Mutation Testing

Mutation testing answers: "Would my tests catch this bug?" by actually introducing bugs and running tests.

Execution Workflow

CRITICAL: This skill actually mutates code and runs tests. Follow this exact process:

Step 1: Identify Target Code

Get changed files on the branch

git diff main...HEAD --name-only | grep -E '.(ts|js|tsx|jsx|vue)$' | grep -v '.test.' | grep -v '.spec.'

Step 2: For Each Function to Test

Execute this loop for each mutation:

1. READ the original file and note exact content
2. APPLY one mutation (edit the code)
3. RUN tests: pnpm test --run (or specific test file)
4. RECORD result: KILLED (test failed) or SURVIVED (test passed)
5. RESTORE original code immediately
6. Repeat for next mutation

Step 3: Report Results

After all mutations, provide a summary table:

Mutation Operators to Apply

Priority 1: Boundary Mutations (Most Likely to Survive)

Original Mutate To Why It Matters

<

<=

Boundary not tested

=

Boundary not tested

<=

<

Equality case missed

=

Equality case missed

Priority 2: Boolean Logic Mutations

Original Mutate To Why It Matters

&&

||

Only tested when both true

||

&&

Only tested when both false

!condition

condition

Negation not verified

Priority 3: Arithmetic Mutations

Original Mutate To Why It Matters

Tested with 0 only

Tested with 0 only

/

Tested with 1 only

Priority 4: Return/Early Exit Mutations

Original Mutate To Why It Matters

return x

return null

Return value not asserted

return true

return false

Boolean return not checked

if (cond) return

// removed

Early exit not tested

Priority 5: Statement Removal

Original Mutate To Why It Matters

array.push(x)

// removed

Side effect not verified

await save(x)

// removed

Async operation not verified

emit('event')

// removed

Event emission not tested

Practical Execution Example

Example: Testing a Validation Function

Original code (src/utils/validation.ts:15 ):

export function isValidAge(age: number): boolean { return age >= 18 && age <= 120; }

Mutation 1: Change >= to >

export function isValidAge(age: number): boolean { return age > 18 && age <= 120; // MUTATED }

Run tests:

pnpm test --run src/tests/validation.test.ts

Result: Tests PASS → SURVIVED (Bad! Need test for isValidAge(18) )

Restore original code immediately

Mutation 2: Change && to ||

export function isValidAge(age: number): boolean { return age >= 18 || age <= 120; // MUTATED }

Run tests:

pnpm test --run src/tests/validation.test.ts

Result: Tests FAIL → KILLED (Good! Tests catch this bug)

Restore original code immediately

Results Interpretation

Mutant States

State Meaning Action

KILLED Test failed with mutant Tests are effective

SURVIVED Tests passed with mutant Add or strengthen test

TIMEOUT Tests hung (infinite loop) Counts as detected

Mutation Score

Score = (Killed + Timeout) / Total Mutations * 100

Score Quality

< 60% Weak - significant test gaps

60-80% Moderate - improvements needed

80-90% Good - minor gaps

90% Strong test suite

Fixing Surviving Mutants

When a mutant survives, add a test that would catch it:

Surviving: Boundary mutation (>= → > )

// Add boundary test it('accepts exactly 18 years old', () => { expect(isValidAge(18)).toBe(true); // Would fail if >= became > });

Surviving: Logic mutation (&& → || )

// Add test with mixed conditions it('rejects when only one condition met', () => { expect(isValidAge(15)).toBe(false); // Would pass if && became || });

Surviving: Statement removal

// Add side effect verification it('saves to database', async () => { await processOrder(order); expect(db.save).toHaveBeenCalledWith(order); // Would fail if save removed });

Quick Checklist During Mutation

For each mutation, ask:

• Before mutating: Does a test exist for this code path?

• After running tests: Did any test actually fail?

• If survived: What specific test would catch this?

• After fixing: Re-run mutation to confirm killed

Common Surviving Mutation Patterns

Tests Only Check Happy Path

// WEAK: Only tests success case it('validates', () => { expect(validate(goodInput)).toBe(true); });

// STRONG: Tests both cases it('validates good input', () => { expect(validate(goodInput)).toBe(true); }); it('rejects bad input', () => { expect(validate(badInput)).toBe(false); });

Tests Use Identity Values

// WEAK: Mutation survives expect(multiply(5, 1)).toBe(5); // 5*1 = 5/1 = 5

// STRONG: Mutation detected expect(multiply(5, 3)).toBe(15); // 5*3 ≠ 5/3

Tests Don't Assert Return Values

// WEAK: No return value check it('processes', () => { process(data); // No assertion! });

// STRONG: Asserts outcome it('processes', () => { const result = process(data); expect(result).toEqual(expected); });

Important Rules

• ALWAYS restore original code after each mutation

• Run tests immediately after applying mutation

• One mutation at a time - don't combine mutations

• Focus on changed code - prioritize branch diff

• Track all results - report full mutation summary

Summary Report Template

After completing mutation testing, provide:

Mutation Testing Results

Target: src/features/workout/utils.ts (functions: X, Y, Z) Total Mutations: 12 Killed: 9 Survived: 3 Score: 75%

Surviving Mutants (Action Required)

Killed Mutants (Tests Effective)

• Line 35: + → - killed by calculation.test.ts
• Line 48: true → false killed by validate.test.ts
• ...

Related Skills

• systematic-debugging

• Root cause analysis

• testing-conventions

• Query priority, expect.poll()

• vue-integration-testing

• Page objects, browser mode

• vitest-mocking

• Test doubles and mocking patterns