Skill Improver

Process user feedback from skill retrospectives and update skill files to improve them over time.

When to Use

• User asks to "review skill feedback" or "improve skills based on usage"

• You notice feedback files in .claude/feedback/

• User mentions a skill didn't work well or missed something

• Periodic review (monthly) to incorporate learnings

How It Works

Step 1: Gather Feedback

Read all feedback files in .claude/feedback/ :

ls -la .claude/feedback/retro-*.md

Look for patterns:

• Multiple users reporting same missing step → add to skill

• Benchmarks don't match user's context → add context-specific ranges

• Workflow confusing → restructure or add clarifications

• Skill incomplete → add missing sections

Step 2: Identify High-Impact Changes

Prioritize updates based on:

High Priority (do first):

• Missing critical steps that users had to figure out themselves

• Incorrect benchmarks or numbers

• Confusing workflow that requires clarification

• Safety issues or errors

Medium Priority:

• Additional examples or templates

• Better explanations of existing steps

• Alternative approaches for different contexts

Low Priority:

• Nice-to-have additions

• Stylistic improvements

• Minor clarifications

Step 3: Update Skill Files

For each skill needing updates:

3a. Add a "Learnings" Section

If the skill doesn't have one, add at the end:

Learnings from Use

[Date]: [Brief description of what was learned]

• Feedback: [What users reported]
• Update: [What we changed]
• Result: [Expected improvement]

3b. Update Main Content

If feedback suggests core changes:

• Add missing steps to checklists

• Update benchmarks with ranges (e.g., "20-30% for B2C, 50-70% for B2B")

• Restructure workflow if confusing

• Add "Common Pitfalls" section if users make same mistakes

3c. Version the Change

At the top of the skill, track versions:

name: skill-name version: 1.2.0 last_updated: 2026-01-22 changelog:

• v1.2.0 (2026-01-22): Added missing step for X based on user feedback
• v1.1.0 (2026-01-15): Updated benchmarks for Y context
• v1.0.0 (2026-01-01): Initial release

Step 4: Archive Processed Feedback

Move processed feedback to archive:

mkdir -p .claude/feedback/archive mv .claude/feedback/retro-2026-01-22-*.md .claude/feedback/archive/

Keep a summary of learnings in .claude/feedback/SUMMARY.md :

Feedback Summary

[Skill Name]

Total feedback sessions: 12 Last updated: 2026-01-22

Key learnings:

• Added step for X (reported by 3 users)
• Updated benchmarks for B2B context (reported by 5 users)
• Clarified workflow around Y (reported by 2 users)

Patterns:

• Users in enterprise context need higher benchmarks
• Early-stage startups need more examples
• Non-technical users need clearer explanations of jargon

Example Workflow

Scenario: product-market-fit skill needs improvement

Step 1: Review Feedback

Read .claude/feedback/retro-2026-01-22-143022.md :

Feedback

Missed important steps? yes

Improvements needed: The Sean Ellis test threshold of 40% seems high for B2B enterprise products. We're at 32% "very disappointed" but our retention is 85% D30 which is excellent. Should the skill mention that thresholds vary by product type?

Step 2: Identify Pattern

Check other feedback files → 3 more users report B2B context needs different benchmarks.

Step 3: Update Skill

Edit .claude/skills/product-market-fit/SKILL.md :

Before:

Sean Ellis Test (40% Rule)

"How would you feel if you could no longer use [product]?"

• ≥40% "Very disappointed" = Strong PMF

After:

Sean Ellis Test (Context-Dependent Thresholds)

"How would you feel if you could no longer use [product]?"

Thresholds by product type:

• Consumer B2C: ≥40% "Very disappointed" = Strong PMF
• SMB B2B: ≥35% "Very disappointed" = Strong PMF
• Enterprise B2B: ≥30% "Very disappointed" = Strong PMF (longer sales cycles, different buying psychology)

Why the difference?

• Enterprise buyers are more rational than emotional
• Switching costs are higher (contracts, integrations)
• Retention is a better PMF signal for B2B (see Step 2)

Add to Learnings section:

Learnings from Use

2026-01-22: Refined Sean Ellis thresholds by product type

• Feedback: 4 users reported 40% threshold too high for B2B enterprise
• Update: Added context-specific thresholds (B2C 40%, SMB 35%, Enterprise 30%)
• Result: More accurate PMF diagnosis for different product types

Step 4: Archive & Track

mv .claude/feedback/retro-2026-01-22-*.md .claude/feedback/archive/

Update .claude/feedback/SUMMARY.md :

product-market-fit

Total feedback sessions: 4 Last updated: 2026-01-22

Key learnings:

• Added context-specific Sean Ellis thresholds (reported by 4 users)
• B2B needs different benchmarks than B2C

Next improvements to consider:

• Add industry-specific retention benchmarks
• Include examples from different verticals

Quality Checklist

Before updating any skill, ensure:

• Feedback is from multiple users (pattern, not outlier)

• Change makes skill more accurate, not just more complex

• Benchmarks are sourced or validated (not anecdotal)

• Update is backward compatible (doesn't break existing workflows)

• Learnings section documents why we made the change

• Version number incremented appropriately (semver)

• Processed feedback archived, not deleted

Feedback Categories

Track feedback by type to identify systemic issues:

Category 1: Missing Steps

Example: "Skill forgot to mention we need to segment cohorts by acquisition channel" Action: Add step to checklist

Category 2: Incorrect Benchmarks

Example: "40% D30 retention is not 'strong' for our B2B SaaS, it's average" Action: Update benchmarks with context (B2C vs B2B vs Enterprise)

Category 3: Confusing Workflow

Example: "I didn't know whether to do cohort analysis before or after Sean Ellis test" Action: Number steps clearly, add workflow diagram

Category 4: Missing Context

Example: "Skill assumes I have 1000+ users, what if I only have 50?" Action: Add "Early Stage Adaptation" section

Category 5: Tool-Specific Issues

Example: "How do I calculate D30 retention in Google Analytics?" Action: Add "Implementation in Common Tools" section

Best Practices

Do:

✅ Look for patterns across multiple feedback sessions ✅ Update skills incrementally (small, tested changes) ✅ Document why changes were made (Learnings section) ✅ Preserve feedback history (archive, don't delete) ✅ Version skills so users know what changed

Don't:

❌ Update based on single piece of feedback (might be outlier) ❌ Make skills overly complex trying to cover every edge case ❌ Remove content without understanding why it was there ❌ Ignore feedback for more than 30 days (patterns emerge) ❌ Update without testing the new version

Automation Ideas

Weekly Digest (optional)

Create a script to summarize new feedback:

#!/bin/bash

.claude/hooks/learning/weekly-feedback-digest.sh

echo "📊 Feedback Digest (Last 7 Days)" echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"

find .claude/feedback -name "retro-*.md" -mtime -7 | while read -r file; do echo "" echo "File: $(basename $file)" grep "Skills Used" -A 5 "$file" grep "Improvements needed:" -A 3 "$file" done

Auto-Tag for Review

When feedback mentions specific issues, auto-tag:

• "missing step" → tag for immediate review

• "wrong number" → tag for fact-check

• "confusing" → tag for clarity rewrite

Success Metrics

Track improvement over time:

• Feedback frequency: Decreasing = skills getting better

• Repeated issues: Should approach zero over time

• User satisfaction: Track "Did this skill help?" responses

• Skill usage: Updated skills should see increased usage

Meta: This Skill Improves Itself

This skill should follow its own advice:

Learnings from Use:

[To be filled as this skill gets used and improved]

Version History:

• v1.0.0 (2026-01-22): Initial release - framework for skill improvement

Next Steps:

• Review feedback in .claude/feedback/

• Identify patterns and prioritize updates

• Update skill files with improvements

• Document learnings

• Archive processed feedback

• Commit changes with clear message

The more you use this system, the better your skills become. It's a continuous improvement loop.