Token Saver Context Compression

Use this skill to reduce token usage while preserving grounded evidence. This integrates:

• pnpm search:code (hybrid retrieval)

• token-saver Python compression scripts

• MemoryRecord persistence into framework memory

• spawn prompt evidence injection ([mem:] / [rag:] )

Activation

The token-saver skill can be invoked in two ways:

Manual Invocation (always available)

Skill({ skill: 'token-saver-context-compression' });

Use this when context pressure is high, pnpm search:tokens shows a file/directory exceeds 32K tokens, or you need query-targeted compression.

Auto-enforcement via compression-reminder.txt (requires AUTO_COMPRESSION_PHASE_3=1)

Set AUTO_COMPRESSION_PHASE_3=1 in .env to enable the compression-reminder.txt trigger:

In .env

AUTO_COMPRESSION_PHASE_3=1

When enabled, compression-trigger.cjs writes .claude/context/runtime/compression-reminder.txt whenever a compression event fires. The router reads this file and spawns context-compressor automatically.

Without this env var: compression events are logged to .claude/context/compression-stats.jsonl but no compression-reminder.txt is written, so the router does not auto-spawn compression. The skill must be invoked manually.

Token thresholds enforced by the router (from CLAUDE.md Section 8):

• 80K tokens — spawn context-compressor proactively

• 120K tokens — compression mandatory before new spawns

• 150K tokens — no new agent spawns until compression completes

Note: These thresholds are router behavioral guidelines checked in CLAUDE.md Section 8. The compression-trigger.cjs triggers are separate heuristics (budget >90%, reads >10KB, fetches >5KB, periodic every 10 ops). There is no automated hook enforcing the 80K/120K/150K thresholds — they rely on the router reading compression-reminder.txt .

When to Use

• pnpm search:tokens shows a file/directory exceeds 32K tokens

• Context is large or expensive and you need a compressed summary

• You need query-targeted compression before synthesis

• You need hard evidence sufficiency gating before persisting memory

• You're building a prompt and search:code results alone aren't enough context

Iron Law

Do not persist compressed content directly to memory files from a subprocess. Emit MemoryRecord payloads and let framework hooks process sync/indexing.

Workflow

• Retrieve candidate context (pnpm search:code "<query>" ).

• Compress using token-saver in JSON mode (run_skill_workflow.py --output-format json ).

• If evidence is insufficient and fail gate is on, stop.

• Map distilled insights into MemoryRecord-ready payloads.

• Persist through MemoryRecord so .claude/hooks/memory/sync-memory-index.cjs runs.

Mapping Rule (Deterministic)

• gotchas.json :

• text contains gotcha|pitfall|anti-pattern|risk|warning|failure

• issues.md :

• text contains issue|bug|error|incident|defect|gap

• decisions.md :

• text contains decision|tradeoff|choose|selected|rationale

• patterns.json :

• default fallback for all remaining distilled evidence

Tooling Commands

Preferred wrapper entrypoint:

node .claude/skills/token-saver-context-compression/scripts/main.cjs --query "<question>" --mode evidence_aware --limit 20 --fail-on-insufficient-evidence

Direct Python engine (advanced):

python .claude/skills/token-saver-context-compression/scripts/run_skill_workflow.py --file <path> --mode evidence_aware --query "<question>" --output-format json --fail-on-insufficient-evidence

Output Contract

• Wrapper emits JSON with:

• search summary

• compression summary

• memoryRecords grouped by target (patterns , gotchas , issues , decisions )

• evidence sufficiency status

Workflow References

• Skill workflow: .claude/workflows/token-saver-context-compression-skill-workflow.md

• Companion tool: .claude/tools/token-saver-context-compression/token-saver-context-compression.cjs

• Command surface: .claude/commands/token-saver-context-compression.md

• Citation format is unchanged:

• memory entries become [mem:xxxxxxxx]

• RAG entries remain [rag:xxxxxxxx]

Integration with search:tokens

Use pnpm search:tokens to decide when to invoke this skill:

Check if you need compression

pnpm search:tokens .claude/lib/memory

Output: 60 files, 500KB, ~128K tokens ⚠ OVER CONTEXT

Then compress with a targeted query

node .claude/skills/token-saver-context-compression/scripts/main.cjs
--query "how does memory persistence work" --mode evidence_aware --limit 10

The tool reads actual file content from search results (not just file paths), compresses via the Python engine, and extracts memory records classified by type (patterns, gotchas, issues, decisions).

Adaptive Compression

Adaptive compression (adjusting compression ratio based on corpus size) is automatic and requires no env var configuration. When the input corpus is small, compression is lighter; when it is large, compression is more aggressive. This is controlled internally by the Python engine based on token counts.

Requirements

• Node.js 18+

• Python 3.10+

Iron Laws

• ALWAYS run hybrid search (pnpm search:code ) before compressing to retrieve grounded evidence for the distilled output

• NEVER compress context that still has open uncertainties — resolve ambiguities before compressing

• ALWAYS persist distilled learnings via MemoryRecord immediately after compression

• NEVER discard evidence that contradicts the current working hypothesis during compression

• ALWAYS inject [mem:] and [rag:] citations in the compressed output for downstream spawn prompt grounding

Anti-Patterns

Anti-Pattern Why It Fails Correct Approach

Compressing without prior hybrid search Output lacks grounded evidence, hallucination risk Run pnpm search:code first, embed citations

Discarding contradicting evidence Creates false confidence in distilled output Preserve all conflicting signals in summary

No MemoryRecord after compression Learnings lost on next context reset Persist key findings immediately via MemoryRecord

Compressing too late (past 80K tokens) Severe accuracy degradation before compression Trigger compression at 80K tokens, not at limit

Skipping [mem:] / [rag:] citations Downstream agents cannot verify claims Always annotate evidence sources in output

Memory Protocol (MANDATORY)

Before work:

cat .claude/context/memory/learnings.md

After work:

• Add integration learnings to .claude/context/memory/learnings.md

• Add integration risks to .claude/context/memory/issues.md