Context Optimization Techniques

Context optimization extends effective capacity through strategic compression, masking, caching, and partitioning. Effective optimization can double or triple effective context capacity.

When to Activate

• Context limits constrain task complexity

• Optimizing for cost reduction (fewer tokens = lower costs)

• Reducing latency for long conversations

• Building production systems at scale

Core Strategies

Compaction

Summarize context contents when approaching limits, reinitialize with summary.

Priority for compression:

• Tool outputs → replace with summaries

• Old turns → summarize early conversation

• Retrieved docs → summarize if recent versions exist

• Never compress system prompt

Summary preservation by type:

• Tool outputs: Key findings, metrics, conclusions

• Conversations: Key decisions, commitments, context shifts

• Documents: Key facts and claims

Observation Masking

Tool outputs can comprise 80%+ of token usage. Replace verbose outputs with compact references once their purpose is served.

Masking Strategy:

Category Action

Never mask Current task observations, most recent turn, active reasoning

Consider masking 3+ turns ago, verbose outputs with extractable key points

Always mask Repeated outputs, boilerplate, already summarized

Example:

if len(observation) > max_length: ref_id = store_observation(observation) return f"[Obs:{ref_id} elided. Key: {extract_key(observation)}]"

KV-Cache Optimization

Reuse cached computations across requests with identical prefixes.

Cache-friendly ordering:

• System prompt (stable, first)

• Tool definitions (stable)

• Frequently reused elements

• Unique content (last)

Design tips:

• Avoid dynamic content like timestamps

• Use consistent formatting

• Keep structure stable across sessions

Context Partitioning

Split work across sub-agents with isolated contexts. Each operates in clean context focused on its subtask.

Aggregation pattern:

• Validate all partitions completed

• Merge compatible results

• Summarize if still too large

Budget Management

Design explicit token budgets:

• System prompt: X tokens

• Tool definitions: Y tokens

• Retrieved docs: Z tokens

• Message history: W tokens

• Reserved buffer: 10-20%

Trigger optimization when:

• Token utilization > 70%

• Response quality degrades

• Costs increase due to long contexts

Decision Framework

Dominant component Apply

Tool outputs Observation masking

Retrieved documents Summarization or partitioning

Message history Compaction with summarization

Multiple Combine strategies

Performance Targets

• Compaction: 50-70% reduction, <5% quality degradation

• Masking: 60-80% reduction in masked observations

• Cache optimization: 70%+ hit rate for stable workloads

Guidelines

• Measure before optimizing—know current state

• Apply compaction before masking when possible

• Design for cache stability with consistent prompts

• Partition before context becomes problematic

• Balance token savings against quality preservation