GrepAI Chunking Configuration

This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase.

When to Use This Skill

• Optimizing search accuracy

• Adjusting for code style (verbose vs. concise)

• Troubleshooting search results

• Understanding how indexing works

What is Chunking?

Chunking is the process of splitting source files into smaller segments for embedding:

┌─────────────────────────────────────┐ │ Large Source File │ │ (1000+ tokens) │ └─────────────────────────────────────┘ ↓ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │ │ ~512 │ │ ~512 │ │ ~512 │ │ tokens │ │ tokens │ │ tokens │ └─────────┘ └─────────┘ └─────────┘ ↓ Each chunk gets its own embedding

Why Chunking Matters

Embedding models have optimal input sizes:

• Too large chunks: Less precise search results

• Too small chunks: Lost context, fragmented results

• Just right: Good balance of precision and context

Configuration

Basic Settings

.grepai/config.yaml

chunking: size: 512 # Tokens per chunk overlap: 50 # Overlap between chunks

Understanding Parameters

Chunk Size

The target number of tokens per chunk.

Size Effect

256 More precise, less context

512 Balanced (default)

1024 More context, less precise

Overlap

Tokens shared between adjacent chunks. Preserves context at boundaries.

Overlap Effect

0 No overlap, may lose context at boundaries

50 Standard overlap (default)

100 More context, larger index

Visualization

With size=512 and overlap=50:

File: auth.go (1000 tokens)

Chunk 1: tokens 1-512 ┌────────────────────────────────────┐ │ func Login(user, pass)... │ └────────────────────────────────────┘ ↘ 50 token overlap ↙ Chunk 2: tokens 463-974 ┌────────────────────────────────────┐ │ ...validate credentials... │ └────────────────────────────────────┘ ↘ 50 token overlap ↙ Chunk 3: tokens 925-1000 ┌──────────────┐ │ ...return │ └──────────────┘

Recommended Settings by Language

Verbose Languages (Java, C#)

chunking: size: 768 # Larger to capture full methods overlap: 75

Concise Languages (Go, Python)

chunking: size: 512 # Standard size overlap: 50

Very Concise (Rust, Zig)

chunking: size: 384 # Smaller for precise results overlap: 40

Recommended Settings by Codebase

Small Functions (Microservices)

chunking: size: 384 # Capture individual functions overlap: 40

Large Classes (Monolith)

chunking: size: 768 # Capture more context overlap: 100

Mixed Codebase

chunking: size: 512 # Balanced default overlap: 50

How Tokens are Counted

GrepAI uses approximate token counting:

• ~4 characters = 1 token (for English text)

• Code varies based on identifiers and syntax

Example:

func calculateTotal(items []Item) float64 { total := 0.0 for _, item := range items { total += item.Price * float64(item.Quantity) } return total }

≈ 45 tokens

Impact on Index Size

Larger overlap = more chunks = larger index:

Size Overlap Chunks per 10K tokens Index Impact

512 0 ~20 Smallest

512 50 ~22 Standard

512 100 ~24 +10%

256 50 ~44 +100%

Impact on Search Quality

Too Small Chunks (size: 128)

Query: "authentication middleware"

Result: "...c.AbortWithStatus(401)..." (Fragment, missing context)

Just Right (size: 512)

Query: "authentication middleware"

Result: "func AuthMiddleware() gin.HandlerFunc { return func(c *gin.Context) { token := c.GetHeader("Authorization") if token == "" { c.AbortWithStatus(401) return } // validate token... } }" (Complete function with context)

Too Large Chunks (size: 2048)

Query: "authentication middleware"

Result: "// Multiple unrelated functions... func AuthMiddleware()... (your match) func LoggingMiddleware()... func CORSMiddleware()..." (Too much noise)

Experimentation

Testing Different Settings

• Try smaller chunks for more precise results:

chunking: size: 384 overlap: 40

• Re-index:

rm .grepai/index.gob grepai watch

• Test with searches:

grepai search "your query"

• Adjust and repeat until satisfied.

Comparing Results

Before changing settings, save a search result:

grepai search "authentication" > before.txt

After changing settings and re-indexing:

grepai search "authentication" > after.txt diff before.txt after.txt

Chunk Boundaries

GrepAI tries to split at logical boundaries:

• Empty lines (function/class boundaries)

• Closing braces

• Statement ends

This means actual chunk sizes may vary slightly from the target.

Best Practices

• Start with defaults: 512/50 works well for most codebases

• Adjust based on code style: Verbose = larger, concise = smaller

• Test with real queries: See what your searches return

• Re-index after changes: Must regenerate embeddings

• Consider overlap: Don't set to 0 unless index size is critical

Common Issues

❌ Problem: Search results are too fragmented ✅ Solution: Increase chunk size:

chunking: size: 768

❌ Problem: Search results have too much irrelevant context ✅ Solution: Decrease chunk size:

chunking: size: 384

❌ Problem: Results miss related code at function boundaries ✅ Solution: Increase overlap:

chunking: overlap: 100

❌ Problem: Index is too large ✅ Solutions:

• Decrease overlap

• Increase chunk size

• Add more ignore patterns

Output Format

Chunking status:

✅ Chunking Configuration

Size: 512 tokens Overlap: 50 tokens

Index Statistics:

• Total files: 245
• Total chunks: 1,234
• Avg chunks/file: 5.0
• Avg chunk size: 478 tokens

Recommendations:

• Current settings are balanced
• Consider size: 384 for more precise results
• Consider size: 768 for more context