Ollama Setup for GrepAI

This skill covers installing and configuring Ollama as the local embedding provider for GrepAI. Ollama enables 100% private code search where your code never leaves your machine.

When to Use This Skill

• Setting up GrepAI with local, private embeddings

• Installing Ollama for the first time

• Choosing and downloading embedding models

• Troubleshooting Ollama connection issues

Why Ollama?

Benefit Description

🔒 Privacy Code never leaves your machine

💰 Free No API costs

⚡ Fast Local processing, no network latency

🔌 Offline Works without internet

Installation

macOS (Homebrew)

Install Ollama

brew install ollama

Start the Ollama service

ollama serve

macOS (Direct Download)

• Download from ollama.com

• Open the .dmg and drag to Applications

• Launch Ollama from Applications

Linux

One-line installer

curl -fsSL https://ollama.com/install.sh | sh

Start the service

ollama serve

Windows

• Download installer from ollama.com

• Run the installer

• Ollama starts automatically as a service

Downloading Embedding Models

GrepAI requires an embedding model to convert code into vectors.

Recommended Model: nomic-embed-text

Download the recommended model (768 dimensions)

ollama pull nomic-embed-text

Specifications:

• Dimensions: 768

• Size: ~274 MB

• Performance: Excellent for code search

• Language: English-optimized

Alternative Models

Multilingual support (better for non-English code/comments)

ollama pull nomic-embed-text-v2-moe

Larger, more accurate

ollama pull bge-m3

Maximum quality

ollama pull mxbai-embed-large

Model Dimensions Size Best For

nomic-embed-text

768 274 MB General code search

nomic-embed-text-v2-moe

768 500 MB Multilingual codebases

bge-m3

1024 1.2 GB Large codebases

mxbai-embed-large

1024 670 MB Maximum accuracy

Verifying Installation

Check Ollama is Running

Check if Ollama server is responding

curl http://localhost:11434/api/tags

Expected output: JSON with available models

List Downloaded Models

ollama list

Output:

NAME ID SIZE MODIFIED

nomic-embed-text:latest abc123... 274 MB 2 hours ago

Test Embedding Generation

Quick test (should return embedding vector)

curl http://localhost:11434/api/embeddings -d '{ "model": "nomic-embed-text", "prompt": "function hello() { return world; }" }'

Configuring GrepAI for Ollama

After installing Ollama, configure GrepAI to use it:

.grepai/config.yaml

embedder: provider: ollama model: nomic-embed-text endpoint: http://localhost:11434

This is the default configuration when you run grepai init , so no changes are needed if using nomic-embed-text .

Running Ollama

Foreground (Development)

Run in current terminal (see logs)

ollama serve

Background (macOS/Linux)

Using nohup

nohup ollama serve &

Or as a systemd service (Linux)

sudo systemctl enable ollama sudo systemctl start ollama

Check Status

Check if running

pgrep -f ollama

Or test the API

curl -s http://localhost:11434/api/tags | head -1

Resource Considerations

Memory Usage

Embedding models load into RAM:

• nomic-embed-text : ~500 MB RAM

• bge-m3 : ~1.5 GB RAM

• mxbai-embed-large : ~1 GB RAM

CPU vs GPU

Ollama uses CPU by default. For faster embeddings:

• macOS: Uses Metal (Apple Silicon) automatically

• Linux/Windows: Install CUDA for NVIDIA GPU support

Common Issues

❌ Problem: connection refused to localhost:11434 ✅ Solution: Start Ollama:

ollama serve

❌ Problem: Model not found ✅ Solution: Pull the model first:

ollama pull nomic-embed-text

❌ Problem: Slow embedding generation ✅ Solution:

• Use a smaller model

• Ensure Ollama is using GPU (check ollama ps )

• Close other memory-intensive applications

❌ Problem: Out of memory ✅ Solution: Use a smaller model or increase system RAM

Best Practices

• Start Ollama before GrepAI: Ensure ollama serve is running

• Use recommended model: nomic-embed-text offers best balance

• Keep Ollama running: Leave it as a background service

• Update periodically: ollama pull nomic-embed-text for updates

Output Format

After successful setup:

✅ Ollama Setup Complete

Ollama Version: 0.1.x Endpoint: http://localhost:11434 Model: nomic-embed-text (768 dimensions) Status: Running

GrepAI is ready to use with local embeddings. Your code will never leave your machine.