Local LLM Ops (Ollama)

Overview

Your localLLM repo provides a full local LLM toolchain on Apple Silicon: setup scripts, a rich CLI chat launcher, benchmarks, and diagnostics. The operational path is: install Ollama, ensure the service is running, initialize the venv, pull models, then launch chat or benchmarks.

Quick Start

./setup_chatbot.sh ./chatllm

If no models are present:

ollama pull mistral

Setup Checklist

• Install Ollama: brew install ollama

• Start the service: brew services start ollama

• Run setup: ./setup_chatbot.sh

• Verify service: curl http://localhost:11434/api/version

Chat Launchers

• ./chatllm (primary launcher)

• ./chat or ./chat.py (alternate launchers)

• Aliases: ./install_aliases.sh then llm , llm-code , llm-fast

Task modes:

./chat -t coding -m codellama:70b ./chat -t creative -m llama3.1:70b ./chat -t analytical

Benchmark Workflow

Benchmarks are scripted in scripts/run_benchmarks.sh :

./scripts/run_benchmarks.sh

This runs bench_ollama.py with:

• benchmarks/prompts.yaml

• benchmarks/models.yaml

• Multiple runs and max token limits

Diagnostics

Run the built-in diagnostic script when setup fails:

./diagnose.sh

Common fixes:

• Re-run ./setup_chatbot.sh

• Ensure ollama is in PATH

• Pull at least one model: ollama pull mistral

Operational Notes

• Virtualenv lives in .venv

• Chat configs and sessions live under ~/.localllm/

• Ollama API runs at http://localhost:11434

Related Skills

• toolchains/universal/infrastructure/docker