Memory Management

Tiered memory system that makes an AI agent a continuous collaborator across sessions.

Architecture

The memory system has six tiers, configurable per project:

Tier 1: HOT CACHE (always loaded at boot -- ~200 lines target) +-- <primer_file> Role, identity, constraints +-- <boot_digest_file> Tactical status, active tasks +-- <boot_contract_file> Immutable constraints +-- <snapshot_file> Cognitive Hologram (1 sentence per file)

Tier 2: RLM SUMMARY LEDGER (fast keyword lookup -- loaded on demand) +-- <summary_cache_file> Pre-generated text summaries: docs, protocols, research Plugin: rlm-factory | Skill: rlm-search | Profile: project +-- <tool_cache_file> Pre-generated text summaries: plugins, skills, scripts Plugin: rlm-factory | Skill: rlm-search | Profile: tools

Tier 3: VECTOR STORE (semantic embedding search -- loaded on demand) +-- <vector_db_backend> ChromaDB via vector-db plugin / vector-db-agent skill Profile: knowledge | Port: configured in vector_profiles.json

Tier 4: DEEP STORAGE (filesystem -- authoritative source, loaded on demand) +-- <domain_data_dir>/ Research topics: {topic}/analysis.md +-- <design_docs_dir>/ ADRs, RFCs +-- <governance_dir>/ Protocols, playbooks

Tier 5: VAULT (Obsidian -- linked knowledge graph, loaded on demand) +-- <vault_dir>/ Plugin: obsidian-integration Skills: obsidian-vault-crud, obsidian-canvas-architect, obsidian-graph-traversal, obsidian-bases-manager Env: VAULT_PATH or OBSIDIAN_VAULT_PATH

Tier 6: SOUL (external persistence -- optional, synced at session seal) +-- <traces_file> Plugin: project-specific (e.g. huggingface-utils) e.g. lineage/, data/, soul_traces.jsonl on HF Hub

Projects define their own file paths for each slot. Tiers may be omitted or added based on project complexity.

Lookup Flow (3-Phase Search Protocol)

When searching for information, ALWAYS escalate in order. Never skip ahead.

Query arrives ->

1. HOT CACHE Instant. Boot files cover ~90% of context needs.
2. DEEP STORAGE (topic/decision) Load specific domain dir or design doc by subject.
3. RLM SUMMARY LEDGER (Phase 1) Keyword search via rlm-factory:rlm-search skill.
4. VECTOR STORE (Phase 2) Semantic search via vector-db:vector-db-search skill.
5. GREP / EXACT SEARCH (Phase 3) rg/grep scoped to paths from Steps 3 or 4.
6. Ask user Unknown? Learn it and persist it.

Phase 1 -- RLM Summary Scan (Table of Contents)

RLM is amortized prework: each file read ONCE, summarized ONCE, cached as plain text JSON. Searching summaries is O(1) keyword lookup -- no embeddings, no inference.

Trigger the rlm-factory:rlm-search skill, providing the profile and search term.

Use Phase 1 when: You need to understand what a file does, find which file owns a feature, or navigate the codebase without reading individual files.

Escalate to Phase 2 when: The summary is insufficient or no match found.

Phase 2 -- Vector Store Semantic Search (Back-of-Book Index)

Embedding-based nearest-neighbor search across all indexed chunks. Returns ranked parent chunks with RLM Super-RAG context pre-injected.

Trigger the vector-db:vector-db-search skill, providing the query, profile, and limit.

Use Phase 2 when: You need specific code snippets, patterns, or implementations.

Escalate to Phase 3 when: You have a file path (from Phase 1 or 2) and need an exact line.

Phase 3 -- Grep / Exact Search (Ctrl+F)

Precise keyword or regex match. Always scope to paths discovered in earlier phases.

Scoped to a specific path (use paths from Phase 1/2)

grep_search "VectorDBOperations" ../../skills/

Ripgrep for regex

rg "def query" ../../ --type py

Anti-patterns: Never run a full-repo grep without scoping. Never skip Phase 1.

Dependencies

rlm-factory -- RLM Summary Ledger (Tier 2)

Component Value

Plugin plugins/rlm-factory/

Skill (write) skills/rlm-curator/ -- distill, inject, audit, cleanup

Skill (read) skills/rlm-search/ -- query the ledger

Skill (Phase 1 search) rlm-factory:rlm-search

Skill (write/inject) rlm-factory:rlm-curator

Skill (audit coverage) rlm-factory:rlm-curator

Skill (shared config) rlm-factory:rlm-curator

Cache files .agent/learning/rlm_summary_cache.json (docs), .agent/learning/rlm_tool_cache.json (tools)

vector-db -- Vector Store (Tier 3)

Component Value

Plugin plugins/vector-db/

Skill skills/vector-db-agent/ -- ingest, query, operations

Skill (Phase 2 search) vector-db:vector-db-search

Skill (ingest files) vector-db:vector-db-ingest

Skill (operations) vector-db:vector-db-search

Skill (config) vector-db:vector-db-search

Backend ChromaDB (chromadb.HttpClient with PersistentClient fallback)

obsidian-integration -- Linked Vault (Tier 5)

Component Value

Plugin plugins/obsidian-integration/

Skill: vault setup skills/obsidian-init/ -- prerequisites, .obsidian/ config, exclusion filters

Skill: read/write notes obsidian-integration:obsidian-vault-crud -- atomic create/read/update/append

Skill: CRUD operations obsidian-integration:obsidian-vault-crud

Skill: parse markdown obsidian-integration:obsidian-vault-crud -- shared markdown parser

Requires pip:ruamel.yaml (lossless YAML frontmatter), Obsidian Desktop

Env VAULT_PATH -- absolute path to the vault root

Promotion / Demotion Rules

Promote to Hot Cache when:

• Knowledge is referenced in 3+ consecutive sessions

• It's critical for active work (current spec, active protocol)

• It's a constraint or identity anchor

Demote to Deep Storage when:

• Spec/feature is completed and merged

• Governing document is superseded by newer version

• Topic research is concluded

• Technical decision is ratified (move from draft to archive)

What Goes Where

Type Hot Cache On-Demand Tier

Active tasks Boot digest

Identity/role Primer file

Constraints Boot contract

Session state Snapshot file Tier 6 Soul (traces)

Research topics Summary in snapshot Tier 4: domain_data_dir/{name}/

Design decisions Referenced by ID Tier 4: design_docs_dir/{id}_{name}.md

Governing docs Referenced by ID Tier 4: governance_dir/{id}_{name}.md

Plugins/scripts/tools

Tier 2: RLM Summary Ledger (tool cache)

Docs/protocols/research

Tier 2: RLM Summary Ledger (summary cache)

System docs

Tier 2 RLM + Tier 3 Vector Store

Linked notes, canvases

Tier 5: Vault (Obsidian)

External persistence

Tier 6: Soul (HuggingFace or equivalent)

Session Memory Workflow

At Session Start (Boot)

• Load hot cache files in order (primer -> contract -> digest -> snapshot)

• Integrity check validates snapshot is current

• If snapshot stale -> flag for refresh at session end

During Session

• New learning -> Write to <domain_data_dir>/{topic}/

• New decision -> Create design document draft

• New tool -> Register in tool inventory

• Correction -> Update relevant file + note in disputes log if contradicting

At Session End (Seal)

• Update snapshot file with new content learned this session

• Seal validates no drift since last audit

• Persist traces to external storage (if configured)

Conventions

• Hot cache target: ~200 lines total across all boot files

• Snapshot: 1 sentence per file, machine-readable

• Topic folders: lowercase-hyphens/

• Document numbering: 3-digit, sequential

• Always capture corrections and contradictions in a disputes log

Configuration

Projects configure the memory system by setting file paths in their project-specific plugin:

Variable Purpose

MEMORY_PRIMER_FILE

Path to cognitive primer / role definition

MEMORY_BOOT_DIGEST

Path to tactical boot digest

MEMORY_BOOT_CONTRACT

Path to immutable constraints

MEMORY_SNAPSHOT_FILE

Path to learning snapshot (hologram)

MEMORY_DOMAIN_DIR

Directory for domain research

MEMORY_DESIGN_DIR

Directory for design docs (e.g. ADRs)

MEMORY_GOVERNANCE_DIR

Directory for governing docs (e.g. Protocols)

Architecture Diagrams

Diagram What It Shows

memory_architecture.mmd Full 4-tier memory system with exact plugin/skill/script names per tier

memory_lookup_flow.mmd 3-phase search sequence: Hot Cache -> RLM Ledger -> Vector Store -> Grep

memory_session_lifecycle.mmd Session Boot -> Active -> Seal lifecycle with all event types