Memory Hybrid Stack

Overview

This stack splits agent memory across three stores:

1. Postgres + pgvector (facts) — structured, durable knowledge with optional embeddings.
2. Redis (state) — low-latency values that expire (locks, session flags, device status).
3. Qdrant (vectors) — semantic recall for long-form text chunks, keyed by collection oc_memory.

All services run via Docker Compose (infra/memory-stack/docker-compose.yml). Connection defaults live in infra/memory-stack/.env; scripts in this skill auto-source that file unless you override MEMORY_STACK_ENV / MEMORY_STACK_ROOT.

Quick Start

1. Ensure the stack is running: cd infra/memory-stack && docker compose ps (all three should be healthy).
2. cd skills/memory-hybrid-stack/scripts or call scripts via absolute path.
3. Use the helper scripts listed below (they inject host/port/user from .env).
4. For schema/endpoint details, open references/connection-map.md when needed.

Facts Layer (Postgres + pgvector)

• Use when: recording validated facts, relationships, high-confidence summaries, or when you need SQL joins/filtering.
• Connection: postgres://oc_memory:…@localhost:55432/oc_facts. Credentials auto-loaded from .env.

Common operations

# Run ad-hoc SQL (string argument)
./scripts/facts_sql.sh "SELECT subject, object->>'value' AS value FROM facts WHERE tags @> ARRAY['preference'];"

# Pipe a multiline query
cat <<'SQL' | ./scripts/facts_sql.sh
INSERT INTO facts (subject, predicate, object, source, confidence, tags)
VALUES (
  'user:xiaobai',
  'prefers_language',
  jsonb_build_object('value', 'zh-CN'),
  'chat/2026-03-18',
  0.92,
  ARRAY['preference','language']
)
ON CONFLICT (subject, predicate)
DO UPDATE SET object = EXCLUDED.object, updated_at = now();
SQL

Tips:

• Store raw text inside object JSON, e.g. { "value": "...", "summary": "..." }.
• Use embedding column when you already have a 1536-d vector (set via UPDATE facts SET embedding = '[...]'::vector WHERE id = ...).
• Always tag rows (tags text[]) so downstream filters are cheap.

State Layer (Redis)

• Use when: caching short-lived context (current task, device status, throttles, locks).
• Key pattern: state:<entity>:<attribute>; keep payloads as JSON strings for readability.

Commands

# Fetch
./scripts/state_kv.sh get state:user:xiaobai:current-task

# Set with 10-minute TTL
./scripts/state_kv.sh set state:user:xiaobai:current-task '{"summary":"researching", "started_at":"2026-03-18T19:00:00Z"}' 600

# Delete
./scripts/state_kv.sh del state:user:xiaobai:current-task

Guidelines:

• Keep TTLs short (seconds/minutes) unless the value is truly session-scoped.
• Use Redis for coordination (e.g., state:lock:calendar-sync) with low TTL to avoid deadlocks.

Vector Layer (Qdrant)

• Use when: storing or retrieving semantic chunks that exceed Markdown recall.
• Endpoints: HTTP http://localhost:6335, gRPC http://localhost:6336.
• Default collection: oc_memory (1536-d cosine). Created via scripts/init_qdrant.sh.

Helper usage

# Check collections
./scripts/qdrant_request.sh GET /collections

# Upsert points from a file
cat > /tmp/points.json <<'JSON'
{
  "points": [
    {
      "id": "memo-001",
      "vector": [/* 1536 floats */],
      "payload": {
        "subject": "user:xiaobai",
        "text": "Prefers in-depth, sourced answers.",
        "tags": ["preference"],
        "timestamp": "2026-03-18T19:05:00Z"
      }
    }
  ]
}
JSON
./scripts/qdrant_request.sh PUT /collections/oc_memory/points @/tmp/points.json

# Semantic search (vector or filter payload inline)
./scripts/qdrant_request.sh POST /collections/oc_memory/points/search '{
  "vector": [/* query vector */],
  "limit": 5,
  "with_payload": true,
  "filter": {"must": [{"key": "subject", "match": {"value": "user:xiaobai"}}]}
}'

Notes:

• The script accepts raw JSON strings or @/path/file.json.
• Generate embeddings via your preferred model (e.g., OpenAI text-embedding-3-small); ensure dimension = 1536.
• Keep payload timestamps (timestamp) to enforce recency filtering.

Workflow Recommendations

1. Ephemeral -> Durable: log immediate events in Redis, then promote confirmed facts into Postgres/Qdrant.
2. Fan-out writes: when capturing a new preference, update Postgres (structured) and Qdrant (semantic search) in the same turn.
3. Read order: Redis (latest state) → Postgres (authoritative fact) → Qdrant (related context) → Markdown fallback.
4. Tags & filters: align tags (Postgres) with Qdrant payload keys so cross-store correlation is simple.

Troubleshooting

• docker compose ps shows unhealthy containers → check host ports (stack uses 55432/56379/6335/6336 to avoid clashes with ai-stack-*).
• Scripts complain about missing .env → copy .env.example → .env, or set env vars manually.
• Qdrant health stuck on "starting" → ensure you rebuilt using memory-qdrant:local (curl installed) or adjust healthcheck.

References

• connection-map.md — Ports, schemas, payload templates, and key conventions.