Agentic Kit RAG Integration

Configure the agentic-kit for Retrieval-Augmented Generation (RAG) using pgvector for semantic search and PGPM for database management. This skill enables AI chat applications to provide contextual responses based on a knowledge base stored in PostgreSQL.

When to Apply

Use this skill when:

• Adding RAG capabilities to agentic-kit applications

• Configuring environment variables for RAG-enabled AI chat

• Setting up a local database for document storage and retrieval

• Building AI assistants that need access to a knowledge base

• Integrating pgvector with the sf-rag-utils app

Architecture Overview

User Query → Agentic Kit → RAG Provider ↓ [RAG_ENABLED=true?] ↓ ↓ Yes No ↓ ↓ Query Embedding Direct Ollama ↓ pgvector Search ↓ Context Retrieval ↓ Ollama + Context → Response

Environment Variables

Configure RAG behavior through environment variables:

Variable Default Description

RAG_ENABLED

true

Enable RAG context retrieval (set to false for direct Ollama)

RAG_DATABASE_URL

PostgreSQL connection string (required when RAG_ENABLED=true)

OLLAMA_HOST

http://localhost:11434

Ollama server URL

RAG_EMBEDDING_MODEL

nomic-embed-text

Model for generating embeddings

RAG_CHAT_MODEL

llama3.2

Model for chat responses

RAG_SIMILARITY_THRESHOLD

0.5

Minimum similarity score for retrieval

RAG_CONTEXT_LIMIT

5

Maximum number of chunks to retrieve

RAG_SCHEMA

intelligence

PostgreSQL schema for RAG tables

Example .env File

RAG Configuration

RAG_ENABLED=true RAG_DATABASE_URL=postgres://postgres:postgres@localhost:5432/rag_dev

Ollama Configuration

OLLAMA_HOST=http://localhost:11434 RAG_EMBEDDING_MODEL=nomic-embed-text RAG_CHAT_MODEL=llama3.2

Retrieval Settings

RAG_SIMILARITY_THRESHOLD=0.5 RAG_CONTEXT_LIMIT=5 RAG_SCHEMA=intelligence

Disabling RAG (Direct Ollama Mode)

To use direct Ollama without RAG context:

RAG_ENABLED=false OLLAMA_HOST=http://localhost:11434 RAG_CHAT_MODEL=llama3.2

Quick Start

1. Set Up Local Database with PGPM

Ensure PostgreSQL is running with a pgvector-enabled image (see pgpm-docker skill) and PG env vars are loaded (see pgpm-env skill).

Run the setup script

bash /mnt/skills/user/agentic-kit-rag/scripts/setup-rag-database.sh

2. Configure Environment

Create a .env file in your application:

RAG_ENABLED=true RAG_DATABASE_URL=postgres://postgres:postgres@localhost:5432/rag_dev OLLAMA_HOST=http://localhost:11434

3. Pull Required Ollama Models

ollama pull nomic-embed-text ollama pull llama3.2

RAG Provider Implementation

Create a RAG-aware provider for agentic-kit:

// src/lib/ai/rag-provider.ts import { Pool } from 'pg'; import type { AgentProvider, GenerateInput, StreamCallbacks, Message } from '@sf-ai/agentic-kit';

interface RAGConfig { databaseUrl: string; ollamaHost?: string; embeddingModel?: string; chatModel?: string; similarityThreshold?: number; contextLimit?: number; schema?: string; }

const formatVector = (embedding: number[]): string => [${embedding.join(',')}];

export class RAGProvider implements AgentProvider { readonly name = 'rag'; private pool: Pool; private ollamaHost: string; private embeddingModel: string; private chatModel: string; private similarityThreshold: number; private contextLimit: number; private schema: string;

constructor(config: RAGConfig) { this.pool = new Pool({ connectionString: config.databaseUrl }); this.ollamaHost = config.ollamaHost || 'http://localhost:11434'; this.embeddingModel = config.embeddingModel || 'nomic-embed-text'; this.chatModel = config.chatModel || 'llama3.2'; this.similarityThreshold = config.similarityThreshold || 0.5; this.contextLimit = config.contextLimit || 5; this.schema = config.schema || 'intelligence'; }

private async generateEmbedding(text: string): Promise<number[]> { const response = await fetch(${this.ollamaHost}/api/embeddings, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ model: this.embeddingModel, prompt: text }), });

if (!response.ok) {
  throw new Error(`Failed to generate embedding: ${response.statusText}`);
}

const data = await response.json();
return data.embedding;

}

private async retrieveContext(query: string): Promise<string> { const embedding = await this.generateEmbedding(query);

const result = await this.pool.query(
  `SELECT string_agg(content, E'\n\n') as context
   FROM ${this.schema}.find_similar_chunks($1::vector, $2, $3)`,
  [formatVector(embedding), this.contextLimit, this.similarityThreshold]
);

return result.rows[0]?.context || '';

}

async generate(input: GenerateInput): Promise<string> { const context = await this.retrieveContext(input.prompt);

const response = await fetch(`${this.ollamaHost}/api/chat`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: input.model || this.chatModel,
    messages: this.buildMessages(input.messages || [], input.prompt, context),
    stream: false,
  }),
});

if (!response.ok) {
  throw new Error(`Chat failed: ${response.statusText}`);
}

const data = await response.json();
return data.message.content;

}

async generateStream(input: GenerateInput, callbacks: StreamCallbacks): Promise<void> { callbacks.onStateChange?.('thinking');

const context = await this.retrieveContext(input.prompt);

const response = await fetch(`${this.ollamaHost}/api/chat`, {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    model: input.model || this.chatModel,
    messages: this.buildMessages(input.messages || [], input.prompt, context),
    stream: true,
  }),
});

if (!response.ok) {
  const error = new Error(`Chat stream failed: ${response.statusText}`);
  callbacks.onStateChange?.('error');
  callbacks.onError?.(error);
  throw error;
}

await this.processStream(response, callbacks);

}

private buildMessages( messages: Message[], prompt: string, context: string ): Array<{ role: string; content: string }> { const systemMessage = messages.find(m => m.role === 'system'); const conversationMessages = messages.filter(m => m.role !== 'system');

const systemContent = systemMessage?.content || 'You are a helpful assistant.';
const contextualSystem = context
  ? `${systemContent}\n\nUse the following context to answer questions. If the context doesn't contain relevant information, say so.\n\nContext:\n${context}`
  : systemContent;

return [
  { role: 'system', content: contextualSystem },
  ...conversationMessages.map(m => ({ role: m.role, content: m.content })),
  { role: 'user', content: prompt },
];

}

private async processStream(response: Response, callbacks: StreamCallbacks): Promise<void> { const reader = response.body?.getReader(); if (!reader) { throw new Error('Failed to get response reader'); }

callbacks.onStateChange?.('streaming');

const decoder = new TextDecoder();
let fullResponse = '';
let done = false;

while (!done) {
  const result = await reader.read();
  done = result.done;
  if (done) break;

  const chunk = decoder.decode(result.value);
  const lines = chunk.split('\n').filter(Boolean);

  for (const line of lines) {
    try {
      const data = JSON.parse(line);
      if (data.message?.content) {
        fullResponse += data.message.content;
        callbacks.onToken?.(data.message.content);
      }
    } catch {
      // Skip malformed JSON lines
    }
  }
}

callbacks.onStateChange?.('complete');
callbacks.onComplete?.(fullResponse);

}

async isAvailable(): Promise<boolean> { try { const [ollamaCheck, dbCheck] = await Promise.all([ fetch(${this.ollamaHost}/api/tags).then(r => r.ok), this.pool.query('SELECT 1').then(() => true).catch(() => false), ]); return ollamaCheck && dbCheck; } catch { return false; } }

async close(): Promise<void> { await this.pool.end(); } }

Creating RAG-Enabled Kit

// src/lib/ai/create-rag-kit.ts import { AgentKit, OllamaProvider } from '@sf-ai/agentic-kit'; import { RAGProvider } from './rag-provider';

interface RAGKitConfig { ragEnabled?: boolean; databaseUrl?: string; ollamaHost?: string; embeddingModel?: string; chatModel?: string; similarityThreshold?: number; contextLimit?: number; schema?: string; }

export function createRAGKit(config: RAGKitConfig = {}): AgentKit { const ragEnabled = config.ragEnabled ?? process.env.RAG_ENABLED !== 'false'; const databaseUrl = config.databaseUrl || process.env.RAG_DATABASE_URL; const ollamaHost = config.ollamaHost || process.env.OLLAMA_HOST || 'http://localhost:11434';

const kit = new AgentKit();

if (ragEnabled && databaseUrl) { kit.addProvider(new RAGProvider({ databaseUrl, ollamaHost, embeddingModel: config.embeddingModel || process.env.RAG_EMBEDDING_MODEL, chatModel: config.chatModel || process.env.RAG_CHAT_MODEL, similarityThreshold: config.similarityThreshold || parseFloat(process.env.RAG_SIMILARITY_THRESHOLD || '0.5'), contextLimit: config.contextLimit || parseInt(process.env.RAG_CONTEXT_LIMIT || '5', 10), schema: config.schema || process.env.RAG_SCHEMA, })); } else { kit.addProvider(new OllamaProvider({ baseUrl: ollamaHost, defaultModel: config.chatModel || process.env.RAG_CHAT_MODEL || 'llama3.2', })); }

return kit; }

Updating useAgent Hook

Modify the useAgent hook to support RAG:

// src/lib/ai/use-agent.ts 'use client';

import { useCallback, useRef, useState } from 'react'; import type { AgentState, Message, StreamCallbacks } from '@sf-ai/agentic-kit'; import { createRAGKit } from './create-rag-kit';

export interface UseAgentOptions { baseUrl?: string; model?: string; systemPrompt?: string; ragEnabled?: boolean; databaseUrl?: string; }

export function useAgent(options: UseAgentOptions = {}) { const { baseUrl, model, systemPrompt, ragEnabled, databaseUrl } = options;

const kitRef = useRef(createRAGKit({ ragEnabled, databaseUrl, ollamaHost: baseUrl, chatModel: model, }));

// ... rest of the hook implementation }

Database Schema

The RAG database requires the following schema (created by the setup script):

-- Schema CREATE SCHEMA IF NOT EXISTS intelligence;

-- Documents table CREATE TABLE intelligence.documents ( id SERIAL PRIMARY KEY, title TEXT, content TEXT NOT NULL, metadata JSONB DEFAULT '{}'::jsonb, embedding VECTOR(768), created_at TIMESTAMPTZ DEFAULT NOW() );

-- Chunks table CREATE TABLE intelligence.chunks ( id SERIAL PRIMARY KEY, document_id INTEGER NOT NULL REFERENCES intelligence.documents(id) ON DELETE CASCADE, content TEXT NOT NULL, embedding VECTOR(768), chunk_index INTEGER NOT NULL, created_at TIMESTAMPTZ DEFAULT NOW() );

CREATE INDEX idx_chunks_document_id ON intelligence.chunks(document_id); CREATE INDEX idx_chunks_embedding ON intelligence.chunks USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

-- Similarity search function CREATE FUNCTION intelligence.find_similar_chunks( p_embedding VECTOR(768), p_limit INTEGER DEFAULT 5, p_similarity_threshold FLOAT DEFAULT 0.5 ) RETURNS TABLE ( id INTEGER, content TEXT, similarity FLOAT ) AS $$ BEGIN RETURN QUERY SELECT c.id, c.content, 1 - (c.embedding <=> p_embedding) AS similarity FROM intelligence.chunks c WHERE c.embedding IS NOT NULL AND 1 - (c.embedding <=> p_embedding) > p_similarity_threshold ORDER BY c.embedding <=> p_embedding LIMIT p_limit; END; $$ LANGUAGE plpgsql;

Adding Documents to the Knowledge Base

import { Pool } from 'pg';

const pool = new Pool({ connectionString: process.env.RAG_DATABASE_URL });

async function addDocument(title: string, content: string, metadata = {}) { const ollamaHost = process.env.OLLAMA_HOST || 'http://localhost:11434'; const embeddingModel = process.env.RAG_EMBEDDING_MODEL || 'nomic-embed-text';

// Generate embedding const response = await fetch(${ollamaHost}/api/embeddings, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ model: embeddingModel, prompt: content }), }); const { embedding } = await response.json();

// Insert document const result = await pool.query( INSERT INTO intelligence.documents (title, content, metadata, embedding) VALUES ($1, $2, $3, $4::vector) RETURNING id, [title, content, metadata, [${embedding.join(',')}]] );

const documentId = result.rows[0].id;

// Create chunks await pool.query('SELECT intelligence.create_document_chunks($1)', [documentId]);

// Generate embeddings for chunks const chunks = await pool.query( 'SELECT id, content FROM intelligence.chunks WHERE document_id = $1', [documentId] );

for (const chunk of chunks.rows) { const chunkResponse = await fetch(${ollamaHost}/api/embeddings, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ model: embeddingModel, prompt: chunk.content }), }); const { embedding: chunkEmbedding } = await chunkResponse.json();

await pool.query(
  'UPDATE intelligence.chunks SET embedding = $1::vector WHERE id = $2',
  [`[${chunkEmbedding.join(',')}]`, chunk.id]
);

}

return documentId; }

Troubleshooting

Issue Solution

"RAG_DATABASE_URL not set" Set the environment variable or pass databaseUrl to createRAGKit

"Connection refused" to database Ensure PostgreSQL is running (see pgpm-docker skill)

"Connection refused" to Ollama Ensure Ollama is running: ollama serve

"type vector does not exist" Run the setup script to install pgvector extension

No context retrieved Lower RAG_SIMILARITY_THRESHOLD or add more documents

Slow responses Reduce RAG_CONTEXT_LIMIT or optimize database indexes

References

• Related skill: pgvector-setup for database schema details

• Related skill: pgvector-embeddings for embedding generation

• Related skill: pgvector-similarity-search for retrieval queries

• Related skill: rag-pipeline for complete RAG implementation

• Related skill: ollama-integration for Ollama client details

• Related skill: pgpm-docker for PostgreSQL container management