Exa Reference Architecture

Overview

Production architecture for Exa neural search integration. Covers search pipeline design, content extraction patterns, embedding-based retrieval, and caching strategies for semantic search applications.

Prerequisites

• Exa API key

• Understanding of neural vs keyword search

• TypeScript project with async support

• Cache layer for search result deduplication

Architecture Diagram

┌─────────────────────────────────────────────────────┐ │ Application Layer │ │ RAG Pipeline │ Research Agent │ Content Discovery │ └───────────┬─────────────┬────────────┬──────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────────────────────────────────────────────────┐ │ Exa Search Service │ │ ┌───────────┐ ┌───────────┐ ┌────────────────┐ │ │ │ Neural │ │ Keyword │ │ Auto (hybrid) │ │ │ │ Search │ │ Search │ │ Search │ │ │ └─────┬─────┘ └─────┬─────┘ └──────┬─────────┘ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌──────────────────────────────────────────────┐ │ │ │ Content Extraction │ │ │ │ Text │ Highlights │ Full HTML │ Summary │ │ │ └──────────────────────┬───────────────────────┘ │ │ │ │ │ ┌──────────────────────┴───────────────────────┐ │ │ │ Result Cache (LRU + Redis) │ │ │ └──────────────────────────────────────────────┘ │ └─────────────────────────────────────────────────────┘

Instructions

Step 1: Exa Client Service Layer

import Exa from 'exa-js';

const exa = new Exa(process.env.EXA_API_KEY!);

interface SearchOptions { query: string; type?: 'neural' | 'keyword' | 'auto'; numResults?: number; startDate?: string; endDate?: string; includeDomains?: string[]; excludeDomains?: string[]; category?: string; }

async function searchWithContents(options: SearchOptions) { return exa.searchAndContents(options.query, { type: options.type || 'auto', numResults: options.numResults || 10, text: { maxCharacters: 3000 }, # 3000: 3 seconds in ms highlights: { numSentences: 3 }, startPublishedDate: options.startDate, endPublishedDate: options.endDate, includeDomains: options.includeDomains, excludeDomains: options.excludeDomains, category: options.category, }); }

Step 2: Search Pipeline with Content Extraction

async function researchTopic(topic: string) { // Step 1: Broad neural search for relevant sources const sources = await exa.searchAndContents(topic, { type: 'neural', numResults: 20, text: true, highlights: { numSentences: 5 }, startPublishedDate: '2024-01-01', # 2024 year category: 'research paper', });

// Step 2: Find similar content to top results const topUrl = sources.results[0]?.url; const similar = topUrl ? await exa.findSimilarAndContents(topUrl, { numResults: 5, text: { maxCharacters: 2000 }, # 2000: 2 seconds in ms }) : { results: [] };

return { primary: sources.results, similar: similar.results, totalSources: sources.results.length + similar.results.length, }; }

Step 3: RAG Integration Pattern

async function ragSearch( userQuery: string, contextWindow = 5 ) { const results = await exa.searchAndContents(userQuery, { type: 'neural', numResults: contextWindow, text: { maxCharacters: 2000 }, # 2000: 2 seconds in ms highlights: { numSentences: 3 }, });

// Format for LLM context injection const context = results.results.map((r, i) => ( [Source ${i + 1}] ${r.title}\n + URL: ${r.url}\n + Content: ${r.text}\n + Key highlights: ${r.highlights?.join(' | ')}\n )).join('\n---\n');

return { context, sources: results.results.map(r => ({ title: r.title, url: r.url, score: r.score, })), }; }

Step 4: Domain-Specific Search Configuration

const SEARCH_PROFILES = { technical: { includeDomains: ['github.com', 'stackoverflow.com', 'arxiv.org', 'docs.python.org'], category: 'github repo' as const, }, news: { includeDomains: ['techcrunch.com', 'theverge.com', 'arstechnica.com'], category: 'news' as const, }, research: { includeDomains: ['arxiv.org', 'scholar.google.com', 'nature.com'], category: 'research paper' as const, }, };

async function profiledSearch(query: string, profile: keyof typeof SEARCH_PROFILES) { const config = SEARCH_PROFILES[profile]; return searchWithContents({ query, ...config, numResults: 10 }); }

Error Handling

Issue Cause Solution

No results Query too specific Broaden query, switch to neural search

Low relevance Wrong search type Use auto type for hybrid results

Content extraction empty Site blocks scraping Use highlights instead of full text

Rate limit Too many concurrent requests Add request queue with delays

Examples

Content Discovery Pipeline

async function discoverCompetitors(companyUrl: string) { const similar = await exa.findSimilar(companyUrl, { numResults: 10, excludeDomains: [new URL(companyUrl).hostname], }); return similar.results.map(r => ({ title: r.title, url: r.url })); }

Resources

• Exa API Documentation

• Exa Search Types

• Exa Content Extraction

Output

• Configuration files or code changes applied to the project

• Validation report confirming correct implementation

• Summary of changes made and their rationale