Vector Databases

When to Use This Skill

Use this skill when:

• Choosing between vector database options

• Designing semantic/similarity search systems

• Optimizing vector search performance

• Understanding ANN algorithm trade-offs

• Scaling vector search infrastructure

• Implementing hybrid search (vectors + filters)

Keywords: vector database, embeddings, vector search, similarity search, ANN, approximate nearest neighbor, HNSW, IVF, FAISS, Pinecone, Weaviate, Milvus, Qdrant, Chroma, pgvector, cosine similarity, semantic search

Vector Database Comparison

Managed Services

Database Strengths Limitations Best For

Pinecone Fully managed, easy scaling, enterprise Vendor lock-in, cost at scale Enterprise production

Weaviate Cloud GraphQL, hybrid search, modules Complexity Knowledge graphs

Zilliz Cloud Milvus-based, high performance Learning curve High-scale production

MongoDB Atlas Vector Existing MongoDB users Newer feature MongoDB shops

Elastic Vector Existing Elastic stack Resource heavy Search platforms

Self-Hosted Options

Database Strengths Limitations Best For

Milvus Feature-rich, scalable, GPU support Operational complexity Large-scale production

Qdrant Rust performance, filtering, easy Smaller ecosystem Performance-focused

Weaviate Modules, semantic, hybrid Memory usage Knowledge applications

Chroma Simple, Python-native Limited scale Development, prototyping

pgvector PostgreSQL extension Performance limits Postgres shops

FAISS Library, not DB, fastest No persistence, no filtering Research, embedded

Selection Decision Tree

Need managed, don't want operations? ├── Yes → Pinecone (simplest) or Weaviate Cloud └── No (self-hosted) └── Already using PostgreSQL? ├── Yes, <1M vectors → pgvector └── No └── Need maximum performance at scale? ├── Yes → Milvus or Qdrant └── No └── Prototyping/development? ├── Yes → Chroma └── No → Qdrant (balanced choice)

ANN Algorithms

Algorithm Overview

Exact KNN: • Search ALL vectors • O(n) time complexity • Perfect accuracy • Impractical at scale

Approximate NN (ANN): • Search SUBSET of vectors • O(log n) to O(1) complexity • Near-perfect accuracy • Practical at any scale

HNSW (Hierarchical Navigable Small World)

Layer 3: ○───────────────────────○ (sparse, long connections) │ │ Layer 2: ○───○───────○───────○───○ (medium density) │ │ │ │ │ Layer 1: ○─○─○─○─○─○─○─○─○─○─○─○─○ (denser) │││││││││││││││││││││││ Layer 0: ○○○○○○○○○○○○○○○○○○○○○○○○○ (all vectors)

Search: Start at top layer, greedily descend • Fast: O(log n) search time • High recall: >95% typically • Memory: Extra graph storage

HNSW Parameters:

Parameter Description Trade-off

M

Connections per node Memory vs. recall

ef_construction

Build-time search width Build time vs. recall

ef_search

Query-time search width Latency vs. recall

IVF (Inverted File Index)

Clustering Phase: ┌─────────────────────────────────────────┐ │ Cluster vectors into K centroids │ │ │ │ ● ● ● ● │ │ /│\ /│\ /│\ /│\ │ │ ○○○○○ ○○○○○ ○○○○○ ○○○○○ │ │ Cluster 1 Cluster 2 Cluster 3 Cluster 4│ └─────────────────────────────────────────┘

Search Phase:

1. Find nprobe nearest centroids
2. Search only those clusters
3. Much faster than exhaustive

IVF Parameters:

Parameter Description Trade-off

nlist

Number of clusters Build time vs. search quality

nprobe

Clusters to search Latency vs. recall

IVF-PQ (Product Quantization)

Original Vector (128 dim): [0.1, 0.2, ..., 0.9] (128 × 4 bytes = 512 bytes)

PQ Compressed (8 subvectors, 8-bit codes): [23, 45, 12, 89, 56, 34, 78, 90] (8 bytes)

Memory reduction: 64x Accuracy trade-off: ~5% recall drop

Algorithm Comparison

Algorithm Search Speed Memory Build Time Recall

Flat/Brute Slow (O(n)) Low None 100%

IVF Fast Low Medium 90-95%

IVF-PQ Very fast Very low Medium 85-92%

HNSW Very fast High Slow 95-99%

HNSW+PQ Very fast Medium Slow 90-95%

When to Use Which

< 100K vectors: └── Flat index (exact search is fast enough)

100K - 1M vectors: └── HNSW (best recall/speed trade-off)

1M - 100M vectors: ├── Memory available → HNSW └── Memory constrained → IVF-PQ or HNSW+PQ

100M vectors: └── Sharded IVF-PQ or distributed HNSW

Distance Metrics

Common Metrics

Metric Formula Range Best For

Cosine Similarity A·B / (||A|| ||B||)

[-1, 1] Normalized embeddings

Dot Product A·B

(-∞, ∞) When magnitude matters

Euclidean (L2) √Σ(A-B)²

[0, ∞) Absolute distances

Manhattan (L1) Σ|A-B|

[0, ∞) High-dimensional sparse

Metric Selection

Embeddings pre-normalized (unit vectors)? ├── Yes → Cosine = Dot Product (use Dot, faster) └── No └── Magnitude meaningful? ├── Yes → Dot Product └── No → Cosine Similarity

Note: Most embedding models output normalized vectors → Dot product is usually the best choice

Filtering and Hybrid Search

Pre-filtering vs Post-filtering

Pre-filtering (Filter → Search): ┌─────────────────────────────────────────┐ │ 1. Apply metadata filter │ │ (category = "electronics") │ │ Result: 10K of 1M vectors │ │ │ │ 2. Vector search on 10K vectors │ │ Much faster, guaranteed filter match │ └─────────────────────────────────────────┘

Post-filtering (Search → Filter): ┌─────────────────────────────────────────┐ │ 1. Vector search on 1M vectors │ │ Return top-1000 │ │ │ │ 2. Apply metadata filter │ │ May return < K results! │ └─────────────────────────────────────────┘

Hybrid Search Architecture

Query: "wireless headphones under $100" │ ┌─────┴─────┐ ▼ ▼ ┌───────┐ ┌───────┐ │Vector │ │Filter │ │Search │ │ Build │ │"wire- │ │price │ │less │ │< 100 │ │head- │ │ │ │phones"│ │ │ └───────┘ └───────┘ │ │ └─────┬─────┘ ▼ ┌───────────┐ │ Combine │ │ Results │ └───────────┘

Metadata Index Design

Metadata Type Index Strategy Query Example

Categorical Bitmap/hash index category = "books"

Numeric range B-tree price BETWEEN 10 AND 50

Keyword search Inverted index tags CONTAINS "sale"

Geospatial R-tree/geohash location NEAR (lat, lng)

Scaling Strategies

Sharding Approaches

Naive Sharding (by ID): ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Shard 1 │ │ Shard 2 │ │ Shard 3 │ │ IDs 0-N │ │IDs N-2N │ │IDs 2N-3N│ └─────────┘ └─────────┘ └─────────┘ Query → Search ALL shards → Merge results

Semantic Sharding (by cluster): ┌─────────┐ ┌─────────┐ ┌─────────┐ │ Shard 1 │ │ Shard 2 │ │ Shard 3 │ │ Tech │ │ Health │ │ Finance │ │ docs │ │ docs │ │ docs │ └─────────┘ └─────────┘ └─────────┘ Query → Route to relevant shard(s) → Faster!

Replication

┌─────────────────────────────────────────┐ │ Load Balancer │ └─────────────────────────────────────────┘ │ │ │ ▼ ▼ ▼ ┌─────────┐ ┌─────────┐ ┌─────────┐ │Replica 1│ │Replica 2│ │Replica 3│ │ (Read) │ │ (Read) │ │ (Read) │ └─────────┘ └─────────┘ └─────────┘ │ │ │ └───────────┼───────────┘ │ ┌─────────┐ │ Primary │ │ (Write) │ └─────────┘

Scaling Decision Matrix

Scale (vectors) Architecture Replication

< 1M Single node Optional

1-10M Single node, more RAM For HA

10-100M Sharded, few nodes Required

100M-1B Sharded, many nodes Required

1B Sharded + tiered Required

Performance Optimization

Index Build Optimization

Optimization Description Impact

Batch insertion Insert in batches of 1K-10K 10x faster

Parallel build Multi-threaded index construction 2-4x faster

Incremental index Add to existing index Avoids rebuild

GPU acceleration Use GPU for training (IVF) 10-100x faster

Query Optimization

Optimization Description Impact

Warm cache Keep index in memory 10x latency reduction

Query batching Batch similar queries Higher throughput

Reduce dimensions PCA, random projection 2-4x faster

Early termination Stop when "good enough" Lower latency

Memory Optimization

Memory per vector: ┌────────────────────────────────────────┐ │ 1536 dims × 4 bytes = 6KB per vector │ │ │ │ 1M vectors: │ │ Raw: 6GB │ │ + HNSW graph: +2-4GB (M-dependent) │ │ = 8-10GB total │ │ │ │ With PQ (64 subquantizers): │ │ 1M vectors: ~64MB │ │ = 100x reduction │ └────────────────────────────────────────┘

Operational Considerations

Backup and Recovery

Strategy Description RPO/RTO

Snapshots Periodic full backup Hours

WAL replication Write-ahead log streaming Minutes

Real-time sync Synchronous replication Seconds

Monitoring Metrics

Metric Description Alert Threshold

Query latency p99 99th percentile latency

100ms

Recall Search accuracy < 90%

QPS Queries per second Capacity dependent

Memory usage Index memory

80%

Index freshness Time since last update Domain dependent

Index Maintenance

┌─────────────────────────────────────────┐ │ Index Maintenance Tasks │ ├─────────────────────────────────────────┤ │ • Compaction: Merge small segments │ │ • Reindex: Rebuild degraded index │ │ • Vacuum: Remove deleted vectors │ │ • Optimize: Tune parameters │ │ │ │ Schedule during low-traffic periods │ └─────────────────────────────────────────┘

Common Patterns

Multi-Tenant Vector Search

Option 1: Namespace/Collection per tenant ┌─────────────────────────────────────────┐ │ tenant_1_collection │ │ tenant_2_collection │ │ tenant_3_collection │ └─────────────────────────────────────────┘ Pro: Complete isolation Con: Many indexes, operational overhead

Option 2: Single collection + tenant filter ┌─────────────────────────────────────────┐ │ shared_collection │ │ metadata: { tenant_id: "..." } │ │ Pre-filter by tenant_id │ └─────────────────────────────────────────┘ Pro: Simpler operations Con: Requires efficient filtering

Real-Time Updates

Write Path: ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ Write │ │ Buffer │ │ Merge │ │ Request │───▶│ (Memory) │───▶│ to Index │ └─────────────┘ └─────────────┘ └─────────────┘

Strategy:

1. Buffer writes in memory
2. Periodically merge to main index
3. Search: main index + buffer
4. Compact periodically

Embedding Versioning

Version 1 embeddings ──┐ │ Version 2 embeddings ──┼──▶ Parallel indexes during migration │ │ ┌─────────────────────┐ └───▶│ Gradual reindexing │ │ Blue-green switch │ └─────────────────────┘

Cost Estimation

Storage Costs

Cost = (vectors × dimensions × bytes × replication) / GB × $/GB/month

Example: 10M vectors × 1536 dims × 4 bytes × 3 replicas = 184 GB At $0.10/GB/month = $18.40/month storage

Note: Memory (for serving) costs more than storage

Compute Costs

Factors: • QPS (queries per second) • Latency requirements • Index type (HNSW needs more RAM) • Filtering complexity

Rule of thumb: • 1M vectors, HNSW, <50ms latency: 16GB RAM • 10M vectors, HNSW, <50ms latency: 64-128GB RAM • 100M vectors: Distributed system required

Related Skills

• rag-architecture

• Using vector databases in RAG systems

• llm-serving-patterns

• LLM inference with vector retrieval

• ml-system-design

• End-to-end ML pipeline design

• estimation-techniques

• Capacity planning for vector systems

Version History

• v1.0.0 (2025-12-26): Initial release - Vector database patterns for systems design

Last Updated

Date: 2025-12-26