Estimation Techniques

This skill provides frameworks for back-of-envelope calculations essential for system design and capacity planning.

When to Use This Skill

Keywords: back-of-envelope, estimation, QPS, storage calculation, bandwidth, latency, capacity planning, scale estimation

Use this skill when:

• Estimating system capacity requirements

• Calculating storage needs for a feature

• Determining bandwidth requirements

• Sizing infrastructure for expected load

• Justifying architectural decisions with numbers

• Preparing for system design interviews

Core Principle

Estimation is not about precision, it's about order of magnitude.

Getting within 10x is usually good enough for architectural decisions. The goal is to identify if you need:

• 1 server or 100 servers

• 1 GB or 1 TB of storage

• 10 ms or 1 second latency

Essential Numbers to Know

Powers of 2

Power Value Approximate

2^10 1,024 ~1 Thousand (KB)

2^20 1,048,576 ~1 Million (MB)

2^30 1,073,741,824 ~1 Billion (GB)

2^40 1,099,511,627,776 ~1 Trillion (TB)

Time Conversions

Unit Seconds Useful For

1 minute 60 Short operations

1 hour 3,600 Batch jobs

1 day 86,400 (~100K) Daily aggregations

1 month 2,592,000 (~2.5M) Monthly calculations

1 year 31,536,000 (~30M) Annual projections

Availability Targets

Availability Downtime/Year Downtime/Month Downtime/Day

99% (two 9s) 3.65 days 7.31 hours 14.4 min

99.9% (three 9s) 8.76 hours 43.8 min 1.44 min

99.99% (four 9s) 52.6 min 4.38 min 8.64 sec

99.999% (five 9s) 5.26 min 26.3 sec 864 ms

Latency Numbers Every Programmer Should Know

See full reference: references/latency-numbers.md

Quick reference:

Operation Latency Relative

L1 cache reference 0.5 ns 1x

L2 cache reference 7 ns 14x

Main memory reference 100 ns 200x

SSD random read 16 us 32,000x

HDD seek 2 ms 4,000,000x

Round trip same datacenter 0.5 ms 1,000,000x

Round trip CA to Netherlands 150 ms 300,000,000x

Estimation Patterns

Pattern 1: QPS (Queries Per Second)

Formula

QPS = (Number of Users) x (Actions per User per Day) / (Seconds per Day)

Example: Twitter-like service

Given:

• 300 million monthly active users
• 50% are daily active = 150M DAU
• Average user reads 20 tweets/day

QPS = 150M * 20 / 86,400 = 3 billion / 100,000 = 30,000 QPS

Peak load (typically 2-3x average): Peak QPS = 30,000 * 3 = 90,000 QPS

Pattern 2: Storage Estimation

Formula

Storage = (Number of Items) x (Size per Item) x (Replication Factor) x (Time Period)

Example: Photo storage service

Given:

• 100 million users
• 10% upload daily = 10M uploads/day
• Average photo size = 2 MB
• Keep 5 years of data
• Replication factor = 3

Daily storage = 10M * 2 MB = 20 TB Yearly storage = 20 TB * 365 = 7.3 PB 5-year storage = 7.3 * 5 = 36.5 PB With replication = 36.5 * 3 = ~110 PB

Pattern 3: Bandwidth Estimation

Formula

Bandwidth = (QPS) x (Request Size or Response Size)

Example: Video streaming service

Given:

• 1 million concurrent viewers
• Average bitrate = 5 Mbps
• Peak hours: 8 PM - 11 PM

Bandwidth = 1M * 5 Mbps = 5 Tbps

With 20% overhead: ~6 Tbps

CDN egress cost (rough): $0.02/GB * 6 Tbps * 3 hours * 3600 sec/hour / 8 bits/byte = massive cost (hence why Netflix built their own CDN)

Pattern 4: Cache Size Estimation

Formula

Cache Size = (QPS) x (Cache TTL) x (Response Size) x (Unique Ratio)

Example: API response cache

Given:

• 10,000 QPS
• Cache TTL = 5 minutes = 300 seconds
• Average response = 10 KB
• 20% of requests are unique

Cache entries = 10,000 * 300 * 0.20 = 600,000 entries Cache size = 600,000 * 10 KB = 6 GB

With overhead (keys, metadata): ~10 GB

Pattern 5: Database Sizing

Formula

DB Size = (Number of Rows) x (Row Size) x (Index Overhead) x (Replication)

Example: User profile database

Given:

• 500 million users
• Average profile = 1 KB (name, email, settings, etc.)
• Index overhead = 30%
• Primary + 2 replicas = 3x

Data size = 500M * 1 KB = 500 GB With indexes = 500 GB * 1.3 = 650 GB With replication = 650 GB * 3 = ~2 TB

Memory for hot data (20%): ~400 GB

Common Estimation Scenarios

Scenario 1: URL Shortener

Requirements:

• 100M new URLs/month
• 10:1 read:write ratio

Writes:

• 100M / (30 * 24 * 3600) = ~40 writes/second
• Peak: ~100 writes/second

Reads:

• 40 * 10 = 400 reads/second
• Peak: ~1000 reads/second

Storage (5 years):

• 100M URLs/month * 60 months = 6 billion URLs
• Average URL = 100 bytes (short) + 500 bytes (long) = 600 bytes
• 6B * 600 bytes = 3.6 TB
• With indexes and overhead: ~5 TB

Scenario 2: Chat Application

Requirements:

• 10M daily active users
• Average 50 messages sent/day
• Average 200 messages received/day

Message throughput:

• Sends: 10M * 50 / 86,400 = ~6,000 messages/second
• Peak: ~20,000 messages/second

Connections:

• Each user maintains 1-3 connections (phone, laptop, tablet)
• Peak concurrent: 10M * 0.1 (10% online) * 2 = 2M connections

Storage (1 year):

• 10M users * 50 msgs/day * 365 days = 182B messages/year
• Average message = 200 bytes
• 182B * 200 bytes = 36.4 TB/year

Scenario 3: Video Streaming

Requirements:

• 100M monthly active users
• 30% watch daily = 30M DAU
• Average 1 hour/day viewing

Concurrent viewers (peak):

• 30M DAU / 24 hours * 3 (peak factor) = ~4M concurrent

Bandwidth:

• Average stream: 5 Mbps
• 4M * 5 Mbps = 20 Tbps peak bandwidth

Storage (library of 10K titles):

• Average video = 2 hours
• Multiple qualities: 480p (1GB), 720p (3GB), 1080p (5GB), 4K (20GB)
• Per title: ~30 GB
• Library: 10K * 30 GB = 300 TB

Estimation Tips

Round Aggressively

Instead of: Use: 86,400 seconds ~100,000 (10^5) 2.5 million ~3 million 7.3 petabytes ~10 petabytes

Use Orders of Magnitude

Think in powers of 10:

• Thousands (10^3)

• Millions (10^6)

• Billions (10^9)

• Trillions (10^12)

State Your Assumptions

Always verbalize:

• "I'm assuming 10% of users are active at peak"

• "I'm estimating average message size at 200 bytes"

• "I'm using a 3x replication factor"

Sanity Check Results

After calculating, ask:

• "Does this make sense?"

• "Is this in the right order of magnitude?"

• "What would change if my assumption is off by 10x?"

Common Mistakes

Mistake 1: Ignoring Peak vs Average

Problem

Sizing for average load.

Average QPS: 10,000 Peak QPS: 30,000 (often 2-3x average)

If you size for 10,000, you'll fail at peak.

Mistake 2: Forgetting Replication

Problem

Calculating raw storage without copies.

Data: 1 TB With 3 replicas: 3 TB With backups: 4-5 TB

Mistake 3: Not Accounting for Growth

Problem

Sizing for current, not future.

Current users: 10M Expected growth: 50%/year Year 3: 10M * 1.5^3 = 34M users

Size for at least 2x current to avoid near-term issues.

Mistake 4: Over-Precision

Problem

Calculating to 3 decimal places.

Bad: "We need exactly 3,456,789 IOPS" Good: "We need roughly 3-4 million IOPS"

Quick Reference Calculations

Need Formula

QPS users * actions/day / 86400

Storage/day items/day * size/item

Bandwidth QPS * response_size

Cache hit rate 1 - (DB_QPS / total_QPS)

Servers needed QPS / QPS_per_server

Shards needed data_size / max_shard_size

Related Skills

• design-interview-methodology

• Overall interview framework

• quality-attributes-taxonomy

• NFR definitions (scalability, performance)

• database-scaling

• Database capacity planning (Phase 3)

• caching-strategies

• Cache sizing and hit rates (Phase 3)

Related Commands

• /sd:estimate <scenario>
• Calculate capacity interactively

Related Agents

• capacity-planner
• Guided estimation with calculations

References

• references/latency-numbers.md
• Complete latency reference table

Version History

• v1.0.0 (2025-12-26): Initial release

Last Updated

Date: 2025-12-26 Model: claude-opus-4-5-20251101