Scalability Patterns

Implement scalability patterns to handle growth, improve performance, and maintain reliability under increasing load.

When to Use This Skill

• Performance bottlenecks

• User growth

• Data volume increase

• High traffic events

• Geographic expansion

• Cost optimization

• Architecture modernization

• SLA requirements

Core Concepts

1. Horizontal vs Vertical Scaling

Vertical Scaling (Scale Up):

• Add more CPU, RAM to existing server
• Limits: Hardware ceiling, single point of failure
• Use case: Databases, legacy apps

Horizontal Scaling (Scale Out):

• Add more servers
• Benefits: No limit, fault tolerance
• Requires: Stateless design, load balancing
• Use case: Web servers, app servers

Recommendation: Design for horizontal scaling

2. Caching Strategies

Cache Architecture

Cache-Aside (Lazy Loading):

• App checks cache

• If miss, read from DB

• Write to cache

• Return data

Write-Through:

• App writes to cache

• Cache writes to DB

• Return success

Write-Behind:

• App writes to cache

• Return success immediately

• Cache async writes to DB (eventual consistency)

Use Cases:

• CDN: Static assets (images, CSS, JS)
• Redis: Session data, API responses
• Browser cache: User-specific data
• Application cache: Configuration, reference data

Example - Redis Caching:

async function getUser(userId) {
  // Try cache first
  let user = await redis.get(`user:${userId}`);
  
  if (!user) {
    // Cache miss - get from database
    user = await database.query('SELECT * FROM users WHERE id = ?', [userId]);
    // Store in cache (TTL: 1 hour)
    await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
  }
  
  return JSON.parse(user);
}

### 3. Database Scaling

```markdown
## Database Scaling Strategies

**Read Replicas:**
- Master: Write operations
- Replicas: Read operations
- Reduces load on master
- Eventual consistency

**Sharding (Horizontal Partitioning):**
- Split data across multiple databases
- Shard key (e.g., user_id % num_shards)
- Challenges: Joins, resharding

**Vertical Partitioning:**
- Split tables by columns
- Separate hot/cold data
- Example: User profile vs user activity logs

**CQRS (Command Query Responsibility Segregation):**
- Separate read and write models
- Optimized for different use cases
- Event sourcing integration

4. Load Balancing

## Load Balancer Strategies

**Round Robin:**
Server 1 → Server 2 → Server 3 → Server 1...

**Least Connections:**
Route to server with fewest active connections

**IP Hash:**
Route based on client IP (sticky sessions)

**Weighted:**
More requests to more powerful servers

**Health Checks:**
- Monitor server health
- Remove unhealthy servers
- Automatic failover

**Example Architecture:**

Internet → CloudFlare CDN
→ AWS ALB (Application Load Balancer)
→ Auto Scaling Group
→ EC2 Instance 1
→ EC2 Instance 2
→ EC2 Instance 3

Best Practices

- Stateless design - Store state externally (Redis, DB)

- Async processing - Use message queues for heavy tasks

- Cache aggressively - Multiple cache layers

- Database optimization - Indexes, query optimization

- Monitor metrics - CPU, memory, response time, error rate

- Auto-scaling - Scale based on metrics

- Graceful degradation - Reduce functionality vs complete failure

- Load testing - Identify bottlenecks before production

Resources

- Scalability Rules: Martin Abbott

- High Performance Browser Networking: Ilya Grigorik