Scaling & Availability — 📈 Building Resilient Applications

Scaling handles growing demand. Availability ensures your app stays online even when things fail.

🎯 Two Types of Scaling

Vertical Scaling (Bigger Server)

Buy a more powerful server.

Before:  4GB RAM, 2 CPU
After:   64GB RAM, 8 CPU

Cost: 2x-3x more expensive
Problem: Eventually hits hardware limits

Pros: Simple, one server to manage

Cons: Expensive, has limits, downtime to upgrade

Horizontal Scaling (More Servers)

Add more servers sharing the load.

Before:  1 server handling 1,000 req/sec
After:   4 servers handling 4,000 req/sec

Cost: Linear growth with demand
Benefit: Infinite scalability

Pros: Scalable, cost-effective, resilient

Cons: Requires load balancer, more complex

💡 Auto-scaling

Automatically add/remove servers based on demand.

8 AM (Peak):     Add servers
│
├─ 1,000 users → 5 servers
├─ 10,000 users → 50 servers
├─ 100,000 users → 500 servers
│
6 PM (Evening):   Remove servers
├─ 10,000 users → 50 servers
└─ 1,000 users → 5 servers

How it works:

1. Monitor CPU usage

2. If CPU > 70% for 5 minutes → Add server

3. If CPU < 30% for 10 minutes → Remove server

4. Automatic, no manual intervention

Savings:

Fixed servers: 50 servers × $100/month = $5,000
Auto-scaled: Average 15 servers × $100 = $1,500
Savings: 70%

📊 Load Balancing

Distributes incoming requests across multiple servers.

Incoming Request
        ↓
   Load Balancer
    /    |    \
   /     |     \
[Server1] [Server2] [Server3]

Algorithm:

• **Round Robin:** Rotate requests (1→2→3→1→2→3)

• **Least Connections:** Send to least busy server

• **IP Hash:** Same IP always goes to same server

Example:

1,000 requests/sec
÷ 10 servers
= 100 requests/sec per server
(manageable, no overload)

🛡️ High Availability

Application stays online even when components fail.

Single Server (No HA):

Server crashes → App down → Customers angry
MTTR: 30 minutes

Multiple Servers (HA):

Server 1 crashes → Load balancer routes to Server 2 & 3
Application still works → Customers don't notice
MTTR: 0 (automatic)

🌍 Geographic Distribution

Deploy across multiple regions worldwide.

Europe             Asia             Americas
├─ London          ├─ Tokyo         ├─ New York
├─ Frankfurt       ├─ Singapore     └─ Los Angeles
└─ Amsterdam       └─ Mumbai

If London data center fails → Traffic routes to Frankfurt

Benefits:

• Faster access (servers near users)

• Disaster recovery (multiple locations)

• Regulatory compliance (data location requirements)

🎨 Real-World Example: Netflix's Infrastructure

Users worldwide
        ↓
CloudFront (Global CDN - caches content)
        ↓
Multiple AWS regions
├─ US-East
├─ US-West
├─ Europe
└─ Asia-Pacific

Each region has:
├─ Load balancers
├─ Multiple app servers (auto-scaling)
└─ Multiple database replicas

Result:

• Never a single point of failure

• If entire region fails, traffic routes elsewhere

• 99.99% uptime SLA

📈 Key Metrics

🔑 Key Takeaways

• ✅ Vertical scaling: bigger server (limited)

• ✅ Horizontal scaling: more servers (unlimited)

• ✅ Auto-scaling: add/remove servers automatically

• ✅ Load balancers distribute traffic

• ✅ High availability: multiple redundant components

• ✅ Geographic distribution: data centers worldwide

• ✅ Proper scaling/HA prevents outages and saves money

Want production patterns? 📈 Scaling & Availability (Experienced)