Compute Options — ⚙️ Advanced Patterns and Optimization

VMs: Instance Type Selection and Performance

Instance Type Families

General Purpose (Balanced):

• t3, t4g: Burstable (good for unpredictable workloads)

• m5, m6i: Fixed performance (consistent workloads)

• Cost per month: $20-100

• Best for: Web servers, app servers, most workloads

CPU Optimized:

• c5, c6i: High clock speed, fewer cores

• Use case: Batch processing, HPC, compilers

• Cost per month: $40-200

Memory Optimized:

• r5, r6i, x1: Large RAM (256GB-4TB)

• Use case: In-memory databases, caching, SAP

• Cost per month: $100-1,000+

Storage Optimized:

• i3, d2: High-throughput storage

• Use case: NoSQL databases, data warehouses

• Cost per month: $50-500

GPU Instances:

• p3, g4dn: NVIDIA/AMD GPUs

• Use case: ML training, rendering, simulations

• Cost per month: $500-5,000+ (GPU = expensive)

Containers and Kubernetes

Kubernetes Scaling Architecture

Three-tier scaling:

1. Pod autoscaling (HPA): 10-200 pods
   - Responds to request volume in seconds
   - CPU/memory/custom metrics

2. Cluster autoscaling: 1-1,000 nodes
   - Adds nodes when needed
   - Responds in 1-2 minutes

3. Multi-cluster: 1-many clusters
   - Global failover
   - Multi-region resilience

Cost Optimization in Kubernetes

Spot instances in Kubernetes:

70% spot nodes (cost optimized)
+ 30% on-demand nodes (availability)
= 40-50% total cost reduction

Pod-disruption-budgets:
├─ Allow rolling updates without service loss
├─ Gradual eviction from spot nodes
└─ Graceful shutdown with connection draining

Serverless: Production Patterns

Cold Start Optimization

Language performance (first request):

Go: 100-200ms (fastest)
Node.js: 300-700ms (good)
Python: 600-1,200ms (slower)
Java: 1,000-3,000ms (slowest)

Optimization for Python:
├─ Use Lambda Layers for dependencies
├─ Move imports outside handler
├─ Minimal package size
└─ Result: 40-50% faster execution

Concurrency and Throttling

Lambda concurrent execution:

Account limit: 1,000 (default)
Per-function limit: Configurable
Per-instance limit: 1 (sequential) or 100 (Python 3.9+)

Scaling formula:
Concurrent invocations = (Invocations per sec) × (Duration in seconds)

Example:
100 requests/sec × 5 second execution = 500 concurrent
Your account limit = 1,000
Overhead: 500 unused concurrency

Hybrid Architectures

Multi-cloud Compute

Portable patterns:

Kubernetes: Works across all clouds
├─ GKE (GCP)
├─ EKS (AWS)
└─ AKS (Azure)

Docker: Standard container format
├─ ECR (AWS)
├─ Artifact Registry (GCP)
├─ ACR (Azure)

Non-portable (lock-in):

Lambda: AWS-only
Cloud Functions: GCP-only
Azure Functions: Azure-only

To migrate: Full rewrite of functions

Performance Benchmarks

Request latency by compute model

Database query latency (target: <10ms):
├─ VM with local SSD: <5ms ✅
├─ Container in Kubernetes: <10ms ✅
├─ Serverless with cold start: 500ms-2s ❌

For latency-sensitive: Use VMs or containers
For API backends: Serverless acceptable (p99 latency)

Key Takeaways

• ✅ VMs: Full control, good for databases

• ✅ Containers: Balance of control and automation

• ✅ Serverless: Cost-effective for APIs and events

• ✅ Spot instances: 70-90% savings for non-critical

• ✅ Kubernetes: Enterprise-grade orchestration

• ✅ Cold starts: <1s usually acceptable for most workloads

• ✅ Choose based on: complexity tolerance, cost, performance needs