📁 Storage Services - Advanced Strategies

Lifecycle Management & Cost Optimization

S3 Lifecycle Policies

<LifecycleConfiguration>
  <Rule>
    <ID>TransitionToIA</ID>
    <Filter><Prefix>logs/</Prefix></Filter>
    <Transitions>
      <Transition>
        <Days>30</Days>
        <StorageClass>STANDARD_IA</StorageClass>
      </Transition>
      <Transition>
        <Days>60</Days>
        <StorageClass>GLACIER</StorageClass>
      </Transition>
    </Transitions>
    <Expiration><Days>365</Days></Expiration>
  </Rule>
</LifecycleConfiguration>

GCP Lifecycle Management

{
  "lifecycle": {
    "rule": [
      {
        "action": {"type": "SetStorageClass", "storageClass": "NEARLINE"},
        "condition": {"age": 30}
      },
      {
        "action": {"type": "SetStorageClass", "storageClass": "COLDLINE"},
        "condition": {"age": 90}
      },
      {
        "action": {"type": "Delete"},
        "condition": {"age": 365}
      }
    ]
  }
}

Cost Analysis Example

100TB data, typical access pattern:
- Hot (current): 10TB accessed daily
- Warm (last 90 days): 30TB accessed monthly
- Cold (archived): 60TB accessed rarely

AWS:
  10TB Standard: $230/month
  30TB IA: $150/month
  60TB Glacier: $240/month
  Total: $620/month

GCP:
  10TB Standard: $200/month
  30TB Nearline: $150/month
  60TB Coldline: $240/month
  Total: $590/month

Savings with lifecycle: ~$70/month = $840/year

Advanced Replication Strategies

Cross-Region Replication (AWS S3)

// Primary bucket → Secondary bucket (async)
const s3 = new AWS.S3();

await s3.putBucketReplication({
  Bucket: 'primary-bucket',
  ReplicationConfiguration: {
    Role: 'arn:aws:iam::123456789012:role/s3-replication',
    Rules: [{
      Status: 'Enabled',
      Priority: 1,
      Filter: { Prefix: 'critical/' },
      Destination: {
        Bucket: 'arn:aws:s3:::secondary-bucket',
        ReplicationTime: { Status: 'Enabled', Time: { Minutes: 15 } },
        Metrics: { Status: 'Enabled', EventThreshold: { Minutes: 15 } }
      }
    }]
  }
}).promise();

Multi-Region Buckets (GCP)

# Automatic multi-region replication
gsutil mb -l US -b on gs://my-bucket/

# Data automatically replicated to secondary US region
# Read from any region

Azure Geo-Replication

Locally Redundant (LRS): Single region
Zone-Redundant (ZRS): Multiple zones
Geo-Redundant (GRS): Primary + Secondary region
Read-Access GRS (RA-GRS): Read from secondary

Performance & Security

S3 Transfer Acceleration

// Use edge locations for faster uploads
const s3 = new AWS.S3({
  endpoint: 'https://bucket.s3-accelerate.amazonaws.com'
});

// Upload via CloudFront edge

Encryption Strategies

// S3 Server-Side Encryption (SSE)
s3.putObject({
  Bucket: 'my-bucket',
  Key: 'sensitive-data.txt',
  Body: data,
  ServerSideEncryption: 'AES256' // or 'aws:kms'
}).promise();

// GCP Customer-Managed Encryption (CMEK)
// Azure Encryption with Customer Key (CMK)

Monitoring & Alerts

// S3 Storage Metrics
const cloudwatch = new AWS.CloudWatch();

await cloudwatch.putMetricAlarm({
  AlarmName: 'S3-High-Delete-Rate',
  MetricName: 'NumberOfObjects',
  Namespace: 'AWS/S3',
  Statistic: 'Sum',
  Period: 86400,
  EvaluationPeriods: 1,
  Threshold: 0.9, // Alert if <90% of baseline
  ComparisonOperator: 'LessThanThreshold'
}).promise();

Key Takeaways

• **Lifecycle policies save 40-60%** on storage costs

• **Replication essential** for disaster recovery

• **Encryption mandatory** for compliance

• **Multi-region buckets automatic** (GCP)

• **Archive for long-term**: 80-90% savings vs standard

• **Versioning protects** against accidental deletion

• **Monitoring detects anomalies** early

• **Encryption at-rest and in-transit** required for PII