Scenario: Cutting a $500K/month AWS Bill 40% in 90 Days Without Breaking SLOs

Q: Scenario: Cutting a $500K/month AWS Bill 40% in 90 Days Without Breaking SLOs

**First 30 days: measure and kill obvious waste.** Set up tagging enforcement, Cost and Usage Report → Athena dashboards, Kubecost for K8s attribution, and per-team cost visibility. Kill the free wins: unattached EBS, old snapshots, idle RDS replicas, DEBUG log retention, forgotten dev environments, missing VPC endpoints. Expected: **~10–15% reduction**, zero SLO risk. **Days 30–60: compute rightsizing and commitment strategy.** Rightsize pods/tasks using p95 utilization, set Karpenter with Spot + on-demand mix, buy **Compute Savings Plans** covering 70% of steady baseline, move log retention from 90 to 14 days with S3 archive, fix log-level discipline. Expected: **another ~15–20%**, careful SLO monitoring. **Days 60–90: architectural shifts.** Migrate steady high-RPS workloads off Lambda to ECS Fargate; introduce scale-to-zero on non-prod + off-hours shutdowns; S3 Intelligent-Tiering + lifecycle; consolidate observability vendors or tune Datadog metrics. Expected: **remaining 5–10%**. Throughout: **cost dashboards per team**, **weekly waste reviews**, **SLO monitors per service**, and **rollback plan** if anything regresses. Communicate clearly: this isn't a one-shot cut, it's installing a **FinOps practice** that sustains. Biggest risks: Spot interruption hitting stateful workloads, aggressive log-retention hiding an incident investigation, commitment overshoot if architecture shifts.

Intermediate·5 min read·

AwsStripeAirbnbPinterestNetflix

Fast recall

Quick revision

Core answer

First 30 days: measure and kill obvious waste. Set up tagging enforcement, Cost and Usage Report → Athena dashboards, Kubecost for K8s attribution, and per-team cost visibility. Kill the free wins: unattached EBS, old snapshots, idle RDS replicas, DEBUG log retention, forgotten dev environments, missing VPC endpoints. Expected: ~10–15% reduction, zero SLO risk. Days 30–60: compute rightsizing and commitment strategy. Rightsize pods/tasks using p95 utilization, set Karpenter with Spot + on-demand mix, buy Compute Savings Plans covering 70% of steady baseline, move log retention from 90 to 14 days with S3 archive, fix log-level discipline. Expected: another ~15–20%, careful SLO monitoring. Days 60–90: architectural shifts. Migrate steady high-RPS workloads off Lambda to ECS Fargate; introduce scale-to-zero on non-prod + off-hours shutdowns; S3 Intelligent-Tiering + lifecycle; consolidate observability vendors or tune Datadog metrics. Expected: remaining 5–10%. Throughout: cost dashboards per team, weekly waste reviews, SLO monitors per service, and rollback plan if anything regresses. Communicate clearly: this isn't a one-shot cut, it's installing a FinOps practice that sustains. Biggest risks: Spot interruption hitting stateful workloads, aggressive log-retention hiding an incident investigation, commitment overshoot if architecture shifts.

Interview Answer5 min

I'd run a 90-day FinOps-driven plan in three phases with SLO guardrails throughout.

Days 0–30: visibility and quick wins (~15%)

Enable Cost and Usage Reports → Athena → QuickSight dashboards per team
Install Kubecost on EKS for per-namespace attribution
Enforce tagging: team, service, env, cost-center, owner
AWS Config flags untagged; Terraform pre-commit blocks new untagged
Publish cost leaderboard — visibility alone drives 5%

Zero-risk quick wins

Unattached EBS volumes and old snapshots
Orphaned ELBs, Elastic IPs, old AMIs
Idle RDS instances and replicas
Forgotten 60-day dev environments
CloudWatch Logs: cut 90-day retention to 14 days + S3 archive
VPC endpoints for S3, DynamoDB, Secrets Manager — avoid NAT egress
NAT Gateway consolidation
Orphaned ECS tasks, old Lambda versions
Install SLO monitors before cutting anything

Days 30–60: rightsize and commit (~15–20%)

Compute rightsizing

Use Kubecost / VPA 14-day p95 recommendations
Most pods 2–4x overprovisioned
Roll per namespace with canary validation
Java: -XX:MaxRAMPercentage=75, UseContainerSupport, AppCDS for startup
ECS tasks via AWS Compute Optimizer
Lambda via Lambda Power Tuning — sometimes higher memory is cheaper

Node strategy

Replace Cluster Autoscaler with Karpenter
30% on-demand baseline + 70% Spot for stateless
PDB + Node Termination Handler
Stateful on on-demand
30–50% savings on Spot-eligible

Commitment

Compute Savings Plans covering 70% of steady baseline
1-year, no upfront — flexible
Reserved DB for steady RDS
Don't overcommit — architecture shifts create stranded cost

Observability tuning

CloudWatch log retention 90 → 14 days + S3 archive — often 10–30K/month
Datadog metric cardinality audit — userId/requestId tags explode cost
APM sampling 100% → 5% + 100% on errors
Custom metrics audit and prune
Log levels: DEBUG only non-prod

Data transfer

VPC Flow Logs → Athena to find top NAT egress
Missing VPC endpoints
Co-locate chatty services in same AZ (topology spread)
CloudFront for hot S3 content

Days 60–90: architectural + governance (~5–10%)

Lambda → ECS Fargate for steady high-RPS

Any Lambda with sustained >10 req/sec often cheaper on Fargate
40–60% savings on migrated workloads
Keep true event-driven on Lambda

Storage lifecycle

S3 Intelligent-Tiering for mixed access
Lifecycle rules: Standard → IA → Glacier → Deep Archive
Delete incomplete multipart uploads
Compress logs and analytics

Database

Aurora Serverless v2 for variable load
Right-size RDS
Aurora I/O-Optimized for I/O-heavy
Stop non-prod nights/weekends
Archive cold data to S3
Read replicas to SLO minimum

Off-hours shutdown

Non-prod scheduled shutdown — 50–70% savings
Auto-start on PR or manual

Governance install

OPA / Gatekeeper / Kyverno on K8s
AWS Service Control Policies
Terraform Sentinel on IaC
Per-team budgets with 50/80/100% alerts
AWS Cost Anomaly Detection → Slack
Weekly waste review — 30 min, close-loop fixes
FinOps champion per product team

SLO safeguards throughout

SLO dashboards per service, live in every meeting
Rollback policy on any regression
Canary rightsizing before namespace rollout
Keep on-demand baseline; don't all-in on Spot

Trade-offs and risks

Spot interruption: stateless only, mixed instances, PDB, graceful shutdown
Savings Plan commitment: 70% baseline, re-evaluate quarterly; document forecast
Aggressive log cuts: keep ERROR full, sample INFO/DEBUG, don't cut during incidents
Rightsizing: canary, monitor GC/p99/memory, roll per namespace
Cost vs reliability: SLO trumps cost always
Engineering time ROI: big levers first, automate rightsizer

Failure modes to avoid

Random cuts without measurement
No tagging first
Cutting Datadog before understanding queries
Overcommit Savings Plans
Spot on stateful
Scale-to-zero on user-facing prod
One-engineer crusade
Cutting dev envs engineers need
No governance after cuts
Overselling to leadership then under-delivering

Communication

To leadership

Phase plan: 15% → 30% → 40% with SLO guardrails
Risk register + mitigation
Weekly dashboard: savings vs target, SLO status
Non-negotiables: no SLO breach, no commitment overbuy
Ask for support: SCP policy, tagging mandate, accountability
This is a practice install, not a one-shot cut

To engineers

Engineering metric, not finance witch-hunt
No shaming; waste is normal, finding is good
Training on costs, tagging, dashboards
Celebrate wins publicly
Empower with Kubecost, right-sizer tools

Bottom line

40% in 90 days is achievable with disciplined phasing
The win isn't just the cut — it's the FinOps practice that sustains
Governance is the only reason bill doesn't regrow next quarter
Product teams own their usage; platform centralizes rates and policy
SLOs protected, celebrate wins, measure weekly

Interview-ready guide

Interview answer