Design a URL Shortener (bit.ly)

Clarifying questions I'd ask first

• Custom aliases — do users get to pick their own short code?
• Expiration — do links expire?
• What level of analytics — just clicks or geo/referrer/device?
• Authentication — anonymous shortening allowed?
• How long should the short code be — 7 characters of base62 give us about three and a half trillion codes

Assumptions for this design

I'll assume 100 million new URLs per month, 10 billion redirects per month — so a 100-to-1 read-to-write ratio — a 5-year horizon, 7-character short codes, and a sub-10-millisecond p99 on the redirect.

Quick capacity check

• Writes: about 40 per second average, 400 peak
• Reads: about 4,000 per second average, 40,000 peak
• Storage: 6 billion rows over 5 years at ~500 bytes each is roughly 3 terabytes — fits a sharded relational cluster easily
• The system is read-dominated by 100×, so the architecture biases toward the read path

High-level shape

• Write path: client → load balancer → Shorten Service → ID Generator (Snowflake) → primary DB → warm Redis
• Read path: client → CDN edge cache → (on miss) Redirect Service → Redis → (on miss) read replica → 302 response
• Click events: Redirect Service publishes to Kafka fire-and-forget; downstream consumer writes to ClickHouse for analytics
• The redirect path never blocks on the database in steady state because Redis absorbs the traffic

Short-code generation — base62 of a Snowflake ID

• Snowflake gives us a 64-bit ID combining timestamp, machine ID, and a sequence number — unique without coordination
• Base62-encoding gives URL-safe, human-typeable codes
• I rejected hash truncation because the same URL would always produce the same code — breaks analytics and creates collision attacks
• I rejected random codes because at billions of rows the birthday-paradox collision rate becomes unacceptable

Data model — two stores, two responsibilities

• OLTP (PostgreSQL or MySQL): short_code, long_url, owner, expiry, click counter — sharded by short_code hash
• Analytics store (ClickHouse): one row per click event — written from Kafka consumer, never from the redirect path
• Custom aliases use INSERT ... ON CONFLICT DO NOTHING so two simultaneous reservations don't both succeed

LLD — keep responsibilities clean

• IdGenerator interface (Snowflake in prod, deterministic counter in tests)
• UrlRepository interface (JDBC in prod, can swap for DynamoDB)
• UrlCache for Redis access
• ClickEventPublisher for fire-and-forget Kafka writes
• ShortenService and RedirectService orchestrate — controllers stay thin

Caching — layered, with hot-key protection

• Cache-aside Redis with 24-hour TTL
• Local Caffeine cache (1–5 second TTL) on each Redirect Service instance absorbs viral hot keys without hitting Redis
• CDN with Cache-Control: public, max-age=86400 on the 302 response means most clicks never reach origin
• Negative caching for 404s (60 seconds) prevents typos on viral non-existent codes from hammering the DB

Failure handling

• Redis down: fall back to read replica; latency increases but service stays up
• Primary DB down: reads continue from replicas; writes fail with 503 + Retry-After
• Kafka down: click events buffer locally and drop on overflow — redirect never fails because of analytics
• Cold cache after Redis restart: pre-warm top 100K codes from a daily snapshot

Production extras

• URL safety scanning at write time against Google Safe Browsing
• Rate limit by IP on the shorten endpoint to prevent spam
• HTTPS-only redirect domain to prevent phishing abuse
• Owner-only access to analytics and delete

At 10× scale

• Multi-region active-active with regional primary DBs and Kafka cross-region replication
• Embedded RocksDB on each Redirect Service for the hottest codes — eliminates the Redis network hop
• CDN edge compute (Cloudflare Workers) to serve cached 302s without origin hit

Recommendation

Snowflake-derived base62 short codes, sharded PostgreSQL primary, layered Redis + Caffeine + CDN cache, Kafka-decoupled click analytics into ClickHouse, and HTTPS-only redirects with safety scanning — essentially the architecture Bitly and TinyURL run today.