Design a File Storage Service (Dropbox / Google Drive)

Clarifying questions I'd ask first

• Folder hierarchy depth, sharing model (read-only links, per-user, group folders)?
• Versioning — keep how many versions, time-bounded?
• Conflict resolution — last-write-wins or fork to conflict copy?
• Selective sync, trash, public links?
• End-to-end encryption?

Assumptions for this design

500M users, 100M DAU, average file 1 MB with long tail to GBs, 100 GB free quota, 4 MB fixed chunks. 2 PB/day growth. Sub-2-second sync visibility for small files. Pre-signed S3 URLs for direct transfer. Conflict forking. No E2EE in v1.

Quick capacity check

• 5 PB/day uploads, ~30% net new after dedup = ~175 GB/sec to S3
• 25 EB raw before dedup, ~17 EB after
• Metadata: 500B files, ~250 TB across sharded PostgreSQL (256 shards at ~1 TB each)
• Sync fan-out: ~2M push notifications/sec
• Critical: bytes go directly client ↔ S3, never through our app tier

The central decisions

1. 1
   Chunk files into 4 MB pieces — enables cross-user dedup, diff-sync, resumable uploads, parallel transfer
2. 2
   Address chunks by SHA-256 of bytes — same content always hashes to same key, so dedup is automatic
3. 3
   Separate metadata (sharded PostgreSQL) from blobs (S3)
4. 4
   Pre-signed S3 URLs for direct client transfer — bytes never traverse app servers

Why 4 MB chunks

• Smaller (64KB): too many chunks per file, huge metadata overhead
• Larger (64MB): hurts partial-edit dedup
• 4 MB is the industry sweet spot (Dropbox uses ~4 MB)
• Variable-size content-defined chunking (Rabin) is better for edited text but more complex; defer to v2

Upload flow — step by step

1. 1
   Client computes SHA-256 of each 4 MB chunk locally
2. 2
   Client POSTs the hash list to Upload Service
3. 3
   Server checks chunks table; returns pre-signed PUT URLs only for missing chunks
4. 4
   Client uploads missing chunks directly to S3 in parallel
5. 5
   Client commits; server creates new file_version row, increments chunk ref counts, publishes change event

Download flow

• Server returns ordered chunk list with pre-signed GET URLs
• Client downloads chunks in parallel from S3
• App tier handles only metadata; bandwidth scales via S3

Sync flow — long-poll with per-user journal

• File change → Kafka → SyncService writes to sync_events for affected users (owner + collaborators)
• Client maintains a cursor (last-seen seq); calls GET /sync/changes?cursor=N&timeout=30s
• Server returns immediately if events exist; otherwise holds connection up to 30s, signals on new event
• Long-poll over WebSocket because mobile clients reconnect frequently and stateless server is simpler
• Push notification fires for offline devices

Conflict resolution

• Two devices edit the same file offline; both upload
• Optimistic version check: second commit detects parent version mismatch, creates a fork as (conflict copy from device X).docx
• User manually merges — same as Dropbox model
• Google Docs-style operational transformation is a separate beast; out of scope here

Data model

• files (sharded by owner_id) — file tree, current_version pointer
• file_versions — ordered chunk_hashes per version
• chunks — the dedup table; chunk_hash PK, s3_key, ref_count
• sync_events — per-user append-only journal
• shares and public_links — access control

LLD — clean abstractions

• ChunkRegistry interface — PostgreSQL impl, could swap to DynamoDB for higher hash-lookup scale
• PresignedUrlFactory interface — S3 impl, could swap to GCS or Azure
• UploadSession server-side state tracks claimed chunks
• SyncService with a waiter registry for long-poll efficiency
• Garbage collector with 7-day grace runs separately

Storage choice rationale

• S3 for chunks: infinite capacity, 11-nines durability, lifecycle policies for cold tier
• PostgreSQL for metadata: ACID for tree mutations, JOINs for shares/versions, sharded by owner_id
• Redis for upload sessions, dedup bloom filter, share permission cache
• Rejected files-in-PostgreSQL — doesn't scale past low-hundred-MB; backups impossible

Reference-counted chunks

• Each commit increments ref_count for chunks; each delete decrements
• Chunks with ref_count = 0 for > 7 days are GC'd
• 7-day grace prevents race with in-flight uploads

Failure handling

• S3 down: cross-region failover; metadata still works
• DB shard down: read replicas serve stale browse; multi-AZ failover ~30s
• Upload session expires: client retries from scratch (chunks already uploaded skip dedup)
• Conflict at commit: 409 Conflict; client forks locally
• Mobile offline: client queues changes, replays on reconnect

Production extras

• TLS everywhere; OAuth/OIDC for clients
• Pre-signed URLs short-lived (15 min) and bound to specific S3 key
• S3 server-side encryption (KMS); per-customer keys for enterprise
• Async virus scan with quarantine on flag
• Audit log for enterprise compliance
• Lifecycle policies move cold files to Standard-IA / Glacier

At 10× scale

• Geo-distributed metadata with CRDTs for shared folders
• Multi-region S3 with nearest-region client routing
• Variable-size content-defined chunking for better text/code dedup
• Edge POP chunk caches for popular downloads
• WebSocket sync for active sessions

Recommendation

4 MB content-addressed chunks deduped at the upload boundary, sharded PostgreSQL metadata + S3 blobs with pre-signed URLs for direct client transfer, per-user long-poll sync journal, reference-counted chunks with grace-period GC, and CDN-cached public shares — the same shape Dropbox, OneDrive, and Box run in production.