Design a Notification Service (Email, SMS, Push)

Clarifying questions I'd ask first

• Which channels: email, SMS, push, web push, in-app?
• Self-serve template management for marketing or always code-deployed?
• User preferences — per channel, per category, both?
• Scheduled or delayed notifications?
• Bulk marketing campaigns vs purely transactional?
• Compliance requirements — GDPR, TCPA, CAN-SPAM?

Assumptions for this design

Three channels: email, SMS, mobile push. 100 million users, ~10 million notifications a day with 70/20/10 mix, plus occasional 50-million-recipient campaigns. Sub-5-second p99 dispatch for transactional. Idempotency keys from every producer. Quiet hours respected for non-transactional categories.

Quick capacity check

• Steady state: ~115 notifications/sec, ~2,000/sec at peak
• Per-channel split at peak: ~1,400 email, ~400 push, ~200 SMS
• Provider rate limits: SendGrid ~10K/sec, Twilio ~500/sec, FCM ~36K/min
• SMS is the cost driver and the bottleneck — design must throttle per-channel independently

High-level shape

• Producers POST to a single Notification API or publish to Kafka
• API does idempotency dedup against notifications.idempotency_key and writes to a Kafka topic
• Notification Orchestrator consumes, resolves preferences and templates, fans out to per-channel topics
• Per-channel worker pools (Email, SMS, Push) consume their topic, acquire rate-limit tokens from Redis, call providers with idempotency headers
• Webhook receivers ingest provider callbacks (delivered, bounced) and update the dispatch status
• DLQ catches poison messages after retry exhaustion

Why per-channel queues, not one big queue

If SMS provider goes down, email shouldn't be blocked. Per-channel topics give independent scaling, independent rate limits, independent failure domains. Same reasoning for per-channel worker pools.

The Orchestrator pipeline — step by step

• Idempotency dedup on idempotency_key
• Look up user preferences for (user_id, category) — list of enabled channels
• Intersect requested channels with enabled channels
• Quiet-hours check for non-transactional categories — schedule for later if needed
• Resolve template (template_id, version, channel, locale), render with template_data
• Persist channel_dispatches rows with status=pending
• Publish to per-channel Kafka topic

Transactional vs marketing

• Transactional (OTP, order shipped) bypasses quiet hours and ignores marketing opt-out — user-expected and often legally required
• Marketing respects both, plus separate rate budgets
• Both share infrastructure but use separate topics with priority consumer ratios so a campaign burst doesn't starve transactional

Worker loop — retries and rate limits

• Acquire token from Redis per-provider rate limiter (token bucket)
• Call provider with idempotency header (so retries don't double-send if prior response was lost)
• 2xx: mark sent, store provider_message_id
• 4xx (bad recipient): mark failed, no retry
• 5xx or timeout: schedule retry with exponential backoff (1s, 4s, 16s, 1m, 5m, 30m)
• After 6 retries: DLQ
• Provider failover: circuit breaker on sustained 5xx switches to alternate provider via config flip

Data model

• notifications table: the request, with idempotency_key UNIQUE
• channel_dispatches table: one row per channel per notification, tracks status and attempts
• user_preferences table: per-user, per-category, per-channel enable/disable plus quiet hours
• Templates in a separate Template Service with versioning — marketing changes templates without code deploy

LLD — clean abstractions

• ChannelDispatcher interface per channel — each has its own policy
• ProviderClient interface inside each dispatcher — SendGrid and Mailgun are interchangeable
• RateLimiter and RetryScheduler as separate collaborators — testable in isolation
• IdempotencyChecker at the controller boundary — duplicates never enter the pipeline

Failure handling

• Provider outage: circuit breaker → failover to alternate provider
• DB down briefly: Kafka is durable record; orchestrator replays on recovery
• Worker crash mid-send: another worker picks up; provider idempotency header prevents double-send
• Webhook loss: daily reconciliation job queries provider APIs for stuck-in-sent dispatches
• Bulk campaign burst: producer-side rate limiting plus per-channel limits prevent SendGrid quota exhaustion

Production extras

• Provider credentials in secret manager, rotated
• Webhook signature verification (HMAC) on every callback
• PII scrubbed from logs; long-term storage encrypted
• Compliance audit log: every send with category, opt-in proof, IP
• Distributed tracing across producer → orchestrator → provider

At 10× scale

• Sharded notification API by user_id hash
• Per-region dispatch infrastructure with regional provider endpoints
• Pre-rendered template cache for high-volume templates
• Batched DB writes for marketing campaigns

Recommendation

A single notification API with idempotency keys, Kafka per-channel topics, a Notification Orchestrator that centralizes preferences and templates, per-channel worker pools with provider abstraction and Redis-backed rate limiters, exponential-backoff retries with DLQ, and a webhook receiver that loops delivery status back into the DB — essentially the architecture Twilio's own customers and most large product companies run.