SQS Dead-Letter Queues: Poison Messages, Retries, Redrive

SQS guarantees at-least-once delivery. When a consumer fails — exception, timeout, crash — the message returns to the queue after the visibility timeout. Without a DLQ, a message that fails for any reason retries forever, consuming processing capacity and blocking progress. A DLQ is the standard safety valve.

The DLQ pattern

• Attach a DLQ (another SQS queue) to your main queue with maxReceiveCount, typically 3 to 5
• After that many failed processing attempts, SQS automatically moves the message to the DLQ
• Main queue keeps processing healthy messages; DLQ holds the bad ones for investigation
• This must be configured from day one, not as an afterthought

The parameters that matter

• maxReceiveCount 3 to 5: enough to handle transient failures (network blips, downstream retries) but not so high that bad messages waste hours in retry loops
• Visibility timeout greater than max processing time: if processing takes 45 seconds but visibility timeout is 30, the message reappears while still being processed, causing duplicate work; set timeout greater than P99 processing + buffer
• DLQ retention 14 days (max): gives time to investigate before losing messages to retention expiration

The end-to-end flow

• Producer writes to main queue
• Consumer receives, processes, deletes on success
• On failure, consumer throws; message reappears after visibility timeout
• After maxReceiveCount failures, SQS moves it to DLQ
• CloudWatch alarm on DLQ depth > 0 fires; ops investigates
• Root cause identified: bad payload? downstream outage? schema mismatch?
• Fix the root cause, then use SQS redrive to send DLQ messages back to main queue

The Lambda partial batch failure gotcha

• When Lambda is the consumer, it receives a batch of messages
• If you return success but one message failed, ALL messages are marked processed — you lose the failed one
• Use ReportBatchItemFailures: return only the failed message IDs; SQS retains those and deletes the successful ones

Idempotency is the paired concern

• SQS is at-least-once; duplicates are inevitable, especially during retries and visibility timeout edge cases
• Consumers must be idempotent — use message ID or idempotency key to deduplicate at the storage layer
• Without idempotency, a retried payment or order creation becomes a double-charge

Distinguish poison from transient

• Poison: malformed payload, schema violation, missing reference data — will never succeed; log and skip immediately
• Transient: downstream API 500, DB timeout — will succeed on retry; exponential backoff via visibility timeout
• Ideally code catches the distinction and handles them differently

The monitoring discipline

• DLQ depth > 0 alarm is the critical one; silent accumulation is worse than no DLQ
• Also monitor ApproximateAgeOfOldestMessage on main queue — if oldest is 1 hour old, processing is backed up
• Gap between NumberOfMessagesReceived and NumberOfMessagesDeleted indicates failure rate

Operational discipline for redrive

• Don't dump 100k DLQ messages back to main queue at once; overwhelms consumers
• Use max-number-of-messages-per-second on redrive task to control rate
• Investigate a sample first; confirm fix works; then redrive the rest

The strategic answer

• Every queue gets a DLQ from day one
• Every DLQ gets an alarm on depth > 0
• Every consumer is idempotent
• Every runbook documents: what does this DLQ mean, who investigates, what's the redrive process
• Failure handling is designed into the system, not bolted on after the first incident