@RetryableTopic — Non-Blocking Retry in Spring Kafka
Fast recall
Quick revision
Core answer
Understanding of why blocking retry (Thread.sleep) is problematic in Kafka consumers
Using Thread.sleep() in a Kafka listener during retry — this stalls the entire partition
Explain the ordering implications: non-blocking retry means messages on the same key can be processed out of order during retries
Interview-ready guide
Interview answer
Blocking retry holds the consumer thread while retrying. If a record fails 10 times with 3-second backoff, that partition is blocked for 30 seconds. No other messages are processed.
@RetryableTopic solves this with non-blocking retry. Failed messages move to separate retry topics (orders-retry-0, orders-retry-1) with increasing delays. The original consumer immediately processes the next message.
Configure with @RetryableTopic(attempts = 4, backoff = @Backoff(delay = 1000, multiplier = 2)). Spring Kafka creates retry topics and DLT automatically.
After all retry topics are exhausted, the message lands in the DLT. Use @DltHandler for custom DLT processing — alerting or storing for manual review.
Blocking retry is still appropriate for transient failures — network blips that resolve in milliseconds. Non-blocking retry is for failures that might take minutes to resolve.
In practice, I use blocking retry with 2 attempts for transient errors and @RetryableTopic with exponential backoff for downstream outages.
@RetryableTopic — Non-Blocking Retry in Spring Kafka
Direct Answer
@RetryableTopic implements non-blocking retry by routing failed messages to intermediate retry topics (e.g., orders-retry-0, orders-retry-1) with increasing delays, finally to a DLT. This avoids blocking the main topic partition during retry waits.
Code Example
1) Listener with @RetryableTopic — Spring auto-creates retry topics + DLT:
@Component
public class OrderEventListener {
@RetryableTopic(
attempts = "4",
backoff = @Backoff(
delay = 1_000, // 1s initial delay
multiplier = 2.0, // 1s, 2s, 4s
maxDelay = 30_000 // cap at 30s
),
autoCreateTopics = "true",
topicSuffixingStrategy = TopicSuffixingStrategy.SUFFIX_WITH_INDEX_VALUE,
dltStrategy = DltStrategy.FAIL_ON_ERROR,
// Don't retry validation errors — they will never succeed.
exclude = { ValidationException.class, IllegalArgumentException.class },
// Only retry these and their subclasses.
include = { TransientDownstreamException.class,
org.springframework.dao.TransientDataAccessException.class }
)
@KafkaListener(topics = "orders", groupId = "order-svc")
public void handle(
@Payload OrderEvent event,
@Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
@Header(name = RetryTopicHeaders.DEFAULT_HEADER_ATTEMPTS,
required = false) Integer attempt) {
log.info("Processing order {} from {} (attempt {})",
event.orderId(), topic, attempt);
orderService.process(event); // throws TransientDownstreamException to trigger retry
}
// Custom DLT handler — same class, runs when message exhausts retries.
@DltHandler
public void onDlt(
@Payload OrderEvent event,
@Header(KafkaHeaders.RECEIVED_TOPIC) String topic,
@Header(KafkaHeaders.EXCEPTION_MESSAGE) String exceptionMessage,
@Header(KafkaHeaders.EXCEPTION_FQCN) String exceptionClass) {
log.error("DLT: order {} from {} class={} msg={}",
event.orderId(), topic, exceptionClass, exceptionMessage);
deadLetterStore.save(event, exceptionClass, exceptionMessage);
alerting.notify("OrderEvent landed in DLT: " + event.orderId());
}
}
Generated topics for topics = "orders" with attempts = 4:
orders— main topicorders-retry-0— first retry (1s delay)orders-retry-1— second retry (2s delay)orders-retry-2— third retry (4s delay)orders-dlt— final dead-letter destination
2) Configure retention for retry/DLT topics in application.yml:
spring:
kafka:
consumer:
enable-auto-commit: false
listener:
ack-mode: record
retry:
topic:
attempts: 4
delay: 1s
multiplier: 2.0
max-delay: 30s
3) Customize topic creation if you don't want auto-create:
@Bean
public RetryTopicConfiguration retryTopicConfig(KafkaTemplate<String, Object> template) {
return RetryTopicConfigurationBuilder
.newInstance()
.maxAttempts(4)
.exponentialBackoff(1_000, 2.0, 30_000)
.retryTopicSuffix("-retry")
.dltSuffix("-dlt")
.dltHandlerMethod("orderEventListener", "onDlt")
.replicationFactor((short) 3)
.create(template);
}
Critical points:
include/excludedecide which exceptions are retryable — never retry validation or business-rule failures (they always fail).- The retry topics must have enough partitions (typically same count as the main topic) and a sensible retention (default 7 days is fine for most workloads).
- Headers like
kafka_dlt-original-topic,kafka_dlt-original-partition,kafka_dlt-original-offset, and the exception class/message are set automatically — invaluable for replay tooling. - Replay from DLT back to the main topic with a small admin tool that reads from
orders-dltand re-produces toordersonce you've fixed the underlying issue.
Common Mistakes & Pitfalls
Using Thread.sleep() in a Kafka listener during retry — this stalls the entire partition
Check your understanding
Answer these without looking back at the article.
- When does @RetryableTopic break message ordering?
- How do you configure retention for retry and DLT topics?
- How do you replay messages from a DLT back to the main topic?
Related interview prep categories
2Want a guided curriculum order?
These modules are also sequenced in our flagship backend interview track — open it when you prefer an ordered path instead of browsing by topic.