@KafkaListener — Partitions, Consumer Groups, and Concurrency
Fast recall
Quick revision
Core answer
Understanding of consumer group partition assignment and the concurrency-partition relationship
Setting concurrency > number of partitions — extra threads are idle
Explain rebalancing when concurrency changes or instances scale, and the difference between group IDs for competing consumers vs fan-out
Interview-ready guide
Interview answer
@KafkaListener creates a ConcurrentMessageListenerContainer that manages consumer threads. These threads poll Kafka and invoke your listener method — completely outside the web request lifecycle.
`concurrency` controls consumer thread count. With 6 partitions and concurrency=3, each consumer handles 2 partitions. Consumers beyond the partition count sit idle.
`groupId` determines partition sharing. Same group = partitions distributed. Different groups = each gets all messages. This is how you have separate processing and analytics consumers on the same topic.
Partition assignment uses RangeAssignor by default. `CooperativeStickyAssignor` is better — incremental rebalances instead of stop-the-world.
Critical gotcha: listener methods run on the consumer thread. If your method blocks longer than max.poll.interval.ms (default 5 minutes), Kafka triggers a rebalance because it thinks the consumer is dead.
In practice, I always set CooperativeStickyAssignor, match concurrency to partition count, and keep listener methods fast.
@KafkaListener — Partitions, Consumer Groups, and Concurrency
Direct Answer
@KafkaListener creates a message-driven POJO consumer. Each listener is part of a consumer group; Kafka assigns partitions to consumers in the group. Concurrency = number of threads = max number of partitions that will be consumed in parallel.
Code Example
One topic with N partitions can be processed by up to N consumers in the same group, in parallel. @KafkaListener(concurrency = …) controls how many consumer threads each app instance runs.
spring:
kafka:
consumer:
bootstrap-servers: kafka:9092
group-id: orders-processor # same group = work splitting
auto-offset-reset: earliest
enable-auto-commit: false # commit manually after success
max-poll-records: 100
properties:
max.poll.interval.ms: 300000 # 5 min — must exceed worst-case processing
@Component
@RequiredArgsConstructor
public class OrderListener {
private final OrderProcessor processor;
/**
* topic has 12 partitions; concurrency = 6 → this instance runs 6 consumer threads,
* each owning 2 partitions. Run two instances and you get 12-way parallelism (one
* thread per partition — the maximum without idle threads).
*/
@KafkaListener(
topics = "orders",
groupId = "orders-processor",
concurrency = "6",
containerFactory = "kafkaListenerContainerFactory")
public void onOrder(OrderEvent event,
@Header(KafkaHeaders.RECEIVED_PARTITION) int partition,
@Header(KafkaHeaders.OFFSET) long offset,
Acknowledgment ack) {
log.info("partition={} offset={} order={}", partition, offset, event.orderId());
processor.process(event);
ack.acknowledge(); // commit AFTER success
}
}
@Configuration
class KafkaConsumerConfig {
@Bean
public ConcurrentKafkaListenerContainerFactory<String, OrderEvent>
kafkaListenerContainerFactory(ConsumerFactory<String, OrderEvent> cf) {
ConcurrentKafkaListenerContainerFactory<String, OrderEvent> f =
new ConcurrentKafkaListenerContainerFactory<>();
f.setConsumerFactory(cf);
f.getContainerProperties().setAckMode(ContainerProperties.AckMode.MANUAL);
f.setBatchListener(false);
return f;
}
}
- Group ID = work-splitting key: every consumer in the same
groupIdshares the partitions; consumers in different groups see all messages independently. - Concurrency never exceeds partition count — extra threads sit idle. Set partitions to your peak parallelism target (
concurrency × instances ≤ partitions). - Order is preserved per partition, not across the topic. Key by something stable (
orderId) so all events for one order land on one partition and process in order. max.poll.interval.msmust be larger than the longest acceptable processing time — exceed it and the broker considers the consumer dead and reassigns its partitions (rebalance storm).
Common Mistakes & Pitfalls
Setting concurrency > number of partitions — extra threads are idle
Check your understanding
Answer these without looking back at the article.
- What happens during a Kafka consumer group rebalance?
- How do you implement a fan-out pattern so multiple services receive the same events?
- What is the difference between poll-based and push-based consumption in Kafka?
Related interview prep categories
2Want a guided curriculum order?
These modules are also sequenced in our flagship backend interview track — open it when you prefer an ordered path instead of browsing by topic.