KafkaTemplate and ProducerFactory Configuration
Fast recall
Quick revision
Core answer
Practical knowledge of Spring Kafka producer setup
Not configuring enable.idempotence=true, leading to duplicate messages on retry
Mention acks=all with min.insync.replicas for durability guarantees, and the difference between send() fire-and-forget and ListenableFuture callback
Interview-ready guide
Interview answer
KafkaTemplate wraps the Kafka producer API with Spring abstractions. Auto-configured from spring.kafka.producer.* properties, you inject it and call send().
send() is asynchronous — returns a CompletableFuture. Messages batch in memory and send when the batch is full or linger.ms expires. Without calling .get(), failures are silent.
The critical configuration is `acks`. Default acks=1 means only the leader confirms — leader failure before replication loses the message. `acks=all` waits for all in-sync replicas, providing the strongest guarantee.
Idempotence (default since Kafka 3.0) prevents duplicates from retries using producer ID and sequence numbers.
The serializer choice (StringSerializer, JsonSerializer, Avro) is set at the ProducerFactory level.
In practice, I always set acks=all, verify enable.idempotence=true, and use CompletableFuture callbacks for error handling instead of fire-and-forget.
KafkaTemplate and ProducerFactory Configuration
Direct Answer
Spring Boot auto-configures KafkaTemplate when spring-Kafka is on the classpath. Configure producers via spring.kafka.producer.* properties. For advanced needs, define a ProducerFactory bean with custom serializers and idempotence settings.
Code Example
ProducerFactory builds the KafkaProducer instance with all configs; KafkaTemplate wraps it with Spring-friendly send APIs (sync, async, transactional, callbacks).
# application.yml — Spring Boot auto-configures one ProducerFactory + KafkaTemplate from these
spring:
kafka:
bootstrap-servers: kafka1:9092,kafka2:9092
producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.springframework.kafka.support.serializer.JsonSerializer
acks: all # wait for all in-sync replicas (durability)
retries: 2147483647 # retry forever; combine with delivery.timeout.ms
properties:
enable.idempotence: true # exactly-once on the broker side
max.in.flight.requests.per.connection: 5
delivery.timeout.ms: 120000
linger.ms: 10 # batch up to 10ms for throughput
batch.size: 32768
compression.type: lz4
@Service
@RequiredArgsConstructor
public class OrderEventPublisher {
private final KafkaTemplate<String, OrderEvent> template;
// Async with callback (recommended)
public void publish(OrderEvent event) {
template.send("orders", event.orderId().toString(), event)
.whenComplete((result, ex) -> {
if (ex != null) log.error("Send failed for {}", event.orderId(), ex);
else log.info("Sent to {} offset {}",
result.getRecordMetadata().topic(),
result.getRecordMetadata().offset());
});
}
// Sync (only when you must block — e.g. on app shutdown)
public void publishSync(OrderEvent event) throws ExecutionException, InterruptedException {
template.send("orders", event.orderId().toString(), event)
.get(5, TimeUnit.SECONDS);
}
}
// Custom factory when you need a second producer (e.g. transactional)
@Configuration
class TxProducerConfig {
@Bean
public ProducerFactory<String, byte[]> txProducerFactory(KafkaProperties props) {
Map<String, Object> cfg = new HashMap<>(props.buildProducerProperties());
cfg.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "tx-orders-" + UUID.randomUUID());
cfg.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, true);
DefaultKafkaProducerFactory<String, byte[]> f = new DefaultKafkaProducerFactory<>(cfg);
f.setTransactionIdPrefix("tx-orders-");
return f;
}
@Bean public KafkaTemplate<String, byte[]> txTemplate(ProducerFactory<String, byte[]> f) {
return new KafkaTemplate<>(f);
}
}
- One
ProducerFactorybuilds many producers; Spring shares a single thread-safeKafkaProduceracross allKafkaTemplate.send()calls (efficient — do not create one per request). acks=all+enable.idempotence=true+ finitedelivery.timeout.msis the durability sweet spot — no duplicates, no silent loss.linger.msandbatch.sizetrade latency for throughput. Defaults are tuned for low latency; bump to 10-20 ms / 32 KB for high-throughput pipelines.- Use two factories if you mix transactional and non-transactional sends — a transactional producer cannot send non-tx records.
Common Mistakes & Pitfalls
Not configuring enable.idempotence=true, leading to duplicate messages on retry
Check your understanding
Answer these without looking back at the article.
- What does enable.idempotence do and why is it important?
- What is the difference between acks=0, acks=1, and acks=all?
- How do you implement transactional Kafka producers in Spring?
Related interview prep categories
2Want a guided curriculum order?
These modules are also sequenced in our flagship backend interview track — open it when you prefer an ordered path instead of browsing by topic.