Kafka Message Headers — Metadata and Context Propagation
Fast recall
Quick revision
Core answer
Awareness of Kafka headers for cross-cutting concerns without polluting message payloads
Putting metadata like correlationId and eventVersion inside the message payload instead of headers
Explain that Spring Kafka's type header (__TypeId__) enables polymorphic deserialization and how to control it
Interview-ready guide
Interview answer
Kafka headers carry metadata separate from the payload — trace IDs, correlation IDs, content types, source service names. They avoid polluting the message body with infrastructure concerns.
In Spring Kafka, use MessageBuilder.withPayload(event).setHeader("X-Trace-Id", traceId).build() to attach headers. On the consumer side, @Header("X-Trace-Id") String traceId extracts them.
Micrometer Tracing automatically propagates trace context via headers — traceparent and tracestate added on produce, extracted on consume. End-to-end tracing across HTTP and Kafka without manual management.
Spring Kafka adds DLT headers automatically — original topic, partition, offset, exception class, and message. Essential for debugging failed messages.
Headers count toward the message size limit (max.message.bytes, default 1MB). Keep them small.
In practice, I use headers for trace IDs, event type routing, and correlation IDs. The payload stays clean with only business data.
Kafka Message Headers — Metadata and Context Propagation
Direct Answer
Kafka message headers are key-value byte pairs attached to each message. Spring Kafka uses them internally (type info, trace IDs) and exposes them via @Header in listeners. Use them for correlation IDs, event versioning, and distributed tracing context.
Code Example
Headers are key-value byte pairs attached to a record alongside the body. Use them for cross-cutting metadata — tracing IDs, schema version, source system, idempotency key — that consumers need without unmarshalling the payload.
// ---- Producer side: add headers ----
@Service
@RequiredArgsConstructor
public class TracedPublisher {
private final KafkaTemplate<String, OrderEvent> template;
private final Tracer tracer;
public void publish(OrderEvent ev) {
Span span = tracer.currentSpan();
ProducerRecord<String, OrderEvent> rec = new ProducerRecord<>(
"orders", null, ev.orderId().toString(), ev);
rec.headers()
.add("traceparent", span.context().traceparent().getBytes()) // W3C trace context
.add("eventType", "OrderPlaced".getBytes())
.add("schemaVersion", "v3".getBytes())
.add("sourceSystem", "checkout-service".getBytes())
.add("idempotencyKey", ev.idempotencyKey().getBytes());
template.send(rec);
}
}
// ---- Consumer side: read headers ----
@KafkaListener(topics = "orders")
public void onOrder(@Payload OrderEvent ev,
@Header(name = "eventType") String type,
@Header(name = "schemaVersion") String version,
@Header(name = "traceparent") String trace,
@Header(name = "idempotencyKey", required = false) String idempKey,
ConsumerRecord<String, OrderEvent> raw) {
// Continue distributed trace
MDC.put("traceId", parseTraceId(trace));
// Route by event type without parsing the body
switch (type) {
case "OrderPlaced" -> handlePlaced(ev);
case "OrderCancelled"-> handleCancelled(ev);
default -> log.warn("Unknown eventType={}", type);
}
// Inspect ALL headers when needed
raw.headers().forEach(h ->
log.debug("header {}={}", h.key(), new String(h.value())));
}
// Spring Cloud Stream / Sleuth automatically propagate traceparent, but you can do it manually:
@Component
class HeaderTracingInterceptor implements ProducerInterceptor<Object, Object> {
@Override public ProducerRecord<Object, Object> onSend(ProducerRecord<Object, Object> r) {
Optional.ofNullable(MDC.get("traceId"))
.ifPresent(t -> r.headers().add("traceparent", t.getBytes()));
return r;
}
@Override public void onAcknowledgement(RecordMetadata m, Exception e) {}
@Override public void close() {}
@Override public void configure(Map<String, ?> configs) {}
}
- Always include:
traceparent(W3C),eventType,schemaVersion,sourceSystem, optionalidempotencyKey. Consumers can route, dedupe, and trace without deserializing the body — useful for poison messages and metrics. - Headers are opaque bytes — encode strings as UTF-8 by convention. Spring Kafka's
DefaultKafkaHeaderMapperhandles the conversions automatically. - Keep header payloads small (under a few KB total). They consume broker memory and bandwidth on every message.
- Distributed tracing libraries (Micrometer Tracing, Brave, OpenTelemetry) propagate
traceparentautomatically when you use the auto-configuredKafkaTemplate— manual injection is the fallback for code that builds rawProducerRecords.
Common Mistakes & Pitfalls
Putting metadata like correlationId and eventVersion inside the message payload instead of headers
Check your understanding
Answer these without looking back at the article.
- How do you propagate distributed trace context through Kafka messages?
- What is the __TypeId__ header and how can it cause deserialization issues?
- How do you handle messages that have an unexpected or missing header?
Related interview prep categories
2Want a guided curriculum order?
These modules are also sequenced in our flagship backend interview track — open it when you prefer an ordered path instead of browsing by topic.