Structured Logging with Logback and Correlation IDs

Structured logging means emitting JSON instead of plain text — which means each log entry becomes a queryable document that log aggregation systems (ELK, CloudWatch Logs Insights, GCP Logging) can filter, aggregate, and alert on by field value.

Setting up JSON logging with Logback

• Add logstash-logback-encoder dependency; configure a JSON encoder in logback-spring.xml: replaces the plain-text pattern encoder
• Each log line becomes {"@timestamp":"2024-01-15T10:30:00Z","level":"ERROR","logger":"com.myapp.OrderService","message":"Payment failed","traceId":"abc123","userId":"456"}
• Which means level = "ERROR" AND userId = "456" in Elasticsearch returns only the relevant entries from billions of log lines, without regex parsing

Correlation IDs with MDC

• A correlation ID is a unique identifier added to every log entry in a request — allowing you to filter all log entries from one request across all services
• Add a servlet filter that extracts or generates the correlation ID from the incoming request: MDC.put("correlationId", correlationId) in the filter; clear it in the finally block with MDC.remove("correlationId")
• The logstash-logback-encoder includes all MDC fields in the JSON output automatically — which means correlationId appears in every log entry without explicit inclusion

Spring Cloud Sleuth auto-populates trace IDs

• Spring Cloud Sleuth (or Micrometer Tracing in Spring Boot 3) automatically sets traceId and spanId in MDC for every request — which means every log entry includes the distributed trace ID, enabling correlation between the log entry and the Jaeger/Zipkin trace
• Sleuth propagates the trace ID through HTTP headers (X-B3-TraceId) and Spring Kafka message headers, so the same trace ID appears in logs across all services that processed the request

Adding business context to logs

• Add business-relevant fields to MDC at the service boundary: MDC.put("orderId", orderId) in the order service; MDC.put("customerId", customerId) — these appear in all downstream log entries automatically
• Structured log fields enable dashboards: count errors per orderId prefix, alert on elevated error rates for a specific customerId