Scenario — Slow and Failing Batch Job, Production Debug Walkthrough
Fast recall
Quick revision
Core answer
Production debugging mindset, familiarity with Batch execution metadata, DB/infra troubleshooting, progressive tuning
Jumping to 'add more threads' without identifying the real bottleneck
Discuss `/actuator`, Micrometer, tailing `execution_context` for reader progress, correlating Batch metrics with DB slow-query logs and deploy markers
Interview-ready guide
Interview answer
I'd treat slowdown and failure as potentially separate problems and triage in three phases.
Phase 1 — gather data (15 minutes):
- StepExecution history from BATCH_* tables — duration, read/write counts, skip and rollback counts per step
- Actuator metrics —
spring.batch.chunk.writeandspring.batch.item.processp50/p99 - DB-side — slow query log,
pg_stat_statements, lock waits, replication lag - Host — CPU, GC pauses, disk IO
- Compare a recent 6h run against an older 2h run; the step that grew most is the bottleneck
Phase 2 — hypothesis testing:
- Volume growth (40% of cases) —
read_countlarger than last month; throughput unchanged - Throughput regression (25%) — same volume, slower; check query plans, missing indexes, lock contention
- Failure-driven (15%) —
rollback_countclimbing; retry storms; identify the exception and decide to skip, retry less, or fail fast - Infrastructure regression (10%) — DB downsized, slower storage, JVM upgrade, network change
Phase 3 — fix in tiers:
- Quick wins first — chunk size 100→500→1000→2000, JDBC batch alignment, missing index, ANALYZE, off-peak
- Scaling — partitioning by hash key, async processor/writer for per-item I/O, read from replica, bulk loaders
- Re-architect — CDC/streaming, materialized views, analytical engines
Patterns to call out:
- Don't scale before diagnosing — adding partitioning when the cause is a missing index just makes 10 workers do 10× the wasted work
- Over-tuning chunk size has diminishing returns; 100→500 is 2-3×, 500→2000 is 10-30%
- Idempotency is non-negotiable for retries, skip, and partitioning
- Retry storms can silently double run time
Documentation and prevention:
- Build a slow-job runbook with exact SQL, actuator endpoints, log commands
- Add alerts on duration thresholds, skip rate, rollback count
- Synthetic load test in CI with scaled-down data
- Track volume growth and alert on projected SLA breach
In an interview I'd emphasize that the first question is always 'how much did the data grow?' — answers the problem in five minutes more often than candidates expect.
Scenario — Slow and Failing Batch Job, Production Debug Walkthrough
Direct Answer
Frame it as a triage — treat slowdown and failure as potentially separate problems. Gather data first: StepExecution history (start/end, read/write/skip counts, exceptions), DB-side metrics (query plans, lock waits, replication lag), host metrics (CPU, GC), and data-volume trends. Compare a recent slow run against an older fast run to identify what changed. Typical culprits in order of likelihood: data growth, index regression, lock contention, chunk size or JDBC batching misconfigured, retry storms, and resource contention. Fix in tiers: optimize query and add index → increase chunk size and JDBC batching → introduce partitioning → move to off-peak window. Always document the diagnosis so it isn't rediscovered next quarter.
Common Mistakes & Pitfalls
Jumping to 'add more threads' without identifying the real bottleneck
Check your understanding
Answer these without looking back at the article.
- How do you detect a retry storm early from metrics alone?
- What's your rollback plan when a tuning change goes wrong in production?
- When would you recommend moving from Spring Batch to streaming CDC?
- How do you test batch performance regressions in CI?
Related interview prep categories
1Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.