Chunk Size Tuning and Custom SkipPolicy Design
Fast recall
Quick revision
Core answer
Practical tuning intuition and the ability to write a non-trivial SkipPolicy that handles different exception classes differently
Picking a chunk size from theory ('larger is faster') without measuring; using `skipLimit(Integer.MAX_VALUE)` and discovering the job has been silently corrupting data for weeks
Discuss percentage-based skip tolerance, exception-specific limits, and a circuit-breaker SkipPolicy that fails fast when error rate spikes mid-job
Interview-ready guide
Interview answer
Chunk size tuning and skip policy design are two of the most-asked production Spring Batch questions, and both have answers that depend on the workload shape — not on framework defaults.
Chunk size tuning is empirical:
- Measure throughput at 100, 250, 500, 1000, 2000 on representative data
- Pick the largest size where memory, DB lock duration, and rollback cost stay acceptable
- Watch for rising
rollback_countat large chunks — that's the signal you've gone past the sweet spot - For JDBC, 250-1000 is typical; for JPA writers, 100-500
- Align chunk size with
hibernate.jdbc.batch_sizeso each chunk maps to one DB batch
Skip policy is a risk-tolerance decision:
SimpleLimitSkipPolicycounts every skip against one global limit — fine for simple jobs- For nuance, implement
SkipPolicy.shouldSkip(throwable, skipCount) - Per-exception limits — allow 1000 parse errors but only 10 DB integrity errors
- Percentage-based tolerance — fail fast if more than 5% of items are skipping; the data is too dirty to trust
- Circuit-breaker — read
StepExecution.getReadCount()and compare against the skip count
Always pair skip with a SkipListener:
- Write skipped items to a dead-letter Kafka topic, file, or table
- Otherwise you've silently corrupted data and can't reprocess or audit
Anti-patterns to call out:
skipLimit(Integer.MAX_VALUE)masks bugsskip(Exception.class)swallows real errors like NullPointerException- No dead-letter destination — skipped items vanish forever
In an interview, I'd emphasize that chunk tuning rewards measurement over theory, and that skip policy is fundamentally a question about how much bad data you're willing to silently discard.
Chunk Size Tuning and Custom SkipPolicy Design
Direct Answer
Chunk size tuning is empirical, not theoretical — start at 100, measure throughput at 250/500/1000/2000, and pick the largest size where memory and DB lock duration stay acceptable. Signals that the chunk size is wrong: rising rollback_count (chunks too large, hitting transient errors that force full re-read), high transaction commit overhead in DB metrics (chunks too small), heap pressure or OOM (chunks too large with heavy items). For skip policy, SimpleLimitSkipPolicy counts skips against a global limit; for nuanced control, implement SkipPolicy.shouldSkip(Throwable t, long skipCount) to skip on specific exception classes, allow more skips for parse errors than for DB errors, and integrate with circuit-breaker logic that fails fast when error rate exceeds a threshold.
Common Mistakes & Pitfalls
Picking a chunk size from theory ('larger is faster') without measuring; using skipLimit(Integer.MAX_VALUE) and discovering the job has been silently corrupting data for weeks
Check your understanding
Answer these without looking back at the article.
- How do you tune chunk size when you cannot run the job in production-like data volumes?
- What happens to the skip counter when a step is restarted from a checkpoint?
- How would you implement a SkipPolicy that fails fast when error rate exceeds 5% of items read?
- Why might `LimitCheckingItemSkipPolicy` be preferable to a custom SkipPolicy?
Related interview prep categories
1Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.