JPA Batch Processing — Inserts, Updates, StatelessSession, COPY

Direct Answer

Default Hibernate sends one SQL per persist/merge — catastrophic for large batches. Enable batching with hibernate.jdbc.batch_size=50 plus order_inserts and order_updates=true, switch generated IDs to SEQUENCE (IDENTITY blocks batching), and call em.flush(); em.clear() every N records to prevent first-level cache OOM. For very large loads, drop to StatelessSession (no persistence context), JdbcClient.batchUpdate (5–10× faster than JPA batch), or Postgres COPY (10M+ rows/minute). Wrap in a single transaction with chunked commits, disable second-level cache during imports, and consider dropping non-essential indexes for one-off migrations.

Code Example

A reusable service that combines StatelessSession, chunked commits, dead-letter handling, and progress metrics.

@Service
public class BulkImportService {
    @Autowired SessionFactory sf;
    private static final int BATCH = 50;

    public ImportResult importStream(Stream<OrderDto> stream) {
        AtomicInteger ok = new AtomicInteger();
        AtomicInteger fail = new AtomicInteger();

        try (StatelessSession s = sf.openStatelessSession()) {
            Transaction tx = s.beginTransaction();
            int i = 0;
            var iter = stream.iterator();
            while (iter.hasNext()) {
                var dto = iter.next();
                try {
                    s.insert(toEntity(dto));
                    if (++i % BATCH == 0) {
                        s.getTransaction().commit();
                        tx = s.beginTransaction();
                    }
                    ok.incrementAndGet();
                } catch (Exception e) {
                    fail.incrementAndGet();
                    log.warn("import fail: {}", dto, e);
                }
            }
            tx.commit();
        }
        return new ImportResult(ok.get(), fail.get());
    }
}

Production additions to layer on top:

• Dead-letter table capturing failed rows with the original payload and exception message.
• Idempotency via INSERT ... ON CONFLICT DO NOTHING (Postgres) or a unique constraint that lets reruns skip already-loaded rows.
• Checkpointing — persist the last successful offset so a restart resumes mid-file.
• Spring Batch for complex pipelines that need restart, retry, skip policies, and parallel step execution.

Common Mistakes & Pitfalls

Calling repository.save() in a loop, leaving @GeneratedValue(strategy = IDENTITY) (which silently disables JDBC batching), and never clearing the EntityManager — producing both linear network round-trips and unbounded heap growth