Design a Task Scheduler / Job Queue in Java

Data structure

Three viable choices:

• PriorityBlockingQueue ordered by (runAt, priority): O(log N) insert/extract, dispatcher computes sleep math
• DelayQueue (Java built-in): take() blocks until head's delay expires, eliminates manual sleep math, same O(log N)
• Hashed timing wheel (Netty HashedWheelTimer): array of buckets, O(1) schedule and cancel, tick processes one bucket

For 99% of cases DelayQueue is right; reach for timing wheels at extreme schedule rates (10K+/sec) or for ms-precision at very large pending sets.

Architecture — dispatcher + worker pool

Two deliberately separate roles:

• Dispatcher thread: sleeps until next task is due, hands it to executor, returns to sleep. Fast — just queue ops.
• Worker pool: runs tasks concurrently. Bounded thread pool sized for throughput.

If dispatcher ran tasks itself, one slow task blocks all subsequent scheduling. Always separate.

Worker pool: bounded LinkedBlockingQueue (10K cap) + CallerRunsPolicy for backpressure (dispatcher slows but never drops) or AbortPolicy if dropping is OK.

Implementation

DelayQueue<ScheduledTask> queue = new DelayQueue<>();
Thread dispatcher = new Thread(() -> {
    while (running) {
        ScheduledTask t = queue.take();   // blocks until due
        workers.submit(() -> safelyRun(t));
    }
});

ScheduledTask implements Delayed; getDelay returns nanos remaining; compareTo orders by runAt then priority.

Use System.nanoTime() for monotonic math — not affected by clock changes; only currentTimeMillis for human display.

Failure isolated in safelyRun — one task can't kill the dispatcher or starve siblings.

Periodic tasks

Self-reschedule in a finally block so failures still continue:

Runnable wrapper = new Runnable() {
    @Override public void run() {
        try { task.run(); }
        catch (Throwable t) { log.error(...); }
        finally { schedule(this, period); }
    }
};

Fixed-rate: next = previous scheduled time + period. Risk: piles up if runs are slow. Fixed-delay: next = completion + period. Cleaner; default unless cadence consistency is required. Java: scheduleAtFixedRate vs scheduleWithFixedDelay.

Cron-style: Quartz CronTrigger evaluates expression at each tick.

Cancellation

DelayQueue.remove is O(N) — it's a heap; arbitrary removal is linear. For O(log N) cancel, use a ConcurrentSkipListMap keyed by runAt + ConcurrentHashMap lookup by id. Or timing wheel (O(1) cancel).

Persistence — survive restart

In-process schedulers lose pending tasks on JVM restart. For persistence:

• Quartz with JDBC store: tasks in DB; row locking for multi-node single-fire; misfire policy for past-due tasks (ignore / fire-now / reschedule)
• External job queue: see job queue design (Redis sorted set, Kafka deferred, SQS delay)
• Snapshot to disk: simple; loses tasks between snapshots

For pure in-process: ScheduledExecutorService or our DelayQueue. For persistence: Quartz or job queue.

LLD

• TaskScheduler facade with schedule / scheduleAt / schedulePeriodic
• TaskQueue interface (DelayQueueImpl, TimingWheelImpl) — swap based on scale
• Dispatcher — single thread, take + submit
• WorkerPool — bounded executor
• ScheduledTask — id, runAt, priority, runnable
• RetryPolicy — nextAttempt for failed tasks

Failure modes

• Task throws — safelyRun catches; siblings unaffected; periodic re-schedules
• Worker pool saturated — CallerRunsPolicy backpressure; or AbortPolicy + alert
• Periodic slower than period — use fixed-delay; alert on cadence drift
• Clock skew — nanoTime for math; currentTimeMillis only for display
• Cancel-heavy on DelayQueue — O(N) penalty; use indexed map + skip list or timing wheel
• Misfires (Quartz) — misfire policy: ignore / fire-now / reschedule
• Two nodes firing the same task — distributed lock (Quartz JDBC) or leader election

Production picks

• ScheduledExecutorService — simple in-process
• Quartz — complex schedules, persistence, multi-node
• Netty HashedWheelTimer — ultra-high schedule rates, ms precision
• Job queue (Redis/Kafka/SQS) — distributed scheduling at scale

Recommendation

Use DelayQueue + dispatcher thread + bounded worker pool for pure in-process scheduling. Periodic tasks self-reschedule in finally, default to fixed-delay over fixed-rate. For O(log N) cancel, layer an indexed skip list. Use System.nanoTime() everywhere for monotonic math. For persistence and multi-node coordination, reach for Quartz with JDBC store. For ultra-high-throughput and ms precision, Netty's HashedWheelTimer. For distributed scheduling at scale, delegate to a job queue (Redis sorted set or Kafka deferred topic).