@Scheduled Observability, Error Handling, and Resilience Patterns

Production @Scheduled tasks need three layers of safety beyond the bare annotation.

Observability:

• Structured logging with MDC — taskName, runId, triggeredBy propagate through the stack
• Micrometer metrics — scheduled.task.duration and scheduled.task.runs with outcome tag
• Heartbeat metric — emit on every successful run; Prometheus alert on staleness (no success in 2x expected period)
• Distributed tracing via @Observed to correlate with downstream calls

Error handling:

• Wrap the method body in try/catch — preserve MDC, emit failure metric, log with context
• Custom ErrorHandler on the TaskScheduler for uncaught exceptions — default just logs
• Distinguish transient from permanent failures — different alert thresholds
• Never silently swallow — even 'expected' exceptions deserve a metric

Resilience:

• Delegate heavy work to a dedicated Executor — scheduled method runs in milliseconds, finds work, dispatches
• Graceful K8s shutdown — setWaitForTasksToCompleteOnShutdown(true) plus terminationGracePeriodSeconds
• Runaway-thread protection — ShedLock lockAtMostFor, per-task timeouts, or circuit breakers

The @Async + @Scheduled pitfall:

• @Async works via AOP proxy; calling this.asyncMethod() bypasses it
• A @Scheduled method calling this.heavyWorkAsync() does NOT run async
• Split across two beans — TriggerScheduler with @Scheduled calling HeavyWorker.runAsync()

Common production failures these patterns prevent:

• Task throws silently for a week — no one notices because default error handler just logs
• Slow task blocks the scheduler thread, delaying all other scheduled methods
• Lock not released after holder crash — task halted until manual intervention
• K8s rolling deploy kills mid-task without graceful shutdown
• Cron fires at unexpected time after DST transition

Testing tips:

• @EnableScheduling is disabled in test slices — test method directly
• Verify cron: CronExpression.parse(...).next(Instant.now())
• Integration tests simulate time advancement

In an interview I'd emphasize the heartbeat metric — that's the single best safety net for catching silent failures.