Scenario — Degrading Upstream and Cascading Failure
Fast recall
Quick revision
Core answer
WebClient connection pool, every request waiting on that upstream piles up in pendingAcquireQueue and the whole service degrades. The mitigation has four layers: tight per-call timeouts so requests fail fast rather than wait forever; a circuit breaker (Resilience4j) that opens after a failure threshold and short-circuits subsequent calls; a bulkhead that caps concurrent calls to the failing upstream so it can't starve other workloads; and a fallback — cached value, default response, or queued retry. Long-term: split clients per upstream so one slow dependency can't poison shared pools, and design endpoints to degrade gracefully with partial responses.Production hygiene around external dependencies, resilience patterns, and the cascading-failure failure mode of WebFlux pools
Increasing pool size to 'absorb' the slowness; missing the bulkhead concept; treating retries as a fix when they amplify the problem
Discuss per-upstream pool isolation, circuit breaker half-open probing, queueing-theory intuition for pendingAcquireQueue, and graceful degradation patterns
Interview-ready guide
Interview answer
When a slow upstream eats your WebClient connection pool, every request piles up in pendingAcquireQueue and the service degrades. The mitigation is layered.
Layer 1 — Tight per-call timeouts
responseTimeout, connect timeout, read / write timeout on theHttpClient- Per-call
timeout(Duration)in the chain - Pick based on upstream healthy p99 — typically 3-5x
- A request that times out fast releases its connection
Layer 2 — Circuit breaker (Resilience4j)
- Three states — Closed, Open, Half-open
- Configure failure-rate threshold, slow-call threshold, sliding window, half-open probe count
- Wire into the chain via
CircuitBreakerOperator - During incident, calls fail fast in microseconds; pool stops getting consumed
Layer 3 — Bulkhead and pool isolation
- Resilience4j Bulkhead caps concurrent calls
- Deeper bulkhead — a separate
WebClientbean per upstream with its own connection pool - Single most impactful architectural change for resilience
Layer 4 — Fallback
- Cached value, static default, partial response
Mono.zipwithonErrorReturnper dependency for graceful degradation- Mark missing pieces in response; clients can degrade their UI
What NOT to do
- Don't blanket-retry — amplifies load on broken upstream
- Don't increase pool size to absorb — queue just grows bigger
- Don't return generic 5xx for partial-failure cases — clients can handle partial data
Tuning rules of thumb
- Timeout — upstream healthy p99 × 3-5
pendingAcquireTimeout— 100-500 ms- Circuit breaker — 50% failure rate over 20 calls minimum 10
- Bulkhead — match upstream capacity, not your load
Observability essentials
- Circuit breaker state per upstream — alert on Open
- Bulkhead available concurrent calls
- Pool acquire time per pool
- Fallback invocation rate — non-zero means degradation
Strategic answer
- Treat external dependencies as adversarial — they will degrade
- Per-upstream pool isolation is the highest-leverage change
- Combine timeouts, circuit breaker, bulkhead, fallback — none alone is enough
- Test failure modes in chaos exercises
- Make degradation visible with partial responses, not blanket 5xx
Scenario — Degrading Upstream and Cascading Failure
Direct Answer
When a slow upstream eats your WebClient connection pool, every request waiting on that upstream piles up in pendingAcquireQueue and the whole service degrades. The mitigation has four layers: tight per-call timeouts so requests fail fast rather than wait forever; a circuit breaker (Resilience4j) that opens after a failure threshold and short-circuits subsequent calls; a bulkhead that caps concurrent calls to the failing upstream so it can't starve other workloads; and a fallback — cached value, default response, or queued retry. Long-term: split clients per upstream so one slow dependency can't poison shared pools, and design endpoints to degrade gracefully with partial responses.
Common Mistakes & Pitfalls
Increasing pool size to 'absorb' the slowness; missing the bulkhead concept; treating retries as a fix when they amplify the problem
Check your understanding
Answer these without looking back at the article.
- How do you size the WebClient connection pool per upstream?
- When does retrying make a degraded upstream worse?
- What metrics do you need to detect a circuit-breaker-induced silent degradation?
- How do you design endpoints that compose many upstreams to degrade gracefully?
Related interview prep categories
1Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.