Streaming Token Responses in Spring AI with Spring Boot REST
Fast recall
Quick revision
Core answer
Flux from ChatClient's .stream() method. In a Spring Boot REST controller, you return that Flux with MediaType.TEXT_EVENT_STREAM_VALUE — Spring WebFlux serializes each token chunk to the client as a Server-Sent Event. This gives users the "typing" experience without waiting for the full response to generate.Whether the candidate knows how to wire reactive streaming from the ChatClient through a REST controller and understands the Server-Sent Events contract
Using .call() instead of .stream() and blocking the response — or not setting the content type to text/event-stream, which breaks the SSE protocol
Mention the difference between Flux<ChatResponse> (full typed response per chunk) and Flux<String> (just the token text), how to extract content with .content(), and how to handle backpressure or error signals in the reactive stream
Interview-ready guide
Interview answer
Streaming in Spring AI uses Reactor's Flux. Here's the pattern.
ChatClient side
- Call
.stream()instead of.call()— this returns aFlux(orFluxif you call.content()on it) - Each element in the
Fluxcarries one chunk of tokens as the model produces them
Controller side
- Return
Fluxfrom your@GetMappingmethod - Set
produces = MediaType.TEXT_EVENT_STREAM_VALUE— this tells Spring WebFlux to use SSE framing - WebFlux serializes each
Fluxemission as adata:SSE line and flushes it to the client
What the client sees
- A sequence of
data:events arriving progressively\n\n - The stream ends with the completion signal from the model
Error handling
- Use standard Reactor operators:
onErrorResumeto return a fallback token,retrywith a back-off for transient failures
Stand-out point: .stream().content() gives you Flux — just the token text. .stream().chatResponse() gives you Flux — useful if you need metadata like token usage or finish reason per chunk.
Streaming Token Responses in Spring AI with Spring Boot REST
Direct Answer
Spring AI returns streaming responses as a Flux<ChatResponse> from ChatClient's .stream() method. In a Spring Boot REST controller, you return that Flux<String> with MediaType.TEXT_EVENT_STREAM_VALUE — Spring WebFlux serializes each token chunk to the client as a Server-Sent Event. This gives users the "typing" experience without waiting for the full response to generate.
Deep Dive & Architecture
ChatClient is Spring AI's fluent, provider-agnostic builder API for sending prompts and reading responses.
Calling the OpenAI API directly (via the OpenAI Java SDK or raw HTTP) ties your service code to OpenAI's specific request/response types, requires manual retry and error handling, and gives you no abstraction layer to add RAG or memory.
ChatClient wraps any ChatModel implementation, integrates with Spring's DI container, and lets you attach Advisors for cross-cutting concerns — all without changing call sites.
A common mistake is confusing Spring AI with just a thin wrapper around the OpenAI SDK — it's a full framework with vector store abstractions, embedding models, prompt templates, and RAG support
Common Mistakes & Pitfalls
Using .call() instead of .stream() and blocking the response — or not setting the content type to text/event-stream, which breaks the SSE protocol
Check your understanding
Answer these without looking back at the article.
- What is Spring AI and how does it fit into the Spring ecosystem?
- How do you configure Spring AI to work with different LLM providers (OpenAI, Anthropic, Gemini)?
- What is `ChatClient` in Spring AI and how does it differ from calling the OpenAI API directly?
Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.