Configuring Spring AI for Multiple LLM Providers — OpenAI, Anthropic, Gemini
Fast recall
Quick revision
Core answer
application.properties, and Spring Boot auto-configures the right ChatModel bean. Your application code injects ChatModel or ChatClient — it doesn't know which provider is underneath. Switching providers is a dependency swap and a config change, not a code change.Whether the candidate understands the auto-configuration pattern, can name the provider starters, and knows how to set model options like temperature and max tokens
Manually instantiating OpenAiChatModel or AnthropicChatModel instead of letting auto-configuration do it — this defeats the portability and makes testing harder
Mention how ChatOptions (provider-specific options like temperature, topP, maxTokens) layer on top of the model default, and how you can override them per-call via the ChatClient fluent API
Interview-ready guide
Interview answer
Spring AI's provider configuration follows a consistent pattern: one starter, one config block, one auto-configured bean.
Steps to configure a provider
- 1Add the starter:
spring-ai-openai-spring-boot-starterfor OpenAI,spring-ai-anthropic-spring-boot-starterfor Anthropic,spring-ai-vertex-ai-gemini-spring-boot-starterfor Gemini - 2Set the API key and model name in application.properties under
spring.ai..chat.options - 3Inject
ChatModelor useChatClient.Builder— auto-configuration wires the implementation
Options hierarchy
- Default options come from application.properties
- You can override them per-call by passing a
ChatOptionsimplementation toPrompt - Temperature, max tokens, and model name are all tunable this way
Multiple providers
- All starters can coexist on the classpath
- Mark one
ChatModelas@Primaryor use@Qualifierto route to the right one - Useful for fallback routing or cost-based dispatch
Important: Never hardcode API keys in properties files. Use ${ENV_VAR} expansion and inject secrets from environment variables or a secrets manager.
Configuring Spring AI for Multiple LLM Providers — OpenAI, Anthropic, Gemini
Direct Answer
Each LLM provider ships its own Spring Boot starter. You add the starter dependency, set the provider-specific API key and model name in application.properties, and Spring Boot auto-configures the right ChatModel bean. Your application code injects ChatModel or ChatClient — it doesn't know which provider is underneath. Switching providers is a dependency swap and a config change, not a code change.
Deep Dive & Architecture
ChatClient is Spring AI's fluent, provider-agnostic builder API for sending prompts and reading responses.
Calling the OpenAI API directly (via the OpenAI Java SDK or raw HTTP) ties your service code to OpenAI's specific request/response types, requires manual retry and error handling, and gives you no abstraction layer to add RAG or memory.
ChatClient wraps any ChatModel implementation, integrates with Spring's DI container, and lets you attach Advisors for cross-cutting concerns — all without changing call sites.
A common mistake is confusing Spring AI with just a thin wrapper around the OpenAI SDK — it's a full framework with vector store abstractions, embedding models, prompt templates, and RAG support
Common Mistakes & Pitfalls
Manually instantiating OpenAiChatModel or AnthropicChatModel instead of letting auto-configuration do it — this defeats the portability and makes testing harder
Check your understanding
Answer these without looking back at the article.
- What is Spring AI and how does it fit into the Spring ecosystem?
- How does Spring AI handle streaming token responses in a Spring Boot REST API?
- What is `ChatClient` in Spring AI and how does it differ from calling the OpenAI API directly?
Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.