LLM Function Calling with Spring AI — FunctionCallback and @Bean Registration

Direct Answer

Function calling lets the LLM decide to invoke a registered Java function instead of returning plain text. The model returns a JSON payload with the function name and arguments; your code executes it and sends the result back. In Spring AI, register functions as FunctionCallback beans and pass them via FunctionCallingOptions. Spring AI handles the round-trip automatically.

Deep Dive & Architecture

All three are supported by Spring AI starters: spring-ai-openai, spring-ai-anthropic, spring-ai-vertex-ai-gemini.

The ChatClient interface is identical across providers — swap the starter and properties to switch.

Key differences: OpenAI uses OpenAiChatModel, Anthropic uses AnthropicChatModel, Google uses VertexAiGeminiChatModel. Auth differs — OpenAI and Anthropic use API keys, Gemini on Vertex AI uses service account credentials. Token context windows and pricing vary significantly.

Manually wiring HTTP clients when Spring AI's auto-configuration handles it; not using the ChatClient builder pattern and going straight to the lower-level model

Common Mistakes & Pitfalls

Thinking function calling executes server-side code directly — the model just returns a JSON intent; your code does the actual execution. Registering functions with wrong parameter types so the model can't serialize arguments.