Implementing a RAG Pipeline with Spring AI and PGVector — Embed, Retrieve, Inject, Generate
Fast recall
Quick revision
Core answer
TokenTextSplitter, embed each chunk via EmbeddingModel, and store into PgVectorStore. Query: embed the user's question, run vector similarity search via DocumentRetriever or directly via VectorStore.similaritySearch(), inject top-k chunks as system context, then call the LLM. Spring AI's QuestionAnswerAdvisor automates the query phase — wire it into ChatClient.Builder.defaultAdvisors() and the full retrieve-augment-generate cycle runs per request. The underlying database needs the pgvector extension and a VECTOR(1536) column — PgVectorStore creates this automatically in Spring AI 1.0.0-M6.Whether the candidate knows the actual mechanics — not just the concept but the concrete Spring AI classes and the two-phase ingestion/query split. Can they wire VectorStore, EmbeddingModel, and QuestionAnswerAdvisor correctly?
Describing only the query phase and forgetting ingestion entirely. Also: not knowing that the same EmbeddingModel must be used in both phases, or thinking the vector store is optional
Mention the token budget problem — you can only inject so many chunks before hitting the context window. Discuss metadata filters, similarity thresholds in SearchRequest, and async batch ingestion for large document sets
Interview-ready guide
Interview answer
A Spring AI RAG pipeline splits into two phases — ingest documents once, query on every user request.
Ingestion phase
- Load documents with a reader like
PagePdfDocumentReader - Split into chunks with
TokenTextSplitter - Call
vectorStore.add(chunks)—PgVectorStoreembeds each chunk viaEmbeddingModeland stores the vector in PostgreSQL - The schema (pgvector extension + vector_store table) auto-creates in development
Query phase — what QuestionAnswerAdvisor does for you
- 1Intercepts the ChatClient call before it hits the LLM
- 2Embeds the user's question via the same EmbeddingModel
- 3Runs cosine similarity search on PgVectorStore, returns top-k documents
- 4Injects those documents as a system context block in the prompt
- 5Forwards the augmented request to the LLM
Wiring it up
- Add
QuestionAnswerAdvisortoChatClient.Builder.defaultAdvisors()— one line in a@Configurationbean - Your controller just calls
chatClient.prompt().user(question).call().content()
Production tuning
SearchRequest.withTopK(6)and.withSimilarityThreshold(0.75)filter out irrelevant chunks- Metadata filters on
SearchRequestlimit retrieval to a document subset without a separate index - Watch token budget — 6 chunks × 300 tokens each eats 1,800 context tokens before the question lands
Implementing a RAG Pipeline with Spring AI and PGVector — Embed, Retrieve, Inject, Generate
Direct Answer
A Spring AI RAG pipeline has two phases. Ingestion: load documents, split into chunks with TokenTextSplitter, embed each chunk via EmbeddingModel, and store into PgVectorStore. Query: embed the user's question, run vector similarity search via DocumentRetriever or directly via VectorStore.similaritySearch(), inject top-k chunks as system context, then call the LLM. Spring AI's QuestionAnswerAdvisor automates the query phase — wire it into ChatClient.Builder.defaultAdvisors() and the full retrieve-augment-generate cycle runs per request. The underlying database needs the pgvector extension and a VECTOR(1536) column — PgVectorStore creates this automatically in Spring AI 1.0.0-M6.
Common Mistakes & Pitfalls
Describing only the query phase and forgetting ingestion entirely. Also: not knowing that the same EmbeddingModel must be used in both phases, or thinking the vector store is optional
Check your understanding
Answer these without looking back at the article.
- What happens if you use a different embedding model at query time than at ingest time?
- How do you handle documents larger than the token limit for a single chunk?
- When would you use VectorStore.similaritySearch() directly instead of QuestionAnswerAdvisor?
- How do you add reranking on top of the initial vector search results?
- What are the pgvector index options (IVFFlat vs HNSW) and how do they affect RAG query latency?
Looking for a structured roadmap?
Every module on this page is also part of the full Java backend interview track.