Agentic Retrieval vs Standard Retrieval in Amazon Bedrock RAG
· Engineer's Notes · Cem Koyluoglu
Comparison of single‑shot and agentic retrieval paths for multi‑intent queries in LangChain and Bedrock.
What happened\n\nThe article describes a Retrieval Augmented Generation (RAG) system built with LangChain that compares two products across three dimensions, effectively posing six separate questions. A single query vector is used for similarity search in the standard retrieval path, producing concise answers that are error‑free and have reasonable relevance scores. However, the retrieved chunks cover only a fraction of the full question. The article then introduces agentic retrieval, a feature of Amazon Bedrock Managed Knowledge Bases that plans the retrieval process. It decomposes the query into sub‑queries, executes them, evaluates whether sufficient evidence has been gathered, and repeats the search if necessary. The langchain‑aws package exposes both the standard Retrieve API and the AgenticRetrieveStream API.\n\nThe solution removes the need for a self‑managed vector store, embeddings, and re‑ranking models. A data source is attached via Amazon S3, and the knowledge base handles chunking, embedding, storage, and retrieval. Two APIs are available: Retrieve, which performs a single hybrid search, and AgenticRetrieveStream, which streams a planning loop and trace events. The article walks through creating a knowledge base, ingesting documents, and querying it with both retrieval methods.\n\n## Why it matters in production\n\nStandard retrieval offers the lowest latency and cost because it performs a single search call and returns the top‑scoring chunks. It is suitable for queries with a single clear intent, where the application can control the generation step. For multi‑part queries, the single‑vector approach can miss intents, leading to incomplete answers and potential user frustration. Agentic retrieval introduces a planning loop that can increase latency and cost due to multiple search iterations and additional model inference. However, it guarantees that evidence sufficiency is evaluated, which improves answer completeness and reliability. The need for additional IAM permissions (bedrock:AgenticRetrieveStream, bedrock:InvokeModelWithResponseStream, bedrock:GetDocumentContent) also raises the security surface area and requires careful policy scoping. The generate_response option returns an answer and citations in a single response, reducing downstream model calls and simplifying evaluation pipelines.\n\n## Engineering takeaways\n\n- Deploy AmazonKnowledgeBasesRetriever with managedSearchConfiguration for single‑intent queries to keep latency and cost minimal.\n- For queries containing multiple intents, use agentic_retrieve with generate_response=True to obtain a grounded answer and citations in one call.\n- Construct IAM policies that include bedrock:AgenticRetrieveStream, bedrock:InvokeModelWithResponseStream, and bedrock:GetDocumentContent, scoped to the specific knowledge‑base ARN to limit privilege escalation.\n- Monitor the number_of_results and the number of retrieval iterations to control cost; consider capping results at ten for agentic retrieval.\n- Implement a polling loop similar to wait_for_ingestion to detect ingestion job completion, avoiding fixed‑interval sleeps and handling failure reasons.
Related work on this site
- RAG & Grounded LLM Systems — LLM features that answer from the right source instead of from memory: document-grounded assistants, transcript-grounded chat and quality gates around generated output.
- Agentic & LLM Automation — Multi-step LLM pipelines that run unattended: model cascades with fallbacks, validation gates, scheduled automation and alerting when something breaks.
- YouTube AI Summarizer — Open-source Chrome extension on the Chrome Web Store: AI summaries, key points, deep analysis, a two-host AI podcast (Gemini TTS) and transcript-grounded chat for any YouTube video, using Groq or Ollama Cloud with the user’s own key.
- Ireland Expat Assistant — A custom GPT that answers from uploaded official documents: step-by-step guidance on Irish visas/IRP, employment permits, tax (PAYE/PRSI/USC), HSE/Medical Card and citizenship.
This note was drafted with AI assistance from the primary source credited on this page, and automatically checked against that source before publishing.