Deep Insight: Production Architecture for Multi‑Agent Systems
· Engineer's Notes · Cem Koyluoglu
Analysis of Deep Insight’s multi‑agent design, deployment options, and engineering practices for reliable, cost‑effective production AI.
What happened
The AWS Solutions Architecture team built Deep Insight, a production‑grade multi‑agent system that turns CSV data and a natural‑language prompt into a full analysis report. The system comprises eight specialized agents—Coordinator, Planner, Plan Reviewer, Supervisor, Coder, Validator, Reporter, and Tracker—each operating in its own context and passing compressed results onward. The architecture leverages the Strands Agents SDK, Amazon Bedrock, Bedrock AgentCore, an Application Load Balancer, and AWS Fargate. Three deployment options are offered: a Self‑Hosted CLI version that sets up in about 10 minutes, a Web UI version with Human‑in‑the‑Loop (HITL) plan review, and a Managed AgentCore version that runs inside a private VPC with per‑user isolation and a separate Code Interpreter on Fargate. The Planner uses Claude Opus with Extended Thinking to generate a 5‑10 step plan, while the Coordinator classifies requests with Claude Haiku to keep costs low. The Plan Reviewer enforces a maximum revision count (MAX_PLAN_REVISIONS) and auto‑approves after a 300‑second countdown if the user does not respond. Execution flow follows a directed graph pattern, and the Supervisor invokes specialized agents via an Agents‑as‑Tools pattern, ensuring each agent’s context remains small.
Why it matters in production
The design addresses several production concerns. Isolating each agent in an independent context reduces the risk of cascading failures and limits memory usage, improving reliability. Using a lightweight model (Claude Haiku) for simple classification cuts inference cost, while reserving the more expensive Claude Opus for planning tasks where higher reasoning quality is needed, balancing cost and latency. The graph‑based execution order makes error handling explicit; failed steps can be retried without re‑executing the entire workflow, lowering overall latency. Deploying the Managed version inside a private VPC and running code generation on Fargate provides network isolation and mitigates security risks associated with executing generated code. The Plan Reviewer’s revision limit and timeout prevent infinite loops, contributing to predictable operational behavior. Together, these choices enable scalable, secure, and cost‑controlled multi‑agent deployments.
Engineering takeaways
- Model selection should match task complexity: use a lightweight model for routing or classification and reserve larger models for planning or reasoning steps.
- Structure agent interactions as a directed graph and invoke specialized agents as tools to keep each agent’s context minimal and simplify error handling.
- Implement a Human‑in‑the‑Loop review node with explicit revision limits and auto‑approval timeouts to avoid indefinite loops and provide user oversight.
- Run any generated code in isolated containers such as AWS Fargate within a private VPC to enforce network and execution security.
- Provide multiple deployment pathways (local CLI, web UI, managed VPC) to accommodate development speed, non‑technical user access, and production‑grade isolation requirements.
Related work on this site
- Agentic & LLM Automation — Multi-step LLM pipelines that run unattended: model cascades with fallbacks, validation gates, scheduled automation and alerting when something breaks.
- RAG & Grounded LLM Systems — LLM features that answer from the right source instead of from memory: document-grounded assistants, transcript-grounded chat and quality gates around generated output.
- YouTube AI Summarizer — Open-source Chrome extension on the Chrome Web Store: AI summaries, key points, deep analysis, a two-host AI podcast (Gemini TTS) and transcript-grounded chat for any YouTube video, using Groq or Ollama Cloud with the user’s own key.
- Automated AI News Pipeline (this site) — The pipeline behind this site’s Tech News: scheduled GitHub Actions scrape sources, an LLM cascade on Groq rewrites and enhances articles, and layered quality gates (date integrity, language checks, instruction-leak detection, duplicate detection) decide what gets published, with Telegram alerts.
This note was drafted with AI assistance from the primary source credited on this page, and automatically checked against that source before publishing.