The AI Engineering Toolbox: CrewAI, LangChain, LangGraph, and the RAG Stack You Must Know
The tooling around AI agents and RAG has exploded, and most "top 10 tools" lists are just logos. This is the opinionated map: what each tool is actually for, when to reach for it, when to skip it, and how the pieces fit into one coherent stack.
Search "top AI agent tools" and you'll get a wall of logos with one-line blurbs that all sound identical: "build powerful AI applications." Useless. The hard part was never finding tools — it's knowing which layer each one belongs to, when it earns its place in your stack, and when adding it just buys you a dependency and a new way to break.
This is the opinionated map. Not every tool that exists — the ones that actually matter, organised by the job they do, with an honest note on when to reach for each and when to skip it. The thesis underneath: most AI systems need far fewer tools than the ecosystem wants to sell you. A direct model SDK plus a vector store gets you remarkably far. Add the rest only when a real constraint demands it.
The layers of the stack
Every AI agent or RAG system is built from the same five layers. Once you see them, the tool landscape stops being a jumble of logos and becomes a grid — each tool slots into exactly one layer, and you pick at most one or two per layer.
Layer ①: evaluation and observability — the layer everyone skips
It sits at the bottom of the stack because everything rests on it, but it is the first thing to set up — and the first thing teams cut under deadline pressure. Don't cut it. An AI system you can't see inside is a system you can't debug, and a system you can't measure is a system you can't improve. Stand this up on day one, before you build a single layer above it.
- Tracing (LangSmith, Langfuse). Every step of every agent run — which tool was called, what came back, what the model decided — captured and inspectable. The first time an agent does something baffling in production, a trace turns a multi-hour mystery into a two-minute read.
- RAG evaluation (Ragas and similar). Score retrieval recall and answer faithfulness on a fixed question set, so every change is measured rather than vibed. This is the harness from the RAG guide made concrete.
- LLM-as-judge. Use a strong model to grade outputs against a rubric at scale — far cheaper than human review for catching regressions, and good enough to gate a deploy.
Layer ②: the model — start here, stay here longest
The provider SDK is the one tool you cannot skip, and the one whose features people most often reach past for a framework that just re-wraps them. Modern model SDKs already give you tool use (the agent loop), structured outputs, streaming, prompt caching, vision, and document handling natively. For app-building, default to the most capable current model — at the time of writing that's Claude Opus 4.8 (model id claude-opus-4-8) — and let the SDK's tool-runner drive the loop for you.
Layer ③: retrieval and memory — the vector store
If you're doing RAG (and most useful systems are), this layer is non-negotiable. The choice is mostly about scale and what you already run.
| Tool | What it is | Reach for it when |
|---|---|---|
| pgvector | Vector search as a Postgres extension | You already run Postgres. Keeps vectors beside your relational data and metadata. The right default for most teams. |
| Qdrant / Weaviate | Dedicated open-source vector engines | Volume, advanced filtering, or latency outgrow what pgvector comfortably handles. Self-hostable. |
| Pinecone | Fully managed vector database | You want zero ops and will pay for it; scaling and availability are someone else's problem. |
| A reranker | Cross-encoder that re-scores candidates | Always, for serious RAG — it's the highest-leverage retrieval upgrade (see the RAG guide). |
Don't start with a managed vector database because a launch blog told you to. Start with pgvector if you have Postgres. The day you genuinely outgrow it is a good day — it means you have real scale — and migrating is a known, bounded job. Premature infrastructure is the same mistake as premature abstraction.
Layer ④: framework glue — LangChain and LlamaIndex
LangChain is the connective tissue: document loaders for a hundred file types, text splitters, retriever interfaces, a unified abstraction over models and vector stores. LlamaIndex overlaps heavily but is more sharply focused on the RAG/data-indexing path. Both are genuinely useful for getting from zero to a working pipeline fast — the loaders and splitters alone save real time.
The honest caveat: these frameworks add abstraction layers, and abstraction has a cost. The same indirection that gets you started fast can get in your way when you need to debug why a particular chunk was retrieved or tune one specific step. Use them for the parts that are pure plumbing — loading and splitting documents is a solved problem you shouldn't rewrite — and don't be afraid to drop to the raw SDK for the parts where you need full control, like the final grounded prompt.
Layer ⑤: orchestration — LangGraph and CrewAI
This is the layer for genuine multi-step, multi-agent agentic systems — and the layer to skip entirely until you actually need it.
| Tool | Mental model | Best when |
|---|---|---|
| LangGraph | Agents as a state graph — you define nodes and the edges between them explicitly | You need control, branching, loops, and human-in-the-loop checkpoints in a complex flow. The more "engineered" choice. |
| CrewAI | Agents as a team of named roles collaborating on a task | Your problem maps naturally to "a researcher, a writer, and a reviewer." Higher-level and faster to express role-based collaboration. |
| Managed agent platforms | The provider runs the agent loop and hosts the tool sandbox for you | You want a stateful, long-running agent with a workspace and don't want to operate the orchestration or the execution environment yourself. |
A coherent default stack
If you want a concrete starting point that covers most real systems without over-buying, here it is:
# A sane default AI stack — add to it only when a real constraint demands it
Model → Provider SDK + native tool use (claude-opus-4-8)
Vector store → pgvector (you already run Postgres)
Retrieval → hybrid search (vector + BM25) + a reranker
Glue → LangChain loaders & splitters (plumbing only)
Orchestration → none yet — SDK tool-runner until you need multi-agent
Eval → a 100-question set + a tracing tool from day oneNotice what's not in there: a managed vector database, a multi-agent framework, a half-dozen integrations. None of those are bad tools. They're just tools you add when a specific constraint forces the choice — not because a tutorial assembled an impressive-looking stack.
How to choose, in one rule
For every tool you're about to add, name the specific constraint it solves that your current stack can't. If you can't name one — if the answer is "everyone uses it" or "it might be useful later" — don't add it. Every dependency is a thing that can break, a surface to learn, and an abstraction between you and the bug you'll eventually have to find.
The teams that ship reliable AI aren't the ones with the most tools. They're the ones who understand the five layers, pick deliberately within each, and can explain why every piece is there. Master the layers, and the endless "top 10 tools" lists stop being noise — they become a menu you order from with intent.
Where this fits
This is the practical middle of the AI series: the concepts map told you what to build at which altitude, the RAG guide went deep on the retrieval layer, and the final post turns all of it into concrete business automation blueprints — support, medical, legal, and finance — showing exactly how these tools assemble behind a simple chat box.
Comments (0)
No comments yet
Be the first to share a thought on this article.
Join the conversation