MCP vs RAG: Which Gives AI Agents Company Context?

MCP and RAG are not competing approaches to the same problem. RAG (retrieval-augmented generation) is a technique for finding the right passages in your company's documents and handing them to a model so it answers from your material, with citations. MCP (Model Context Protocol) is a standard for how an AI application connects to tools and data sources, so that Claude, Cursor, Copilot and your own agents can all reach the same systems the same way. RAG decides what the model knows. MCP decides what the model can reach. Most teams that ask "MCP or RAG?" need both, and a growing number are finding that for heavy multi-system work, a third option (letting the agent write code against APIs directly) beats both on cost and speed.

This post explains what each one does, where each one breaks, and a practical architecture for teams that want grounded answers, scoped system access, and the ability to use company knowledge from whichever AI tool they already work in.

Key Takeaways

RAG is a retrieval technique. It indexes your documents, pulls the relevant passages at question time, and lets the model answer from them with citations. It solves "the model doesn't know our stuff."
MCP is a connection standard. It defines how an AI app discovers and calls tools, reads resources, and authenticates against a server. It solves "every AI app needs its own integration with every system."
They stack rather than compete. A RAG system can be served over MCP, which is how you make company knowledge available inside Claude, Cursor, or Copilot without building four separate plugins.
MCP has a hidden cost: tool calling load. Every connected server adds tool definitions to the context window, every call is a round trip through the model, and every result gets read in full. Answering one question across five systems can mean a dozen sequential calls.
The efficient alternative for multi-step work is code. Let the agent write a short script that calls the APIs, filters the data, and returns only what matters. Anthropic measured a 98.7% token reduction on one such workflow. This is how QX agents work whenever the task allows.

What is RAG?

RAG is a way of giving a language model access to information it was never trained on. Your documents are split into passages, converted into searchable representations, and stored in an index. When someone asks a question, the system retrieves the handful of passages most relevant to it and passes them to the model alongside the question. The model writes the answer from those passages and cites where each claim came from.

The important word is retrieval. The model does not read your whole document set on every question, and it does not memorise it. It gets a small, relevant slice at answer time. That is what makes RAG cheap to run and what keeps answers checkable: a cited passage can be opened and verified in seconds.

We covered the mechanics of grounding and why citations matter in How AI Agents Use Your Company Knowledge, so this post will not repeat it. The short version: RAG is the standard answer to hallucination on company-specific questions, and it works well for "needle" questions like "what is the notice period in the Acme contract?"

It is less good at broad questions. Ask "tell me everything about Acme" or "which companies have we evaluated this year?" and a plain vector index returns its top ten hits and silently drops the rest. Our founder Jai wrote about this failure mode and how QX handles it in Why AI Company Brains Fail: QX layers a lightweight entity graph over the text index (companies, people, events and so on, resolved and linked as they appear across documents) so the agent can choose between passage search, entity lookup, and corpus-wide counting depending on the question. Keep that in mind, because "RAG" in practice ranges from a basic vector search to something considerably more capable, and the choice affects what you can safely ask it.

What is MCP?

MCP is an open protocol, originally released by Anthropic in November 2024 and donated to the Linux Foundation's Agentic AI Foundation in December 2025, that standardises how an AI application talks to external systems. Before it, every AI product needed its own bespoke integration with Slack, its own with Salesforce, its own with your internal database. MCP replaces that with one interface: a system exposes an MCP server, any MCP-capable client can connect to it, and the client discovers what the server offers at runtime.

An MCP server can expose three kinds of thing: tools (actions the model can call, like "search tickets" or "create a record"), resources (readable data such as a file or a table), and prompts (reusable templates). The current spec uses OAuth 2.1 for authentication against remote servers and a streamable HTTP transport, which is what makes hosted, multi-user MCP servers practical for businesses rather than just local developer setups.

The reason MCP matters is client support. Claude, ChatGPT, Gemini, Microsoft Copilot, GitHub Copilot, Cursor, VS Code and Windsurf all speak it. Build one MCP server for your internal knowledge and every one of those tools can use it. Before MCP that was four or five separate integration projects.

What MCP does not do is make the model any better at finding the right information. It is plumbing. If the server behind it does a bad job of search, the model gets bad results, delivered over a very well-standardised pipe.

MCP vs RAG: the side-by-side

	RAG	MCP
What it is	A retrieval technique	A connection protocol
Problem it solves	The model doesn't know your documents	Every AI app needs its own integrations
What it returns	Relevant passages, with citations	Whatever the tool returns (records, files, actions)
Where it lives	Inside your knowledge system (index + retriever)	Between the AI app and the system it's calling
Reads or writes	Read only	Read and write (tools can take actions)
Typical question	"What does our refund policy say about partial returns?"	"Create a Linear ticket for this bug and post the link in #eng"
Cost model	Cheap per query; indexing cost up front	Per call, and the calls add up (see below)
Failure mode	Silently drops results on broad questions; stale index	Context bloat, sequential round trips, over-broad permissions

The clearest way to hold the distinction: RAG is one of the things you might put behind an MCP server. QX does exactly this. Our Knowledge Vaults are a RAG system with an entity layer on top, and we expose them to Claude, Cursor and Copilot via a single MCP endpoint. Same retrieval, same citations, different front door.

When should you use RAG?

Use RAG when the job is answering questions from a body of documents that a human would otherwise have to read. Contracts, policies, product specs, meeting notes, past proposals, research reports. The signals that RAG is the right tool:

The answer exists in a document somewhere and the problem is finding it.
You need citations, because a person will act on the answer and needs to verify it.
The corpus changes on a schedule you can sync (daily, hourly), not second by second.
The question is read-only. Nobody needs the model to change anything.

A support agent answering "does our enterprise plan include SSO?" from the pricing docs is a RAG problem. So is a diligence analyst asking "what did the CIM say about customer concentration?" across 40 PDFs. So is an onboarding agent that lets a new hire ask questions of the employee handbook.

RAG is the wrong tool when the answer lives in a live system rather than a document. "How many open tickets does Acme have right now?" is not a retrieval question. Indexing your helpdesk into a document store and searching it would give you a stale count from the last sync. That is a tool call.

When should you use MCP?

Use MCP when the model needs to reach a live system, take an action, or when you need one integration to work across several AI clients. The signals:

The data is transactional and changes constantly (CRM records, tickets, calendar, inventory).
The task involves doing something: send, create, update, assign.
Your team uses more than one AI app and you don't want to build the integration more than once.
You want scoped, revocable access with a proper consent flow rather than a shared API key pasted into a prompt.

The last point is underrated. A well-built MCP server authenticates the user with OAuth, exposes only the tools that user is allowed to use, and can be disconnected in one click. That is a much better security posture than the alternative most teams start with, which is an API key in a system prompt.

MCP is the wrong tool when you have a large document corpus and your MCP server just wraps a keyword search over it. You will get the connection standard without the retrieval quality, and the model will confidently answer from the three passages it happened to find.

The hidden cost of MCP: tool calling load

This is the part most "MCP vs RAG" explainers skip, and it is the thing that bites teams once they get past the demo.

Connecting an MCP server to an agent does two things to the context window. First, every tool the server exposes gets its definition loaded up front: name, description, parameters. One server with eight tools is fine. Ten servers with fifteen tools each is 150 definitions the model reads before it has seen your question. Anthropic's engineering team described agents connected to thousands of tools having to process hundreds of thousands of tokens before reading the request.

Second, and worse, every tool result flows back through the model. Suppose someone asks an agent: "Which of our portfolio companies raised a support escalation last week, and what did we commit to in their contracts?" A faithful MCP-based agent has to do something like this:

Call the helpdesk server to list escalations from the last seven days. Read the full result.
For each escalation, call the CRM server to resolve the customer to a portfolio company. Read each result.
For each company, call the document server to find the contract. Read each result.
For each contract, call it again to pull the relevant clause. Read each result.
Assemble the answer.

That is not one tool call. It is one call, then N calls, then N more, then N more, each one a full round trip through the model, each result ingested in full even if the agent only needed one field from it. Five escalations turns into sixteen sequential calls. The latency stacks, the token bill stacks, and every intermediate result sits in the context window for the rest of the conversation. The Anthropic post gives the simple case: fetching a two-hour meeting transcript through one tool and writing it into another system means the model reads and re-emits the entire transcript, on the order of an extra 50,000 tokens, to move data it never needed to see.

None of this is a flaw in the protocol. It is a consequence of routing every operation through a language model. MCP makes connecting systems easy, and the ease is what leads teams to connect twenty of them and then wonder why a simple question takes ninety seconds and costs a dollar.

The third option: let the agent write code

There is an alternative that sidesteps most of the tool calling load, and it is the approach we default to at QX Labs: instead of having the agent call tools one at a time, have it write a short program that does the whole job, run that program in a sandbox, and show the agent only the output.

Take the portfolio question above. Rather than sixteen round trips, the agent writes something close to this in one pass:

escalations = helpdesk.list(since="7d")
companies = crm.lookup([e.customer_id for e in escalations])
contracts = [docs.find(company=c.name, type="contract") for c in companies]
clauses = [extract_sla(c) for c in contracts]
return summarise(escalations, companies, clauses)

The loops, the filtering and the joins all happen in the execution environment. The model sees the five escalations, the five company names, and the five clauses. It never sees the 200-row helpdesk export or the full text of five contracts. Anthropic's measurement on a comparable workflow was a drop from 150,000 tokens to 2,000, a 98.7% reduction, and the same post proposes presenting MCP servers as code APIs for exactly this reason.

Three things improve at once:

Cost. Tokens are the bill. Intermediate data that never enters the context is data you never pay for.
Speed. One execution pass replaces a chain of model round trips. The work that used to be sequential (call, wait, read, decide, call again) runs as ordinary code.
Reliability. Error handling, retries and conditionals are things code does well and language models do inconsistently. A loop in Python does not forget to process row 37.

This is why QX agents with Workspace Access can write and run code, and why we push them to do so whenever a task involves more than a couple of operations. The agent still uses the same connected apps and the same scoped credentials it would use for a direct tool call. It just batches the work.

The trade-off is that you now have an agent running code, which means you need a proper sandbox: isolated execution, credentials injected at runtime rather than visible to the model, network restricted to what the task needs. We wrote about what happens when that isolation is misconfigured in Agent Sandbox Escapes. If you are not prepared to run code execution properly, direct tool calls over MCP are the safer default, and you accept the token cost as the price of simplicity.

A practical architecture: RAG, MCP and code together

For a team that wants grounded answers, scoped system access, and company knowledge available inside the AI tools people already use, the pieces fit together like this.

Layer 1: a knowledge layer that retrieves and cites

Put your documents (drive folders, SharePoint, Notion, uploaded files) into an indexed store that re-syncs on a schedule, retrieves by meaning rather than keyword, and returns citations with every passage. Add an entity layer if you need to answer "everything about X" or "how many" questions, because plain vector retrieval will drop results on those. This layer is read-only by design.

Layer 2: a scoped access layer over MCP

Expose the knowledge layer, and your live systems, as MCP servers with OAuth and per-user scopes. Each agent, and each human's AI client, gets access to the specific vaults, folders and tools it needs and nothing else. In QX, an agent's editor lists exactly the apps, tools and vault folders it can reach, and only the tools you see in its editor exist for it. An agent that can read an inbox but not send from it cannot send, however it is asked.

Layer 3: a code execution layer for multi-step work

For anything that touches more than one system or more than a handful of records, let the agent write a script against those same scoped connections and run it in a sandbox. The agent reads only the output. Keep direct tool calls for single actions where a script would be overkill: post a message, create one ticket, look up one record.

Layer 4: the same knowledge, in every tool

Because the knowledge layer is behind MCP, it does not matter whether a person is working in Claude, Cursor, Copilot or a QX agent in Slack. They connect once, pick which vaults to authorise, and get the same grounded, cited retrieval. For QX Vaults the setup is a single server URL (https://platform.qxlabs.com/api/mcp/knowledge) added as a custom connector in Claude, an MCP entry in Cursor, or a tool in Copilot Studio. The first connection runs a browser consent flow where you choose all vaults or a selected few, and access can be revoked instantly from Settings.

Here is how the three mechanisms divide the work on real questions:

Question	Mechanism	Why
"What does the Acme MSA say about termination?"	RAG	Answer is in a document; needs a citation
"Create a follow-up task for the Acme renewal in Salesforce"	Single MCP tool call	One write action, one system
"Summarise every account where a contract renews in Q4 and there's an open escalation"	Code execution	Joins across CRM, helpdesk and documents; hundreds of intermediate rows the model doesn't need to read
"Ask our product docs a question from inside Cursor while I write this spec"	RAG over MCP	Same retrieval, different client

When this isn't the right fit

If your "company context" is a single folder of 30 documents and one person needs to ask it questions, a Knowledge Vault attached to an agent is enough. You do not need MCP, and you certainly do not need code execution. Reach for MCP when a second AI client enters the picture or when the agent needs to act on live systems. Reach for code execution when tool calls start chaining.

Equally, if your data is entirely transactional and lives in two systems, RAG adds nothing. Give the agent scoped tool access and skip the index.

And if your retrieval quality is poor, no amount of protocol will fix it. Test the knowledge layer on the hard questions (the broad ones, the counting ones) before you spend time on how it connects to anything.

FAQ

Is MCP a replacement for RAG?

No. RAG is a retrieval technique that finds relevant passages in your documents and grounds the model's answer in them. MCP is a protocol for connecting AI apps to tools and data. A RAG system can sit behind an MCP server, which is the common pattern. You choose RAG for knowledge quality and MCP for reach and portability.

Can I use RAG and MCP together?

Yes, and most production setups do. The knowledge system does the retrieval and citation work, and MCP is how AI clients like Claude, Cursor or Copilot connect to it. QX Knowledge Vaults work this way: one MCP server URL, a consent flow to pick which vaults are shared, and the same cited answers in every connected tool.

Why do MCP tool calls get expensive?

Every connected server loads its tool definitions into the context window, and every tool result is read by the model in full. A question spanning several systems becomes a chain of sequential calls, each one a round trip. Anthropic measured one multi-tool workflow at 150,000 tokens via direct calls versus 2,000 tokens when the agent wrote code instead.

What is code execution for agents and why is it more efficient?

Instead of calling tools one at a time, the agent writes a script that calls the APIs, loops over results, filters and joins them, then returns only the final output. Intermediate data never enters the model's context, so token cost drops sharply and the work runs in one pass rather than many round trips. It needs a proper sandbox.

Which is better for hallucination, MCP or RAG?

RAG, because it is designed for it. Grounding answers in retrieved passages with citations is the standard control for hallucination on company-specific questions. MCP does not affect answer quality either way; it only governs how the model reaches a system. A poor search tool exposed over MCP will still produce confident wrong answers.

How do I connect company knowledge to Claude or Cursor?

Expose your knowledge base as a remote MCP server and add it as a connector in the client. With QX, add the Vaults MCP URL under Settings then Connectors in Claude, or the MCP config in Cursor, authorise the vaults you want to share, and queries return passages with citations back to the source files.

Want to see grounded, cited retrieval running inside the tools your team already uses? Explore QX Knowledge Vaults or book a demo and we'll connect your documents to Claude or Cursor in the session.

Sources: Anthropic, Code execution with MCP, Linux Foundation, Formation of the Agentic AI Foundation, Jai Juneja, Why AI Company Brains Fail, QX Labs docs, Connect via MCP, WorkOS, Everything your team needs to know about MCP in 2026

MCP vs RAG: How to Give AI Agents Company Context