The model is the commodity. The map is the moat.

Most organizations approaching AI agents ask which model to use. It is the wrong first question, and the answer keeps changing anyway.

Here is the more useful framing, which I picked up from Eran Yahav of Tabnine and have not been able to unsee since. An enterprise agent stack has three parts, not one:

• The LLM — powerful, general, and completely ignorant of your organization. It has never seen your systems, your history, your incidents, or the reason that one service is named after someone's dog.

• The agent — orchestration, tool use, interaction with humans.

• The context engine — a persistent, continuously maintained map of the organization itself.

Almost everyone invests in the first two and improvises the third. That is why agents fail most complex enterprise tasks — and the failures are usually not reasoning failures. They are onboarding failures. The agent lacks the knowledge a human engineer accumulates in their first few months and then never thinks about again.

The blast radius example

Give an agent a newly disclosed vulnerability and ask what it affects.

Without a map, it has to crawl the estate to find out. In a large organization it will not finish, and it will not be right — not because it reasons badly, but because the search space exceeds any single agent's visibility and reasoning horizon.

With a precomputed dependency map, the same agent eliminates the vast majority of the estate immediately and concentrates on the handful of systems that matter. The reported effect of adding that layer: roughly double the success rate, with up to eighty percent less token spend.

Same model. Same reasoning. Different starting knowledge.

Precomputation beats retrieval-on-demand

The architectural move here is a division of labour between agents.

Context agents do the homework, continuously. They crawl code, design documents, incident history, telemetry and dependencies, and maintain a representation that is exhaustively current. This work has its own cadence and its own cost profile, and it happens whether or not anyone is asking a question.

Task agents consume that map. They do not rebuild it per request.

The mistake worth avoiding is conflating the two — systems that discover context at query time rediscover the same context endlessly, paying for it every time.

Where retrieval fits, and where it stops

This is memory architecture, and it is worth being precise about the layers. Short-term memory is the transient reasoning state in the context window. Long-term memory is durable knowledge: outcomes, learned strategies, the map itself. Working memory bridges them for the current cycle.

A context engine is institutional long-term memory, externalized and shared across agents. That is how it sidesteps the finite context window: knowledge becomes queryable rather than resident.

Retrieval-augmented generation is the mechanism — a retriever, a knowledge store, a generator that fuses retrieved context into the prompt at inference time. Unlike fine-tuning, it needs no retraining when facts change, because the weights never move.

But RAG on its own is retrieval without a map, and it comes with an uncomfortable property: retrieval yields evidence, not necessarily good evidence, and generation yields answers, not necessarily faithful ones. Which is why a serious pipeline adds a judgment layer:

• Relevancy grading — discard retrieved chunks that do not actually bear on the question.

• Groundedness checking — verify every claim against the retrieved documents.

• Source attribution — trace each claim back to the verbatim segment it came from.

Those three are the difference between a demo and a system whose answers can be defended to someone who has to sign off on them.

Why this reframing changes decisions

If the map is the moat, then a knowledge graph over your systems is not a better retrieval widget. It is infrastructure — an asset that makes every future agent cheaper and more accurate, and that compounds instead of resetting with each model release.

It also has a property that matters to any organization with data-protection or resilience obligations: the map can live entirely inside your own boundary. You get agent capability without exporting your estate to someone else.

And there is a sting in the tail, for anyone who has read about Conway's Law. A map that spans the whole estate needs a team mandated across the whole estate. Whoever owns the context engine sets a ceiling on how complete it can ever be.

The model, meanwhile, you can rent from whoever is winning this month.

Why it matters

For anyone passionate about building AI systems that work at organizational scale, this reframes where the effort should go.

• Stop shopping for models, start building the map. Model capability is converging and rented; the precomputed map of an organization's systems, dependencies and history is the durable asset. It is also the thing that makes every future agent cheaper and more accurate — compounding, rather than resetting with each model release.

• A knowledge graph is a context engine, not "better RAG." Framing graph work as a context engine changes the business case entirely: it is infrastructure other capabilities depend on, not an incremental retrieval improvement.

• Separate enrichment from reasoning. The context-agent / task-agent split is a genuine architectural principle: continuous background enrichment of the map is a different concern, with a different cadence and cost profile, from query-time reasoning. Systems that conflate the two rediscover the same context endlessly.

• Keeping the map inside the perimeter is a real advantage. For any organization with data-protection or resilience obligations, a context engine that stays within its own boundary delivers agent capability without exporting the estate to a third party.

• The reliability layer is what makes output defensible. Relevancy grading, groundedness checking and source attribution are the difference between a demo and something whose answers can be audited. Retrieval yields evidence, not good evidence; generation yields answers, not faithful ones.

• Conway's Law applies here too. A map spanning the whole estate needs a team mandated across the whole estate. Whoever owns the context engine caps how complete it can ever be.

Do you use context agents, knowledge graphs,…? What’s the impact of Google new Open Knowledge Format (OKF) into the organisation knowledge organisation and context agents? I am curios to hear your experience. 

Sources

• Why Enterprise AI Fails Without a Context Engine — Eran Yahav (Tabnine) on the AI in Business podcast; link goes to Emerj's written analysis of the episode

• Agentic Artificial Intelligence — Pascal Bornet, Jochen Wirtz, Thomas H. Davenport et al.

• RAG Made Simple — Nir Diamant

• AI Prompt Engineering — Nelson Ming

• The Age of Agentic AI — One Book Ahead Publishing

Flashcards

These are the cards I use for spaced repetition on this material. The whole Second Brain deck is public on AnkiWeb if you want it.

Q1: What are the three components of the enterprise agent stack (Yahav)?

The LLM (reasoning), the agent (orchestration and human interaction), and the context engine (organizational knowledge). All three are required for organizational intelligence.

Q2: Why do AI agents fail most complex enterprise tasks?

Not from weak reasoning — they lack the organization-specific onboarding knowledge a human engineer accumulates over months.

Q3: What is a context engine?

A persistent, precomputed map of the organization — built by crawling code, design docs, incidents, telemetry and dependencies — that agents query instead of rediscovering context each time.

Q4: What is the difference between a context agent and a task agent?

Context agents continuously crawl the organization and maintain the dependency map ("do the homework"); task agents consume that map to execute specific requests.

Q5: Illustrate the value of precomputed context with the blast-radius example.

Without context, an agent must crawl the whole estate to find what a vulnerability affects and stands no chance at scale. With the map, it eliminates irrelevant areas instantly and focuses on the few relevant systems.

Q6: What are the reported effects of adding a context engine?

Roughly double the agent success rate and up to ~80% reduction in token cost — while keeping the knowledge inside the enterprise security perimeter.

Q7: Why is precomputation the key architectural insight, rather than better retrieval?

Dependencies and relationships are built once and reused; rediscovering them per request exceeds the visibility and reasoning horizon of any single agent at enterprise scale.

Q8: What are the three layers of agent memory?

Short-term (transient reasoning state, context window), long-term (durable knowledge, outcomes, learned strategies), and working memory bridging the two for the current reasoning cycle.

Q9: How does an externalized knowledge store overcome the context-window constraint?

It makes indexed knowledge queryable rather than resident — the agent retrieves what it needs instead of holding everything in finite internal memory.

Q10: What are the three components of a RAG system?

The retriever (finds relevant content — keyword, semantic, or hybrid), the knowledge store (memory substrate: documents, embeddings, structured or hybrid), and the generator (fuses retrieved context with the prompt at inference).

Q11: Why is RAG alone insufficient for enterprise reliability?

Retrieval gives evidence but not all evidence is good; generation gives answers but not all answers are faithful. A judgment layer is needed on top.

Q12: What three checkpoints make a RAG pipeline reliable?

Relevancy grading (discard irrelevant retrieved chunks), hallucination checking (verify every claim is grounded in the retrieved documents), and source highlighting (trace each claim to verbatim source segments).

Q13: How does RAG differ from fine-tuning?

RAG supplies external context at inference time without changing model weights, so fresh information needs no retraining; fine-tuning adds data to the training set and adjusts the weights.

Q14: Why is a context engine a regulatory advantage for a financial institution?

It can be kept entirely inside the security perimeter — delivering agent capability without exporting the enterprise estate to an external provider.

Q15: What does Conway's Law imply about who should own the context engine?

A map spanning the whole estate requires a team mandated across the whole estate — the owner's organizational scope caps how complete the map can be.