Agents running on prompts and retrieval break down around two hundred rules or less if those rules span several systems or when rules conflict.
There is no definitive limit to the number of rules an agent can successfully read and operate on, but every agent will hit a limit where performance drops. We find agents tend to break down when given 200 or more instructions.
Instruction-following accuracy as instructions rise from 10 to 500, for six of the 20 models tested. Source: Distyl AI, IFScale (arXiv, July 15, 2025), Figure 1. The benchmark tested older models, so the breaking point may be closer to 500 instructions with newer models. However, this data does not consider other typical business complexities.
Why agents break down past the threshold
Inside the prompt
IFScale (Distyl AI, arXiv preprint, July 15, 2025) tested 20 models on 10 to 500 instructions.
As the chart above shows, the strongest reasoning models declined after about 150 to 200 instructions, and even the best model reached 68% accuracy at 500.
The paper sorts the models into three patterns of decline, all visible in that chart.
The kind of error changed as instructions piled up. When only a few instructions are given at once (low instruction density), models either modified the instruction or omitted a rule entirely. As density rose, omission became the dominant error, so rules dropped silently with nothing to flag the gap.
Position mattered too. Later instructions were dropped more often than earlier ones, and that bias peaked around 150 to 200 instructions. At very high densities, errors spread evenly, so putting important rules first stopped helping.
Reasoning models held up longer but slowed down sharply. In the same study, o4-mini went from 12.40 seconds at 10 instructions to 436.19 seconds at 250.
It's important to understand that the IFScale benchmark only measures a small piece of the complexity agents face in a real business environment. It doesn't measure how agents handle rules scattered across teams and systems, or how agents resolve outdated and conflicting rules.
Tools show a similar pattern. Drew Breunig (June 22, 2025) reports a quantized Llama 3.1 8b that failed with 46 tools and succeeded with 19. Anthropic Engineering (September 29, 2025) calls context "a finite resource with diminishing marginal returns."
In the context itself
Breunig's June 22, 2025 post names four ways context fails.
We covered clash in our earlier post.
Chroma's "Context Rot" report (July 14, 2025) tested 18 LLMs and found performance fell as input grew, in every experiment. Even a single distractor passage lowered performance against the baseline.
Anthropic's "Components of agent memory" (November 6, 2026) found that an incorrect note from the start could be read and trusted in a later session. When agents overwrote each other's shared notes, the output kept only 41 of 57 facts (72%).
The piece does not discuss access control, retention or human review, which a governed graph has to handle.
These problems grow as an agent takes on more rules, and an agent whose rules span several systems runs into issues first. A wrong or conflicting rule then gets applied with a confident explanation and raises no error, so standard monitoring will not flag it.
In retrieval
IFScale covers instructions in a prompt. For passages from retrieval, the best evidence is Jin et al., an arXiv preprint by Google Cloud AI Research and UIUC authors (October 8, 2024). Output quality first improved and then declined as more passages were retrieved.
The authors trace much of the decline to hard negatives, passages that look relevant but do not answer the question. Hard negatives from stronger retrievers were harder for the models, and the decline could be more severe. The study used older, smaller models on general question sets, not enterprise rules.
Business rules make this worse, because retrieval ranks by similarity rather than by authority. A stronger retriever surfaces more relevant exceptions and so hands the model more conflicts to settle alone.
Anthropic's fixes for long context (compaction, structured notes, sub-agents and just-in-time retrieval) help with length, but none of them decides which rule wins.
That 200 figure falls sharply when the rules come from many systems, because more of them can apply to the same case and disagree. An agent also fails at any count once its rules lose owners, review dates or a stated winner in a conflict.
That gives two kinds of agent. A single-team agent works inside one team, with a few dozen rules set by one owner, and people catch its mistakes. Prompts and retrieval usually serve it well.
A cross-system agent takes rules from several teams and systems, and some of those rules disagree with no stated winner. That agent needs a context graph, which stores which rule outranks which and why.
Saturation is the point where new rules stop improving decisions. A cross-system agent reaches it much sooner, and starting a graph before then costs less than patching prompts later.
You need a context graph if
Your agent works with an ERP or CRM
An ERP or CRM is where rules, exceptions and approvals from several teams meet. Once an agent drafts quotes in the CRM, rules from sales, finance and the deal desk all apply to the same deal.
Take a quoting agent at a company whose policy says any discount above 15% needs finance approval. Last quarter, a renewal promotion let reps give up to 25% without approval. The promotion has ended, but its playbook still sits in the sales team's shared drive.
A customer up for renewal asks for 20%. Retrieval returns both the discount policy and the promotion playbook, because both closely match the question. Nothing in either passage says the promotion ended, so the model picks one on its own.
Our post on the issues with bigger context windows talks about the importance of rule precedence. Rule precedence is the stored record of which rule outranks which, under what condition, and is set by the rule's owner.
Trail, the company brain for AI, is the context layer your agents run on, and it stores that decision. The promotion is a node with an end date and an override edge to the 15% policy. Because the promotion has ended, Trail applies the 15% policy, routes the 20% request to finance, and explains that the promotion is over.
You are migrating an ERP or another major system such as a CRM
Business rules and exceptions live inside the old system, in its customizations, and in the heads of the people who configured it. A migration is when they either get written down or lost.
Major SAP migrations are coming in the next three years. Mainstream maintenance for SAP Business Suite 7 (ECC) ends in 2027, with paid extended maintenance through 2030.
Additionally, Dynamics GP technical support ends in 2029, and new subscriptions stopped after April 1, 2026.
Whether companies continue using these major ERPs or move on, preserving context will be a major issue to solve for.
A migration that copies old processes also carries forward stale rules and drops exceptions nobody wrote down. The cheapest time to record which rule wins is during the migration.
You are switching AI models or AI vendors
Rules tuned inside old prompts do not carry over to a new model, so each swap means rebuilding and retesting them. A graph you own holds the rules outside any one model, which keeps your agents portable between models and vendors.
A merger raises the same need. Two companies bring two sets of rules to one agent, and nobody has decided which set governs.
Test your own agents
Regardless of rule count, check your agents against these five signals.
Next, sort your last 20 to 30 agent errors into two piles, lacked a rule and applied the wrong one. If most land in the second pile, the agent already had the rules it needed, so more rules will not fix those errors.
To find where your own agent saturates, add rules in steps against fixed test cases and watch where accuracy flattens or falls.
Pushbacks and honest answers
"It's too early for a context graph." Starting with one process is the best way to implement a context graph, and it is much easier to add new teams and workflows after. We recommend starting with the most complex process you're considering for an agent.
"Our prompt works." Prompts often hold up for a single-team agent and drift once it becomes cross-system.
"RAG is enough." Retrieval finds rules but does not pick between them when they disagree.
"Longer context windows will fix it." Chroma and Breunig both show quality falling as context grows, across models. Bigger windows move the drop later without removing it.
"It is another system to maintain." Fair, but the graph replaces patches across prompts and wikis. Trail's self-learning proposes updates and brings in a person when a call needs one, and that person approves any change to how decisions get made.
"We will wait until the migration is finished." After go-live, the old system and the reasons behind its customizations are gone.
Why now
Many teams spent 2025 on single-team agents, and in 2026 more agents are becoming cross-system. The ERP dates above put migrations on the calendar for 2027 and 2029.
Article 12 of the EU AI Act requires high-risk AI systems to allow automatic event logging, which makes an unexplained agent decision harder to defend.
A 15 minute readiness check
Three questions will tell you where you stand:
- Sort your last 20 to 30 agent errors into "lacked a rule" and "had the rules, applied the wrong one." Which pile is bigger?
- Two of your rules both apply to the same deal or invoice and they disagree. What does your system return?
- Who resolves rule conflicts today, and is their reasoning written down?
If any of the five signals holds, or your agent works with an ERP or CRM, or you face a migration or a model switch, the graph should come next. If you still run single-team agents, start now, while the rules are few enough to give each an owner, a review date and a precedence.

