Governance Decay, Explained: How Context Compaction Erodes Agent Policy — and Where Enforcement Belongs

A new term just gave enterprise AI's quietest failure mode a name: governance decay — the silent erosion of an agent's in-context rules as its history is compressed. The coinage is a June 2026 research paper, "Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents" (arXiv:2606.22528), and its opening scenario deserves to be famous. An agent is instructed never to email a contract outside the organization; for dozens of turns it complies, refusing the request correctly. Then its history outgrows the context window and compaction runs — the routine, LLM-based summarization step every long-horizon harness now performs, engineered to preserve task continuity. The summary faithfully records the task state and, optimizing for continuity, quietly drops the "old" compliance preamble. Asked again, the agent emails the contract. No jailbreak, no model change, no new request: the rule the agent was obeying is simply no longer in front of it. The paper's findings sharpen the alarm: the decay is a property of the harness, not the model — stronger models fall too; it is weaponizable, since an adversary who can place content in context can accelerate the forgetting; the proposed defense, constraint pinning, helps but is defeated by operator impersonation, which the authors note requires "a trusted out-of-band operator channel" to close. And the conclusion arrives in seven words we quote once: "Governing agents requires governing how they forget." This guide is the enterprise reference the paper implies but does not write: what governance decay is, why compaction made it inevitable, how it differs from the failure modes it resembles — and the architectural answer, which is that the policies that matter must live where compaction cannot reach them: outside the context window, enforced at the layer every call already crosses. As with all our young-term references, we will keep this page updated as the research and vocabulary evolve.

What Is Governance Decay? A Definition

Governance decay is the progressive, silent loss of an agent's in-context governance — the organizational policies, standing instructions, and memory-loaded constraints that specify what a deployed agent must and must not do — as the agent's conversation history is compressed to fit its context window. The mechanism is mundane, which is what makes it dangerous. Long-horizon agents accumulate history faster than windows grow; the dominant engineering response is context compaction — an LLM-based summarization or eviction pass that periodically compresses the transcript — and compaction has been engineered for exactly one objective: preserving task accuracy. A standing policy is, from the summarizer's perspective, old text that is not the current sub-goal, competing for a shrinking token budget against active task state; so it is summarized away, softened, or evicted — and the agent, whose behavior was only ever as good as what was in front of it, reverts to ungoverned defaults without any signal that anything changed. The research adds two properties that elevate this from bug to category. First, a soft/hard gradient: models' trained-in hard norms (the things safety benchmarks probe) survive, while the soft, deployment-specific rules — your rules, the operator-specified ones — are precisely what erodes, which is why the failure went unnoticed by standard evaluations. Second, adversarial availability: an attacker who can place content in the agent's context (the standard indirect-injection assumption) can deliberately bloat and steer the history so compaction erases the constraint standing in their way. A disambiguation: governance decay is not model drift (the weights are unchanged), not context rot (retrieval quality degrading with length), and not forgetting in the ML sense — it is a harness-level information loss with governance consequences, and it happens by design, every time compaction runs.

A Brief History: How Compaction Became a Governance Surface

The term is new; the collision it names was building for two years. 2024–2025: agents go long-horizon — multi-step tool use, day-long runs, persistent sessions — and histories outgrow even frontier windows, making compression a standard harness feature rather than an optimization; the practitioner ecosystem converges on threshold-triggered summarization, with compaction firing at 70–90% of window in mainstream frameworks and, in aggressive configurations, at a few thousand tokens. In parallel, governance moves into the context: the era's deployable control surface is in-context — system preambles, standing instructions, AGENTS.md-style operating rules, loaded memory — because it is the layer operators can actually specify without retraining. 2025–2026: the research community starts treating context management as a first-class agent subsystem — structured eviction schemes, context-folding with branch-and-return, KV-cache-aware compression — all still optimizing for task fidelity. June 2026: arXiv 2606.22528 names the collision — the governance channel and the compression step were pointed at each other all along — demonstrates the decay empirically across models, shows the adversarial variant, proposes constraint pinning (quarantine the extractable rules so the summarizer must carry them forward), and candidly lists pinning's limits: implicit constraints can't be pinned, and pinning is defeated by operator impersonation in recent context, closing which "needs a trusted out-of-band operator channel." The term is useful less as a name than as a diagnosis the enterprise can act on for incidents it has already had and could not explain, the agent that "just stopped following the rule" with nothing in the logs but a routine compaction.

Governance Decay vs Prompt Injection, Context Rot, and Model Drift

Three distinctions keep the term precise, and the first is the one that matters operationally. Versus prompt injection: injection is an addition attack — hostile instructions smuggled into context, the threat our injection-defense analysis covers; decay is a subtraction failure — protective instructions silently removed by the system's own maintenance. They compose viciously: the paper's adversarial results show injected content steering compaction to erase defenses, meaning a hardened prompt is not a durable defense if the hardening itself can be compacted away. Versus context rot: rot names quality degradation as context grows — retrieval misses, attention dilution, the model getting worse at using what is there; decay names the governance-specific case where the content is no longer there at all, and the distinction matters because rot's mitigations (better retrieval, longer windows) do not touch decay, while decay's mitigation (move the rule out of context) is unavailable for rot. Versus model drift: drift is behavior change from changed weights or versions; decay occurs with the model frozen — which is exactly why it evades the monitoring built for drift, and why the paper's harness-not-model finding is the practical headline: you cannot upgrade your way out, and you cannot benchmark your way into noticing, because the eroding rules are the deployment-specific ones benchmarks never contain. The common thread across all three contrasts: governance decay is invisible to every watchtower pointed at the model, because it is not a model behavior — it is an information-lifecycle behavior, and the lifecycle belongs to the harness.

Governance Decay in the Enterprise: Policy That Cannot Be Compacted

Here is where TrueFoundry enters, and for once the vendor thesis is simply the paper's conclusion with an architecture attached. If governing agents requires governing how they forget, the enterprise has two options: make the forgetting safe (pin constraints, audit summaries, harden the summarizer — necessary, fragile, and per the authors' own limitations, incomplete), or make the constraints unforgettable — locate them where the forgetting mechanism cannot reach. The second option has a name: infrastructure. A guardrail at the configured hooks — LLM input/output, and MCP pre/post-tool for matching traffic — evaluates the data-boundary rule ("no contracts to external addresses") on the actual outbound action, at turn 3 and at turn 3,000, identically, because the rule is attached to the request path, not the prompt; there is nothing in the transcript for compaction to drop. The same relocation applies down the control list: tool reach is scoped by identity and task at the MCP layer rather than by in-context pleading (TBAC); spend is bounded by budgets the agent cannot summarize away; and the "trusted out-of-band operator channel" the researchers call for is precisely what a gateway is — operators change policy on the plane, the change applies to every request from that moment, and no agent's context is involved, so operator impersonation inside a transcript moves nothing. Two supporting practices complete the posture. Compaction itself becomes an observed event: harness step-traces record when history was compressed, which is the difference between "the agent inexplicably changed behavior" and "behavior changed around a compaction" — a diagnosis that may require application-level instrumentation or transcript comparison, since current documentation exposes context growth in traces but no dedicated compaction-event marker. And in-context governance gets demoted, not abolished: it remains the right channel for steering — tone, preferences, task framing, the things that should evolve with the conversation — while everything with a compliance or blast-radius character migrates to the plane, the division of labor our context-engineering and just-in-time context work develops from the construction side. The one-sentence enterprise translation of the research: your agent's memory is a lossy channel, so stop shipping your compliance posture through it.

Enterprise Governance-Decay Checklist

Six questions for any long-horizon agent estate: Do your agents compact — and do you know each harness's trigger threshold and summarizer? Is every compliance-grade rule (data boundaries, tool scope, spend, approvals) enforced at the infrastructure layer rather than carried in context? Are compaction events recorded in the run's trace, so behavior changes can be correlated with history compression? Do guardrails evaluate outbound actions — especially tool calls — independently of whatever the transcript currently says? Can operators change policy through the plane, out-of-band, without any agent's context being touched or trusted? And has someone red-teamed the subtraction path — not "can an attacker inject an instruction" but "can an attacker make the system forget one"? Six yeses and decay is a managed property; the noes are rules with expiration dates nobody set.

The Future of Governance Decay: What We'll Be Watching

Predictions, lightly held. Expect the term to cross from research to practitioner discourse within quarters, on the strength of its explanatory power: it names incidents teams have already experienced — the agent that abandoned a rule mid-session with clean logs — and terms that explain existing pain travel fastest, the pattern that carried context rot and tool poisoning before it. Expect compaction to professionalize in response: governance-aware summarizers, constraint-pinning options in mainstream harnesses, compaction events as first-class telemetry — all worth adopting, none sufficient alone, per the original paper's own limitations. Expect the adversarial variant to make the term mainstream the hard way: subtraction attacks are a natural next move for the injection ecosystem, and the first public incident where an attacker compacted away a control will do for governance decay what the poisoning disclosures did for MCP security. Expect regulators and auditors to catch the implication on their usual lag — "show me the controls that survive context compression" is a question we would bet appears in AI audits within the year, and it is answerable only by architecture. And expect the deepest effect to be a redrawn map: the era's default — govern the agent by writing rules into its head — gets replaced by a two-tier doctrine, steering in context and survival on the plane, which has been this blog's standing argument and now has a failure mode, a paper, and a name to enforce it. We will keep this guide updated as the research and vocabulary evolve; the durable claim is the paper's, and it deserves the last word it earned: governing agents requires governing how they forget — and the strongest way to govern forgetting is to keep the things that must not be forgotten somewhere forgetting cannot reach.

Governance Decay FAQ

What is governance decay in AI agents?

Governance decay is the silent erosion of an agent's in-context rules — organizational policies, standing instructions, memory-loaded constraints — as context compaction compresses the agent's history. Summarization optimized for task continuity drops "old" rules that aren't the current sub-goal, and the agent reverts to ungoverned behavior with no jailbreak, no model change, and no obvious signal. The term comes from a June 2026 research paper (arXiv:2606.22528).

How is governance decay different from prompt injection?

Prompt injection adds hostile instructions to an agent's context; governance decay removes protective ones through the system's own maintenance. They combine: an adversary who can place content in context can steer compaction to erase the constraints that would have stopped them — meaning hardened prompts are not durable defenses if the hardening itself can be compacted away.

Can better models or longer context windows fix governance decay?

No. The research finds decay is a property of the harness, not the model — stronger models fall too, because the failure is in what the summarizer preserves, not what the model can do. Longer windows delay compaction, and agents whose harnesses use lossy summarization or eviction can decay when deployer rules live only in compactable content — deployment-specific rules being exactly what standard benchmarks never test.

How do enterprises prevent governance decay?

Relocate survival-grade rules out of the context window: enforce data boundaries, tool scope, spend, and approvals as infrastructure — guardrails on every model and tool call, task-based access control, budgets on the plane — where compaction cannot reach them; monitor context growth in traces (correlating behavior changes with compaction may need application-level instrumentation); and keep in-context governance for steering (tone, preferences, framing) that is safe to lose.

References

• Anchor — "Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents" (arXiv:2606.22528, June 2026: the coinage; the contract-email scenario; the harness-not-model finding; the soft/hard gradient; the adversarial variant; constraint pinning and its stated limitations, including the out-of-band operator channel; the quoted conclusion). Context-management background: "Beyond Compaction: Structured Context Eviction for Long-Horizon Agents" (compaction thresholds; context-folding and eviction approaches).
• TrueFoundry documentation — guardrails (four hooks; official flow diagram); budgets; analytics (official image).
• This blog — prompt-injection defense at the gateway; TBAC: task-based access control; context engineering and session management; just-in-time context; loop engineering at enterprise grade.

One direct quotation under fifteen words is used, from the cited paper's conclusion, attributed inline; the paper's scenario, findings, defense, and limitations are otherwise paraphrased, and readers should consult the original for the full experimental treatment. The relocation thesis — survival rules as infrastructure, steering rules in context — is TrueFoundry editorial carrying the paper's conclusion to an architecture; the paper's authors neither mention nor endorse TrueFoundry. Product capabilities are paraphrased from public documentation current at the time of writing; the product images are TrueFoundry's own documentation assets, reproduced with attribution. This is a living reference and will be revised as the research and vocabulary evolve.

‍