Governance Decay, Explained: How Context Compaction Erodes Agent Policy — and Where Enforcement Belongs

A new term just gave enterprise AI's quietest failure mode a name: governance decay — the silent erosion of an agent's in-context rules as its history is compressed. The coinage is a June 2026 research paper, "Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents" (arXiv:2606.22528), and its opening scenario deserves to be famous. An agent is instructed never to email a contract outside the organization; for dozens of turns it complies, refusing the request correctly. Then its history outgrows the context window and compaction runs — the routine, LLM-based summarization step every long-horizon harness now performs, engineered to preserve task continuity. The summary faithfully records the task state and, optimizing for continuity, quietly drops the "old" compliance preamble. Asked again, the agent emails the contract. No jailbreak, no model change, no new request: the rule the agent was obeying is simply no longer in front of it. The paper's findings sharpen the alarm: the decay is a property of the harness, not the model — stronger models fall too; it is weaponizable, since an adversary who can place content in context can accelerate the forgetting; the proposed defense, constraint pinning, helps but is defeated by operator impersonation, which the authors note requires "a trusted out-of-band operator channel" to close. And the conclusion arrives in seven words we quote once: "Governing agents requires governing how they forget." This guide is the enterprise reference the paper implies but does not write: what governance decay is, why compaction made it inevitable, how it differs from the failure modes it resembles — and the architectural answer, which is that the policies that matter must live where compaction cannot reach them: outside the context window, enforced at the layer every call already crosses. As with all our young-term references, we will keep this page updated as the research and vocabulary evolve.

What Is Governance Decay? A Definition

Governance decay is the progressive, silent loss of an agent's in-context governance — the organizational policies, standing instructions, and memory-loaded constraints that specify what a deployed agent must and must not do — as the agent's conversation history is compressed to fit its context window. The mechanism is mundane, which is what makes it dangerous. Long-horizon agents accumulate history faster than windows grow; the dominant engineering response is context compaction — an LLM-based summarization or eviction pass that periodically compresses the transcript — and compaction has been engineered for exactly one objective: preserving task accuracy. A standing policy is, from the summarizer's perspective, old text that is not the current sub-goal, competing for a shrinking token budget against active task state; so it is summarized away, softened, or evicted — and the agent, whose behavior was only ever as good as what was in front of it, reverts to ungoverned defaults without any signal that anything changed. The research adds two properties that elevate this from bug to category. First, a soft/hard gradient: models' trained-in hard norms (the things safety benchmarks probe) survive, while the soft, deployment-specific rules — your rules, the operator-specified ones — are precisely what erodes, which is why the failure went unnoticed by standard evaluations. Second, adversarial availability: an attacker who can place content in the agent's context (the standard indirect-injection assumption) can deliberately bloat and steer the history so compaction erases the constraint standing in their way. A disambiguation: governance decay is not model drift (the weights are unchanged), not context rot (retrieval quality degrading with length), and not forgetting in the ML sense — it is a harness-level information loss with governance consequences, and it happens by design, every time compaction runs.

A Brief History: How Compaction Became a Governance Surface

The term is new; the collision it names was building for two years. 2024–2025: agents go long-horizon — multi-step tool use, day-long runs, persistent sessions — and histories outgrow even frontier windows, making compression a standard harness feature rather than an optimization; the practitioner ecosystem converges on threshold-triggered summarization, with compaction firing at 70–90% of window in mainstream frameworks and, in aggressive configurations, at a few thousand tokens. In parallel, governance moves into the context: the era's deployable control surface is in-context — system preambles, standing instructions, AGENTS.md-style operating rules, loaded memory — because it is the layer operators can actually specify without retraining. 2025–2026: the research community starts treating context management as a first-class agent subsystem — structured eviction schemes, context-folding with branch-and-return, KV-cache-aware compression — all still optimizing for task fidelity. June 2026: arXiv 2606.22528 names the collision — the governance channel and the compression step were pointed at each other all along — demonstrates the decay empirically across models, shows the adversarial variant, proposes constraint pinning (quarantine the extractable rules so the summarizer must carry them forward), and candidly lists pinning's limits: implicit constraints can't be pinned, and pinning is defeated by operator impersonation in recent context, closing which "needs a trusted out-of-band operator channel." The term is useful less as a name than as a diagnosis the enterprise can act on for incidents it has already had and could not explain, the agent that "just stopped following the rule" with nothing in the logs but a routine compaction.

Governance Decay vs Prompt Injection, Context Rot, and Model Drift

Three distinctions keep the term precise, and the first is the one that matters operationally. Versus prompt injection: injection is an addition attack — hostile instructions smuggled into context, the threat our injection-defense analysis covers; decay is a subtraction failure — protective instructions silently removed by the system's own maintenance. They compose viciously: the paper's adversarial results show injected content steering compaction to erase defenses, meaning a hardened prompt is not a durable defense if the hardening itself can be compacted away. Versus context rot: rot names quality degradation as context grows — retrieval misses, attention dilution, the model getting worse at using what is there; decay names the governance-specific case where the content is no longer there at all, and the distinction matters because rot's mitigations (better retrieval, longer windows) do not touch decay, while decay's mitigation (move the rule out of context) is unavailable for rot. Versus model drift: drift is behavior change from changed weights or versions; decay occurs with the model frozen — which is exactly why it evades the monitoring built for drift, and why the paper's harness-not-model finding is the practical headline: you cannot upgrade your way out, and you cannot benchmark your way into noticing, because the eroding rules are the deployment-specific ones benchmarks never contain. The common thread across all three contrasts: governance decay is invisible to every watchtower pointed at the model, because it is not a model behavior — it is an information-lifecycle behavior, and the lifecycle belongs to the harness.

Governance Decay in the Enterprise: Policy That Cannot Be Compacted

Here is where TrueFoundry enters, and for once the vendor thesis is simply the paper's conclusion with an architecture attached. If governing agents requires governing how they forget, the enterprise has two options: make the forgetting safe (pin constraints, audit summaries, harden the summarizer — necessary, fragile, and per the authors' own limitations, incomplete), or make the constraints unforgettable — locate them where the forgetting mechanism cannot reach. The second option has a name: infrastructure. A guardrail at the configured hooks — LLM input/output, and MCP pre/post-tool for matching traffic — evaluates the data-boundary rule ("no contracts to external addresses") on the actual outbound action, at turn 3 and at turn 3,000, identically, because the rule is attached to the request path, not the prompt; there is nothing in the transcript for compaction to drop. The same relocation applies down the control list: tool reach is scoped by identity and task at the MCP layer rather than by in-context pleading (TBAC); spend is bounded by budgets the agent cannot summarize away; and the "trusted out-of-band operator channel" the researchers call for is precisely what a gateway is — operators change policy on the plane, the change applies to every request from that moment, and no agent's context is involved, so operator impersonation inside a transcript moves nothing. Two supporting practices complete the posture. Compaction itself becomes an observed event: harness step-traces record when history was compressed, which is the difference between "the agent inexplicably changed behavior" and "behavior changed around a compaction" — a diagnosis that may require application-level instrumentation or transcript comparison, since current documentation exposes context growth in traces but no dedicated compaction-event marker. And in-context governance gets demoted, not abolished: it remains the right channel for steering — tone, preferences, task framing, the things that should evolve with the conversation — while everything with a compliance or blast-radius character migrates to the plane, the division of labor our context-engineering and just-in-time context work develops from the construction side. The one-sentence enterprise translation of the research: your agent's memory is a lossy channel, so stop shipping your compliance posture through it.

‍