A process that worked in March produces subtly worse output in September. Nobody changed the code. No release went out. There is no ticket, no incident, no alert. Somewhere upstream, something moved.
Change management rests on an assumption so basic that nobody states it: software stays still unless someone moves it. Conventional software does. Hosted AI does not. Change is an event — initiated by you, passing through a process you control, recorded somewhere you can find it later.
Three things break that assumption, at three different timescales. None of them appear in a change register.
Months: the model moves beneath you.
Hosted models are a category of software dependency the discipline has not caught up with. Behaviour can change without explicit version control or transparency.
A paper published in June documented it in passing, while investigating something else entirely. Re-running an identical protocol against the same pinned model identifier — gpt-4o-2024-08-06, with protocol and endpoint identity verified — the researchers obtained a result that collapsed completely between a May run and a June run. The effect size was large and the statistical significance unambiguous. They concluded that a silent model update was the only plausible explanation.
That it surfaced as an obstacle to unrelated work makes it more persuasive rather than less. Nobody was looking for it. It interrupted something else, and the authors documented why.
The paper is careful to note that one case does not establish a general law, and that caution is worth preserving. However, published failure taxonomies describe the same pattern in general terms: a provider updating or retuning a model such that previously stable workflows change their output format, reasoning style, or tool-call behaviour, with no change whatsoever to the deploying organisation’s code.
Minutes: the system degrades while it works.
Chroma Research evaluated eighteen leading models on simple tasks at increasing context lengths in 2025, naming the result context rot. All eighteen degraded. The degradation is not linear, it tends to arrive as a cliff rather than a slope.
The finding that matters operationally is where the degradation starts. It is not at the edge of the context window, where an engineer would expect it and plan for it. It begins well inside, at lengths a long working session reaches without anyone considering it unusual.
The arithmetic matters more than the mechanism. If degradation takes per-step reliability from near-perfect down to 85%, then ten dependent steps produce a clean end-to-end run roughly one time in five. That is not a claim about AI. It is multiplication, and it explains why long sessions fail in slow motion rather than announcing themselves.
Silently: the constraints thin out.
Agents compact their context as a run grows long, summarising earlier turns to stay within a token budget. Research published this year looked at what that does to the constraints they were given.
With the policy present in context, the tested model never violated it, even across a long uncompressed run. Replace that context with a summary that omits the policy, and violations reached 60%. Restore the policy to the summary, and violations returned to zero. The constraint had not been overwhelmed by length. It had been deleted — by the housekeeping that keeps the agent running.
Put in governance terms: the rules in force at the end of an agent run are not necessarily the rules set at the beginning, and nothing reports the difference. An agent’s authority is defined by its constraints. If the constraints decay, the authority has changed, and no register records it.
None of the three sets off an alarm, and the reason is structural.
Conventional software fails visibly. It works, then it does not, and someone raises a ticket within the hour. These failures do not announce themselves. A process that moves from 94% to 88% accuracy on a dimension nobody measures generates no error. It generates slightly worse decisions, indefinitely, while the people closest to the work adapt around the change without reporting it — because from where they sit, nothing happened.
And the obvious safeguard has the same weakness. Research published this year tested monitoring models on agent transcripts and found their recall degrades as transcripts lengthen. Dangerous actions are missed more often when they follow long stretches of benign context, and detection is worst when the problematic action sits in the middle of a long run.
The control built to catch the problem is subject to the same decay as the thing it watches.
At this point most technical readers will want to say: pin the model version.
They are right that it is the correct first move, and it genuinely closes one of the three gaps. But it is incomplete for reasons the research makes clear. Hosted models have been observed changing behaviour without the version string changing, which is precisely what the June case documents. Pinned versions get retired on the provider’s schedule, not yours — you are given a notice period and then migrated, whether or not the replacement behaves the way your workflow needs. And pinning addresses provider-side change only; it does nothing about context degradation or constraint decay, which occur entirely within your own deployment.
Pinning reduces the frequency of an event you still have to be able to detect. It does not remove the need to detect it.
Which brings the argument to what your validation actually proved. You tested at a point in time, on short inputs, against the model as it then was, with the full constraint set in place. Production runs long contexts on a model that has since moved, with constraints that thin as the session proceeds. The validation has expired, and nothing told you.
The control is not new, and that is the most useful thing about it.
A reference test set with defined expected outputs, replayed whenever the model changes, with results retained. This is regression testing. It has existed for as long as software has. The entire novelty lies in recognising the trigger: a supplier model update is a change event and belongs in the change register.
The research adds three things to the conventional version.
Test at the context lengths production actually uses. A suite run on short inputs validates a condition your system never operates in. Given where accuracy drops, the test material has also to sit in the middle of the context, not at the edges.
Treat session length as a limit. If reliability falls past a certain point, that point is a control parameter, not a curiosity.
Do not rely on a single accuracy number. A regression that is obvious within one category disappears once performance is averaged across a whole suite. A headline figure is not a regression test.
It is worth noting that Bain and BCG have both arrived independently at the same control — a human-calibrated baseline with a continuously maintained evaluation set, and periodic performance review of deployed agents. The idea is not scarce. Its execution is.
Whether this is a real control failure or an inconvenience is not a matter of opinion. The profession settled it some time ago.
The IIA’s guidance on IT change management holds that the only acceptable number of unauthorised changes is zero.
Apply that to a hosted model. The provider update altered production behaviour. Nobody authorised it. Nobody tested it. Nothing recorded it.
This is not a new weakness for a well-prepared organisation. Change management is already among the most common areas for control failure in assurance work: undocumented processes, informal approvals and inadequate testing. That weakness has now met a dependency that modifies itself, on someone else’s schedule.
Responsibility is unambiguous even where ownership is not. The provider has no obligation to preserve your workflow’s behaviour across versions, and does not claim to. The organisation running the process carries the risk, whether or not anyone has been assigned it.
In most organisations the AI initiative has an owner. The model dependency does not.
What makes this enforceable is a standard that applies to everyone: no AI-supported process in production without a reference test set, a replay procedure on model change, and a tested limit on session length. It applies to what a central function builds, what a business unit builds, and what a vendor delivers. Its neutrality is the point — it surfaces weak practice through evidence rather than accusation, which matters where this conversation has already turned political.
For an internal audit function, none of this requires new methodology. It is IT change management applied to AI-supported processes. Which makes it one of the few available handles on AI assurance today.
Every organisation running AI in production has a change log. The changes that matter most are not in it.
Sources: silent model drift, arXiv:2606.29719, section 5.3. Context degradation, Chroma Research, Context Rot (2025). Constraint decay, Gamage, arXiv:2604.20911, and arXiv:2606.22528 on context compaction. Change management standards from the IIA’s Global Technology Audit Guide on IT Change Management.
Alexandre Castaing is Managing Director of Axon Advisory & Consulting, working with supervised financial entities in Luxembourg on digital resilience, AI governance and regulatory compliance. Articles are AI-assisted, human-directed, and source-verified.
#AIGovernance #InternalAudit #ChangeManagement #AIAssurance #RiskManagement #ITGC