Over 1,000 enterprise AI professionals subscribed to this newsletter after the first issue. The pattern in their responses confirmed what the data already showed: the AI delivery discipline gap is not theoretical. It is operational. This issue diagnoses the most recognisable symptom.
The most dangerous AI pilot is the one that works perfectly.
Twelve weeks ago, a mid-tier insurer approved the production rollout of an agentic claims triage system. The leadership team had seen it work. The pilot processed 800 sample claims in a controlled environment and routed them with 94% accuracy. The business case projected $2.4 million in annual savings. The CTO called it the most successful AI initiative in the company's history. Six weeks later, they quietly pulled it from production.
The CTO was not negligent. The evaluation was thorough by every standard available. The problem was that every standard available was designed for a different category of system.
In production, the system started routing identical claim types differently on different days. Not because the data changed. Because the system is probabilistic. Input phrasing, context window state, model updates, and interaction between agents within the multi-agent architecture all influenced outputs in ways that no deterministic test could predict. Confidence scores remained high while output quality quietly degraded. The system did not crash. It did not throw errors. It improvised on ambiguous inputs rather than flagging them. It was subtly, confidently wrong.
Within six weeks, the system was silently misrouting 12% of complex claims. The dashboard still showed performance in the low 90s because it was measuring against the same categories used in the pilot. The 12% were invisible. They only surfaced when a regulator queried response times on a specific claim category and the operations team traced the problem back to the system.
The cost of rework, customer complaints, regulatory correspondence, and emergency re-engineering exceeded the projected savings for the entire first year. The $2.4 million savings projection became a $3.1 million remediation cost.
The CTO presented, the product owner demoed, finance had validated the business case, and the delivery lead sat in the room but was never asked whether the system was ready for production conditions. That delivery lead told me later: "I knew the test set was narrow. But nobody asked, and the number looked too good to challenge."
This is the Demo God Curse. The CTO did everything right by the evaluation framework available. But the evaluation framework was built for deterministic software, where same input produces same output, where "done" means done at release, where bugs are reproducible and costs are predictable. Agentic AI systems are probabilistic. Same input, different outputs. Never truly "done." Failures are not reproducible in the traditional sense. The organisation applied a deterministic evaluation framework to a probabilistic system. That is not a competence failure. It is a category error.
So what would have caught this before production? A structured transition. In AI ROF, the transition from pilot to production is not a single decision. It is gated.
AI ROF defines three gates. Gate 1 (Charter Approved) aligns business value and risks before building. Gate 2 (Alpha Verified) proves reliability under controlled conditions. Gate 3 (Limited Production Trial) tests production readiness with real users at limited scale. Most enterprises have none of them. They have a demo, a leadership sign-off, and a Gantt chart. You cannot apply a Gantt chart to a system that changes its own behaviour daily.
The insurer's agentic system cleared no gate. Not Gate 1. Not Gate 2. Not Gate 3. It cleared a leadership presentation.
Gate 2 is the one the insurer skipped. It does not ask "does the pilot work?" It asks "have you proved this system is reliable under conditions that resemble production?" In a probabilistic system, building the technology is only half the job. The other half is building the stewardship model: the evaluation cadence, the accountability structure, the drift detection approach, the cost monitoring under real conditions, the escalation logic for when the system starts being wrong in ways nobody predicted. That stewardship model must be designed and built alongside the system during delivery, not bolted on after go-live.
The insurer built an excellent system. They did not build the stewardship discipline around it. No evaluation cadence was designed during delivery. No one was named as accountable for ongoing behavioural performance. No drift baseline was established. No cost model was stress-tested against production variability. All of that was deferred to "operations." By the time the system reached production, there was no stewardship infrastructure to receive it. Operations cannot run what delivery never built.
This is not unique. A financial institution spent millions on an agentic loan approval system that hit every accuracy benchmark in development and never served a single customer. The model did not fail. The delivery system did.
Gate 2 creates the structural moment where the delivery lead's concern is not a voice in the room hoping to be heard. It is the gate criteria. The gate ensures the organisation is delivering a governed system, not just shipping a model. Without these gates, the Demo God Curse is the default outcome.
If your team cannot answer all three with evidence, the pilot is not ready to scale. It is ready for a better delivery process.
Question 1: Has your team defined what "working" means for this system beyond the accuracy number from the demo?
Deterministic software either passes or fails. Probabilistic systems operate on a spectrum. If your only success metric is the accuracy score from the pilot, you are measuring the system you tested, not the system you are about to deploy. If nobody on your team can describe what early degradation looks like for this system, you have not built the stewardship model during delivery. You have built the system and hoped for the best.
Question 2: What happens to this system's behaviour when conditions change and nobody touches the code?
Model updates, upstream data shifts, user behaviour changes, seasonal volume patterns. In deterministic software, the system behaves the same until someone changes it. Probabilistic systems change themselves. What is your plan for detecting behavioural drift that nobody triggered? If that plan does not exist, it needs to be built during delivery, not discovered during an incident.
Question 3: Who is accountable for this system's behavioural performance on day 91, and were they involved in the delivery?
The pilot team ships and moves on. The vendor provides support tickets. The platform team monitors infrastructure. But who stewards the ongoing behavioural performance of a system that can change its own outputs without warning? If that person was not involved during the build, they are inheriting a system they were never equipped to govern.
Three questions. If your team cannot answer them today, the good news is you found the gap before production did.
Field observation: Last issue I asked how many weeks since your team ran a formal agent evaluation. I am still collecting responses and will share the pattern once the dataset is meaningful.
This issue's question: How does your organisation currently decide that an AI pilot is ready for production? Reply with one answer:
(a) Accuracy metrics from the pilot. (b) Stakeholder sign-off. (c) A formal readiness gate with defined criteria. (d) We do not have a defined process.
One letter. I am building the industry's first dataset on how enterprises actually make the pilot-to-production decision. The results will inform the Enterprise AI Delivery Maturity Assessment I am developing.
Next issue: Silent Drift. The insurer's 12% misrouting was Silent Drift, the failure mode that starts the day your system goes live, runs for weeks before anyone notices, and costs more to fix than the original build. If you think shipping is the hard part, that one will change your mind.
Every issue of this newsletter includes a diagnostic you can run inside your organisation. If the Demo God Curse resonated, my free guide "The First 90 Days as an AI Delivery Manager" goes deeper into the stewardship structures and production gates that prevent it. I send it to email subscribers along with frameworks that do not appear here. Link in my Featured section to subscribe.
Vijayan Seenisamy Enterprise Agentic AI Systems Delivery | Creator, AI ROF™ Author, The AI Delivery Manager Blueprint
New issues arrive by email first. Subscribe here.
