Fault reports work best when they stick to four headings: symptom, diagnosis, repair, prevention. This one documents a fault now common enough to deserve a pattern number, if anyone numbered these things: an organisation turns on AIOps for Security tooling — the shorthand, still lacking any formal standard, for security work where machine learning and LLMs share the load — and the assistant begins closing cases on its own from the first hour. Nobody chose that behaviour. Nobody signed for it. Nobody noticed for eleven days. The pages that follow treat it as a fault, diagnose why it keeps happening, and set out the repair in steps a two-person team can complete without a budget request. The tone is deliberately unglamorous; faults deserve checklists, not essays.
Symptom: The AIOps for Security Queue Started Closing Itself
Here is how the fault presents. Mail marked suspicious starts disappearing from the triage queue within minutes of arriving. Analysts notice cases marked “resolved” that they never touched, and the closure notes read fluently enough that nobody argues with them — which is part of the problem, because fluency reads as competence even when the underlying judgment has never been measured. Weekly metrics look wonderful: queue times collapse, “response” figures improve, everyone breathes out. Then a colleague recognises a customer’s legitimate invoice among the closed items, or a near-miss surfaces through luck, and the two questions nobody can answer arrive at once: what exactly has it been closing, and on what evidence? When someone asks when the observe-only period ended, the honest answer is that there never was one: the Microsoft Phishing Triage Agent, in its documented state as of October 2026, starts closing mail by itself from its first minute in service, and no observe-only switch appears anywhere in the product documentation. The contrast with Google’s Triage and Investigation Agent is instructive: that product holds to level one, recommending and drafting while people decide, at roughly a minute per triage. One vendor made observe a product decision the customer could skip; the other built it in. Teams running the first kind experience the fault; teams running the second rarely do. That asymmetry alone tells you most of the diagnosis.
Diagnosis: An Autonomy Claim With No Evidence Chain
Strip the symptom to its structure and the problem is plain: software holds a level of authority that no measurement supports. The published ladder prices authority in evidence. Recommending begins after roughly 200 graded alerts, of which 30-plus must be genuine. Silent closing waits until the miss rate has been bounded at or below 2% over 150-plus genuine cases, with decoys planted, a blind holdout scored, and one-click reopen live. Reversible actions wait longer: wrong-action rates must sit below 1% across 300-plus verdicts on cases where acting would have been right, with rollback rehearsed and hard rate limits in force. Irreversible actions are never granted, on any evidence. An agent closing mail on day one holds rung two while holding none of that evidence. The missing artefacts — the graded sample, the bound, the decoys — are not paperwork. They are the difference between a tool that earns its closures and a tool that asserts them, and audits of collapsed programmes almost always find the same artefacts absent. Think of it as an evidence chain, the way maintenance organisations think of a component’s history: every hour of flight is accounted for, every inspection signed. An authority granted without its chain is not authority; it is hope with a logo.
Root Cause: No Observe Stage Was Ever Installed
Why does this keep happening? Three causes recur. First, defaults: act-from-launch is the vendor’s default posture because observe-only reads as slowness in a demo, and demos sell products — nobody books a meeting to watch software wait. Second, vocabulary: procurement documents ask whether the product “uses AI”, which every product now answers yes to, and never ask which actions it may take unaided — so the authority question never enters the contract, and the contract is where bargaining power lives. Third, evaluation debt: teams know labelling takes hours nobody has, so the measurement is deferred, and the deferred measurement quietly becomes permanent while the closure count grows in the dark. The vendor’s own behaviour compounds all three; Gartner placed AI SOC agents on the inflated-expectations peak of its 2026 hype cycle and flagged AI washing for a reason. None of the causes is technical. All of them are cheap to fix once named, which is the good news buried in this section.
The Repair: Restoring the AIOps for Security Evidence Chain
Repairs run in order, and none needs vendor permission. Treat this as the work order:
- Demote the authority today. Re-enable human confirmation for every closure, or route auto-closures to a review bin, until the evidence below exists. Document the change with a date and a name; the paper trail is part of the repair, not overhead.
- Label a blind sample. Grade a recent sample of cases — 1,240 emails was the documented exercise, 71 of them genuine — with answers fixed before the system sees them. Two labellers, one tie-break rule agreed in advance, and no peeking afterwards.
- Compute the bound, not the rate. In that documented trial the agent missed 2 of 71 genuine cases: 2.8% observed, but a 95% ceiling near 8.6% on so thin a sample — failing the 2% gate despite 99.8% closing precision and 86% precision on its own “genuine” verdicts. The lesson generalises: midpoints flatter, bounds decide.
- Split every result by category. The documented pooling failure left cloud-identity cases failing 42.3% of the time beneath a blended 9.1% headline; your worst category is the one attackers will use.
- Plant canaries before re-promoting. Fifty seeded genuine threats per quarter, withdrawal after three get closed — because random spot-checks prove almost nothing: audit a twentieth of roughly 1,845 closures, find nothing, and the miss rate can still reach 38%.
- Write the register. One signed page listing every autonomous action, its evidence, its owner, and its rollback. No entry, no authority. Review it at every renewal, because defaults have a way of drifting back while nobody watches.
Teams that run this sequence usually discover the tool is genuinely useful at recommend-only — the documented trial attributed 93.5% of triage workload to the removable pile, none of it requiring autonomy. The repair often ends with better outcomes than the original purchase promised, minus the silent closures. Expect the demotion period to last a quarter, not a week: that is how long collecting enough graded cases honestly takes, and shortening it dishonestly is how the fault re-installs itself.
Prevention: What to Demand Before the Next Switch-On
Prevention is a short list of demands, made while the contract is still negotiable:
- An observe-only mode, in writing, with the autonomous path off until you enable it.
- Per-category results measured on your data, with sample sizes stated against the 236-and-313 standard for small claimed miss rates.
- Canary support, one-click reopen, and a rate limit on any autonomous action class, all demonstrated live during the trial rather than promised in slides.
- Least privilege and no outbound channels for anything reading attacker-controlled text — the NCSC’s December 2025 assessment calls language models an inherently confusable deputy on injection risk and says filtering alone cannot close it.
- Clear answers on data residency, including defaults: Security Copilot enables customer-data sharing unless the organisation holds E5 or E7 licensing, and Gemini’s Security Ops processes on global endpoints, though prompts stay outside model training.
None of these demands is exotic; all of them are verifiable; and a vendor who refuses them has answered the procurement question honestly, in writing, before any money moved. The definitions, gates, and worked case behind every number in this report are collected in the Ops Atlas dossier behind the label. File this one under faults that were never really the software’s: the machine did what it was allowed to do, promptly and tirelessly, exactly as configured. The fault was that nobody wrote the permission slip — and writing permission slips down, signed and dated, is the whole of the fix.
