Tailor the Framing, Fix the Facts
A factual-invariance gate for distributing AI incident information.
Apart Research & CeSIA, AI Incident Response Research Sprint, September 2026 · Track 4, Communication · Iven Cui, EverTO Ventures, with Trajectory Labs
A warning shot only works if it reaches people. The July 2026 incident, in which OpenAI evaluation agents escaped a sandbox and spent days inside Hugging Face's production infrastructure, produced an exceptional public record that travelled through security and AI-policy circles and largely stopped there. Rewriting it for wider audiences is the fix. Moving the facts is the risk. This is the control that separates the two.
Where this came from
This project started as a distribution idea, not an accuracy one. There is already a large population of people whose job is finding a story before anyone else has it: influencers/creators across YouTube, Twitter, LinkedIn, Substack, podcasts and short-form video, competing continuously for novelty. They are hungry for newsjacking opportunities, and a fully documented incident that nobody has covered properly is exactly that. What stops them is cost. The primary record takes a day to read and is full of ways to be wrong in public.
So the plan was to build a platform between the record and the influencer: an opt-in list they join, where we tailor the story to each influencer's audience. The same verified account, rendered differently for the people who actually watch them, by age, technical background and channel. A short-form channel reaching eighteen-year-olds needs a different piece than a policy newsletter read by regulatory lawyers, and neither is served by the same explainer. Hand it over with the sources attached and explicit permission to disagree with our framing. Their research cost goes to roughly zero, and the story reaches people who were never going to read a technical timeline.
The four briefs below are that idea demonstrated: one incident, four audiences, four genuinely different pieces of writing.
Then the obstacle. One wrong number, published to a large audience, ends the relationship: the correction travels a fraction as far as the error, the influencer wears it in public, and we may lose influencers' trust. Reach is cheap once the incentive is aligned. The influencer's trust that your material will not embarrass them is the whole asset, and tailoring is exactly the operation that puts it at risk. So the binding constraint is not distribution. It is fidelity under tailoring, at volume, without a human reading every draft.
That constraint had to be solved before the relay could responsibly be switched on, and nobody had measured it. This artifact is that measurement, and the control it produced. The distribution protocol it was built for is protocol.md.
What this is
A claim ledger of 63 claims about the incident. Fifty-six come from six primary sources and carry a verbatim support quote and a status recording whether the claim is established, a party's assertion about itself, divergent between sources, or contested. Seven more record regulatory context from secondary material.
A gate (gate.py) that holds a tailored brief's numbers, dates, named entities and attributions to that ledger using seven deterministic checks, while constraining nothing about tone, length, structure or framing.
Eight briefs: two for each of four audience segments, one written under a naive engagement-first prompt and one written under the gate.
A distribution protocol covering opt-in recipient eligibility, embargo, corrections and dual-use control.
Findings
Drift concentrates in one place. Against an audited set of ten errors in the ungated briefs, seven sat in the single general-audience brief, at roughly eight times the rate of the three specialist briefs.
| Ungated brief | Words | Errors | Per 1k words |
|---|---|---|---|
| S1 general audience | 410 | 7 | 17.1 |
| S2 tech explainer | 462 | 1 | 2.2 |
| S3 security practitioner | 436 | 1 | 2.3 |
| S4 policy newsletter | 438 | 1 | 2.3 |
Accuracy risk is highest exactly where reach is most valuable and readers are least equipped to catch the error.
The gate is precise and partial. No finding it raised was a false alarm. Three errors escaped it, all narrative distortions of timing or intent.
| Gate version | TP | FP | Precision | Recall |
|---|---|---|---|---|
| v0.1, atoms only | 5 | 0 | 100% | 50% |
| v0.2, + word numbers + closed sets | 7 | 0 | 100% | 70% |
A clean gate is not a clean brief. All four gated briefs passed with zero findings and were still wrong in four places. The clearest: three of them presented the 17,600 action count as actions taken inside Hugging Face, when it is the total for the whole campaign and the intrusion into Hugging Face was roughly its last two and a half days. The count is a correct ledger atom. The date range is a correct ledger atom. The claim built from them is false, and no atom-level check can see it.
That is why the artifact ships with a protocol requiring a named human reviewer who is not the drafter, rather than more automation.
Reproduce it
Python 3.8 or later. No third-party packages. Runs in under a second.
python3 score.py # reproduces every number above python3 gate.py --factbase factbase.json yourfile.md # run the gate on your own text
The gate exits non-zero on any finding, so it drops into a pre-commit hook or CI step unchanged.
Files
| README.md | Start here. What the seven checks are and what the gate cannot do. |
| factbase.json | The claim ledger. Every claim with its source, status and verbatim support quote. |
| gate.py | The gate. Seven deterministic checks, standard library only. |
| score.py | Scoring harness. Reproduces the tables above. |
| ground_truth.json | The audited reference set of ten errors. |
| protocol.md | The Relay Protocol v0.1: eligibility, embargo, corrections, dual-use control. |
| SOURCES.md | The six primary sources, with URLs and verification status. |
Links open plain-text views so they render in the browser. The originals, with their real extensions, are in the zip.
Briefs. Four segments, two conditions each. The _ungated pair is the naive engagement-first draft; the _gated pair is written against the ledger.
| S1 general | ungated · gated |
| S2 tech | ungated · gated |
| S3 security | ungated · gated |
| S4 policy | ungated · gated |
Reports. The error audit with reasoning and error classes, and scores.txt, the raw scoring output.
Download
Limits
Four segments, eight briefs, ten reference errors. The seven-of-ten concentration is a striking pattern in a small sample, not a measured rate. Ground truth was adjudicated by the same author who wrote the briefs and the gate, which is the work's most serious methodological weakness. The gate checks agreement with the ledger, not truth, so a fabricated ledger would produce confidently gate-clean falsehoods; the main defence is that the ledger ships public with a verbatim quote per claim. The briefs have not been sent to anyone and the protocol has never been run.