# Ground-truth error audit of the four ungated briefs

Method: after generating the four ungated briefs, the author re-read each one
sentence by sentence against the claim ledger and the six primary sources, and
recorded every factual error independently of what `gate.py` reported. This list
is the reference set used to score the gate's precision and recall in Table 3.
It was written before the gate's v0.2 upgrades and has not been revised to
flatter them.

Caveat stated plainly: the same author produced the briefs and the reference
set. This is not blind adjudication. See Limitations.

---

## S1 general audience (7 errors)

| # | Error | Class | Correct position |
|---|---|---|---|
| S1-a | "about 20,000 moves" | inflated quantity | ~17,600 recovered actions (C001) |
| S1-b | "Over the next week, the AI broke in" | timeline compression | Intrusion into Hugging Face ran 11-13 July, roughly 2.5 days (C003) |
| S1-c | "Hugging Face didn't notice for almost a week" | false duration + false characterisation | Detected and cut off within the 4.5-day window; detection fired and correlated, escalation failed (C002, C026) |
| S1-d | "they asked ChatGPT and Claude for help" | wrong named entity | The models that refused were Claude Opus and Fable (C027) |
| S1-e | "OpenAI stayed quiet about it for five days" | implied concealment | OpenAI's monitoring alerted 19 July, connected 20 July, disclosed 21 July (C035) |
| S1-f | "Their own CEO called it unprecedented" | misattributed speaker | "Unprecedented" is OpenAI's corporate wording; the CEO quoted in that post is Hugging Face's, saying "possibly the first of its kind" (C051) |
| S1-g | "a Chinese AI model" | unnamed nationality-framed attribution | The fallback was a quantized open-weight GLM-5.2 run on Hugging Face's own infrastructure (C027) |

## S2 tech explainer (1 error)

| # | Error | Class | Correct position |
|---|---|---|---|
| S2-a | "Roughly 20,000 recorded actions" | inflated quantity | ~17,600 (C001) |

## S3 security practitioner (1 error)

| # | Error | Class | Correct position |
|---|---|---|---|
| S3-a | "Around 20,000 actions in 4.5 days" | inflated quantity | ~17,600 (C001) |

Note: "Recon was over 6,000" is correct (6,191) and the gate correctly does not
flag it once directional rounding tolerance is enabled. An earlier gate version
flagged it, which is what motivated that feature.

## S4 policy newsletter (1 error)

| # | Error | Class | Correct position |
|---|---|---|---|
| S4-a | "eight days after the last attacker action" | wrong arithmetic, spelled out | Last attacker action 13 July, OpenAI detection 19 July: six days, not eight (C002, C035) |

---

## Error classes

Ten errors fall into five classes:

1. **Inflated quantity** (S1-a, S2-a, S3-a). A ledger figure rounded past the
   point of accuracy. Three of ten, and the single most repeated failure: the
   same 17,600 to 20,000 inflation appeared independently in three of four
   briefs.
2. **Wrong or vague named entity** (S1-d, S1-g). A real entity swapped for a
   more familiar one, or replaced by a category label.
3. **Misattributed speaker** (S1-f). A corporate statement promoted to a named
   individual.
4. **Narrative distortion** (S1-b, S1-c, S1-e). Timing and intent restated in a
   way no single atom contradicts, but which the record does not support.
5. **Spelled-out arithmetic error** (S4-a).

Classes 1 and 2 are mechanically checkable. Class 5 is checkable once numerals
written as words are parsed. Classes 3 and 4 are not reachable by any
atom-level check, and are the reason the protocol in Section 3.4 keeps a named
human reviewer in the loop rather than treating a green gate as sufficient.