Runbook — quarantine triage

Audience: whoever is on corpus duty. Trigger: an ingest run reports N quarantined, or openregs quarantine list is not empty. Time: minutes for the decision, longer for whichever path it points at.

A document in quarantine was fetched, archived and pinned in SOURCES.lock, and then refused by one of the four validation gates (or by the OCR confidence bar its source sets). Nothing was written to the canon for it and nothing will be until you resolve it: the gates are the only way in, not a warning beside one. An item that sits in the queue is a piece of law the corpus does not contain, so the queue is worked, not watched.

Everything below is executable against the fixture corpus, from the engine checkout, offline. Substitute your own paths for $WORK.

WORK=/tmp/triage          # scratch: canon, state, quarantine for this walkthrough
mkdir -p $WORK

Two things have to be set up first, and neither is optional — without them the walkthrough runs green and quarantines nothing, which is the one outcome a page about a queue must not produce.

A queue needs something in it. The shipped eur-lex source resolves 32024R0001 to the clean act, so a poll of it captures two documents and holds neither. Two catalogues are shipped for exactly this, one per path below, and they differ only in which rendering of 32024R0001 the offline poll resolves to:

catalogue	serves	walked in
`fixtures/scope/triage-mapper-gap.catalogue.json`	`list-in-a-paragraph.formex.xml` — the defect is in our transform	§ 4a
`fixtures/scope/triage-source-defect.catalogue.json`	`dangling-reference.formex.xml` — the defect is in the document	§ 4b

Write a configuration that names one. Everything else is the shipped file, because the claim is about a poll meeting a defective document and not about a poll configured strangely:

CATALOGUE=fixtures/scope/triage-source-defect.catalogue.json   # § 4b; § 4a uses the other

uv run --frozen python - "$CATALOGUE" > $WORK/sources.yaml <<'PY'
import pathlib, sys, yaml
document = yaml.safe_load(pathlib.Path("config/sources.yaml").read_text())
for source in document["sources"]:
    if source["source_id"] == "eur-lex":
        source["offline_fixtures"]["catalogue"] = sys.argv[1]
yaml.safe_dump(document, sys.stdout, sort_keys=False)
PY

A pull request needs a repository to be opened against. There is no forge in this project and none is needed (§ 2) — a bare repository with one commit on main is one:

REMOTE=$WORK/regime.git
git init --quiet --bare -b main $REMOTE
git clone --quiet $REMOTE $WORK/seed
echo "FIXREG regime repository (fixture remote)." > $WORK/seed/README.md
git -C $WORK/seed add README.md
git -C $WORK/seed -c user.name=fixture -c user.email=fixture@openregs.io \
  commit --quiet -m "seed"
git -C $WORK/seed push --quiet origin main

Where the items come from

Both ingests fill this queue, and each has to be told where it is — a quarantine belongs to a regime and a source can serve several, so neither is guessed:

# a poll of one source, draining what its discovery found
openregs ingest --source eur-lex --offline-fixtures --config $WORK/sources.yaml \
  --state-dir $WORK/state --lock $WORK/SOURCES.lock \
  --canon $WORK/canon --inbox $WORK/inbox --quarantine $WORK/quarantine

# a regime's scope crawl — the ingest a regime repository runs (new-regime runbook § 6)
openregs ingest --regime <id> --offline-fixtures --quarantine quarantine/

--state-dir $WORK/state is what makes § 4a’s replay mean anything, and leaving it off does not fail: the snapshots go to the checkout’s own state/ instead, and a replay pointed at $WORK/state finds an empty store and reports 0 captures re-transformed, 0 units rewritten, 0 unchanged at exit 0. A green no-op is the worst answer a step like that can give, so keep the store, the canon, the ledger and the queue in one place. --lock for the same reason in reverse: without it the walkthrough appends to the checkout’s own regimes/fixreg/SOURCES.lock.

The crawl holds its items inside the regime repository, beside the canon they were refused from, because an item becomes a branch on that repository (§ 2).

A held document is the pipeline working, so a run that has somewhere to put one reports it and succeeds:

32024R0001: fetched (depth 0, seed), sha256 38290b429609, QUARANTINED — 1 of 4 gate(s) failed: reference-resolution -> quarantine/eur-lex--32024R0001
ingest: regime <id>: citation depth 1, 1 instrument in scope and captured, 0 held out beyond the depth, 0 Expressions written to canon, 1 quarantined
ingest: 1 document held for review — work the queue with `openregs quarantine --quarantine quarantine/ list` (https://docs.openregs.io/runbooks/quarantine-triage/)

A crawl told no --quarantine fails the instrument instead, printing the gate report and then:

no quarantine directory was named, so nothing holds it for review; re-run with --quarantine <path>
...
FAIL regimes/<id>/SCOPE.yaml: 1 in-scope instrument(s) did not reach the canon

That is deliberate. The bytes are archived and pinned either way, so nothing is lost from the archive — but the act is in neither the canon nor a queue, and a corpus quietly missing an instrument it declared is exactly what a green run must not be able to hide. Re-run with --quarantine and work the item.

The item is the same whichever ingest wrote it — same reason, same snapshot reference, same gate report, same mapping report — so nothing below depends on which one you ran. The capture lines quoted in §§ 4a–4b are a poll’s; a crawl’s say where in the frontier the document sat instead, as the one above does.

0. See the queue

openregs quarantine --quarantine $WORK/quarantine list

quarantine /tmp/triage/quarantine: 1 open item(s)
  eur-lex--32024R0001: 32024R0001 — reference-resolution

One line per item: the item’s directory name (how every other command addresses it), the document id, the gates that refused it, and — once somebody has taken it — who claimed it and which branch its request is on. --closed lists the resolved ones instead, each with the resolution that answered it.

The item directory itself holds everything the decision needs:

file	what it is
`item.yaml`	snapshot reference (uri, sha256, storage path), the per-gate report, the mapping report, and the claim and pull request once they exist
`extracted-text.txt`	for a scanned source only: exactly what the recogniser read

1. Claim it

openregs quarantine --quarantine $WORK/quarantine claim eur-lex--32024R0001 --by <you>

Writes claim: {by, at} into item.yaml. It refuses a second claim by somebody else rather than overwriting the first — two people fixing one mapper twice is the failure this prevents.

2. Open the pull request

openregs quarantine --quarantine $WORK/quarantine \
  open eur-lex--32024R0001 --remote $REMOTE --by <you>

eur-lex--32024R0001: pull request opened — quarantine/eur-lex--32024R0001 -> /tmp/triage/regime.git (faf2734ca0db), body quarantine/eur-lex--32024R0001/PULL_REQUEST.md

(The commit is a fresh one every run, so that sha will not be yours.)

There is no forge in this project and none is needed: a pull request is a branch on the regime repository plus a request body, and both are ordinary git. The command clones the remote into a temporary directory (never your checkout), creates quarantine/<item>, commits the item and a generated PULL_REQUEST.md under quarantine/<item>/, pushes the branch, and records pull_request: {remote, branch, base, commit, body, opened_at} back into item.yaml. The body carries the failing check name, the full gate report with its detail lines, the mapping report and the snapshot sha256 — everything needed to decide without re-running the pipeline. Read it first with:

openregs quarantine --quarantine $WORK/quarantine open eur-lex--32024R0001 --dry-run

which prints the body and pushes nothing.

Review happens on the branch; a maintainer merges it once the item closes, so the history holds both the refusal and its answer.

3. Decide: fix the mapper, or hand-annotate

One question decides it: where is the defect?

	fix the mapper	hand-annotate
the defect is in	our transform	the document the publisher served
typical report	`structural-coverage` names an element type; `round-trip-integrity` names text that vanished	`reference-resolution` names a citation with no target; low OCR confidence on a damaged scan
why	ordinary markup the mapper has no rule for — and the same gap is silently deforming every other document that uses it	no mapper can invent a target that does not exist, or read a page that cannot be read
the fix reaches	every snapshot ever captured, through `openregs replay`	this one document
costs	a code change, review, and a replay	a person’s judgement, recorded with their name

When both readings are arguable, prefer fix the mapper: one document annotated by hand is a permanent exception, and an exception that turns out to have been a mapper gap will be reproduced on every future document of the same shape.

4a. Path one — fix the mapper, then replay

Set CATALOGUE=fixtures/scope/triage-mapper-gap.catalogue.json and re-run the setup, so the poll resolves 32024R0001 to this path’s document rather than § 4b’s. Nothing else changes.

The item that motivates it: fixtures/sources/gates/list-in-a-paragraph.formex.xml, whose Article 4(1) enumerates its points in a <LIST> inside the <PARAG>. The mapper read a <LIST> hanging off an <ARTICLE> and nothing else, so the list was never visited — its elements unmapped, its text absent from the output. Two gates fire, and both are the same fact:

32024R0001: captured, sha256 7de26acb1bf4, QUARANTINED — 2 of 4 gate(s) failed: round-trip-integrity, structural-coverage

Fix the mapper. For this walkthrough the fix is shipped as a patch:
Terminal window
```
git apply fixtures/patches/mapper-list-in-a-paragraph.patch
```
It teaches _paragraphs to read a LIST inside a PARAG through the same _points an article’s own list goes through, and takes the Formex mapper to formex-akn/1.1.0 — a minor bump, because the mapper handles something new and no existing output changes shape.

Replay every snapshot the source ever captured, offline. This is the point of fixing the mapper rather than the document: the gap is repaired everywhere it applies, in one run, with no network access at all.

openregs replay --source eur-lex --offline \
  --canon $WORK/canon --state-dir $WORK/state --report $WORK/replay.yaml

32024R0001: sha256 7de26acb1bf4, transform formex-akn/1.1.0 -> /akn/eu/act/reg/2024/1/eng@2024-06-01 (0 rewritten, 0 unchanged, 12 added)
32025R0007: sha256 657455d6f1e8, transform formex-akn/1.1.0 -> /akn/eu/act/reg/2025/7/eng@2025-03-01 (0 rewritten, 2 unchanged)
replay: eur-lex, 2 captures re-transformed, 0 units rewritten, 2 unchanged, 12 added

The refused document’s twelve units are added — they never existed — while the act that was already fine is re-emitted identically and left untouched, version bump and all. A version bump is not a change.

Close the item. It closes on evidence, without you asserting anything:
Terminal window
```
openregs quarantine --quarantine $WORK/quarantine close --all \
  --canon $WORK/canon --state-dir $WORK/state
```
```
eur-lex--32024R0001: closed — fix_mapper: re-mapped under formex-akn/1.1.0: 4 gate(s) passed
```
close re-reads the archived snapshot by digest, re-maps it with the transform as it stands now, re-runs all four gates, and requires the canon to actually hold the Expression that produced. Both halves are required: a mapper that could now handle the document does not by itself put the document in the corpus. --all sweeps the whole queue, which is how one mapper fix closes every item it repaired.

Run it before the fix and it tells you so, and closes nothing:

eur-lex--32024R0001: still open — the transform still refuses it: 2 of 4 gate(s) failed: round-trip-integrity, structural-coverage (under formex-akn/1.0.0); either fix the mapper and replay, or correct the document by hand with `openregs quarantine annotate`

4b. Path two — hand-annotate the one document

The item that motivates it: fixtures/sources/gates/dangling-reference.formex.xml, whose Article 4(2) cites an Article 42(9) that this twelve-article act does not contain. Every word is present and every element understood, so only one gate fires:

32024R0001: captured, sha256 38290b429609, QUARANTINED — 1 of 4 gate(s) failed: reference-resolution

No mapper can repair this: the mapper is right to refuse to guess, and the target genuinely does not exist. A person decides, and signs for the decision.

Start from the mapper’s own output, so you are correcting a document rather than writing one. The item names the archived bytes; map them:
Terminal window
```
STORAGE=$(uv run --frozen python -c "import yaml; print(yaml.safe_load(
  open('$WORK/quarantine/eur-lex--32024R0001/item.yaml'))['snapshot']['storage_path'])")

openregs normalize $WORK/state/snapshots/$STORAGE \
  --doc-id 32024R0001 -o $WORK/draft.akn.xml
```
(storage_path is the sha256/aa/bb/<digest> key printed in item.yaml; any way of reading it out of that file will do — the parser above is the one this project already depends on, so it needs nothing installed. yq would read it in one line and is not a dependency here.)

The flag is -o, or --output in full. --out happens to work today only because argparse accepts any unambiguous prefix, which is a property of the parser and not something the CLI promises.
Correct it. Here the judgement is that the printed citation addresses the incident-reporting paragraph, so the link is supplied and the words are left exactly as the publisher printed them — Akoma Ntoso separates the text of a reference from what it points at, which is what makes this annotation rather than redrafting:
```
<p>Those measures shall be reviewed following any incident reported under Article 42(9).</p>
<p>Those measures shall be reviewed following any incident reported under <ref href="#art_5__para_3">Article 42(9)</ref>.</p>
```
Never edit the legal text itself. Provenance offsets index it and every atom’s quoted_text is a byte-for-byte slice of it.
Admit it under the override:
Terminal window
```
openregs quarantine --quarantine $WORK/quarantine \
  annotate eur-lex--32024R0001 --akn $WORK/draft.akn.xml --by <you> --canon $WORK/canon
```
```
eur-lex--32024R0001: /akn/eu/act/reg/2024/1/eng@2024-06-01: 12 units in eu-act-reg-2024-1/eng@2024-06-01
  method: human_authored, authored_by: <you> — recorded in every unit's metadata
```
This is the only way into the canon that does not go through the four gates, and what replaces them is attribution. Every unit’s meta.yaml records:
```
source_hash: 38290b4296092dbb2e9c024e09e296b45880984ed1e5e68f50e9ea05e3eb7ee9
publisher: eur-lex
transform_version: formex-akn/1.0.0
method: human_authored
authored_by: <you>
```
method: human_authored is the warning label a later reader needs: re-running transform_version over the same snapshot will not reproduce this file. The transform is still recorded because it says what the annotation started from. authored_by is required whenever method is human_authored — a hand-authored canonical text with nobody’s name on it is a text nobody can be asked about (spec/schemas/unit-meta.schema.json).

The scope is the document, not the line you touched: a maintainer vouches for the document they inspected and committed.
Check what you wrote, before it goes anywhere:
Terminal window
```
openregs validate unit-meta $WORK/canon/eu-act-reg-2024-1/eng@2024-06-01/*.meta.yaml
```
```
OK /tmp/triage/canon/eu-act-reg-2024-1/eng@2024-06-01/art-012.meta.yaml: valid unit-meta (unit-meta.schema.json)
```
Twelve of those, one per unit. The schema is where authored_by becomes required rather than advisory, so this is the check that would catch a hand-authored text with nobody’s name on it.

make check-canon is not that check, here. It runs the canon rules over the regimes in the checkout — regimes/fixreg/ and its ledger — and never reads $WORK/canon, so it passes whatever you did in $WORK and passing means nothing about it. Where it becomes the real check is in the regime repository: these units reach regimes/<id>/canon/ through the branch § 2 pushed, and it is that repository’s CI that runs the rules over them — id-reuse against its retired ids, sources-integrity tracing each unit to a payload pinned in its own SOURCES.lock. The hand-authored units are ordinary canon in every other respect: the override buys an exemption from the gates, never from provenance. Nothing in $WORK is a substitute for that, and this walkthrough deliberately stops at the branch.

Close the item.

openregs quarantine --quarantine $WORK/quarantine \
  close eur-lex--32024R0001 --canon $WORK/canon --state-dir $WORK/state --by <you>

eur-lex--32024R0001: closed — hand_annotate: 12 hand-authored unit(s) of /akn/eu/act/reg/2024/1/eng@2024-06-01 admitted by <you>

The evidence close looks for is exactly the override: units in the canon whose method is human_authored and whose source_hash is this item’s own snapshot digest. Nothing else counts, and the item cannot be closed by saying so.

5. What closing does

The item directory moves to quarantine/closed/<item>/, keeping the failure report, with a resolution: block appended naming the path taken, when, by whom, the Expression and units it produced, and the evidence.
If the item has a pull request, the same move is committed and pushed to its branch, so the request on the remote carries its own outcome:
```
2588275 quarantine: close eur-lex--32024R0001 (hand_annotate)
c1085e5 quarantine: eur-lex/32024R0001 failed reference-resolution
```
A queue that says “closed” locally while the request still asks for a decision is exactly the drift triage exists to prevent, so this is not optional; use --no-push only when the remote is genuinely unreachable, and say so in the request.
Merging the branch is the maintainer’s, and is what puts the record in the regime repository’s history.

Nothing is ever deleted. quarantine/closed/ is the standing answer to “why is there a hand-authored text in this corpus”, and the only place that answer lives.

6. If neither path applies

The document is out of scope. Do not annotate it. Fix the regime’s SCOPE.yaml or the source’s discovery bounds so it is not captured, and note that in the request before closing the branch by hand.
A threshold is genuinely wrong for a source (a publisher whose markup this project will never fully map, a scanned series that cannot reach 100% coverage): lower it in config/sources.yaml, in a reviewed commit, with the reason written beside it — the gazette source’s coverage_percent: 95.0 is the worked example. Lowering a threshold is a judgement about a corpus. Deleting a gate is not an option.
The bytes are damaged (the payload no longer matches its digest): openregs verify --snapshots reports it. Re-capture rather than annotate; an item is only as good as the archive under it.