Runbook — standing up a new regime
Audience: whoever is adding a jurisdiction, and whoever administers the org. Trigger: an adapter can ingest a real instrument, and there is data to hold. Time: minutes to create the repository; the corpus takes as long as the law does.
A regime lives in its own repository. openregs/openregs is the engine —
spec, tooling, fixtures, and regimes/fixreg — and it never grows a second
regime. Each real corpus (openregs/regime-eu, regime-uk, regime-us-cfr)
is a separate public repository under CC-BY-4.0, with its own reviewers, its own
releases and its own signing.
“Never grows a second regime” is a check rather than an intention.
make check-regime-boundary refuses any directory under the engine’s regimes/
whose provenance is not committed under fixtures/ — which is what a real
corpus’s SOURCES.lock can never be, since it records the regulator the text was
fetched from. The engine’s suite runs it over the engine’s own tree, so
regimes/eu/ added there fails the pull-request gate, pointing back at this
runbook and at the decision in
ARCHITECTURE.md § fixreg is a fixture, not data.
Creating one is a command, not a checklist:
cd <the directory the engine checkout sits in> # not the checkout itselfuv run --project openregs openregs init-regime euopenregs is not on your PATH and nothing here puts it there — a bare
openregs is command not found, exit 127. Every openregs line on this page is
run out of a checkout with uv, and until step 6 the only checkout there is is the
engine’s own: openregs/ above is git clone https://github.com/openregs/openregs,
the directory you are standing beside. It has to be a checkout rather than a
packaged binary, because init-regime reads that checkout’s licence and roster and
pins its HEAD as the engine this corpus is built by. --engine <path> names one
somewhere else.
From step 6 the prefix changes once and for good: the corpus clones the engine
commit it pins into .openregs-core/, and every command after that runs out of
that clone, so what checks the corpus is the commit the corpus declares.
That writes regime-eu/ complete and green: its CI passes before it holds a
single word of law. That is deliberate — a repository whose checks only work once
somebody has filled it in is a repository whose checks were never tried.
What is enforced by a machine, and what is not
Section titled “What is enforced by a machine, and what is not”| Rule | Enforced by |
|---|---|
| the corpus holds exactly one regime, and it is not one the engine ships | openregs check-regime, in the repository’s own CI |
| the engine is pinned to an exact commit, and CI runs that commit | check-regime, plus a rev-parse the workflow does itself |
| the scope stops being a placeholder before any text lands | check-regime |
| the charter says what the corpus covers before any text lands | check-regime |
| somebody is answerable before any text lands | check-regime |
| the owning team is not one scoped to the engine | init-regime, and check-regime thereafter |
README.md names every required context, and clones the engine the pin names | check-regime |
every ledger validates against spec/schemas/ | openregs validate, in CI |
| the canon does not contradict itself | the canon rules, in CI |
| the committed graph is what the canon produces | a rebuild and a diff, in CI |
CODEOWNERS matches MAINTAINERS.yaml | python -m openregs.governance --check, in CI |
| no check reaches the network | the engine’s tooling/ci/offline-shell, by kernel |
| a scope crawl reads the engine’s config and fixtures, and writes the corpus’s canon | the engine’s suite, against a repository this tool created — tooling/tests/test_corpus_ingest.py |
| a crawl that a gate refuses holds the document instead of losing it, and fails outright if it has nowhere to hold it | the engine’s suite, from a corpus — tooling/tests/test_scope_quarantine.py |
| branch protection on the created repository | a person, by hand — step 5 |
| the reviewer team existing on the forge | a person, by hand — step 4 |
| a source resolving a real instrument | nothing — every shipped catalogue is a fixture catalogue, see step 6 |
The first two are the honest gap. Everything the repository can enforce about itself, it enforces from the first commit; everything that is server-side configuration is configuration until somebody applies it.
1. Create it
Section titled “1. Create it”uv run --project openregs openregs init-regime eu \ --seed 32024R0575 \ --seed-title "Commission Regulation (EU) 2024/575" \ --seed-source eur-lex| Flag | What it decides |
|---|---|
<id> | the regime id: lowercase, hyphenated, the pattern scope.schema.json states. It is the directory name and SCOPE.yaml’s regime, for ever |
--into | where to write it. Default ./regime-<id>, which is the org’s own repository name |
--engine | the engine checkout to pin, and to read the roster and the licence from. Default: the checkout the CLI is running from |
--core-ref | the engine commit to pin. Default: that checkout’s HEAD |
--maintainer, --maintainer-name, --maintainer-email | the one person answerable for the corpus. All three or none; left out, no roster is written and nobody is invented |
--maintainer-team | the team that owns the corpus. Default regime-<id>-reviewers. It must have write access to the repository this becomes; the engine’s own teams are refused |
--seed, --seed-title, --seed-source | the first instrument. All three or none |
The id is permanent. It is in every atom id, every eId, every release tag and
every index/combinations entry that ever names this regime. Changing it later
is not a rename, it is a deprecation of everything.
Leaving the seed out is allowed and temporary. The scope then carries an
UNSCOPED placeholder so the file is still valid, and two things refuse it: the
source id it names is one no config/sources.yaml defines, so openregs ingest cannot run against it; and openregs check-regime fails the moment the
regime holds any canon or atoms while the placeholder survives. An empty
repository may be unscoped. A repository with text in it may not.
--seed-source eur-lex names a source that is bound to fixreg today. The
shipped eur-lex entry declares discovery.regimes: [fixreg] and a FIXREG
catalogue, so naming it here records an intent rather than something a crawl can
act on — see step 6, which is where that bites. It is still the right id to
write: it is the source this instrument really comes from, and the binding is
what has to move.
Two things the command will not do:
- emit a regime the engine ships.
openregs init-regime fixregexits 2 and writes nothing.fixregis a fixture, not data: the spec, the tooling, the fixtures and that regime form one verification boundary and a release is a function of one commit across all four. It stays in the engine permanently. - create the repository inside the engine checkout. A regime repository has its own remote, its own releases and its own reviewers; nested in the engine it is none of those.
2. Read what it wrote
Section titled “2. Read what it wrote”Fifteen files, and the command lists every one of them as it writes it — read that list rather than this tree if the two ever disagree:
regime-eu/├── core-pin.yaml the engine commit every check runs out of├── LICENSE-DATA CC-BY-4.0 over the whole repository├── README.md├── .gitignore .openregs-core/, state/, dist/, and the per-release│ signature and eval directories a rebuild must not carry├── .github/│ ├── workflows/regime.yml validate · check-canon · graph│ └── workflows/dco.yml dco — outside the offline seal, because it asks the forge├── config/│ ├── trust.yaml whose signature this corpus's releases are verified│ │ against — ANCHORS NOBODY YET, and refuses until it does│ └── eval.yaml the retrieval eval's floors — PLACEHOLDERS, at the│ highest score there is, until somebody measures here└── regimes/eu/ ├── SCOPE.yaml SOURCES.lock DEPRECATIONS.yaml ├── graph/edges.yaml └── atoms/.gitkeep canon/.gitkeep releases/.gitkeep
# and, only when --maintainer named somebody:├── MAINTAINERS.yaml who may approve what└── .github/CODEOWNERS GENERATED from MAINTAINERS.yamlThe two files under config/ are emitted empty of claims on purpose. Both are
read from the root of whatever repository the command runs in, so a corpus needs
its own rather than the engine’s: an anchor states whose signature this
deployment accepts, and a floor is the measured behaviour of the serving plane
over this corpus. Neither could be filled in here — when init-regime ran this
repository had no release pipeline to name an identity for and had answered no
question anybody scored — and a guessed anchor is worse than none, because one
refuses and the other vouches. So both refuse: openregs verify stops on lists
no keys, so there is nothing to anchor release-meta.yaml to, and openregs eval --gate admits nothing short of a perfect answer. Each file says in its own
comments what to write there and what a reviewer of that pull request is
approving.
The question set openregs eval scores — fixtures/eval/questions.jsonl — is
not scaffolded at all. Every question is a claim about specific law, and the
engine’s own set is about the engine’s fixture regime: scored against it, a
corpus defers every question and certifies nothing.
The three .gitkeep files are not decoration. Git tracks files and not
directories, so atoms/, canon/ and releases/ would not survive the first
commit without them, and a corpus whose canon directory does not exist is one
where the first ingest creates it under whatever name it felt like.
Why regimes/<id>/ inside a repository named for that regime. Every OpenRegs
command addresses a regime as <root>/regimes/<id>/. Laid out this way, a regime
repository is checked, built, signed and released by the shipped commands with no
special case — which is the whole point of having a template. The repository root
is still the corpus root for licensing, which is why LICENSE-DATA sits there
rather than under regimes/.
Read SCOPE.yaml properly before anything else. It is the charter: what the
regime covers, and how far ingestion follows citations out of it. The default
citation_depth: 1 reaches the acts a seed cites for its definitions; each
further hop multiplies the frontier.
Its description: is an instruction to you, not a charter. It is prose, so
no schema can tell one from the other — check-regime gates it the way it gates
the UNSCOPED seed: legal while the corpus is empty, refused once it holds any
text. It is the one sentence a reader uses to decide whether this corpus answers
their question, so write it before the law arrives, not after.
3. Commit it and push it
Section titled “3. Commit it and push it”cd regime-eugit init -b main
# Say who you are, HERE. A fresh `git init` inherits the global identity, not# the engine checkout's — that checkout has a local `user.email` and this# directory is not in it. Two things go wrong if you skip it: DCO is checked on# the address, so the sign-off fails on the forge rather than here; and the# global identity on a machine that has worked on this project is often a# personal address, in which case the corpus's first commit publishes exactly# the kind of address the engine's own history rewrite existed to remove.# Unlike a wrong roster this is not cheaply fixable — it is in the commit.git config user.name "<your name>"git config user.email "<the address the roster names>"
git add -Agit commit -s -m "regime: the eu corpus, empty and checked"-s is not optional: contributions are DCO-signed
(CONTRIBUTING.md).
Then push, and which command depends on whether the repository already exists — normally it does, because creating repositories in the org needs rights the person standing up a corpus may not have:
# The repository already exists (the usual case):git remote add origin https://github.com/openregs/regime-eu.gitgit push -u origin main
# Only if you are creating it yourself, and it does NOT exist yet:gh repo create openregs/regime-eu --public --source=. --pushgh repo create fails outright against an existing repository — it is a create,
not an upsert — so reaching for it out of habit costs a confusing error at the
one moment the corpus is not yet anywhere.
CI runs on that first push and must be green while the repository is empty. If it is not, the template is wrong — fix it in the engine (this runbook’s whole premise is that standing up the second regime takes no bespoke work), not in the repository it produced.
4. Give it reviewers
Section titled “4. Give it reviewers”A new repository is unowned, and that is deliberate. Run without
--maintainer, init-regime writes no MAINTAINERS.yaml and no CODEOWNERS at
all. It does not invent anyone and it does not borrow anyone: a roster is a
statement about people, and the only thing that can make that statement is a
person. openregs check-regime refuses this corpus the moment it holds any
canonical text or atoms while it is unowned, so the gap closes before the first
line of law and not after.
Two failure modes to avoid, and they look identical from the outside — every
CODEOWNERS rule resolves to nobody, silently:
- a team that does not exist;
- a team that exists somewhere else. The forge honours a team as code owner
only in repositories that team has write access to.
@openregs/core-maintainershas it on the engine and not here, which is whyinit-regimerefuses the engine’s own teams outright andcheck-regimerefuses a committed roster that names one.
So:
-
Create
regime-eu-reviewerson the forge and give it write access toopenregs/regime-eu. Creating the team is not enough; the grant is the part that makes it own anything. -
Put the domain reviewers in it — the people who can read the law this corpus holds, not the people who built the pipeline. An obligation atom is a machine’s reading of a legal text, and whoever says yes to it should know the law.
-
Write
MAINTAINERS.yamlnaming them, or start it from one person at creation time:Terminal window uv run --project openregs openregs init-regime eu \--maintainer <handle> --maintainer-name "<Name>" --maintainer-email <email>One person is a starting point and not a governance model.
-
Generate
CODEOWNERSand commit both. You are insideregime-eu/by now (step 3), and the engine checkout is its sibling —.openregs-core/does not exist until step 6, so it is not what generates this:Terminal window uv run --project ../openregs python -m openregs.governance --root .wrote .github/CODEOWNERS from MAINTAINERS.yaml--root .is the corpus and--projectis the engine, and they are two different questions — the same split step 6 turns on. Naming the corpus for both isModuleNotFoundError: No module named 'openregs', which is what this line said to run before it was executed.Never edit
.github/CODEOWNERSby hand. It says so at the top, and CI fails when it and the roster disagree — the workflow runs the same module with--check, out of.openregs-core, which by then exists.
Where the roster will live. The direction of travel is one org-level roster
in the engine generating outward into every regime repository, rather than one
per corpus. The generator already takes --root, and the pinned engine checkout
is on disk in CI, so that is a change to where the roster is read from and not
to how ownership is expressed. Whatever it reads from, it must never read the
engine’s people: — part of that roster is a test fixture, and inheriting it
publishes invented maintainers into a public corpus of real law. That has
happened once.
5. Protect the branch
Section titled “5. Protect the branch”On openregs/regime-eu, require pull requests into main, and require the three
contexts the workflow publishes: validate, check-canon, graph. Require
code-owner review — that is what makes an atom change need a domain reviewer
rather than any maintainer with write access.
Renaming a job in regime.yml silently removes a required check. Add jobs;
do not rename these three.
6. Ingest
Section titled “6. Ingest”Clone the pinned engine beside the corpus once, and run the crawl from the corpus:
git clone https://github.com/openregs/openregs .openregs-coregit -C .openregs-core checkout "$(uv run --project .openregs-core python -c \ 'import pathlib, yaml; print(yaml.safe_load(pathlib.Path("core-pin.yaml").read_text())["ref"])')"
uv run --project .openregs-core openregs ingest \ --regime eu --offline-fixtures --quarantine quarantine/--quarantine is not optional on a crawl, and leaving it off fails the run.
A document one of the four gates refuses has to go somewhere, and told nowhere the
crawl fails the instrument rather than dropping it quietly:
32024R0001: fetched (depth 0, seed), sha256 38290b429609, NOT normalized — 32024R0001: 1 of 4 gate(s) failed: reference-resolution no quarantine directory was named, so nothing holds it for review; re-run with --quarantine <path>FAIL /…/regimes/eu/SCOPE.yaml: 1 in-scope instrument(s) did not reach the canonexit 1, and one refusal takes the whole crawl with it. With the flag the same run
holds the document and succeeds — 1 quarantined, an item under
quarantine/eur-lex--32024R0001, and a pointer at
quarantine-triage.md, which is where that item is worked.
The path is inside the corpus on purpose: an item becomes a branch on this
repository. Commit quarantine/ with the rest.
Parse the pin, do not match it. core-pin.yaml is YAML and ref: is a
string, so the value is read by a YAML parser — the clone above already carries
one, and asking it is cheaper than being right about quoting. A sed matching
^ref: "\(.*\)" reads today’s template and returns the empty string for an
unquoted ref:, at which point git checkout "" fails with empty string is not a valid pathspec and the corpus is checked by whatever commit the clone landed
on. Nothing in between says a pin was not applied.
No --config: which sources exist, how politely they are asked, and which
recorded fixtures replay them are the engine’s statements about itself, read from
the engine this corpus pins. Naming one explicitly still wins, and a corpus never
needs to.
--offline-fixtures is not optional today and the command says so: resolving an
instrument id to a live regulator URL arrives with the per-source adapters. Until
then a scope crawl replays recorded feeds.
The corpus is the root, and the engine is not. A regime repository is a
repository root in its own right (regimes/ beside core-pin.yaml is what says
so), so --regime eu reads regimes/eu/SCOPE.yaml here, writes canon here, and
appends to SOURCES.lock here. Everything the engine owns — the schemas, the
taxonomies, config/sources.yaml and the fixtures it names — comes from the
engine build, wherever the command was run. state/ is written beside the corpus
and is gitignored.
Those are two different questions with two different answers, and answering the
second with the first is what used to stop this section working at all: run from
a corpus, the command looked for config/sources.yaml and then the engine’s
fixtures/ inside the corpus, and stopped at
error: no source configuration at /…/regime-eu/config/sources.yamlThat is fixed for the crawl, and only for the crawl. A scope crawl resolves
config/sources.yaml and every offline_fixtures path it names against the
engine-owned file that declares them, so the command above runs from a corpus and
writes the corpus’s canon. The poll path — ingest --source <id> — still
resolves its instrument catalogue against the run root, one _catalogue_for per
adapter, and a corpus is the run root. It never gets that far today for a
different reason (below), which is the only thing keeping that from being the
error a reader sees.
Either way, do not resolve it by giving a corpus a fixtures/ directory or a copy
of the engine’s config. sources.schema.json pins those paths to ^fixtures/…
and that is correct — they are engine-relative, and the fix is to resolve them
against the engine.
What the crawl does from a corpus, and what still does not work
Section titled “What the crawl does from a corpus, and what still does not work”The crawl itself works. Against a corpus whose scope declares an instrument a
shipped catalogue lists, it captures, maps, gates and writes canon in the corpus.
This is a scratch regime-demo seeded with a FIXREG instrument rather than the
regime-eu above, for the reason the first row of the table below gives — a real
seed is the one thing no shipped catalogue can resolve:
32024R0001: fetched (depth 0, seed), sha256 860a67521d58 -> /akn/eu/act/reg/2024/1/eng@2024-06-01 (12 units) 32024R0099: fetched (depth 1, cited by 32024R0001), sha256 f9d23c753385 -> /akn/eu/act/reg/2024/99/eng@2024-04-01 (3 units) 32024R0042: out-of-scope (depth 2, cited by 32024R0099) — 2 citation hops from a declared instrument, beyond the configured depth of 1ingest: regime demo: citation depth 1, 2 instruments in scope and captured, 1 held out beyond the depth, 2 Expressions written to canonWhat is blocked is a corpus of real law, and it is one thing wearing two faces:
| what | why it blocks a corpus |
|---|---|
| every shipped catalogue is a fixture catalogue | the crawl resolves an instrument id through the catalogue its source configures, and the five shipped sources configure four fixture catalogues between them. A SCOPE.yaml naming source: eur-lex — as the example in step 1 does — resolves through fixtures/scope/fixreg-instruments.catalogue.json, so a real seed comes back 32024R0575: unresolved (depth 0, seed) — nothing resolves it to a URI; fixreg-instruments.catalogue.json lists 4 instruments and not this one, and the run fails the instrument. Resolving an id to a live regulator URL arrives with the per-source adapters |
| no corpus can run the poll path at all | not “not this source”: resolve_sources() binds every configured entry before --source selects anything and before enabled is consulted, and eur-lex is the first. It is bound by discovery.regimes: [fixreg] — as are uk, us and gazette — so the run dies at error: eur-lex: cannot bound discovery by regime 'fixreg': no scope declaration at /…/regime-eu/regimes/fixreg/SCOPE.yaml, exit 2. --source uk prints that. So does --source fixreg-local, which declares no regime at all. Which regimes a source serves is a fact about a corpus, living in an engine-owned file |
Both are engine-side and neither is fixable from inside a corpus. Do not work
around them there: the bar this project set for replicating a regime is that the
next one stands up with zero changes to the tooling — met three times now,
regime-eu, regime-uk and regime-us-cfr — so a workaround downstream is a
lie told to that criterion.
What to do meanwhile: nothing puts a real act in a corpus’s canon today, and
this page is not going to pretend otherwise. In particular it is not
fetch-expression, which this section used to send readers to. That command takes
a Work IRI and a language, not a URI, and it captures a sibling Expression of
a Work the canon already holds — so on an empty corpus it refuses, exit 2:
error: no regime among eu holds /akn/eu/act/reg/2024/575 in its canon; ingest the Work first, or name the canon directory with --canonIt is the command for reaching the French text of an act you already have, and it cannot take a first capture by construction.
The one real act this project has captured was taken through the poll path from
the engine checkout, with the eur-lex source’s discovery.regimes pointed at a
regime that declares that CELEX number —
eurlex-capture.md walks it end to end, and its §1 is where
that binding is set. That is the same engine-owned binding the
table above names, which is why it is a thing done to the engine and not something
a corpus can do for itself. The result lives in captures/eurlex-32024R0575/ as
evidence, not canon: nothing reads it, and it would not clear the gates if
anything tried.
What is proven to work from a corpus, because a test runs it there: the scope
crawl (tooling/tests/test_corpus_ingest.py), the crawl holding a gate-refused
document in the corpus’s own quarantine — --quarantine reaches the crawl now,
see quarantine-triage.md
(tooling/tests/test_scope_quarantine.py) — and the
three jobs the emitted workflow publishes, validate, check-canon and graph,
executed against an empty corpus (tooling/tests/test_regime_template.py); and,
since this wave, release build from a corpus with a populated canon
(tooling/tests/test_corpus_release.py) — which found a real defect on its way in,
the engine’s config/embeddings.yaml resolved against the corpus, so a corpus’s
release shipped fixture vectors under whatever model the deployment had named.
Consolidate and atomize from a populated corpus are still run by nothing, here or in the suite. Step 7 is written from the engine’s behaviour, so treat a surprise there as a defect in the engine and report it rather than patching the corpus around it — the embedding defect above is what that looks like when it is done properly.
7. Consolidate, atomize, release
Section titled “7. Consolidate, atomize, release”Consolidation addresses a canon directory directly and needs no root:
uv run --project .openregs-core openregs consolidate \ regimes/eu/canon/<work>/<expression> --amendment <path>uv run --project .openregs-core openregs atomize --regime eu --root .Atom proposals go through review: see the atom-review runbook. Building and
signing a release is release-build.md and
release-sign.md, unchanged — a
release from a regime repository is a release like any other, and it records this
repository’s commit and the engine pin it was built under.
Record the published release in index/combinations/ in the engine, which is
where a consumer finds out which releases are meant to be used together.
Moving the engine pin
Section titled “Moving the engine pin”core-pin.yaml and env.CORE_REF in the workflow hold the same commit. A
workflow cannot read a file to decide what to check out, which is the only reason
the value appears twice; openregs check-regime fails when the two disagree, and
the workflow additionally checks that the checkout it produced is that commit.
So a bump is one pull request touching two lines, and it is a real change: it moves what every check in the repository means. Read the engine’s changes between the two commits before merging one. Never bump it in passing with other work.
When something is wrong
Section titled “When something is wrong”| Symptom | Cause |
|---|---|
check-regime fails on fixture-boundary | the repository holds a regime the engine ships. This is the guard against migrating fixreg out of the engine; it is not a check to work around |
check-regime fails on scope | text landed against the UNSCOPED placeholder, or against the charter nobody wrote. Say what the regime covers |
check-regime fails on ownership | text landed while the corpus has no roster, or the roster names a team scoped to the engine. Give it reviewers the forge can resolve here |
check-regime fails on workflow | the pin file and the workflow disagree. One of the two lines was bumped alone |
check-regime fails on readme | README.md names fewer than all the required contexts, or gives an engine sha the pin does not. A repository scaffolded before the sign-off gate joined the template has no context table at all: copy it in from a freshly generated one. A stale sha means the pin was bumped in core-pin.yaml and the workflow and not here, so the commands the README calls “the same commands CI runs” are not |
the graph job’s diff is non-empty | graph/edges.yaml is stale. Rebuild it and commit the result |
CODEOWNERS matches MAINTAINERS.yaml fails | somebody edited the generated file. Regenerate |
| a step fails with a network error | it should. Every step runs sealed to loopback; a check that needs a regulator’s endpoint is a check in the wrong place |
Rendered from openregs/openregs@f3a2d10:docs/runbooks/new-regime.md