OpenRegs — Architecture

This document is the component-and-connection reference for OpenRegs. Where OVERVIEW.md tells the story of the six planes and GLOSSARY.md defines the vocabulary, this file enumerates every component and every connection between components, so an implementer can check a piece of work against a single list: is the thing I built one of these boxes, and does it consume and emit exactly the arrows named here?

Three sections follow:

• Components — every part of the system, one bullet each, grouped by plane.
• Data-flow edges — an explicit edge list, each written as source -> destination: what flows.
• Repository boundaries — which repository each part is developed and released from, and why the line falls where it does.

regulator sources
      │
      ▼
watcher adapter ── scheduler ▸ fetcher ▸ raw snapshot store
                                              │  (replay re-reads this store)
                                              ▼
                              parser ▸ AKN mapper ▸ validation gates
                                                        │
                                       pass ◂───────────┴───────────▸ fail
                                        │                              │
                                        ▼                              ▼
                                   canon inbox                    quarantine
                                        │
                                        ▼
                     consolidation engine ◂ amendment parser
                                        │
                                        ├──▸ atomization pipeline ▸ human review
                                        ▼                                │
                                   canon repos ◂── merged atom PRs ◂─────┘
                                     (CI checks)
                                        │
                                        ▼
                            release builder ▸ signing chain
                                        │
                          ┌─────────────┴─────────────┐
                          ▼                           ▼
                 hosted API + MCP server         local mirror (CLI)
                          │                           │
                          └─────────────┬─────────────┘
                                        ▼
              bank CI bots · copilots · audit tooling · coverage reports

Components

Regulator sources

The outside world. Each is an official publisher; none is under OpenRegs’ control, and no test or CI job may contact one.

• EUR-Lex — the EU’s official law portal. Publishes OJ acts identified by CELEX (e.g. 32024R0001) in Formex XML and XHTML, with consolidated versions for amended acts.
• legislation.gov.uk — the UK’s official statute book. Publishes Acts and statutory instruments as CLML XML with point-in-time versions.
• eCFR — the Electronic Code of Federal Regulations: the continuously updated consolidated text of the US CFR, with title/part XML and an amendment-date index.
• Federal Register — the daily US journal carrying proposed and final rules before they land in the CFR; the source of US amendment events and their effective dates.
• National gazettes — member-state and other national official journals, many of which publish scanned PDF only and offer no structured feed.

Watcher adapter internals

One adapter per source, all implementing the same five-member SourceAdapter protocol (source_id, discover, fetch, parse, normalize). The adapter is split by purity: the network-facing side may touch the network, the clock and randomness; the pure side may not. That split is what makes replay possible.

Network-facing side:

• Scheduler — decides when each source is due, reads the poll cadence from config/sources.yaml and the per-source cursor from state/cursors/, and drives discover()/fetch(). The only component that owns wall-clock time.
• Fetcher — fetch(): retrieves the exact bytes for a discovered reference, handling auth, retries, rate limits and conditional requests. The only component that speaks HTTP to a regulator.
• Raw snapshot store — content-addressed object store (S3-compatible interface, filesystem driver for dev and tests) keyed by sha256 of the fetched bytes, holding the immutable evidence base: original payloads stored once each, one WARC record per capture event beside them, and the digests pinned in SOURCES.lock. Its integrity scan (openregs verify --snapshots) re-reads every object and re-derives every digest rather than trusting an index.

Pure side:

• Parser — parse(): turns snapshot bytes into a source-native tree (Formex, CLML, eCFR XML, or an OCR layout tree). Pure: no network, no clock, no randomness.
• AKN mapper — normalize(): emits Akoma Ntoso XML with stable eIds, plus a mapping report listing every source element it did not handle and, for scanned sources, an OCR confidence score. Pure and deterministic; stamped with a transform_version.
• Validation gates — the four checks standing between adapter output and the canon: (1) XSD validity against the vendored AKN schema; (2) round-trip text integrity under spec/whitespace-policy.md; (3) structural coverage above the configured threshold; (4) cross-reference resolution rate. Gates decide the document’s outcome; they never repair it.

The two outcomes:

• Canon inbox — staging directory of gate-approved AKN units awaiting consolidation. Everything that reaches the canon passes through here.
• Quarantine — queue of documents that failed a gate or fell below the OCR confidence threshold. Each item becomes a pull request bundling the snapshot reference and the failure/mapping report; a maintainer either fixes the mapper (then replays) or hand-annotates with a human_authored override. The decision rule is where the defect is — in our transform, or in the document the publisher served — and closing an item re-derives the answer rather than accepting an assertion: docs/runbooks/quarantine-triage.md.

Consolidation

• Amendment parser — reads amending acts and extracts structured amendment instructions (replace / insert / delete, target eId, old and new text) with their effective dates, classifying each as substantive or editorial.
• Consolidation engine — applies those instructions to base texts to produce point-in-time Expressions with validity windows, recording which instruments produced each one so an as-of query resolves a date to exactly one text. openregs consolidate <base-expression-dir> --amendment <act> is the command. Four properties are worth knowing before reading the code (openregs/consolidate/engine.py, and run.py for the part that touches the repository):
  • the effective date is derived, never declared. The new Expression is addressed at the date the amending act’s own Expression IRI carries, because an amending act’s Expression date is its entry into force. A date on the command line could address a text to a day no act commenced on.
  • the engine works on a whole act, then the canon writer splits it again. An insertion lands between two units and a cross-reference in one unit points at another, so a per-unit edit cannot express a consolidation. The reassembly exactly inverts the split, which is what lets a run rewrite three units and leave nine byte-identical.
  • publisher_verified is evidence, not a default. Where the regulator publishes its own consolidated text, the engine compares unit by unit and marks the agreeing units verified — two independent consolidations reaching the same words. Without one, or where they disagree, the unit is computed and carries a confidence. Verification records something about a text; it never changes one, and the bytes are identical either way.
  • each unit traces to the payload that determined it. A unit no operation touched carries the base capture’s digest; a unit an amendment rewrote or created carries the amending capture’s, because the words that make this version different are literally in those bytes.

Atomization

• Atomization pipeline — an LLM proposes obligation atoms (actor, modality, action, conditions, exceptions, executable blocks) over consolidated text, each carrying a provenance span whose quoted_text must equal source_text[char_start:char_end] byte-for-byte.
• Human review — maintainers accept, correct or reject every proposed atom in a pull request. The model proposes; it never decides. Only merged atom PRs are canon.

Canon

• Canon repos — the git repositories under regimes/<regime>/ holding consolidated texts, atoms, the cross-reference graph, SCOPE.yaml, SOURCES.lock and DEPRECATIONS.yaml. Human-auditable ground truth.
• CI checks — the gate on every push and PR to a canon repo: schema validation, provenance (quoted_text equality against SOURCES.lock-pinned text), ID permanence (no reused atom id or eId; retirement is status: deprecated + superseded_by + a DEPRECATIONS.yaml entry), and graph integrity.

Release

• Release builder — compiles the canon at one git commit into a date-tagged release directory plus release-meta.yaml (git commit, schema version range, build timestamp derived from the commit, artifact digests). Deterministic: same commit ⇒ byte-identical artifacts.
• Schema compatibility gate (tooling/openregs/compat.py) — the one place the two version axes meet. spec/ versions by semver in spec/VERSION; content releases version by date tag. A release records the spec version at the commit it was built from — never the checkout’s, so a spec bump can neither restamp nor invalidate a release already published under an earlier one — plus the spec_schema_range of consumers that can read it, derived from the major because minors are additive. Every consumer declares SUPPORTED_SCHEMA_MAJORS and refuses anything outside it, naming both versions and pointing at schema-compatibility.md. The gate runs before digests, signatures and any bundle read, including under --insecure-skip-verify, and the builder applies the same rule in reverse: it will not build a canon whose spec major it does not implement.

Its five artifacts:

• SQLite bundle (corpus.sqlite) — units, atoms, edges, validity windows and the amendment modifications behind each consolidated version, for offline query. Its schema and the queries it is meant to answer — including the canonical as-of query — are documented and doc-tested in docs/sqlite-queries.md.

• Graph index (graph.idx) — the cross-reference edge list in the serving engine’s load format.

• Embedding chunks (embeddings.parquet) — one row per retrieval chunk, and a chunk is one legal unit version (an article or a paragraph), never a fixed token window: a hit is an address with a validity window, so the serving layer’s temporal filter is a comparison rather than a guess. Every row carries eid, work_iri, jurisdiction, regime, entity-type tags, valid_from/valid_to, the corpus tag, the chunk text and the vector. The model is pluggable and named in config/embeddings.yaml, resolved before the build reads anything; the deterministic fixture-hash embedder is the one this repository ships, so CI is network-free, and a deployment registers its own from a plugin module.

• Codegen constants (constants-src/) — generated source for every atom executable block, e.g. incident_report_threshold_eur.

• Diff manifest (<tag>.diff.json) — unit- and atom-level changes versus the previous tag, each classified substantive or editorial. Editorial only when the amendment parser marked the operation editorial — the amending act’s own recital, recorded at consolidation time and carried in the bundle — or the texts are equal under the whitespace policy. openregs diff <regime>@<a>..<regime>@<b> --json computes the same document from two built releases.

• Release range (tooling/openregs/release/aggregate.py) — openregs diff’s reader-facing side. Without --json the changes are grouped by classification, substantive first, each with its citation and the words on both sides. A range spanning several release moves is aggregated from the per-release manifests, one step per published tag, with every entry naming the release that reported it; a one-step range emits that release’s own manifest unchanged. A chain that the published release-meta.yaml files do not agree on is refused rather than double-counted.

• Signing chain (openregs release sign) — writes attestation/ beside the artifacts and never inside the build, because a build is reproducible and a signature is not. Three parts:

  • a signature per file, in cosign’s key-based sign-blob format — ECDSA-P-256 over the blob, DER, base64, with a PEM public key — produced either by the cosign binary as a subprocess or by the in-process signer, which emit the identical artifact;
  • a SLSA provenance statement (attestation/provenance.json, an in-toto v1 Statement with a slsa.dev/provenance/v1 predicate) binding the tag to the git commit and regime tree, to the CI run id, and to every source snapshot sha256 — the whole SOURCES.lock ledger travels inside the document, because a consumer holding a bundle has no checkout to consult;
  • transparency-log entries in Rekor’s hashedrekord shape, appended to an RFC 6962 Merkle log (local implementation for tests and self-hosted deployments), with each entry’s inclusion proof and the log’s signed checkpoint bundled into the release.

• Verification (openregs verify --release <regime>@<tag>) — signatures, digests, the provenance chain and log inclusion, all checked entirely offline against the bundled checkpoint. The trust anchor is --key or config/trust.yaml, never the release’s own copy of its public key. Every check runs and the report is the union, so a tampered release names all the ways it is wrong; a single problem is a non-zero exit. Verified before any release is unpacked or served.

• Distribution (openregs pull <regime>@<tag> --from <registry>) — the consumer’s end of the chain. A registry is a file tree laid out as <registry>/<regime>/<tag>/, never an archive, so nothing is ever extracted; the file list is derived from the release’s own signature manifest rather than from a listing the registry composes. Every file is staged into a sibling of the target, the five checks above run over the staged copy, and only a report with no problem at all is followed by the single os.replace that installs it. A refused release leaves the target directory untouched, which for a first pull means empty. Pulling twice reads one file, re-checks what is on disk and writes nothing.

Serving

• Hybrid retriever (openregs.serve.retrieval) — candidate generation over a release’s chunk index: BM25 over the chunk text and cosine similarity over the chunk vectors, fused by a weighted sum of each retriever’s min-max normalised scores. Filters — as_of, jurisdiction, regime, language, eId, entity type — are hard constraints applied before any score is computed, so a chunk whose validity window excludes the date or whose tags fail the request is absent from the result set rather than ranked low. Entity types are read through spec/taxonomies/entity-types.yaml, so a duty on operators binds a small operator and a duty written only for small operators does not answer a request about operators. Everything below answers through it.
• Retrieval closure (openregs.serve.answer) — the answering primitive, and the reason an isolated text match is never returned. Each retrieved unit is expanded with its structural ancestry (the eId is a path, so the ancestors are its __-prefixes), the definitions in scope for it on the date (defines edges, both endpoints in force), whatever amends it (modifies, gathered by provision because an amendment names the version it operated on) and whatever disapplies it (exempts), plus the obligation atoms written into it. Every content block carries an eId or an atom id and the release tag. Applicability against an entity profile is decided by openregs.profiles, the same engine openregs coverage reports from; the profile is a lens over the verdict and never a filter over the content, so a duty that does not bind the caller is still in the answer, reported as disapplied with the derogating provision cited. The applicability section is a statement about the population, not about the ranking: it lists that profile’s whole disposition on the date rather than only the duties retrieval surfaced, and each entry says (retrieved) whether the answer also quotes the duty’s words. A request that names no profile still gets the duties an in-force derogation operates on, each applicable: null, because a derogation withheld is the trap the closure exists to close.
• Hosted API — the public REST service (/v1/answer and friends) answering against a pinned release with retrieval closure and citations. The deployment policy in front of it is openregs.serve.gateway, configured by config/serving.yaml: bearer-token authentication (anonymous access only where the file says so, never by omission), per-token rate limiting over a rolling minute returning 429 with Retry-After, CORS for browser clients, and one error schema — {error: {code, message, request_id}} — on every failure path from every layer, with that request_id on the X-Request-Id header and in the log line. The socket binds before the release is verified, so GET /healthz (liveness) answers 200 from boot while GET /readyz (readiness) answers 503 until a verified release is loaded; until it is, every other route answers 503 too, so a bound listener that is not ready serves no regulation. SIGTERM turns readiness off, stops accepting, and drains the requests already in flight to the last byte on the wire. Every request is also timed and counted into openregs.observe.metrics and logged as one JSON record in spec/schemas/log-record.schema.json’s schema; GET /metrics renders the registry in the Prometheus text exposition format, from the moment the socket is bound, never rate limited, behind the same authentication as every other route.
• MCP server — the same codebase over the Model Context Protocol (openregs serve --mcp, stdio), exposing five tools — search_regulation, get_unit, get_atom, diff_releases, list_obligations — so AI agents query the corpus directly. The tools are openregs.serve.tools.ToolService, the object the REST routes call, so an MCP payload and its REST twin are one document produced once; every result is built in one place (envelope) and carries the release tag beside the payload.
• Local mirror — the identical stack run inside a consumer’s own network from a pulled, verified release (openregs serve --release <tag>, or --path <dir> for a release vendored outside a checkout), the air-gapped path. Hosted and local are pure projections of the same artifact and must answer identically. The verification gate is openregs.serve.startup, which both doors call: a signed release goes through the five checks openregs verify runs, this checkout’s own unsigned build has its digests and provenance checked, and either way a release that fails a check is refused with the failing check named — before anything is served and before the MCP transport is offered. --insecure-skip-verify is the only way past it, and every response then carries verification: skipped, as every response carries what was checked.
• Retrieval eval (openregs.evaluate, openregs eval) — the adversarial benchmark that gates releases. It asks a serving instance built on one release every question in fixtures/eval/questions.jsonl that release may be asked — exemption traps, as-of dates, cross-regime near-misses — scores the answers per category against the floors in config/eval.yaml, and writes <release>/eval/report.json. It reads only what an answer says: its citations, its prose and the verdict they amount to. With --gate a category below its floor is exit 1, which is what leaves publish unrun in release.yml. See runbooks/eval.md.
• CLI — the openregs binary (PyInstaller, per platform): pull, verify, serve, text, diff, impact, coverage, compile, eval, index, plus the pipeline-side ingest, replay, atomize, graph, release, validate. It is how humans and CI touch every plane.

Observability

Not a seventh plane — a cross-cut. Every plane above already produces the two things an operator reads, a narrative of what happened and a measurement of how much, and the observability plane is the decision to produce them in one shape rather than one per plane. tooling/openregs/observe/ is that shape’s only implementation, so a component emits a log line by rendering the record type, never by formatting a string of its own.

• Log records (openregs.observe.log) — one JSON object per line on standard error: six keys, always all six — timestamp, level, component, request_id, message, fields. The contract is spec/schemas/log-record.schema.json, versioned in spec/ like every other schema and checkable with openregs validate log-record; LogRecord.render is the only thing in this repository that produces a line, so the emitter cannot drift from the schema. A key is never omitted — an event belonging to no request carries request_id: null, which is a statement rather than an absence — and everything event-specific goes in fields rather than being interpolated into message, so searching for a message finds every occurrence of it. Standard output stays prose, which is what lets openregs serve 2> server.log yield a file every line of which parses while a human still reads sentences.
• Metric registry (openregs.observe.metrics) — counters, gauges and histograms declared in one module and rendered in the Prometheus text exposition format, so dashboards and alert rules written against a real Prometheus work unchanged. Rendering sorts by name and then label set, so two scrapes of one state are byte-identical for the same reason every other artifact here is.
• Request id — what joins the two into a trace. Minted (or adopted) per request by the serving gateway, returned on X-Request-Id and in every error body, and bound to a context variable so a component three frames down carries it without being handed it. Deliberately not in a successful response body: two instances of one release must answer byte for byte, and an id in the document would end that.

The two planes it attaches to differently are worth naming, because the shape of each decides how it is watched. Serving is request-shaped and has a socket, so it answers GET /metrics from the moment that socket is bound — before the release is verified, which is exactly when an operator wants numbers. Ingestion and its replay path are commands that start, work and exit, with no port anybody could scrape by the time the question is asked, so they write the same exposition format to <state-dir>/metrics/<command>.prom for a textfile collector, and a run that cannot write its metrics warns rather than fails — a corpus ingested correctly was ingested correctly whether or not the observer took notes. Consolidation, atomization, canon and release are git-shaped: what they did is a commit, a diff manifest and a release’s own metadata, which is a better record than any counter and is why they publish none. runbooks/observability.md is the operator-facing page — every metric, what to alert on, and what to do with a request id.

Downstream integrations

• controls.yaml mapping — the consumer’s private file mapping internal controls to atom ids (atom, control, owner, evidence) or explicit waivers (atom, status: not_applicable, rationale), validated against spec/schemas/controls.schema.json.
• Impact report — openregs impact --diff <A>..<B> --map controls.yaml (tooling/openregs/impact.py): the release’s own diff manifest intersected with the firm’s mapping. Exit code 1 if and only if an atom mapped to a control changed substantively; a waived change and an unmapped change are reported and never fail a build, and an editorial change to a mapped atom never does either. Ticket-ready JSON on stdout — control, owner, atom, classification, citation — with waived and unmapped as their own sections. The classification is read from the shipped manifest and never recomputed, and the document carries no timestamp, so the same bump always produces the same bytes.
• Compiled constants — openregs compile --release <tag> --lang <python|typescript> --out <dir> (tooling/openregs/codegen.py): every executable block in force on the release’s as-of date, written as a typed constant carrying its value, unit, atom id, source citation and release tag in a docstring or JSDoc block — plus a generated test beside the module that asserts each value against a release (OPENREGS_RELEASE / OPENREGS_RELEASE_DIR choose which). Compiled from the release’s own corpus.sqlite after its digests are recomputed, stamped with the tag, as-of date and commit and never the clock, so one release compiles to identical bytes every time — and a bump to a release that moved a value fails the generated test rather than changing a number nobody reviewed.
• CI action — the reusable GitHub Action (with a documented GitLab CI equivalent) that opens a release-bump PR in the consuming repo, runs openregs impact against its controls.yaml, posts the impact table as the PR body, and fails the check when a mapped atom changed substantively. .github/actions/release-bump/action.yml and docs/integrations/gitlab-ci.md are two wrappers around one command, openregs bump (tooling/openregs/bump.py): it rewrites the release: line of the consumer’s own controls.yaml — that field is the pin, there is no second pin file — on a branch pushed to the consumer’s remote, and writes the report, the body, the title, the verdict and the branch name as five files a CI job reads. The exit code is impact’s unchanged, so the check is red exactly when a mapped duty moved. docs/integrations/github-actions.md is the consumer-facing document; fixtures/consumers/ holds two consuming repositories the whole thing is exercised against offline.
• Data packages — openregs release publish --release <tag> --mode <dry-run|live> --out <dir> (tooling/openregs/release/packages.py builds them, publish.py places them): one release as openregs-data-<regime> on PyPI and @openregs/data-<regime> on npm, each wrapping corpus.sqlite, release-meta.yaml verbatim and the whole attestation/ directory, plus a manifest.json both languages read. The signature travels with the data, so a consumer who installed from a registry can check the bytes that registry served without trusting it. Both modes write the packages into a registry on the filesystem — a PEP 503 index and an npm registry directory — and only live uploads; a missing credential is a failure, never a skipped upload. Archives are stamped from the release’s own built_at, so republishing a release produces identical bytes.
• Coverage reports — the auditor-facing JSON + self-contained HTML artifact from openregs coverage, stating for every atom applicable to an entity profile at a date whether it is mapped, waived or unmapped.
• Release feeds — JSON and RSS feeds per filter (jurisdiction, regime, topic tags, entity type) plus HMAC-signed webhook subscriptions with at-least-once delivery, so downstream systems learn about a new release without polling git. openregs feed (tooling/openregs/feed/): the entries are the diff manifest’s atom changes, copied verbatim including their classification, with the entity types and jurisdiction each one binds resolved out of the release’s own corpus.sqlite. Subscriptions live in one SQLite file under state/ — the only state this system keeps that is not a function of a commit. Not to be confused with the inbound regulator discovery feeds, which belong to openregs ingest.

Data-flow edges

Every connection in the system, written as source -> destination: what flows. If an implementation produces or consumes an arrow that is not on this list, either the implementation or this list is wrong.

Sources into adapters

Each source feeds exactly one adapter, and only via discover()/fetch().

• EUR-Lex -> eur-lex watcher adapter: CELEX-identified OJ acts as Formex/XHTML, plus the publication metadata discover() compares against its stored cursor.
• legislation.gov.uk -> uk-legislation watcher adapter: CLML XML for Acts and statutory instruments, with the point-in-time version list discover() walks.
• eCFR -> us-ecfr watcher adapter: consolidated CFR title/part XML and the amendment-date index that bounds point-in-time retrieval.
• Federal Register -> us-ecfr watcher adapter: final-rule notices carrying the effective dates that drive US consolidation.
• National gazettes -> gazette watcher adapter: scanned PDF issues with no structured feed, entering the OCR and layout-parsing path.

Inside the watcher adapter

• Scheduler -> Fetcher: a due-work signal per source — poll cadence from config/sources.yaml and the cursor from state/cursors/ — after which discover() yields references and each reference is handed to fetch().
• Fetcher -> Raw snapshot store: the exact response bytes, written content-addressed by sha256, plus a WARC record of the exchange that produced them; the digest is pinned into SOURCES.lock as the provenance root.
• Raw snapshot store -> Parser: snapshot bytes read back by digest — the only input parse() is permitted, which is what keeps parsing pure and re-runnable.
• Parser -> AKN mapper: the source-native tree (Formex / CLML / eCFR XML / OCR layout tree) passed to normalize().
• AKN mapper -> Validation gates: Akoma Ntoso XML with stable eIds, plus the mapping report (unmapped elements, coverage percentage, OCR confidence) the gates score.

The replay path (the least obvious edge)

• Raw snapshot store -> Parser (replay, re-run on mapper upgrade): when parse() or normalize() code changes, every stored snapshot is re-read from the snapshot store and pushed through the new transform with zero network access; outputs are stamped with the new transform_version, only units whose output actually changed are rewritten, and a replay diff report lists them. This is the payoff of the purity rule — normalization bugs are fixed retroactively across the whole corpus without re-fetching anything from a regulator, and a quarantine item closed by “fix the mapper” re-enters the pipeline here rather than at the fetcher.
• “Output actually changed” is judged leaving the transform stamp aside. The transform’s own version is written into every document it emits, so a version bump alone makes every re-emitted unit differ from the one in canon; a replay therefore relabels the new output back to the version the stored unit carries and compares that. A unit whose only difference is its stamp keeps both its files, timestamps included, and the ones that really moved are named in the report with their old and new content hashes. SOURCES.lock is never written by a replay — nothing new is learned about where the bytes came from, and an untouched ledger is the evidence that nothing was refetched.

Gate outcomes

• Validation gates -> Canon inbox (on pass): units clearing all four gates, staged as gate-approved AKN awaiting consolidation.
• Validation gates -> Quarantine (on fail): a quarantine item bundling the snapshot reference and the failure/mapping report, opened as a triage pull request; low-confidence OCR routes here too, so a bad scan never silently enters the canon.

Canon layout, and who wins when two publishers disagree

A canon Expression is a directory, and the directory is keyed by the Work:

regimes/<regime>/canon/<work-key>/<language>@<date>/art-005.akn.xml
                                                    art-005.meta.yaml

<work-key> is derived from the FRBR Work IRI — /akn/eu/act/reg/2024/1 files as eu-act-reg-2024-1 — and never from the id the publishing document happens to carry. That distinction is the whole point. EUR-Lex calls this act 32024R0001 and legislation.gov.uk calls the same act eur-2024-1; keying on the publisher’s id put two renderings of one Expression in two directories with nothing recording that they were the same law, and a corpus that holds two accounts of a provision without knowing it has a hole in the middle of it. The Work IRI is carried by the normalized document itself, so every publisher of an act necessarily agrees on it and no lookup table stands between a citation and a directory. Which source served a given text is recorded in that unit’s meta.yaml (publisher), where a fact about provenance belongs.

Keying on the Work makes two publishers of one act collide on purpose. The rule that resolves the collision, in order:

1. Identical text is idempotent. Byte-identical AKN is accepted as a no-op and the rendering already in canon is kept, publisher and all. Re-running an ingest therefore changes no byte, which is what lets the committed canon be verified by reproduction.
2. Differing text is decided by declared precedence. The regime’s publisher_precedence in regimes/<regime>/SCOPE.yaml lists sources most authoritative first. A higher-ranked incoming rendering replaces what is there; a lower-ranked one is superseded — refused, not written — and the run reports how many units it kept out, because a run that discards a rendering silently is a run nobody can audit.
3. An unranked disagreement is refused. If either publisher is absent from the list — including a unit written by hand, which records no publisher — the write fails naming both. Ranking is a human deciding which publisher speaks for a regime; inferring it (alphabetically, first-seen, most-recent) would make the canon depend on the order polls happened to run in.
4. A publisher may not disagree with itself. Same source, same Expression, different text is refused rather than overwritten: re-capture it instead.

The consequence worth stating plainly: precedence decides which rendering is canon, not which is true. A persistent disagreement between two publishers of one act is a finding about the sources, and the right response is to look at it, not to raise a rank until it disappears.

Canon formation

• Canon inbox -> Consolidation engine: gate-approved base and amending acts with their effective dates.
• Amendment parser -> Consolidation engine: structured amendment instructions (target eId, operation, old/new text, effective date, substantive vs editorial) extracted from those amending acts.
• Consolidation engine -> Canon repos: point-in-time Expressions with validity windows and the amending instruments that produced them. The superseded Expression’s valid_to is set to the day before the new one’s valid_from in the same run, so no two versions of a unit ever claim the same instant.
• Consolidation engine -> Atomization pipeline: consolidated text plus its eId addressing and validity window, the input the LLM proposes atoms over.
• Atomization pipeline -> Human review: proposed obligation atoms with provenance spans and executable blocks, opened as pull requests for a maintainer to accept, correct or reject.
• Merged atom PRs -> Canon repos: human-reviewed atoms, which together with the consolidated texts and the cross-reference graph are the canon.
• Canon repos -> CI checks: every push and PR, gated on schema validation, provenance equality, ID permanence and graph integrity before it can merge.

Release

• Canon repos -> Release builder: the canon at one git commit — texts, atoms, graph edges, scope and sources lock — compiled into the five artifacts plus release-meta.yaml.
• spec/VERSION at that commit -> Release builder: the spec semver stamped into release-meta.yaml and into the bundle’s release_info. Read out of the commit, never the checkout, which is what decouples the semver axis from the date-tag one.
• release-meta.yaml -> Schema compatibility gate -> every consumer: the declared schema_version and spec_schema_range, checked against the consumer’s SUPPORTED_SCHEMA_MAJORS before the release is read at all.
• Release builder -> Signing chain: artifact digests, the git commit and the snapshot sha256 set, turned into cosign signatures, SLSA-style provenance and a transparency log entry.
• Signed release -> Retrieval eval: the signed artifacts served and asked the adversarial question set, producing a per-category score report beside them. The edge is one-way and it gates: a category below its floor stops the release before publishing, and nothing the eval produces enters the release itself.

Serving

• Signed release -> Hosted serving (hosted API + MCP server): a verified, pinned release directory loaded read-only; the service answers only from it and stamps every response with the release tag.
• Signed release -> Local mirror: the same artifact pulled by openregs pull and served by openregs serve, after the full chain (signatures, digests, provenance, log inclusion) is verified before unpacking.

Serving into consumers

• Serving (hosted API, MCP server or local mirror) -> Bank CI bots: closure answers, atom diffs and impact results that gate a deployment.
• Serving (hosted API, MCP server or local mirror) -> Copilots: retrieval-closure answers with citations into the canonical text, over MCP or REST.
• Serving (hosted API, MCP server or local mirror) -> Audit tooling: point-in-time text and atoms with provenance, proving what a rule said on a given date.

Observability

Cross-cutting, so these edges leave several planes and arrive at the same two places. Nothing downstream of them re-enters the system: an arrow out of the observability plane goes to an operator, never back into a corpus.

• Serving -> log stream: one JSON record per event on standard error, in spec/schemas/log-record.schema.json’s schema, each carrying the request id the gateway minted and returned on X-Request-Id.
• Ingestion (and the replay path) -> log stream: the same records under the ingest / ingest.gates components, with request_id: null, because a pipeline run belongs to no request.
• Serving -> GET /metrics: the registry rendered in the Prometheus text exposition format, behind the same bearer token as every other route and never rate limited.
• Ingestion -> <state-dir>/metrics/<command>.prom: the same format written to disk for a textfile collector, since the command has exited before a scraper could reach it.

Downstream integrations

• Diff manifest -> CI action: the atom-level change set for a release bump, rendered as the impact table in the bump PR body.
• controls.yaml mapping -> CI action: the consumer’s atom-to-control mappings and waivers, resolved against the pinned release so a changed mapped atom fails the check.
• controls.yaml mapping -> Coverage reports: the same mappings scored per applicable atom as mapped, waived or unmapped for an entity profile at a date.
• Release feeds -> Downstream subscribers: filtered atom changes as JSON/RSS entries and HMAC-signed webhook payloads, so a bump can be triggered without polling git.
• Codegen constants -> Compiled constants: the release’s executable blocks compiled into a consumer’s own source tree, with a generated test that fails the consumer’s build when the release it is run against states a different value.
• Signed release -> Data packages: the SQLite bundle, the release metadata and the attestation, installed through pip and npm rather than pulled, and verifiable from inside the installed package.

Repository boundaries

The two sections above say what the system is made of and what flows between its parts. This one says where each part is maintained: which repository it is developed and released from, and why the line falls there. It answers a different question — where does this live rather than what does it consume — and the answers do not line up plane by plane, because a plane is a stage in a pipeline and a repository is something else.

One rule generates most of the map: a repository is a verification boundary. Whatever has to be green in one commit stays in one repository. Whatever crosses a repository boundary crosses it as a pinned, versioned artifact — a schema $id, a release tag, a digest — and never as a relative path into somebody else’s checkout.

The org map

Repository	Visibility	Licence	What it holds
openregs/openregs	public	Apache-2.0 code + CC-BY-4.0 corpus	This repository: the engine. spec/, tooling/, fixtures/, regimes/fixreg, index/, and the documentation content under docs/
openregs/docs	public	Apache-2.0	The documentation site: rendering, navigation, versioning machinery, search, and guides authored there rather than here
openregs/website	private	proprietary	The marketing site
openregs/cloud	private	proprietary	The hosted product’s control plane — accounts, API keys, metering, billing — in front of an unmodified OSS data plane
openregs/regime-eu, openregs/regime-uk, openregs/regime-us-cfr	public	CC-BY-4.0	One real legal corpus each: canon/, atoms/, graph/, releases/, SCOPE.yaml, SOURCES.lock, DEPRECATIONS.yaml
openregs/openregs-js, openregs/openregs-go	public	Apache-2.0	Client SDKs over the REST and MCP surface
openregs/.github	public	—	The org profile and the community health files other repositories inherit

Two of the columns carry a decision rather than a fact. Visibility is private only where a repository holds no part of running the system: the marketing site and the control plane. Licence splits the way the artifacts split — code Apache-2.0, corpus CC-BY-4.0 — which is why one repository can carry both and a corpus repository carries only the second.

Which repository does X live in

The lookup, with the reason beside each answer, because the reason is what decides the next thing somebody adds. core below is this repository, openregs/openregs.

Thing	Repository	Why there
spec/ — schemas, taxonomies, the vendored AKN XSD	core	A schema and the code that enforces it are one claim, and the claim is checked at one commit. spec/ is nonetheless consumable alone: schemas are addressed by $id, so a consumer validates against them without depending on tooling/
tooling/openregs/ — every line of pipeline, CLI and server code	core	It is one determinism contract with one implementation. A second implementation of a mapper or a builder would be a second answer to “what are the bytes of this release”
fixtures/ — the offline ground truth	core	Fixtures are the spec. They state what the code must do, so they move in the same commit as the code that satisfies them
regimes/fixreg — the synthetic FIXREG corpus	core, permanently	It is a fixture, not data — see below
A real EU, UK or US corpus — its canon, atoms, graph and releases	regime-<jurisdiction>	Different reviewers, different cadence. A domain reviewer approving an atom about EU law should be reviewing EU law, and a corpus should be releasable the week the law moved without an engine change standing in front of it
index/combinations/ — pinned cross-regime release sets	core, for now	It is the one artifact that spans regimes, and it names releases rather than containing them. Whether it graduates to its own repository is a decision for when two real corpora publish
Documentation content — this file, OVERVIEW.md, GLOSSARY.md, the runbooks	core	A runbook is right or wrong about the code beside it, so it changes in the commit that changes the code. The docs site pulls this directory at build time and never forks it
The documentation site — theme, navigation, version switcher, search index	docs	Rendering is a web application with its own dependency tree and its own deploy cadence, and none of it is part of running OpenRegs
The marketing site	website	Positioning copy is not a component of the system and is the one thing here with no reason to be public
The cloud control plane — accounts, orgs, API keys, metering, billing	cloud	It is the commercial half, and it is the only commercial half
The data plane the cloud runs	core	It is openregs serve over a published release. The cloud pins a version of it; it does not carry a copy
The JS and Go SDKs	openregs-js, openregs-go	They version with the published API surface, not with the engine, and they ship to ecosystem registries on their own tags
Data packages (openregs-data-<regime> on PyPI, @openregs/data-<regime> on npm)	built by core, from a regime’s release	openregs release publish wraps an already-signed release; the package is a distribution channel for bytes that exist, not a source tree

Why the engine is one repository

spec/, tooling/, fixtures/ and regimes/fixreg are a single verification boundary and are never split from each other. The reason is concrete rather than stylistic: the repository’s checks are all claims about two directories at one commit.

• make check-canon runs the canon rules — schema validity, id permanence, temporal overlap, provenance spans, sources-lock integrity — which is spec/, tooling/ and a corpus checked against each other.
• make conformance re-runs every adapter’s pure side over every captured document and compares the Akoma Ntoso byte for byte against fixtures/goldens/, which is tooling/ checked against fixtures/.
• The standing releases under regimes/fixreg/releases/ are rebuilt from the shas their own release-meta.yaml records and compared byte for byte, which is tooling/, spec/ and the corpus checked against each other and against their own history.

Split those across repositories and each claim becomes a claim about two commits in two histories. Nothing can make that true at merge time — only afterwards, and only for whoever remembers to look. The split the org map describes is therefore a split of corpora and consumers away from the engine, never a split of the engine.

fixreg is a fixture, not data

regimes/fixreg is the synthetic FIXREG corpus — twelve articles, one amending act, a derogation, a typo fixed editorially, standing releases with known as-of answers. It stays in this repository permanently and does not graduate to a regime-* repository when the real corpora are stood up.

The reason is what it is for. FIXREG is not a small legal corpus that happens to be ours; it is the executable statement of what the engine must do, and the tests that assert on it are the specification of the pipeline. Moving it out would put that specification behind a release pin and a fetch, and leave it free to drift from the code it exists to hold still. It is also part of why the test suite is offline by construction: a corpus that ships in the tree is a corpus no test has to go anywhere for.

The rule that follows is worth stating plainly, because it is easy to violate by accident and it looks harmless when you do: regimes/ in this repository contains exactly fixreg and its LICENSE-DATA. A real regime added here is a mistake even when it validates cleanly and even though the licence would be the same — the boundary is not about licensing, it is about who reviews the corpus and what has to be green before it can move. Real corpora live in their own repositories; this one holds the fixture.

The code↔data contract

A corpus and the engine that reads it sit in different repositories, released on different schedules by different reviewers. Nothing versions the pair, and deliberately so — a version of the pair would make every corpus update an engine release. What keeps them compatible instead is four pins, each written into an artifact by a program rather than into a README by a person.

• schema_version and spec_schema_range — the shape. Every release’s release-meta.yaml records the spec version it was built under, read out of spec/VERSION at the commit it was built from and never from the checkout doing the building (openregs.release.source.spec_version_at). Beside it, spec_schema_range states which spec versions may read the release, derived from that version’s major by openregs.compat.range_for: floor is the first version of the major, ceiling is the next major, exclusive. Minors are additive, so a consumer implementing 1.0.0 reads a release built under 1.2.0, and writing the build version as the floor would be a release claiming a break that did not happen. openregs.compat.check_document is the single gate, and it refuses on either of two independent statements: the consumer’s (the declared major is outside SUPPORTED_SCHEMA_MAJORS) or the release’s (this build’s own spec version falls outside the range the release declares). It runs before digests, signatures or a row of the bundle, including under --insecure-skip-verify, and the builder applies the same rule pointed the other way — it will not build a canon whose spec major it does not implement. The full account, including what to do when it refuses, is schema-compatibility.md.
• builder — which code emitted the bytes. A string such as openregs-release/1.1.0, from BUILDER_VERSION in tooling/openregs/release/build.py, bumped when the builder changes what it writes — 1.1.0 is the manifest’s licenses block, the first field 1.0.0 did not emit. It is the release plane’s own version, not the CLI’s and not the spec’s: its job is that two releases built by different code say so rather than looking identical, so a corpus repository pinning an engine version and a release declaring its builder are two halves of one audit trail. The same string travels into the SLSA provenance statement (openregs.release.provenance), where it names the builder that made the claim.
• commit and regime_tree — which corpus. A release is a function of a commit; the working tree is never read. The manifest records the commit sha, the sha of the regime subtree at that commit, and built_at as that commit’s own committer date, so rebuilding a tag from its own sha reproduces its bytes. The tree sha is there because a commit sha dies in a history rewrite and a content-addressed tree does not — which matters most exactly when the corpus is a repository somebody else rewrites.
• The release tag — which corpus, when. Releases are per regime and always per regime: the tag is <regime>@<YYYY.MM>, the directory is regimes/<regime>/releases/<tag>/, as_of is derived from the tag rather than declared so a release cannot disagree with its own name, and the diff manifest compares against the previous tag of the same regime. There is no release of the corpus as a whole, because there is no commit the corpora share.

index/combinations/ is what says two releases go together. Since regimes release independently, two separately valid releases are not automatically a set anyone has run together: they may have been built against spec versions that mean different things, and neither release says a word about the other. A combination (index/combinations/<name>.yaml) is the artifact that turns “these go together” into a checkable claim. It names the pinned releases as <regime>@<tag>, the spec_schema_range they were validated under, and the commit of the eval suite that passed over them — the evidence, without which the file asserts that a set works together on nobody’s authority. openregs index verify (tooling/openregs/combinations.py) checks that the file is schema-valid, that its name matches its filename, that every pinned release exists, and that each release’s declared schema_version falls inside the combination’s range. It deliberately does not re-run the eval or re-verify digests: a release’s signature and provenance answer for its own bytes, and a combination answers only for the set. index/combinations/fixture-set.yaml is the shipped one, pinning fixreg@2025.02 and fixreg@2025.04.

Where the contract is exercised today, stated so nobody infers more than is there. The builder reads its input out of git in the repository it is invoked in: openregs.release.source.regime_at archives the regimes/<regime> path at a commit, and spec_version_at reads spec/VERSION at that same commit, falling back to 1.0.0 for commits made before that file existed. Both facts are true of this repository’s layout, so the four pins above are exercised end to end against fixreg in one checkout and nowhere else yet. A corpus in its own repository has its canon at the repository root rather than under regimes/<regime>/, and carries no spec/VERSION at all — where the fallback would quietly stamp 1.0.0 rather than fail. Building a regime repository’s releases therefore needs the builder to take the corpus location and the spec version from the pinned engine rather than from the corpus commit. That is work in tooling/, not a change to the contract: the four pins are what travels across the boundary, and they travel unchanged. openregs index verify has the same seam — it resolves each pin at a path under the checkout, so once the pinned releases belong to other repositories, resolving a pin means pulling and verifying one.

What stays open, and where the commercial line is

Everything required to self-host is open, and that is the rule the licence boundary is derived from rather than an outcome of it. The engine is Apache-2.0 and lives here in full: the pipeline, the release builder, the signing chain, the verifier, the REST and MCP server, the CLI, deploy/Dockerfile and the composed stack it boots. Every public-law corpus is CC-BY-4.0. The split is recorded in each release’s own release-meta.yaml —

licenses:
  code: Apache-2.0
  data: CC-BY-4.0

— so it travels with the bytes rather than living on a page a consumer would have to go and trust.

The commercial line is drawn at operating the system, not at the data and not at the code. What is sold is hosting, freshness, private customer-supplied regimes and attestation — the work of running the thing continuously and standing behind it. The cloud repository holds only the control plane, and its data plane is this repository’s engine consumed as a published artifact: a container built from deploy/Dockerfile at a pinned released version, running openregs pull → openregs verify → openregs serve against public releases. It never vendors or forks tooling/. When the hosted product needs an engine change, the change lands here, in the open, and the hosted product picks it up by moving its pin.

What that rules out, written so it can be checked rather than believed: no corpus behind a paywall, no private fork of the engine, no hosted-only capability a self-hoster cannot obtain. And the strongest form of the promise is already an invariant of the serving plane rather than a sentence in a licence — hosted and local serving are projections of one artifact and must answer identically, which tooling/tests/test_serve_parity.py asserts literally: two openregs serve processes started from the same signed release directory, fifty queries POSTed to both, and the raw response bytes compared pair by pair.