OpenRegs — Overview

OpenRegs is an open-source system that turns financial regulation — EU regulations, UK statutes, US CFR rules and comparable national instruments — into versioned, machine-readable, cryptographically verifiable data.

A regulator publishes prose. A bank, an auditor or an AI copilot needs something else: a specific obligation, as it stood on a specific date, traceable byte-for-byte back to the official text. OpenRegs is the machinery in between. Every artifact it emits carries provenance back to a published source document, and every claim it serves carries a citation.

Terms used below are defined in GLOSSARY.md.

The six planes

The system is organised as six planes. Each plane consumes the plane before it and publishes a stable interface to the plane after it, so a plane can be rebuilt or replayed without disturbing its neighbours.

1. Ingestion

Per-source watcher adapters poll official publishers (EUR-Lex, legislation.gov.uk, the eCFR and the Federal Register, national gazettes), download the official legal texts, and store the exact raw bytes in a content-addressed snapshot store keyed by sha256. The adapter then normalizes those bytes into Akoma Ntoso (AKN), the OASIS XML standard for legal documents, and validates the result against the AKN schema and the project’s own gates.

The adapter is split by purity: only discover() and fetch() touch the network, the clock or randomness; parse() and normalize() are pure functions of the snapshot bytes. That split is what makes replay possible — when a mapper improves, old snapshots are re-run through the new mapper and the output is diffed, with no re-fetch from the regulator.

2. Consolidation

Regulators publish base acts and, later, amending acts that say things like “in Article 5(3), replace ‘EUR 5000’ with ‘EUR 10000’”. The consolidation engine parses those amendment instructions and applies them to base texts to produce point-in-time consolidated versions: the text as it read on any given date. Amendments are classified as substantive or editorial, and each consolidated expression records the amending instruments that produced it.

3. Atomization

An LLM proposes obligation atoms — small YAML records stating which actor MUST, SHOULD or MAY do what, under which conditions, with character-offset provenance spans into the consolidated source text. The model proposes; it never decides. Every atom reaches the canon through a pull request that a human maintainer reviews and merges. CI enforces the invariant that an atom’s quoted_text equals source_text[char_start:char_end] byte-for-byte, so an atom can never drift from the text it claims to quote.

4. Canon

Git repositories hold the reviewed output: consolidated texts, atoms, and a cross-reference graph (defined-term links, internal references, derogations, amendment lineage). The canon is the human-auditable ground truth, and it is protected by CI: schema validation, provenance checks, ID-permanence checks (an atom id or eId is never reused; retirement is status: deprecated plus superseded_by plus a DEPRECATIONS.yaml entry), and graph integrity.

Repositories, plural, and on purpose: each real legal corpus is its own repository with its own domain reviewers and its own release cadence. The one in this repository, regimes/fixreg, is the synthetic fixture the engine is tested against, and it stays here permanently — see ARCHITECTURE.md.

5. Release

A release builder compiles the canon at a git commit into date-tagged, signed artifacts:

1. a SQLite bundle — texts, atoms, and metadata for offline query;
2. a graph index — the cross-reference graph in queryable form;
3. embedding chunks — deterministic, chunked text for retrieval;
4. codegen constants — generated code (e.g. incident_report_threshold_eur) for direct use in application logic;
5. a diff manifest — atom-level changes since the previous release.

Artifacts are signed with cosign and carry SLSA provenance; builds are deterministic, so the same canon commit produces byte-identical artifacts.

6. Serving

A REST API and an MCP server — one codebase, two protocols — answer questions against a pinned release, always returning citations into the canonical text. The same release can be pulled and served locally by the openregs CLI, so an answer given in a bank’s own network is identical to the hosted one. pull and serve verify signatures, digests and provenance before unpacking or using a release. Downstream consumers — bank CI bots, AI copilots, audit tooling, controls.yaml mappings — build on the released artifacts, never on live scraping.

Data flow: regulator sources through to consumers

1. Regulator source publishes or updates a document (EUR-Lex, legislation.gov.uk, eCFR, Federal Register, national gazettes).
2. Watcher adapter discover() detects the change against a stored cursor.
3. Watcher adapter fetch() downloads the exact bytes.
4. Snapshot store records those bytes content-addressed by sha256; the digest is pinned in SOURCES.lock.
5. parse() turns the snapshot into a structured intermediate form (pure).
6. normalize() / AKN mapper emits Akoma Ntoso XML with stable eIds (pure).
7. Validation gates check schema, provenance and invariants; passing documents go to the canon inbox, failing ones to quarantine.
8. Consolidation engine applies amendment instructions to base texts, producing point-in-time expressions.
9. Atomization proposes obligation atoms with provenance spans over those expressions.
10. Human review — maintainers accept, correct or reject each atom in a pull request.
11. Canon (git) holds merged texts, atoms and the cross-reference graph; CI validates every change.
12. Release builder compiles a canon commit into the five artifacts and signs them (cosign, SLSA provenance).
13. Distribution — releases are published as date-tagged, verifiable bundles; index/combinations/ pins cross-regime release sets.
14. Serving — the REST API and MCP server answer over a pinned release, with citations; the CLI mirrors the same release locally after verifying it.
15. Consumers — bank CI bots gate deployments on controls.yaml mappings, AI copilots answer with citations, audit tooling proves what a rule said on a date, and diff manifests drive change-impact review.

Invariants that hold across all six planes

• Provenance. Every canonical text traces to a SOURCES.lock sha256; every atom’s quoted_text equals its provenance span in the source text, byte-for-byte.
• Purity. parse() and normalize() never touch network, wall clock or randomness.
• Determinism. Same input ⇒ byte-identical output for mappers, builds and the fixture embedder.
• Permanent IDs. Atom ids and eIds are never reused; retirement is explicit.
• Verify before use. Signatures, digests and provenance are checked before a release is unpacked or served.
• Offline tests. No test or CI job contacts a real regulator endpoint; all network behaviour is exercised against local test servers and the FIXREG fixture corpus.

Where things live

Inside this repository — the engine — spec/ holds versioned schemas, taxonomies and the vendored AKN XSD; tooling/openregs/ holds all source, one package per plane; fixtures/ holds the offline test corpus; and regimes/fixreg/ holds the fixture regime’s canon, atoms, graph and releases in the layout every regime uses. See CLAUDE.md for the full directory listing and ARCHITECTURE.md for the component-and-edge reference.

Not everything lives here. A real regime’s corpus is its own repository, the documentation site that renders these pages is another, and the hosted product’s control plane is a third — each consuming this repository’s released artifacts rather than its files. Which repository holds what, and why the line falls there, is ARCHITECTURE.md § Repository boundaries.