Runbook — the release feed service

Audience: whoever runs the feed service, and whoever operates a system that subscribes to it. Trigger: a release was built and downstream needs to hear about it, a subscriber says it never got a payload, or a subscription database needs backing up or restoring. Time: minutes.

The feed service is how a downstream system learns about a release without watching a git repository. It reads a built release’s diff manifest, keeps its atom changes as feed entries, serves them as a JSON feed and an RSS feed per filter, and POSTs the ones each subscription asked for to that subscriber’s endpoint, signed and retried until they land.

It is outbound. The feeds a watcher adapter polls — a regulator’s own change notifications — are the inbound ones and belong to openregs ingest; nothing here ever contacts a regulator.

---

Running it

openregs feed serve --store /var/lib/openregs/subscriptions.sqlite \
                    --host 0.0.0.0 --port 8787 \
                    --base-url https://feeds.example.com

--base-url is what the links inside the feeds are written against, so set it to whatever a reader sees. Without it the service writes links against the address it bound to, which is right for a laptop and wrong behind a proxy.

The service does no authentication and terminates no TLS. It publishes material that is already public and holds exactly one thing that is not — the subscription secrets, which it never returns in any response. Put it behind whatever does the rest.

Route	What it is
POST /subscriptions	register an endpoint, its secret and its filter
GET /subscriptions	what is registered; secrets are never included
DELETE /subscriptions/<id>	stop sending
POST /publish	announce a built release ({"regime": …, "tag": …})
POST /deliveries/retry	one retry pass over everything now due
GET /deliveries	what is owed, and what state each delivery is in
GET /log	the service log: every attempt at every delivery
GET /feed.json, GET /feed.xml	the feeds, filtered by the query string
GET /releases/<regime>/<tag>/diff.json	the release’s own diff manifest

Filtering

Four dimensions, named the same way in a subscription’s filter and in a feed request’s query string: jurisdiction, regime, topic, entity_type. A dimension left unnamed matches everything; a dimension named matches when the change carries any of the values (?regime=fixreg&entity_type=operator).

Entity types follow the taxonomy: small_operator receives changes to obligations declared on operator, because a small operator is an operator. It does not work the other way round.

topic matches nothing today. No atom in spec/schemas/atom.schema.json carries a subject vocabulary, so no entry declares a topic. The dimension is wired and will start matching the day atoms carry tags:; until then a subscription filtered on a topic is a subscription that hears nothing, which is deliberate — the alternative, treating an unknown dimension as “everything”, would silently widen every such subscription later.

Publishing a release

openregs feed publish --regime fixreg --tag 2025.04 \
                      --store /var/lib/openregs/subscriptions.sqlite

The release must already be built under regimes/<regime>/releases/<tag>/. The entries are the manifest’s atom changes, copied verbatim — including their substantive/editorial classification and its basis — plus the entity types and jurisdiction resolved out of that release’s own corpus.sqlite. Unit changes are not fanned out; they are one click away in the diff each entry links to.

publish exits 1 while a delivery is still owed. That is not a failure of the publication — the release was announced and every payload is durable — it is the signal that a drain is still to come.

Publishing the same tag twice is a no-op: a delivery’s id is derived from the subscription and the payload bytes, so a re-run does not re-send anything.

What a subscriber does with the announcement

A feed entry says what changed. It is not the release and it is not evidence about the release: it is a notification this service composed, signed with the subscriber’s own shared secret, from a diff manifest it read off disk. Nobody should act on a regulatory change because a webhook said so.

So the second half of a subscriber’s handler is a pull:

openregs pull "$REGIME@$TAG" --from "https://github.com/openregs/regime-$REGIME" \
    --into /var/lib/openregs/releases --trust /opt/openregs-$REGIME/config/trust.yaml

which fetches the release from the distribution channel and runs the five checks over the fetched bytes before any of them reach that directory. The channel, the URL layout, and what the pull refuses are in release-publish.md.

--trust is not optional decoration on a subscriber. A handler runs on a machine that is not a checkout, and neither the wheel nor the standalone binaries carry a roster, so without it the pull has nothing to anchor against and refuses — which is the right answer and not a useful one at three in the morning. Obtain the roster of the repository that owns the regime once, review it, and name it every time: Getting an anchor onto a machine that is not a checkout, in release-publish.md.

Two consequences worth stating out loud, because they are the reason the split exists:

• The HMAC on a delivery authenticates the messenger, not the law. It proves the payload came from a service holding the subscription secret. The signature that means a corpus is what a maintainer signed is the release’s own, checked by pull and verify against config/trust.yaml.
• A feed can be dropped without losing anything. Deliveries are at-least-once and can be exhausted; the release is immutable and addressable by tag forever. A subscriber that missed every notification catches up by pulling the tag.
• The tag is what the subscriber has, so the pull checks it. An announcement carries a regime and a tag and no digest, which is all a notification can honestly carry — so the release that comes back is held to that name by pull itself: release-meta.yaml states which release it is, and one that says something else is refused before a byte is staged. That matters most in exactly the case this handler exists for, because a release served under the tag of a later one is superseded law arriving as current, and it passes all five checks. Every pull prints the release’s digest; a subscriber that records it can pin the same bytes with --expect anywhere else it vendors them.

Signatures — what a subscriber must check

Every POST carries:

Header	Value
X-OpenRegs-Signature	sha256=<hex>, HMAC-SHA256 of the request body bytes under the subscription secret
X-OpenRegs-Delivery	the delivery id — stable across retries, so deduplicate on it
X-OpenRegs-Subscription	which subscription this answers
X-OpenRegs-Event	release.published

Verify over the raw bytes as they arrived, before parsing them. Re-serializing the JSON and signing that will not match, and is not meant to.

What an endpoint must be

A delivery carries two things worth keeping off the wire, and carries them together: the payload, which says which obligations in which regime just changed, and the signature over it. So the endpoint is held to two rules, checked when the subscription is registered and again before each POST — a row written before a rule existed is still a row drain picks up.

Refused	Why
cleartext http to anything but loopback	which regulations you subscribe to is metadata about your compliance posture, and the signature travels beside the payload. Loopback is exempt: a subscriber in the same pod has no hop to wiretap
anything but http(s)	a subscriber is reached over HTTP; nothing else is a delivery
a 3xx from the endpoint	redirects are not followed at all. urllib answers a redirect on a POST by reissuing it as a GET with no body, so a followed redirect delivers the payload nowhere while handing X-OpenRegs-Signature to whatever host the Location named — and the 200 that came back would be recorded as delivered. A subscriber that moved re-registers; it does not redirect

This is the opposite of what pull does with a redirect, and deliberately: a pull is fetching bytes that are verified afterwards, so a hop is survivable, while a delivery is handing bytes over and a hop that drops them has already failed.

A refused endpoint fails the attempt with the reason verbatim in the log, is retried on the schedule below, and ends exhausted. That is the intended shape: the fix is to re-register the subscription, not to wait.

Delivery, retries and the log

Delivery is at-least-once. The row is durable before the first POST, so a service that dies mid-flight still owes the payload when it comes back. A failed attempt — a refused connection, a timeout, or any non-2xx — sets a next-attempt time and the delivery stays pending:

Attempt	Sent after
1	immediately, at publication
2	1 minute
3	5 minutes
4	30 minutes
5	2 hours

After the fifth the delivery is exhausted: it stops being retried and stays in the log saying so. A retry re-sends the same bytes with the same signature and the same id — a retry is the same delivery, not a new rendering of it.

Run the retry pass from a cron entry, or by hand:

openregs feed drain --store /var/lib/openregs/subscriptions.sqlite
openregs feed log   --store /var/lib/openregs/subscriptions.sqlite

drain exits 1 while anything is still pending.

“A subscriber says it never got the payload”

1. openregs feed log --store <db> — find the delivery and read its attempts. A failed line names the endpoint and the error verbatim.
2. If it is pending, fix the subscriber and run openregs feed drain.
3. If it is exhausted, the five attempts are spent. Re-register or re-enable the subscriber and re-publish the tag: the delivery id is a function of the subscription and the payload, so a new subscription gets a new delivery while the old one stays in the log as the record of what happened.
4. If nothing was ever created for that subscriber, its filter did not match. The publication response and GET /log both name the subscriptions that were silent; compare the filter against the entry’s facets in GET /feed.json.

Backing it up

Everything else this system produces is a function of a git commit and can be rebuilt. The subscription database cannot: it is somebody’s registration, made at a moment. One SQLite file holds all of it —

state/feed/subscriptions.sqlite      (the default; --store overrides it)

— with the subscriptions, the publications and their entries, the deliveries and the attempt log. state/ is gitignored on purpose: a subscription is nobody’s reviewed data and must never reach the canon. Back the file up whole (it is opened in WAL mode, so copy the -wal and -shm siblings with it, or use sqlite3 <db> ".backup"), and treat it as a credential store: the secret column is the key each subscriber verifies with.

What is deterministic and what is not

Everything a feed publishes is a function of the release: an entry’s date is the release’s built_at, which is its commit’s committer date, so two services announcing one release render identical feeds and identical payload bytes.

A delivery attempt is the exception and is stamped with a real moment, because no commit can say when a subscriber was tried. --now pins it, which is how the backoff schedule is tested without sleeping through it.