Skip to content

openregs ingest

Poll sources, durably enqueue what they discover, then advance their cursors.

Poll configured sources in the only safe order: discover, enqueue durably, then advance the cursor. A run stopped before the cursor moves re-discovers the same window, so documents are delivered at least once and never skipped.

openregs ingest [-h] [--source <source_id>] [--document <doc_id>] [--offline-fixtures]
[--config <path>] [--state-dir <path>] [--crash-at <point>]
[--regime <name>] [--citation-depth <n>] [--scope <path>] [--lock <path>]
[--canon <path>] [--inbox <path>] [--quarantine <path>]
Option
--source <source_id>poll this source now, whatever its interval says (repeatable; default: every enabled source that is due) (repeatable)
--document <doc_id>enqueue only these documents out of what discovery offers (repeatable). A poll narrowed this way does not advance the cursor, so the documents it left alone are still discovered next time (repeatable)
--offline-fixturesreplay each source’s recorded fixture feed instead of its discovery endpoint
--config <path>read this source configuration instead of config/sources.yaml
--state-dir <path>keep cursors and the fetch queue here instead of state/
--crash-at <point>test hook: stop the run as if killed at this point (after-discover, mid-enqueue, after-enqueue) (one of after-discover, mid-enqueue, after-enqueue)
--canon <path>write normalized units here — with —regime, instead of regimes/<regime>/canon/; on a poll, which otherwise only captures
--inbox <path>stage approved units in this canon inbox, each entry carrying an entry.yaml naming the capture it came from
--quarantine <path>hold documents that did not clear their source’s quality bar here, with the snapshot reference and whatever was recovered from them

Capture one regime’s declared instruments and whatever they cite, no further than the configured citation depth, instead of polling source feeds.

Option
--regime <name>run this regime’s scope crawl, reading regimes/<name>/SCOPE.yaml
--citation-depth <n>follow citations this many hops instead of the depth the scope file sets
--scope <path>read this scope declaration instead of regimes/<regime>/SCOPE.yaml
--lock <path>append provenance entries to this ledger instead of regimes/<regime>/SOURCES.lock

Rendered from openregs/openregs@cbe9615:tooling/openregs/cli/main.py