openregs ingest
Poll sources, durably enqueue what they discover, then advance their cursors.
Poll configured sources in the only safe order: discover, enqueue durably, then advance the cursor. A run stopped before the cursor moves re-discovers the same window, so documents are delivered at least once and never skipped.
openregs ingest [-h] [--source <source_id>] [--document <doc_id>] [--offline-fixtures] [--config <path>] [--state-dir <path>] [--crash-at <point>] [--regime <name>] [--citation-depth <n>] [--scope <path>] [--lock <path>] [--canon <path>] [--inbox <path>] [--quarantine <path>]| Option | |
|---|---|
--source <source_id> | poll this source now, whatever its interval says (repeatable; default: every enabled source that is due) (repeatable) |
--document <doc_id> | enqueue only these documents out of what discovery offers (repeatable). A poll narrowed this way does not advance the cursor, so the documents it left alone are still discovered next time (repeatable) |
--offline-fixtures | replay each source’s recorded fixture feed instead of its discovery endpoint |
--config <path> | read this source configuration instead of config/sources.yaml |
--state-dir <path> | keep cursors and the fetch queue here instead of state/ |
--crash-at <point> | test hook: stop the run as if killed at this point (after-discover, mid-enqueue, after-enqueue) (one of after-discover, mid-enqueue, after-enqueue) |
--canon <path> | write normalized units here — with —regime, instead of regimes/<regime>/canon/; on a poll, which otherwise only captures |
--inbox <path> | stage approved units in this canon inbox, each entry carrying an entry.yaml naming the capture it came from |
--quarantine <path> | hold documents that did not clear their source’s quality bar here, with the snapshot reference and whatever was recovered from them |
corpus scoping
Section titled “corpus scoping”Capture one regime’s declared instruments and whatever they cite, no further than the configured citation depth, instead of polling source feeds.
| Option | |
|---|---|
--regime <name> | run this regime’s scope crawl, reading regimes/<name>/SCOPE.yaml |
--citation-depth <n> | follow citations this many hops instead of the depth the scope file sets |
--scope <path> | read this scope declaration instead of regimes/<regime>/SCOPE.yaml |
--lock <path> | append provenance entries to this ledger instead of regimes/<regime>/SOURCES.lock |
Rendered from openregs/openregs@cbe9615:tooling/openregs/cli/main.py