What is verified, and how¶
The claims this documentation makes are checkable, and this page says which checks stand behind which claims — so the limits read as scope rather than as apology, and so a claim you are about to rely on can be traced to the evidence for it.
The ported arithmetic matches the originals¶
Every algorithm, reward and metric in this package that reconciles or ports research code is verified differentially: the original implementation and this one run on the same generated inputs, at volume, and every output is compared. Reading the source is how the ports were written; running both is how they are believed.
| Layer | Comparisons | Against |
|---|---|---|
| The reward system (all four specifications, every component) | 181,084 | both reference reward stacks |
| The harness's arithmetic (advantage estimation, token slicing, hindsight relabelling, EWC's Fisher, the context encoder, the featuriser, effective rank) | 126,718 | the reference training code |
| The six-metric protocol | 175,404 | the reference evaluation code |
| The data-relative anchor | 101,350 | the reference analysis code |
All at zero mismatches, and a representative slice of each battery is frozen into the test suite as regression fixtures, so the result is re-established on every run rather than remembered.
The adapter is verified against a live deployment¶
The MORPHEUS adapter's contract with a real world is checked by using one: real incidents projected into the fixed observation schema, a remediation executed and accepted by the world, a verifier scoring it into a non-zero composed reward, forbidden actions refused client-side before reaching the deployment, a configuration shift pushed on schedule and read back, and a training update taken on the resulting experience. These tests skip when no deployment answers — a recorded, deliberate limit — and everything else runs without one, against an in-memory double that implements the same backend protocol.
The boundary is measured, not asserted¶
Everything above the adapter is written against the Gymnasium API, and the claim is enforced by a
test that imports each core module in a subprocess and records exactly which modules loaded:
the core and the environment layer load no adapter module and no ML stack. The check itself is
verified by deliberately breaking it — an adapter import injected into a core module fails the
boundary test and only that test. The contract has real consumers: the harness types against
gymnasium.Env and is tested on CartPole (which genuinely terminates, as a world never does) and
a text-action toy world; DiscoveryWorld runs through everything from a
configuration document with nothing in the package edited; and the metrics read recorded traces
rather than any environment at all.
The numbers are pinned against drift¶
A fixed sweep runs end to end — runner, collector, trace, ingest, metrics, aggregator, both commands — and its machine-readable output is compared against a committed baseline on every supported interpreter: every key exactly, so a field appearing or disappearing fails, and every number to a tolerance six orders of magnitude tighter than any real change. A seeded PPO update is pinned the same way at a tolerance measured across platforms rather than assumed. The demo's published numbers are a third baseline: the table the quickstart-level tutorial shows is compared line-for-line against what the command prints.
The gates are shown to fail¶
A check that has never failed has not been shown to work, so every gate in this project is teeth-tested when it lands: a planted defect — a perturbed number, a misplaced key, an undocumented flag, a leaked label — must fail the named test and only that test, before the gate is trusted. Several gates have since caught their own authors, which is the point.
This documentation is under the same discipline¶
Every Python fence parses and its imports resolve; every YAML fence is validated against the real configuration models; the tutorial pages are executed when the documentation is tested, and every output block they publish must match what actually printed; the CLI reference is checked against the argument parsers in both directions, and the configuration reference against the models' own field lists; committed figures must equal what the deterministic generator draws. A number nobody measured cannot ship here.
Continuous integration¶
Six jobs on every push: lint (ruff and mypy), the full suite on Python 3.11, 3.12 and 3.13, a cached job that installs the ML stack and runs the training backends, and a documentation job that builds both editions strictly and runs every documentation gate.
What verification does not cover¶
No algorithm here is verified to learn — the arithmetic matches the originals, and the originals' published training runs need GPUs to reproduce. The live checks cover the MORPHEUS adapter against one deployment, not every deployment's configuration. And the differential batteries establish fidelity to the reference implementations, not to the paper's published figures — see the τ note in Limits.