Command-line reference

One command, six subcommands:

skyfall-crl demo | run | eval | aggregate | serve | conformance

Every flag on this page is checked against the actual parsers when the documentation is tested — a flag added to the code without a row here fails the build, and a row here naming a flag that does not exist fails it too.

Exit codes, everywhere. 0 on success. 1 on a named error (an unreadable file, an unknown name, a failed run) — always accompanied by a message saying what and, where there is one, the remedy. 2 for a usage error (unknown flag, missing required argument), from the argument parser itself. run over a matrix exits 1 if any cell failed, after running the rest; conformance exits 1 when any required check fails, and 0 when only recommended checks do.

Shared option groups

Three groups recur across subcommands; each is defined once here.

Plugins — accepted by every subcommand except demo

Flag	Argument	Default	Meaning
--plugin	MODULE | PATH	—	Import before reading anything, so a name it registers — a backend, a reward component, a provider, an environment module — can be selected. A dotted module or a path to a .py file; repeatable, imported in order. A file is importable afterwards under its stem, which is how env: {id: windy:make} resolves after --plugin windy.py.

The ceiling — accepted by eval and aggregate

What, if anything, the bound-anchored metrics are measured against. With none of these, the performance gap reports -- and everything else uses each segment's own behaviour.

Flag	Argument	Default	Meaning
--spec	NAME | PATH	—	Reward specification to derive the ceiling from: a built-in name, a YAML file, or a JSON document. Also names the episode-end components, which adds the policy-attributable settled-reward column.
--upper-bound	VALUE	—	State the ceiling directly, overriding any derivation.
--data-relative		off	Also report against what the run actually reached. Moves with the policy measured, so it does not compare two policies fairly.
--quantile	Q	1.0	Quantile of observed reward for the data-relative ceiling (1.0 is the peak).

The protocol — accepted by eval and aggregate

The measurement parameters, one per knob the metrics chapter defines.

Flag	Argument	Default	Meaning
--anchor	segment | ceiling	segment	What adaptation and recovery are measured against: the segment's own behaviour, or a theoretical ceiling.
--alpha	ALPHA	per anchor	Fraction of the reference that counts as adapted: 0.9 under the segment anchor, 0.5 under the ceiling.
--epsilon	EPSILON	half the ceiling	Recovery tolerance under the ceiling anchor, as an absolute distance.
--epsilon-fraction	F	0.3	Recovery tolerance under the segment anchor, as a fraction of the segment's own asymptote.
--tail-fraction	F	0.2	Fraction of an interval treated as its settled state.
--zero-shot-window	N	10	Steps after a shift counted as zero-shot.
--window	segment | post_shift	segment	Which steps a shift-anchored metric measures over.
--gamma	GAMMA	from the traces	Discount for the returns, overriding whatever the traces recorded.
--solve-component	NAME	—	Reward component whose positive values mark solved work; adds the solve-rate row.

---

skyfall-crl demo

Train a policy on a world that changes underneath it, score the run, and explain the numbers. No configuration, no code; the five-minute tutorial reads its output line by line.

Flag	Argument	Default	Meaning
--traces	PATH	temporary	Write the run here instead of a temporary directory, to keep it.
--quiet		off	Print the metric table alone, without the commentary.

skyfall-crl run

Execute one experiment, or a whole matrix of them.

Flag	Argument	Default	Meaning
--config	PATH		One experiment configuration. Exactly one of --config/--matrix is required.
--matrix	PATH		A sweep: what every cell shares, and the families, seeds and tasks that vary.
--output-dir	DIR	from the document	Write each run's trace to DIR/<run name>/traces.jsonl, overriding the document.
--export	DIR	—	Also write an export bundle — schemas, reward spec, provenance and weights. A matrix writes DIR/<run name>/ per cell.
--export-external	DIR	—	Write the bundle's weights to DIR instead of copying them inside it. Smaller, and the bundle is then only usable where that path resolves.
--dry-run		off	Print what would run, and run nothing. A matrix can be worth hours of compute; see it first.

A failing cell does not end a matrix: it is reported, counted, reflected in the exit code, and the steps it did take are kept — a truncated run shows up downstream as a short one rather than a missing one. Also accepts --plugin.

skyfall-crl eval

Score recorded traces against the six-metric protocol. Reads trace files and nothing else — no environment, no model.

Flag	Argument	Default	Meaning
--traces	PATH	required	A trace file or a directory tree of them; repeat for several.
--format	table | json | markdown	table	The JSON is deterministic — no timestamp, no absolute paths — so two runs over the same traces are byte-identical, which is what lets it be diffed and pinned.
--output	PATH	stdout	Write here instead of standard output.

Also accepts --plugin, the ceiling group, and the protocol group.

skyfall-crl aggregate

Compare families of runs: group traces, score each family's seeds, report mean ± spread per family — the cross-family table.

Flag	Argument	Default	Meaning
--traces	PATH	required	A trace file or a directory tree of them; repeat for several.
--group-by	FIELD	algorithm_id	Run-metadata field naming the family; source groups by the file a run was read from, for a corpus whose recorded labels collide.
--baseline	FAMILY	ppo if present, else the first	Family the adaptation advantage is measured against.
--min-steps	N	—	Exclude runs shorter than N steps, and say which were excluded. Nothing is dropped silently.
--format	table | markdown | csv | json	table	CSV is long-form, one row per family × metric. The JSON is deterministic, like eval's.
--output	PATH	stdout	Write here instead of standard output.

Also accepts --plugin, the ceiling group, and the protocol group. Anomalies — a family whose seeds produced identical runs, a family pooling more than one task, a truncated run — are reported as notes under the table rather than silently absorbed.

skyfall-crl serve

Run an exported policy against an environment, inference only: the policy acts, nothing updates, and the run writes the same trace a training run writes — so eval scores a served policy with no special case.

Flag	Argument	Default	Meaning
--bundle	DIR	required	The exported bundle. It names the environment and the policy, so serving needs nothing else.
--steps	N	100	Steps to run.
--traces	PATH	not recorded	Write the run to PATH, which skyfall-crl eval then scores.
--env	ID	from the bundle	Environment to serve against, overriding the one the bundle names: a Gymnasium id or a dotted package.module:factory. Serving against a world the policy was not trained on should be something you asked for.
--env-kwargs	JSON	—	JSON object merged into the environment's recorded keyword arguments.
--window-steps	N	64	Steps per window in the served run's record.
--auto-reset		off	Reset the environment when an episode ends. Needed for an environment with genuine terminal states; never for a persistent world, where a reset destroys and rebuilds it.
--seed	N	unseeded	Seed for the environment. Unseeded inference is legitimately different every time; a scored serve should name its seed.
--stationary		off	Ignore the configuration schedule the bundle recorded and run under one set of conditions. By default serve replays the recorded schedule, so a served trace segments exactly as the training trace did.

Also accepts --plugin.

skyfall-crl conformance

Check an implementation against the contracts — before building anything on it. Runnable against your own code with nothing from this repository.

Flag	Argument	Default	Meaning
--env	MODULE:FACTORY	—	An environment factory, as a dotted path or a registered Gymnasium id.
--tier	1 | 2	1	1 = a plain gymnasium.Env; 2 = one that also emits a per-step record, labels its steps, and describes itself.
--reward	NAME	—	A registered reward component, or MODULE:FACTORY.
--algorithm	NAME	—	A registered algorithm backend, or MODULE:FACTORY.
--regime	NAME	—	A registered configuration provider, or MODULE:FACTORY.
--format	table | json	table
--output	PATH	stdout	Write here instead of standard output.

Naming an environment and an algorithm together also drives a short real run, because an implementation can satisfy every method and still be untrainable. Required failures exit 1; recommended failures — reproducibility under a seed, a missing checkpoint hook — exit 0 and say what is given up. Also accepts --plugin.