Five minutes¶

One command trains a policy on a world that changes underneath it, scores the run against the continual-learning protocol, and explains what it found. This page runs it and reads everything it prints — because the output is a complete, worked instance of what this package measures, and every later tutorial produces tables shaped exactly like it.

skyfall-crl demo --traces demo-run

--traces keeps the run's record in demo-run/ so it can be re-scored afterwards; without it the demo uses a temporary directory.

The setup it announces¶

  CartPole, 1,200 steps, three configurations of 400:

    calm         gravity 9.8
    windy        gravity 18.0
    calm_again   gravity 9.8   -- the first configuration again, under a new name

Three regimes. The third is the first again — an alias, which is what makes forgetting measurable: without a return to known conditions there is nothing to compare. And, as the demo says next, the policy is never told which regime it is in; inferring that is the problem being posed, and the label is recorded for the benchmark while being withheld from the agent.

Training, one line per window¶

  window 1  calm         mean episode length   97.5
  window 2  calm         mean episode length    4.3
  window 3  windy        mean episode length    3.9
  window 4  windy        mean episode length   63.3
  window 5  calm_again   mean episode length   97.5
  window 6  calm_again   mean episode length  200.0

Six windows of 200 steps — administrative slices, one algorithm update each. The story is already visible: competent under calm (97.5), a collapse that begins with an unlucky update before the shift (4.3) and deepens when gravity nearly doubles (3.9), recovery under windy (63.3), and then a return to the original conditions the policy now handles better than it did the first time (97.5 → 200.0, the environment's cap).

The score¶

  Metric                     segment
  ----------------------------------
  Per-configuration reward  0.804167
  Adaptation speed                 1
  Forgetting                 -0.2625
  Recovery time                    1
  Stability (variance)      0.926222
  Performance gap                 --
  Zero-shot reward              0.75
  Regret                          --
  Effective rank                  --
  Settled reward            0.791667
  Return (discounted)        92.3276
  Return (undiscounted)          965
  Invalid-action rate              0

The header row says segment: every number here is measured against each regime's own behaviour — the default anchor. Reading down:

Per-configuration reward 0.804 — the mean per-step reward, averaged over the three regimes. The demo's reward pays 1 for a step upright and −5 for a fall, so this says the pole was held most of the time.
Adaptation speed 1 — after each shift, the running mean was back within a fraction of its eventual level in a single step. At this reward's granularity there is nothing to wait for; that is what an easy adaptation looks like, not a missing measurement. L3 shows the same world producing an adaptation of 57 the moment the reward can resolve quality.
Forgetting −0.2625 — negative, so the revisit went better than the first encounter: backward transfer, not forgetting. The per-regime sections below put numbers to it.
The three -- rows are absences, not zeros. The performance gap needs a ceiling and CartPole's reward has no specification to derive one from; regret needs a reference trajectory nobody supplied; effective rank was never measured because capture is off by default. Two further rows — the policy-attributable settled reward and the solve rate — appear only when their inputs are named, so this table has thirteen rows of a possible fifteen.

Per-regime sections¶

The command then prints one section per regime. The revisit's is the one worth pausing on:

Configuration 'calm_again'

  Metric                    segment
  ---------------------------------
  Per-configuration reward   0.9875
  Adaptation speed                1
  Forgetting                -0.2625
  Recovery time                   1
  Stability (variance)       0.0625
  Performance gap                --
  Zero-shot reward                1
  Regret                         --
  Effective rank                 --
  Settled reward                  1
  ...

calm earned 0.725 the first time; calm_again — the same conditions — earned 0.9875, hence the −0.2625. Both labels report the same forgetting, because the comparison pairs the two encounters through what the alias repeats, not the two labels. Stability of 0.0625 against 1.30 under the first calm says the second visit was not just better but steadier.

The run is a file¶

  skyfall-crl eval --traces demo-run

The demo closes by pointing at its own trace. Scoring reads that file and nothing else — not the environment, not the policy — so this run can be re-scored on another machine, months from now, by someone with neither. That command is the whole of the next rung's second half.

A final honesty note, in the demo's own words: the algorithm is a hill climber, chosen as the simplest thing that genuinely improves, and the numbers demonstrate the measurement, not a result. A first run in code puts the same machinery under your control.