A first run in code¶
Twenty lines put the demo's machinery under your control: an environment, an algorithm, a schedule of regimes, and a recorded trace — all declared in one configuration document. This rung's table will come out deliberately disappointing, and the reason it disappoints is the most important thing on the page.
The run¶
from skyfall_crl.train import ExperimentConfig, run_experiment
CONFIG = """
env: {id: CartPole-v1}
algorithm: {name: discrete_hill_climbing}
rollout: {window_steps: 200, total_steps: 1200, auto_reset: true}
regime:
provider: scheduled
params:
schedule:
regimes:
- {regime_id: calm, duration_steps: 400}
- {regime_id: windy, duration_steps: 400}
- {regime_id: calm_again, alias_of: calm, duration_steps: 400}
run: {name: first-run, seed: 0, trace_path: traces.jsonl}
"""
result = run_experiment(ExperimentConfig.from_yaml(CONFIG))
print(f"{result.steps} steps, {len(result.window_metrics)} windows, "
f"configurations {', '.join(result.regimes)} -> {result.trace_path}")
Everything here is one of the four units: six windows of 200
steps pace the algorithm's updates; auto_reset: true starts a new episode when CartPole
genuinely ends one (the default is off, because in a persistent world a reset is destructive);
and the schedule labels three configuration intervals, the third an alias of the first. The
document is the whole experiment — swap env.id for a dotted package.module:factory and the
same twenty lines run something you built.
The score¶
1 trace(s), 3 configuration(s)
Metric segment
---------------------------------
Per-configuration reward 1
Adaptation speed 1
Forgetting 0
Recovery time 1
Stability (variance) 0
Performance gap --
Zero-shot reward 1
Regret --
Effective rank --
Settled reward 1
Return (discounted) 99.9994
Return (undiscounted) 1200
Invalid-action rate 0
Why every number is a constant — and why that is correct¶
Reward 1 in every regime, forgetting exactly 0, variance exactly 0. The table looks broken. It is the opposite: the measurement is telling the truth about the run. CartPole pays +1 for every step regardless of how the pole is held, and nothing in this configuration made the world change — the schedule labelled three intervals, but no code read those labels and adjusted gravity, so the agent lived in one unchanging world with a constant reward. A constant reward under unchanging conditions has nothing to adapt to, nothing to forget, and no variance. The protocol measured exactly that.
This is the honest gap between labelling a run and shifting a world, and it is deliberate at this rung: the schedule is the record of change, and something still has to make the change. Two rungs fix it in turn — Your own environment makes the world genuinely shift underneath the policy, and Instrumenting to Tier 2 replaces the flat reward with one that can resolve quality. Watch this table sharpen at each step.