Training

The problem. Research training code picks its algorithm by which entrypoint you run, with the trainer class written into each script. Swapping algorithms means forking the harness, and the harness knows what environment it is driving.

The shape here. One configuration document names the environment, the algorithm, the policy, the reward and the configuration schedule. Backends resolve by name on first use, so naming a GPU trainer costs no import and a document validates on a laptop with no ML stack installed. The harness types against gymnasium.Env, and nothing below it names an environment.

ExperimentConfig  →  registry  →  Algorithm ← Policy
RolloutCollector  →  RolloutWindow  →  update()  →  metrics + trace

What you implement: the Algorithm protocol — structurally, inheriting nothing: a policy property and an update(window); on_regime_shift and save_checkpoint are optional. Extending is the recipe; skyfall-crl conformance --algorithm is the check.

A run, end to end

from skyfall_crl.train import ExperimentConfig, run_experiment

config = ExperimentConfig.from_yaml("""
env:       {id: CartPole-v1}
algorithm: {name: discrete_hill_climbing}
rollout:   {window_steps: 200, total_steps: 1200, auto_reset: true}
regime:
  provider: scheduled
  params:
    schedule:
      regimes:
        - {regime_id: calm,       duration_steps: 400}
        - {regime_id: windy,      duration_steps: 400}
        - {regime_id: calm_again, alias_of: calm, duration_steps: 400}
run:       {name: cartpole-demo, seed: 0, output_dir: runs}
""")
result = run_experiment(config)

Or without Python at all: skyfall-crl run --config experiment.yaml. The full document schema is the configuration reference; L1 walks a run line by line, and L4/L5 grow it into a matrix of families and seeds. Two of its lines carry the layer's two standing rules:

• auto_reset is off by default because in a persistent world a reset destroys the world; CartPole, with genuine terminal states, needs it on.
• The regime section is what makes the run continual. Adaptation, recovery and forgetting are defined at configuration boundaries, so a run with none records one segment and those metrics report nothing; result.stationary says which happened. run_experiment also tells the algorithm when a configuration ends — a continual-learning backend's only signal to consolidate.

In a sweep, say where the seed goes or it goes only to the environment: write ${seed} into the parameters that should vary. A sweep whose seeds vary only the world reports a spread of exactly zero, which reads as certainty and means the seed reached nothing; aggregate flags it, but the fix belongs in the document.

The built-in that needs nothing

discrete_hill_climbing is the backend a base install trains with: observation features to one score per action, greedy action, keep a random perturbation when a window scored better. It fits any discrete-action environment, flattens Dict observations by sorted key, and is deliberately the simplest thing that genuinely improves — so "trains from a configuration document" is checkable, not a claim. One detail worth borrowing if you write your own: its fitness is reward per episode, not per window — a window is a fixed step budget, so on an environment paying a constant per step every window sums to the same number and a search on that sum climbs a flat surface.

The ported algorithms

ppo, ppo_ewc and ppo_lcm are the reference algorithms, ported. Each trains a language policy it is handed, so the document names the policy beside the backend:

algorithm:
  name: ppo_ewc
  policy:
    id: skyfall_crl.train.backends.policy:language_policy
    kwargs: {model: Qwen/Qwen3-4B, max_new_tokens: 256}
  params: {parameter_scope: all_trainable, fisher_seed: 0, replay_seed: 0}

They need the [train] extra. Point model at a checkpoint directory and the saved value head restores with the weights, so an SFT stage's critic carries into PPO. Operational facts to know before a long run — each with its full story in Fidelity:

• Consume the collector lazily (for window in collector.windows(n)), one update per window. A language policy records the tokens it generated per window; materialising the run first pairs one window's rewards with another's actions, and the backend refuses loudly.
• HER is a setting, not a backend: params: {replay_size: 256, replay_seed: 0} on ppo, exactly as the reference wired it. Set replay_seed — the reference draws from the global RNG.
• ppo_ewc's parameter_scope defaults to lora_only, which matches PEFT naming; on a model without adapters it matches nothing and raises rather than training as plain PPO behind a healthy log line. fisher_gamma is the Fisher's decay; gamma is PPO's discount.
• ppo_lcm's regime prefix is exposed, not injected: call backend.observe(observation) and prepend backend.regime_prefix() to your prompt, or the latent is computed and discarded.

The backend name is checked when the document loads, not when the model builds — a typo fails immediately, and a plugin's backend must register before its document is read (--plugin does exactly that).

Selecting a MORPHEUS world

A live world needs a backend object, which a document cannot express — so the adapter ships a factory that takes scalars, selected like any other environment:

env:
  id: skyfall_crl.env.morpheus:morpheus_world
  kwargs: {base_url: "http://localhost:8282", layout: process-outbound, max_steps: 60}

Anything MorpheusEnv accepts goes under kwargs — including world_id to attach to an existing world as a guest, and a regime schedule, which for a MORPHEUS world belongs to the environment rather than the harness section (L6 shows why). Leave auto_reset off: a MORPHEUS reset destroys the world and builds another. The harness imports none of this — the dotted path resolves at build time, and that boundary is asserted by test.

What the harness asks of an environment

Only gymnasium.Env. Everything else is optional and probed:

It offers	Used for	If absent
describe()	run provenance, recorded verbatim	provenance is {}
info["regime"] = {"id", "boundary"}	segmenting by configuration	the document's provider labels the run
info["trace"] = an EpisodeStep	per-component breakdown on the trace	a minimal record is built for you

That last row is what puts an ordinary Gymnasium environment on the same footing as one that decomposes its own reward. Probing goes through Gymnasium's wrapper-aware attribute lookup, so a capability added by a middle wrapper is found.

Recording, and the plasticity column

Every run records a trace — one row per step, written as it happens, so a run that dies halfway leaves what it did. A language policy built with capture_effective_rank=True additionally stamps each row with its representation's effective rank, the plasticity diagnostic; off by default because it costs a forward pass and a singular-value decomposition per step, and a measurement not taken is recorded as absent rather than zero.

Limits

Nothing on this page claims an algorithm learns — the ported arithmetic is verified against the reference implementations and the harness against a live deployment (What is verified), but training to convergence needs GPUs and no result here is a research result. The ported backends train language policies only, and several reference behaviours are reproduced deliberately, defaults and switches recorded in Fidelity.

---

API: every public symbol, with signatures — Training — API reference.