skyfall-crl

Continual reinforcement learning drops the assumption that the world holds still: the environment's conditions change while the policy is learning, and the questions that matter are how quickly it adapts, what it forgets when old conditions return, and whether it keeps the capacity to learn at all. Most RL tooling is built for the opposite setting — episodic, cheaply reset, stationary by design.

This package is infrastructure for the continual setting: composable verifiable rewards defined as data, regime scheduling (a regime and a configuration are the same thing here — the prose says both) that shifts a world's conditions on a plan the policy never sees, a substrate-neutral environment layer with an adapter for live MORPHEUS worlds, a config-driven training harness, the six-metric protocol for behaviour around change, policy export and serving, and a conformance checker for the four contracts you implement yourself. Each layer is usable on its own, and only the adapter knows what MORPHEUS is.

pip install -e .
skyfall-crl demo

  Metric                     segment
  ----------------------------------
  Per-configuration reward  0.804167
  Adaptation speed                 1
  Forgetting                 -0.2625
  ...

That is a real run — gravity shifting underneath a policy, the returning conditions handled better the second time — and five minutes reads all of it.

Three ways in

You are…	Start here
evaluating whether this fits your research	Background — the field, the two experimental shapes, where this sits — then the metrics, then Limits for the honest scope
bringing your own environment	the tutorial ladder: twenty lines to a first table (L1), your world genuinely shifting (L2), composed rewards (L3), a full experiment ending in a served policy (L5)
operating a MORPHEUS deployment	L6 — Train on a MORPHEUS world — provision, remediate, train and score against a live world

The eight layers

Layer	In one line
Rewards	the objective as weighted, clipped components in a serialisable specification — changing it is editing data
The environment layer	one contract, two tiers: any gymnasium.Env works; one that describes its steps gains the composed rewards
MORPHEUS adapter	a gymnasium.Env over a live operational world — incidents observed, remediations scoped and verified
Configuration shifts	schedules, recurrence, and the label the policy never sees
Training	environment, algorithm, policy, reward and schedule in one document; a matrix of families × seeds in one more
Benchmark metrics	the six-metric protocol from recorded traces alone — scoreable years later, by someone with neither the world nor the policy
Export and serving	a policy as one movable bundle: manifest, schemas, reward, weights — served back and scored with no special case
Conformance	the four core contracts, as checks you run

What to trust, and how far

Every number on this documentation's executed pages is produced by executing them (the two live-infrastructure tutorials, L6 and L7, publish captures instead — each says so); the arithmetic ported from research code is verified against the originals; and the limits are stated as precisely as the features — What is verified is the evidence, Limits and status the scope, and Fidelity the record a reproducing researcher needs. Nothing here has been trained to convergence, and no number this package prints is a research result.