Skip to content

Troubleshooting

Most errors here name the problem and the fix. This page covers the ones where the cause is a design decision worth understanding, and the failures that are quiet rather than loud.

Index

Quiet failures — the ones to know about

Errors, and what they mean

Environment and install

Quiet failures — the ones to know about

These do not raise. They produce a run that completes and reports numbers.

Every reward is 0.0, or every advantage looks flat

A spec that can read nothing from its environment. Every reward spec that ships describes MORPHEUS's operational model. Compose one against an environment reporting something else and each component reads an absent field and returns zero, forever.

The engine warns once when a spec and a context are describing different worlds. Ask it directly:

from skyfall_crl.rewards import CompositionEngine, get_spec

CompositionEngine(get_spec("paper")).declared_reads()

If that names fields your environment never fills, you want a different spec or your own component — see Extending.

Or: advantage estimation is degenerate. With gae="degenerate" — the default — every sample is its own one-step episode, the estimate collapses to r - V, and gamma and lam have no effect. That is the published baselines' behaviour. Pass gae="episodic" to use the window's terminal flags.

Episode-scoped reward terms never contribute

Your rollout window is shorter than an episode. Components with Cadence.EPISODE_END fire on the episode's last step, and a window boundary is not one. With the paper spec and a short window you see the failure term and nothing else, until the window that ends the episode.

This is correct behaviour, not a bug — see Concepts.

Replay is enabled but never replays

Replay draws len(batch) // 4 rows. A window under four steps fills the buffer and never draws from it. Widen the window.

Two seeded runs still differ

run.seed seeds the environment only. Hindsight replay (replay_seed) and EWC's Fisher draw (fisher_seed) are unseeded by default, faithfully — the original research code seeds neither. Set all three for a reproducible run. See Limits.

ppo_lcm behaves exactly like ppo

Because nothing is injecting the regime prefix. The backend exposes it; it does not inject it, because it does not own the prompt. Wire observe and regime_prefix into whatever builds your prompt, or ppo_lcm is PPO plus an auxiliary loss that changes no decision.

Adaptation speed and recovery time report the same number

This is a property of --anchor ceiling, not of the default. Under that anchor adaptation asks for a running mean at or above α·R_ub while recovery asks for one within ε of R_ub, with ε defaulting to half the ceiling — so below 1.5·R_ub, which is every real run, the two conditions coincide. Inherited behaviour, not a bug here: the original tables show it as a recovery time of exactly 1.0 wherever adaptation was fast.

Either pass a smaller --epsilon to separate them, or drop the flag: the default anchor measures adaptation against the configuration's own peak and recovery against its own tail asymptote, which are different questions and do not collapse into each other.

Every family's spread across seeds is ± 0

The seeds changed nothing. run.seed seeds the environment; it reaches a policy's initialisation, a replay draw or a Fisher estimate only where a sweep writes ${seed} into them. Write it nowhere and every seed of a family runs identically — so the deviation reported across them is zero by construction, and a table of ± 0 reads as a strongly repeatable result when it means the seed was never consumed.

skyfall-crl aggregate names any family this happened to. The fix is in the sweep document:

families:
  ppo:
    algorithm:
      params: {replay_seed: "${seed}"}
      policy: {id: my.module:policy, kwargs: {seed: "${seed}"}}

A genuine zero is possible too — on an environment whose reward does not depend on anything the seed touches, every seed really does agree. The note tells you to check, not that you are wrong.

A family's numbers average two different problems

Grouping is by algorithm, so a sweep that varies families and tasks puts both tasks in one column. The protocol's tables are per task — adapting to a scheduling shift and adapting to an allocation shift are not the same quantity — so pooling them produces a mean that describes neither. aggregate names any family that spans more than one task; run it once per task to compare within one.

The regression gate fails and you did not change a metric

tests/test_regression_gate.py compares a fixed sweep's output against a committed baseline: every key exactly, every number to a tolerance far tighter than any real change. It fails when a number moved, and also when a field appeared or disappeared — which is intended, since that changes what every consumer reads. tests/test_demo_baseline.py does the same for skyfall-crl demo, and additionally checks that the table published in the quickstart is the one the command prints — so a change to the demo's numbers fails until the documentation catches up.

If the change is deliberate, regenerate and read the diff before committing it:

SKYFALL_CRL_UPDATE_BASELINE=1 pytest tests/test_regression_gate.py tests/test_train_drift.py \
    tests/test_demo_baseline.py

If it is not, something moved a number that nothing else checks. The failure names the path that differed.

The metric table shows -- where you expected a number

-- means no value exists, never zero. Each has one cause:

  • Adaptation speed, recovery time, performance gap — no ceiling was given. Pass --spec to derive one or --upper-bound to state it.
  • Forgetting — the configuration was seen once. It needs a schedule that brings one back; see Configuration shifts.
  • Effective rank — nothing measured it. Capture is off by default because it costs a decomposition per step; enable it on the policy.
  • Regret — no reference trajectory was supplied. The protocol is deliberately oracle-free.

Errors, and what they mean

the reward spec … has no closed-form upper bound

A component in it declares no limit on the side its weight points to, so the ceiling would be infinite. Of the built-in specifications this is eval8, whose chaos component is unclipped.

It raises rather than substituting a number because substituting one is the fault this derivation exists to correct — the implementation this was ported from defaulted to a ceiling roughly three times the real one, and every policy then looked equally far from it. Estimate a bound with numeric_upper_bound, state one with --upper-bound, or register a derivation — see Extending.

no upper bound given for configuration …

You passed compute_all_metrics a per-configuration mapping that does not cover every configuration in the traces. Deriving it with bounds_by_configuration always covers them; a hand-written mapping may not. It raises rather than falling back, because measuring one configuration against another's ceiling is wrong in a way the resulting number does not reveal.

… is not a valid JSONL trace

A file matched the trace pattern and could not be parsed. It names the file. Reading stops rather than skipping it, because a sweep reported as complete when part of it was unreadable is worse than one that fails.

EWC parameter_scope='lora_only' matched no parameters

The scope matches on a naming convention — PEFT's lora_ prefix. A full fine-tune has no such parameters, so the Fisher would be empty and EWC would train as plain PPO while reporting a healthy sample count. That case raises on purpose. Use parameter_scope="all_trainable", or attach adapters.

the adapter at … loaded with every parameter frozen

A LoRA checkpoint saved with inference_mode=true. Loaded frozen, every update would run, report finite losses, and change nothing. Re-save it from a trainable model.

Repo id must use alphanumeric chars… when loading a checkpoint

An adapter directory whose adapter_config.json names no base model — the empty name is read as a repository. The loader raises with the real cause; if you see the raw Hub error, you are calling transformers directly rather than language_policy.

a response of N tokens does not fit this model's M-token context

The generated response alone exceeds the model's context, so trimming the prompt cannot help. Lower the policy's max_new_tokens, or use a model with more context. Prompts are trimmed automatically; responses are not, because the response is what is being reinforced.

configuration 'x' has a key named 'axes'

Axes are written at the top level of a configuration, not nested:

{"regime_id": "surge", "duration_steps": 500, "ticket_arrival_rate": 0.8}   # yes
{"regime_id": "surge", "axes": {"ticket_arrival_rate": 0.8}}                # no

A near-miss of a known axis name warns rather than raising — tickt_arival_rate is almost certainly a typo, but a genuinely new axis name is allowed, because the format is open.

the policy recorded N generations for a window of M steps

Consume the collector lazily, one window at a time:

for window in collector.windows(total_steps):   # not list(...)
    backend.update(window)

A language policy records the tokens it generated and the backend collects that record at update time. Materialising the run first collects every window's generations before the first update, and the backend refuses rather than pairing one window's rewards with another window's actions.

reset() must be called before step()

Gymnasium's contract. For a live world this matters more than usual: a reset destroys the world and provisions a new one, so the environment will not do it implicitly.

unknown algorithm backend 'x' when a config loads

The name is checked against the registry when the configuration is read, so a typo fails immediately rather than an hour into a run. A backend registered by your own plugin must therefore be registered before the configuration is read.

From Python that is your import order. From the command line, pass --plugin — a dotted module or a path to a .py file, repeatable, and available on run, eval and aggregate alike:

skyfall-crl run --config experiment.yaml --plugin my_package.backends
skyfall-crl run --config experiment.yaml --plugin ./my_backend.py

the manifest names … which is not there

A bundle was copied without its checkpoint/. A self-contained bundle is a directory — manifest and weights together — so copy the directory, not the manifest.

The same error with "an external checkpoint at … no longer exists" means the bundle was exported with --export-external, which records an absolute path and copies nothing. It works only where those weights remain. Re-export without the flag for something portable.

this bundle records no policy to rebuild

The run that produced it did not name its policy, so the weights have nothing to go back into. Name it in the experiment's algorithm.policy section — id plus kwargs — and export again. A backend that constructs its own policy internally gives a bundle that records schemas and weights but cannot reconstitute the policy.

… has no load_weights(path)

The bundle's weights were saved by a policy that could be filled, and the policy its recorded factory builds cannot be. Serving is refused rather than running an untrained policy while reporting success. Give the policy a load_weights(path) — see Extending — or point the bundle at the policy that wrote it.

cannot record a … in a bundle

The environment declares a space this package cannot rebuild faithfully — usually a subclass of a built-in space, which would encode as its base and come back as its base. That is a different schema from the one the policy was trained against, so it is refused rather than recorded. Declare the base space, or add the type to the space encoder.

Environment and install

The [train] extra will not install on macOS

If your interpreter is x86_64 (including Rosetta), PyTorch's last macOS wheel is 2.2.2 and transformers 5.x refuses any torch below 2.5. Use an arm64 interpreter.

Failed to build 'pyarrow'

Should no longer happen: pyarrow used to arrive through trl → datasets, and trl is no longer in the extra because nothing imported it. If something in your environment reintroduces it and pip resolves a version with no wheel for your interpreter, force one: pip install --only-binary=pyarrow -e ".[train]".

Tests skip instead of running

By design. Tests needing a live deployment skip when none answers; tests needing the ML stack skip when [train] is absent. Install the extra, or start a deployment, and they run.