Make a run reproducible

Reproducibility is available, not automatic: some of what varies is seeded by default and some deliberately is not — the ported algorithms seed neither their replay draw nor their Fisher sample, faithfully to the originals.

Set the seeds that are not set for you — each parameter below exists on the backend named beside it, and nothing else takes one:

run: {seed: 0}            # the environment's reset, and nothing else
algorithm:
  name: ppo_ewc
  params:
    seed: 0               # torch's generator: weight init and sampling (ppo and ppo_ewc)
    replay_seed: 0        # the hindsight-replay draw (ppo and ppo_ewc)
    fisher_seed: 0        # the EWC Fisher sample (ppo_ewc only -- plain ppo has no Fisher)
  policy:
    id: skyfall_crl.train.backends.policy:language_policy
    kwargs: {model: ./sft-ckpt}

run.seed reaches the environment and nothing else. Everything else the seed should reach is written where it belongs — the algorithm's params, as above — and in a sweep that is what ${seed} does: substituted wherever it appears, so every cell's seed reaches whatever the document says it reaches. A misplaced key is an error, not a no-op: a document that puts policy: at the top level is refused at load, naming the key — and a parameter the backend does not take (fisher_seed on plain ppo, say) is refused when the backend is built. One caveat on seed: itself: it calls torch.manual_seed, which is global to the process — right for a reproduction run, wrong to rely on with two seeded backends alive at once.

Prove it, don't assume it. Tutorial L5 does exactly this, gate-verified: the same document run twice produces 600 byte-identical trace rows. The equivalent check for your own run is two runs into two paths — the same document twice, with run.trace_path set to first.jsonl and then second.jsonl — and a comparison of the deterministic JSON:

skyfall-crl eval --traces first.jsonl --format json --output a.json
skyfall-crl eval --traces second.jsonl --format json --output b.json
diff a.json b.json

The JSON carries no timestamp and no absolute path, so identical runs produce byte-identical files — an empty diff is the proof.

What can still differ. A reward component that reads a clock scores by observed_at when the row carries one — traces recorded without it replay against now. Unseeded inference differs every time, which is why a scored serve names --seed. And identical results across seeds is its own warning sign: if every seeded run matches, the seed reached nothing.