Tune the protocol

Every measurement parameter is a flag, shared by eval and aggregate — the flag table lists them all, and the metrics chapter defines what each moves. The recipe below builds the metrics chapter's worked-example trace and shows the two changes you will actually make: switching the anchor, and tightening a threshold.

from pathlib import Path

from skyfall_crl.rewards.trace import EpisodeStep, TraceWriter

rewards = [0.8] * 10 + [0.1, 0.9, 0.9] + [0.6] * 7 + [0.5] * 10
labels = ["a"] * 10 + ["b"] * 10 + ["a_again"] * 10
with TraceWriter(Path("toy.jsonl"), meta={"run_id": "toy"}) as writer:
    for i in range(30):
        writer.write(EpisodeStep(step=i, reward=rewards[i], regime_id=labels[i],
                                 regime_origin="a" if labels[i] == "a_again" else None))
print("30 rows -> toy.jsonl")

The default asks a data-relative question — steps to reach 90% of each segment's own peak:

skyfall-crl eval --traces toy.jsonl

  Metric                      segment
  -----------------------------------
  Per-configuration reward   0.636667
  Adaptation speed                  2
  Forgetting                      0.3
  Recovery time                   1.5
  ...

--anchor ceiling asks an absolute one — steps to reach half of a stated bound — and needs that bound (--spec derives one; --upper-bound states it):

skyfall-crl eval --traces toy.jsonl --anchor ceiling --upper-bound 1.0

  ceiling (stated): 1

  Metric                       stated
  -----------------------------------
  Per-configuration reward   0.636667
  Adaptation speed                1.5
  Forgetting                      0.3
  Recovery time                   1.5
  Performance gap            0.366667
  ...

The column header names what the numbers are measured against, and the two adaptation speeds are answers to different questions — never compare a segment-anchored number with a ceiling-anchored one. Note the gap appearing: it is the one metric that always needs a bound.

Three knobs cover most remaining needs: --alpha moves the adapted threshold (its default is per anchor — 0.9 of a peak, 0.5 of a ceiling — and does not carry across), --epsilon-fraction widens or narrows recovery's band under the segment anchor (--epsilon, an absolute distance, is the ceiling anchor's), and --tail-fraction decides how much of an interval counts as settled. Reproducing someone's table means matching the parameters their analysis used, which are not always their library's defaults — every table this library prints records its anchor for exactly that reason.