Score a run you already have¶
The evaluation layer reads trace files and nothing
else, so a run produced by any system scores identically — the whole contract is one JSON object
per step with step and reward, and every further field sharpens what can be measured.
Write rows (from anything — this block fakes four), then point eval at them:
import json
rows = [
{"step": 0, "reward": 0.2, "regime_id": "before", "run_id": "mine"},
{"step": 1, "reward": 0.4, "regime_id": "before", "run_id": "mine"},
{"step": 2, "reward": 0.1, "regime_id": "after", "run_id": "mine"},
{"step": 3, "reward": 0.5, "regime_id": "after", "run_id": "mine"},
]
with open("yours.jsonl", "w") as handle:
for row in rows:
handle.write(json.dumps(row) + "\n")
print("wrote yours.jsonl:", len(rows), "rows")
1 trace(s), 2 configuration(s)
Metric segment
---------------------------------
Per-configuration reward 0.3
Adaptation speed 2
Forgetting --
...
What sharpened what: regime_id gave the run two configuration intervals, which is what made
adaptation measurable at all; forgetting stays -- because no configuration recurs
(regime_origin on a revisit is what pairs encounters); adding reward_components per row would
light up the per-component readings, and identity fields (run_id, seed, algorithm_id) are
what let a directory of files group into runs and families. --traces takes a directory tree and
is repeatable; a ceiling comes from --spec or --upper-bound
when you want the gap.