Skip to content

Export and serve a policy

A trained policy leaves the experiment as a bundle — one movable directory: manifest beside checkpoint — and comes back as an inference run that writes the same trace training writes, so it is scored with no special case.

The whole flow needs a complete experiment document — the policy named, the environment named, and auto_reset set for an environment whose episodes genuinely end:

# file: experiment.yaml
env:
  id: CartPole-v1
algorithm:
  name: discrete_hill_climbing
  params: {seed: 1}
  policy:
    id: skyfall_crl.train.backends.hill_climbing:linear_menu_policy
    kwargs: {n_actions: 2, seed: 1}
rollout:
  window_steps: 64
  total_steps: 256
  auto_reset: true
regime:
  provider: scheduled
  params:
    schedule:
      regimes:
        - {regime_id: calm, duration_steps: 128}
        - {regime_id: brisk, duration_steps: 128}
run:
  name: my-policy
  seed: 1

Export during the run, serve from the bundle, score the served trace:

skyfall-crl run --config experiment.yaml --export bundles/my-policy

[1/1] my-policy: 256 steps, 2 configurations

skyfall-crl serve --bundle bundles/my-policy --steps 200 --seed 7 --auto-reset --traces served.jsonl

skyfall-crl eval --traces served.jsonl

Tutorial L5 runs this whole flow, gate-verified, including the served table. The parts that go wrong in practice:

  • The policy must be named in the document. A backend's internal default policy cannot be recorded, and a bundle without a policy reference cannot be served.
  • A custom array policy needs load_weights(path). A bundle whose weights the rebuilt policy cannot take is refused, never restored empty — a partially restored policy acts like something untrained while every log line reports success.
  • --seed on any serve you will score; unseeded inference is legitimately different every time. --auto-reset for environments with genuine terminal states, same as training.
  • The recorded schedule is replayed by default, so served traces segment like training traces; --stationary declines it, and --env/--env-kwargs deliberately override the environment when serving against a different world is what you asked for — nothing validates that the bundle's schemas fit the substitute, so whether they still do is your call.
  • A matrix export (run --matrix --export DIR) gives every cell its own bundle subdirectory; --export-external records a path instead of copying weights, and the bundle then only works where that path resolves.