Export and serving — API reference

Every public symbol in skyfall_crl.serve, generated from the source.

Generated page — built from the source docstrings, so this file is blank when read on GitHub. Run mkdocs serve to read it locally, or read the docstrings in the module itself.

The bundle

bundle

A trained policy, described well enough to be used somewhere else.

A checkpoint on its own is not a servable policy. To drive an environment with it you need the shapes it was trained against, the reward it was trained under, and enough about the policy to rebuild it -- and none of that is in the weights. A bundle is that record.

The bundle is self-contained by default. It is a directory holding manifest.yaml beside a checkpoint/ copied into it, and the manifest stores a relative path, so the whole thing can be moved, archived, or handed to someone as one object. external_checkpoint records an absolute path to a checkpoint that stays where it is and copies nothing -- for weights too large to duplicate. The manifest records which of the two it is, so a reader can tell a portable bundle from one that only works on the machine that wrote it.

Everything the environment supplies is optional, and its absence is recorded rather than invented. A Tier-1 environment has no reward spec, may declare no spaces, and need not describe itself; a bundle for one is still a useful record of what it was and what it was trained on. This follows the rule describe_run already states: recording that nothing was reported is honest where inventing a description is not.

Two details are deliberate and easy to get wrong:

• The reward spec is stored by value, not by name. A world can declare an inline spec through its reward-config endpoint, so the spec a policy was trained under may exist under no registered name and in no file. A name would not survive the trip.
• describe() output is stored opaquely, under its own key. Its keys are unspecified by contract, and the one implementation today happens to include a reward_spec entry holding a bare name -- which is not the manifest's own full spec and must never be confused for it.

MANIFEST_NAME module-attribute

MANIFEST_NAME = 'manifest.yaml'

CHECKPOINT_DIRNAME module-attribute

CHECKPOINT_DIRNAME = 'checkpoint'

MissingCheckpoint

Bases: FileNotFoundError

A bundle names weights that are not where it says they are.

WeightsFormat

Bases: str, Enum

How a policy is reconstituted from its checkpoint.

There are genuinely two, because a language model is rebuilt from its checkpoint while an array policy is built and then filled -- and forcing one shape onto both would mean constructing a base model only to discard it, which for a large model is not a detail.

CHECKPOINT The recorded factory takes the checkpoint directory as its first argument and returns a ready policy. language_policy is one. WEIGHTS The recorded factory takes no checkpoint; the policy it returns is then filled by its own load_weights(path). This is the path an array policy takes, and it is why the two example backends can both write weights.npy with different shapes and still load: the shape is the rebuilt policy's business, which is where it was always known.

CHECKPOINT class-attribute instance-attribute

CHECKPOINT = 'checkpoint'

WEIGHTS class-attribute instance-attribute

WEIGHTS = 'weights'

PolicyRef

Bases: BaseModel

How to get the trained policy back.

id and kwargs mirror the experiment configuration's policy section, so a bundle names its policy the same way a run does. format selects which of the two reconstruction protocols applies; None means no weights were recorded at all, which is what a backend that cannot checkpoint produces.

model_config class-attribute instance-attribute

model_config = ConfigDict(extra='allow')

id class-attribute instance-attribute

id: str | None = None

kwargs class-attribute instance-attribute

kwargs: dict[str, Any] = Field(default_factory=dict)

format class-attribute instance-attribute

format: WeightsFormat | None = None

checkpoint class-attribute instance-attribute

checkpoint: str | None = None

external class-attribute instance-attribute

external: bool = False

Bundle

Bases: BaseModel

A policy, its schemas, its reward and its provenance, as one serialisable record.

extra="allow", as the trace record is, so a manifest written by a later version survives a round-trip through an earlier reader instead of losing the fields it did not recognise.

model_config class-attribute instance-attribute

model_config = ConfigDict(extra='allow')

version class-attribute instance-attribute

version: int = 1

policy class-attribute instance-attribute

policy: PolicyRef = Field(default_factory=PolicyRef)

observation_space class-attribute instance-attribute

observation_space: dict[str, Any] | None = None

action_space class-attribute instance-attribute

action_space: dict[str, Any] | None = None

reward_spec class-attribute instance-attribute

reward_spec: dict[str, Any] | None = None

env class-attribute instance-attribute

env: dict[str, Any] = Field(default_factory=dict)

regime class-attribute instance-attribute

regime: dict[str, Any] = Field(default_factory=dict)

run class-attribute instance-attribute

run: dict[str, Any] = Field(default_factory=dict)

provenance class-attribute instance-attribute

provenance: dict[str, Any] = Field(default_factory=dict)

from_dict classmethod

from_dict(data: Mapping[str, Any]) -> Bundle

to_dict

to_dict() -> dict[str, Any]

from_yaml classmethod

from_yaml(text: str) -> Bundle

to_yaml

to_yaml() -> str

save

save(directory: str | Path) -> Path

Write manifest.yaml into directory, creating it if needed.

load classmethod

load(directory: str | Path) -> Bundle

Read a bundle from a directory, or from a manifest file directly.

observation

observation() -> Any

The recorded observation space, or None if the environment declared none.

action

action() -> Any

The recorded action space, or None if the environment declared none.

reward

reward() -> RewardSpec | None

The reward this policy was trained under, or None for a Tier-1 run.

checkpoint_path

checkpoint_path(directory: str | Path) -> Path | None

Where this bundle's weights actually are. None when it recorded none.

The two ways a bundle can be broken get named errors rather than a stray failure deep in a model loader: a relative checkpoint that is not there (a bundle copied in part), and an absolute one that has since moved or been deleted.

export_bundle

export_bundle(
    destination: str | Path,
    *,
    env: Any,
    algorithm: Any = None,
    policy_id: str | None = None,
    policy_kwargs: Mapping[str, Any] | None = None,
    env_id: str | None = None,
    env_kwargs: Mapping[str, Any] | None = None,
    reward_spec: RewardSpec | None = None,
    regime: Mapping[str, Any] | None = None,
    run: Mapping[str, Any] | None = None,
    external_checkpoint: str | Path | None = None,
) -> Bundle

Write a bundle for algorithm's policy as trained against env.

Called while both are still alive, which is the only moment the schemas can be read at all -- and the reason the export hook lives in the run rather than in a command of its own.

external_checkpoint writes the weights to that path and records it absolutely instead of copying them into the bundle. The result is smaller and not portable, and says so.

A backend that cannot checkpoint still exports. The result records spaces, reward and provenance with no weights, which is the honest thing for the array-policy example backends the base install ships with -- and refusing would make export unusable on exactly those.

Serving one

runner

Driving an environment with a bundled policy, and nothing else.

Serving here means inference against a live environment, not a web server. That is what the research code this generalises actually does -- every one of its inference backends is a client to somebody else's process, and a trained checkpoint never goes over HTTP -- and it is what makes a served policy measurable: the run writes the same trace a training run writes, so skyfall-crl eval scores it with no special case.

Inference is a rollout that never updates, so the collector does the driving. That is deliberate reuse rather than convenience: a served run then labels configurations, stamps provenance and writes trace rows by exactly the code a trained run used, and cannot quietly disagree with it.

The task view comes from the bundle, which is the defect this design exists to avoid. The reference selects the environment and the task separately at inference time, so a policy trained on one task can be evaluated under another with nothing objecting; here the schemas and the environment travel with the weights, and an override is something you ask for.

Nothing at module scope imports torch. A language-model bundle pulls it in when its policy is built, and an array bundle never does -- so a policy exported from the example backends can be served on a base install.

ServeResult dataclass

What one inference run produced.

trace_path instance-attribute

trace_path: Path | None

steps class-attribute instance-attribute

steps: int = 0

total_reward class-attribute instance-attribute

total_reward: float = 0.0

regimes class-attribute instance-attribute

regimes: tuple[str, ...] = ()

provenance class-attribute instance-attribute

provenance: Mapping[str, Any] = field(default_factory=dict)

load_policy

load_policy(bundle: Bundle, directory: str | Path) -> Any

Rebuild the policy a bundle describes.

Two reconstruction protocols, chosen by the recorded format, because there genuinely are two: a language model is rebuilt from its checkpoint, while an array policy is built and then filled. See WeightsFormat.

Every way this can fail names what is missing. A policy restored partially -- built fresh and then not filled -- would act, and act like something that was never trained, which is the one failure mode worth ruling out by construction.

build_environment

build_environment(bundle: Bundle, **overrides: Any) -> Any

Rebuild the environment a bundle was trained against.

Keyword overrides are merged into the recorded kwargs, so serving the same policy against a different world is an argument rather than a rewrite.

serve

serve(
    directory: str | Path,
    *,
    steps: int = 100,
    env: Any | None = None,
    trace_path: str | Path | None = None,
    window_steps: int = 64,
    auto_reset: bool = False,
    seed: int | None = None,
    env_kwargs: Mapping[str, Any] | None = None,
    regime: Any | None = None,
    use_recorded_regime: bool = True,
) -> ServeResult

Run a bundled policy against an environment and record what it did.

env overrides the environment the bundle names -- for serving a policy against a world it was not trained on, which is a legitimate thing to want and should be explicit. An environment passed in is still closed on the way out, for the same reason the training run closes one it was handed: a rule about ownership is easy to state and easy to forget, and a world left open costs money.

The configuration schedule the bundle recorded is rebuilt and applied, so a served trace is segmented exactly as the training trace was and the two are comparable. regime overrides it; use_recorded_regime=False runs stationary, which is what you want when serving against a world the recorded schedule does not describe.

Spaces as plain data

spaces

Spaces as plain data, so a schema survives leaving the process.

An exported policy is only usable if the shapes it was trained against travel with it, and Gymnasium offers nothing for that. to_jsonable/from_jsonable exist on every space, but their signature is to_jsonable(self, sample_n) -- they serialise batches of samples, not the space itself; flatten_space returns a different space. So this is written rather than reused.

Four things are recorded here that a naive encoder loses, and each was a real round-trip break in the spaces this package already ships:

• Infinite bounds. Box(-inf, inf) is ordinary and JSON has no infinity. json.dumps emits a bare -Infinity, which is not valid JSON, and yaml.safe_load reads that back as the string '-Infinity'. Non-finite bounds are encoded as tokens instead.
• Text.min_length. Gymnasium defaults it to 1; every Text space in this package sets it to 0. A reconstruction that took the default would reject the empty request body the tool action space emits routinely.
• Text character sets. character_set is a frozenset, and the sets used here are string.printable -- which contains tab, newline and carriage return. The repr is unordered and elides, so the characters themselves are stored, sorted for determinism.
• Discrete.start and MultiDiscrete.start, both silently defaulted to zero on reconstruction.

Correctness is space == space_from_dict(space_to_dict(space)) under Gymnasium's own equality, which is what the tests assert rather than comparing encodings.

None is a supported input and output. Not every environment declares spaces -- three of this package's own test doubles declare none -- so absence is a case to record, not an error.

Dispatch is on the exact type. A subclass of Box encodes as something this module could only reconstruct as a plain Box, so it raises UnsupportedSpace rather than quietly recording a schema that is not the one the policy was trained against.

UnsupportedSpace

Bases: TypeError

A space this module cannot record faithfully, or an encoding it cannot read.

space_to_dict

space_to_dict(
    space: Space[Any] | None,
) -> dict[str, Any] | None

Encode a space as JSON- and YAML-safe plain data.

None in gives None out: an environment that declares no space has said something, and recording that is more honest than inventing a shape for it.

space_from_dict

space_from_dict(
    data: Mapping[str, Any] | None,
) -> spaces.Space[Any] | None

Rebuild a space from space_to_dict output. None round-trips as None.