Adoption cost¶

The question. What does it actually cost to point this package at a simulator it was not written for? Not in principle — measured, on a specific simulator, with the numbers and the friction both written down.

The answer, in one line. The environment adapter is 251 statements in one file outside the package; making it Tier 2 costs 87 more; adding a reward signal costs 8 statements and under a minute; and no module under src/skyfall_crl/ was edited. What it also cost was finding one real gap in this package, described at the bottom.

What was integrated, and why that one¶

DiscoveryWorld — AI2's benchmark for agents that have to discover something rather than execute a known procedure. It was chosen for being unlike the environment this package grew up on, because an integration that only works on something familiar proves very little:

	MORPHEUS	DiscoveryWorld
Where it runs	a server, over HTTP	in-process
Identity	a world id per instance	none
Time	wall-clock, via a scheduler	an explicit `tick()`
Agents	one	many
Actions	tool calls against a live API	JSON verbs against object UUIDs
Ending	never — the world persists	it terminates when the task is solved
Configurations	a chaos profile, changed in place	a scenario, which must be rebuilt

That last row is the interesting one. For this simulator a configuration is a scenario, so a configuration shift rebuilds the world while the policy carries across — which is the task-sequence form of continual learning, not the parameter-drift form this package was built around. It works, and the run stays one continuous run: see the shift is not an episode boundary below.

The walkthrough¶

This is what writing the integration involved, tier by tier — which is what gives the cost table below its meaning. To run it, the commands live in one place: the DiscoveryWorld tutorial.

1. Write the adapter. A gymnasium.Env whose reset loads a scenario and whose step maps a discrete action onto the simulator's JSON verb, ticks, observes, and returns the change in the task score as the reward. That is the whole of Tier 1 — see examples/discoveryworld_env.py.

2. Make it Tier 2, if you want the composable reward. Three additions: a describe() returning whatever provenance you want recorded, info["regime"] naming the active configuration, and info["trace"] carrying a per-step record. The reward is then composed from components you write:

from skyfall_crl.rewards import Cadence, RewardComponent, RewardSignal, StepContext, register


class TaskProgressComponent(RewardComponent):
    name = "task_progress"
    cadence = Cadence.STEP
    clip = (0.0, 1.0)
    default_weight = 1.0
    reads = ("extras",)

    def compute(self, ctx: StepContext) -> RewardSignal:
        return self._signal(max(0.0, float(ctx.extras.get("score_delta", 0.0))))


register(TaskProgressComponent.name, TaskProgressComponent)

ctx.extras is the seam. Six of StepContext's eighteen fields describe MORPHEUS's operational model — incident tickets, a ledger, throughput — and a simulator that reports none of those puts its own payload in extras instead. Do not reuse a shipped spec: every one of them reads those six fields, so composing one here would score a structural zero on every step forever. The composition engine warns when a spec can read nothing from its environment, which is that mistake being caught rather than shipped.

3. Run it. One configuration document and two commands, both in the DiscoveryWorld tutorial.

Expect the numbers to be poor, and negative. That is the demo being honest about what it is: a measurement of integration cost, not of learning. The agent is a hill climber on a 21-entry action menu, and its invalid-action rate is high enough that the small penalty for a refused action dominates the positive terms. Reading the result any other way would be reading it as a research result, which Limits is explicit that nothing here is — and the DiscoveryWorld tutorial works through what those numbers actually say.

What it cost¶

Statements exclude blank lines, comments and docstrings. The files themselves are heavily commented, because most of what was learned is in the comments.

	Tier 1	Tier 2 (extra)	The backend	Adding a reward
Statements	251	87	0	8
Files	1	same file	none	same file
Core modules edited	0	0	0	0
What you implement	`gymnasium.Env`	`describe()`, `info["regime"]`, `info["trace"]`	nothing	`RewardComponent`
Wall-clock	— see below —			37 s

Supporting files, not part of the integration proper: 25 statements of configuration and 157 of tests.

On the wall-clock, honestly. The whole integration — first contact with the simulator through to a green suite and a metrics table — took about 40 minutes. Read that as a floor, not an estimate: it was written by an author who already knew this package intimately and who started from a completed survey of the simulator's API. What it does bound is the shape of the work, which is the part that transfers. The one cleanly isolated figure is the last column: adding a reward signal through the documented plugin path, from nothing to a passing test, was 37 seconds — and that number is meaningful because the path is short, not because the author was fast.

The backend column was the surprise, and closing it changed the package. When this was measured, no shipped backend could train this environment: the example backend owned a policy shaped for CartPole, and the backends that accept an injected policy all train a language model and need the [train] extra — so an environment whose actions were neither had to bring an algorithm as well as an adapter, at a measured cost of 73 statements. Those statements became the package's discrete_hill_climbing built-in, because a cost of the package belongs to the package; an environment with a discrete action menu now selects it by name on a base install, and the column reads zero.

What the integration had to absorb¶

The honest content of "integration cost" is not the line count; it is this list. None of it is a defect in either project — it is what adapting one real system to another actually involves.


Observing is not free, or pure	`getAgentObservation` renders the viewport twice and writes three PNGs per call, with no flag to disable it, and mutates cached state that the step counter and completion check then read. So it is called exactly once per step and its output directory is redirected to a scratch path.
A strict call order	`observe → act → tick`, enforced: a second action before `tick()` is refused. A Gym `step` is therefore act, tick, observe, with the opening observation taken in `reset`.
Worlds cannot be reloaded in place	`loadScenario` appends to its interface list without clearing it, so a second load leaves the previous world's interface live. Every configuration gets a fresh API object with its own thread id.
Failure is not an exception	An unsupported scenario/difficulty pair prints and returns `False`; a valid pair that fails to build calls `exit(1)`, which would end the training process rather than the run. The pair is validated before the call.
The score can go down	It is recomputed every tick with no latch, so a delta can be negative and "completed" can flip back to false. The reward component clips negatives away deliberately, which is also what gives it a ceiling worth measuring against.
No step limit exists	The simulator implements none; its 100/1000-step budgets are a convention in its documentation. The adapter imposes truncation itself.
An unbounded history	Every tick pickles the whole world onto a list that is never trimmed. Contained in the adapter rather than patched upstream.

And one property worth stating as a result rather than a cost: a configuration shift does not end the episode. Rebuilding the world could easily have surfaced as a termination, which would hand the agent the boundary for free — and adapting to an unannounced change is the entire problem being posed. It has its own test.

What it found in this package¶

The shipped command could not run any third-party extension. A configuration naming a registered backend is validated the moment it loads, so a plugin imported afterwards is already too late — and skyfall-crl run had no seam to import one first. Every documented extension point was reachable from a Python script and unreachable from the command the documentation tells you to run.

Fixed by --plugin, which imports a module — dotted name or path to a file — before anything is read, on run, eval and aggregate alike. The gap is pinned by a test that runs the command in a clean interpreter, because the first version of that test passed with the fix deleted: a fixture had already imported the examples, so the command found the backend whether or not it imported anything itself.

What this does and does not establish¶

It does establish that the environment-agnostic core is real: a simulator with no server, no world id, no incident tickets and a genuine terminal state runs through the harness, the configuration schedule, the trace format and the metrics library without a line changing inside the package.
It does not establish that any Gymnasium environment gets the full Tier-2 experience for free. One simulator is a witness, not a family — see Limits.
Nothing here was trained to convergence. The backend is a hill climber on a discrete action menu; the demo measures integration, not learning.
The demo is verified locally, not in CI. Its test skips unless the simulator is installed, the same standing as the live MORPHEUS tests, because the library will not take a dependency on a simulator it merely demonstrates.