Skip to content

Run DiscoveryWorld

The ladder's last rung, and its hardest test. AI2's DiscoveryWorld is a benchmark of scientific-discovery tasks, and nobody here wrote it. It has no server, no world id, no notion of an incident, several agents, and it genuinely ends when its task is solved — which is to say it shares almost nothing with the operational worlds this package was built for.

That is why it is worth running. Everything above the environment adapter is written against the Gymnasium API, and this is the evidence: a simulator with none of MORPHEUS's shape trains and scores here with nothing changed inside the package.

If you want to know what writing that integration cost, Adoption cost measures it. This page is how to run it.

1. Install the simulator

It is not a dependency of this package and will not become one — a simulator brings its own, and carrying them for everyone would be the wrong trade.

pip install pygame pathfinding termcolor matplotlib
pip install --no-deps git+https://github.com/allenai/discoveryworld

2. Run the shipped configuration

PYTHONPATH=. skyfall-crl run --config examples/discoveryworld_experiment.yaml \
    --output-dir runs --plugin examples.discoveryworld_env

[1/1] discoveryworld: 18 steps, 3 configurations, 4.1s -> discoveryworld

Two flags are doing work here.

--plugin imports a module before the configuration document is read, which is what lets the document name something that module defines. One is enough: the environment is a plugin, and the algorithm the config names — discrete_hill_climbing — is a built-in.

PYTHONPATH=. is needed only because examples/ is not an installed package. Your own environment, installed, is importable already. A loose file works too: --plugin ./my_env.py, then name it my_env:make_env.

3. What is different about this world

The interesting part, and the reason this is a separate tutorial rather than a footnote to the MORPHEUS one.

A configuration here is a scenario. In a MORPHEUS world a configuration shift reconfigures a world that persists — the same world, under new conditions. In DiscoveryWorld a shift rebuilds the world entirely: a different task, a different map, a different set of objects. The policy carries across; the world does not.

regimes:
  - {regime_id: nutrients,       duration_steps: 6, scenario: Plant Nutrients,    difficulty: Easy}
  - {regime_id: archaeology,     duration_steps: 6, scenario: Archaeology Dating, difficulty: Easy}
  - {regime_id: nutrients_again, alias_of: nutrients, duration_steps: 6}

Both are legitimate continual-learning settings, and they are not the same one — they are the two experimental shapes the field takes: a schedule that rebuilds the world is the task-sequence setting most continual-RL benchmarks use, and a schedule that reconfigures a persistent one is the setting MORPHEUS poses. The package spans both, and that is the claim this page exists to support.

Two smaller differences worth noticing in that config:

  • auto_reset: true, because this environment genuinely terminates. A MORPHEUS world never does — terminated there is always false, since an operational world has no goal state.
  • alias_of on the third entry, for the same reason as everywhere else: forgetting is a comparison, and without a revisit it has nothing to compare.

4. Run it properly

Eighteen steps is a smoke test — enough to prove the wiring, too short to measure anything. A real run is cheap, at roughly 6–7 steps a second:

env:
  id: examples.discoveryworld_env:make_env
  kwargs:
    tier: 2
    max_steps: 60
    schedule: &schedule
      regimes:
        - {regime_id: nutrients,       duration_steps: 60, scenario: Plant Nutrients,    difficulty: Easy}
        - {regime_id: archaeology,     duration_steps: 60, scenario: Archaeology Dating, difficulty: Easy}
        - {regime_id: nutrients_again, alias_of: nutrients, duration_steps: 60}
algorithm: {name: discrete_hill_climbing, params: {n_actions: 21, noise: 0.3, seed: 0}}
rollout: {window_steps: 30, total_steps: 180, auto_reset: true}
regime: {provider: scheduled, params: {schedule: *schedule}}
run: {name: dw-long, seed: 0}

PYTHONPATH=. skyfall-crl run --config long.yaml --output-dir runs \
    --plugin examples.discoveryworld_env
skyfall-crl eval --traces runs

180 steps takes about 27 seconds.

5. Read what it says

  Per-configuration reward      -0.045
  Adaptation speed                  31
  Forgetting                0.00583333
  Recovery time                     19
  Stability (variance)      0.00134416
  Invalid-action rate         0.983333

Adaptation speed 31 and recovery 19 are real numbers, which the smoke run could not give you — at eighteen steps both collapse to 1, because there are not enough steps after a shift for a running mean to do anything. That is the difference between proving the wiring and measuring something.

The reward is negative, and that is not a bug. Look at the last row: the agent proposes an invalid action 98% of the time, and the reward spec charges a small penalty for each one. The penalty dominates the two positive terms, so the total goes below zero.

That number is information, not noise. DiscoveryWorld's real action space is JSON verbs addressed to object UUIDs. A 21-slot discrete menu is a crude projection of it, and a hill climber choosing uniformly from that menu will mostly name something that does not apply to what is in front of it. The metrics are working correctly and telling you the policy is bad.

Two honest consequences:

  • This is a measurement of integration, not of learning. Nothing here has been trained to convergence, and reading these numbers as a research result would be reading them as something Limits is explicit that nothing here is.
  • If you want a run that only ever climbs, drop action_validity from the spec. But the 98% is real information about how well a discrete menu fits this simulator, and it is worth seeing before you hide it.

What the numbers permit in general is Reading the numbers without being misled.

Where next