Archipelago World Generation Tester

Disclaimer: AI was used in this repository

This repository contains a python packages that can check whether community Archipelago worlds can generate a valid seed without bringing down the multi-game world.

The community worlds spreadsheet is a great resource to showcase the games that are in development, but the stability label may not reflect its current state. I wanted to play games that were labeled unstable but were at least be stable enough to allow reliable world generation. This tester uses the sheet to collect the latest APworlds and apply a series of tests to assess their generation stability (see #testing-overview). Only APworlds that are available via direct links or Git repositories are tested.

To see the results of this live in action, see worlds.archipelago.skillysue.nl for an overview of all listed games and their test results.

Contents

• Requirements
• Installation
  • System packages
  • Install the package
  • Configuration
  • GitHub
• Testing overview
• Test worlds
• Test all worlds
• Test a local .apworld
• Related Projects

Requirements

• Python 3.10 or newer
• Git — used to clone Archipelago and check out the version being tested
• Docker — must be usable by your user without sudo
• A GitHub token — required to access the GitHub API. It does not need any scopes
• Base ROMs — optional; some worlds cannot generate without them

Installation

System packages

On Debian-based distros:

# update and install the required packages
sudo apt update
sudo apt install -y git python3 python3-venv

Install Docker:

# Download docker
curl -fsSL https://get.docker.com | sudo sh

# Create docker usergroup
sudo usermod -aG docker "$USER"
newgrp docker

# Test if it can run without sudo
docker run --rm hello-world

Install the package

Install the latest version with pip:

pip install --no-cache-dir https://git.skillysue.nl/Skilly/APWorldTester/archive/latest.tar.gz

Or clone the repo and install it:

# Clone repo
git clone git.skillysue.nl/APWorldTester && cd APWorldTester

# Create environment
python3 -m venv .venv
source .venv/bin/activate

# Install APWorldTester
pip install -e .

Configuration

Write a starter config and edit the paths in it:

apworld-tester init

That writes the template config to ~/.config/apworld-tester/config.yaml. Two settings that require attention:

• general.output_directory The output directory for running the tester, including cloning the Archipelago repo and downloaded apworlds. Must be changed, otherwise an error is thrown.
• general.roms_directory Directory where base ROMs are located for apworlds that need one. Optional, but increases the number of testable worlds.

GitHub

The tester checks and downloads hundreds of apworlds from GitHub repositories. It is highly recommended to register a GitHub token and store it in an .env file as follows:

GITHUB_TOKEN=ghp_<your_token_here>

Testing overview

Each world is tested five ways:

• by itself
• with two players of the same game
• with 2–5 of Archipelago's built-in worlds
• by itself with its options randomized
• with other worlds and its options randomized

Each test is repeated with different seeds. The results are combined into one of six stability values:

Value	Meaning
Stable	Every attempt succeeded
Minor issues	Solo and multiworld tests worked, but a randomized test failed
Flaky	Some attempts succeeded and others failed
Solo only	Works by itself, but fails with other worlds
Broken	Could not generate at all
Unknown	No usable result, for example because the .apworld was missing or a ROM was required

A successful test means the world loaded, generated a seed, and computed the spoiler log at the configured level. With the default playthrough level, this also checks that the seed is completable.

Test worlds

The TestRun class can be used to test one or more worlds:

from apworld_tester import TestRun

test = TestRun("Baba Is You")
test.run()

To test multiple worlds, supply the names as a list:

test = TestRun(["Baba Is You", "Anodyne"])
test.run()

To test all worlds that have already been downloaded, add cached=True:

test = TestRun(cached=True)
test.run()

The first run clones Archipelago and builds the Docker image, which can take several minutes.

Test all worlds

The real benefit of the tester is to parse the community worlds spreadsheet and re-test all changed apworlds. The UpdateRun class does exactly this. It updates Archipelago if needed, refreshes the community worlds spreadsheet, downloads and tests changed worlds:

from apworld_tester import UpdateRun

updater = UpdateRun()
updater.run()

Test a local .apworld

In case you want to test a world for which you have an apworld locally, you can supply a directory instead. First, put the .apworld in its own directory:

~/my-apworlds/
└── My Game/
    └── my_game.apworld

Then run the following:

from apworld_tester import TestRun

test = TestRun(
    game="My Game",
    root_directory="~/my-apworlds",
)
test.run()

Related Projects

• Community Spreadsheet: A curated list of games and resources. Without the spreadsheet, this work would be impossible.
• Archipelago Games Library: A list of available games and tools made by the community. Very player friendly.
• Archipelago fuzzer: A set of scripts to generate multiworlds to record failures.