181 lines
6.1 KiB
Markdown
181 lines
6.1 KiB
Markdown
# Archipelago World Generation Tester
|
||
_Disclaimer: AI was used in this repository_
|
||
|
||
This repository contains a python packages that can check whether community Archipelago worlds can generate a valid seed without bringing down the multi-game world.
|
||
|
||
The [community worlds spreadsheet](https://docs.google.com/spreadsheets/d/1iuzDTOAvdoNe8Ne8i461qGNucg5OuEoF-Ikqs8aUQZw/edit?gid=58422002#gid=58422002) is a great resource to showcase the games that are in development, but the stability label may not reflect its current state. I wanted to play games that were labeled `unstable` but were at least be stable enough to allow reliable world generation. This tester uses the sheet to collect the latest APworlds and apply a series of tests to assess their generation stability (see #testing-overview). Only APworlds that are available via direct links or Git repositories are tested.
|
||
|
||
To see the results of this live in action, see [worlds.archipelago.skillysue.nl](https://worlds.archipelago.skillysue.nl) for an overview of all listed games and their test results.
|
||
|
||
## Contents
|
||
- [Requirements](#requirements)
|
||
- [Installation](#installation)
|
||
- [System packages](#system-packages)
|
||
- [Install the package](#install-the-package)
|
||
- [Configuration](#configuration)
|
||
- [GitHub](#github)
|
||
- [Testing overview](#testing-overview)
|
||
- [Test worlds](#test-worlds)
|
||
- [Test all worlds](#test-all-worlds)
|
||
- [Test a local `.apworld`](#test-a-local-apworld)
|
||
- [Related Projects](#related-projects)
|
||
|
||
## Requirements
|
||
|
||
- **Python 3.10 or newer**
|
||
- **Git** — used to clone Archipelago and check out the version being tested
|
||
- **Docker** — must be usable by your user without `sudo`
|
||
- **A GitHub token** — required to access the GitHub API. It does not need any scopes
|
||
- **Base ROMs** — optional; some worlds cannot generate without them
|
||
|
||
## Installation
|
||
|
||
### System packages
|
||
|
||
On Debian-based distros:
|
||
|
||
```sh
|
||
# update and install the required packages
|
||
sudo apt update
|
||
sudo apt install -y git python3 python3-venv
|
||
```
|
||
|
||
Install Docker:
|
||
|
||
```sh
|
||
# Download docker
|
||
curl -fsSL https://get.docker.com | sudo sh
|
||
|
||
# Create docker usergroup
|
||
sudo usermod -aG docker "$USER"
|
||
newgrp docker
|
||
|
||
# Test if it can run without sudo
|
||
docker run --rm hello-world
|
||
```
|
||
|
||
### Install the package
|
||
Install the latest version with pip:
|
||
```sh
|
||
pip install --no-cache-dir https://git.skillysue.nl/Skilly/APWorldTester/archive/latest.tar.gz
|
||
```
|
||
|
||
Or clone the repo and install it:
|
||
```sh
|
||
# Clone repo
|
||
git clone git.skillysue.nl/APWorldTester && cd APWorldTester
|
||
|
||
# Create environment
|
||
python3 -m venv .venv
|
||
source .venv/bin/activate
|
||
|
||
# Install APWorldTester
|
||
pip install -e .
|
||
```
|
||
|
||
### Configuration
|
||
|
||
Write a starter config and edit the paths in it:
|
||
|
||
```bash
|
||
apworld-tester init
|
||
```
|
||
|
||
That writes the template config to `~/.config/apworld-tester/config.yaml`.
|
||
Two settings that require attention:
|
||
|
||
- `general.output_directory` The output directory for running the tester, including cloning the Archipelago repo and downloaded apworlds. Must be changed, otherwise an error is thrown.
|
||
- `general.roms_directory` Directory where base ROMs are located for apworlds that need one. Optional, but increases the number of testable worlds.
|
||
|
||
### GitHub
|
||
The tester checks and downloads hundreds of apworlds from GitHub repositories. It is highly recommended to register a GitHub token and store it in an .env file as follows:
|
||
|
||
```text
|
||
GITHUB_TOKEN=ghp_<your_token_here>
|
||
```
|
||
|
||
## Testing overview
|
||
|
||
Each world is tested five ways:
|
||
|
||
- by itself
|
||
- with two players of the same game
|
||
- with 2–5 of Archipelago's built-in worlds
|
||
- by itself with its options randomized
|
||
- with other worlds and its options randomized
|
||
|
||
Each test is repeated with different seeds. The results are combined into one of six stability values:
|
||
|
||
| Value | Meaning |
|
||
| --- | --- |
|
||
| Stable | Every attempt succeeded |
|
||
| Minor issues | Solo and multiworld tests worked, but a randomized test failed |
|
||
| Flaky | Some attempts succeeded and others failed |
|
||
| Solo only | Works by itself, but fails with other worlds |
|
||
| Broken | Could not generate at all |
|
||
| Unknown | No usable result, for example because the `.apworld` was missing or a ROM was required |
|
||
|
||
A successful test means the world loaded, generated a seed, and computed the spoiler log at the configured level. With the default `playthrough` level, this also checks that the seed is completable.
|
||
|
||
|
||
## Test worlds
|
||
The `TestRun` class can be used to test one or more worlds:
|
||
|
||
```python
|
||
from apworld_tester import TestRun
|
||
|
||
test = TestRun("Baba Is You")
|
||
test.run()
|
||
```
|
||
|
||
To test multiple worlds, supply the names as a list:
|
||
|
||
```python
|
||
test = TestRun(["Baba Is You", "Anodyne"])
|
||
test.run()
|
||
```
|
||
|
||
To test all worlds that have already been downloaded, add `cached=True`:
|
||
|
||
```python
|
||
test = TestRun(cached=True)
|
||
test.run()
|
||
```
|
||
|
||
The first run clones Archipelago and builds the Docker image, which can take several minutes.
|
||
|
||
## Test all worlds
|
||
The real benefit of the tester is to parse the community worlds spreadsheet and re-test all changed apworlds. The `UpdateRun` class does exactly this. It updates Archipelago if needed, refreshes the community worlds spreadsheet, downloads and tests changed worlds:
|
||
|
||
```python
|
||
from apworld_tester import UpdateRun
|
||
|
||
updater = UpdateRun()
|
||
updater.run()
|
||
```
|
||
|
||
## Test a local `.apworld`
|
||
In case you want to test a world for which you have an apworld locally, you can supply a directory instead. First, put the `.apworld` in its own directory:
|
||
|
||
```text
|
||
~/my-apworlds/
|
||
└── My Game/
|
||
└── my_game.apworld
|
||
```
|
||
|
||
Then run the following:
|
||
|
||
```python
|
||
from apworld_tester import TestRun
|
||
|
||
test = TestRun(
|
||
game="My Game",
|
||
root_directory="~/my-apworlds",
|
||
)
|
||
test.run()
|
||
```
|
||
|
||
## Related Projects
|
||
- [Community Spreadsheet](https://docs.google.com/spreadsheets/d/1iuzDTOAvdoNe8Ne8i461qGNucg5OuEoF-Ikqs8aUQZw/edit?gid=58422002#gid=58422002): A curated list of games and resources. Without the spreadsheet, this work would be impossible.
|
||
- [Archipelago Games Library](https://mk-404.github.io/Archipelago-Games-Library/): A list of available games and tools made by the community. Very player friendly.
|
||
- [Archipelago fuzzer](https://github.com/Eijebong/Archipelago-fuzzer): A set of scripts to generate multiworlds to record failures. |