Check it yourself
stepmaniay-verification Public
Tests anyone can run against StepManiaY's published builds and ITGmania 1.3.0, the release it grew from.
Clone
Clone with HTTPS
Plain git over HTTPS: no account, no app, nothing else to install.
Download the source tarball 3.0 MB100 lines · 5.6 KB
RawStepManiaY verification kit
Tests anyone can run to check what stepmaniay.com/why says StepManiaY fixed, against ITGmania 1.3.0, the release StepManiaY's engine grew from. You don't need StepManiaY's source: every test runs the public downloads, after checking each one's SHA-256 against the published value, and records what each engine did.
git clone https://stepmaniay.com/verification.git stepmaniay-verification
cd stepmaniay-verification
./verify.sh stability
Without git, the same files are a tarball at stepmaniay.com/verification, with its SHA-256 and the commit it was made from.
What you need
- Docker (Docker Desktop on macOS and Windows; on Windows, run the scripts from WSL or Git Bash).
- git, for the ITGmania build.
- About 3 GB of disk: both engines' builds, and ITGmania's source for the timing test.
Every run is headless (a virtual display, software OpenGL, the Null sound driver), offline, and in a fresh portable copy of the engine holding only the test's songs, so nothing touches a game you already have installed.
The tests
| Command | Time | What it checks |
|---|---|---|
./verify.sh stability |
5 min | Damaged files that broke the old song scan: a WAV whose chunk size is -8, an .ssc with #SPEEDS:=;, a .dwi with #single:, an .ssc with a 1/256 time signature. Each engine loaded the song list, crashed, or hung (no song list after 180 s). |
./verify.sh playback |
3 min | The 1/256 song played through a minimal theme on each engine's own _fallback, the same Lua on both: played, crashed or hung. |
tests/timing/build_itgmania.sh then ./verify.sh timing |
30 min the first time | StepManiaY's 653-note test chart played by a self-test that presses the panels through the engine's own input system at exact moments, with a recorder that writes every judged step and the note it was meant for. |
tests/stability/run.py lists, for every damaged file, the line of ITGmania 1.3.0's source it
reaches. tests/stability/make_songs.py and tests/timing/make_song.py make every file from
nothing (the timing chart is StepManiaY's own, its music a generated click track), so the kit
carries no one else's music or charts.
Stability and playback run the official x86_64 builds (ITGmania publishes no other Linux build), so an Apple silicon Mac runs them under emulation. Timing runs on your machine's own architecture, so an emulator can't stand in for the long frames it makes on purpose.
The ITGmania side of the timing test
ITGmania has no self-test or recorder, so tests/timing/build_itgmania.sh builds ITGmania 1.3.0
(tag v1.3.0, commit 8a3bf8884a, the official release's build) from its own repository, in its
own build container (Utils/Dockerfile-linux), with tests/timing/itgmania-1.3.0-timing-harness.patch
added. The patch only adds lines; ITGmania's judging is untouched and every judgment is its own:
git apply --numstat tests/timing/itgmania-1.3.0-timing-harness.patch # the second column, lines removed, is 0 throughout
To build it without the script, follow ITGmania's own build instructions after:
git clone --branch v1.3.0 --recurse-submodules https://github.com/itgmania/itgmania.git
cd itgmania && git apply ../stepmaniay-verification/tests/timing/itgmania-1.3.0-timing-harness.patch
then point path in tests/timing/engines-<arch>.json at your itgmania binary. The script prints
the binary's SHA-256; each run's results name the binaries they ran.
The self-test's flags are the same on both engines: --smytest=timing turns it on when gameplay
starts, --timingoffsets=16 presses every step 16 ms late (late positive), --timingrate=1.5 sets
the music rate, --timingstutter=60 makes one frame 60 ms long every second.
Results so far
From results/ (November 2026, StepManiaY 1.3.1.2611032302 against ITGmania 1.3.0):
| Test | ITGmania 1.3.0 | StepManiaY |
|---|---|---|
| WAV with a chunk size of -8 | hung | loaded (the file refused, with the reason in userlog.txt) |
.ssc with #SPEEDS:=; |
crashed (segmentation fault) | loaded (the entry skipped and logged) |
.dwi with #single: |
crashed (Unrecognized DWI notes format) |
loaded |
.ssc with a 1/256 time signature, scanned |
loaded | loaded (the signature skipped and logged) |
| The same song, played | hung | played |
| 653-note chart, exact steps, rate 1.0 and 0.75 | 653 W1 | 653 W1 |
| Exact steps, a 60 ms frame every second (3 runs) | 4–5 notes missed | 653 W1 |
| Every step 16 ms late, rate 1.5 (3 runs) | 48–55 steps on the wrong note, 12–13 missed | 653 W1 |
| Every step 16 ms early, rate 1.5 | 16 missed | 653 W1 |
| 10 ms late, a 60 ms frame every second | 5 missed | 653 W1 |
python3 tools/summarize.py results/*/ prints every run's table. Each results folder keeps every
run's log.txt, userlog.txt, crash report, console output and recorder CSV beside results.json.
Not tested here yet
Claims that need hardware: XInput controllers polled every millisecond on Windows, the sound clock's noise and drift over hours on real sound cards, Play Together's sync between machines, and frame times on a Raspberry Pi 5. Those need a capture of the sound or the controller from outside the game, so this kit will carry their procedures and analysis rather than run them in a container.
Sharing what you found
Run it on your machine and publish your results/ folder wherever you like, or send it through
stepmaniay.com/feedback. Results that disagree with ours are
the ones we most want to see.