Tests that passed for the wrong reason
With about 170 test entries and agents working in parallel, the worst failures were checks that said PASS while proving nothing. Each one got a rule, and five tests still flake under load.
Street Simulator kept its no-hardcoded-numbers rule only because a script enforces it before every full check.
In the first week of Street Simulator we set one hard rule: no number in the game is written into the code. Every speed, price and timer is a setting you can change.
Street Simulator is an open-world survival game built on the Godot 4.7 engine. You start with almost nothing, pick a backstory, and try to climb out of a bad situation in a city that keeps moving without you. Other players share that city, and so do hundreds of computer-run people.
This article covers what the game is, the rules we set before writing code, and how the first stretch of building went. The game's repository holds 590 commits, with dates running from 30 October to 6 November 2026. Most of that happened in about one week.
The core loop is a personal climb-out arc plus shared camps. Those camps are contested by turf wars. Players and computer-run homeless groups fight over territory: begging spots, shelter sites, scavenging routes.
On top of that sit crime (breaking into cars, stealing wheels with the right tools), a law system, jobs, weather and seasons. The design notes list every system we wanted and every decision we made in six rounds on the first day.
The most important line in the design file is short. Every number in the game lives in a settings file that a server admin can edit. Drain rates, prices, crime odds, police response, turf rules and difficulty are all settings. Code reads settings and never contains the number itself.
There is a second part. One config file holds presets such as grounded and harsh, and an admin command reloads it while the server runs. Anyone can run the headless server with their own settings file.
A rule like this only holds if something enforces it. Early on we added a "literal gate", a script that scans the game files for bare numbers and fails the whole check if it finds one. We will come back to what that gate did to us in the third article.
A few decisions shaped everything after them:
The first slice was small on purpose. We split the initial implementation across parallel workers, verifying each step against automated tests.
When we re-ran the full check, seven tests passed:
Each check was tried against a deliberately broken copy (a "mutant"). With the wall removed, the wall test failed. With the night multiplier set to 1, the sun test failed. A test that cannot fail tells you nothing, so we kept this habit for the whole project.
One small thing went wrong: two parts of the work were built side by side, and one waited idle for the other after it had already finished. A quick check from us got it moving again.
The second plan covered the core sandbox systems in 10 steps. It started with two agents and grew to four mid-run. The result was 20 tests, and five full checks in a row passed on the same commit.
Two real flakes showed up on a clean tree. A police officer spawned inside a building in 1 of 12 runs. A cars test read only the last server answer instead of all of them. Both got a regression check and a mutant.
The biggest win was speed. The check script was serial: it ran behind one lock on fixed network ports, so it used 1 of 16 CPU cores, and four agents queued behind each other. Moving to a separate port per test cut the full suite to about 24 seconds.
Parallel workers editing the same configuration file repeatedly overwrote preset sections. The next plan began with a presets gate that fails if a preset is missing a section.
Plan 3 had 14 steps and made the city persistent, again with four agents. It ended with five full passes in a row on a clean checkout.
Two lessons came out of it. First, the sun test was flaky because of the test, not the game. It timed the server's daylight drain with the client's wall clock, so a stalled server looked like no drain at all. We changed it to count server physics ticks. A mutant with a ratio of 1.00 now fails.
Second, verify on a clean checkout. A dirty shared working tree once gave a false pass. Concurrent processes overwrote git staging areas until each worker was given an isolated environment.
Key milestones remained untried: no human had played the game, the Windows build was untested, and multiplayer was unverified beyond two clients.
Plan 4 item 3 was the UI: a table-driven file of 215 lines with 6 tabs. The first UI test passed with the Use button disabled, because the client read its starting hunger before the server's value of 50 arrived. A mutant caught it. Reading the rendered tabs with OCR caught a blank zone owner that no assertion had covered.
The game was renamed from its working title to Street Simulator on 3 November. The repository, notes and search index were renamed with it.
On 5 November a full check gave 194 passing tests. The status page lists what is built: places and rooms, phones, animals, a law system with trials, cars, an underground with a metro and sewers, squats, heists and more. It also lists what is planned and not in code, such as a named heir when you die.
Five tests still fail only inside the full run, and pass when run alone. We treat them as load flakes, and the third article covers them.
Commenti
Ancora nessun commento.
Accedi per commentare.