← Blog
✎publicación de blog

The game runs a bandit, and the network waits for a harder question

A cinematic photograph of a lone teal slot machine with a chrome lever in an empty room in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

The game runs a three-answer bandit per NPC and player, not the network, until a question needs more than three answers and a per-player table.

Part 5 of 5 in NPC-mind. Previous: Most of the extra machinery did not help, and the game gave no reward to learn from

The game does not run our network today. It runs a three-answer bandit for each NPC and each player, and the scripted behaviour stays as the fallback. That is the state of the project as we can verify it. It is not the decision we first wrote down, and this part explains the gap between the two.

Two decisions, one day, and they disagree

On 2026-11-02 we wrote an owner decision: the network becomes the whole NPC in homeless-sim. The goal was that players should meet NPCs that remember them. The same day, the measured receipts said the network did not earn a place over a simple bandit, and we wrote that down too. The game as built follows the receipts. The owner decision still stands in the plan, and the two cannot both hold as written. We have not settled which governs. That is the open question for the rest of this part.

What the game runs now

The game runs a preset we call hybrid. Its needs and its first answers come from a linear readout, which is a weighted sum of inputs, trained once. Remembering a player, the short lines NPCs shout (we call them barks), and answers to witnesses are fixed rules, not learned. The reservoir network, the recurrent brain described in Part 2, is not in the game. The script stays as the fallback and as the baseline the learner has to beat.

The learner is the three-answer bandit. A bandit is a learner that tries each answer and keeps the one that pays. Ours works per NPC and per player, so the same NPC can answer one player differently from another. Its three answers are comply, refuse and report. A cumulative-sum test, a standard way to detect a drift in a stream of numbers, watches how outcomes compare with what the script expected. When they drift, the bandit takes over from the script. When they do not, the script keeps the decision.

What the game pays for an answer

The game pays an NPC for each answer, in health, cash and need, one tick after the answer. The measured costs were these: comply cost 2, refuse cost 6, and report cost 8, in the game's units. That ordering matters. Refusing a threat is dearer than complying in the short run, and reporting is dearer still. A learner that counts the cost can learn that.

One test shows the effect. We made complying more expensive, and an NPC that had been complying switched to refusing. The learner followed the price. We would not call that wisdom yet. It is a test of the mechanism, and a mechanism is all we have measured so far.

Under game noise, it still flipped

Real games are noisy. We tested four NPCs in a scene with snow, night, a fight and police. The hybrid was a fair test only if it could respond to the threats through all of that. Once the police were set up in the scene, the hybrid flipped its answer in four of four pairs within two threats. It also never took over falsely: it did not switch away from the script when nothing had changed.

We had to change one part of the gate to get that result. A baseline that updates only when the alarm is quiet is biased toward its own alarm, so it now tracks every outcome, slowly. That keeps the baseline honest, and it means the gate reacts to a real change rather than to its own history.

Bugs that only more NPCs showed

With one NPC, two bugs stayed hidden. With more than one, both showed. In the first, one connection to the sidecar served at a time. The sidecar is a separate process that answers the game's questions for the brain, so one connection serving at a time meant the NPCs queued behind each other. In the second, a shelter override in the game swallowed the brain's answer, so the brain's decision never reached the NPC.

Both are game-integration bugs, not brain bugs, and we list them because they are the kind of failure that a single-NPC demo will never show. A test that runs one NPC cannot find them.

The cost of the brain in the game

Our measured cost was about 20 milliseconds per tick for 100 NPCs. The plan asked for a mean server tick under 2 milliseconds for 100 brains. So the brain is well outside its budget. We have not measured the bandit's cost in the same units, so we do not claim a number for it here.

The plan, and what is still open

The plan has nine steps. Step 1 is done: the branch was rebased onto the main game branch and a fix for event handling shipped first. The agreed order for the rest is 2, 5, 6, 8, 7, then 3.

Step 2 is to remember. An extorter and a stranger should get different answers, in three of three runs. Step 5 is life: places for warmth and sleep, and saving and loading an NPC's memory. Step 6 is witness and society: answers when a player is testified against, bribed or scared. Step 8 is to show it, with barks drawn from memory and one debug overlay of intent. Step 7 is scale: a shared route grid and measuring 100 NPCs. Step 3 is learning on the readout, with a forgetting layer, measured against a zero-learning arm on the arena twist, on fresh seeds.

Step 4, training the model on real game data, is postponed. Step 9 is the head-to-head: the reservoir network against a linear readout on the player transfer test. We will report that result either way. It stays open, and it comes last.

What would change our mind

The decision stands until a question needs more than three answers and a per-player table. That is the falsifier we wrote for the game. A bandit can handle a small set of answers per player. It cannot hold a rich memory of what each player did across many kinds of request. If the game needs that, the network earns its place. If it does not, the network stays in the research log and the game keeps the bandit.

We have other levers on the brain itself, and each one has a cost. Telling, where a rewarded speaker shares a page of experience with listeners who learn from it, is a medium-sized change. Lower-variance learning, which gives each state its own baseline or pays each need separately instead of the total, is medium too, and it targets the one-step reflexes our learners still miss. A learned world model could test whether foresight works once the critic can represent value, which is also medium. Fixing the base arm, by choosing the most likely action instead of sampling from the copy, is small. We will do it first, then ask again whether the owned layer beats the script in a stable world.

So, the open question for the next round is the one from the top of this part. Which decision governs the game: the one that ships the network, or the receipts that say the bandit already covers the arena? We will settle it with the next test, not with another argument.

Comentarios

Aún no hay comentarios.

Iniciar sesión para comentar.