The NPC brain still loses to the script it copied
A shared frozen instinct plus a small owned layer per NPC, tested against a kill line we wrote before running anything. The brain passed the line and still lost to the script it came from.
Each NPC's own learning touches only 216 readout weights on top of a shared base of 7,873 synapses.
Part 2 of 5 in NPC-mind. Previous: The NPC brain still loses to the script it copied · Next: In-life learning wins back about 0.10, but only after a shift
Each NPC in our test world has 192 hidden neurons, and most of the brain is shared. Only 216 of its weights belong to that one NPC. That split is the design, and this part explains what the brain sees, what it can do, and what it is allowed to change.
The test world is a 32 by 32 grid. Trees block walking and sight. Each NPC has a facing (north, east, south or west), and it sees only what is in front of it. Every tick, the brain receives 64 numbers. We group them into six kinds.
First, vision. An NPC has eight rays spread across a 90 degree cone in front of it, plus a gaze direction it can move. Each ray reports five kinds of thing it might hit: food, a forge (the place where work happens), a wolf, another NPC, or a tree or wall. The reading is closeness: 1 minus the distance divided by 6, so a thing six cells away reads 0 and a thing right next to the NPC reads near 1. Eight rays times five kinds gives 40 numbers.
Second, the four needs: hunger, safety, social contact and purpose. Third, four touch and hearing flags. These say whether the NPC stands on ready food, on a forge, next to another NPC, or near a wolf. Fourth, three numbers for gaze direction. Fifth, a copy of what the NPC just did: eight numbers for its last body action and three for its last eye action. This is called an efference copy, and it lets the brain know its own previous move. Sixth, two numbers for the time of day, as a sine and a cosine, so that the clock wraps around smoothly.
Count them up: 40 plus 4, plus 4, plus 3, plus 11, plus 2. That is 64.
What is missing matters as much as what is there. The brain gets no global position and no map. It does not get a vector pointing to the nearest food. An NPC knows food only when its rays hit it. We built it this way on purpose, because a brain that knows where everything is would be a cheat, and the script does not have that knowledge either.
The brain has two output heads. The body head has eight choices: move forward, turn left, turn right, eat, work, talk, flee, or rest. The eye head has three: look left, look centre, or look right. Each head turns its scores into probabilities, using a rule called softmax, and the NPC picks one choice from each head at random according to those probabilities. A high score makes a choice likely, but not certain.
Inside, the brain has 192 hidden neurons in six groups. Visual neurons take the 64 vision numbers, drive neurons track the needs, integration neurons combine them, memory neurons hold on to what was seen, and two motor groups drive the body and the eyes. Each neuron is a leaky unit: it keeps part of its previous state every tick. That is what lets the brain carry memory from one tick to the next. The whole base has 7,873 connections, called synapses.
The base is shared by every NPC in a world and is frozen during play. We created it in two steps. First, we trained it to copy the scripted schedule, choosing what the script would choose in each situation. The copy matched the script about 59 percent of the time in the vision world. Second, we evolved only the readout, the final layer that turns activity into choices, using a method called evolution strategies. Each round, we try many small random changes to the readout weights, score each change on training worlds, and move toward the changes that scored best. We ran 40 rounds, with 16 paired nudges each, for each seed. That step took the base from 0.419 to 0.530 in satisfaction.
On top of the frozen base sits the owned layer. For the body head, it is a set of corrections to the readout: 8 body choices times 24 body motor neurons gives 192 weights. For the eye head, it is 3 eye choices times 8 eye motor neurons, which gives another 24. Together that is 216 live weights. Every other weight in the brain is shared and frozen.
The learning rule is simple to state, even if the maths takes more work. When an NPC makes a choice, the weights that led to that choice move a little. How much they move depends on two things: a learning rate, which is part of the NPC's inherited traits, and a signal called m, which is how much the choice changed the NPC's needs. If the choice helped, the weights that led to it strengthen. If it hurt, they weaken. Scientists call this kind of rule a third-factor rule, because a separate signal decides whether the change counts. Our third factor is the need change.
Each NPC also has a genome of about 32 numbers. These are inherited traits: the learning rate, how quickly each group of neurons forgets, gain (how strongly a neuron responds), the weight given to each need, and a seed that decides which connections are masked. Children get a mix of their parents' genomes with a small random change, a process called mutation. They never get the learned weights. When an NPC dies, its owned layer dies with it.
The brain is written in Mojo, a language built for fast numeric code. Because a fast brain can be wrong in fast ways, we check it against a slow reference written in Python, which we call the oracle. The Mojo version matches the oracle to a worst relative error of 1.8 times ten to the minus thirteen on the parity checks. The GPU version, which runs the same brains on a graphics card, matches the Mojo version to a worst relative error of about 4.5 times ten to the minus seven. That is the precision you would expect from single-precision floating point.
The full gate, which runs in check.sh, takes 11 minutes. It covers the parity checks, a self-test of the world with 11 scenes (for example, a tree must block a ray, and a turn must hide food from view), and seven mutants. A mutant is a deliberately broken copy of the code, such as one with the learning sign flipped. The gate passes only if every mutant fails. A check that passes a broken copy proves nothing, so we test the test.
Speed was the reason for the GPU. A brain takes about 5.6 kilobytes on the graphics card, against about 41 kilobytes on the processor. The graphics card we use runs the brains of a whole world at once, so a sweep of many arms and seeds finishes in one process rather than many.
The owned layer is supposed to stay tied to one specific base. If the base changed between saves, every NPC's learned weights would silently mean something else. We planned a hash check of the base on every save load to catch that. The check is not built yet, so the guarantee is currently by convention, not by code.
Part 3 covers what the owned layer does once the world changes, which is the one place where it clearly won.
Kommentare
Noch keine Kommentare.
Anmelden um zu kommentieren.