← Blog
✎Blogbeitrag

A Memory Line Agents Did Not Follow

A cinematic photograph of a closed teal logbook with a red ribbon on a wooden table between brass oil lamps in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

A learned file-memory line that beats recency and frequency offline did not change how often agents re-read files in a live A/B of 1,090 matched turns.

This is the seventh part of the series, and the first one that is not about the network itself. Since the last part, on 2026-11-06, we ran two more experiments on the same machine, and both changed how we work. One is a memory line that tells an agent which files it probably needs next. The other is four rules that came out of two outages in the rebuild and lock code that keeps the network fresh.

The memory line is the sad part. It won on paper and showed no effect we could detect in use.

What the memory line does

Every file, note or skill an agent has touched in a session becomes a small item with its own memory. The memory is a 59-parameter GRU, a small recurrent network that reads each event and keeps a short state. When an agent starts a turn, the model ranks the items seen earlier in the session and shows the top five in a memory: line in the preflight packet.

Offline, on the 300 newest of 1,000 sessions, with settings picked on the 700 older ones, the model scored recall@5 of 0.599. That means: of the already-seen files an agent touched in its next turn, about 60 percent were in its top five. The usual rule, recency plus frequency, scored 0.534. The ceiling for that test is 0.928, so there was room left.

The live miss

Offline numbers do not travel by themselves. On 2026-11-04 we replayed the live sessions. On 721 live predictions, precision was 0.217 and recall was 0.144 of the files that were touched again. The same 47 sessions, run offline with the live settings, gave 0.609. So the model was fine and the loss was in the path between the model and the agent.

The diagnosis found that 347 of the 724 live rows fired on turns with no mined prompt within 10 seconds, which are meta prompts. Half of the items the agent touched were directories, which were never shown. Under strict matching, live file recall was 0.379 and the replay gave 0.510. The gap came from matching rows to turns, and after the fix the two agreed on the path. We kept a pool of 30 candidates and showed the count and age of each file.

The A/B: no effect we could detect

The line was then switched on for a random half of turns, and the other half got nothing. The test ran on 2026-11-07 with 2,030 logged rows, 1,090 matched turns, 559 with the line and 531 without.

measure line on line off p
repeat reads per read 0.281 0.245 0.28
repeat reads per turn 0.159 0.179 0.43
file recall 0.383 0.393 0.80

None of the differences is anywhere near significant. The honest reading is that agents did not follow the line. The test was also too small. A drop from 0.25 to 0.20 repeat reads per read needs about 1,175 reads in each arm, and we had about 350. The test could not see a drop from 0.25 to 0.20, so it cannot rule out a change of that size.

The v2 line, not yet measured

The second version makes the line an instruction. It says "already read, unchanged, do not re-read" only when the file's modification time is no later than the agent's last touch and the last read was under an hour ago. Otherwise it says the file is unchanged but old, or changed since the last read. Rows from this version are tagged v2, and the timer for the next A/B was set at 5,000 v2 rows. That test has not run to its end, so we report no result for it.

A gate that blocks repeat reads, not yet measured

A second tool sits beside the memory line. read_dedupe_gate.py denies a whole-file Read when the file is already in the agent's context, unchanged, and there has been no compaction since. Eleven unit tests pass, and one planted bug in the compaction check was caught. On a 2 MB transcript it takes 20 ms. Its A/B splits by session, using a hash parity, and logs each block to a file. The live report now also counts tokens per turn for each arm. None of that has a result yet. We will not quote a token saving until the log has one.

Seven rules from the network's rebuild

The network's rebuild runs on every commit. Several failures in the same week turned into entries in docs/rules.md. Seven entries came from the work since 2026-11-06. Each is written as "what broke, then the rule". The four below are the ones about the rebuild and the locks:

  • A rebuild triggered by a hook ran in full on every commit: 50 seconds and 8.4 GB each time. The rule is to coalesce, stamp and rate-limit any work a hook triggers, with one trailing run for the changes that came in meanwhile.
  • A detached worker held its own lock while it ran the job the lock guarded, so a job it should have scheduled was lost. The rule is to drop the lock before running the job, and to test with a forced run during the sleep.
  • A detached child inherited a lock file descriptor and held the lock for 30 minutes. The rule is to close lock descriptors on every background spawn and to check the descriptors of the child process.
  • A memory-mapped file's owner was destroyed before a copy that still read from it, which crashed a build. The rule is to keep the owner alive past its last raw use.
  • A test build was written over the live binary and broke the hooks' queries for about two minutes. The rule now is to build candidates to a separate path and move them into place only after they pass.

Two more entries came from the same work. A speed shortcut changed how duplicate keys were handled for 0.6 seconds of gain, so a shortcut must keep the slow path's result on every input class. A debounce must be tested under real churn, with many commits in a short window, not on two idle reruns.

Where the series stands

Today the merged network holds 1,640,443 neurons across the Brain notes, the paper vaults, the math results and the text stores, counted on 2026-11-13. The Brain notes alone are 7,382 neurons, up from 5,341 in our earlier count. The Wikipedia store is separate, with 7,219,311 neurons and 79,377,928 synapses. A warm refresh now takes about 16 seconds at 5.8 GB, down from 49 seconds and 8.4 GB, and runs at most once per 30 minutes.

Two findings from earlier parts rest on measurements we have not repeated. The vault had about two links per note on 2026-10-21, and spreading along those links did not beat plain search there. The best gains we have measured remain the held-out link recall on dense graphs and the reranker's accuracy, not the spread alone.

What is not yet measured: the v2 memory line, the read dedupe gate, the token savings per arm, and whether the skill hooks suggest the right skill. We list them as open rather than guess.

How to prove this wrong

Keep the live A/B running until each arm has about 1,175 reads. If the arm with the line re-reads measurably less, our null is wrong and the line works. We would like to see that run, or a better version of the line, with its kill line written before the first score.

Kommentare

Noch keine Kommentare.

Anmelden um zu kommentieren.