← Blog
✎article de blog

Our Notes Do Not Have Enough Gravity

Loose dark rock fragments on a barren surface in black space, one fragment glowing amber, no planet formed, the AMARBARO symbol centered in the frame.

Spreading activation over a note network beat vector search only inside a small, densely linked topical neighborhood: no gain on our vault (23 of 29 with and without), +6.7% held-out link recall on 3,000 Wikipedia articles, +0.7% on 5% of the dump.

The network did nothing for our notes. On the frozen brain-ask eval, 58 queries over the Brain vault, the spreading-activation layer scored 23 of 29 paraphrase hits with the network on and 23 of 29 with it off. It reordered 6 of 58 result lists. It showed 6 notes the control did not show, and none of those 6 was a target.

It did win once. On 3,000 Wikipedia articles around one topic, the uncapped network recovered 6.7 percent more held-out links than vector search alone. On 5 percent of the whole dump the same method gained 0.7 percent. The claim is narrow: spreading activation helped only inside a small, dense topical neighborhood, and neither more links nor a bigger graph made it work.

What was built

tools/brainnet.mojo turns the vault into a network. Each note is a neuron whose data is the mean of its chunk vectors. Synapses come from three places: the wikilinks written by hand (3,166), edges grown by the build, where an under-linked note is wired to its most similar notes in its category (5,825), and Hebbian weights from notes that one past brain-ask run returned together (131, with 706 single co-retrievals pruned). The vault had 4,183 neurons and 9,095 synapses. Most synapses were grown by the build, not written.

At query time brainvec's hits are the stimulus. The network runs three leaky steps with degree normalization and k-winners-take-all (cap 32), and prints five distinct notes. It stays behind an opt-in flag, BRAIN_ASK_NET=1, and it is off by default. Build time is 0.95 s at 175 MB peak (commit 36b455e).

The vault: a miss

The eval has 29 verbatim and 29 paraphrase queries. Every arm scored 29 of 29 on verbatim. The paraphrase arms:

arm paraphrase strict MRR distinct notes shown
brain-ask today 23/29 0.662 4.62
network off (same 20 candidates) 23/29 0.664 5.0
network on, BETA 0.5 23/29 0.664 5.0
network on, no cap 23/29 0.664 5.0
hub network (every note wired to its category hub) 23/29 0.664 5.0

The receipts are newera/.work/brainnet/{A,B,C,D,E}.txt. The no-cap arm moved the target rank in 0 of 58 queries. The kill line was set before the run: beat 23 of 29 or stop. No variant beat it, so we stopped. BETA was frozen at 0.5 and was never tuned on the eval.

One retraction. The first commit message (36b455e) said the network recalled no notes beyond the vectors. That was a counting bug: the eval cuts hits at .md#, so the (net marker we were counting never appeared. Commit 1010a41 corrected it to the numbers above. The picture of the vault as a network, newera/.work/brainnet/brain-net.png, shows the layout and is not evidence.

Why did it miss? The six missed targets are verdicts.md files, and those carry few wikilinks. Spreading reaches link neighbors, so a verdict gives it little to reach from. That is an inference. We did not check it.

Wikipedia, small: a win, with a setting in the way

The second test used 3,000 articles crawled breadth-first from "Neuron", inside the first 3 GB of the enwiki dump. The extractor logged 82,777 visible internal links for the seed 7 split and 82,641 for seed 11 (newera/.work/wikinet/extract.log, extract11.log). We held out 10 percent and measured recall@10 of the held-out targets.

split vectors only network, cap 32 (default) network, no cap
seed 7 (where the cap change was chosen) 763 / 9,139 742 806 (+5.6%)
seed 11 (fresh, rule fixed before the run) 815 / 9,275 816 870 (+6.7%)

Seed 7 is where we found the cap problem, so its +5.6 percent is the number we trust least. Seed 11 is the one that counts: 870 held-out links against 815 from vectors alone. The capped network returned 816, which is no gain at all. The cap of 32 stays the default because it is what the vault eval measured (commit d2015f8). On the vault it is harmless. On this subset it stops activation from reaching the second hop, and the gain needs that second hop. We have not found one value that works on both.

Wikipedia, at scale: no gain

Then we ran the same test on 5 percent of the dump: 176,955 articles, 5.8 million visible links, 646,000 held out, with 2,029 sources sampled evenly. The leads were embedded on the GPU at 1,044 rows per second, and the GPU vectors matched the CPU vectors at cosine 1.0.

arm recall@10
vectors only 294 / 9,457
network, cap 32 289
network, no cap 296 (+0.7%)

The uncapped result is 0.7 percent above vectors, and the capped one is below them. Neither is a gain we would defend (commit 71d667b). Across the whole graph, three steps of spreading dilute into too many neighbors, and the 6.7 percent from the small subset does not carry over.

Gravity is a core, not a count

k-core measures how tightly a graph holds together. A node has core number k if it sits in a subgraph where every node keeps at least k links inside it. The 3,000-article subset reaches core 33. The vault reaches core 6, and that 6 is shaped by the growth rule, which puts a floor of core 3 under every note. Above that floor the vault has little structure. It is a cloud.

Spread activation only where the links already form a core.

A planet holds its moons because of a dense center, and a diffuse cloud of mass holds nothing. The 3,000-article subset is a core, the vault is a cloud, and the whole dump is a large cloud with a few dense patches. The hub test fits this. With BRAINNET_HUBS=1, every note is wired to its category's whiteboard, index or README. That adds structure to the vault, and the eval stayed at 23 of 29. Our reading, unproven, is that links which follow the topic matter and links added for their own sake do not.

What counts:

  • A held-out link recall measured against vectors on the same candidates.
  • A rule fixed before the run, on a split nobody tuned on.
  • A control arm with the network off, on the same 20 candidates.

What does not count:

  • A split where the setting was chosen.
  • A gain inside the noise, reported as a gain.
  • A picture of the network.

What it cost

Extraction of the 3,000-article subset took 167 seconds and 19 GB of RAM, because the whole byte prefix of the dump was held in memory. The Mojo dump parser, tools/wikinet.mojo, streamed the 5.5 GB slice in 34.8 seconds at 13.8 GB peak. Predict takes 5 seconds. The cost that matters more is the direction one: two attempts to make the network help the vault (lifting the cap, wiring in hubs) moved no hit.

The reach of the result is small. One vault, 29 paraphrase queries, one topic seed, two splits on the small subset, one sample of 2,029 sources on the large slice. With 29 queries one hit is 3.4 points, so the vault numbers cannot separate arms that differ by a hit. The 0.7 percent is inside the noise, so we call it no gain and not a loss. A different method is not ruled out. The next one we would try is personalized PageRank with restart, or a per-query subgraph, rather than more synapses.

How to prove this wrong

Build a per-query subgraph, or personalized PageRank with restart, over the 5 percent slice and run brainnet predict at 2,029 sources. If recall@10 beats the 294 of 9,457 vector baseline by more than the noise, the claim that only a small dense neighborhood works is wrong. A second route: rerun the vault eval with the no-cap network on a bigger query set than 29. If it shows a gain a hit-sized difference could not explain, the vault is not a cloud. Send us the numbers.

Provenance

Repo newera, commits 36b455e (network and vault eval), d2015f8 (Wikipedia predict), 1010a41 (vault correction), 71d667b (5 percent slice, k-core, hubs). Run on one box with an RX 7900 XTX for the GPU embedding, 2026-10-21; the README section "brainnet" carries the same figures.

Commentaires

Pas encore de commentaires.

Se connecter pour commenter.