Where BrainNet Still Fails
On our own vault, every network idea tied plain search at 23 of 29 and 17 of 30. Bridges, usage synapses, learned weights, Hebbian plasticity and a graph inside the model were all killed against a kill line set in advance.
Spreading activation found 13 percent more hidden Wikipedia links than plain search, but the accuracy gain in brain-ask came from the reranker, not the network.
Spreading a question along links found 13 percent more of the hidden Wikipedia links than plain search did. That is the network's best result. Some of the other wins in this article belong to something else, and we say which, because a network is a nice picture and the honest question is what it bought us.
The cleanest test asks: hide 10 percent of the links in a graph, then check how many hidden links appear in the top 10 of a search. This tests whether links help the search find relevant notes that pure text matching missed.
On Wikipedia, where an article has about 66 links, personalized PageRank wins every time. The "vectors" column is plain vector search; "PageRank" is the spread from article 3:
| graph | vectors | PageRank | change |
|---|---|---|---|
| 5% of Wikipedia (177k articles) | 294 | 360 | +22.4% |
| a 3,000-article subset (seed 11) | 815 | 907 | +11.3% |
| 20% of Wikipedia (1.18M articles, 8,426 hidden links) | 332 | 375 | +13.0% |
The 20% run had a kill line of 336 set in advance: PageRank had to beat it or we would drop it. It scored 375.
The older, simpler spread (activation capped to the 32 strongest neurons) did nothing here: 326 against 332 for vectors on the 20% graph. The uncapped walk with restart was the fix. Math paper results saw a similar 10 to 13 percent lift over pure vector search.
One caveat: the gain did not grow with the graph. A 10 percent graph rebuilt with one link per article pair gave +9.0 percent. We expected it to shrink with size, and the 20 percent run says it does not. Why it stays flat is open.
For brain-ask on our own vault, we compared four arms on 59 scored questions (a frozen test of paraphrased and held-out queries):
| arm | strict hits of 59 |
|---|---|
| plain vectors | 41 |
| network only | 41 |
| plain plus reranker | 44 |
| network plus reranker (today's default) | 44 |
The +3 is the cross-encoder, not the network. We say so plainly in our own notes: "do not credit PageRank for it". What the network adds is variety. An answer now shows 5.0 distinct notes, against 4.66 or 3.93 without it.
The right note also sits about as high as before. On the two question sets, the mean reciprocal rank (a score for how high the right note sits) is 0.767 and 0.534, against 0.766 and 0.529. Small, real, and not an accuracy win.
When you ask for answers from the PaperBrain vault, a few hub pages link to almost everything. One commentary page links to most papers. Activation poured through it and it filled the tail of the answer list.
We built a 60-question paper test, 30 for picking settings and 30 for scoring once. Strict accuracy was already at ceiling (29 of 30), so we counted a different thing: hub pages in the top 5 across the scored 30. Default settings put 22 hub slots there.
Dividing a hub's signal by the square root of its link count cut that to 9, with strict accuracy and ranking quality unchanged. All 9 remaining were the same one commentary page. Paper and combined modes now use this damping by default. Brain mode is untouched.
Our vault has only about two links per note, so many related notes are not joined. We tried a "level 3" idea: PageRank proposes its 30 best candidates for a note, then a local 9-billion-parameter model reads the note, with its links removed, and reorders them.
We hid 10 percent of the real links and checked how many came back in the top 10, on two splits:
| split | vectors | PageRank | model reorders PageRank's 30 |
|---|---|---|---|
| 7 | 96 of 325 | 124 | 134 |
| 11 | 87 of 312 | 107 | 139 |
Pooled over 392 cases, the model added 42 hits over PageRank, with a 95 percent interval of 20 to 70. The spread between splits is large (+8 percent against +30 percent), so treat the average with care. It became a tool, brain-link-propose: it prints up to 10 numbered link proposals for a note, and a human accepts some with --accept 1,3. It runs in 0.67 s a note.
One honest detail. Adding 957 model-suggested links changed only 1 of 88 search results, with no accuracy change. The links help tidy our notes, but they do not change search. The links are real, but they do not move search answers. We use the tool for vault hygiene, not for retrieval.
The model-load fix found by timing a single query was almost silly. Loading a 90 MB model file took 135 ms because of a slow read call. Reading the file with its size known took 25 ms. A query dropped from 0.93 s to 0.63 s, and with a second fix (one batched text lookup instead of 20 full scans) to about 0.55 s. Result lists were identical before and after, so nothing was traded for speed.
A trap from the same day: comparing two runs on the live vault is noise, because each search is logged and boosts often-returned notes. We compare through the frozen test.
643 skill files are neurons too, counted on 2026-11-14. When an agent loads one, a hook spreads from it and suggests three related skills. Loading memory-optimization really suggested the Rust, C++ and ROCm optimization skills. How often the suggestion is right, we have not yet measured. That is listed as open.
| win | earned by |
|---|---|
| held-out links on dense graphs | the network |
| accuracy +3 of 59 | the cross-encoder |
| answer variety | the network |
| fewer hub pages | hub damping |
| proposed links | the network plus a local model |
| speed | file-read and lookup fixes |
Next: where it fails.
Commentaires
Pas encore de commentaires.
Se connecter pour commenter.