← Blog
✎blogbericht

What the Network Fixed

A cinematic photograph of an old wooden library card catalogue with a few drawers open and glowing faintly, in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

Spreading activation found 13 percent more hidden Wikipedia links than plain search, but the accuracy gain in brain-ask came from the reranker, not the network.

Spreading a question along links found 13 percent more of the hidden Wikipedia links than plain search did. That is the network's best result. Some of the other wins in this article belong to something else, and we say which, because a network is a nice picture and the honest question is what it bought us.

Win 1: the spread beats plain search on dense graphs

The cleanest test asks: hide 10 percent of the links in a graph, then check how many hidden links appear in the top 10 of a search. This tests whether links help the search find relevant notes that pure text matching missed.

On Wikipedia, where an article has about 66 links, personalized PageRank wins every time. The "vectors" column is plain vector search; "PageRank" is the spread from article 3:

graph vectors PageRank change
5% of Wikipedia (177k articles) 294 360 +22.4%
a 3,000-article subset (seed 11) 815 907 +11.3%
20% of Wikipedia (1.18M articles, 8,426 hidden links) 332 375 +13.0%

The 20% run had a kill line of 336 set in advance: PageRank had to beat it or we would drop it. It scored 375.

The older, simpler spread (activation capped to the 32 strongest neurons) did nothing here: 326 against 332 for vectors on the 20% graph. The uncapped walk with restart was the fix. Math paper results saw a similar 10 to 13 percent lift over pure vector search.

One caveat: the gain did not grow with the graph. A 10 percent graph rebuilt with one link per article pair gave +9.0 percent. We expected it to shrink with size, and the 20 percent run says it does not. Why it stays flat is open.

Win 2: answers get more varied

For brain-ask on our own vault, we compared four arms on 59 scored questions (a frozen test of paraphrased and held-out queries):

arm strict hits of 59
plain vectors 41
network only 41
plain plus reranker 44
network plus reranker (today's default) 44

The +3 is the cross-encoder, not the network. We say so plainly in our own notes: "do not credit PageRank for it". What the network adds is variety. An answer now shows 5.0 distinct notes, against 4.66 or 3.93 without it.

The right note also sits about as high as before. On the two question sets, the mean reciprocal rank (a score for how high the right note sits) is 0.767 and 0.534, against 0.766 and 0.529. Small, real, and not an accuracy win.

Win 3: hub pages stopped flooding paper answers

When you ask for answers from the PaperBrain vault, a few hub pages link to almost everything. One commentary page links to most papers. Activation poured through it and it filled the tail of the answer list.

We built a 60-question paper test, 30 for picking settings and 30 for scoring once. Strict accuracy was already at ceiling (29 of 30), so we counted a different thing: hub pages in the top 5 across the scored 30. Default settings put 22 hub slots there.

Dividing a hub's signal by the square root of its link count cut that to 9, with strict accuracy and ranking quality unchanged. All 9 remaining were the same one commentary page. Paper and combined modes now use this damping by default. Brain mode is untouched.

Our vault has only about two links per note, so many related notes are not joined. We tried a "level 3" idea: PageRank proposes its 30 best candidates for a note, then a local 9-billion-parameter model reads the note, with its links removed, and reorders them.

We hid 10 percent of the real links and checked how many came back in the top 10, on two splits:

split vectors PageRank model reorders PageRank's 30
7 96 of 325 124 134
11 87 of 312 107 139

Pooled over 392 cases, the model added 42 hits over PageRank, with a 95 percent interval of 20 to 70. The spread between splits is large (+8 percent against +30 percent), so treat the average with care. It became a tool, brain-link-propose: it prints up to 10 numbered link proposals for a note, and a human accepts some with --accept 1,3. It runs in 0.67 s a note.

One honest detail. Adding 957 model-suggested links changed only 1 of 88 search results, with no accuracy change. The links help tidy our notes, but they do not change search. The links are real, but they do not move search answers. We use the tool for vault hygiene, not for retrieval.

Win 5: questions got fast

The model-load fix found by timing a single query was almost silly. Loading a 90 MB model file took 135 ms because of a slow read call. Reading the file with its size known took 25 ms. A query dropped from 0.93 s to 0.63 s, and with a second fix (one batched text lookup instead of 20 full scans) to about 0.55 s. Result lists were identical before and after, so nothing was traded for speed.

A trap from the same day: comparing two runs on the live vault is noise, because each search is logged and boosts often-returned notes. We compare through the frozen test.

Win 6: skills find their neighbours

643 skill files are neurons too, counted on 2026-11-14. When an agent loads one, a hook spreads from it and suggests three related skills. Loading memory-optimization really suggested the Rust, C++ and ROCm optimization skills. How often the suggestion is right, we have not yet measured. That is listed as open.

Scorecard

win earned by
held-out links on dense graphs the network
accuracy +3 of 59 the cross-encoder
answer variety the network
fewer hub pages hub damping
proposed links the network plus a local model
speed file-read and lookup fixes

Next: where it fails.

Reacties

Nog geen reacties.

Inloggen om te reageren.