← Blog
✎publicación de blog

Rank Was Half the Answer Until One Lookup Fix

A cinematic photograph of a steel card-catalogue cabinet with one drawer pulled half open and a single brass index tab, in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

On 2026-11-13 the rank stage took 0.82 of a 1.56 second brain-ask answer, and a lookup fix plus a resident server cut it to 0.093 seconds with no change in the top five lists. On 2026-11-19 the served rank stage measured 0.100 seconds at the median, under the 0.15 second line.

Part 8 of 12 in BrainNet. Previous: A Memory Line Agents Did Not Follow · Next: The Reranker Won Three Questions and Lost None

On 2026-11-13, a brain-ask answer took 1.56 seconds at the median over our frozen 88-question eval. The rank stage, which spreads the vector hits along the synapses, took 0.82 of those seconds. That was half the answer. With the servers on, the same answer took 0.574 seconds that day, and rank took 0.093 of it when its server was warm. This part shows what each stage cost and what each change did to the output.

The stages, as we measured them

stage 2026-10-21 trace (ms) 2026-11-13 trace (ms)
rerank (MiniLM) 538 320
embed (bge) 257 260
vector search (brainvec) 38 80
rank (network) 12 820

The October trace came from one query in brain mode, when our notes store held 4,183 neurons. The November trace came from bash timestamps, and it sums to 1.48 seconds against a 1.56 median. Rank grew 68 times while the store grew 392 times, from 4,183 to 1,640,443 neurons. We read that as the store size at work. On 2026-11-19 we timed rank on one question, cold, against a vault-only store and the merged store: 20.0 ms against 271.0 ms. That supports the store-size reading, though it is one question.

Four fixes, each checked against the same answers

The model file read. The embedding model was loaded with a read call that took 135 ms for 90 MB. Reading with the known file size took 25 ms. One query fell from 0.93 to 0.63 seconds. The paper eval lists stayed identical on 60 of 60 queries.

The rerank text fetch. The rerank step looked up each hit's text with a separate full scan of the text table. One indexed query for all 20 hits cut the benchmark from 414 to 296 ms. Eval lists differed on 0 of 88 queries.

The rank lookup. Rank loaded 1.64 million path strings and a dictionary before it looked up its 20 hits. It now reads the hits first and scans the path file once. Cold rank went from 0.82 to 0.24 seconds, and the output was byte-identical on 87 of 87 captured calls.

The resident servers. The embed, rerank and rank models stay loaded behind unix sockets. If no server answers, the client does the work itself, so a dead server only costs time. The median answer went from 934 ms with servers off to 574 ms with them on. Hit lists differed on 0 of 88 queries.

Where the copy was the cost

At the 20 percent Wikipedia slice (1,181,724 articles), each question copied a 540 MB link file and a 1.8 GB vector file into memory. Firing the network took 0.66 seconds of a 5.3 second question. The other 3.9 seconds were copying. Memory-mapping the files cut the question to 1,076 ms and peak memory from 6.9 GB to 1.9 GB. All 32 outputs stayed identical.

At 40 percent of the dump (2.85 million articles), the question spent its time elsewhere. Over three questions, the stages were: embed 170 to 220 ms, network 600 to 640 ms, text fetch 1,400 to 1,820 ms, rerank 280 to 350 ms, print 1,250 to 1,740 ms. Fetch and print were two awk scans of a 1.4 GB file, for 20 rows. One index with binary search brought fetch and print to 31 to 47 ms. The question went from 3.6 seconds to 0.86 seconds, with identical output.

What we kept on purpose

Two speed ideas did not ship. The rank walk is still single-threaded, because a parallel scatter changes the order of float additions, and the outputs would stop being byte-identical. The walk is about half of a served rank, so the ceiling on that change is small. A GPU does not help the query path either: the time goes to loading files, and the walk itself takes milliseconds.

The reranker stayed, and its cost is the 0.2 seconds it adds to each answer. Our speed note says it carries the paraphrase and held-out quality. Part 9 measures how much.

Retraction and open items

The 2026-11-13 note said the embed model is the largest warm stage. On 2026-11-19 that was wrong. On the served path, rerank was the largest stage at 209 ms median, and embed took 34 ms. We withdraw the claim. The 0.093 second rank figure in the 2026-11-13 receipts was timed on the rank server's own clock. The 2026-11-19 served rank stage, 100 ms, was timed from the BRAIN_ASK_TIME lines.

Measured again on 2026-11-19

Source: .work/amarbaro/bench/RESULTS.md, 88 frozen questions, 5 reps each, on a snapshot copy of the store.

Served path, servers on. Median per stage, pooled over n=440 runs, range in brackets:

stage median ms range ms
cois 109.5 34.0 to 990.0
embed 34.0 29.0 to 50.0
brainvec 55.0 47.0 to 70.0
rank 100.0 82.0 to 117.0
dedupe 57.0 54.0 to 62.0
rerank 209.0 125.0 to 280.0
total 564.0 422.0 to 1480.0

Wall time for brain-ask, served: median 658.5 ms (range 524.0 to 2747.0). The 2026-11-13 figure of 574 ms was not reproduced.

Cold path, servers off, n=440: rank 283.0 ms (241.0 to 380.0), embed 179.0 ms (166.0 to 206.0), rerank 311.0 ms (225.0 to 425.0), total 988.5 ms (832.0 to 1407.0). Cold rank is a little above the 0.24 seconds in the table above, on the larger store.

Rank at one fixed question, cold, n=5: vault-only store 20.0 ms (19.0 to 23.0), merged store 271.0 ms (264.0 to 273.0).

Rank is now about 18 percent of the served total (100 of 564 ms). On the served path it is no longer half the answer.

Falsifier: the served rank median is 0.100 seconds, under 0.15 seconds. It did not fire.

How to prove this wrong

Run BRAIN_ASK_TIME=1 on the 88 frozen questions with the servers on, and take the median of the rank stage. If that median is above 0.15 seconds, the speed claim is wrong. The 2026-11-19 run gave 0.100 seconds, so it did not fire. The embed-versus-rank ranking is now measured (embed 34 ms, rank 100 ms).

Provenance

  • .work/amarbaro/bench/RESULTS.md items 1, 2 and 13 (2026-11-19), made by .work/amarbaro/bench/01-02-stages.sh and .work/amarbaro/bench/13-time.sh

  • ~/Brain/iTools/2026-11-13-brain-ask-speed.md: receipts table (1,562, 574, 934 and 352 ms), 0 of 88 and 87 of 87 identity checks

  • ~/Brain/newera/2026-10-21-brain-ask-latency.md: bash -x trace (538, 257, 38, 12, 9 ms), model read 135 to 25 ms, text fetch 414 to 296 ms, paper eval 900 to 684 ms
  • ~/Brain/newera/2026-10-21-brainnet-eps.md: wiki copy fix 5005 to 1076 ms, 6.9 to 1.9 GB, 32 of 32 identical
  • ~/Brain/newera/2026-10-21-brainnet.md: 4,183 neurons on 2026-10-21
  • ~/Brain/newera/2026-10-22-wiki-40.md: 40 percent stage timings and the row index (BRAIN_ASK_TIME=1, BRAINNET_TIME=1)
  • ~/iTools/brain/brain-ask/PLAN-speed-ax.md: 2026-11-13 stage trace and status
  • ~/Brain/Skills/own/brainnet/SKILL.md: rank 0.82 to 0.24 s, 0.093 s served, 1,640,443 neurons
  • ~/iTools/brain/brain-ask/eval/runs/now-2026-11-13.jsonl, final-serve.jsonl, final-noserve.jsonl, final-norerank.jsonl: per-query ms and hits; medians recomputed as the mean of the two middle values
  • ~/.claude/INDEX.md: 1,855,596 neurons on 2026-11-18

Commands: ~/iTools/brain/brain-ask/eval/.bin/eval OUT.jsonl with BRAIN_ASK_RERANK=0 (no rerank) or BRAIN_ASK_SERVE=0 (no servers), as named in the speed note. The eval usage line is in eval/eval.mojo.

Comentarios

Aún no hay comentarios.

Iniciar sesión para comentar.