Filtering redundant hook context before it reaches the model removed a 964-byte block that one real session received 50 times.
Update (2026-10-19). The filter described here was retired on 2026-10-06; the emitters it filtered were cut at the source instead, which is the rule this post ends on. An earlier version opened with a single payload falling from 1790 to 826 bytes; we have no receipt for those two numbers and removed them. The 964-byte block and its 50 occurrences are recorded.
One path-hint block was 964 bytes, and one real session received it 50 times, about 48 KB, before the filter stripped it ahead of the model. Prefix caching still matters, but it cannot cache away a block that should never have been emitted.
Context that never arrives is the cheapest context
The filter removed repeated path, evidence, and scout sections from a hook payload. It preserved the task contract and the active skill, which are the pieces that change what the agent is allowed to do. The output stayed actionable while losing the ceremony around it.
This is a different lever from compression. Compression makes an existing payload cheaper to transport or tokenize. The filter changes the source so the model receives fewer tokens in the first place. That distinction keeps the measurement honest: compare the bytes before and after the exact hook boundary.
The sample exposed a trap
The capability surface gate saved 525 bytes over a 40-prompt sample and emitted zero bytes on repeats. A different 40-prompt window reported no saving because it contained no path block. Both statements were true. The sample window is part of the claim.
The same session showed that tool results and tool commands were the largest context classes. That points to batching and shorter command output as the next levers. It does not justify claiming that every session saves the same amount.
Ownership beats clever dedupe
Remove repetition where the repetition is created, and preserve bytes everywhere else.
What counts:
a before and after receipt from the same payload;
a list of removed fields;
an assertion that the task contract remains;
a repeat test against the component that emits the hint.
What does not count:
a smaller prompt produced by changing the task;
a cache hit reported without the input bytes;
dedupe hidden in a transport wrapper;
a saving measured on a window that did not contain the target block.
The cost was one reverted optimization
The wrapper-side dedupe looked efficient but violated byte-for-byte passthrough on repeated payloads. It was removed in fc7c893. The capability hook kept its own session dedupe because it owns that emission. The correction made the system slightly less clever and materially more trustworthy.
How to prove this wrong
Replay the recorded payload with the filter at the cited commits. Count bytes before and after. Check that the task contract and active skill remain, and that the removed path, evidence, and scout lines are exactly the expected ones. Then send the same payload twice through the wrapper and assert identical output bytes. If the 964-byte reduction does not reproduce, correct the post.
Provenance
Receipt: our token-reduction index records 964 bytes per occurrence and 50 occurrences in one session for the path block, and 525 bytes over 40 prompts for the capability hook. Implementation: hooks/caveman_filter.sh and hooks/caveman_filter_test.py (retired 2026-10-06). Related capability hook: hooks/capability_surface.py. Commits 97f3a63, 7e4aef0, and fc7c893.
Sending a text screenshot as a picture cost about 13 times the tokens of its recognised text. A hook now denies that pixel read, and the oversized case shows 2,280 billed tokens that bought no pixels the encoder used.
A duplicated instruction file cost room in every request, not much money, because cached reads are cheap. The 7k figure is an estimate, and the measurement that would confirm it has not been run.
The panel said enforced for six days while nothing enforced it. The fix was a check that names unwired hooks, and the only cut with a measurement is also the only one a hook now denies.
Komentarze
Brak komentarzy.
Zaloguj się aby skomentować.