← Blog
✎Blogbeitrag

Raw hidden handoff still sits at the receiver floor

A monolithic dark slab resting above a dense sea of cloud, under a heavy sky broken by a single bright band at the horizon.

Eight raw hidden vectors did not improve accuracy in a same-model 120-task rerun.

On 30 September we repeated a same-model hidden-state handoff experiment. Both producer and receiver used the same Qwythos 9B Q4_0 model and matching tokenizer. The question was whether eight raw hidden vectors could help the receiver solve tasks without a text explanation.

Handoff Exact correct Accuracy
Receiver only 27/120 22.5%
Text explanation 84/120 70.0%
Eight raw hidden vectors 26/120 21.7%

All 120 tasks completed in all three arms without execution errors. The set comprised 20 JSON tasks and 100 math tasks. The receiver had a 32-token answer cap; the text producer could emit up to 300 tokens. The raw arm captured eight post-final-norm states, fed each state back through the producer, and inserted those vectors as the receiver prefix.

Faster production did not mean useful transfer

Median source work was 0.263 seconds for raw vectors and 3.319 seconds for text. Those durations buy different outputs: eight state steps versus a text explanation. They are not an equal-information speed comparison.

The paired bootstrap 95% interval for raw transfer minus receiver-only was [-2.5, 0] percentage points. Raw transfer minus text was [-58.3, -38.3] points. The tested raw handoff did not improve task accuracy.

What this result covers

This repeated the same 120 task IDs used in an earlier run. It is a reproducibility check, not an independent generalization result. It tests an eight-vector continuation handoff, not a single-vector replay at a prompt boundary, a trained adapter, or all possible latent protocols.

The earlier run scored 27/120 for receiver-only, 79/120 for text, and 27/120 for raw vectors. The rerun preserves the main finding: text helps substantially; this raw protocol stays near the receiver floor.

Our LLM work continues to separate transport, state consumption, and task success. Moving a tensor successfully is one gate. Showing that it helps a receiver is another. A future positive result needs a disjoint held-out set and receiver-only, zero, and shuffled controls before it can support a semantic-transfer claim.

Discuss the experiment on gllm.forum.

Kommentare

Noch keine Kommentare.

Anmelden um zu kommentieren.