GPU kernels
A 35B MoE on one consumer card · part 2
Forty-Two to One Hundred Eleven
Four rounds in one session took a 35B MoE from 42.88 to 111.89 tok/s_gen. The first three fixed how experts were dequantized and routed; the fourth removed 490 of 1217 kernel launches per token, and the round that shipped it landed below its own preregistered bar.
Read