The slow part was a memory layout, not a kernel
A strided tensor layout made AOTriton attention run 6 times slower than on the same data made contiguous. One small custom node that makes the data contiguous fixed it, and two hand-built kernel rewrites bought nothing.
Our ComfyUI Work · part 3