← Blog
✎post de blog

A Z-Image picture in 1.7 seconds, and why no kernel could have got us there

A cinematic photograph of a vintage brass stopwatch resting on a dark steel workbench beside a graphics card, in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

Most of the drop from 7.3 to 1.7 seconds per Z-Image picture on a 7900 XTX came from memory budget flags, int8 weights and fewer steps, not from faster kernels.

A picture used to take 7.3 seconds on our graphics card. It now takes 1.7, and almost none of that came from faster math. It came from memory settings, smaller weights, and a hidden 2 second wait inside our own timer.

We run ComfyUI on one AMD Radeon RX 7900 XTX (24 GB, architecture name gfx1100, ROCm 7.2, torch 2.13). ComfyUI is a node graph for image models: you wire a loader, a sampler and a decoder together, and a small server runs the graph. Our picture model is Z-Image Turbo, a text to image model that needs few steps.

This article is about one number: seconds per picture. It started at 7.3 and ended at 1.7. Most of the road was not about speed of math at all.

The setup

We tested Z-Image Turbo at 1024x1024 resolution and 8 steps. It used a Qwen3-4B text encoder and the Flux VAE to turn generated latents into pixels. Four generations per arm. Run 0 includes loading the model from disk, so we report the steady number from runs 1 to 3.

One rule shaped everything: each arm changes one thing on top of the current winner. Our first attempt stacked flags in a ladder, and it hid that one of them was a loss. That mistake is why the table below looks the way it does.

Flags: what helped and what hurt

The starting launch used --reserve-vram 2, which tells ComfyUI to keep 2 GB of the card free.

Arm Steady s/image Verdict
--reserve-vram 2 (start) 7.3 old
--reserve-vram 0.5 6.8 keep
plus COMFYUI_AMD_VAE_PYTORCH_ATTENTION=1 6.8 cold start only, later removed
plus TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 6.8 rejected: server abort on run 2
TorchCompileModel (inductor), bf16 13.2 rejected: twice as slow
int8 tensorwise weights 6.5 keep
int8 plus TorchCompileModel 6.5 to 6.8 after 2 warmups rejected: no gain
--fast 1.25 it/s, 8 % slower rejected
COMFYUI_ENABLE_MIOPEN=1 MIOPEN_FIND_MODE=FAST 1.19 it/s, plus a 200 second tune rejected

The reserve flag is the interesting one. The text encoder (4.8 GB), the diffusion transformer (11.3 GB) and the VAE decode did not fit inside the budget, so every prompt partly unloaded and reloaded weights. The win was not faster compute. It was less shuffling. A lower reserve and smaller weights removed the shuffle.

--fast hurts on AMD because its autotune does nothing while ComfyUI keeps cudnn and MIOpen off, so only the fp16 accumulation cost remains.

int8 weights

We wrote a small script, quantize_int8.py, that turns the 12.3 GB bf16 transformer into a 6.3 GB int8 file in 19 seconds on the CPU. Only the big 2D attention and feed-forward weights of the repeated blocks go to int8, with one scale per matrix. Norms, embedders and the final layer stay bf16. The int8 matrix multiply runs through the comfy_kitchen HIP kernel gemm_wmma_kernel.

Side-by-side inspection confirmed the composition stayed identical, with only tiny pixel shifts. Both images looked equally sharp.

A profile of one transformer step explained where the gain lives. A step took 738 ms in bf16 and 675 ms in int8. The whole gain is one layer shape, the feed-forward w2 (10240 to 3840): 160 ms in bf16 against 114 ms in int8. In bf16 that layer ran 1.6 times slower than its twin w1 at the same arithmetic, because the library picked a poorer kernel for it. int8 simply runs it at the normal rate. The text encoder in int8 added a little more: 6.52 to 6.02 s.

Keeping models resident saved a second

Moving from ComfyUI v0.34 to v0.43 changed little on its own: 7.0 to 6.8 s steady. The same seed gave a PSNR of 29.4 dB between versions, same composition.

The useful find was --highvram. Each VAE decode was evicting about 2.5 GB of the transformer to make room for a 160 MB VAE, then reloading it. Keeping models resident took Z-Image from 6.8 to 5.8 s. We keep it for pictures only, because it can run Wan video out of memory.

Updating itself was cheap once we copied the Python environment with cp -a --reflink=always (4 seconds on btrfs) and only installed the difference. A fresh ROCm torch download had passed 560 seconds. Before an update we also set a git tag as a rollback point.

The roofline

At this point we asked whether a better kernel could reach 1 to 2 seconds. Arithmetic first. A roofline is that arithmetic: the fastest a job can possibly run, given how much data it must move and how much math it must do on this card. If the roofline says no, no kernel can say yes.

Z-Image Turbo has 30 layers, a width of 3840 and 5.3 billion int8 linear parameters. At 1024x1024, one step requires 55 trillion floating-point calculations. We measured 0.65 seconds per step, reaching 69% of the graphics card's theoretical peak speed. Even at the int8 peak, 9 steps would take 2.0 s before any overhead. So a custom kernel could not give us a one or two second picture. The levers left were steps, token count and client overhead.

A note on our own history: an earlier Wan note of ours assumed a peak of 61 TFLOPS, and a later measurement retracted it to about 123. Check the roof before you trust it.

Steps and size

We swept warm seconds per picture, client side:

Steps 1024 by 1024 768 by 768
9 7.1 3.5
6 5.5 2.4
4 4.4 1.7

Server speed was 1.54 it/s at 1024 and 2.83 it/s at 768. We judged the look on a sheet of three prompts (a shop, a park, a face). Four steps at 768 kept structure, composition and light. It lost fine texture such as leaves, clutter and skin pores.

Getting the texture back for free

A schedule sweep at 4 steps and 768 pixels used the ModelSamplingAuraFlow shift node, with shift 3 as the base. Every arm ran in 1.7 s (5 steps: 2.1 s).

  • Shift 1.5 brought back skin wrinkles, stubble, jacket grain, plaster cracks and produce density.
  • Shift 6 flattened the picture.
  • The beta scheduler recomposed the scene and washed the sky.
  • Sampler euler matched the base, and 5 steps added nothing.

Default now: 4 steps, 768 pixels, shift 1.5. Three default jobs gave 8.1 s cold, then 1.7 s and 1.7 s warm. A quality arm (9 steps, 1024) stays one field away. We did not train a 4-step distillation LoRA for Z-Image Turbo. None exists that we could use, and training one is about a day of work that shift 1.5 already made unnecessary.

The timer was lying by up to 2 seconds

When we compared our client time with the server's own "Prompt executed" line, they disagreed. Our client script checked for completed jobs only once every 2 seconds. Every picture recorded since September carried an accidental 0 to 2 second delay. We changed the sleep to 50 ms. The 9 step, 1024 job went from 8.0 to 7.1 s and the two clocks now agree within 0.1 s. A client that times a server must poll at a small fraction of what it measures.

What we learned

  • Look for memory shuffling before you look for slow math.
  • Do the roofline before writing a kernel.
  • A knob that does not change quality can still cost you; test it alone.
  • Check your timer against the server's own.

Comentarii

Niciun comentariu încă.

Conectare pentru a comenta.