← Blog
✎Post blogowy

Porting a hot loop to Goose, and the benchmark that lied

A cinematic photograph of a single grey goose standing on a dark wet stone dock at dawn beside coiled steel cable, in teal and steel with a warm amber light, the AMARBARO mark and wordmark centred over it.

The Goose nearest-triangle search beats scipy only once the tree is built once and shared, and in the real pipeline the win is a real but small 1.4 s per prop.

Our first benchmark said the Goose port was 4 times faster. Inside the real pipeline, that same version took 6.4 seconds and lost. After the fix it won for real, but small: 1.4 seconds saved on a 30 second job.

After the GPU work, the slowest part of our 3D prop pipeline was on the CPU. A profile of the live ComfyUI server (details in the props article) put 26.9 % of its cycles in scipy's _ckdtree, called from one function in the remesh step: _udf_exact. This article is about porting that function to Goose, and about how our first measurement was wrong.

What the function does

Remeshing requires measuring distance: for thousands of sample points in space, how far is the nearest surface triangle? ComfyUI's _udf_exact builds a cKDTree over triangle centroids, takes the 8 nearest candidates per point, then computes exact point to triangle distances. Because it only looks at 8 candidates by centroid, it is approximate. On AMD it also mixes CPU and GPU work, so the tree queries contend with the samplers.

The real cost is large with big meshes. A tea cup run at 32,000 tokens produced a mesh of 32.6 million faces before remeshing, and the CPU stages scale with it.

What Goose is

Goose is a systems language that avoids dynamic memory allocation by keeping data on structured stacks. It compiles to C. It gives a plain C-ABI shared library that Python can call. (Our default language is Mojo, but Goose is the choice when the target is a CPU kernel behind a C interface. It is CPU only.)

The recipe that works from Python

goose --standalone --header lib.h -o lib.c src.goose
gcc -O3 -march=native -shared -fPIC -o libX.so lib.c -lm -pthread

--standalone produces C with no main, plus goose_init and goose_exit. A function marked export fn is callable from outside. Slices cross the boundary as a struct of a data pointer and a length, writable and zero copy from numpy through a.ctypes.data_as(...). Call goose_init(0, NULL) once before any export, and on the same thread as every later call.

Language gotchas that each cost us a compile:

  • min and max need import std;, sqrt needs import math;, and float3, dot and cross need import vec;.
  • Indexing takes i64, so an i32 index is written a[i as i64].
  • float3 * float3 is component-wise. To scale, use v * float3 { s, s, s }.

Threads: one runtime per thread

A Goose runtime is single-threaded, and Goose's own threads share nothing. Their arguments must be flat, so a numpy buffer cannot be handed to a worker. The pattern that works: copy the .so to a temp file once per worker thread, load it with CDLL, unlink it, and call goose_init on that thread. N copies give N runtimes, and ctypes drops the Python GIL during each call. Sixteen runtimes ran sixteen workers.

The port

The Goose routine organizes triangles into a bounding-box tree so it can search for the exact closest surface point without testing every triangle. It replaces the approximate k=8 centroid search with an exact one. It is wired into ComfyUI by an environment variable, GOOSE_UDF=path/libudf.so.

The benchmark

150,000 triangles, 1 million query points:

Version Time
cKDTree k=8, 16 workers (ComfyUI today) 3.63 s
Goose, 1 thread 10.4 s
Goose v1, 16 runtimes, BVH built per call on each thread 0.93 s
Goose v2, BVH built once into numpy buffers, 16 runtimes query it 0.79 s

v2 was also exact: 0.0 difference against brute force.

Where the benchmark lied

The first version looked four times faster in isolation, but took 6.4 seconds in the real pipeline. The real workflow calls the function six times with small inputs, where tree setup costs outweighed search speed. Each call made all 16 threads rebuild the same tree, and the rebuild dominated the small calls. One big benchmark call hid that fixed per-call cost.

The fix was to split the work. One thread builds the BVH once, into buffers owned by numpy. Then all runtimes only read it. In the pipeline (crate prop, with UDF_TIME=1 receipts) the UDF total went from 2.87 s to 1.48 s, and the whole prompt from 31.18 s to 29.74 s.

That is a 1.4 s win on a job of 30 s. It is real and small. Each prop saves a bit and the exactness is a bonus, but it is not a revolution.

Other CPU ideas we did not keep

  • A HIP port of the BVH any-hit rays used for ambient occlusion: it matched the CPU result and gave no speedup.
  • A pipeline overlap so the GPU samples prop N+1 while the CPU post-processes prop N: sequential 599 s, overlapped 608 s. GPU idle time fell and the wall time did not, because _udf_exact contends with the samplers.
  • OMP_NUM_THREADS=8: 101.8 to 94.5 s in a single run, then drowned by the real cause, the OpenBLAS pool.

Rules we took from this

  • Measure in the real call pattern. A micro-benchmark of one big call hides per-call cost.
  • Split build from query: build once on one thread into host-owned buffers, query read-only on all threads.
  • Check exactness with brute force, not with a second approximation.
  • A port that saves 5 % of a job still needs a cause, a receipt, and a check before it becomes a default.

Komentarze

Brak komentarzy.

Zaloguj się aby skomentować.