← Blog
✎blog post

A ComfyUI cache held 14.74 GB of text encoder between prompts

A steel cold-storage canister with frost on its shoulders on a dark slate bench, a single amber indicator light at its side, in teal and steel.

On a resident ComfyUI video server, a conditioning-cache miss kept the 14.74 GB Gemma encoder in the CLIPLoader cache entry between prompts, and a cache hit never loaded it: RAM after a shot fell from 31.7 GB to 3.1 GB and the warm shot from 85 s to 49 s.

On our resident ComfyUI video server, one cache entry kept a 14.74 GB text encoder in RAM between prompts. When the cache hit, the encoder was not loaded again. RAM after a shot fell from 31.7 GB to 3.1 GB, and the warm shot dropped from 85 seconds to 49. The holder was something we could name. The change was in the cache, not in the render: the output was identical on both paths.

What the cache is

ComfyUI keeps the output of its text encoder for prompts it has already seen. That output is the conditioning, the numbers the video model reads in place of the prompt text. The encoder itself is a large model, here Gemma at 14.74 GB of tensors. A cache entry for a loader node can hold a reference to the encoder. That reference is what keeps the encoder in RAM.

What we measured

We compared three arms on the same server. The table gives the numbers as the reports record them.

Arm Memory Warm shot Encoder loads
Default, cache miss 31.7 GB RAM after a shot; 30,540 MB held 85 s loaded on each miss
Cache hit 3.1 GB RAM after a shot; 3,062 to 3,094 MB 49 s 0
--cache-none 2.9 GB RssAnon over five prompts 61 s (54 to 67) not stated

The census named the holder. The CLIP object in the cache entry of the CLIPLoader node held the Gemma encoder, at 14.74 GB of tensors. A cache miss did the same thing. A cache hit never loaded it at all.

The --cache-none arm kept memory flat and produced output that was pixel-identical to the default. It was slower than a cache hit. The miss arm with the cache limit set to eight entries ended in an out-of-memory kill, as the table's row records.

The headroom setting

ComfyUI's cache has a headroom: the amount of memory it may hold before it evicts entries. At our memory setting of 36 GB, the default headroom is 3.6 GB. That was too small to evict the encoder before the next prompt loaded its own, so the next prompt's loads passed the memory cap.

A separate arm raised the headroom to 12 GB. It stayed flat at 5.0 to 5.1 GB over three prompts, with no kill. That arm was not the cache-hit arm, and the record reports it as a memory result. The miss arm with the eight-entry limit ended in a kill, which is the failure the flat arm did not show. Raising the headroom changes when entries are evicted, not what the render computes. The record gives the flat memory figure for that arm and nothing about its output.

What failed

The default cache headroom evicts too late. The next prompt's loads pass the memory cap before the old encoder is freed, which is how the miss arm ran into the kill. The cache held the encoder longer than the memory allowed. The census is what showed the encoder, so it is the number to check first.

What we still do not know

About 15 GB of the server's anonymous memory is not attributed to a live tensor. The heap report puts it at about 15 GB. Subtracting the 14.74 GB from the 31.6 GB total gives about 17 GB, so the two numbers differ by the way the report counted. We quote the report's figure. A private mapping of the encoder file accounts for 12.4 GB of it, and three anonymous regions for the rest. We have not traced those regions.

Switching to a different allocator did not change the total. This is one server with one card. We do not know how the holder behaves on other hardware.

How to prove this wrong

Run the miss arm with a census after each of the first three prompts. If the 14.74 GB of encoder tensors are not held by the CLIP object in the CLIPLoader entry, the holder claim is wrong, and the memory was going somewhere else. Then run a hit arm on a second server with the same model and check whether it also drops to about 3 GB.

Provenance

  • /home/mario/Projects/AssetsHQ/docs/lanes/video-heap-report.md, lines 10 to 14 (cache-none holds RssAnon at 2.9 GB over five prompts, warm shot 61 s, pixel-identical output; cache-ram 12 flat at 5.0 to 5.1 GB; the census names the CLIP object in the CLIPLoader entry at 14.74 GB; about 15 GB of RssAnon not attributed), line 24 (cache hits: encoder never loaded, 3,062 to 3,094 MB), line 25 (cache misses: 30,540 MB with 14.74 GB held, row ends in an OOM kill).
  • /home/mario/Projects/AssetsHQ/docs/lanes/video-fast-report.md, lines 47 to 48 (cache hit: warm shot 85 s to 49 s, RAM after a shot 31.7 GB to 3.1 GB; output identical on the miss path).
  • /home/mario/Brain/AssetsHQ/2026-11-17-vt-heap.md, lines 17 to 18 (census holder; jemalloc did not change the 31.5 GB after prompt 1).

Comments

No comments yet.

Log in to comment.