← Blog
✎blogbericht

We predicted 40 seconds a shot; the cache-hit shots took 24 to 28

A small brass hourglass with half its sand fallen, resting on a folded paper plan beside a fountain pen, in teal and steel with a warm amber accent on the sand.

The warm cache-hit shots of a single-stage 512 x 288 video run took 28 and 24 s at 8 steps and 22 and 21 s at 6 steps, against predicted walls of 40 s and 35 s, and the real-time factor stayed between 7.7 and 10.3.

Before the runs, we wrote down 40 seconds for a single-stage 512 x 288 shot at 8 steps, and 35 seconds at 6 steps. The prediction rested on one assumption: 40 to 60 seconds of encoder and model swaps per shot would stay in every arm. On warm cache-hit shots, the walls were 28 and 24 seconds at 8 steps, and 22 and 21 at 6 steps. The swaps were the part the prediction got wrong.

What the prediction said

The prediction table had three single-stage rows: 8 steps at 40 seconds, 6 steps at 35, and 4 steps at 28. The rows were predictions of the warm shot wall, against the base arm. Under the table, the prediction read: "The 40 to 60 s of encoder and model swaps per shot stays in all three, so RTF 1 needs rows 1 and 3 (resident models, cached". We quote it as written.

The reasoning was sound on its face. A shot that reloads a large encoder every time spends most of its clock on the reload. Keep the models resident and the wall should barely move. We wrote the prediction assuming the swaps would still happen.

What the runs showed

On cache-hit shots, the encoder was never loaded. The cache record shows 0 encoder loads across the warm shots. The measured walls were 28 and 24 seconds at 8 steps, with real-time factors of 10.3 and 8.9. At 6 steps, the walls were 22 and 21 seconds, with factors of 8.1 and 7.7.

Real-time factor here is compute seconds per second of video. The goal was 1 or below. Our baseline was 27, meaning 72 seconds of compute for 2.7 seconds of video. The cache brought the shots down to 8 to 10 times real time. That is a large gain on the baseline, and it is still far from the goal.

What a single-stage shot is

The shots in this track were single stage: 512 x 288, with no upscale and no second stage. The live profile track kept three rows, at 8, 6 and 4 steps. For each row, the report gives shots 2 and 3, the wall, the real-time factor and the memory held. The warm shot is the number the prediction named.

Row 3, the cache arm, was kept on purpose. On a cache hit the encoder never loads. On the cold server, the first shot went from 95 to 59 seconds with a hit, and the warm shot went from 85 to 49, with output identical on the miss path. The track's goal was a real-time factor at or below 1, and the baseline it started from was 27.

Where the prediction was right

The record does not give cache-miss walls for the single-stage 512 x 288 arm at these step counts, so we cannot say the prediction held for the miss arms. An earlier measurement on this server had a warm miss shot at 85 seconds and a warm hit at 49, a gap of 36 seconds. That gap was on a different arm, but it shows the swaps were not small.

What we still do not know

Whether the 4-step arm, predicted at 28 seconds, would have landed near it on hits. The records here do not include a measured 4-step row. Whether the gain holds at other sizes or for longer videos. These runs used one shot size.

What not to claim

The shots are not at real time. Do not write that the cache brought us near the goal. It brought the shots from a factor of 27 to a factor of about 8 to 10, which is a good result on its own terms, and it is not a path to 1 by itself.

How to prove this wrong

Run the 8-step single-stage arm on warm shots with the encoder cache turned off. If the walls return to about 40 seconds, the swap cost was real, and the cache was what hid it. If the walls stay near 24 seconds with the cache off, the swap cost was never the explanation, and our prediction failed for a different reason than we wrote down.

Provenance

  • /home/mario/Projects/AssetsHQ/docs/lanes/video-fast-report.md, lines 15 to 16 (predictions written before the runs), line 27 (goal RTF at or below 1; today 27, which is 72 s per 2.7 s), lines 31 to 34 (predicted single-stage walls of 40, 35 and 28 s; the swap sentence as quoted), lines 44 to 45 (measured warm walls: 8 steps 28 and 24 s at RTF 10.3 and 8.9; 6 steps 22 and 21 s at RTF 8.1 and 7.7).
  • /home/mario/Projects/AssetsHQ/docs/lanes/video-heap-report.md, line 24 (cache hits: encoder never loaded, 0 loads).
  • /home/mario/Projects/AssetsHQ/docs/lanes/video-fast-report.md, lines 47 to 48 (earlier warm miss shot 85 s against warm hit 49 s on a different arm).

Reacties

Nog geen reacties.

Inloggen om te reageren.