MusaMahmood/mcap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
3b76b6dcb9
mcap/tools/gpu_bench
History
Musa Mahmood 3b76b6dcb9 [0.1.26] gpu-bench: a memory trace per step, -ocr-workers and -ort-log 
Each result now has the GPU's memory over the baseline after loading, the
first image, each warm-up pass and each timed pass (with the pass's peak), in
the output and as trace= in the results file, so growth across passes (a leak)
shows apart from a working set. -ocr-workers sets OCR's GPU worker threads, and
-ort-log info prints onnxruntime's arena extensions: what found the cause of
0.1.24's memory fix.
2026-10-17 21:13:30 -04:00
..
corpus [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
corpus.txt [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
gpu_bench.jai [0.1.26] gpu-bench: a memory trace per step, -ocr-workers and -ort-log 2026-10-17 21:13:30 -04:00
make_corpus.py [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
README.md [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
results.jai [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
sha256.jai [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00
system_info.jai [0.1.25] GPU benchmark (gpu-bench branch, not for master) 2026-10-17 21:13:01 -04:00

README.md

gpu-bench (the gpu-bench branch, not for master)

Times every model mcap runs on the GPU over a fixed corpus, so machines can be compared like for like:

Task What runs
ocr/ppocr-v5-mobile, ocr/ppocr-v5-server detection, orientation and recognition, 8 images per call, as a scan does
labels/siglip2-base-patch16-384 the label model's image encoder
segment/mask2former-swin-base-coco-panoptic panoptic segmentation
labels-text/... (with -cpu) the text encoder, one phrase at a time as a like: search runs it (always on the CPU)

Keeping it consistent

  • The corpus (corpus/, 30 images: 16 COCO val2017 photos, 2 of them scaled to 12 MP, and 12 synthetic screenshots from 720p to 4K made by make_corpus.py) is in git. The build copies it to bin/gpu-bench-corpus. Before anything runs, every file is checked against its SHA-256 in corpus.txt, and a run stops if one differs. The corpus fingerprint, a hash of those name and hash pairs, goes in the results. .gitattributes marks the images binary, so git never changes them.
  • The models: each task's fingerprint covers the hash of each of its files and its models.config entry (comments and spacing left out). It also says whether every file is the one the fetch scripts pin. The segmentation .onnx is exported on each machine, so it has no pin: copy it from one machine to the others.
  • The machine: each result records the GPU, its memory, enforced power limit and clocks, the driver, the CUDA runtime and cuDNN actually loaded (with their paths), onnxruntime, the CPU, RAM, and AC or battery.
  • The run: each model gets one untimed run on the first image, one untimed warm-up pass over everything, then N timed passes (default 3). The median is reported, with the best and worst passes. While the timed passes run, NVML samples the GPU every 100 ms: peak memory over the device's use before the model loaded, utilisation, power, SM clock and temperature.
  • The output: what the models found (OCR lines and characters, segments, best phrases) and a digest of it. Different GPUs can round differently, so a different digest is a hint, not an error.

Running

jai first.jai - release builds bin/gpu-bench.exe with mcap. Then either:

  • In mcap: click GPU Bench above the grid, choose what to run, and click Run the benchmark. It runs gpu-bench in a separate process with no console and shows its progress. Close mcap and use the terminal for the cleanest numbers, as mcap's own models stay loaded while it runs.
  • In a terminal: bin\gpu-bench.exe [-only ocr,labels,segment] [-models a,b] [-repeats N] [-warmup N] [-cpu | -cpu-only] [-note TEXT]

Results go to bin/gpu-bench-results/<machine>_<date>.txt (plain text, results.jai). To compare machines, copy their files into that folder: the window puts the results side by side. Each result shows its speed against the baseline (the oldest file, or the one you pick). Its corpus row turns red when the images differ, and a * marks a model whose files or settings differ. Hover a time for its details.

The server OCR model can take a lot of GPU and system memory and a long time (see the main README's known issue). Leave it out with -models ppocr-v5-mobile,siglip2-base-patch16-384,mask2former-swin-base-coco-panoptic.