Musa Mahmood
da616d2802
bin/gpu-bench times every GPU model (OCR mobile and server, the label image encoder, segmentation; optionally the CPU and the label text encoder) over a fixed 30-image corpus in tools/gpu_bench/corpus. Every corpus file is checked against corpus.txt's SHA-256 before anything runs; each model's files and models.config entry are fingerprinted; the GPU, driver, power limit, CUDA runtime, cuDNN and onnxruntime are recorded, and NVML samples memory, load, power and clocks during the timed passes. Results go to bin/gpu-bench-results as text files. The GPU Bench button opens a window that runs gpu-bench in its own process (no console) and lays results files from several machines side by side against a baseline, marking a different corpus or model.
51 lines
3.3 KiB
Markdown
51 lines
3.3 KiB
Markdown
# gpu-bench (the `gpu-bench` branch, not for master)
|
|
|
|
Times every model mcap runs on the GPU over a fixed corpus, so machines can be compared like for
|
|
like:
|
|
|
|
| Task | What runs |
|
|
| --- | --- |
|
|
| `ocr/ppocr-v5-mobile`, `ocr/ppocr-v5-server` | detection, orientation and recognition, 8 images per call, as a scan does |
|
|
| `labels/siglip2-base-patch16-384` | the label model's image encoder |
|
|
| `segment/mask2former-swin-base-coco-panoptic` | panoptic segmentation |
|
|
| `labels-text/...` (with `-cpu`) | the text encoder, one phrase at a time as a `like:` search runs it (always on the CPU) |
|
|
|
|
## Keeping it consistent
|
|
|
|
- **The corpus** (`corpus/`, 30 images: 16 COCO val2017 photos, 2 of them scaled to 12 MP, and 12
|
|
synthetic screenshots from 720p to 4K made by `make_corpus.py`) is in git. The build copies it to
|
|
`bin/gpu-bench-corpus`. Before anything runs, every file is checked against its SHA-256 in
|
|
`corpus.txt`, and a run stops if one differs. The corpus fingerprint, a hash of those name and hash
|
|
pairs, goes in the results. `.gitattributes` marks the images binary, so git never changes them.
|
|
- **The models:** each task's fingerprint covers the hash of each of its files and its
|
|
`models.config` entry (comments and spacing left out). It also says whether every file is the one
|
|
the fetch scripts pin. The segmentation `.onnx` is exported on each machine, so it has no pin:
|
|
copy it from one machine to the others.
|
|
- **The machine:** each result records the GPU, its memory, enforced power limit and clocks, the
|
|
driver, the CUDA runtime and cuDNN actually loaded (with their paths), onnxruntime, the CPU, RAM,
|
|
and AC or battery.
|
|
- **The run:** each model gets one untimed run on the first image, one untimed warm-up pass over
|
|
everything, then N timed passes (default 3). The median is reported, with the best and worst
|
|
passes. While the timed passes run, NVML samples the GPU every 100 ms: peak memory over the
|
|
device's use before the model loaded, utilisation, power, SM clock and temperature.
|
|
- **The output:** what the models found (OCR lines and characters, segments, best phrases) and a
|
|
digest of it. Different GPUs can round differently, so a different digest is a hint, not an error.
|
|
|
|
## Running
|
|
|
|
`jai first.jai - release` builds `bin/gpu-bench.exe` with mcap. Then either:
|
|
|
|
- **In mcap:** click **GPU Bench** above the grid, choose what to run, and click **Run the benchmark**.
|
|
It runs gpu-bench in a separate process with no console and shows its progress. Close mcap and use
|
|
the terminal for the cleanest numbers, as mcap's own models stay loaded while it runs.
|
|
- **In a terminal:** `bin\gpu-bench.exe [-only ocr,labels,segment] [-models a,b] [-repeats N] [-warmup N] [-cpu | -cpu-only] [-note TEXT]`
|
|
|
|
Results go to `bin/gpu-bench-results/<machine>_<date>.txt` (plain text, `results.jai`). To compare
|
|
machines, copy their files into that folder: the window puts the results side by side. Each result
|
|
shows its speed against the baseline (the oldest file, or the one you pick). Its corpus row turns red
|
|
when the images differ, and a `*` marks a model whose files or settings differ. Hover a time for its
|
|
details.
|
|
|
|
The server OCR model can take a lot of GPU and system memory and a long time (see the main README's
|
|
known issue). Leave it out with `-models ppocr-v5-mobile,siglip2-base-patch16-384,mask2former-swin-base-coco-panoptic`.
|