MusaMahmood/mcap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

[0.1.16] README: the server OCR model's bimodal speed on the GPU, as a known issue

Browse Source 
The README's "about 0.4 s" for the server model was measured on 2026-10-07 (e024b7f), and the
batching it ran with has not changed since. Measured now, the same build on the same 20 captures
runs at about 0.27 s a capture with 2.4 GB at peak, or about 1.5 s with 6.6 GB, with either
allocator; longer runs fare worse. Until that is fixed, the mobile model is the one to scan a large
library with.
This commit is contained in:
Musa Mahmood 2026-10-13 08:30:50 +01:00
parent 03f23f7122
commit 2bd77e8121
1 changed files with 12 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
README.md
View File

@ -98,7 +98,18 @@ grid (in its order, so through a search's matches).
### The OCR window ### The OCR window
- **Model:** PP-OCRv5 mobile (light, about 85 ms a capture on the GPU) or server (more accurate, - **Model:** PP-OCRv5 mobile (light, about 85 ms a capture on the GPU) or server (more accurate,
about 0.4 s). The choice is kept in the settings file (`model` under `[ocr]`). about 0.3 s when it runs well; see below). The choice is kept in the settings file (`model` under
`[ocr]`).
- **Known issue, the server model on the GPU:** its speed is bimodal. The same program on the same
20 captures (RTX 5080 laptop, 16 GB) runs either at about 0.27 s a capture with a peak of about
2.4 GB of process memory, or at about 1.5 s with about 6.6 GB, with no change between runs and
15 GB of the GPU's memory free beforehand. Longer runs fare worse: 50 captures peaked at 16 GB, and
100 averaged about 4.6 s a capture. The extra memory looks like GPU allocations spilling into shared
system memory, which Windows then pages: what the GPU memory defaults (onnxruntime's
kNextPowerOfTwo arena, no cuDNN max workspace, 4 workers) were meant to stop, and which they
only partly did. It is the same with Jai's allocator and with rpmalloc. Until it is fixed, scan a
large library with the mobile model. Measured on 2026-10-13 with
`ocr-bench -models ppocr-v5-server -gpu-only -limit 20`.
- **Running on:** the GPU or the CPU, and if the CPU, what to install for the GPU. - **Running on:** the GPU or the CPU, and if the CPU, what to install for the GPU.
- **Scan the library** reads every capture the model has not read yet and, with **Labels too**, - **Scan the library** reads every capture the model has not read yet and, with **Labels too**,
labels every capture not yet labelled with the current list. While a search is active the button labels every capture not yet labelled with the current list. While a search is active the button