From 2bd77e8121cdd03f7c039b487fd6803a947eea69 Mon Sep 17 00:00:00 2001 From: Musa Mahmood Date: Tue, 13 Oct 2026 08:30:50 +0100 Subject: [PATCH] [0.1.16] README: the server OCR model's bimodal speed on the GPU, as a known issue The README's "about 0.4 s" for the server model was measured on 2026-10-07 (e024b7f), and the batching it ran with has not changed since. Measured now, the same build on the same 20 captures runs at about 0.27 s a capture with 2.4 GB at peak, or about 1.5 s with 6.6 GB, with either allocator; longer runs fare worse. Until that is fixed, the mobile model is the one to scan a large library with. --- README.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 26a8435..8964c99 100644 --- a/README.md +++ b/README.md @@ -98,7 +98,18 @@ grid (in its order, so through a search's matches). ### The OCR window - **Model:** PP-OCRv5 mobile (light, about 85 ms a capture on the GPU) or server (more accurate, - about 0.4 s). The choice is kept in the settings file (`model` under `[ocr]`). + about 0.3 s when it runs well; see below). The choice is kept in the settings file (`model` under + `[ocr]`). +- **Known issue, the server model on the GPU:** its speed is bimodal. The same program on the same + 20 captures (RTX 5080 laptop, 16 GB) runs either at about 0.27 s a capture with a peak of about + 2.4 GB of process memory, or at about 1.5 s with about 6.6 GB, with no change between runs and + 15 GB of the GPU's memory free beforehand. Longer runs fare worse: 50 captures peaked at 16 GB, and + 100 averaged about 4.6 s a capture. The extra memory looks like GPU allocations spilling into shared + system memory, which Windows then pages: what the GPU memory defaults (onnxruntime's + kNextPowerOfTwo arena, no cuDNN max workspace, 4 workers) were meant to stop, and which they + only partly did. It is the same with Jai's allocator and with rpmalloc. Until it is fixed, scan a + large library with the mobile model. Measured on 2026-10-13 with + `ocr-bench -models ppocr-v5-server -gpu-only -limit 20`. - **Running on:** the GPU or the CPU, and if the CPU, what to install for the GPU. - **Scan the library** reads every capture the model has not read yet and, with **Labels too**, labels every capture not yet labelled with the current list. While a search is active the button