71 lines
3.4 KiB
Markdown
71 lines
3.4 KiB
Markdown
# OCR
|
|
|
|
Text in images, with where each line is: a batch of RGBA images in, each image's text lines out,
|
|
each with a box and a confidence. It runs ONNX models through onnxruntime
|
|
([`modules/onnxruntime`](../onnxruntime)): on an NVIDIA GPU through CUDA when every library that
|
|
needs is present, and on the CPU otherwise.
|
|
|
|
## Setting up
|
|
|
|
The binaries and the models are downloaded, not committed:
|
|
|
|
```
|
|
cd modules/onnxruntime && jai fetch.jai # onnxruntime 1.30.0, GPU build (CUDA 13); `- cpu` for the CPU-only build
|
|
cd modules/OCR && jai fetch_models.jai # both models (about 195 MB); `- mobile` for the light one only
|
|
jai generate.jai # in modules/onnxruntime, only when changing onnxruntime versions
|
|
```
|
|
|
|
Each checks the SHA-256 of what it downloads. Building the app (`jai first.jai`) copies onnxruntime
|
|
beside the executable and the models into `bin/ocr-models`.
|
|
|
|
## Using it
|
|
|
|
```jai
|
|
#import "OCR";
|
|
|
|
engine, ok, error := ocr_create(models_folder, "ppocr-v5-mobile"); // prefer_gpu = true
|
|
in_use, explanation := ocr_gpu_status(engine); // why not the GPU, and what to install
|
|
results := ocr_process(engine, images); // [] Ocr_Image (RGBA) -> [] Ocr_Result
|
|
for results[0].lines print("% at %,%: %\n", it.text, it.x, it.y, it.confidence);
|
|
ocr_destroy(engine);
|
|
```
|
|
|
|
`ocr_process` shares a batch out over worker threads: detection per image, then recognition of
|
|
every line of every image in batches of similar width. The results come from the context
|
|
allocator, or the one passed. `ocr_stats` gives the time spent in each stage.
|
|
|
|
## Models
|
|
|
|
`models.config` lists them; `ocr_create` takes a name. Both are PaddleOCR's PP-OCRv5 (Apache 2.0),
|
|
as converted to ONNX by RapidOCR, and read Chinese, English and Japanese:
|
|
|
|
| Name | Files | On the 20 sample captures, CPU |
|
|
| --- | --- | --- |
|
|
| `ppocr-v5-mobile` | detector 4.8 MB, recognizer 16.6 MB | about 0.5 s per capture |
|
|
| `ppocr-v5-server` | detector 88 MB, recognizer 85 MB | about 2.5 s per capture |
|
|
|
|
Another model with the same two-stage design is a new `[model NAME]` section; a different design
|
|
needs its own detection or recognition code beside `detect.jai` and `recognize.jai`.
|
|
|
|
## The GPU
|
|
|
|
onnxruntime's CUDA build needs, on PATH or beside the executable:
|
|
|
|
- the NVIDIA driver;
|
|
- the CUDA Toolkit 13.x (`cudart64_13`, `cublas64_13`, `cublasLt64_13`, `cufft64_12`), from
|
|
[developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads), whose installer puts its bin folder on PATH;
|
|
- cuDNN 9.x for CUDA 13 (`cudnn64_9` and its `cudnn_*64_9` companions), from
|
|
[developer.nvidia.com/cudnn-downloads](https://developer.nvidia.com/cudnn-downloads), whose bin folder for CUDA 13 you add to PATH yourself.
|
|
|
|
`ocr_create` looks for each of these first (`cuda_check.jai`) and says which are missing, rather
|
|
than letting onnxruntime fail later and obscurely. If they are all there but CUDA still cannot
|
|
start, it runs on the CPU and keeps onnxruntime's message.
|
|
|
|
## Measuring
|
|
|
|
`jai first.jai - release ocr_bench`, then `bin/ocr-bench MCAP_FOLDER`: each model on the GPU (when
|
|
available) and the CPU, the time per capture and per stage, and each capture's lines in
|
|
`bin/ocr-out/<model>-<provider>/`. Like any command-line tool here it sets
|
|
`runtime_support_headless` (`modules/Runtime_Support.jai`): a failure is printed and exits with
|
|
code 1, with no message box to wait on.
|