OCR filled the GPU: 9 GB for the mobile model on gpu-bench's 30 images, past
16 GB for the server model, which Windows then paged into shared memory (1.5 to
11 s a capture). onnxruntime's logged arena extensions showed why: the arena
keeps every block it takes from CUDA and never joins or frees them, and OCR's
inputs grow from run to run (recognizer batches went narrowest first; images in
any order across calls), so each larger run left another block behind: 5.7 GB
of recognizer blocks where the widest batch needs 0.9 GB.
- Detection runs the largest images first, recognition the widest batches.
- The detector's and recognizer's arenas shrink once each stage of an
ocr_process call is done, by a run on a small input with
memory.enable_memory_arena_shrinkage. Shrinking after every run cost 33 to
59% of the speed.
- The worker threads are made with the engine and kept, not made for every
stage of every call (onnxruntime keeps GPU state for each thread that runs a
session, microsoft/onnxruntime#33037); 6% faster.
- OCR's arenas grow by what is asked (kSameAsRequested).
- ort.log_level sets what the engines' onnxruntime environments log; INFO shows
every arena extension.
Mobile OCR on gpu-bench (RTX 5080 laptop, 4 workers): peak 5.7 GB over the
baseline, 2 GB between calls, 142 ms a capture, against 9.0 GB and 126 ms. The
text read differs by a line in 2,986 (batches are grouped differently). The
server model is not yet measured again.
- .library-folders in the library (src/app/library_folders.jai): each library folder's options,
starting with segment: true. Every folder starts off; unknown keys are kept when it is rewritten.
- The Segmentation window (Segments, above the grid): each folder's Segment checkbox, how many of
its images are segmented, and Erase per folder; a button to segment the folders set to segment.
- The segmentation scan (src/app/segment_ui.jai) runs on the OCR thread as its own task, with the
OCR scan's progress, pause and cancel: images only (never videos), not yet segmented, in folders
set to segment. The engine loads with the first scan (2.4 GB of GPU memory). Each image's segs and
mask are stored in one rewrite and its catalog entry loaded again.
- Importing into a folder set to segment segments the new images; the Import window's checkbox is
the folder's setting.
- -segment segments and exits; -erase segments, and -erase all now includes segments.
- The catalog reads each capture's segs (class list hash, segment count). The exit overlay names the
segmentation scan and model. segment_mcap_chunks moved into modules/Segment, shared with
segment-probe. The quoting helpers of Capture_File's text formats are public.
- Tests: .library-folders read and written back (quoted names, unknown keys, versions).
- segmentation-plan.html: the plan for panoptic segmentation of images, per library folder, stored
in .mcap chunks (no file version bump), in six phases.
- tools/export_mask2former.py: exports facebook/mask2former-swin-base-coco-panoptic to ONNX (opset 17,
dynamic batch, 384 x 384 input) and checks it against PyTorch: on 25 images, all 180 segments
identical (IoU 1.000). Writes the class list with names for people ("sky-other-merged" is "sky"),
and PyTorch's own segmentations for segment-probe -check.
- modules/Segment: runs the model with onnxruntime (CUDA, else the CPU) and makes segments of its
outputs as transformers' post_process_panoptic_segmentation does: a segment id map (longest side at
most 1024) and each segment's class, score, area and bounds. Matches PyTorch on 99.8% of pixels,
every segment scored 0.6 or more found. 57 ms an image at batch 4 on an RTX 5080 laptop.
- tools/segment_probe.jai (jai first.jai - segment_probe): segments image files; -out draws them,
-check compares with PyTorch, -dump-input writes the model's input.
- modules/onnxruntime: ort_run_float_outputs, for a model with several outputs.
- first.jai copies the model to bin/segment-models. The model itself is not in git
(modules/Segment/models, 462 MB); tools/export_mask2former.py makes it.
