- modules/Capture_File/segment_chunks.jai: a front "segs" chunk (text: engine, date, the image read
and its size, the class list's hash, and each segment's id, score, area, box and class) and a back
"mask" chunk (a segment id per pixel, run-length encoded with LEB128 lengths, longest side at most
1024), keyed by model. No new file version: chunks came in version 3.
- mcap_store takes back chunks too, so a model's summary and mask are stored in one rewrite.
- Tests: the text round trip, the runs' edge cases (one segment over everything, none, runs across
rows, multi-byte lengths, id 255), bad masks refused, and storing, replacing and erasing both
chunks in a file with other chunks kept.
- segment-probe takes .mcap files (the image each is shown as; videos skipped) and with -store stores
its segments in them. On 27 images a mask is 0.8 to 6.4 KB.
- The format README's "Segmentation chunks" section; the plan's phase 3 results.
- segmentation-plan.html: the plan for panoptic segmentation of images, per library folder, stored
in .mcap chunks (no file version bump), in six phases.
- tools/export_mask2former.py: exports facebook/mask2former-swin-base-coco-panoptic to ONNX (opset 17,
dynamic batch, 384 x 384 input) and checks it against PyTorch: on 25 images, all 180 segments
identical (IoU 1.000). Writes the class list with names for people ("sky-other-merged" is "sky"),
and PyTorch's own segmentations for segment-probe -check.
- modules/Segment: runs the model with onnxruntime (CUDA, else the CPU) and makes segments of its
outputs as transformers' post_process_panoptic_segmentation does: a segment id map (longest side at
most 1024) and each segment's class, score, area and bounds. Matches PyTorch on 99.8% of pixels,
every segment scored 0.6 or more found. 57 ms an image at batch 4 on an RTX 5080 laptop.
- tools/segment_probe.jai (jai first.jai - segment_probe): segments image files; -out draws them,
-check compares with PyTorch, -dump-input writes the model's input.
- modules/onnxruntime: ort_run_float_outputs, for a model with several outputs.
- first.jai copies the model to bin/segment-models. The model itself is not in git
(modules/Segment/models, 462 MB); tools/export_mask2former.py makes it.
