Semantic segmentation in mcap

1. Scope

Each opted-in image gets a panoptic segmentation. Every pixel gets a class ("person", "dog", "sky", "wall"), and countable things are split into instances: two people are two segments. The results serve three purposes:

• Search: has:dog, has:person, and combined with the other search terms.
• Stats: which things the shown captures hold most.
• Looking: coloured masks over the image in the Capture window, with the class under the mouse.

What counts as an image. Imported photos and screenshots, Snagit captures and animated GIFs (their first frame) qualify. Videos are skipped even in opted-in folders: a video's poster frame is an image technically, but frames are the later project. A capture is a video when its catalog entry's media is VIDEO, which comes from VideoCodec in its import.json.

2. The model

Which Mask2Former

COCO panoptic is the default because a photo library's questions are mostly about things ("photos with the dog", "group shots"), and only the panoptic models separate instances.

Export to ONNX

There is no official ONNX release, so phase 1 exports one with a Python script kept in the repo (tools/export_mask2former.py). It runs once on a developer machine and is never shipped.

• Deformable attention. The pixel decoder uses multi-scale deformable attention. Transformers has a pure-PyTorch version built on grid_sample, which exports at opset 17 or later. The script forces that path.
• Outputs. The script exports up to class_queries_logits [B, 100, 134] and masks_queries_logits [B, 100, H/4, W/4]. The panoptic merge (below) is done in Jai, as it is short, and keeping it out of the graph keeps the export simple.
• Input. pixel_values [batch, 3, 384, 384]: RGB squashed to 384 × 384, ignoring the aspect ratio, as this checkpoint's processor does (bilinear), scaled to 0..1, then ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225). The batch size is dynamic; the size is fixed.
• Check. On 20 or more images, compare the ONNX output, run through mcap's post-processing, with transformers' post_process_panoptic_segmentation. Per-segment IoU should be at least 0.98.

Packaging, as Labels and OCR are

• A new module, modules/Segment, laid out like modules/Labels.
  • models.config: @version 1, then [model mask2former-swin-base-coco-panoptic] with title, model (the .onnx file name), classes (a text file with one class name per line, plus whether each is a "thing"), input_size, batch_gpu, batch_cpu, score_threshold (0.8) and overlap_threshold (0.8).
  • fetch_models.jai: the files, their SHA-256 hashes, and where they come from. The exported .onnx has to be hosted somewhere you control, such as a Hugging Face repo or a release asset. See open questions.
  • first.jai copies the model folder to bin/segment-models, as it does bin/label-models.
• segment_create and segment_destroy follow labels_create. That means ort_cuda_check, a GPU session made under ort_gpu_lock, and a CPU fallback with the tiny model.
• segment_images(engine, images, cancel) -> [] Segment_Result, given RGBA in, as Labels_Image and Ocr_Image are:
  1. Squash to 384 × 384 and normalise, as above.
  2. Run in batches.
  3. The panoptic merge, as transformers 5.19 does it:
     1. Softmax over the classes, and sigmoid over the masks scaled up to 384, then to the map.
     2. Drop queries scoring 0.5 or under, or whose class is "no object".
     3. Weight each mask by its query's score, and assign each pixel to its best query.
     4. Drop a segment if it keeps 80% or less of its own mask.
     Stuff of one class is not merged.
  4. Result: a segment-id map at the mask resolution, plus a list of segments (id, class, score, area, box).
  It checks cancel between batches, as ocr_process now does.
• tools/segment_probe.jai, like image-probe: segment some files, print their segments, and write the masks out as PNGs to look at.

3. Per-folder opt-in

A library folder is a top-level folder of the library, one per place imported from (SnagitLibraryWork, Phone, Photos 2014). Segmentation runs only in the folders you turn it on for. A new folder starts with it off, so adding a 40,000-photo archive never starts an hours-long GPU job by surprise.

Where the setting lives

It goes in a new file in the library root, .library-folders, next to .import-index and .import-sources. Not in %APPDATA%\mcap\settings.config, because the choice belongs to the library, not to the machine. It follows the library when it is opened elsewhere, or with -library FOLDER. The format follows the Ex4 style used in the rest of mcap, so later per-folder options (OCR off for a photos folder, say) go in the same file:

@version 1

[folder "Photos 2014"]
segment: true

[folder "SnagitLibraryWork"]
segment: false

• Missing file or folder: every option is off. Unknown keys are kept on rewrite, as unknown chunks are.
• Who reads and writes it: library_folders_load and library_folders_save in a new src/app/library_folders.jai, on the main thread, written beside itself and swapped in, as the import index is. It lives in a table keyed by folder name.
• A renamed folder loses its setting. The Segmentation window lists settings for folders that no longer exist, with a button to remove them.

Where you set it

• The Segmentation window (a new button next to Labels) has a table of library folders with these columns:
  • Segment: the checkbox.
  • Images: the count, excluding videos.
  • Segmented: how many are done with the current model.
  • Erase: a per-folder button.
  Turning a folder off stops future scans but erases nothing.
• The Import window adds a checkbox, "Segment the images in this folder". It shows and sets the destination folder's option. For a new folder it starts off; for an existing folder it shows that folder's setting.
• The folder combo above the grid marks opted-in folders, for example with a small "seg" after the name. This is a nicety; it can come last.

4. The segmentation scan

• The thread. It runs on the existing scan thread (the "ocr" thread in ocr_ui.jai), as a new task kind SEGMENT. One thread doing GPU work at a time is already the rule there, and it brings pause, cancel, the progress bar and -scan-style headless runs for free.
• Which captures. Every capture in an opted-in folder whose media is not VIDEO and has no segs chunk from the current model. Unlike the OCR scan, it ignores the grid's folder filter and search: the scope is the folder settings. A "Segment the captures shown" button can be added later if wanted.
• Batches. Batches of SEGMENT_SCAN_BATCH (4 to start, since Mask2Former is heavier per image than OCR). Each image is decoded once with mcap_read_shown_image, so a Snagit capture uses its rendered image. Cancel is checked between batches and inside segment_images; a cut-short batch stores nothing, as for OCR.
• After an import. When "Read text and labels" is on and the destination folder segments, the import's follow-up scan becomes OCR and labels, then segmentation of the new images. These are two tasks queued one after the other, not one combined pass. They write different chunks, and keeping them apart keeps each simple.
• Headless. -segment segments every opted-in folder and exits, 0 or 1, as -scan does.
• Erase. -erase segments, and -erase all now includes it. Add Erase_What.SEGMENTS, its tags (segs, mask) and its "has" test. Per-folder erase in the Segmentation window uses the same task, limited to that folder's captures.
• Shutdown. It is the same thread, so the 0.0.10 shutdown status covers it. Add a "Stopping the segmentation scan" line.

5. Storage in .mcap

segs, front chunk, keyed by model name

This is what search and Stats need, small enough for the front that every library browse reads. It is text in the style of labl:

@version 1
engine:  "onnxruntime 1.30.0, CUDA"
date:    2026-10-12T09:30:00Z
image:   rendered 4032 3024
classes: 5c1e9a0d77b2f413
segment: 1 0.981 0.2140 1210 380 900 2210 "person"
segment: 2 0.955 0.1022 2300 900 700 1400 "person"
segment: 3 0.912 0.0811 400 1900 800 700 "dog"
segment: 4 0.993 0.3105 0 0 4032 1100 "sky"

A busy photo has 10 to 30 segments, about 1 to 2 KB.

mask, back chunk, keyed by model name

This is the segment-id map, for drawing only. It is binary:

u8   format        1: run-length encoded, row-major
u16  width         the map's size: the image scaled so its longest side is at most 1024
u16  height
u8   id_bits       8 (up to 255 segments), or 16 later if needed
then runs:         (id, length) pairs over the whole map, row by row, wrapping across rows
                   id as u8; length as a LEB128 varint
                   id 0: no segment (below threshold, or "no object")

• Why run-length encoding: segment maps are large flat regions, so runs are few. It decodes in one pass with no dependencies, and an expected 5 to 40 KB per photo is a few percent of a typical JPEG.
• The alternative: a palette PNG through stb_image_write. It is smaller on noisy maps but encodes and decodes more slowly, and RLE is simpler to test. Revisit only if mask chunks turn out large.
• Why at most 1024 on the long side: it is plenty for an overlay, and the model's own output is at a quarter of its input size, so storing more would invent detail. The overlay scales it up.

Code changes

• modules/Capture_File/segment_chunks.jai, next to label_chunks.jai:
  • The tags MCAP_SEGMENTS_TAG "segs" and MCAP_MASK_TAG "mask".
  • The text read and write functions, mcap_segments_text and mcap_segments_parse.
  • The mask functions, mcap_mask_bytes and mcap_mask_parse.
• mcap_store gains a back: [] Mcap_Chunk parameter, so segs and mask are written in one rewrite. Today a back chunk needs its own call (mcap_store_chunk(..., back = true)), which would rewrite the file twice and could leave one without the other.
• mcap_read_without_original already reads the back chunks, so the overlay needs no new reader.
• The format README gets a "Segmentation chunks" section.

6. Catalog, search, Stats

• Catalog. Catalog_Entry gains:
  • segments: [..] Catalog_Segment: class, area and score for each segment, from the front segs chunk keyed by the current model.
  • segment_classes: the hash, so stale results can be found.
  • find_segments: the class names, lowercased, one per line.
  catalog_entry_free frees them. The mask is never loaded into the catalog.
• Search. A new field, has: (Search_Field.HAS), matching a whole class name: has:dog, has:"traffic light".
  • Plain words do not match segment classes, as with folder:, so dog keeps meaning text and titles, and has:dog means it is in the picture.
  • Later: counts and sizes, such as has:person:3+ or has:dog>20%. The front chunk already holds what they need.
• Stats. Add "Segmented: N of M images shown" and a top-classes list: how many captures contain each class. Clicking a class searches has:class, as the top labels search label:.

7. Capture window overlay

• A Segments row under the labels line, shown when the capture has segs from the current model:
  • A toggle and an opacity slider.
  • A legend: each class with its colour swatch and a checkbox to hide it. Clicking a class name searches has:class.
• The mask texture. When the toggle is on, a viewer worker reads the capture's mask chunk (mcap_read_without_original, which does not read the original). It turns the run-length map into RGBA, colour per class and alpha from the slider, at the map's resolution. The main thread uploads it with gpu_texture_create, as it does the full image. Hidden classes are drawn transparent; changing which are hidden or the opacity rebuilds the texture on the worker, at most a few times a second.
• Drawing. In viewer_full_window, after Image(...), next to ocr_ui_capture_overlay: draw.AddImage(mask_texture, image_min, image_min + source * zoom). The GPU scales it up, and the map's aspect ratio matches the image.
• Hover. The pixel under the mouse is mapped into the map, and its segment's class and score are shown as a tooltip ("dog · 0.91 · 8% of the image").
• Colours. A fixed palette by class index, the same in every capture, so a class keeps its colour. Pick it for contrast in both themes.

8. Phases

Each phase ends with something to run and check, and is a commit (0.0.11 onwards).

9. Open questions

1. COCO panoptic or ADE20K? COCO counts things (people, pets, cars); ADE20K knows more indoor objects and scene parts, but has no instances. The default is COCO. Revisit after phase 1 by trying both on a sample of your photo libraries.
2. Where to host the exported .onnx for fetch_models.jai: a private Hugging Face repo, a release asset, or committing it as the FFmpeg DLLs are. At about 400 MB for Swin-B, committing it is probably too much.
3. The mask resolution cap. Answered 1024 on the long side costs 0.8 to 6.4 KB per photo (phase 3), so there is no reason to go lower.
4. Snagit and screenshot folders. They stay off by default, and COCO classes mean little on UI. Turning one on is allowed, but the Segmentation window could warn that most of the folder is screenshots, judged from Format and AppName.
5. Re-scanning on a model change. A new model name means a new key, so old results stay until erased. Should the scan offer to erase other models' masks to save space? Probably yes, as an explicit button rather than automatically.

10. Later: video frames Shelved

Treat strategically sampled frames of a video as images, and run the same analysis (segmentation, and OCR and labels too) on them. Notes for when it comes back:

• Sampling: scene changes rather than a fixed rate. Compare small thumbnails of successive decoded frames, or use FFmpeg's scene score. Cap the frames per minute.
• Storage: keep the frames themselves out of the .mcap except as small thumbnails. Store per-frame results in chunks keyed by model and frame time (for example a key like mask2former…@83.40), or in one chunk holding every frame's results.
• The per-folder option would grow a second key, segment_videos, in the same .library-folders file.