Semantic segmentation in mcap
+Planning document · 2026-10-10 · for mcap after 0.0.10
+
+ In short
+
+
+
+
+-
+
- Decided Mask2Former, exported to ONNX and run with the onnxruntime mcap already ships. It is swappable later, because results are keyed by model name. +
- Decided Images only, and only in the library folders you opt in. The setting is per folder, kept in a file in the library root, so it travels with the library. +
- Decided No .mcap version bump. Results go in two new chunk kinds: a small
segssummary at the front for search, and amaskat the back for drawing.
+ - It is built in six phases. The first proves the ONNX export matches PyTorch before anything else depends on it. +
- Progress Phases 1 and 2 are done (2026-10-10). The model is exported and checked, and
modules/Segmentruns it in mcap at about 57 ms per image on the RTX 5080. Next is phase 3: storing results in .mcap files.
+ - Later Video, by treating sampled frames as images, is shelved. +
1. Scope
+Each opted-in image gets a panoptic segmentation. Every pixel gets a class ("person", "dog", "sky", "wall"), and countable things are split into instances: two people are two segments. The results serve three purposes:
+-
+
- Search:
has:dog,has:person, and combined with the other search terms.
+ - Stats: which things the shown captures hold most. +
- Looking: coloured masks over the image in the Capture window, with the class under the mouse. +
What counts as an image. Imported photos and screenshots, Snagit captures and animated GIFs (their first frame) qualify. Videos are skipped even in opted-in folders: a video's poster frame is an image technically, but frames are the later project. A capture is a video when its catalog entry's media is VIDEO, which comes from VideoCodec in its import.json.
2. The model
+Which Mask2Former
+
+
+
+
+| Checkpoint (Hugging Face) | Classes | Size | Use |
|---|---|---|---|
facebook/mask2former-swin-base-coco-panoptic | 133 COCO panoptic: 80 things, 53 stuff | ~107M | Default. Things are counted as instances, and the stuff classes (sky, wall, grass, road) cover the rest. |
facebook/mask2former-swin-large-coco-panoptic | same | ~216M | If base misses too much and the GPU can take it. |
facebook/mask2former-swin-tiny-coco-panoptic | same | ~47M | The CPU fallback, when CUDA is unavailable. |
facebook/mask2former-swin-base-ade-semantic | 150 ADE20K, no instances | ~107M | The alternative, if indoor scenes and objects matter more than counting people. See open questions. |
COCO panoptic is the default because a photo library's questions are mostly about things ("photos with the dog", "group shots"), and only the panoptic models separate instances.
+ +Export to ONNX
+There is no official ONNX release, so phase 1 exports one with a Python script kept in the repo (tools/export_mask2former.py). It runs once on a developer machine and is never shipped.
-
+
- Deformable attention. The pixel decoder uses multi-scale deformable attention. Transformers has a pure-PyTorch version built on
grid_sample, which exports at opset 17 or later. The script forces that path.
+ - Outputs. The script exports up to
class_queries_logits[B, 100, 134] andmasks_queries_logits[B, 100, H/4, W/4]. The panoptic merge (below) is done in Jai, as it is short, and keeping it out of the graph keeps the export simple.
+ - Input.
pixel_values[batch, 3, 384, 384]: RGB squashed to 384 × 384, ignoring the aspect ratio, as this checkpoint's processor does (bilinear), scaled to 0..1, then ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225). The batch size is dynamic; the size is fixed.
+ - Check. On 20 or more images, compare the ONNX output, run through mcap's post-processing, with transformers'
post_process_panoptic_segmentation. Per-segment IoU should be at least 0.98.
+
Phase 1 result (2026-10-10): passed.
+
+-
+
- Test set: 25 images: 16 COCO val2017 photos, 5 of your photos and 4 video frames. +
- Accuracy: 180 segments, every one with an IoU of 1.000 against PyTorch, none missing and none extra. +
- Batching: batches of 2, 4 and 8 give bit-identical results to one at a time. +
- Speed: CPU 348 ms per image at batch 1, 290 ms at batch 2. +
- Size: 462 MB at opset 17. +
- Load warning: the "missing"
swin.layernormthat transformers 5.19 reports when loading is unused; randomizing it changes no output.
+ - Not yet measured: GPU timing and memory. Those come in phase 2, with mcap's own onnxruntime build. +
tools/export_mask2former.py.Risk. The export is the one part of this plan that may fail outright. If
+
+grid_sample export misbehaves, fall back to the ADE20K semantic SegFormer, which exports trivially, to build phases 2 to 6. Then return to Mask2Former, since everything after phase 1 is keyed by model name.Packaging, as Labels and OCR are
+-
+
- A new module,
modules/Segment, laid out likemodules/Labels. +-
+
models.config:@version 1, then[model mask2former-swin-base-coco-panoptic]withtitle,model(the .onnx file name),classes(a text file with one class name per line, plus whether each is a "thing"),input_size,batch_gpu,batch_cpu,score_threshold(0.8) andoverlap_threshold(0.8).
+ fetch_models.jai: the files, their SHA-256 hashes, and where they come from. The exported .onnx has to be hosted somewhere you control, such as a Hugging Face repo or a release asset. See open questions.
+ first.jaicopies the model folder tobin/segment-models, as it doesbin/label-models.
+
+ segment_createandsegment_destroyfollowlabels_create. That meansort_cuda_check, a GPU session made underort_gpu_lock, and a CPU fallback with the tiny model.
+ segment_images(engine, images, cancel) -> [] Segment_Result, given RGBA in, asLabels_ImageandOcr_Imageare: +-
+
- Squash to 384 × 384 and normalise, as above. +
- Run in batches. +
- The panoptic merge, as transformers 5.19 does it:
+
-
+
- Softmax over the classes, and sigmoid over the masks scaled up to 384, then to the map. +
- Drop queries scoring 0.5 or under, or whose class is "no object". +
- Weight each mask by its query's score, and assign each pixel to its best query. +
- Drop a segment if it keeps 80% or less of its own mask. +
+ - Result: a segment-id map at the mask resolution, plus a list of segments (id, class, score, area, box). +
cancelbetween batches, asocr_processnow does.
+ tools/segment_probe.jai, likeimage-probe: segment some files, print their segments, and write the masks out as PNGs to look at.
+
3. Per-folder opt-in
+A library folder is a top-level folder of the library, one per place imported from (SnagitLibraryWork, Phone, Photos 2014). Segmentation runs only in the folders you turn it on for. A new folder starts with it off, so adding a 40,000-photo archive never starts an hours-long GPU job by surprise.
Where the setting lives
+It goes in a new file in the library root, .library-folders, next to .import-index and .import-sources. Not in %APPDATA%\mcap\settings.config, because the choice belongs to the library, not to the machine. It follows the library when it is opened elsewhere, or with -library FOLDER. The format follows the Ex4 style used in the rest of mcap, so later per-folder options (OCR off for a photos folder, say) go in the same file:
@version 1
+
+[folder "Photos 2014"]
+segment: true
+
+[folder "SnagitLibraryWork"]
+segment: false-
+
- Missing file or folder: every option is off. Unknown keys are kept on rewrite, as unknown chunks are. +
- Who reads and writes it:
library_folders_loadandlibrary_folders_savein a newsrc/app/library_folders.jai, on the main thread, written beside itself and swapped in, as the import index is. It lives in a table keyed by folder name.
+ - A renamed folder loses its setting. The Segmentation window lists settings for folders that no longer exist, with a button to remove them. +
Where you set it
+-
+
- The Segmentation window (a new button next to Labels) has a table of library folders with these columns:
+
-
+
- Segment: the checkbox. +
- Images: the count, excluding videos. +
- Segmented: how many are done with the current model. +
- Erase: a per-folder button. +
+ - The Import window adds a checkbox, "Segment the images in this folder". It shows and sets the destination folder's option. For a new folder it starts off; for an existing folder it shows that folder's setting. +
- The folder combo above the grid marks opted-in folders, for example with a small "seg" after the name. This is a nicety; it can come last. +
4. The segmentation scan
+-
+
- The thread. It runs on the existing scan thread (the "ocr" thread in
ocr_ui.jai), as a new task kindSEGMENT. One thread doing GPU work at a time is already the rule there, and it brings pause, cancel, the progress bar and-scan-style headless runs for free.
+ - Which captures. Every capture in an opted-in folder whose media is not
VIDEOand has nosegschunk from the current model. Unlike the OCR scan, it ignores the grid's folder filter and search: the scope is the folder settings. A "Segment the captures shown" button can be added later if wanted.
+ - Batches. Batches of
SEGMENT_SCAN_BATCH(4 to start, since Mask2Former is heavier per image than OCR). Each image is decoded once withmcap_read_shown_image, so a Snagit capture uses its rendered image. Cancel is checked between batches and insidesegment_images; a cut-short batch stores nothing, as for OCR.
+ - After an import. When "Read text and labels" is on and the destination folder segments, the import's follow-up scan becomes OCR and labels, then segmentation of the new images. These are two tasks queued one after the other, not one combined pass. They write different chunks, and keeping them apart keeps each simple. +
- Headless.
-segmentsegments every opted-in folder and exits, 0 or 1, as-scandoes.
+ - Erase.
-erase segments, and-erase allnow includes it. AddErase_What.SEGMENTS, its tags (segs,mask) and its "has" test. Per-folder erase in the Segmentation window uses the same task, limited to that folder's captures.
+ - Shutdown. It is the same thread, so the 0.0.10 shutdown status covers it. Add a "Stopping the segmentation scan" line. +
5. Storage in .mcap
+No file version bump. Version 3 added tagged, model-keyed chunk lists, and the format README already names segmentation masks as what back chunks are for. A reader that does not know a chunk keeps it on rewrite. Each new chunk carries its own internal version, as
+
+labl does.segs, front chunk, keyed by model name
+This is what search and Stats need, small enough for the front that every library browse reads. It is text in the style of labl:
@version 1
+engine: "onnxruntime 1.30.0, CUDA"
+date: 2026-10-12T09:30:00Z
+image: rendered 4032 3024
+classes: 5c1e9a0d77b2f413
+segment: 1 0.981 0.2140 1210 380 900 2210 "person"
+segment: 2 0.955 0.1022 2300 900 700 1400 "person"
+segment: 3 0.912 0.0811 400 1900 800 700 "dog"
+segment: 4 0.993 0.3105 0 0 4032 1100 "sky"
+
+
+
+| Key | Value |
|---|---|
engine, date | As in OCR text. |
image | Which image was read (rendered or original) and its size. Boxes are in its pixels. |
classes | A 64-bit FNV-1a hash of the model's class list, so a model with new classes is seen as needing a new scan. |
segment | The segment's id in the mask, its score, its area (as a fraction of the image), its box (x y w h), and its class, quoted. Largest first. Stuff classes have one segment per class. |
A busy photo has 10 to 30 segments, about 1 to 2 KB.
+ +mask, back chunk, keyed by model name
+This is the segment-id map, for drawing only. It is binary:
+u8 format 1: run-length encoded, row-major
+u16 width the map's size: the image scaled so its longest side is at most 1024
+u16 height
+u8 id_bits 8 (up to 255 segments), or 16 later if needed
+then runs: (id, length) pairs over the whole map, row by row, wrapping across rows
+ id as u8; length as a LEB128 varint
+ id 0: no segment (below threshold, or "no object")-
+
- Why run-length encoding: segment maps are large flat regions, so runs are few. It decodes in one pass with no dependencies, and an expected 5 to 40 KB per photo is a few percent of a typical JPEG. +
- The alternative: a palette PNG through stb_image_write. It is smaller on noisy maps but encodes and decodes more slowly, and RLE is simpler to test. Revisit only if mask chunks turn out large. +
- Why at most 1024 on the long side: it is plenty for an overlay, and the model's own output is at a quarter of its input size, so storing more would invent detail. The overlay scales it up. +
Code changes
+-
+
modules/Capture_File/segment_chunks.jai, next tolabel_chunks.jai: +-
+
- The tags
MCAP_SEGMENTS_TAG "segs"andMCAP_MASK_TAG "mask".
+ - The text read and write functions,
mcap_segments_textandmcap_segments_parse.
+ - The mask functions,
mcap_mask_bytesandmcap_mask_parse.
+
+ - The tags
mcap_storegains aback: [] Mcap_Chunkparameter, sosegsandmaskare written in one rewrite. Today a back chunk needs its own call (mcap_store_chunk(..., back = true)), which would rewrite the file twice and could leave one without the other.
+ mcap_read_without_originalalready reads the back chunks, so the overlay needs no new reader.
+ - The format README gets a "Segmentation chunks" section. +
6. Catalog, search, Stats
+-
+
- Catalog.
Catalog_Entrygains: +-
+
segments: [..] Catalog_Segment: class, area and score for each segment, from the frontsegschunk keyed by the current model.
+ segment_classes: the hash, so stale results can be found.
+ find_segments: the class names, lowercased, one per line.
+
catalog_entry_freefrees them. The mask is never loaded into the catalog.
+ - Search. A new field,
has:(Search_Field.HAS), matching a whole class name:has:dog,has:"traffic light". +-
+
- Plain words do not match segment classes, as with
folder:, sodogkeeps meaning text and titles, andhas:dogmeans it is in the picture.
+ - Later: counts and sizes, such as
has:person:3+orhas:dog>20%. The front chunk already holds what they need.
+
+ - Plain words do not match segment classes, as with
- Stats. Add "Segmented: N of M images shown" and a top-classes list: how many captures contain each class. Clicking a class searches
has:class, as the top labels searchlabel:.
+
has: and label: answer different questions. Labels (SigLIP 2) describe the whole image against your own list ("Settings", "Installer", "Beach"). Segments say which COCO things are in it, and where. Both stay, and they can be combined: label:beach has:dog.7. Capture window overlay
+-
+
- A Segments row under the labels line, shown when the capture has
segsfrom the current model: +-
+
- A toggle and an opacity slider. +
- A legend: each class with its colour swatch and a checkbox to hide it. Clicking a class name searches
has:class.
+
+ - The mask texture. When the toggle is on, a viewer worker reads the capture's
maskchunk (mcap_read_without_original, which does not read the original). It turns the run-length map into RGBA, colour per class and alpha from the slider, at the map's resolution. The main thread uploads it withgpu_texture_create, as it does the full image. Hidden classes are drawn transparent; changing which are hidden or the opacity rebuilds the texture on the worker, at most a few times a second.
+ - Drawing. In
viewer_full_window, afterImage(...), next toocr_ui_capture_overlay:draw.AddImage(mask_texture, image_min, image_min + source * zoom). The GPU scales it up, and the map's aspect ratio matches the image.
+ - Hover. The pixel under the mouse is mapped into the map, and its segment's class and score are shown as a tooltip ("dog · 0.91 · 8% of the image"). +
- Colours. A fixed palette by class index, the same in every capture, so a class keeps its colour. Pick it for contrast in both themes. +
8. Phases
+Each phase ends with something to run and check, and is a commit (0.0.11 onwards).
+ +
+
+
+Phase 1: Export and check the model Done
+tools/export_mask2former.py exports Swin-B COCO panoptic. It is checked against transformers on 20 or more photos, with the IoU target above, and the timings and peak GPU memory are recorded at batch sizes 1, 2 and 4 on your GPU.
Done when: an .onnx that loads in onnxruntime 1.30 with CUDA and passes the IoU check, plus a note of the numbers.
+
+Phase 2:
+
+
+Phase 2: modules/Segment and the probe Done
+The module (create, segment, panoptic merge, cancel), models.config, the copy step in first.jai, and tools/segment_probe.jai. fetch_models.jai waits on where the model will be hosted (open question 2); until then tools/export_mask2former.py makes it.
Done when: segment-probe on the phase 1 photos writes masks that match phase 1's, and runs on both CPU and CUDA.
Result (2026-10-10).
+
+-
+
- Agreement: 99.8% of pixels have the same class as PyTorch's, on both CUDA and the CPU. Every segment PyTorch scores 0.6 or more is found. The test set is the same 25 images. +
- Differences: two people at 0.56 and 0.58 are missed, and there are two extra segments. These are borderline cases, tipped by the input being a step or two (out of 255) off PyTorch's, because stb_image decodes JPEGs slightly differently from libjpeg. +
- Release build, RTX 5080: 57 ms per image at batch 4: preparing 3 ms, the model 21 ms, making segments 30 ms. Peak GPU memory is 2.4 GB. Batch 8 is no faster; batch 1 is 38 ms for the model. On the CPU, about 670 ms per image. +
Things learned on the way:
+-
+
- The score threshold. transformers 5.19's default is 0.5, not the 0.8 this plan assumed, and the check compares with that default. It's
score_thresholdinmodels.config, to raise if weak segments prove noisy in search.
+ - Stuff isn't merged per class by default, so two patches of sky can be two segments. +
- Making segments now takes longer than the model. Spreading it across threads can roughly double throughput if the scan needs it. +
- Overlay colours. The overlay colours by class, so neighbouring people share a colour. Phase 6 should vary the shade per instance. +
+
+
+Phase 3: Chunks and storing
+segment_chunks.jai (text, RLE), mcap_store with back chunks, the README section, and tests:
-
+
- Round trip. +
- RLE edge cases: one segment covering everything, a map with no segments, runs wrapping across rows, the largest id. +
- Storing
segsandmasktogether leaves the original and every other chunk as they were.
+
Done when: the tests pass and a probe run can store its results into a scratch library's files.
+
+
+
+Phase 4: Per-folder opt-in and the scan
+The pieces:
+-
+
.library-folders(load, save, keep unknown keys).
+ - The Segmentation window's folder table. +
- The Import window's checkbox. +
- The
SEGMENTtask and its item selection.
+ - The import follow-up scan. +
-segmentand-erase segments.
+ - The shutdown status line. +
Done when: on a scratch library with two folders, only the opted-in one is segmented. Videos in it are skipped. Cancelling mid-batch stores nothing. Erase per folder works.
+
+Phase 5: Catalog,
+
+
+Phase 5: Catalog, has: and Stats
+Done when: has:person finds the right captures in the scratch library, catalog tests cover the new field, and Stats shows the top classes, each clickable.
+
+
+Phase 6: The overlay
+The mask texture on a worker, the legend, hiding classes, opacity and hover.
+Done when: masks line up with the image at every fit and zoom, including on Snagit captures (rendered image) and on photos turned by their EXIF orientation.
+9. Open questions
+-
+
- COCO panoptic or ADE20K? COCO counts things (people, pets, cars); ADE20K knows more indoor objects and scene parts, but has no instances. The default is COCO. Revisit after phase 1 by trying both on a sample of your photo libraries. +
- Where to host the exported .onnx for
fetch_models.jai: a private Hugging Face repo, a release asset, or committing it as the FFmpeg DLLs are. At about 400 MB for Swin-B, committing it is probably too much.
+ - The mask resolution cap. 1024 on the long side is a guess. Phase 3 should measure chunk sizes on real photos at 512, 1024 and 2048. +
- Snagit and screenshot folders. They stay off by default, and COCO classes mean little on UI. Turning one on is allowed, but the Segmentation window could warn that most of the folder is screenshots, judged from
FormatandAppName.
+ - Re-scanning on a model change. A new model name means a new key, so old results stay until erased. Should the scan offer to erase other models' masks to save space? Probably yes, as an explicit button rather than automatically. +
10. Later: video frames Shelved
+Treat strategically sampled frames of a video as images, and run the same analysis (segmentation, and OCR and labels too) on them. Notes for when it comes back:
+-
+
- Sampling: scene changes rather than a fixed rate. Compare small thumbnails of successive decoded frames, or use FFmpeg's scene score. Cap the frames per minute. +
- Storage: keep the frames themselves out of the .mcap except as small thumbnails. Store per-frame results in chunks keyed by model and frame time (for example a key like
mask2former…@83.40), or in one chunk holding every frame's results.
+ - The per-folder option would grow a second key,
segment_videos, in the same.library-foldersfile.
+