Musa Mahmood
cb21b73460
- segmentation-plan.html: the plan for panoptic segmentation of images, per library folder, stored
in .mcap chunks (no file version bump), in six phases.
- tools/export_mask2former.py: exports facebook/mask2former-swin-base-coco-panoptic to ONNX (opset 17,
dynamic batch, 384 x 384 input) and checks it against PyTorch: on 25 images, all 180 segments
identical (IoU 1.000). Writes the class list with names for people ("sky-other-merged" is "sky"),
and PyTorch's own segmentations for segment-probe -check.
- modules/Segment: runs the model with onnxruntime (CUDA, else the CPU) and makes segments of its
outputs as transformers' post_process_panoptic_segmentation does: a segment id map (longest side at
most 1024) and each segment's class, score, area and bounds. Matches PyTorch on 99.8% of pixels,
every segment scored 0.6 or more found. 57 ms an image at batch 4 on an RTX 5080 laptop.
- tools/segment_probe.jai (jai first.jai - segment_probe): segments image files; -out draws them,
-check compares with PyTorch, -dump-input writes the model's input.
- modules/onnxruntime: ort_run_float_outputs, for a model with several outputs.
- first.jai copies the model to bin/segment-models. The model itself is not in git
(modules/Segment/models, 462 MB); tools/export_mask2former.py makes it.
19 lines
1011 B
Plaintext
19 lines
1011 B
Plaintext
@version 1
|
|
|
|
# Panoptic segmentation models (modules/Segment). Each file named here is in the models folder: made
|
|
# by tools/export_mask2former.py, which also checks the .onnx against PyTorch.
|
|
|
|
[model mask2former-swin-base-coco-panoptic]
|
|
title: "Mask2Former Swin-B, COCO panoptic (133 classes)"
|
|
model: "mask2former-swin-base-coco-panoptic.onnx"
|
|
classes: "mask2former-swin-base-coco-panoptic.classes.txt"
|
|
image_size: 384 # squashed to 384 x 384, not keeping its shape, as the checkpoint's processor does
|
|
mean: 0.485 0.456 0.406
|
|
std: 0.229 0.224 0.225
|
|
batch_gpu: 4
|
|
batch_cpu: 2 # the fastest per image on the CPU (tools/export_mask2former.py's check)
|
|
score_threshold: 0.5 # transformers 5.19's post_process_panoptic_segmentation defaults, which the check compares with
|
|
mask_threshold: 0.5
|
|
overlap_threshold: 0.8
|
|
map_max_side: 1024 # the stored segment map's longest side
|