- The Capture window draws a segmented image's segments over it: each class in its colour, a shade of it for each instance, edges darker; under the OCR boxes, at any fit and zoom. A viewer worker reads the segs and mask chunks (never the images); the main thread makes the texture from the mask when the capture, the opacity or the classes hidden change. - Above the image: Segments on or off, the opacity, and a legend of the classes (largest first, with counts): untick one to hide it, right click it to search has: for it. Hovering the image shows the segment's class, score and share of the image. - Colours: a class's own is a hue from its name's hash. Its legend swatch is a colour picker (ImGui ColorEdit3) to choose another, kept in %APPDATA%\mcap\segment-colours.config for every image and library; right click it to go back to its own. - Tests: the overlay's pixels (shades, edges, hidden classes, opacity), reading a file's segments for it, and chosen colours. - This completes segmentation-plan.html's six phases.
518 lines
43 KiB
HTML
518 lines
43 KiB
HTML
<!doctype html>
|
||
<html lang="en">
|
||
<head>
|
||
<meta charset="utf-8">
|
||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||
<title>Segmentation Plan</title>
|
||
<style>
|
||
:root {
|
||
--bg: #fbfaf8; --surface: #ffffff; --text: #1d1d1f; --dim: #5d5d66; --line: #e3e1dc;
|
||
--accent: #3a5bd9; --accent-soft: #e9edfb; --warn: #9a5b00; --warn-soft: #fbf1e1;
|
||
--code-bg: #f2f0ec; --ok: #2e7d4f; --ok-soft: #e5f3ea;
|
||
}
|
||
@media (prefers-color-scheme: dark) {
|
||
:root:not([data-theme="light"]) {
|
||
--bg: #16171a; --surface: #1d1f23; --text: #e7e7ea; --dim: #a2a3ab; --line: #30333a;
|
||
--accent: #8aa2ff; --accent-soft: #232a45; --warn: #f0b45a; --warn-soft: #33281a;
|
||
--code-bg: #25272c; --ok: #6fcf97; --ok-soft: #1c2e24;
|
||
}
|
||
}
|
||
:root[data-theme="dark"] {
|
||
--bg: #16171a; --surface: #1d1f23; --text: #e7e7ea; --dim: #a2a3ab; --line: #30333a;
|
||
--accent: #8aa2ff; --accent-soft: #232a45; --warn: #f0b45a; --warn-soft: #33281a;
|
||
--code-bg: #25272c; --ok: #6fcf97; --ok-soft: #1c2e24;
|
||
}
|
||
* { box-sizing: border-box; }
|
||
body {
|
||
margin: 0; background: var(--bg); color: var(--text);
|
||
font: 16px/1.6 system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
|
||
}
|
||
main { max-width: 860px; margin: 0 auto; padding: 40px 16px 80px; }
|
||
header p { color: var(--dim); margin-top: 4px; }
|
||
h1 { font-size: 2rem; line-height: 1.2; margin: 0; }
|
||
h2 { font-size: 1.35rem; margin: 48px 0 8px; padding-top: 16px; border-top: 1px solid var(--line); }
|
||
h3 { font-size: 1.05rem; margin: 28px 0 6px; }
|
||
p, li { max-width: 72ch; }
|
||
a { color: var(--accent); }
|
||
code, pre { font-family: ui-monospace, "Cascadia Code", Consolas, monospace; font-size: 0.88em; }
|
||
code { background: var(--code-bg); padding: 1px 5px; border-radius: 4px; }
|
||
pre { background: var(--code-bg); padding: 12px 14px; border-radius: 8px; overflow-x: auto; line-height: 1.45; }
|
||
pre code { background: none; padding: 0; }
|
||
table { border-collapse: collapse; width: 100%; margin: 12px 0; font-size: 0.94rem; }
|
||
th, td { text-align: left; padding: 7px 10px; border-bottom: 1px solid var(--line); vertical-align: top; }
|
||
th { color: var(--dim); font-weight: 600; }
|
||
.table-wrap { overflow-x: auto; }
|
||
.summary { background: var(--surface); border: 1px solid var(--line); border-radius: 10px; padding: 16px 20px; margin-top: 24px; }
|
||
.summary ul { margin: 6px 0 0; padding-left: 20px; }
|
||
.note { background: var(--accent-soft); border-radius: 8px; padding: 10px 14px; margin: 14px 0; }
|
||
.risk { background: var(--warn-soft); border-radius: 8px; padding: 10px 14px; margin: 14px 0; }
|
||
.risk strong { color: var(--warn); }
|
||
.tag { display: inline-block; font-size: 0.78rem; font-weight: 600; padding: 1px 8px; border-radius: 99px; background: var(--accent-soft); color: var(--accent); }
|
||
.tag.later { background: var(--code-bg); color: var(--dim); }
|
||
.tag.decided { background: var(--ok-soft); color: var(--ok); }
|
||
nav ol { columns: 2; column-gap: 32px; padding-left: 20px; margin: 8px 0 0; }
|
||
@media (max-width: 600px) { nav ol { columns: 1; } h1 { font-size: 1.6rem; } }
|
||
.phase { border-left: 3px solid var(--accent); padding: 2px 0 2px 14px; margin: 16px 0; }
|
||
.phase h3 { margin-top: 0; }
|
||
figure { margin: 18px 0; }
|
||
figcaption { color: var(--dim); font-size: 0.88rem; margin-top: 6px; }
|
||
svg text { fill: var(--text); font: 13px system-ui, sans-serif; }
|
||
svg .dim { fill: var(--dim); }
|
||
svg .box { fill: var(--surface); stroke: var(--line); }
|
||
svg .hl { fill: var(--accent-soft); stroke: var(--accent); }
|
||
svg .arrow { stroke: var(--dim); fill: none; marker-end: url(#a); }
|
||
</style>
|
||
</head>
|
||
<body>
|
||
<main>
|
||
|
||
<header>
|
||
<h1>Semantic segmentation in mcap</h1>
|
||
<p>Planning document · 2026-10-10 · for mcap after 0.0.10 · <strong>all six phases built, mcap 0.0.15</strong></p>
|
||
</header>
|
||
|
||
<div class="summary">
|
||
<strong>In short</strong>
|
||
<ul>
|
||
<li><span class="tag decided">Decided</span> <strong>Mask2Former</strong>, exported to ONNX and run with the onnxruntime mcap already ships. It is swappable later, because results are keyed by model name.</li>
|
||
<li><span class="tag decided">Decided</span> <strong>Images only</strong>, and only in the <strong>library folders you opt in</strong>. The setting is per folder, kept in a file in the library root, so it travels with the library.</li>
|
||
<li><span class="tag decided">Decided</span> <strong>No .mcap version bump.</strong> Results go in two new chunk kinds: a small <code>segs</code> summary at the front for search, and a <code>mask</code> at the back for drawing.</li>
|
||
<li>It is built in six phases. The first proves the ONNX export matches PyTorch before anything else depends on it.</li>
|
||
<li><span class="tag decided">Done</span> <strong>All six phases are built</strong> (2026-10-10, mcap 0.0.11 to 0.0.15). See <a href="#state">Where it stands</a> for what was built, the commits, and what is left.</li>
|
||
<li><span class="tag later">Later</span> Video, by treating sampled frames as images, is shelved.</li>
|
||
</ul>
|
||
</div>
|
||
|
||
<nav>
|
||
<ol>
|
||
<li><a href="#scope">Scope</a></li>
|
||
<li><a href="#model">The model</a></li>
|
||
<li><a href="#folders">Per-folder opt-in</a></li>
|
||
<li><a href="#scan">The segmentation scan</a></li>
|
||
<li><a href="#storage">Storage in .mcap</a></li>
|
||
<li><a href="#search">Catalog, search, Stats</a></li>
|
||
<li><a href="#overlay">Capture window overlay</a></li>
|
||
<li><a href="#phases">Phases</a></li>
|
||
<li><a href="#state">Where it stands</a></li>
|
||
<li><a href="#questions">Open questions</a></li>
|
||
<li><a href="#later">Later: video frames</a></li>
|
||
</ol>
|
||
</nav>
|
||
|
||
<h2 id="scope">1. Scope</h2>
|
||
<p>Each opted-in image gets a <em>panoptic</em> segmentation. Every pixel gets a class ("person", "dog", "sky", "wall"), and countable things are split into instances: two people are two segments. The results serve three purposes:</p>
|
||
<ul>
|
||
<li><strong>Search:</strong> <code>has:dog</code>, <code>has:person</code>, and combined with the other search terms.</li>
|
||
<li><strong>Stats:</strong> which things the shown captures hold most.</li>
|
||
<li><strong>Looking:</strong> coloured masks over the image in the Capture window, with the class under the mouse.</li>
|
||
</ul>
|
||
<p><strong>What counts as an image.</strong> Imported photos and screenshots, Snagit captures and animated GIFs (their first frame) qualify. Videos are skipped even in opted-in folders: a video's poster frame is an image technically, but frames are the <a href="#later">later</a> project. A capture is a video when its catalog entry's <code>media</code> is <code>VIDEO</code>, which comes from <code>VideoCodec</code> in its <code>import.json</code>.</p>
|
||
|
||
<h2 id="model">2. The model</h2>
|
||
<h3>Which Mask2Former</h3>
|
||
<div class="table-wrap">
|
||
<table>
|
||
<tr><th>Checkpoint (Hugging Face)</th><th>Classes</th><th>Size</th><th>Use</th></tr>
|
||
<tr><td><code>facebook/mask2former-swin-base-coco-panoptic</code></td><td>133 COCO panoptic: 80 things, 53 stuff</td><td>~107M</td><td><strong>Default.</strong> Things are counted as instances, and the stuff classes (sky, wall, grass, road) cover the rest.</td></tr>
|
||
<tr><td><code>facebook/mask2former-swin-large-coco-panoptic</code></td><td>same</td><td>~216M</td><td>If base misses too much and the GPU can take it.</td></tr>
|
||
<tr><td><code>facebook/mask2former-swin-tiny-coco-panoptic</code></td><td>same</td><td>~47M</td><td>The CPU fallback, when CUDA is unavailable.</td></tr>
|
||
<tr><td><code>facebook/mask2former-swin-base-ade-semantic</code></td><td>150 ADE20K, no instances</td><td>~107M</td><td>The alternative, if indoor scenes and objects matter more than counting people. See <a href="#questions">open questions</a>.</td></tr>
|
||
</table>
|
||
</div>
|
||
<p>COCO panoptic is the default because a photo library's questions are mostly about things ("photos with the dog", "group shots"), and only the panoptic models separate instances.</p>
|
||
|
||
<h3>Export to ONNX</h3>
|
||
<p>There is no official ONNX release, so phase 1 exports one with a Python script kept in the repo (<code>tools/export_mask2former.py</code>). It runs once on a developer machine and is never shipped.</p>
|
||
<ul>
|
||
<li><strong>Deformable attention.</strong> The pixel decoder uses multi-scale deformable attention. Transformers has a pure-PyTorch version built on <code>grid_sample</code>, which exports at opset 17 or later. The script forces that path.</li>
|
||
<li><strong>Outputs.</strong> The script exports up to <code>class_queries_logits</code> [B, 100, 134] and <code>masks_queries_logits</code> [B, 100, H/4, W/4]. The panoptic merge (below) is done in Jai, as it is short, and keeping it out of the graph keeps the export simple.</li>
|
||
<li><strong>Input.</strong> <code>pixel_values</code> [batch, 3, 384, 384]: RGB <em>squashed</em> to 384 × 384, ignoring the aspect ratio, as this checkpoint's processor does (bilinear), scaled to 0..1, then ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225). The batch size is dynamic; the size is fixed.</li>
|
||
<li><strong>Check.</strong> On 20 or more images, compare the ONNX output, run through mcap's post-processing, with transformers' <code>post_process_panoptic_segmentation</code>. Per-segment IoU should be at least 0.98.</li>
|
||
</ul>
|
||
<div class="note"><strong>Phase 1 result (2026-10-10): passed.</strong>
|
||
<ul>
|
||
<li><strong>Test set:</strong> 25 images: 16 COCO val2017 photos, 5 of your photos and 4 video frames.</li>
|
||
<li><strong>Accuracy:</strong> 180 segments, every one with an IoU of 1.000 against PyTorch, none missing and none extra.</li>
|
||
<li><strong>Batching:</strong> batches of 2, 4 and 8 give bit-identical results to one at a time.</li>
|
||
<li><strong>Speed:</strong> CPU 348 ms per image at batch 1, 290 ms at batch 2.</li>
|
||
<li><strong>Size:</strong> 462 MB at opset 17.</li>
|
||
<li><strong>Load warning:</strong> the "missing" <code>swin.layernorm</code> that transformers 5.19 reports when loading is unused; randomizing it changes no output.</li>
|
||
<li><strong>Not yet measured:</strong> GPU timing and memory. Those come in phase 2, with mcap's own onnxruntime build.</li>
|
||
</ul>
|
||
Script: <code>tools/export_mask2former.py</code>.</div>
|
||
<div class="risk"><strong>Risk.</strong> The export is the one part of this plan that may fail outright. If <code>grid_sample</code> export misbehaves, fall back to the ADE20K semantic SegFormer, which exports trivially, to build phases 2 to 6. Then return to Mask2Former, since everything after phase 1 is keyed by model name.</div>
|
||
|
||
<h3>Packaging, as Labels and OCR are</h3>
|
||
<ul>
|
||
<li>A new module, <code>modules/Segment</code>, laid out like <code>modules/Labels</code>.
|
||
<ul>
|
||
<li><code>models.config</code>: <code>@version 1</code>, then <code>[model mask2former-swin-base-coco-panoptic]</code> with <code>title</code>, <code>model</code> (the .onnx file name), <code>classes</code> (a text file with one class name per line, plus whether each is a "thing"), <code>input_size</code>, <code>batch_gpu</code>, <code>batch_cpu</code>, <code>score_threshold</code> (0.8) and <code>overlap_threshold</code> (0.8).</li>
|
||
<li><code>fetch_models.jai</code>: the files, their SHA-256 hashes, and where they come from. The exported .onnx has to be hosted somewhere you control, such as a Hugging Face repo or a release asset. See <a href="#questions">open questions</a>.</li>
|
||
<li><code>first.jai</code> copies the model folder to <code>bin/segment-models</code>, as it does <code>bin/label-models</code>.</li>
|
||
</ul>
|
||
</li>
|
||
<li><code>segment_create</code> and <code>segment_destroy</code> follow <code>labels_create</code>. That means <code>ort_cuda_check</code>, a GPU session made under <code>ort_gpu_lock</code>, and a CPU fallback with the tiny model.</li>
|
||
<li><code>segment_images(engine, images, cancel) -> [] Segment_Result</code>, given RGBA in, as <code>Labels_Image</code> and <code>Ocr_Image</code> are:
|
||
<ol>
|
||
<li>Squash to 384 × 384 and normalise, as above.</li>
|
||
<li>Run in batches.</li>
|
||
<li>The panoptic merge, as transformers 5.19 does it:
|
||
<ol>
|
||
<li>Softmax over the classes, and sigmoid over the masks scaled up to 384, then to the map.</li>
|
||
<li>Drop queries scoring 0.5 or under, or whose class is "no object".</li>
|
||
<li>Weight each mask by its query's score, and assign each pixel to its best query.</li>
|
||
<li>Drop a segment if it keeps 80% or less of its own mask.</li>
|
||
</ol>
|
||
Stuff of one class is not merged.</li>
|
||
<li>Result: a segment-id map at the mask resolution, plus a list of segments (id, class, score, area, box).</li>
|
||
</ol>
|
||
It checks <code>cancel</code> between batches, as <code>ocr_process</code> now does.</li>
|
||
<li><code>tools/segment_probe.jai</code>, like <code>image-probe</code>: segment some files, print their segments, and write the masks out as PNGs to look at.</li>
|
||
</ul>
|
||
|
||
<h2 id="folders">3. Per-folder opt-in</h2>
|
||
<p>A library folder is a top-level folder of the library, one per place imported from (<code>SnagitLibraryWork</code>, <code>Phone</code>, <code>Photos 2014</code>). Segmentation runs only in the folders you turn it on for. A new folder starts with it off, so adding a 40,000-photo archive never starts an hours-long GPU job by surprise.</p>
|
||
|
||
<h3>Where the setting lives</h3>
|
||
<p>It goes in a new file in the library root, <code>.library-folders</code>, next to <code>.import-index</code> and <code>.import-sources</code>. Not in <code>%APPDATA%\mcap\settings.config</code>, because the choice belongs to the library, not to the machine. It follows the library when it is opened elsewhere, or with <code>-library FOLDER</code>. The format follows the Ex4 style used in the rest of mcap, so later per-folder options (OCR off for a photos folder, say) go in the same file:</p>
|
||
<pre><code>@version 1
|
||
|
||
[folder "Photos 2014"]
|
||
segment: true
|
||
|
||
[folder "SnagitLibraryWork"]
|
||
segment: false</code></pre>
|
||
<ul>
|
||
<li><strong>Missing file or folder:</strong> every option is off. Unknown keys are kept on rewrite, as unknown chunks are.</li>
|
||
<li><strong>Who reads and writes it:</strong> <code>library_folders_load</code> and <code>library_folders_save</code> in a new <code>src/app/library_folders.jai</code>, on the main thread, written beside itself and swapped in, as the import index is. It lives in a table keyed by folder name.</li>
|
||
<li><strong>A renamed folder</strong> loses its setting. The Segmentation window lists settings for folders that no longer exist, with a button to remove them.</li>
|
||
</ul>
|
||
|
||
<h3>Where you set it</h3>
|
||
<ul>
|
||
<li><strong>The Segmentation window</strong> (a new button next to Labels) has a table of library folders with these columns:
|
||
<ul>
|
||
<li><strong>Segment</strong>: the checkbox.</li>
|
||
<li><strong>Images</strong>: the count, excluding videos.</li>
|
||
<li><strong>Segmented</strong>: how many are done with the current model.</li>
|
||
<li><strong>Erase</strong>: a per-folder button.</li>
|
||
</ul>
|
||
Turning a folder off stops future scans but erases nothing.</li>
|
||
<li><strong>The Import window</strong> adds a checkbox, "Segment the images in this folder". It shows and sets the destination folder's option. For a new folder it starts off; for an existing folder it shows that folder's setting.</li>
|
||
<li><strong>The folder combo above the grid</strong> marks opted-in folders, for example with a small "seg" after the name. This is a nicety; it can come last.</li>
|
||
</ul>
|
||
|
||
<h2 id="scan">4. The segmentation scan</h2>
|
||
<figure>
|
||
<svg viewBox="0 0 820 190" width="100%" role="img" aria-label="Scan flow: opted-in folders give image items, which are decoded once, segmented in batches, then stored and handed to the catalog">
|
||
<defs><marker id="a" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto"><path d="M0,0 L10,5 L0,10 z" fill="currentColor" style="color: var(--dim)"/></marker></defs>
|
||
<rect class="box" x="10" y="30" width="170" height="70" rx="8"/>
|
||
<text x="95" y="60" text-anchor="middle">.library-folders</text>
|
||
<text x="95" y="80" text-anchor="middle" class="dim">segment: true</text>
|
||
<rect class="box" x="220" y="30" width="170" height="70" rx="8"/>
|
||
<text x="305" y="58" text-anchor="middle">Items</text>
|
||
<text x="305" y="78" text-anchor="middle" class="dim">images not yet done</text>
|
||
<rect class="hl" x="430" y="30" width="170" height="70" rx="8"/>
|
||
<text x="515" y="58" text-anchor="middle">Decode, segment</text>
|
||
<text x="515" y="78" text-anchor="middle" class="dim">batches on the scan thread</text>
|
||
<rect class="box" x="640" y="30" width="170" height="70" rx="8"/>
|
||
<text x="725" y="58" text-anchor="middle">mcap_store</text>
|
||
<text x="725" y="78" text-anchor="middle" class="dim">segs front, mask back</text>
|
||
<path class="arrow" d="M180 65 H216"/>
|
||
<path class="arrow" d="M390 65 H426"/>
|
||
<path class="arrow" d="M600 65 H636"/>
|
||
<path class="arrow" d="M725 100 V140 H520"/>
|
||
<text x="520" y="160" text-anchor="middle" class="dim">catalog_load → viewer.catalog_updates → search and Stats</text>
|
||
</svg>
|
||
<figcaption>Only the scan's choice of captures is new. Batching, pause and cancel, storing and the catalog hand-off are the OCR scan's.</figcaption>
|
||
</figure>
|
||
|
||
<ul>
|
||
<li><strong>The thread.</strong> It runs on the existing scan thread (the "ocr" thread in <code>ocr_ui.jai</code>), as a new task kind <code>SEGMENT</code>. One thread doing GPU work at a time is already the rule there, and it brings pause, cancel, the progress bar and <code>-scan</code>-style headless runs for free.</li>
|
||
<li><strong>Which captures.</strong> Every capture in an opted-in folder whose media is not <code>VIDEO</code> and has no <code>segs</code> chunk from the current model. Unlike the OCR scan, it ignores the grid's folder filter and search: the scope is the folder settings. A "Segment the captures shown" button can be added later if wanted.</li>
|
||
<li><strong>Batches.</strong> Batches of <code>SEGMENT_SCAN_BATCH</code> (4 to start, since Mask2Former is heavier per image than OCR). Each image is decoded once with <code>mcap_read_shown_image</code>, so a Snagit capture uses its rendered image. Cancel is checked between batches and inside <code>segment_images</code>; a cut-short batch stores nothing, as for OCR.</li>
|
||
<li><strong>After an import.</strong> When "Read text and labels" is on and the destination folder segments, the import's follow-up scan becomes OCR and labels, then segmentation of the new images. These are two tasks queued one after the other, not one combined pass. They write different chunks, and keeping them apart keeps each simple.</li>
|
||
<li><strong>Headless.</strong> <code>-segment</code> segments every opted-in folder and exits, 0 or 1, as <code>-scan</code> does.</li>
|
||
<li><strong>Erase.</strong> <code>-erase segments</code>, and <code>-erase all</code> now includes it. Add <code>Erase_What.SEGMENTS</code>, its tags (<code>segs</code>, <code>mask</code>) and its "has" test. Per-folder erase in the Segmentation window uses the same task, limited to that folder's captures.</li>
|
||
<li><strong>Shutdown.</strong> It is the same thread, so the 0.0.10 shutdown status covers it. Add a "Stopping the segmentation scan" line.</li>
|
||
</ul>
|
||
|
||
<h2 id="storage">5. Storage in .mcap</h2>
|
||
<div class="note">No file version bump. Version 3 added tagged, model-keyed chunk lists, and the format README already names segmentation masks as what back chunks are for. A reader that does not know a chunk keeps it on rewrite. Each new chunk carries its own internal version, as <code>labl</code> does.</div>
|
||
|
||
<h3><code>segs</code>, front chunk, keyed by model name</h3>
|
||
<p>This is what search and Stats need, small enough for the front that every library browse reads. It is text in the style of <code>labl</code>:</p>
|
||
<pre><code>@version 1
|
||
engine: "onnxruntime 1.30.0, CUDA"
|
||
date: 2026-10-12T09:30:00Z
|
||
image: rendered 4032 3024
|
||
classes: 5c1e9a0d77b2f413
|
||
segment: 1 0.981 0.2140 1210 380 900 2210 "person"
|
||
segment: 2 0.955 0.1022 2300 900 700 1400 "person"
|
||
segment: 3 0.912 0.0811 400 1900 800 700 "dog"
|
||
segment: 4 0.993 0.3105 0 0 4032 1100 "sky"</code></pre>
|
||
<div class="table-wrap">
|
||
<table>
|
||
<tr><th>Key</th><th>Value</th></tr>
|
||
<tr><td><code>engine</code>, <code>date</code></td><td>As in OCR text.</td></tr>
|
||
<tr><td><code>image</code></td><td>Which image was read (rendered or original) and its size. Boxes are in its pixels.</td></tr>
|
||
<tr><td><code>classes</code></td><td>A 64-bit FNV-1a hash of the model's class list, so a model with new classes is seen as needing a new scan.</td></tr>
|
||
<tr><td><code>segment</code></td><td>The segment's id in the mask, its score, its area (as a fraction of the image), its box (x y w h), and its class, quoted. Largest first. Stuff classes have one segment per class.</td></tr>
|
||
</table>
|
||
</div>
|
||
<p>A busy photo has 10 to 30 segments, about 1 to 2 KB.</p>
|
||
|
||
<h3><code>mask</code>, back chunk, keyed by model name</h3>
|
||
<p>This is the segment-id map, for drawing only. It is binary:</p>
|
||
<pre><code>u8 format 1: run-length encoded, row-major
|
||
u16 width the map's size: the image scaled so its longest side is at most 1024
|
||
u16 height
|
||
u8 id_bits 8 (up to 255 segments), or 16 later if needed
|
||
then runs: (id, length) pairs over the whole map, row by row, wrapping across rows
|
||
id as u8; length as a LEB128 varint
|
||
id 0: no segment (below threshold, or "no object")</code></pre>
|
||
<ul>
|
||
<li><strong>Why run-length encoding:</strong> segment maps are large flat regions, so runs are few. It decodes in one pass with no dependencies, and an expected 5 to 40 KB per photo is a few percent of a typical JPEG.</li>
|
||
<li><strong>The alternative:</strong> a palette PNG through stb_image_write. It is smaller on noisy maps but encodes and decodes more slowly, and RLE is simpler to test. Revisit only if mask chunks turn out large.</li>
|
||
<li><strong>Why at most 1024 on the long side:</strong> it is plenty for an overlay, and the model's own output is at a quarter of its input size, so storing more would invent detail. The overlay scales it up.</li>
|
||
</ul>
|
||
|
||
<h3>Code changes</h3>
|
||
<ul>
|
||
<li><code>modules/Capture_File/segment_chunks.jai</code>, next to <code>label_chunks.jai</code>:
|
||
<ul>
|
||
<li>The tags <code>MCAP_SEGMENTS_TAG "segs"</code> and <code>MCAP_MASK_TAG "mask"</code>.</li>
|
||
<li>The text read and write functions, <code>mcap_segments_text</code> and <code>mcap_segments_parse</code>.</li>
|
||
<li>The mask functions, <code>mcap_mask_bytes</code> and <code>mcap_mask_parse</code>.</li>
|
||
</ul></li>
|
||
<li><code>mcap_store</code> gains a <code>back: [] Mcap_Chunk</code> parameter, so <code>segs</code> and <code>mask</code> are written in <strong>one</strong> rewrite. Today a back chunk needs its own call (<code>mcap_store_chunk(..., back = true)</code>), which would rewrite the file twice and could leave one without the other.</li>
|
||
<li><code>mcap_read_without_original</code> already reads the back chunks, so the overlay needs no new reader.</li>
|
||
<li>The format README gets a "Segmentation chunks" section.</li>
|
||
</ul>
|
||
|
||
<h2 id="search">6. Catalog, search, Stats</h2>
|
||
<ul>
|
||
<li><strong>Catalog.</strong> <code>Catalog_Entry</code> gains:
|
||
<ul>
|
||
<li><code>segments: [..] Catalog_Segment</code>: class, area and score for each segment, from the front <code>segs</code> chunk keyed by the current model.</li>
|
||
<li><code>segment_classes</code>: the hash, so stale results can be found.</li>
|
||
<li><code>find_segments</code>: the class names, lowercased, one per line.</li>
|
||
</ul>
|
||
<code>catalog_entry_free</code> frees them. The mask is never loaded into the catalog.</li>
|
||
<li><strong>Search.</strong> A new field, <code>has:</code> (<code>Search_Field.HAS</code>), matching a whole class name: <code>has:dog</code>, <code>has:"traffic light"</code>.
|
||
<ul>
|
||
<li>Plain words do not match segment classes, as with <code>folder:</code>, so <code>dog</code> keeps meaning text and titles, and <code>has:dog</code> means it is in the picture.</li>
|
||
<li>Later: counts and sizes, such as <code>has:person:3+</code> or <code>has:dog>20%</code>. The front chunk already holds what they need.</li>
|
||
</ul></li>
|
||
<li><strong>Stats.</strong> Add "Segmented: N of M images shown" and a top-classes list: how many captures contain each class. Clicking a class searches <code>has:class</code>, as the top labels search <code>label:</code>.</li>
|
||
</ul>
|
||
<div class="note"><code>has:</code> and <code>label:</code> answer different questions. Labels (SigLIP 2) describe the whole image against your own list ("Settings", "Installer", "Beach"). Segments say which COCO things are in it, and where. Both stay, and they can be combined: <code>label:beach has:dog</code>.</div>
|
||
|
||
<h2 id="overlay">7. Capture window overlay</h2>
|
||
<ul>
|
||
<li>A <strong>Segments</strong> row under the labels line, shown when the capture has <code>segs</code> from the current model:
|
||
<ul>
|
||
<li>A toggle and an opacity slider.</li>
|
||
<li>A legend: each class with its colour swatch and a checkbox to hide it. Clicking a class name searches <code>has:class</code>.</li>
|
||
</ul></li>
|
||
<li><strong>The mask texture.</strong> When the toggle is on, a viewer worker reads the capture's <code>mask</code> chunk (<code>mcap_read_without_original</code>, which does not read the original). It turns the run-length map into RGBA, colour per class and alpha from the slider, at the map's resolution. The main thread uploads it with <code>gpu_texture_create</code>, as it does the full image. Hidden classes are drawn transparent; changing which are hidden or the opacity rebuilds the texture on the worker, at most a few times a second.</li>
|
||
<li><strong>Drawing.</strong> In <code>viewer_full_window</code>, after <code>Image(...)</code>, next to <code>ocr_ui_capture_overlay</code>: <code>draw.AddImage(mask_texture, image_min, image_min + source * zoom)</code>. The GPU scales it up, and the map's aspect ratio matches the image.</li>
|
||
<li><strong>Hover.</strong> The pixel under the mouse is mapped into the map, and its segment's class and score are shown as a tooltip ("dog · 0.91 · 8% of the image").</li>
|
||
<li><strong>Colours.</strong> A fixed palette by class index, the same in every capture, so a class keeps its colour. Pick it for contrast in both themes.</li>
|
||
</ul>
|
||
|
||
<h2 id="phases">8. Phases</h2>
|
||
<p>Each phase ends with something to run and check, and is a commit (0.0.11 onwards).</p>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 1: Export and check the model <span class="tag decided">Done</span></h3>
|
||
<p><code>tools/export_mask2former.py</code> exports Swin-B COCO panoptic. It is checked against transformers on 20 or more photos, with the IoU target above, and the timings and peak GPU memory are recorded at batch sizes 1, 2 and 4 on your GPU.</p>
|
||
<p><strong>Done when:</strong> an .onnx that loads in onnxruntime 1.30 with CUDA and passes the IoU check, plus a note of the numbers.</p>
|
||
</div>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 2: <code>modules/Segment</code> and the probe <span class="tag decided">Done</span></h3>
|
||
<p>The module (create, segment, panoptic merge, cancel), <code>models.config</code>, the copy step in <code>first.jai</code>, and <code>tools/segment_probe.jai</code>. <code>fetch_models.jai</code> waits on where the model will be hosted (open question 2); until then <code>tools/export_mask2former.py</code> makes it.</p>
|
||
<p><strong>Done when:</strong> <code>segment-probe</code> on the phase 1 photos writes masks that match phase 1's, and runs on both CPU and CUDA.</p>
|
||
<div class="note"><strong>Result (2026-10-10).</strong>
|
||
<ul>
|
||
<li><strong>Agreement:</strong> 99.8% of pixels have the same class as PyTorch's, on both CUDA and the CPU. Every segment PyTorch scores 0.6 or more is found. The test set is the same 25 images.</li>
|
||
<li><strong>Differences:</strong> two people at 0.56 and 0.58 are missed, and there are two extra segments. These are borderline cases, tipped by the input being a step or two (out of 255) off PyTorch's, because stb_image decodes JPEGs slightly differently from libjpeg.</li>
|
||
<li><strong>Release build, RTX 5080:</strong> 57 ms per image at batch 4: preparing 3 ms, the model 21 ms, making segments 30 ms. Peak GPU memory is 2.4 GB. Batch 8 is no faster; batch 1 is 38 ms for the model. On the CPU, about 670 ms per image.</li>
|
||
</ul>
|
||
<p>Things learned on the way:</p>
|
||
<ul>
|
||
<li><strong>The score threshold.</strong> transformers 5.19's default is 0.5, not the 0.8 this plan assumed, and the check compares with that default. It's <code>score_threshold</code> in <code>models.config</code>, to raise if weak segments prove noisy in search.</li>
|
||
<li><strong>Stuff isn't merged</strong> per class by default, so two patches of sky can be two segments.</li>
|
||
<li><strong>Making segments</strong> now takes longer than the model. Spreading it across threads can roughly double throughput if the scan needs it.</li>
|
||
<li><strong>Overlay colours.</strong> The overlay colours by class, so neighbouring people share a colour. Phase 6 should vary the shade per instance.</li>
|
||
</ul></div>
|
||
</div>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 3: Chunks and storing <span class="tag decided">Done</span></h3>
|
||
<p><code>segment_chunks.jai</code> (text, RLE), <code>mcap_store</code> with back chunks, the README section, and tests:</p>
|
||
<ul>
|
||
<li>Round trip.</li>
|
||
<li>RLE edge cases: one segment covering everything, a map with no segments, runs wrapping across rows, the largest id.</li>
|
||
<li>Storing <code>segs</code> and <code>mask</code> together leaves the original and every other chunk as they were.</li>
|
||
</ul>
|
||
<p><strong>Done when:</strong> the tests pass and a probe run can store its results into a scratch library's files.</p>
|
||
<div class="note"><strong>Result (2026-10-10).</strong>
|
||
<ul>
|
||
<li><strong>Tests:</strong> 24 new checks pass, in the import suite's "segment chunks" group.</li>
|
||
<li><strong>A real library:</strong> on a scratch library of 25 photos and frames plus 6 imported videos and GIFs, <code>segment-probe -store</code> stored <code>segs</code> and <code>mask</code> in all 27 images, and skipped the 4 videos. A second run reads every one back.</li>
|
||
<li><strong>Size:</strong> a mask is 0.8 to 6.4 KB and the <code>segs</code> text 200 to 400 bytes, so the whole library grew by 141 KB. Open question 3 is answered: 1024 on the long side is cheap.</li>
|
||
<li><strong>For phase 4:</strong> the conversion from a <code>Segment_Result</code> to the two chunks (<code>segment_chunks</code>) lives in <code>tools/segment_probe.jai</code> for now. Phase 4 moves it into the app's scan.</li>
|
||
</ul></div>
|
||
</div>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 4: Per-folder opt-in and the scan <span class="tag decided">Done</span></h3>
|
||
<p>The pieces:</p>
|
||
<ul>
|
||
<li><code>.library-folders</code> (load, save, keep unknown keys).</li>
|
||
<li>The Segmentation window's folder table.</li>
|
||
<li>The Import window's checkbox.</li>
|
||
<li>The <code>SEGMENT</code> task and its item selection.</li>
|
||
<li>The import follow-up scan.</li>
|
||
<li><code>-segment</code> and <code>-erase segments</code>.</li>
|
||
<li>The shutdown status line.</li>
|
||
</ul>
|
||
<p><strong>Done when:</strong> on a scratch library with two folders, only the opted-in one is segmented. Videos in it are skipped. Cancelling mid-batch stores nothing. Erase per folder works.</p>
|
||
<div class="note"><strong>Result (2026-10-10).</strong>
|
||
<ul>
|
||
<li><strong>The scratch library:</strong> three folders. Photos (16) and Videos (4 videos and 2 GIFs) are set to segment; Other (9 images) is not.</li>
|
||
<li><strong>First <code>-segment</code>:</strong> it segmented exactly the 16 photos and 2 GIFs. The model loaded in 4.5 s and the scan took 6 s. Other and the videos were left alone.</li>
|
||
<li><strong>Second <code>-segment</code>:</strong> it found nothing to do and exited 0.</li>
|
||
<li><strong><code>-erase segments</code>:</strong> it removed all 18.</li>
|
||
<li><strong>Closing mid-scan:</strong> no crash, and exit took 0.3 to 1.8 s. Only whole engine batches were stored.</li>
|
||
<li><strong>Tests:</strong> 14 new checks of <code>.library-folders</code> pass: quoted names, unknown keys kept, a later version refused.</li>
|
||
</ul>
|
||
<p><strong>Changes from the plan:</strong></p>
|
||
<ul>
|
||
<li><strong>Cancel granularity.</strong> Cancelling between the engine's batches of 4 keeps any whole batch already stored, rather than storing nothing for the scan's batch of 8. Each stored image is complete; the counter counts those.</li>
|
||
<li><strong>Stale results.</strong> A capture segmented with another class list is scanned again only once the engine is up. Until then a <code>segs</code> chunk counts as done.</li>
|
||
<li><strong>The import follow-up</strong> runs whether or not "Read text and labels" is ticked, since the folder's setting is the opt-in. It waits for the new captures' catalog entries, so it knows which are videos. Headless <code>-import</code> does not segment.</li>
|
||
<li><strong>Erase per folder</strong> uses the OCR window's erase task, filtered to the folder, and its progress shows there.</li>
|
||
</ul>
|
||
<p><strong>Not yet checked by eye:</strong> the Segmentation window's layout. It is drawn during the GUI <code>-segment</code> runs (scan and cancel), but no screenshot was taken.</p></div>
|
||
</div>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 5: Catalog, <code>has:</code> and Stats <span class="tag decided">Done</span></h3>
|
||
<p><strong>Done when:</strong> <code>has:person</code> finds the right captures in the scratch library, catalog tests cover the new field, and Stats shows the top classes, each clickable.</p>
|
||
<div class="note"><strong>Result (2026-10-10).</strong> The test library is phase 4's: 18 segmented images.
|
||
<ul>
|
||
<li><strong><code>has:</code> searches</strong> (headless <code>-stats</code> with each search):
|
||
<ul>
|
||
<li><code>has:person</code>: 9 images.</li>
|
||
<li><code>has:cat</code>: 3, the COCO cats photo and the 2 GIFs.</li>
|
||
<li><code>has:table</code>: 1, the only plain "table", not the "dining table" images.</li>
|
||
<li>A plain <code>person</code>: none, since plain words don't search segments.</li>
|
||
</ul></li>
|
||
<li><strong>Stats</strong> lists person 9, wall 7, sky 4, tree 4, cat 3 and so on, each class counted once per image.</li>
|
||
<li><strong>Tests:</strong> 10 new catalog checks cover <code>has:</code>: whole names, quoted names with spaces, case, combinations, and a plain word not matching.</li>
|
||
</ul>
|
||
<p>As built:</p>
|
||
<ul>
|
||
<li><strong>The catalog</strong> keeps each segment's class, score and area, and a lookup string of the classes between newlines. <code>search_parse</code> wraps a <code>has:</code> term in newlines once, so matching a whole name is one <code>contains</code> per capture.</li>
|
||
<li><strong>The Stats section</strong> "What is in the images" shows even when nothing has been OCR'd.</li>
|
||
</ul>
|
||
<p><strong>Not yet checked by eye:</strong> the Stats section's layout.</p>
|
||
<p><strong>Left for later:</strong> counts and sizes (<code>has:person:3+</code>, <code>has:dog>20%</code>).</p></div>
|
||
</div>
|
||
|
||
<div class="phase">
|
||
<h3>Phase 6: The overlay <span class="tag decided">Done</span></h3>
|
||
<p>The mask texture on a worker, the legend, hiding classes, opacity and hover.</p>
|
||
<p><strong>Done when:</strong> masks line up with the image at every fit and zoom, including on Snagit captures (rendered image) and on photos turned by their EXIF orientation.</p>
|
||
<div class="note"><strong>Built (2026-10-10); waiting on a check by eye.</strong>
|
||
<ul>
|
||
<li><strong>Reading:</strong> a viewer worker reads the capture's <code>segs</code> and <code>mask</code>, not its images. It reads them again when the capture's catalog entry changes: stepped to, or segmented again.</li>
|
||
<li><strong>The texture:</strong> the main thread makes it at the mask's size, and remakes it only when the capture, the opacity or the hidden classes change.</li>
|
||
<li><strong>Colours:</strong> fixed by the class name's hash, a shade per instance (the plan's note from phase 2), with darker, stronger edges.</li>
|
||
<li><strong>The row above the image:</strong> a Segments toggle, an opacity slider, and a legend that wraps across lines. Each class can be hidden, and right-clicking one searches <code>has:</code>.</li>
|
||
<li><strong>Hover:</strong> the tooltip shows the class, score and share of the image.</li>
|
||
<li><strong>Placement:</strong> the overlay covers the image as drawn, whatever the fit or zoom. It sits under the OCR boxes.</li>
|
||
<li><strong>Tests:</strong> 12 checks on the pixels (shades, edges, hidden classes, opacity) and on reading a file's segments.</li>
|
||
</ul>
|
||
<p><strong>Checked by eye (2026-10-10):</strong> it looks right.</p>
|
||
<p><strong>Then, colours you choose:</strong></p>
|
||
<ul>
|
||
<li><strong>The picker:</strong> each legend swatch is an ImGui colour picker (<code>ColorEdit3</code>, a hue wheel). The colour chosen becomes the class's in every image and library, and its instances shade from it.</li>
|
||
<li><strong>Reset:</strong> right-clicking the swatch takes the class back to its own colour, from its name's hash.</li>
|
||
<li><strong>Where they're kept:</strong> <code>%APPDATA%\mcap\segment-colours.config</code>, one line per class, such as <code>colour: "person" #e05a47</code>. Colours are a way of looking, not part of a library. The file is written once the picker closes, not on every frame of a drag.</li>
|
||
<li><strong>Tests:</strong> 5 more checks: a chosen colour, its shades, other classes unchanged, reset.</li>
|
||
</ul></div>
|
||
</div>
|
||
|
||
<h2 id="state">Where it stands (2026-10-10)</h2>
|
||
<p>All six phases are built and in <code>master</code>. You've checked them in the app: <code>has:person</code> and <code>has:cat</code> work, and the overlay with its colour picker looks right.</p>
|
||
|
||
<div class="table-wrap">
|
||
<table>
|
||
<tr><th>Commit</th><th>Version</th><th>Phase</th></tr>
|
||
<tr><td><code>cb21b73</code></td><td>0.0.11</td><td>1–2: the ONNX export and its check; <code>modules/Segment</code> and <code>segment-probe</code></td></tr>
|
||
<tr><td><code>8b66021</code></td><td>0.0.12</td><td>3: the <code>segs</code> and <code>mask</code> chunks, <code>mcap_store</code> with back chunks</td></tr>
|
||
<tr><td><code>3e6e733</code></td><td>0.0.13</td><td>4: <code>.library-folders</code>, the segmentation scan and window, <code>-segment</code>, <code>-erase segments</code></td></tr>
|
||
<tr><td><code>3c2c07b</code></td><td>0.0.14</td><td>5: <code>has:</code> search; "What is in the images" in Stats</td></tr>
|
||
<tr><td><code>cf3cecf</code></td><td>0.0.15</td><td>6: the overlay in the Capture window, colours chosen in its legend</td></tr>
|
||
</table>
|
||
</div>
|
||
|
||
<h3>What you can do now</h3>
|
||
<ul>
|
||
<li><strong>Choose folders:</strong> in <strong>Segments</strong> above the grid, tick the library folders to segment. They are kept in the library's <code>.library-folders</code>. Importing into a ticked folder segments the new images.</li>
|
||
<li><strong>Scan:</strong> segment those folders' images, with pause and cancel, from the window or with <code>-segment</code>. It runs at about 17 images a second on the RTX 5080 laptop and stores about 5 KB per image.</li>
|
||
<li><strong>Search:</strong> <code>has:dog</code>, <code>has:"dining table"</code>, combined with any other search term.</li>
|
||
<li><strong>Stats:</strong> how many images hold each class, each one a <code>has:</code> search when clicked.</li>
|
||
<li><strong>Look:</strong> the Capture window draws the segments over the image. It has opacity, a legend to hide classes or search them, colours you can choose (kept in <code>%APPDATA%\mcap\segment-colours.config</code>), and the class under the mouse.</li>
|
||
<li><strong>Erase:</strong> per folder in the window, or the whole library with <code>-erase segments</code>.</li>
|
||
</ul>
|
||
|
||
<h3>Tests</h3>
|
||
<ul>
|
||
<li><strong>Unit tests:</strong> 55 checks in the import suite cover the chunks (24), <code>.library-folders</code> (14), and the overlay and colours (17). 10 in the catalog suite cover <code>has:</code>.</li>
|
||
<li><strong><code>segment-probe -check</code>:</strong> mcap's segmentation matches PyTorch's on 99.8% of pixels over 25 images.</li>
|
||
<li><strong>Scratch libraries:</strong> the scan, cancel, erase and search were run end to end.</li>
|
||
</ul>
|
||
|
||
<h3>Left to do</h3>
|
||
<ul>
|
||
<li><strong>Push.</strong> The repo has no remote yet; adding one is all that's needed to push.</li>
|
||
<li><strong>Getting the model.</strong> <code>fetch_models.jai</code> for <code>modules/Segment</code> waits on where the 462 MB .onnx is hosted (open question 2). Until then, <code>tools/export_mask2former.py</code> makes it, and a fresh clone shows "the model is not there" in the Segmentation window.</li>
|
||
<li><strong>SnagitConverter.</strong> Its copy of <code>Capture_File</code> predates file version 4 (0.0.9), so it refuses files with a video's 64-bit size. The segment chunks themselves need no change: it keeps chunks it does not know.</li>
|
||
<li><strong>Untested:</strong> writing <code>segment-colours.config</code> and reading it back. The tests set colours in memory only, so they never touch the real data folder.</li>
|
||
</ul>
|
||
|
||
<h3>Ideas for later</h3>
|
||
<ul>
|
||
<li><strong>Counts and sizes in <code>has:</code>:</strong> <code>has:person:3+</code>, <code>has:dog>20%</code>. The front chunk already has what they need.</li>
|
||
<li><strong>Faster scans:</strong> making segments (30 ms) takes longer than the model (21 ms). Spreading it over threads would roughly double throughput, if large libraries make the scan's speed matter.</li>
|
||
<li><strong>The score threshold</strong> is 0.5, transformers' default. Raise <code>score_threshold</code> in <code>models.config</code> toward 0.7 if weak segments make <code>has:</code> noisy on your photo libraries.</li>
|
||
<li><strong>A warning for screenshot folders</strong> (open question 4), and <strong>erasing other models' masks</strong> after a model change (open question 5).</li>
|
||
<li><strong>Video frames</strong> (section 10), shelved.</li>
|
||
</ul>
|
||
|
||
<h2 id="questions">9. Open questions</h2>
|
||
<ol>
|
||
<li><strong>COCO panoptic or ADE20K?</strong> COCO counts things (people, pets, cars); ADE20K knows more indoor objects and scene parts, but has no instances. The default is COCO. <span class="tag decided">Built with COCO</span> Revisit when your photo libraries are in, by trying both on a sample; results are keyed by model, so the two can sit side by side.</li>
|
||
<li><strong>Where to host the exported .onnx</strong> for <code>fetch_models.jai</code>: a private Hugging Face repo, a release asset, or committing it as the FFmpeg DLLs are. At about 400 MB for Swin-B, committing it is probably too much.</li>
|
||
<li><strong>The mask resolution cap.</strong> <span class="tag decided">Answered</span> 1024 on the long side costs 0.8 to 6.4 KB per photo (phase 3), so there is no reason to go lower.</li>
|
||
<li><strong>Snagit and screenshot folders.</strong> They stay off by default, and COCO classes mean little on UI. Turning one on is allowed, but the Segmentation window could warn that most of the folder is screenshots, judged from <code>Format</code> and <code>AppName</code>.</li>
|
||
<li><strong>Re-scanning on a model change.</strong> A new model name means a new key, so old results stay until erased. Should the scan offer to erase other models' masks to save space? Probably yes, as an explicit button rather than automatically.</li>
|
||
</ol>
|
||
|
||
<h2 id="later">10. Later: video frames <span class="tag later">Shelved</span></h2>
|
||
<p>Treat strategically sampled frames of a video as images, and run the same analysis (segmentation, and OCR and labels too) on them. Notes for when it comes back:</p>
|
||
<ul>
|
||
<li><strong>Sampling:</strong> scene changes rather than a fixed rate. Compare small thumbnails of successive decoded frames, or use FFmpeg's scene score. Cap the frames per minute.</li>
|
||
<li><strong>Storage:</strong> keep the frames themselves out of the .mcap except as small thumbnails. Store per-frame results in chunks keyed by model and frame time (for example a key like <code>mask2former…@83.40</code>), or in one chunk holding every frame's results.</li>
|
||
<li>The per-folder option would grow a second key, <code>segment_videos</code>, in the same <code>.library-folders</code> file.</li>
|
||
</ul>
|
||
|
||
</main>
|
||
</body>
|
||
</html>
|