Musa Mahmood
811c096dd0
- Import window (and -import SOURCE DEST): every image under a folder (PNG, JPEG, HEIC, AVIF, WebP, GIF, BMP, TIFF, camera RAW) becomes an .mcap holding the file as it was, a preview turned by its EXIF orientation, and metadata.json and import.json Capture Text (date from EXIF, the file name or its time; file name as title; camera as app). .snagx files become the same .mcap SnagitConverter makes. Duplicates across the whole library are skipped by a hash index (.import-index); videos and sidecars are skipped; source files are only read. - modules/Image_Decode: stb_image, then WIC for everything else; EXIF; orientation; previews. tools/image_probe.jai shows what it reads. - modules/Snagx (and modules/jaison), copied from SnagitConverter. - The library is a folder of folders: recursive scan, a folder filter, and a folder: search prefix. - OCR, the viewer and ocr-bench decode any image format, turned the right way up. - Rename D:\SnagitIndex to D:\McapIndex.
168 lines
9.2 KiB
Markdown
168 lines
9.2 KiB
Markdown
# The .mcap file
|
|
|
|
One capture in one file: a fixed magic, a file version stamp, the capture's data as
|
|
[Capture Text](capture_text.md), what OCR read in it, what other models added (chunks), a small preview,
|
|
then the images, then any large chunks. There is no ZIP and no compression around it; the images keep
|
|
their own compression.
|
|
|
|
A capture is a Snagit capture (from a `.snagx`) or an imported image: a photo or screenshot in any image
|
|
format (the mcap app's Import window).
|
|
|
|
The parts a library browser needs (header, text, OCR, front chunks and preview) come first, so it reads
|
|
the start of each file and never touches the images; a search, which needs no preview, reads less still.
|
|
|
|
## Layout, version 3
|
|
|
|
All integers are little-endian. Sizes count bytes.
|
|
|
|
| Field | Type | |
|
|
| --- | --- | --- |
|
|
| magic | 8 bytes | `89 4D 43 41 50 0D 0A 1A`, which is `\x89MCAP\r\n\x1A`. Says this is an .mcap, in every version. |
|
|
| file version | u32 | `3`. Versions 1 and 2 are still read; see below. |
|
|
| text size, text | u32, bytes | Capture Text, UTF-8: every JSON file of the capture, as `[[file]]` sections. |
|
|
| OCR size, OCR | u32, bytes | OCR text, UTF-8 (below): what each OCR model read. Size 0 when none has. |
|
|
| front chunks | chunk list | Small things models added, such as labels and an image vector (below). |
|
|
| preview width, height | u16, u16 | In pixels. |
|
|
| preview size, preview | u32, bytes | PNG. Size 0 when there is none. |
|
|
| original name size, name | u16, bytes | The original image's name: in the .snagx, such as `{GUID}.png` (the page's `CaptureBackgroundImage` names it too), or an imported image's file name, such as `IMG_0001.HEIC`. |
|
|
| original size, original | u32, bytes | The capture as taken, exactly as it was: PNG from a .snagx; an imported image in its own format (JPEG, HEIC, WebP, RAW ...), told by its first bytes, not its name. |
|
|
| rendered kind | u8 | 0: none. 1: the same as the original (stored once). 2: its own, below. |
|
|
| rendered size, rendered | u32, bytes | Only for kind 2. The capture with its annotations drawn in (Snagit's `thumbnail.png`): PNG, exactly as it was. |
|
|
| back chunks | chunk list | Large things models added, such as segmentation masks: after the images, so reading the front never reaches them. |
|
|
|
|
The file ends there: anything after the last part is a mistake.
|
|
|
|
The magic borrows PNG's: the first byte has its high bit set, so a channel that strips it shows; `\r\n`
|
|
shows line endings being converted; and `\x1A` stops `type` on Windows.
|
|
|
|
A **chunk list** is a u32 count, then each chunk: a 4-byte tag of ASCII letters saying what it is
|
|
(`labl`, `embd`), a u32 size, and that many bytes: a u16 key size, the key (whose it is: a model's
|
|
name), and the chunk's own data. A file holds at most one chunk per tag and key.
|
|
|
|
**Version 2** is version 3 without the two chunk lists; **version 1** is version 2 without the OCR size
|
|
and OCR. A reader takes all three; anything written is version 3, so an older file becomes version 3
|
|
the first time something is stored in it.
|
|
|
|
## The parts
|
|
|
|
- **Text.** Everything the capture's JSON held: `index.json`, `metadata.json`, the page's `{GUID}.json`,
|
|
and `{GUID}.backup.json` when there is one. An imported image has two files instead (below).
|
|
- **OCR.** One section per OCR model, so a capture read by several models keeps every reading;
|
|
reading it again with a model replaces only that model's section. See *OCR text* below.
|
|
- **Chunks.** What other models add, each keyed by the model's name, so models do not overwrite each
|
|
other; storing a chunk again replaces only the one with its tag and key. A chunk a reader does not
|
|
know is kept as it is when the file is rewritten, so a later build can add kinds without a new file
|
|
version. See *Label chunks* below.
|
|
- **Preview.** The rendered image (or the original, when there is no rendered one) scaled so its longest
|
|
side is at most 320 pixels, as a PNG. A Snagit PNG already that small is used as it is, never scaled
|
|
up. An imported image's preview is always a PNG made from it, turned the right way up by its EXIF
|
|
orientation, which a reader applies to the original too.
|
|
- **Original and rendered.** A capture with no annotations has a rendered image identical to the original,
|
|
byte for byte; it is then stored once, as kind 1.
|
|
- **One page.** An .mcap holds one page, as every capture so far does; a .snagx with more is not converted.
|
|
- **Left out:** `{GUID}.backup.png`, Snagit's image from before an edit. Its `{GUID}.backup.json` is kept,
|
|
as plenty of captures have one without the image.
|
|
|
|
## Imported images
|
|
|
|
An image imported from a folder has no rendered image (kind 0), and its Capture Text holds two files:
|
|
|
|
```
|
|
[[file "metadata.json"]]
|
|
AppName: "Apple iPhone 15 Pro" the camera or phone, from EXIF; "" when unknown
|
|
CaptureDate: "2024-05-12 10:15:30" when it was taken, local time, as Snagit writes it
|
|
WindowName: "IMG_4021.HEIC" the file's name
|
|
|
|
[[file "import.json"]]
|
|
SourcePath: "E:/Phone/DCIM/IMG_4021.HEIC" where it was imported from
|
|
SourceSize: 2994394
|
|
SourceModified: "2024-05-12 10:15:31"
|
|
ContentHash: "d3fa07db58597cec" 64-bit FNV-1a of the original's bytes
|
|
Format: "HEIF"
|
|
Width: 3024 as shown: after the orientation is applied
|
|
Height: 4032
|
|
DateFrom: "EXIF" or "file name", or "modified time"
|
|
ImportDate: "2026-10-08 22:28:17"
|
|
Orientation: 6 and, when the EXIF has them: CameraMake, CameraModel,
|
|
Software, TimeZoneOffset, Latitude, Longitude, Altitude
|
|
```
|
|
|
|
So the parts a library reads from Snagit's `metadata.json` (date, title, app) work for photos too.
|
|
|
|
## Versions
|
|
|
|
A reader checks the magic, then the file version. It refuses a version above its own, never writing
|
|
over a file from a later version, and reads every version up to its own. Version 0 does not exist.
|
|
|
|
## Converting
|
|
|
|
`snagit-converter mcap` converts .snagx files (or extracted captures) to .mcap, and `snagit-converter
|
|
snagx` converts .mcap files back to a .snagx Snagit can open. See the README for what each checks.
|
|
The mcap app's Import makes the same .mcap from a .snagx (modules/Snagx). Only captures from a .snagx
|
|
convert back: an imported image has no Snagit JSON to make one from.
|
|
|
|
Back in a .snagx, the JSON is laid out as Snagit writes it, the images are the same bytes, and only the
|
|
backup image is missing. JSON numbers come back as values: `3.0` is written `3`.
|
|
|
|
## OCR text
|
|
|
|
In the style of the Ex4 text formats:
|
|
|
|
```
|
|
@version 2
|
|
|
|
[ocr ppocr-v5-mobile]
|
|
engine: "onnxruntime 1.30.0, CUDA"
|
|
image: rendered 1920 1080
|
|
date: 2026-10-08T21:04:11Z
|
|
line: 412 96 640 18 412 96 1052 96 1052 114 412 114 0.993 "Assign Footprints - KiCad"
|
|
line: 20 140 220 14 20 140 240 140 240 154 20 154 0.981 "Symbol : Footprint Assignments"
|
|
line: 1700 300 20 160 1720 300 1720 460 1700 460 1700 300 0.952 "Read downwards"
|
|
```
|
|
|
|
| Key | Value |
|
|
| --- | --- |
|
|
| `[ocr NAME]` | A section: the model's name, as in its `models.config`. |
|
|
| `engine` | What ran it, quoted. |
|
|
| `image` | Which image was read (`rendered` or `original`) and its width and height; the lines' boxes are in its pixels. |
|
|
| `date` | When it was read, UTC. |
|
|
| `line` | A text line: x, y, width and height, then its four corners as x y pairs, its confidence (0 to 1), and its text, quoted with `""` for a quote. A control character in the text is written as a space. |
|
|
|
|
The corners go clockwise from the text's top left: top left, top right, bottom right, bottom left, as
|
|
the text reads. Level text has the box's corners; a turned line's say which way it reads (the third
|
|
line above reads downwards, so its top left is the box's top right). x, y, width and height bound them.
|
|
|
|
A line that cannot be read is skipped, as are keys this version does not know. **Version 1** OCR text
|
|
had no corners (`line: x y width height confidence "text"`); it is still read, with the box's corners.
|
|
The OCR text carries its own version, so this did not change the .mcap file version. `mcap_store_ocr`
|
|
(`modules/Capture_File/ocr_text.jai`) stores a section in a file.
|
|
|
|
## Label chunks
|
|
|
|
A label model (an image-text encoder such as SigLIP 2) adds two front chunks, keyed by its name.
|
|
|
|
**`labl`**: the capture's best labels from a label list, as text in the style above:
|
|
|
|
```
|
|
@version 1
|
|
engine: "onnxruntime 1.30.0, CUDA"
|
|
date: 2026-10-08T21:04:11Z
|
|
list: 9c4f0b2d81e3a675
|
|
label: 0.6720 "Settings"
|
|
label: 0.0951 "Installer"
|
|
```
|
|
|
|
| Key | Value |
|
|
| --- | --- |
|
|
| `engine`, `date` | As in OCR text. |
|
|
| `list` | Which label list they were scored against: a 64-bit FNV-1a hash of its names and prompts. Labels from another list are out of date. |
|
|
| `label` | A score, 0 to 1 (the model's own sigmoid; small, but their order is what counts), and the label's name, quoted with `""` for a quote. Best first. |
|
|
|
|
**`embd`**: the image's vector, binary: a u32 length, a u8 format (1: 16-bit floats), then the values,
|
|
little-endian. It is a unit vector, so any phrase's vector (from the same model's text encoder) scores
|
|
against it with a dot product: labels from a new list, or a search by meaning, need no new pass over
|
|
the images. 768 values, 1.5 KB, for SigLIP 2 base.
|
|
|
|
`mcap_store` (`modules/Capture_File/module.jai`) stores an OCR section and chunks in one rewrite;
|
|
`label_chunks.jai` reads and writes these two.
|