[0.0.9] Import: videos and animated GIFs, in .mcap version 4

- Import brings in videos (MP4, MOV, MKV, WebM, AVI, WMV, MPEG, MTS/M2TS, FLV) on Windows, read with
  FFmpeg: the file as it was in the original, a frame a tenth of the way in (at most 3 s), turned
  the right way up, as the rendered image (what the grid, Capture window, OCR and labels see), and
  import.json with Duration, FrameRate, VideoCodec, Rotation, PosterTime, Audio, and the camera,
  date and location from the container. Elsewhere videos are skipped as "video (Windows only)".
- .mcap version 4: the original's size is a u64, so a video can be any size. Versions 1-3 still read.
  mcap_read_without_original and mcap_write_streamed read and rewrite a file with its original left
  on disk; mcap_store, mcap_erase, the import index and the import itself never hold a video in memory.
- Animated GIFs: Frames and Duration in import.json (gif_info).
- Grid: a play mark and duration on video thumbnails, GIF on animations. Capture window: what the
  video is, and Play in the default player (written out to %TEMP%\mcap).
- modules/ffmpeg: FFmpeg 6.0 bindings and the five DLLs used (avformat, avcodec, avutil, swscale,
  swresample), copied beside the executable by the build.
  swscale converts into a buffer of its own allocator with padded rows: an exact-sized one let
  its SIMD write past the end, crashing on frames of odd widths (1916, 1106 ...).
- image-probe shows videos and GIF frame counts; ocr-bench reads a video's frame.
- Tests: new video formats, display-matrix rotation, ISO 6709, video dates, GIF frames, and v4
  serialize/parse, v3 compatibility, streamed writes, store/erase and extraction.

SnagitConverter's copy of Capture_File needs syncing to read version 4 files.
This commit is contained in:
Musa Mahmood 2026-10-09 23:04:35 +01:00
parent 811c096dd0
commit baa188b0e4
30 changed files with 15316 additions and 226 deletions

View File

@ -7,7 +7,8 @@ with [Dear ImGui](https://github.com/ocornut/imgui), for Windows (macOS and Linu
and labels are tested on Windows with an NVIDIA GPU).
The `.mcap` files come from the **Import** window, which takes any folder of images (PNG, JPEG, HEIC,
AVIF, WebP, GIF, BMP, TIFF, camera RAW) and Snagit `.snagx` captures, or from `snagit-converter`
AVIF, WebP, GIF, BMP, TIFF, camera RAW), videos (MP4, MOV, MKV, WebM, AVI, WMV, MPEG) and Snagit
`.snagx` captures, or from `snagit-converter`
(`D:\McapIndex\SnagitConverter`); `modules/Capture_File/README.md` describes the format.
## Getting started
@ -113,7 +114,7 @@ counted (click one to search for it), and the commonest words (click one to sear
## Importing
**Import** (above the grid) brings a folder of screenshots and photos into the library:
**Import** (above the grid) brings a folder of screenshots, photos and videos into the library:
1. **Import from:** the folder, typed or chosen with **Browse...**. Everything under it is looked
at, its folders too (not hidden ones, such as `.thumbnails`).
@ -134,13 +135,24 @@ What becomes of each file:
`2024-05-12_10-15-30_1a2b3c4d.mcap`, the date and the start of its hash. HEIC, AVIF, WebP and RAW
are read by Windows' own codecs: the HEIF, AV1, WebP and Raw Image Extensions from the Microsoft
Store.
- **Animated GIFs** are images; their thumbnail is marked **GIF**, and the Capture window says how
long they run and opens them in the default viewer.
- **Videos** (MP4, MOV, M4V, 3GP, MKV, WebM, AVI, WMV, MPEG, MTS and M2TS, FLV), read with FFmpeg
(`modules/ffmpeg`, Windows only): an `.mcap` holding the file exactly as it was, of any size (it is
copied a few MB at a time, never read whole), and a frame from it a tenth of the way in (at most 3
seconds), turned the right way up, which is what the grid, the Capture window, OCR and labels see.
The date is the phone's (Apple's creation date), else one in its name, else the video's own
creation time, else the file's time; the phone and its location are kept too. The thumbnail shows a
play mark and how long it runs; **Play in the default player** in the Capture window writes the
video out to the temporary folder and opens it.
- **Snagit `.snagx`:** the same `.mcap` as `snagit-converter mcap` makes, byte for byte, named as
the `.snagx` is.
- **Duplicates:** an image already in the library, in any of its folders, is not stored again; the
Import window lists each with where it is. Importing a folder a second time imports only what is
new.
- **Skipped:** videos (`.mp4`, `.mov`, ...), sidecar files (`.aae`, `.xmp`, `.json`) and anything
that is not an image, counted by kind.
- **Skipped:** sidecar files (`.aae`, `.xmp`, `.json`), a GoPro's thumbnails and low-resolution
copies (`.thm`, `.lrv`), videos on macOS and Linux, and anything that is not an image or a video,
counted by kind.
- **Failed:** files that cannot be read or decoded, listed with why.
The files imported from are only read: nothing there is changed, moved or deleted.
@ -259,10 +271,13 @@ converted copy.
several threads; captures written are handed to the grid each frame.
- `modules/Image_Decode`: any image file's bytes to pixels, the right way up: stb_image for PNG,
JPEG, GIF, BMP, PSD and TGA, Windows Imaging Component for the rest; EXIF (from JPEG, PNG, WebP,
HEIF and TIFF); previews. `bin\image-probe FILE...` (`jai first.jai - image_probe`) shows what it
makes of files.
HEIF and TIFF); previews; and videos' frames and what their containers say, with FFmpeg
(`video.jai`), and how many frames a GIF has. `bin\image-probe FILE...` (`jai first.jai -
image_probe`) shows what it makes of files.
- `modules/Snagx`: `.snagx` to `.mcap`, copied from SnagitConverter.
- `modules/Capture_File`: reading and writing `.mcap` files.
- `modules/Capture_File`: reading and writing `.mcap` files, with a video's original left on disk.
- `modules/ffmpeg`: FFmpeg 6.0's bindings and DLLs (`modules/ffmpeg/windows`, in git; copied beside
the executable by the build), from the gyan.dev shared build, for reading videos.
# The application template

View File

@ -29,7 +29,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap";
GIT_BRANCH :: "check";
GIT_HASH :: "-";
VERSION :: "0.0.8";
VERSION :: "0.0.9";
BUILD_DATE :: "-";
DEBUG :: true;
COMPILE_TESTS :: true;

View File

@ -8,7 +8,7 @@ APP_NAME :: "mcap";
APP_FOLDER_NAME :: "mcap";
APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
VERSION :: "0.0.8";
VERSION :: "0.0.9";
// Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
// builds never write over the ones you run by hand.
build_directory := "bin";
@ -286,6 +286,11 @@ copy_assets :: () {
log("OCR: the models have not been fetched; run jai fetch_models.jai in modules/OCR for OCR.");
}
// Videos: FFmpeg's DLLs beside the executable (modules/ffmpeg/windows, in git), the same way.
#if OS == .WINDOWS {
for file_list("modules/ffmpeg/windows") if ends_with(it, ".dll") copy_if_changed(it, tprint("%/%", build_directory, path_filename(it)));
}
// Labels: its models in bin/label-models, the same way.
make_directory_if_it_does_not_exist(tprint("%/label-models", build_directory));
copy_if_changed("modules/Labels/models.config", tprint("%/label-models/models.config", build_directory));

View File

@ -2,32 +2,32 @@
One capture in one file: a fixed magic, a file version stamp, the capture's data as
[Capture Text](capture_text.md), what OCR read in it, what other models added (chunks), a small preview,
then the images, then any large chunks. There is no ZIP and no compression around it; the images keep
their own compression.
then the original and rendered image, then any large chunks. There is no ZIP and no compression around
it; the images and videos keep their own compression.
A capture is a Snagit capture (from a `.snagx`) or an imported image: a photo or screenshot in any image
format (the mcap app's Import window).
A capture is a Snagit capture (from a `.snagx`), an imported image (a photo or screenshot in any image
format), or an imported video (the mcap app's Import window).
The parts a library browser needs (header, text, OCR, front chunks and preview) come first, so it reads
the start of each file and never touches the images; a search, which needs no preview, reads less still.
## Layout, version 3
## Layout, version 4
All integers are little-endian. Sizes count bytes.
| Field | Type | |
| --- | --- | --- |
| magic | 8 bytes | `89 4D 43 41 50 0D 0A 1A`, which is `\x89MCAP\r\n\x1A`. Says this is an .mcap, in every version. |
| file version | u32 | `3`. Versions 1 and 2 are still read; see below. |
| file version | u32 | `4`. Versions 1 to 3 are still read; see below. |
| text size, text | u32, bytes | Capture Text, UTF-8: every JSON file of the capture, as `[[file]]` sections. |
| OCR size, OCR | u32, bytes | OCR text, UTF-8 (below): what each OCR model read. Size 0 when none has. |
| front chunks | chunk list | Small things models added, such as labels and an image vector (below). |
| preview width, height | u16, u16 | In pixels. |
| preview size, preview | u32, bytes | PNG. Size 0 when there is none. |
| original name size, name | u16, bytes | The original image's name: in the .snagx, such as `{GUID}.png` (the page's `CaptureBackgroundImage` names it too), or an imported image's file name, such as `IMG_0001.HEIC`. |
| original size, original | u32, bytes | The capture as taken, exactly as it was: PNG from a .snagx; an imported image in its own format (JPEG, HEIC, WebP, RAW ...), told by its first bytes, not its name. |
| original name size, name | u16, bytes | The original's name: in the .snagx, such as `{GUID}.png` (the page's `CaptureBackgroundImage` names it too), or an imported file's name, such as `IMG_0001.HEIC` or `IMG_4022.MOV`. |
| original size, original | u64, bytes | The capture as taken, exactly as it was: PNG from a .snagx; an imported image or video in its own format (JPEG, HEIC, WebP, RAW, MP4, MOV, MKV ...), told by its first bytes, not its name. |
| rendered kind | u8 | 0: none. 1: the same as the original (stored once). 2: its own, below. |
| rendered size, rendered | u32, bytes | Only for kind 2. The capture with its annotations drawn in (Snagit's `thumbnail.png`): PNG, exactly as it was. |
| rendered size, rendered | u32, bytes | Only for kind 2. The capture with its annotations drawn in (Snagit's `thumbnail.png`): PNG, exactly as it was. A video's is a frame from it, as PNG (below). |
| back chunks | chunk list | Large things models added, such as segmentation masks: after the images, so reading the front never reaches them. |
The file ends there: anything after the last part is a mistake.
@ -39,9 +39,14 @@ A **chunk list** is a u32 count, then each chunk: a 4-byte tag of ASCII letters
(`labl`, `embd`), a u32 size, and that many bytes: a u16 key size, the key (whose it is: a model's
name), and the chunk's own data. A file holds at most one chunk per tag and key.
**Version 2** is version 3 without the two chunk lists; **version 1** is version 2 without the OCR size
and OCR. A reader takes all three; anything written is version 3, so an older file becomes version 3
the first time something is stored in it.
**Version 3** is version 4 with the original's size a u32, so an original of at most 4 GB; **version 2**
is version 3 without the two chunk lists; **version 1** is version 2 without the OCR size and OCR. A
reader takes all four; anything written is version 4, so an older file becomes version 4 the first time
something is stored in it.
An original can be larger than memory, so a file is read and rewritten with its original left on disk:
`mcap_read_without_original` reads everything else and says where the original lies, and
`mcap_write_streamed` copies it across a few MB at a time. Storing what a scan adds works that way.
## The parts
@ -89,6 +94,33 @@ Orientation: 6 and, when the EXIF has them: Camer
So the parts a library reads from Snagit's `metadata.json` (date, title, app) work for photos too.
## Imported videos
A video imported from a folder (MP4, MOV, MKV, WebM, AVI, WMV, MPEG; read with FFmpeg, on Windows) is
stored as it was in the original. Its rendered image (kind 2) is a frame from it as a PNG, at its own
size and turned the right way up: a tenth of the way in, at most 3 seconds, past the black or blurred
frames phones often start with. The preview is made from that frame, and OCR and labels read it, as
they read a Snagit capture's rendered image; nothing reads the video itself but the import.
Its metadata.json is a photo's. Its import.json has the keys above (Format is the container: "MP4",
"QuickTime", "WebM" ...; Width and Height as shown) and these:
```
Duration: 83.4 seconds
FrameRate: 30
VideoCodec: "h264" FFmpeg's name for it: "hevc", "vp9", "av1" ...
Rotation: 90 degrees clockwise its frames are turned to be shown
PosterTime: 3 seconds into it of the rendered frame
Audio: true
DateFrom: "metadata" Apple's creationdate (local time), else the file name, else the
container's creation_time (UTC, as this computer's local time), else
the modified time
and, when the container has them: CameraMake, CameraModel, Software,
TimeZoneOffset, Latitude, Longitude, Altitude
```
An animated GIF is imported as an image; its import.json adds `Frames` and `Duration` (seconds).
## Versions
A reader checks the magic, then the file version. It refuses a version above its own, never writing

View File

@ -1,6 +1,7 @@
// .mcap: one capture in one file. A fixed magic, a file version stamp, the capture's Capture Text,
// its OCR text (version 2), its front chunks (version 3), a small preview, the images, then its back
// chunks (version 3). See docs/mcap.md for the layout.
// its OCR text (version 2), its front chunks (version 3), a small preview, the original (an image or a
// video; its size 64-bit from version 4) and the rendered image, then its back chunks (version 3). See
// README.md for the layout.
//
// Chunks hold what models add (labels, embeddings, later masks), each named by a tag and a key (the
// model's name): small ones in front, where a library browser reads them without the images, large
@ -8,11 +9,15 @@
//
// Reading gives views into the bytes it was given; nothing is copied. Images stay PNG.
//
// An original can be larger than memory (a video), so a file is also read and written with its
// original left on disk: mcap_read_without_original and mcap_write_streamed. Storing what a scan adds
// (mcap_store and the others) works that way, copying the original across a few MB at a time.
//
// The main copy is mcap's (D:/jai_projects/active/mcap/modules/Capture_File); SnagitConverter has
// the same files.
MCAP_MAGIC :: u8.[0x89, #char "M", #char "C", #char "A", #char "P", 0x0D, 0x0A, 0x1A];
MCAP_VERSION :: 3; // what is written; every version from 1 up is read
MCAP_VERSION :: 4; // what is written; every version from 1 up is read
MCAP_EXTENSION :: "mcap";
MCAP_PREVIEW_MAX_SIDE :: 320; // the preview's longest side, in pixels
@ -41,10 +46,10 @@ Mcap :: struct {
ocr: string; // OCR text (ocr_text.jai), one section per model; "" when none, and always in version 1
front_chunks: [] Mcap_Chunk; // small: read with the front. None before version 3
preview: Mcap_Image;
original_name: string; // the original image's name in the .snagx, such as {GUID}.png
original: [] u8; // PNG
original_name: string; // the original's name: in the .snagx, such as {GUID}.png, or an imported file's
original: [] u8; // PNG from a .snagx; an imported image or video in its own format
rendered_kind: Mcap_Rendered_Kind;
rendered: [] u8; // PNG; OWN only, see mcap_rendered
rendered: [] u8; // PNG; OWN only, see mcap_rendered. A video's is a frame from it
back_chunks: [] Mcap_Chunk; // large: after the images. None before version 3
}
@ -85,42 +90,71 @@ mcap_rendered :: (m: Mcap) -> [] u8 {
// The file's bytes. A part too large for its size field fails the whole file rather than writing one
// that cannot be read back.
mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string {
size := MCAP_HEADER_SIZE + 4 + m.text.count + 4 + m.ocr.count + mcap_chunks_size(m.front_chunks) + 8 + m.preview.png.count + 2 + m.original_name.count + 4 + m.original.count + 1;
if m.rendered_kind == .OWN size += 4 + m.rendered.count;
size += mcap_chunks_size(m.back_chunks);
s := new_serializer(size);
magic := MCAP_MAGIC;
AddArray_NoSize(*s, magic);
Add(*s, cast(u32) MCAP_VERSION);
if !AddString(*s, m.text, u32) return s, false, "the text is too large for its size field";
if !AddString(*s, m.ocr, u32) return s, false, "the OCR text is too large for its size field";
ok, error := mcap_add_chunks(*s, m.front_chunks);
s, ok, error := mcap_serialize_head(m, m.original.count, extra = m.original.count + mcap_tail_size(m));
if !ok return s, false, error;
Add(*s, m.preview.width);
Add(*s, m.preview.height);
if !AddSizedArray(*s, m.preview.png, u32) return s, false, "the preview is too large for its size field";
if !AddString(*s, m.original_name, u16) return s, false, "the original image's name is too long";
if !AddSizedArray(*s, m.original, u32) return s, false, "the original image is too large for its size field";
Add(*s, m.rendered_kind);
if m.rendered_kind == .OWN && !AddSizedArray(*s, m.rendered, u32) return s, false, "the rendered image is too large for its size field";
ok, error = mcap_add_chunks(*s, m.back_chunks);
if !ok return s, false, error;
assert(s.count == size);
return s, true, "";
AddArray_NoSize(*s, m.original);
ok, error = mcap_add_tail(*s, m);
return s, ok, error;
}
// Writes beside path and then swaps it in, so a failure leaves any earlier file as it was.
mcap_write :: (path: string, bytes: [] u8) -> ok: bool, error: string {
temp_path := tprint("%.tmp", path);
if !write_entire_file(temp_path, bytes.data, bytes.count) return false, tprint("cannot write %", temp_path);
if !atomic_file_replace_or_just_rename_if_destination_file_does_not_exist(temp_path, path) {
file_delete(temp_path);
return false, tprint("cannot move % to %", temp_path, path);
replaced, replace_error := mcap_replace(temp_path, path);
return replaced, replace_error;
}
// An original left in a file: size bytes at offset in path. See mcap_read_without_original.
Mcap_Original_Ref :: struct {
path: string;
offset: s64;
size: s64;
}
// Writes m to path as mcap_write does, with the original copied from original, a file, a few MB at a
// time, in place of m.original: it is never all in memory. original may be in the file at path
// itself, which is only replaced once it has been read.
mcap_write_streamed :: (path: string, m: Mcap, original: Mcap_Original_Ref) -> ok: bool, error: string {
head, ok, error := mcap_serialize_head(m, original.size, extra = mcap_tail_size(m),, temp);
if !ok return false, error;
ok, error = mcap_add_tail(*head, m);
if !ok return false, error;
tail := array_view(head, head.count - mcap_tail_size(m));
head.count -= tail.count;
temp_path := tprint("%.tmp", path);
failed := "";
{
out, opened := file_open(temp_path, for_writing = true, log_errors = false);
if !opened return false, tprint("cannot write %", temp_path);
defer file_close(*out);
source, source_opened := file_open(original.path, log_errors = false);
if !source_opened failed = tprint("cannot read %", original.path);
defer if source_opened file_close(*source);
if !failed && !file_write(*out, head.data, head.count) failed = tprint("cannot write %", temp_path);
if !failed && original.size > 0 {
if !file_set_position(source, original.offset) failed = tprint("cannot read %", original.path);
buffer := NewArray(min(original.size, MCAP_COPY_SIZE), u8, initialized = false,, context.default_allocator);
defer array_free(buffer,, context.default_allocator);
left := original.size;
while !failed && left > 0 {
count := min(left, buffer.count);
read_ok, read := file_read(source, buffer.data, count);
if !read_ok || read != count { failed = tprint("% ends before its original does", original.path); break; }
if !file_write(*out, buffer.data, count) { failed = tprint("cannot write %", temp_path); break; }
left -= count;
}
}
if !failed && !file_write(*out, tail.data, tail.count) failed = tprint("cannot write %", temp_path);
}
return true, "";
if failed {
file_delete(temp_path);
return false, failed;
}
replaced, replace_error := mcap_replace(temp_path, path);
return replaced, replace_error;
}
// ---------------------------------------------------------------- reading
@ -133,17 +167,11 @@ mcap_parse :: (data: [] u8) -> Mcap, ok: bool, error: string {
if !ok return m, false, error;
m.original_name = ReadSizedString(*d, u16);
m.original = ReadSizedArray(*d, u8, u32);
m.rendered_kind = Read(*d, Mcap_Rendered_Kind);
if d.ok && m.rendered_kind == .OWN m.rendered = ReadSizedArray(*d, u8, u32);
if d.ok && m.version >= 3 {
m.back_chunks, ok, error = mcap_read_chunks(*d);
if !ok return m, false, error;
}
size := mcap_read_original_size(*d, m.version);
m.original = ReadArrayView(*d, size, u8);
if !d.ok return m, false, "the file ends early (it is truncated or damaged)";
if cast(u8) m.rendered_kind > cast(u8) Mcap_Rendered_Kind.OWN return m, false, tprint("the rendered image is of unknown kind %", cast(u8) m.rendered_kind);
if d.cursor != d.count return m, false, tprint("% bytes follow the last part", d.count - d.cursor);
return m, true, "";
ok, error = mcap_read_after_original(*d, *m);
return m, ok, error;
}
// Only the front of the file, as a library browser needs it: the header, text, OCR text, front chunks
@ -156,27 +184,98 @@ mcap_read_front :: (path: string, preview := true) -> Mcap, ok: bool, error: str
defer file_close(*file);
length, length_ok := file_length(file);
if !length_ok return m, false, "cannot be read";
ok, error := mcap_read_front_file(file, length, *m, preview);
return m, ok, error;
}
wanted := min(length, MCAP_FRONT_READ_SIZE);
for attempt: 1..MCAP_FRONT_READS {
buffer := NewArray(wanted, u8, initialized = false);
if !file_read(file, buffer.data, wanted) { array_free(buffer); return m, false, "cannot be read"; }
// The whole file but the original, which stays on disk: the ref says where it is. When the rendered
// image is the original (SAME_AS_ORIGINAL), rendered stays empty and the ref is it too. The parts are
// views into buffers from the context allocator.
mcap_read_without_original :: (path: string) -> Mcap, Mcap_Original_Ref, ok: bool, error: string {
m: Mcap;
ref: Mcap_Original_Ref;
file, opened := file_open(path, log_errors = false);
if !opened return m, ref, false, "cannot be opened";
defer file_close(*file);
length, length_ok := file_length(file);
if !length_ok return m, ref, false, "cannot be read";
d := to_deserializer(buffer);
ok, error := mcap_read_front_part(*d, *m, preview);
if ok return m, true, "";
if d.ok || wanted == length { array_free(buffer); return m, false, error; } // not a matter of too few bytes
front_end: s64;
ok, error := mcap_read_front_file(file, length, *m, true, *front_end);
if !ok return m, ref, false, error;
// Short: the sizes read so far say how far the front reaches at least. Read that and as much
// again as the first read, so the size fields after it (the OCR, the chunk count, each chunk's
// size) come in the same read rather than one read each.
needed := mcap_front_size(buffer, preview);
array_free(buffer);
if needed <= wanted || needed > length return m, false, "the file ends early (it is truncated or damaged)";
wanted = min(needed + MCAP_FRONT_READ_SIZE, length);
if !file_set_position(file, 0) return m, false, "cannot be read";
// The original's name and size: at most 2 + 65535 + 8 bytes.
middle := NewArray(min(length - front_end, 2 + 0xFFFF + 8), u8, initialized = false);
if !middle.count || !file_set_position(file, front_end) || !file_read(file, middle.data, middle.count) return m, ref, false, "the file ends early (it is truncated or damaged)";
d := to_deserializer(middle);
m.original_name = ReadSizedString(*d, u16);
size := mcap_read_original_size(*d, m.version);
ref = .{path = path, offset = front_end + d.cursor, size = size};
if !d.ok || size > length - ref.offset return m, ref, false, "the file ends early (it is truncated or damaged)";
// What follows the original: the rendered image and the back chunks.
rest := NewArray(length - ref.offset - size, u8, initialized = false);
if rest.count && (!file_set_position(file, ref.offset + size) || !file_read(file, rest.data, rest.count)) return m, ref, false, "cannot be read";
rest_d := to_deserializer(rest);
ok, error = mcap_read_after_original(*rest_d, *m);
return m, ref, ok, error;
}
// The bytes ref points at, from the context allocator.
mcap_read_original :: (ref: Mcap_Original_Ref) -> [] u8, ok: bool {
bytes := NewArray(ref.size, u8, initialized = false);
if !ref.size return bytes, true;
file, opened := file_open(ref.path, log_errors = false);
if !opened { array_free(bytes); return .[], false; }
defer file_close(*file);
if !file_set_position(file, ref.offset) || !file_read(file, bytes.data, ref.size) { array_free(bytes); return .[], false; }
return bytes, true;
}
// The bytes ref points at, written to a file of their own at path (beside it, then swapped in), a few
// MB at a time: an original as it was imported, to open in another program.
mcap_extract_original :: (ref: Mcap_Original_Ref, path: string) -> ok: bool, error: string {
temp_path := tprint("%.tmp", path);
failed := "";
{
source, opened := file_open(ref.path, log_errors = false);
if !opened return false, tprint("cannot read %", ref.path);
defer file_close(*source);
out, out_opened := file_open(temp_path, for_writing = true, log_errors = false);
if !out_opened return false, tprint("cannot write %", temp_path);
defer file_close(*out);
if !file_set_position(source, ref.offset) failed = tprint("cannot read %", ref.path);
buffer := NewArray(max(min(ref.size, MCAP_COPY_SIZE), 1), u8, initialized = false,, context.default_allocator);
defer array_free(buffer,, context.default_allocator);
left := ref.size;
while !failed && left > 0 {
count := min(left, buffer.count);
read_ok, read := file_read(source, buffer.data, count);
if !read_ok || read != count { failed = tprint("% ends before its original does", ref.path); break; }
if !file_write(*out, buffer.data, count) { failed = tprint("cannot write %", temp_path); break; }
left -= count;
}
}
return m, false, tprint("its front was not all read in % reads", MCAP_FRONT_READS);
if failed {
file_delete(temp_path);
return false, failed;
}
replaced, replace_error := mcap_replace(temp_path, path);
return replaced, replace_error;
}
// The image the capture is shown as, as the Capture window shows it: the rendered one (annotations
// and all, or a video's frame), else the original; which says which, "rendered" or "original". The
// rest of the file comes with it in m, and m.original only when it is the image. A video's original
// is never read: it always has a frame.
mcap_read_shown_image :: (path: string) -> Mcap, image: [] u8, which: string, ok: bool, error: string {
m, ref, ok, error := mcap_read_without_original(path);
if !ok return m, .[], "", false, error;
if m.rendered_kind == .OWN return m, m.rendered, "rendered", true, "";
bytes, read_ok := mcap_read_original(ref);
if !read_ok return m, .[], "", false, "cannot be read";
m.original = bytes;
return m, bytes, ifx m.rendered_kind == .SAME_AS_ORIGINAL then "rendered" else "original", true, "";
}
#scope_module
@ -184,6 +283,106 @@ mcap_read_front :: (path: string, preview := true) -> Mcap, ok: bool, error: str
MCAP_HEADER_SIZE :: 12; // magic and version
MCAP_FRONT_READ_SIZE :: 64 * 1024; // holds the header, text and preview of nearly every capture
MCAP_FRONT_READS :: 16; // reads of the front at most: each reaches past at least one more size field
MCAP_COPY_SIZE :: 4 * 1024 * 1024; // an original is copied this much at a time
// The front of file (length bytes) into m, as mcap_read_front; end, when given, is where it ends.
mcap_read_front_file :: (file: File, length: s64, m: *Mcap, preview: bool, end: *s64 = null) -> ok: bool, error: string {
wanted := min(length, MCAP_FRONT_READ_SIZE);
for attempt: 1..MCAP_FRONT_READS {
if wanted <= 0 return false, "is not an .mcap file (it is empty)";
buffer := NewArray(wanted, u8, initialized = false);
if !file_set_position(file, 0) || !file_read(file, buffer.data, wanted) { array_free(buffer); return false, "cannot be read"; }
d := to_deserializer(buffer);
ok, error := mcap_read_front_part(*d, m, preview);
if ok {
if end end.* = d.cursor;
return true, "";
}
if d.ok || wanted == length { array_free(buffer); return false, error; } // not a matter of too few bytes
// Short: the sizes read so far say how far the front reaches at least. Read that and as much
// again as the first read, so the size fields after it (the OCR, the chunk count, each chunk's
// size) come in the same read rather than one read each.
needed := mcap_front_size(buffer, preview);
array_free(buffer);
if needed <= wanted || needed > length return false, "the file ends early (it is truncated or damaged)";
wanted = min(needed + MCAP_FRONT_READ_SIZE, length);
}
return false, tprint("its front was not all read in % reads", MCAP_FRONT_READS);
}
// The original's size: a u64 from version 4, a u32 before.
mcap_read_original_size :: (d: *Deserializer, version: u32) -> s64 {
if version < 4 return cast(s64) Read(d, u32);
size := Read(d, u64);
if size > 0x7FFF_FFFF_FFFF_FFFF { d.ok = false; return 0; }
return cast,no_check(s64) size;
}
// The parts after the original: the rendered kind and image, then the back chunks, which end d.
mcap_read_after_original :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string {
m.rendered_kind = Read(d, Mcap_Rendered_Kind);
if d.ok && m.rendered_kind == .OWN m.rendered = ReadSizedArray(d, u8, u32);
if d.ok && m.version >= 3 {
ok: bool;
error: string;
m.back_chunks, ok, error = mcap_read_chunks(d);
if !ok return false, error;
}
if !d.ok return false, "the file ends early (it is truncated or damaged)";
if cast(u8) m.rendered_kind > cast(u8) Mcap_Rendered_Kind.OWN return false, tprint("the rendered image is of unknown kind %", cast(u8) m.rendered_kind);
if d.cursor != d.count return false, tprint("% bytes follow the last part", d.count - d.cursor);
return true, "";
}
// Everything up to the original's bytes, for an original of original_size bytes, in a serializer
// with room for extra bytes more.
mcap_serialize_head :: (m: Mcap, original_size: s64, extra: s64 = 0) -> Serializer, ok: bool, error: string {
size := MCAP_HEADER_SIZE + 4 + m.text.count + 4 + m.ocr.count + mcap_chunks_size(m.front_chunks) + 8 + m.preview.png.count + 2 + m.original_name.count + 8;
s := new_serializer(size + extra);
magic := MCAP_MAGIC;
AddArray_NoSize(*s, magic);
Add(*s, cast(u32) MCAP_VERSION);
if !AddString(*s, m.text, u32) return s, false, "the text is too large for its size field";
if !AddString(*s, m.ocr, u32) return s, false, "the OCR text is too large for its size field";
ok, error := mcap_add_chunks(*s, m.front_chunks);
if !ok return s, false, error;
Add(*s, m.preview.width);
Add(*s, m.preview.height);
if !AddSizedArray(*s, m.preview.png, u32) return s, false, "the preview is too large for its size field";
if !AddString(*s, m.original_name, u16) return s, false, "the original's name is too long";
Add(*s, cast(u64) original_size);
assert(s.count == size);
return s, true, "";
}
// The bytes after the original: the rendered kind and image, and the back chunks.
mcap_tail_size :: (m: Mcap) -> s64 {
size := 1 + mcap_chunks_size(m.back_chunks);
if m.rendered_kind == .OWN size += 4 + m.rendered.count;
return size;
}
mcap_add_tail :: (s: *Serializer, m: Mcap) -> ok: bool, error: string {
start := s.count;
Add(s, m.rendered_kind);
if m.rendered_kind == .OWN && !AddSizedArray(s, m.rendered, u32) return false, "the rendered image is too large for its size field";
ok, error := mcap_add_chunks(s, m.back_chunks);
if !ok return false, error;
assert(s.count - start == mcap_tail_size(m));
return true, "";
}
mcap_replace :: (temp_path: string, path: string) -> ok: bool, error: string {
if !atomic_file_replace_or_just_rename_if_destination_file_does_not_exist(temp_path, path) {
file_delete(temp_path);
return false, tprint("cannot move % to %", temp_path, path);
}
return true, "";
}
// Header, text, OCR text, front chunks and, with preview, the preview.
mcap_read_front_part :: (d: *Deserializer, m: *Mcap, preview := true) -> ok: bool, error: string {
@ -295,15 +494,13 @@ mcap_read_chunks :: (d: *Deserializer) -> [] Mcap_Chunk, ok: bool, error: string
// Stores what a scan adds to the .mcap at path in one rewrite: with ocr, that OCR section in place of
// any by the same model (the others kept); and each of front, in place of any front chunk with the
// same tag and key (every other chunk kept, known to this build or not). Upgrades the file to the
// current version. Rewritten beside itself and swapped in (mcap_write), so a failure leaves it as it was.
// current version. Rewritten beside itself and swapped in (mcap_write_streamed, which copies the
// original across without holding it), so a failure leaves it as it was.
mcap_store :: (path: string, ocr: *Mcap_Ocr_Section = null, front: [] Mcap_Chunk = .[]) -> ok: bool, error: string {
data, read_ok := read_entire_file(path, log_errors = false);
if !read_ok return false, "cannot be read";
defer free(data);
m, parsed, parse_error := mcap_parse(cast([] u8) data);
push_allocator(temp);
m, original, parsed, parse_error := mcap_read_without_original(path);
if !parsed return false, parse_error;
push_allocator(temp);
if ocr {
sections, ok, error := mcap_ocr_parse(m.ocr);
if !ok return false, error;
@ -313,11 +510,7 @@ mcap_store :: (path: string, ocr: *Mcap_Ocr_Section = null, front: [] Mcap_Chunk
m.ocr = mcap_ocr_text(sections);
}
m.front_chunks = mcap_chunks_with(m.front_chunks, front);
m.version = MCAP_VERSION;
serializer, serialized, serialize_error := mcap_serialize(m);
if !serialized return false, serialize_error;
written, write_error := mcap_write(path, serializer);
written, write_error := mcap_write_streamed(path, m, original);
return written, write_error;
}
@ -325,13 +518,10 @@ mcap_store :: (path: string, ocr: *Mcap_Ocr_Section = null, front: [] Mcap_Chunk
// tag is one of tags (whatever its key). Everything else stays as it is. The file is rewritten (and
// upgraded to the current version) only when there was something to take out; changed says whether.
mcap_erase :: (path: string, ocr := false, tags: [] string = .[]) -> ok: bool, error: string, changed: bool {
data, read_ok := read_entire_file(path, log_errors = false);
if !read_ok return false, "cannot be read", false;
defer free(data);
m, parsed, parse_error := mcap_parse(cast([] u8) data);
push_allocator(temp);
m, original, parsed, parse_error := mcap_read_without_original(path);
if !parsed return false, parse_error, false;
push_allocator(temp);
changed := false;
if ocr && m.ocr.count { m.ocr = ""; changed = true; }
without :: (chunks: [] Mcap_Chunk, tags: [] string, changed: *bool) -> [] Mcap_Chunk {
@ -348,10 +538,7 @@ mcap_erase :: (path: string, ocr := false, tags: [] string = .[]) -> ok: bool, e
m.back_chunks = without(m.back_chunks, tags, *changed);
if !changed return true, "", false;
m.version = MCAP_VERSION;
serializer, serialized, serialize_error := mcap_serialize(m);
if !serialized return false, serialize_error, false;
written, write_error := mcap_write(path, serializer);
written, write_error := mcap_write_streamed(path, m, original);
return written, write_error, written;
}
@ -361,17 +548,11 @@ mcap_store_chunk :: (path: string, chunk: Mcap_Chunk, back := false) -> ok: bool
ok, error := mcap_store(path, front = .[chunk]);
return ok, error;
}
data, read_ok := read_entire_file(path, log_errors = false);
if !read_ok return false, "cannot be read";
defer free(data);
m, parsed, parse_error := mcap_parse(cast([] u8) data);
if !parsed return false, parse_error;
push_allocator(temp);
m, original, parsed, parse_error := mcap_read_without_original(path);
if !parsed return false, parse_error;
m.back_chunks = mcap_chunks_with(m.back_chunks, .[chunk]);
m.version = MCAP_VERSION;
serializer, serialized, serialize_error := mcap_serialize(m);
if !serialized return false, serialize_error;
written, write_error := mcap_write(path, serializer);
written, write_error := mcap_write_streamed(path, m, original);
return written, write_error;
}

View File

@ -7,13 +7,16 @@
// What the bytes are comes from their first bytes (image_kind), never from a file name, except for
// the camera RAW formats that are TIFF inside: image_kind_from_name tells those apart.
//
// Videos (video.jai) are told apart here too, by their first bytes or their extension; video_poster
// reads a frame from one with FFmpeg, on Windows, and gif_info counts an animated GIF's frames.
//
// Thread-safe: every call stands alone, and WIC is set up and released on the calling thread.
Image_Kind :: enum u8 {
UNKNOWN;
PNG; JPEG; GIF; BMP; PSD; TGA; // stb_image
WEBP; HEIF; AVIF; TIFF; RAW; // WIC
VIDEO; // an MP4 or QuickTime file: not an image
VIDEO; // MP4, QuickTime, Matroska and WebM, AVI, WMV, MPEG: see video_poster
}
// Pixels from image_decode: RGBA, 4 bytes a pixel, rows packed. Free with image_free.
@ -63,10 +66,39 @@ image_kind :: (bytes: [] u8) -> Image_Kind {
if found[Image_Kind.RAW] return .RAW;
return .VIDEO; // an ftyp with no image brand: an MP4, MOV, 3GP and the like
}
if starts(bytes, 4, "moov") || starts(bytes, 4, "mdat") || starts(bytes, 4, "wide") return .VIDEO; // old QuickTime
if starts(bytes, 4, "moov") || starts(bytes, 4, "mdat") || starts(bytes, 4, "wide") || starts(bytes, 4, "free") return .VIDEO; // old QuickTime
if starts(bytes, 0, "\x1A\x45\xDF\xA3") return .VIDEO; // Matroska and WebM (EBML)
if starts(bytes, 0, "RIFF") && starts(bytes, 8, "AVI ") return .VIDEO;
if starts(bytes, 0, "\x30\x26\xB2\x75\x8E\x66\xCF\x11") return .VIDEO; // ASF: WMV
if starts(bytes, 0, "\x00\x00\x01\xBA") || starts(bytes, 0, "\x00\x00\x01\xB3") return .VIDEO; // MPEG program stream, or bare MPEG video
if starts(bytes, 0, "FLV\x01") return .VIDEO;
if bytes.count >= 189 && bytes[0] == 0x47 && bytes[188] == 0x47 return .VIDEO; // MPEG transport stream: a sync byte every 188
if bytes.count >= 197 && bytes[4] == 0x47 && bytes[196] == 0x47 return .VIDEO; // ... with a 4-byte time code before each (.m2ts, .mts)
return .UNKNOWN;
}
// The container a video is in, for people to read: "MP4", "QuickTime", "WebM" ... from its first
// bytes and, where those are shared (MP4 and QuickTime, Matroska and WebM), its extension.
video_format_name :: (head: [] u8, name: string) -> string {
extension := to_lower_copy(path_extension(name),, temp);
starts :: (bytes: [] u8, at: s64, magic: string) -> bool {
return bytes.count >= at + magic.count && memcmp(bytes.data + at, magic.data, magic.count) == 0;
}
if starts(head, 4, "ftyp") {
if starts(head, 8, "qt ") return "QuickTime";
if starts(head, 8, "3g") return "3GP";
return "MP4";
}
if starts(head, 4, "moov") || starts(head, 4, "mdat") || starts(head, 4, "wide") || starts(head, 4, "free") return "QuickTime";
if starts(head, 0, "\x1A\x45\xDF\xA3") return ifx extension == "webm" then "WebM" else "Matroska";
if starts(head, 0, "RIFF") return "AVI";
if starts(head, 0, "\x30\x26\xB2\x75") return "WMV";
if starts(head, 0, "FLV") return "FLV";
if starts(head, 0, "\x00\x00\x01") return "MPEG";
if head.count && (head[0] == 0x47 || (head.count > 4 && head[4] == 0x47)) return "MPEG-TS";
return "video";
}
// The kind for bytes that image_kind cannot place, or places as TIFF, from the file's name: camera RAW
// formats built on TIFF, and TGA, which has no magic of its own. kind is image_kind's answer.
image_kind_from_name :: (kind: Image_Kind, name: string) -> Image_Kind {
@ -217,7 +249,9 @@ png_size :: (png: [] u8) -> width: s64, height: s64, ok: bool {
}
#load "exif.jai";
#load "video.jai";
#if OS == .WINDOWS #load "wic.jai";
#if OS == .WINDOWS #load "video_ffmpeg.jai";
#scope_module

View File

@ -0,0 +1,111 @@
// Videos and animations: a video's poster frame and what its container says (FFmpeg, modules/ffmpeg,
// on Windows), and how many frames an animated GIF has. A video is read from its file, a little at a
// time, never all in memory.
// What a video is, as video_poster finds it. The strings are temp.
Video_Info :: struct {
width, height: s32; // as shown: turned, and stretched by a non-square pixel shape
duration: float64; // seconds; 0 when unknown
frame_rate: float64; // frames a second; 0 when unknown
codec: string; // "h264", "hevc", "vp9" ...
rotation: s32; // degrees clockwise the frames are turned to be shown: 0, 90, 180 or 270
poster_time: float64; // seconds into the video of the poster frame
has_audio: bool;
// From the container, as written; "" when it has none.
creation_date: string; // Apple's com.apple.quicktime.creationdate: local time and its offset, "2024-05-12T10:15:30+0100"
creation_time: string; // creation_time: UTC, "2024-05-12T09:15:30.000000Z"
make, model, software: string;
location: string; // ISO 6709, "+51.5072-000.1276+011.000/"; see iso6709_parse
}
// The frame a video is shown as, turned the right way up, and what it is. The frame is a tenth of
// the way in (at most VIDEO_POSTER_MAX_SECONDS), past the black or blurred first frames phones often
// start with; the first frame of a clip shorter than a second.
VIDEO_POSTER_MAX_SECONDS :: 3.0;
#if OS != .WINDOWS {
video_poster :: (path: string) -> Image, Video_Info, ok: bool, error: string {
return .{}, .{}, false, "videos are only read on Windows";
}
}
// Degrees clockwise to turn a video's frames, from its display matrix (as ffmpeg's
// av_display_rotation_get, whose angle is anticlockwise): 0, 90, 180 or 270.
display_matrix_rotation :: (m: [9] s32) -> s32 {
fixed :: (v: s32) -> float64 { return cast(float64) v / 65536.0; }
scale0 := sqrt(fixed(m[0]) * fixed(m[0]) + fixed(m[3]) * fixed(m[3]));
scale1 := sqrt(fixed(m[1]) * fixed(m[1]) + fixed(m[4]) * fixed(m[4]));
if scale0 == 0 || scale1 == 0 return 0;
anticlockwise := -atan2(fixed(m[1]) / scale1, fixed(m[0]) / scale0) * 180 / PI;
clockwise := cast(s32) floor(-anticlockwise / 90 + 0.5) * 90;
clockwise %= 360;
if clockwise < 0 clockwise += 360;
return clockwise;
}
// "+51.5072-000.1276+011.000/" (ISO 6709: latitude, longitude, then an altitude when there is one)
// as numbers. ok is false when it is not that.
iso6709_parse :: (text: string) -> latitude: float64, longitude: float64, altitude: float64, has_altitude: bool, ok: bool {
values: [3] float64;
count := 0;
at := 0;
while at < text.count && count < 3 {
c := text[at];
if c == #char "/" break;
if c != #char "+" && c != #char "-" return 0, 0, 0, false, false;
end := at + 1;
while end < text.count && (is_digit(text[end]) || text[end] == #char ".") end += 1;
if end == at + 1 return 0, 0, 0, false, false;
value, value_ok := string_to_float64(slice(text, at, end - at));
if !value_ok return 0, 0, 0, false, false;
values[count] = value;
count += 1;
at = end;
}
if count < 2 || values[0] < -90 || values[0] > 90 || values[1] < -180 || values[1] > 180 return 0, 0, 0, false, false;
return values[0], values[1], values[2], count == 3, true;
}
// How many frames a GIF has, and how long it runs in milliseconds, from its blocks; no pixels are
// decoded. A frame with no delay, or 10 ms, shows for 100 ms, as browsers show it. ok is false
// when the bytes are not a GIF; one cut short counts the frames up to where it ends.
gif_info :: (bytes: [] u8) -> frames: s64, duration_ms: s64, ok: bool {
if bytes.count < 13 || (memcmp(bytes.data, "GIF87a".data, 6) != 0 && memcmp(bytes.data, "GIF89a".data, 6) != 0) return 0, 0, false;
at := 13;
flags := bytes[10];
if flags & 0x80 at += 3 * (1 << ((flags & 7) + 1)); // the global colour table
frames, duration := 0, 0;
delay := 0;
skip_blocks :: (bytes: [] u8, at: s64) -> s64 { // data sub-blocks, to the empty one that ends them
while at < bytes.count {
size := cast(s64) bytes[at];
at += 1 + size;
if size == 0 break;
}
return at;
}
while at < bytes.count {
if bytes[at] == {
case 0x21; // an extension
if at + 1 >= bytes.count break;
if bytes[at + 1] == 0xF9 && at + 6 < bytes.count { // graphic control: the next frame's delay, in hundredths
delay = (cast(s64) bytes[at + 4]) | ((cast(s64) bytes[at + 5]) << 8);
}
at = skip_blocks(bytes, at + 2);
case 0x2C; // a frame
if at + 10 > bytes.count break;
local := bytes[at + 9];
at += 10;
if local & 0x80 at += 3 * (1 << ((local & 7) + 1));
at = skip_blocks(bytes, at + 1); // after the LZW code size
if at > bytes.count break;
frames += 1;
duration += ifx delay <= 1 then 100 else delay * 10;
delay = 0;
case; // 0x3B ends it; anything else is damage
break;
}
}
return frames, duration, true;
}

View File

@ -0,0 +1,176 @@
// video_poster on Windows, with FFmpeg (modules/ffmpeg). See video.jai.
video_poster :: (path: string) -> Image, Video_Info, ok: bool, error: string {
image: Image;
info: Video_Info;
av_log_set_level(AV_LOG_ERROR); // FFmpeg's notes would go to the console
format: *AVFormatContext;
result := avformat_open_input(*format, temp_c_string(path), null, null); // UTF-8 paths, on Windows too
if result < 0 return image, info, false, av_text("FFmpeg cannot open it", result);
defer avformat_close_input(*format);
result = avformat_find_stream_info(format, null);
if result < 0 return image, info, false, av_text("FFmpeg cannot find its streams", result);
codec: *AVCodec;
index := av_find_best_stream(format, .VIDEO, -1, -1, *codec, 0);
if index < 0 || !codec return image, info, false, ifx index == AVERROR_DECODER_NOT_FOUND then "FFmpeg has no decoder for its video" else "it has no video stream";
stream := format.streams[index];
for 0..format.nb_streams - 1 if format.streams[it].codecpar.codec_type == .AUDIO info.has_audio = true;
info.codec = to_string(avcodec_get_name(stream.codecpar.codec_id));
if format.duration > 0 && format.duration != AV_NOPTS info.duration = cast(float64) format.duration / AV_TIME_BASE;
else if stream.duration > 0 && stream.duration != AV_NOPTS info.duration = cast(float64) stream.duration * rational(stream.time_base);
if stream.avg_frame_rate.num > 0 && stream.avg_frame_rate.den > 0 info.frame_rate = rational(stream.avg_frame_rate);
else if stream.r_frame_rate.num > 0 && stream.r_frame_rate.den > 0 info.frame_rate = rational(stream.r_frame_rate);
side_size: u64;
matrix := cast(*s32) av_stream_get_side_data(stream, .DISPLAYMATRIX, *side_size);
if matrix && side_size >= 9 * size_of(s32) {
m: [9] s32;
memcpy(m.data, matrix, size_of(type_of(m)));
info.rotation = display_matrix_rotation(m);
}
meta :: (format: *AVFormatContext, stream: *AVStream, key: string) -> string {
for dictionary: *AVDictionary.[format.metadata, stream.metadata] {
entry := av_dict_get(dictionary, temp_c_string(key), null, 0);
if entry && entry.value && entry.value[0] return copy_string(to_string(entry.value),, temp);
}
return "";
}
info.creation_date = meta(format, stream, "com.apple.quicktime.creationdate");
info.creation_time = meta(format, stream, "creation_time");
info.make = meta(format, stream, "com.apple.quicktime.make");
if !info.make info.make = meta(format, stream, "make");
if !info.make info.make = meta(format, stream, "com.android.manufacturer");
info.model = meta(format, stream, "com.apple.quicktime.model");
if !info.model info.model = meta(format, stream, "model");
if !info.model info.model = meta(format, stream, "com.android.model");
info.software = meta(format, stream, "com.apple.quicktime.software");
if !info.software info.software = meta(format, stream, "encoder");
info.location = meta(format, stream, "com.apple.quicktime.location.ISO6709");
if !info.location info.location = meta(format, stream, "location");
// The decoder.
decoder := avcodec_alloc_context3(codec);
if !decoder return image, info, false, "FFmpeg could not make a decoder";
defer avcodec_free_context(*decoder);
result = avcodec_parameters_to_context(decoder, stream.codecpar);
if result < 0 return image, info, false, av_text("FFmpeg cannot set up its decoder", result);
decoder.thread_count = 2; // several imports run at once
result = avcodec_open2(decoder, codec, null);
if result < 0 return image, info, false, av_text(tprint("FFmpeg cannot open its % decoder", info.codec), result);
// Seek to the poster's time (to the key frame before it), then decode up to it.
if info.duration >= 1 info.poster_time = min(info.duration / 10, VIDEO_POSTER_MAX_SECONDS);
target: s64 = AV_NOPTS;
if info.poster_time > 0 && stream.time_base.num > 0 {
target = cast(s64) (info.poster_time / rational(stream.time_base));
if stream.start_time != AV_NOPTS target += stream.start_time;
if av_seek_frame(format, index, target, AVSEEK_FLAG_BACKWARD) < 0 {
// Cannot seek (some AVI, MPEG): the first frame, then.
target = AV_NOPTS;
info.poster_time = 0;
av_seek_frame(format, index, 0, AVSEEK_FLAG_BACKWARD);
}
avcodec_flush_buffers(decoder);
}
packet := av_packet_alloc();
defer av_packet_free(*packet);
frame := av_frame_alloc();
defer av_frame_free(*frame);
got := av_frame_alloc(); // the last frame decoded: the poster, once it reaches the target
defer av_frame_free(*got);
have := false;
packets := 0;
ended := false; // the end of the file was sent to the decoder
while true {
// The frames the decoder has, until it wants another packet.
if avcodec_receive_frame(decoder, frame) == 0 {
av_frame_unref(got);
av_frame_move_ref(got, frame);
have = true;
if target == AV_NOPTS || got.best_effort_timestamp == AV_NOPTS || got.best_effort_timestamp >= target break;
continue;
}
if ended || packets >= VIDEO_MAX_PACKETS break;
// The next packet of the video.
while true {
if av_read_frame(format, packet) < 0 {
avcodec_send_packet(decoder, null); // the end: the frames the decoder still holds
ended = true;
break;
}
ours := packet.stream_index == index;
if ours avcodec_send_packet(decoder, packet);
av_packet_unref(packet);
if ours { packets += 1; break; }
}
}
if !have return image, info, false, tprint("FFmpeg decoded no frame of its % video", info.codec);
if got.best_effort_timestamp != AV_NOPTS && stream.time_base.num > 0 {
start := ifx stream.start_time != AV_NOPTS then stream.start_time else 0;
info.poster_time = max(cast(float64) 0, cast(float64) (got.best_effort_timestamp - start) * rational(stream.time_base));
}
// To RGBA, stretched to square pixels when they are not.
w, h := got.width, got.height;
if w <= 0 || h <= 0 return image, info, false, "its frame has no size";
out_w, out_h := w, h;
aspect := got.sample_aspect_ratio;
if aspect.num > 0 && aspect.den > 0 && aspect.num != aspect.den out_w = max(cast(s32) 1, cast(s32) (cast(float64) w * rational(aspect) + 0.5));
if cast(s64) out_w * out_h * 4 > S32_MAX return image, info, false, tprint("its frames are % x %, too large", out_w, out_h);
scaler := sws_getContext(w, h, cast(AVPixelFormat) got.format, out_w, out_h, .RGBA, SWS_BILINEAR, null, null, null);
if !scaler return image, info, false, tprint("FFmpeg cannot convert its frames (pixel format %)", got.format);
defer sws_freeContext(scaler);
// swscale writes with SIMD, a row at a time, past the last pixel of a row and of the image: into a
// buffer of its own allocator (aligned), its rows padded and a spare row after them, then copied
// out packed. An exact-sized buffer of ours let it write past the end, a crash in swscale.
packed_stride := cast(s64) out_w * 4;
stride := (packed_stride + SCALE_ALIGN - 1) / SCALE_ALIGN * SCALE_ALIGN;
scaled := cast(*u8) av_malloc(cast(u64) (stride * (out_h + 1) + SCALE_ALIGN));
if !scaled return image, info, false, tprint("there is no memory for its % x % frame", out_w, out_h);
defer av_free(scaled);
planes: [4] *u8;
strides: [4] s32;
planes[0] = scaled;
strides[0] = cast(s32) stride;
rows := sws_scale(scaler, got.data.data, got.linesize.data, 0, h, planes.data, strides.data);
if rows <= 0 return image, info, false, "FFmpeg could not convert its frame";
pixels := cast(*u8) alloc(packed_stride * out_h,, context.default_allocator);
for y: 0..out_h - 1 memcpy(pixels + y * packed_stride, scaled + y * stride, packed_stride);
image = .{pixels = pixels, width = out_w, height = out_h};
if info.rotation == {
case 90; image_orient(*image, 6); // turned clockwise, as EXIF orientation 6
case 180; image_orient(*image, 3);
case 270; image_orient(*image, 8);
}
info.width, info.height = image.width, image.height;
return image, info, true, "";
}
#scope_file
SCALE_ALIGN :: 64; // bytes: swscale's widest SIMD writes
VIDEO_MAX_PACKETS :: 3000; // packets decoded to reach the poster at most: a key frame every few seconds is usual
AV_NOPTS :: cast,no_check(s64) AV_NOPTS_VALUE;
AVERROR_DECODER_NOT_FOUND :: -1128613112; // FFERRTAG(0xF8, 'D', 'E', 'C')
rational :: (r: AVRational) -> float64 {
if r.den == 0 return 0;
return cast(float64) r.num / r.den;
}
av_text :: (what: string, error: s32) -> string {
buffer: [256] u8;
av_strerror(error, buffer.data, buffer.count);
return tprint("% (%)", what, to_string(buffer.data));
}
#import "ffmpeg";

25
modules/ffmpeg/README.md Normal file
View File

@ -0,0 +1,25 @@
# FFmpeg module for jai
FFmpeg bindings for jai and some examples decoding and play videos.
## Setup
The FFmpeg libaries and header files are not included in this repo as I didn't want to have to deal with license
things. Unfortunately this means that there is some labour to get things working. The approximate steps are:
- Download "shared" binaries. I got them from [here](https://www.gyan.dev/ffmpeg/builds/#release-builds). Eg
"ffmpeg-release-full-shared.7z". At the time of writing the latest release build was version 6.0.
- Copy all the directories in the Include directory to ffmpeg/src. There will then be, eg, a
ffmpeg/src/libavcodec/codec.h file.
- Copy all the dlls from the bin directory to ffmpeg/windows
- Copy all the libs from the lib directory to ffmpeg/windows. You may then need to rename the lib files so the
names match the versioned dll names. (This may not actually be necessary, but that is how I generated bindings.
I'm not worrying about it as probably it makes more sense to some day work out how to compile the libraries
myself so they don't include verion names in them).
There may be additional work for platforms other than windows, I only tried on windows!
You may need to add aditional source files in generate.jai if bindings you want are not present.
You will also need to copy the dlls into whatever directory your exe is in.

266
modules/ffmpeg/generate.jai Normal file
View File

@ -0,0 +1,266 @@
// Generate bindings for ffmpeg.
//
// @TODO:
// - Get rid of the dlls having version number in their name, it's annoying. Probably need to compile ffmpeg
// myself to do that...
// - Fix warning about generating type info for large ffmpeg struct. Probably just don't generate type info
// for any ffmpeg types? (Hmm, type info is probably useful for the smaller types, I think it's needed to
// be able to print % structs out nicely).
//
AT_COMPILE_TIME :: true;
#if AT_COMPILE_TIME {
#run {
set_build_options_dc(.{do_output=false});
if !generate_bindings() {
compiler_set_workspace_status(.FAILED);
}
}
} else {
#import "System";
main :: () {
set_working_directory(path_strip_filename(get_path_of_running_executable()));
if !generate_bindings() {
exit(1);
}
}
}
HEADER :: #string DONE
AV_NOPTS_VALUE :: 0x8000000000000000;
DONE
FOOTER :: #string DONE
// Bindings generator stopped generating this for some reason?
// Looks like it only adds the Basic import if you have the struct checks on, but it is also
// needed for print like functions.
// @TODO: Test and report if so.
//
// Needed to compile print like log functions (although not sure you'll ever really want the av log functions?).
#import "Basic";
DONE
INCLUDE_PATH :: "src";
generate_bindings :: () -> bool {
output_filename := "windows.jai";
opts: Generate_Bindings_Options;
{
using opts;
try_to_preserve_comments = true; // @Tmp
generate_compile_time_struct_checks = false;
// generate_printf_wrappers = true;
// c_enum_emulation = true;
// auto_detect_enum_prefixes = false;
// strip_enum_prefixes = false;
// alias_original_enum_names = true;
visitor = my_visitor;
// Can't load all the libraries for some reason, just disable stripping thingz. Nevermind, this
// doesn't work, just get a load of __UnknownLib dont you :roll:.
// strip_flags &= ~.SYMBOLS_WITH_UNKNOWN_FOREIGN_LIBS;
array_add(*libpaths, "windows");
libraries := string.[
"avcodec-60",
"avdevice-60",
"avfilter-9",
"avformat-60",
"avutil-58",
"postproc-57",
"swresample-4",
"swscale-7",
];
for libraries array_add(*libnames, it);
array_add(*include_paths, INCLUDE_PATH);
// An attmept to just add all includes. For some great reason the headers also include a bunch of
// third party headers not included with the ffmpeg headers...
// I could just make empty dummy headers for these things. But instead I am just going to manually
// pick the headers I want!
#if false {
dirs_to_visit: [..] string;
for libraries array_add(*dirs_to_visit, tprint("%/lib%", INCLUDE_PATH, it));
file_visitor :: (info: *File_Visit_Info, source_files : *[..] string) {
assert(info != null, "info (%) != null", info);
assert(source_files != null, "modules (%) != null", source_files);
assert(!info.is_directory);
assert(has_extension(info.short_name, ".h"));
if ends_with(info.short_name, "qsv.h") {
print("Skipping % because it contains an include to mfx/mfxvideo.h that I don't want to deal with right now!\n", info.short_name);
} else if ends_with(info.short_name, "vdpau.h") {
print("More fuckers\n");
} else {
array_add(source_files, copy_string(info.full_name));
}
}
for dirs_to_visit {
print("Visiting: %\n", it);
visit_files(it, false, *source_files, file_visitor, true, false);
}
#if false {
print("Found % files: \n", source_files.count);
for source_files print(" %\n", it);
}
}
array_add(*source_files, tprint("%/libavcodec/avcodec.h", INCLUDE_PATH));
array_add(*source_files, tprint("%/libavformat/avformat.h", INCLUDE_PATH));
array_add(*source_files, tprint("%/libswscale/swscale.h", INCLUDE_PATH));
array_add(*system_include_paths, GENERATOR_DEFAULT_SYSTEM_INCLUDE_PATH);
builder: String_Builder;
append(*builder, HEADER);
generate_error_codes(*builder);
header = builder_to_string(*builder);
footer = FOOTER;
}
print("Generating ffmpeg bindings...\n");
result := generate_bindings(opts, output_filename);
return result;
}
DECLARATIONS_TO_OMIT :: string.[
"av_log_set_callback",
"av_fopen_utf8",
"av_pkt_dump2",
"av_hex_dump",
];
Pair :: struct {
type: string;
replacement: string;
}
TYPE_REPLACEMENTS_TO_MAKE :: Pair.[
Pair.{"__some_type", "some_cooler_type"},
];
// This one warns about being large. Probably can also get rid of others?
TYPES_WITHOUT_TYPE_INFO :: string.[
"AVCodecContext",
];
// NOTE(Charles): Most of this stuff doesn't acheive anything. I just yoinked it from other examples of
// visitor functions for reference.
my_visitor :: (decl: *Declaration, parent_decl: *Declaration) -> Declaration_Visit_Result {
if decl.kind == .MACRO_DEFINITION && (ends_with(decl.name, "_H") || ends_with(decl.name, "_h_")) {
print("MACRO_DEFINITION removed: %\n", decl.name);
decl.decl_flags |= .OMIT_FROM_OUTPUT;
return .STOP;
}
if !parent_decl {
if array_find(DECLARATIONS_TO_OMIT, decl.name) {
print("DECLARATIONS_TO_OMIT: %\n", decl.name);
decl.decl_flags |= .OMIT_FROM_OUTPUT;
return .STOP;
}
if decl.kind == .TYPEDEF {
for TYPE_REPLACEMENTS_TO_MAKE if decl.name == it.type {
print("TYPE_REPLACEMENTS_TO_MAKE: % -> %\n", it.type, it.replacement);
decl.output_name = it.replacement;
decl.decl_flags |= .OMIT_FROM_OUTPUT;
return .STOP;
}
}
if array_find(TYPES_WITHOUT_TYPE_INFO, decl.name) {
decl.decl_flags |= .TYPE_INFO_NONE;
}
}
return .RECURSE;
}
// ffmpeg defines error codes using some weird ass macros. Need to manually define them. See libavutil/error.h.
generate_error_codes :: (b: *String_Builder) {
fferrtag :: (a: u32, b: u32, c: u32, d: u32) -> s32 #expand {
// #define MKTAG(a,b,c,d) ((a) | ((b) << 8) | ((c) << 16) | ((unsigned)(d) << 24))
// #define FFERRTAG(a, b, c, d) (-(int)MKTAG(a, b, c, d))
tag : u32 = a | (b << 8) | (c << 16) | (d << 24);
return -cast(s32) tag;
}
Av_Error_Codes :: struct {
AVERROR_BSF_NOT_FOUND := #run fferrtag( 0xF8, #char "B", #char "S", #char "F"); ///< Bitstream filter not found
AVERROR_BUG := #run fferrtag(#char "B", #char "U", #char "G", #char "!"); ///< Internal bug, also see AVERROR_BUG2
AVERROR_BUFFER_TOO_SMALL := #run fferrtag(#char "B", #char "U", #char "F", #char "S"); ///< Buffer too small
AVERROR_DECODER_NOT_FOUND := #run fferrtag( 0xF8, #char "D", #char "E", #char "C"); ///< Decoder not found
AVERROR_DEMUXER_NOT_FOUND := #run fferrtag( 0xF8, #char "D", #char "E", #char "M"); ///< Demuxer not found
AVERROR_ENCODER_NOT_FOUND := #run fferrtag( 0xF8, #char "E", #char "N", #char "C"); ///< Encoder not found
AVERROR_EOF := #run fferrtag(#char "E", #char "O", #char "F", #char " "); ///< End of file
AVERROR_EXIT := #run fferrtag(#char "E", #char "X", #char "I", #char "T"); ///< Immediate exit was requested; the called function should not be restarted
AVERROR_EXTERNAL := #run fferrtag(#char "E", #char "X", #char "T", #char " "); ///< Generic error in an external library
AVERROR_FILTER_NOT_FOUND := #run fferrtag( 0xF8, #char "F", #char "I", #char "L"); ///< Filter not found
AVERROR_INVALIDDATA := #run fferrtag(#char "I", #char "N", #char "D", #char "A"); ///< Invalid data found when processing input
AVERROR_MUXER_NOT_FOUND := #run fferrtag( 0xF8, #char "M", #char "U", #char "X"); ///< Muxer not found
AVERROR_OPTION_NOT_FOUND := #run fferrtag( 0xF8, #char "O", #char "P", #char "T"); ///< Option not found
AVERROR_PATCHWELCOME := #run fferrtag(#char "P", #char "A", #char "W", #char "E"); ///< Not yet implemented in FFmpeg, patches welcome
AVERROR_PROTOCOL_NOT_FOUND := #run fferrtag( 0xF8, #char "P", #char "R", #char "O"); ///< Protocol not found
AVERROR_STREAM_NOT_FOUND := #run fferrtag( 0xF8, #char "S", #char "T", #char "R"); ///< Stream not found
AVERROR_BUG2 := #run fferrtag(#char "B", #char "U", #char "G", #char " ");
AVERROR_UNKNOWN := #run fferrtag(#char "U", #char "N", #char "K", #char "N"); ///< Unknown error, typically from an external library
// AVERROR_EXPERIMENTAL (-0x2bb2afa8) ///< Requested feature is flagged experimental. Set strict_std_compliance if you really want to use it.
// AVERROR_INPUT_CHANGED (-0x636e6701) ///< Input changed between calls. Reconfiguration is required. (can be OR-ed with AVERROR_OUTPUT_CHANGED)
// AVERROR_OUTPUT_CHANGED (-0x636e6702) ///< Output changed between calls. Reconfiguration is required. (can be OR-ed with AVERROR_INPUT_CHANGED)
// HTTP & RTSP errors
AVERROR_HTTP_BAD_REQUEST := #run fferrtag(0xF8, #char "4", #char "0", #char "0");
AVERROR_HTTP_UNAUTHORIZED := #run fferrtag(0xF8, #char "4", #char "0", #char "1");
AVERROR_HTTP_FORBIDDEN := #run fferrtag(0xF8, #char "4", #char "0", #char "3");
AVERROR_HTTP_NOT_FOUND := #run fferrtag(0xF8, #char "4", #char "0", #char "4");
AVERROR_HTTP_OTHER_4XX := #run fferrtag(0xF8, #char "4", #char "X", #char "X");
AVERROR_HTTP_SERVER_ERROR := #run fferrtag(0xF8, #char "5", #char "X", #char "X");
}
// My plan was to define constants on a struct, loop of the struct members and print out the
// name :: value's. I couldn't see a way to get the values of constant struct members like that though, so
// instead I just declared them as fields and used a dummy instance. Probably there is a smarter thing to do
// here!
dummy: Av_Error_Codes;
info_struct := type_info(Av_Error_Codes);
longest_name_length := 0;
for info_struct.members longest_name_length = max(longest_name_length, it.name.count);
for info_struct.members {
slot := cast (*u8) *dummy + it.offset_in_bytes;
value := << cast(*s32) slot;
append(b, it.name);
num_spaces := longest_name_length - it.name.count;
for 1..num_spaces append(b, " ");
print_to_builder(b, " :: 0x%; // %\n", formatInt(value, base = 16, minimum_digits = 8), value);
}
}
has_extension :: (name: string, extension: string) -> bool {
if name.count < extension.count return false;
test := name;
advance(*test, name.count - extension.count);
return test == extension;
}
#import "Basic";
#import "String";
#import "File_Utilities";
#import "Compiler";
#import "Bindings_Generator";
#import "File";

13
modules/ffmpeg/module.jai Normal file
View File

@ -0,0 +1,13 @@
// Bindings for ffmpeg.
//
#scope_module
size_t :: u64;
#if OS == .WINDOWS {
#load "windows.jai";
} else {
#assert false, "Only windows has bindings atm, sorry bros!";
}

12938
modules/ffmpeg/windows.jai Normal file

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,780 @@
FFmpeg 64-bit shared Windows build from www.gyan.dev
Version: 6.0-full_build-www.gyan.dev
License: GPL v3
Source Code: https://github.com/FFmpeg/FFmpeg/commit/ea3d24bbe3
External Assets
frei0r plugins: https://www.gyan.dev/ffmpeg/builds/ffmpeg-frei0r-plugins
lensfun database: https://www.gyan.dev/ffmpeg/builds/ffmpeg-lensfun-db
release-full build configuration:
ARCH x86 (generic)
big-endian no
runtime cpu detection yes
standalone assembly yes
x86 assembler nasm
MMX enabled yes
MMXEXT enabled yes
3DNow! enabled yes
3DNow! extended enabled yes
SSE enabled yes
SSSE3 enabled yes
AESNI enabled yes
AVX enabled yes
AVX2 enabled yes
AVX-512 enabled yes
AVX-512ICL enabled yes
XOP enabled yes
FMA3 enabled yes
FMA4 enabled yes
i686 features enabled yes
CMOV is fast yes
EBX available yes
EBP available yes
debug symbols yes
strip symbols yes
optimize for size no
optimizations yes
static no
shared yes
postprocessing support yes
network support yes
threading support pthreads
safe bitstream reader yes
texi2html enabled no
perl enabled yes
pod2man enabled yes
makeinfo enabled yes
makeinfo supports HTML yes
xmllint enabled yes
External libraries:
avisynth libilbc libtheora
bzlib libjxl libtwolame
chromaprint liblensfun libuavs3d
frei0r libmodplug libvidstab
gmp libmp3lame libvmaf
gnutls libmysofa libvo_amrwbenc
iconv libopencore_amrnb libvorbis
ladspa libopencore_amrwb libvpx
libaom libopenjpeg libwebp
libaribb24 libopenmpt libx264
libass libopus libx265
libbluray libplacebo libxavs2
libbs2b librav1e libxml2
libcaca librist libxvid
libcdio librubberband libzimg
libdav1d libshaderc libzmq
libdavs2 libshine libzvbi
libflite libsnappy lzma
libfontconfig libsoxr mediafoundation
libfreetype libspeex sdl2
libfribidi libsrt zlib
libgme libssh
libgsm libsvtav1
External libraries providing hardware acceleration:
amf dxva2 nvenc
cuda ffnvcodec opencl
cuda_llvm libmfx vulkan
cuvid libvpl
d3d11va nvdec
Libraries:
avcodec avformat swresample
avdevice avutil swscale
avfilter postproc
Programs:
ffmpeg ffplay ffprobe
Enabled decoders:
aac fraps pcm_u8
aac_fixed frwu pcm_vidc
aac_latm ftr pcx
aasc g2m pfm
ac3 g723_1 pgm
ac3_fixed g729 pgmyuv
acelp_kelvin gdv pgssub
adpcm_4xm gem pgx
adpcm_adx gif phm
adpcm_afc gremlin_dpcm photocd
adpcm_agm gsm pictor
adpcm_aica gsm_ms pixlet
adpcm_argo h261 pjs
adpcm_ct h263 png
adpcm_dtk h263i ppm
adpcm_ea h263p prores
adpcm_ea_maxis_xa h264 prosumer
adpcm_ea_r1 h264_cuvid psd
adpcm_ea_r2 h264_qsv ptx
adpcm_ea_r3 hap qcelp
adpcm_ea_xas hca qdm2
adpcm_g722 hcom qdmc
adpcm_g726 hdr qdraw
adpcm_g726le hevc qoi
adpcm_ima_acorn hevc_cuvid qpeg
adpcm_ima_alp hevc_qsv qtrle
adpcm_ima_amv hnm4_video r10k
adpcm_ima_apc hq_hqa r210
adpcm_ima_apm hqx ra_144
adpcm_ima_cunning huffyuv ra_288
adpcm_ima_dat4 hymt ralf
adpcm_ima_dk3 iac rasc
adpcm_ima_dk4 idcin rawvideo
adpcm_ima_ea_eacs idf realtext
adpcm_ima_ea_sead iff_ilbm rka
adpcm_ima_iss ilbc rl2
adpcm_ima_moflex imc roq
adpcm_ima_mtf imm4 roq_dpcm
adpcm_ima_oki imm5 rpza
adpcm_ima_qt indeo2 rscc
adpcm_ima_rad indeo3 rv10
adpcm_ima_smjpeg indeo4 rv20
adpcm_ima_ssi indeo5 rv30
adpcm_ima_wav interplay_acm rv40
adpcm_ima_ws interplay_dpcm s302m
adpcm_ms interplay_video sami
adpcm_mtaf ipu sanm
adpcm_psx jacosub sbc
adpcm_sbpro_2 jpeg2000 scpr
adpcm_sbpro_3 jpegls screenpresso
adpcm_sbpro_4 jv sdx2_dpcm
adpcm_swf kgv1 sga
adpcm_thp kmvc sgi
adpcm_thp_le lagarith sgirle
adpcm_vima libaom_av1 sheervideo
adpcm_xa libaribb24 shorten
adpcm_xmd libdav1d simbiosis_imx
adpcm_yamaha libdavs2 sipr
adpcm_zork libgsm siren
agm libgsm_ms smackaud
aic libilbc smacker
alac libjxl smc
alias_pix libopencore_amrnb smvjpeg
als libopencore_amrwb snow
amrnb libopenjpeg sol_dpcm
amrwb libopus sonic
amv libspeex sp5x
anm libuavs3d speedhq
ansi libvorbis speex
anull libvpx_vp8 srgc
apac libvpx_vp9 srt
ape libzvbi_teletext ssa
apng loco stl
aptx lscr subrip
aptx_hd m101 subviewer
arbc mace3 subviewer1
argo mace6 sunrast
ass magicyuv svq1
asv1 mdec svq3
asv2 media100 tak
atrac1 metasound targa
atrac3 microdvd targa_y216
atrac3al mimic tdsc
atrac3p misc4 text
atrac3pal mjpeg theora
atrac9 mjpeg_cuvid thp
aura mjpeg_qsv tiertexseqvideo
aura2 mjpegb tiff
av1 mlp tmv
av1_cuvid mmvideo truehd
av1_qsv mobiclip truemotion1
avrn motionpixels truemotion2
avrp movtext truemotion2rt
avs mp1 truespeech
avui mp1float tscc
ayuv mp2 tscc2
bethsoftvid mp2float tta
bfi mp3 twinvq
bink mp3adu txd
binkaudio_dct mp3adufloat ulti
binkaudio_rdft mp3float utvideo
bintext mp3on4 v210
bitpacked mp3on4float v210x
bmp mpc7 v308
bmv_audio mpc8 v408
bmv_video mpeg1_cuvid v410
bonk mpeg1video vb
brender_pix mpeg2_cuvid vble
c93 mpeg2_qsv vbn
cavs mpeg2video vc1
cbd2_dpcm mpeg4 vc1_cuvid
ccaption mpeg4_cuvid vc1_qsv
cdgraphics mpegvideo vc1image
cdtoons mpl2 vcr1
cdxl msa1 vmdaudio
cfhd mscc vmdvideo
cinepak msmpeg4v1 vmnc
clearvideo msmpeg4v2 vnull
cljr msmpeg4v3 vorbis
cllc msnsiren vp3
comfortnoise msp2 vp4
cook msrle vp5
cpia mss1 vp6
cri mss2 vp6a
cscd msvideo1 vp6f
cyuv mszh vp7
dca mts2 vp8
dds mv30 vp8_cuvid
derf_dpcm mvc1 vp8_qsv
dfa mvc2 vp9
dfpwm mvdv vp9_cuvid
dirac mvha vp9_qsv
dnxhd mwsc vplayer
dolby_e mxpeg vqa
dpx nellymoser vqc
dsd_lsbf notchlc wady_dpcm
dsd_lsbf_planar nuv wavarc
dsd_msbf on2avc wavpack
dsd_msbf_planar opus wbmp
dsicinaudio paf_audio wcmv
dsicinvideo paf_video webp
dss_sp pam webvtt
dst pbm wmalossless
dvaudio pcm_alaw wmapro
dvbsub pcm_bluray wmav1
dvdsub pcm_dvd wmav2
dvvideo pcm_f16le wmavoice
dxa pcm_f24le wmv1
dxtory pcm_f32be wmv2
dxv pcm_f32le wmv3
eac3 pcm_f64be wmv3image
eacmv pcm_f64le wnv1
eamad pcm_lxf wrapped_avframe
eatgq pcm_mulaw ws_snd1
eatgv pcm_s16be xan_dpcm
eatqi pcm_s16be_planar xan_wc3
eightbps pcm_s16le xan_wc4
eightsvx_exp pcm_s16le_planar xbin
eightsvx_fib pcm_s24be xbm
escape124 pcm_s24daud xface
escape130 pcm_s24le xl
evrc pcm_s24le_planar xma1
exr pcm_s32be xma2
fastaudio pcm_s32le xpm
ffv1 pcm_s32le_planar xsub
ffvhuff pcm_s64be xwd
ffwavesynth pcm_s64le y41p
fic pcm_s8 ylc
fits pcm_s8_planar yop
flac pcm_sga yuv4
flashsv pcm_u16be zero12v
flashsv2 pcm_u16le zerocodec
flic pcm_u24be zlib
flv pcm_u24le zmbv
fmvc pcm_u32be
fourxm pcm_u32le
Enabled encoders:
a64multi hevc_qsv pcm_u16le
a64multi5 huffyuv pcm_u24be
aac jpeg2000 pcm_u24le
aac_mf jpegls pcm_u32be
ac3 libaom_av1 pcm_u32le
ac3_fixed libgsm pcm_u8
ac3_mf libgsm_ms pcm_vidc
adpcm_adx libilbc pcx
adpcm_argo libjxl pfm
adpcm_g722 libmp3lame pgm
adpcm_g726 libopencore_amrnb pgmyuv
adpcm_g726le libopenjpeg phm
adpcm_ima_alp libopus png
adpcm_ima_amv librav1e ppm
adpcm_ima_apm libshine prores
adpcm_ima_qt libspeex prores_aw
adpcm_ima_ssi libsvtav1 prores_ks
adpcm_ima_wav libtheora qoi
adpcm_ima_ws libtwolame qtrle
adpcm_ms libvo_amrwbenc r10k
adpcm_swf libvorbis r210
adpcm_yamaha libvpx_vp8 ra_144
alac libvpx_vp9 rawvideo
alias_pix libwebp roq
amv libwebp_anim roq_dpcm
anull libx264 rpza
apng libx264rgb rv10
aptx libx265 rv20
aptx_hd libxavs2 s302m
ass libxvid sbc
asv1 ljpeg sgi
asv2 magicyuv smc
av1_amf mjpeg snow
av1_nvenc mjpeg_qsv sonic
av1_qsv mlp sonic_ls
avrp movtext speedhq
avui mp2 srt
ayuv mp2fixed ssa
bitpacked mp3_mf subrip
bmp mpeg1video sunrast
cfhd mpeg2_qsv svq1
cinepak mpeg2video targa
cljr mpeg4 text
comfortnoise msmpeg4v2 tiff
dca msmpeg4v3 truehd
dfpwm msvideo1 tta
dnxhd nellymoser ttml
dpx opus utvideo
dvbsub pam v210
dvdsub pbm v308
dvvideo pcm_alaw v408
eac3 pcm_bluray v410
exr pcm_dvd vbn
ffv1 pcm_f32be vc2
ffvhuff pcm_f32le vnull
fits pcm_f64be vorbis
flac pcm_f64le vp9_qsv
flashsv pcm_mulaw wavpack
flashsv2 pcm_s16be wbmp
flv pcm_s16be_planar webvtt
g723_1 pcm_s16le wmav1
gif pcm_s16le_planar wmav2
h261 pcm_s24be wmv1
h263 pcm_s24daud wmv2
h263p pcm_s24le wrapped_avframe
h264_amf pcm_s24le_planar xbm
h264_mf pcm_s32be xface
h264_nvenc pcm_s32le xsub
h264_qsv pcm_s32le_planar xwd
hap pcm_s64be y41p
hdr pcm_s64le yuv4
hevc_amf pcm_s8 zlib
hevc_mf pcm_s8_planar zmbv
hevc_nvenc pcm_u16be
Enabled hwaccels:
av1_d3d11va hevc_nvdec vc1_nvdec
av1_d3d11va2 mjpeg_nvdec vp8_nvdec
av1_dxva2 mpeg1_nvdec vp9_d3d11va
av1_nvdec mpeg2_d3d11va vp9_d3d11va2
h264_d3d11va mpeg2_d3d11va2 vp9_dxva2
h264_d3d11va2 mpeg2_dxva2 vp9_nvdec
h264_dxva2 mpeg2_nvdec wmv3_d3d11va
h264_nvdec mpeg4_nvdec wmv3_d3d11va2
hevc_d3d11va vc1_d3d11va wmv3_dxva2
hevc_d3d11va2 vc1_d3d11va2 wmv3_nvdec
hevc_dxva2 vc1_dxva2
Enabled parsers:
aac dvdsub opus
aac_latm flac png
ac3 ftr pnm
adx g723_1 qoi
amr g729 rv30
av1 gif rv40
avs2 gsm sbc
avs3 h261 sipr
bmp h263 tak
cavsvideo h264 vc1
cook hdr vorbis
cri hevc vp3
dca ipu vp8
dirac jpeg2000 vp9
dnxhd misc4 webp
dolby_e mjpeg xbm
dpx mlp xma
dvaudio mpeg4video xwd
dvbsub mpegaudio
dvd_nav mpegvideo
Enabled demuxers:
aa idf pcm_f64le
aac iff pcm_mulaw
aax ifv pcm_s16be
ac3 ilbc pcm_s16le
ace image2 pcm_s24be
acm image2_alias_pix pcm_s24le
act image2_brender_pix pcm_s32be
adf image2pipe pcm_s32le
adp image_bmp_pipe pcm_s8
ads image_cri_pipe pcm_u16be
adx image_dds_pipe pcm_u16le
aea image_dpx_pipe pcm_u24be
afc image_exr_pipe pcm_u24le
aiff image_gem_pipe pcm_u32be
aix image_gif_pipe pcm_u32le
alp image_hdr_pipe pcm_u8
amr image_j2k_pipe pcm_vidc
amrnb image_jpeg_pipe pjs
amrwb image_jpegls_pipe pmp
anm image_jpegxl_pipe pp_bnk
apac image_pam_pipe pva
apc image_pbm_pipe pvf
ape image_pcx_pipe qcp
apm image_pfm_pipe r3d
apng image_pgm_pipe rawvideo
aptx image_pgmyuv_pipe realtext
aptx_hd image_pgx_pipe redspark
aqtitle image_phm_pipe rka
argo_asf image_photocd_pipe rl2
argo_brp image_pictor_pipe rm
argo_cvg image_png_pipe roq
asf image_ppm_pipe rpl
asf_o image_psd_pipe rsd
ass image_qdraw_pipe rso
ast image_qoi_pipe rtp
au image_sgi_pipe rtsp
av1 image_sunrast_pipe s337m
avi image_svg_pipe sami
avisynth image_tiff_pipe sap
avr image_vbn_pipe sbc
avs image_webp_pipe sbg
avs2 image_xbm_pipe scc
avs3 image_xpm_pipe scd
bethsoftvid image_xwd_pipe sdns
bfi imf sdp
bfstm ingenient sdr2
bink ipmovie sds
binka ipu sdx
bintext ircam segafilm
bit iss ser
bitpacked iv8 sga
bmv ivf shorten
boa ivr siff
bonk jacosub simbiosis_imx
brstm jv sln
c93 kux smacker
caf kvag smjpeg
cavsvideo laf smush
cdg libgme sol
cdxl libmodplug sox
cine libopenmpt spdif
codec2 live_flv srt
codec2raw lmlm4 stl
concat loas str
dash lrc subviewer
data luodat subviewer1
daud lvf sup
dcstr lxf svag
derf m4v svs
dfa matroska swf
dfpwm mca tak
dhav mcc tedcaptions
dirac mgsts thp
dnxhd microdvd threedostr
dsf mjpeg tiertexseq
dsicin mjpeg_2000 tmv
dss mlp truehd
dts mlv tta
dtshd mm tty
dv mmf txd
dvbsub mods ty
dvbtxt moflex v210
dxa mov v210x
ea mp3 vag
ea_cdata mpc vc1
eac3 mpc8 vc1t
epaf mpegps vividas
ffmetadata mpegts vivo
filmstrip mpegtsraw vmd
fits mpegvideo vobsub
flac mpjpeg voc
flic mpl2 vpk
flv mpsub vplayer
fourxm msf vqf
frm msnwc_tcp w64
fsb msp wady
fwse mtaf wav
g722 mtv wavarc
g723_1 musx wc3
g726 mv webm_dash_manifest
g726le mvi webvtt
g729 mxf wsaud
gdv mxg wsd
genh nc wsvqa
gif nistsphere wtv
gsm nsp wv
gxf nsv wve
h261 nut xa
h263 nuv xbin
h264 obu xmd
hca ogg xmv
hcom oma xvag
hevc paf xwma
hls pcm_alaw yop
hnm pcm_f32be yuv4mpegpipe
ico pcm_f32le
idcin pcm_f64be
Enabled muxers:
a64 h263 pcm_s24be
ac3 h264 pcm_s24le
adts hash pcm_s32be
adx hds pcm_s32le
aiff hevc pcm_s8
alp hls pcm_u16be
amr ico pcm_u16le
amv ilbc pcm_u24be
apm image2 pcm_u24le
apng image2pipe pcm_u32be
aptx ipod pcm_u32le
aptx_hd ircam pcm_u8
argo_asf ismv pcm_vidc
argo_cvg ivf psp
asf jacosub rawvideo
asf_stream kvag rm
ass latm roq
ast lrc rso
au m4v rtp
avi matroska rtp_mpegts
avif matroska_audio rtsp
avm2 md5 sap
avs2 microdvd sbc
avs3 mjpeg scc
bit mkvtimestamp_v2 segafilm
caf mlp segment
cavsvideo mmf smjpeg
chromaprint mov smoothstreaming
codec2 mp2 sox
codec2raw mp3 spdif
crc mp4 spx
dash mpeg1system srt
data mpeg1vcd stream_segment
daud mpeg1video streamhash
dfpwm mpeg2dvd sup
dirac mpeg2svcd swf
dnxhd mpeg2video tee
dts mpeg2vob tg2
dv mpegts tgp
eac3 mpjpeg truehd
f4v mxf tta
ffmetadata mxf_d10 ttml
fifo mxf_opatom uncodedframecrc
fifo_test null vc1
filmstrip nut vc1t
fits obu voc
flac oga w64
flv ogg wav
framecrc ogv webm
framehash oma webm_chunk
framemd5 opus webm_dash_manifest
g722 pcm_alaw webp
g723_1 pcm_f32be webvtt
g726 pcm_f32le wsaud
g726le pcm_f64be wtv
gif pcm_f64le wv
gsm pcm_mulaw yuv4mpegpipe
gxf pcm_s16be
h261 pcm_s16le
Enabled protocols:
async http rtmp
bluray httpproxy rtmpe
cache https rtmps
concat icecast rtmpt
concatf ipfs_gateway rtmpte
crypto ipns_gateway rtmpts
data librist rtp
fd libsrt srtp
ffrtmpcrypt libssh subfile
ffrtmphttp libzmq tcp
file md5 tee
ftp mmsh tls
gopher mmst udp
gophers pipe udplite
hls prompeg
Enabled filters:
a3dscope dctdnoiz pad
abench ddagrab pad_opencl
abitscope deband pal100bars
acompressor deblock pal75bars
acontrast decimate palettegen
acopy deconvolve paletteuse
acrossfade dedot pan
acrossover deesser perms
acrusher deflate perspective
acue deflicker phase
addroi deinterlace_qsv photosensitivity
adeclick dejudder pixdesctest
adeclip delogo pixelize
adecorrelate derain pixscope
adelay deshake pp
adenorm deshake_opencl pp7
aderivative despill premultiply
adrawgraph detelecine prewitt
adrc dialoguenhance prewitt_opencl
adynamicequalizer dilation program_opencl
adynamicsmooth dilation_opencl pseudocolor
aecho displace psnr
aemphasis dnn_classify pullup
aeval dnn_detect qp
aevalsrc dnn_processing random
aexciter doubleweave readeia608
afade drawbox readvitc
afdelaysrc drawgraph realtime
afftdn drawgrid remap
afftfilt drawtext remap_opencl
afifo drmeter removegrain
afir dynaudnorm removelogo
afirsrc earwax repeatfields
aformat ebur128 replaygain
afreqshift edgedetect reverse
afwtdn elbg rgbashift
agate entropy rgbtestsrc
agraphmonitor epx roberts
ahistogram eq roberts_opencl
aiir equalizer rotate
aintegral erosion rubberband
ainterleave erosion_opencl sab
alatency estdif scale
alimiter exposure scale2ref
allpass extractplanes scale_cuda
allrgb extrastereo scale_qsv
allyuv fade scale_vulkan
aloop feedback scdet
alphaextract fftdnoiz scharr
alphamerge fftfilt scroll
amerge field segment
ametadata fieldhint select
amix fieldmatch selectivecolor
amovie fieldorder sendcmd
amplify fifo separatefields
amultiply fillborders setdar
anequalizer find_rect setfield
anlmdn firequalizer setparams
anlmf flanger setpts
anlms flip_vulkan setrange
anoisesrc flite setsar
anull floodfill settb
anullsink format shear
anullsrc fps showcqt
apad framepack showcwt
aperms framerate showfreqs
aphasemeter framestep showinfo
aphaser freezedetect showpalette
aphaseshift freezeframes showspatial
apsyclip frei0r showspectrum
apulsator frei0r_src showspectrumpic
arealtime fspp showvolume
aresample gblur showwaves
areverse gblur_vulkan showwavespic
arnndn geq shuffleframes
asdr gradfun shufflepixels
asegment gradients shuffleplanes
aselect graphmonitor sidechaincompress
asendcmd grayworld sidechaingate
asetnsamples greyedge sidedata
asetpts guided sierpinski
asetrate haas signalstats
asettb haldclut signature
ashowinfo haldclutsrc silencedetect
asidedata hdcd silenceremove
asoftclip headphone sinc
aspectralstats hflip sine
asplit hflip_vulkan siti
ass highpass smartblur
astats highshelf smptebars
astreamselect hilbert smptehdbars
asubboost histeq sobel
asubcut histogram sobel_opencl
asupercut hqdn3d sofalizer
asuperpass hqx spectrumsynth
asuperstop hstack speechnorm
atadenoise hstack_qsv split
atempo hsvhold spp
atilt hsvkey sr
atrim hue ssim
avectorscope huesaturation ssim360
avgblur hwdownload stereo3d
avgblur_opencl hwmap stereotools
avgblur_vulkan hwupload stereowiden
avsynctest hwupload_cuda streamselect
axcorrelate hysteresis subtitles
azmq identity super2xsai
backgroundkey idet superequalizer
bandpass il surround
bandreject inflate swaprect
bass interlace swapuv
bbox interleave tblend
bench join telecine
bilateral kerndeint testsrc
bilateral_cuda kirsch testsrc2
biquad ladspa thistogram
bitplanenoise lagfun threshold
blackdetect latency thumbnail
blackframe lenscorrection thumbnail_cuda
blend lensfun tile
blend_vulkan libplacebo tiltshelf
blockdetect libvmaf tinterlace
blurdetect life tlut2
bm3d limitdiff tmedian
boxblur limiter tmidequalizer
boxblur_opencl loop tmix
bs2b loudnorm tonemap
bwdif lowpass tonemap_opencl
cas lowshelf tpad
cellauto lumakey transpose
channelmap lut transpose_opencl
channelsplit lut1d transpose_vulkan
chorus lut2 treble
chromaber_vulkan lut3d tremolo
chromahold lutrgb trim
chromakey lutyuv unpremultiply
chromakey_cuda mandelbrot unsharp
chromanr maskedclamp unsharp_opencl
chromashift maskedmax untile
ciescope maskedmerge v360
codecview maskedmin vaguedenoiser
color maskedthreshold varblur
colorbalance maskfun vectorscope
colorchannelmixer mcompand vflip
colorchart median vflip_vulkan
colorcontrast mergeplanes vfrdet
colorcorrect mestimate vibrance
colorhold metadata vibrato
colorize midequalizer vidstabdetect
colorkey minterpolate vidstabtransform
colorkey_opencl mix vif
colorlevels monochrome vignette
colormap morpho virtualbass
colormatrix movie vmafmotion
colorspace mpdecimate volume
colorspace_cuda mptestsrc volumedetect
colorspectrum msad vpp_qsv
colortemperature multiply vstack
compand negate vstack_qsv
compensationdelay nlmeans w3fdif
concat nlmeans_opencl waveform
convolution nnedi weave
convolution_opencl noformat xbr
convolve noise xcorrelate
copy normalize xfade
corr null xfade_opencl
cover_rect nullsink xmedian
crop nullsrc xstack
cropdetect openclsrc xstack_qsv
crossfeed oscilloscope yadif
crystalizer overlay yadif_cuda
cue overlay_cuda yaepblur
curves overlay_opencl yuvtestsrc
datascope overlay_qsv zmq
dblur overlay_vulkan zoompan
dcshift owdenoise zscale
Enabled bsfs:
aac_adtstoasc h264_redundant_pps opus_metadata
av1_frame_merge hapqa_extract pcm_rechunk
av1_frame_split hevc_metadata pgs_frame_merge
av1_metadata hevc_mp4toannexb prores_metadata
chomp imx_dump_header remove_extradata
dca_core media100_to_mjpegb setts
dts2pts mjpeg2jpeg text2movsub
dump_extradata mjpega_dump_header trace_headers
dv_error_marker mov2textsub truehd_core
eac3_core mp3_header_decompress vp9_metadata
extract_extradata mpeg2_metadata vp9_raw_reorder
filter_units mpeg4_unpack_bframes vp9_superframe
h264_metadata noise vp9_superframe_split
h264_mp4toannexb null
Enabled indevs:
dshow lavfi vfwcap
gdigrab libcdio
Enabled outdevs:
caca sdl2

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -14,10 +14,22 @@ Catalog_Entry :: struct {
label_list: string; // the hash of the label list they were scored against
embedding: [] u16; // the label model's image vector, 16-bit floats; empty when not labelled
// What an imported file is, from its import.json (app/import.jai).
media: Catalog_Media;
duration: float64; // seconds: a video's, or an animated GIF's; 0 for a still image
video_codec: string; // "h264", "hevc" ...; "" when not a video
frame_rate: float64;
// Lowercased copies, for matching without case.
find_title, find_url, find_app, find_notes, find_ocr, find_labels, find_folder: string;
}
Catalog_Media :: enum u8 {
IMAGE; // a still image, or a Snagit capture
VIDEO;
ANIMATION; // an animated GIF
}
// Everything the entry holds is from the default allocator.
catalog_entry_free :: (entry: *Catalog_Entry) {
if !entry return;
@ -32,6 +44,7 @@ catalog_entry_free :: (entry: *Catalog_Entry) {
for entry.labels free(it.name);
array_free(entry.labels);
free(entry.label_list);
free(entry.video_codec);
array_free(entry.embedding);
free(entry.find_title); free(entry.find_url); free(entry.find_app); free(entry.find_notes); free(entry.find_ocr); free(entry.find_labels); free(entry.find_folder);
free(entry);
@ -62,6 +75,14 @@ catalog_load :: (path: string) -> *Catalog_Entry, error: string {
entry.url = catalog_string(metadata.root, "WebURL");
entry.app = catalog_string(metadata.root, "AppName");
}
imported := capture_find_file(*doc, "import.json");
if imported {
entry.video_codec = catalog_string(imported.root, "VideoCodec");
entry.duration = catalog_number(imported.root, "Duration");
entry.frame_rate = catalog_number(imported.root, "FrameRate");
if entry.video_codec entry.media = .VIDEO;
else if catalog_number(imported.root, "Frames") > 1 entry.media = .ANIMATION;
}
notes: String_Builder;
notes.allocator = temp;
for file: doc.files {
@ -308,6 +329,12 @@ catalog_string :: (object: Capture_Value, key: string) -> string {
return copy_string(value.text);
}
catalog_number :: (object: Capture_Value, key: string) -> float64 {
value := capture_object_find(*object, key);
if !value || value.kind != .NUMBER return 0;
return value.number;
}
rtf_hex :: (c: u8) -> s64 {
if c >= #char "0" && c <= #char "9" return c - #char "0";
if c >= #char "a" && c <= #char "f" return c - #char "a" + 10;

View File

@ -1,12 +1,16 @@
// Importing a folder of screenshots and photos into the library: every image under it (PNG, JPEG, HEIC,
// AVIF, WebP, GIF, BMP, TIFF, camera RAW) and every Snagit .snagx becomes an .mcap in a folder of its own
// in the library, one folder for each place imported from (D:\McapIndex\mcap\SnagitLibraryWork, ...).
// The files imported from are only read, never changed.
// Importing a folder of screenshots, photos and videos into the library: every image under it (PNG,
// JPEG, HEIC, AVIF, WebP, GIF, BMP, TIFF, camera RAW), every video (MP4, MOV, MKV, WebM, AVI, WMV, MPEG;
// on Windows, with FFmpeg) and every Snagit .snagx becomes an .mcap in a folder of its own in the
// library, one folder for each place imported from (D:\McapIndex\mcap\SnagitLibraryWork, ...). The
// files imported from are only read, never changed.
//
// An image's .mcap holds the file's bytes exactly as they were (its original), a preview turned the
// right way up, and Capture Text with metadata.json (CaptureDate, WindowName as the file's name,
// AppName as the camera or phone) and import.json (where it came from, its EXIF). A .snagx becomes the
// same .mcap SnagitConverter makes of it (modules/Snagx).
// AppName as the camera or phone) and import.json (where it came from, its EXIF). A video's is the
// same, with a frame from it, turned the right way up, as its rendered image (what OCR and labels
// read), and what its container says in import.json. A video is copied into its .mcap a few MB at a
// time, never read whole, so it can be any size. A .snagx becomes the same .mcap SnagitConverter
// makes of it (modules/Snagx).
//
// Each image is stored once in the whole library: the library index (IMPORT_INDEX_NAME, in the library
// folder) holds a hash of every .mcap's original image, made once for the files already there and kept
@ -19,12 +23,15 @@
IMPORT_WINDOW :: "Import";
IMPORT_INDEX_NAME :: ".import-index"; // in the library folder: path, size, time and hash of each .mcap
IMPORT_SOURCES_NAME :: ".import-sources"; // in the library folder: the folder each library folder was last imported from
IMPORT_MAX_FILE :: 1024 * 1024 * 1024; // larger files are not read
IMPORT_MAX_FILE :: 1024 * 1024 * 1024; // larger images are not read (videos are streamed, any size)
IMPORT_NOTES_MAX :: 5000; // failures and duplicates listed; the rest are only counted
IMPORT_SAVE_EVERY :: 250; // captures imported between saves of the index
IMPORT_HEAD_SIZE :: 256; // the first bytes looked at, to tell what a file is
// Not images, by extension: skipped without being read. Anything else is read and looked at.
IMPORT_VIDEO_EXTENSIONS :: string.["mp4", "mov", "m4v", "avi", "mkv", "webm", "3gp", "3g2", "wmv", "mts", "m2ts", "mpg", "mpeg", "flv"];
// Videos, by extension, for those whose first bytes do not say so (MPEG transport streams).
IMPORT_VIDEO_EXTENSIONS :: string.["mp4", "mov", "m4v", "avi", "mkv", "webm", "3gp", "3g2", "wmv", "mts", "m2ts", "ts", "mpg", "mpeg", "flv"];
// Not images or videos, by extension: skipped without being read. Anything else is read and looked
// at. thm and lrv are a GoPro's thumbnails and low-resolution copies of its videos.
IMPORT_OTHER_EXTENSIONS :: string.["aae", "xmp", "json", "txt", "ini", "db", "pdf", "zip", "lnk", "url", "html", "htm", "doc", "docx", "xls", "xlsx", "ppt", "pptx", "mp3", "m4a", "wav", "aac", "exe", "dll", "mcap", "tmp", "thm", "lrv", "mpo"];
Import_Phase :: enum u8 { IDLE; LISTING; INDEXING; IMPORTING; DONE; }
@ -282,26 +289,29 @@ import_worker :: (thread: *Thread) -> s64 {
return 0;
}
// Imports one file. The text says why it failed or was skipped (what kind of file: "mp4", "video"),
// or, for a duplicate, where the image already is; new_path is where an import went. Temp.
// Imports one file. The text says why it failed or was skipped (what kind of file: "aae", "empty"),
// or, for a duplicate, where it already is; new_path is where an import went. Temp.
import_file :: (path: string) -> Import_Outcome, text: string, new_path: string {
name := path_filename(path);
extension := to_lower_copy(path_extension(name),, temp);
if array_find(IMPORT_VIDEO_EXTENSIONS, extension) || array_find(IMPORT_OTHER_EXTENSIONS, extension) return .SKIPPED, extension, "";
if array_find(IMPORT_OTHER_EXTENSIONS, extension) return .SKIPPED, extension, "";
modified, size, stat_ok := file_modtime_and_size(path);
if !stat_ok return .FAILED, "cannot be read", "";
if size == 0 return .SKIPPED, "empty", "";
if size > IMPORT_MAX_FILE return .FAILED, tprint("is % MB, larger than the % MB an import reads", size / (1024 * 1024), IMPORT_MAX_FILE / (1024 * 1024)), "";
// A look at the first bytes, before reading what may not be an image at all.
head := import_read_head(path, 64);
head := import_read_head(path, IMPORT_HEAD_SIZE);
kind := image_kind_from_name(image_kind(head), name);
is_snagx := extension == "snagx" && head.count >= 4 && memcmp(head.data, "PK\x03\x04".data, 4) == 0;
if !is_snagx {
if kind == .VIDEO return .SKIPPED, "video", "";
if kind == .VIDEO || (kind == .UNKNOWN && array_find(IMPORT_VIDEO_EXTENSIONS, extension)) {
outcome, text, new_path := import_video_file(path, name, head, size, modified);
return outcome, text, new_path;
}
if kind == .UNKNOWN return .SKIPPED, ifx extension then extension else "no extension", "";
}
if size > IMPORT_MAX_FILE return .FAILED, tprint("is % MB, larger than the % MB an import reads", size / (1024 * 1024), IMPORT_MAX_FILE / (1024 * 1024)), "";
data, read_ok := read_entire_file(path, log_errors = false,, temp);
if !read_ok return .FAILED, "cannot be read", "";
@ -323,21 +333,10 @@ import_file :: (path: string) -> Import_Outcome, text: string, new_path: string
}
// Once in the whole library: the hash is held for this file until it is written, or let go.
lock(*importer.mutex);
found, existing := table_find(*importer.by_hash, hash);
if found {
where := copy_string(existing,, temp);
unlock(*importer.mutex);
return .DUPLICATE, ifx where then where else "(another file in this import)", "";
}
table_set(*importer.by_hash, hash, ""); // "" while it is being written
unlock(*importer.mutex);
claimed, where := import_claim(hash);
if !claimed return .DUPLICATE, where, "";
let_go := true;
defer if let_go {
lock(*importer.mutex);
table_remove(*importer.by_hash, hash);
unlock(*importer.mutex);
}
defer if let_go import_let_go(hash);
if !is_snagx {
ok: bool;
@ -349,8 +348,58 @@ import_file :: (path: string) -> Import_Outcome, text: string, new_path: string
out_bytes = cast([] u8) string.{serializer.count, serializer.data};
}
// Its name in the library folder: another file there by that name (a different image) gets _2, _3 ...
out_path := import_reserve_path(out_name);
written, write_error := mcap_write(out_path, out_bytes);
new_path := import_written(out_path, hash, written);
if !written return .FAILED, write_error, "";
let_go = false;
return .IMPORTED, "", new_path;
}
// A video: hashed and copied into its .mcap a few MB at a time, so it can be larger than memory.
import_video_file :: (path: string, name: string, head: [] u8, size: s64, modified: Apollo_Time) -> Import_Outcome, text: string, new_path: string {
if OS != .WINDOWS return .SKIPPED, "video (Windows only)", "";
hash, hashed := import_hash_file(.{path = path, offset = 0, size = size});
if !hashed return .FAILED, "cannot be read", "";
claimed, where := import_claim(hash);
if !claimed return .DUPLICATE, where, "";
let_go := true;
defer if let_go import_let_go(hash);
m, out_name, ok, error := import_video_mcap(path, name, head, size, hash, modified);
if !ok return .FAILED, error, "";
out_path := import_reserve_path(out_name);
written, write_error := mcap_write_streamed(out_path, m, .{path = path, offset = 0, size = size});
new_path := import_written(out_path, hash, written);
if !written return .FAILED, write_error, "";
let_go = false;
return .IMPORTED, "", new_path;
}
// Holds hash for a file being imported, so no other worker imports the same bytes; false, with where
// they already are, when the library has them, or another file of this import does.
import_claim :: (hash: u64) -> claimed: bool, where: string {
lock(*importer.mutex);
defer unlock(*importer.mutex);
found, existing := table_find(*importer.by_hash, hash);
if found return false, ifx existing then copy_string(existing,, temp) else "(another file in this import)";
table_set(*importer.by_hash, hash, ""); // "" while it is being written
return true, "";
}
// Lets go of a hash claimed for a file that was not imported after all.
import_let_go :: (hash: u64) {
lock(*importer.mutex);
table_remove(*importer.by_hash, hash);
unlock(*importer.mutex);
}
// Where out_name goes in the library folder, held until import_written: another file there by that
// name (a different capture) gets _2, _3 ... Heap.
import_reserve_path :: (out_name: string) -> string {
lock(*importer.mutex);
defer unlock(*importer.mutex);
base := path_strip_extension(out_name);
out_path := tprint("%/%", importer.dest_folder, out_name);
number := 2;
@ -360,9 +409,12 @@ import_file :: (path: string) -> Import_Outcome, text: string, new_path: string
}
out_path = copy_string(out_path);
table_set(*importer.reserved_names, out_path, true);
unlock(*importer.mutex);
return out_path;
}
written, write_error := mcap_write(out_path, out_bytes);
// After out_path (from import_reserve_path, which this frees) was written, or failed to be: the
// index learns of it. Where it is (temp), for the grid.
import_written :: (out_path: string, hash: u64, written: bool) -> string {
out_modified, out_size, _ := file_modtime_and_size(out_path);
relative := import_relative(out_path);
lock(*importer.mutex);
@ -372,10 +424,9 @@ import_file :: (path: string) -> Import_Outcome, text: string, new_path: string
import_record_set(relative, .{size = out_size, modified = to_nanoseconds(out_modified), hash = hash});
}
unlock(*importer.mutex);
new_path := tprint("%/%", importer.library, relative);
free(out_path);
if !written return .FAILED, write_error, "";
let_go = false;
return .IMPORTED, "", tprint("%/%", importer.library, relative);
return new_path;
}
// An image file as an .mcap (temp) and the name it gets: the date it was taken and its hash.
@ -393,68 +444,209 @@ import_image_mcap :: (path: string, name: string, bytes: [] u8, kind: Image_Kind
if !date { date = import_name_date(name); date_from = "file name"; }
if !date { date = import_calendar_text(to_calendar(modified, .LOCAL)); date_from = "modified time"; }
device := import_text(exif.model);
make := import_text(exif.make);
if make && !begins_with_nocase(device, make) device = tprint("% %", make, device);
device = trim(device);
doc: Capture_Document;
capture_document_init(*doc, temp);
doc.comment = tprint("Imported by % % from %.", APP_NAME, VERSION, import_text(path_filename(path)));
add :: (doc: *Capture_Document, file: *Capture_File, key: string, value: string) {
capture_object_add(doc, *file.root, key, capture_string(doc, value));
}
add_number :: (doc: *Capture_Document, file: *Capture_File, key: string, value: float64) {
capture_object_add(doc, *file.root, key, capture_number(value));
}
metadata := capture_add_file(*doc, "metadata.json");
add(*doc, metadata, "AppName", device);
add(*doc, metadata, "CaptureDate", date);
add(*doc, metadata, "WindowName", import_text(name));
info := capture_add_file(*doc, "import.json");
add(*doc, info, "SourcePath", import_text(path));
add_number(*doc, info, "SourceSize", xx bytes.count);
add(*doc, info, "SourceModified", import_calendar_text(to_calendar(modified, .LOCAL)));
add(*doc, info, "ContentHash", import_hash_text(hash));
add(*doc, info, "Format", image_kind_name(kind));
add_number(*doc, info, "Width", xx image.width);
add_number(*doc, info, "Height", xx image.height);
add(*doc, info, "DateFrom", date_from);
add(*doc, info, "ImportDate", import_calendar_text(to_calendar(current_time_consensus(), .LOCAL)));
if exif.found {
add_number(*doc, info, "Orientation", xx exif.orientation);
if make add(*doc, info, "CameraMake", make);
if exif.model add(*doc, info, "CameraModel", import_text(exif.model));
if exif.software add(*doc, info, "Software", import_text(exif.software));
if exif.offset_original add(*doc, info, "TimeZoneOffset", import_text(exif.offset_original));
if exif.has_gps {
add_number(*doc, info, "Latitude", exif.latitude);
add_number(*doc, info, "Longitude", exif.longitude);
if exif.altitude != 0 add_number(*doc, info, "Altitude", exif.altitude);
make := import_text(exif.make);
doc, info := import_document(path, name, import_device(make, exif.model), date, bytes.count, modified, hash, image_kind_name(kind), image.width, image.height, date_from);
if kind == .GIF {
frames, duration_ms := gif_info(bytes);
if frames > 1 { // an animation
import_add_number(doc, info, "Frames", xx frames);
import_add_number(doc, info, "Duration", duration_ms / 1000.0);
}
}
text := capture_text_write(*doc);
back: Capture_Document;
capture_document_init(*back, temp);
if !capture_text_parse(*back, path, text) || back.mistakes || back.files.count != 2 return m, "", false, "its Capture Text does not read back cleanly (see above)";
if exif.found {
import_add_number(doc, info, "Orientation", xx exif.orientation);
if make import_add(doc, info, "CameraMake", make);
if exif.model import_add(doc, info, "CameraModel", import_text(exif.model));
if exif.software import_add(doc, info, "Software", import_text(exif.software));
if exif.offset_original import_add(doc, info, "TimeZoneOffset", import_text(exif.offset_original));
if exif.has_gps {
import_add_number(doc, info, "Latitude", exif.latitude);
import_add_number(doc, info, "Longitude", exif.longitude);
if exif.altitude != 0 import_add_number(doc, info, "Altitude", exif.altitude);
}
}
text, text_ok := import_document_text(doc, path);
if !text_ok return m, "", false, "its Capture Text does not read back cleanly (see above)";
m.text = text;
m.preview = .{width = cast(u16) preview_w, height = cast(u16) preview_h, png = png};
m.original_name = name;
m.original = bytes;
m.rendered_kind = .NONE;
return m, import_out_name(date, hash), true, "";
}
// 2024-05-12_10-15-30_1a2b3c4d.mcap: newest last, as Snagit names its captures.
// A video as an .mcap (temp), but for its original, which mcap_write_streamed copies from the file,
// and the name it gets. Its rendered image is a frame from it, which OCR and labels read.
import_video_mcap :: (path: string, name: string, head: [] u8, size: s64, hash: u64, modified: Apollo_Time) -> Mcap, name: string, ok: bool, error: string {
m: Mcap;
image, video, decoded, decode_error := video_poster(path);
if !decoded return m, "", false, decode_error;
defer image_free(*image);
png, preview_w, preview_h, preview_ok, preview_error := image_preview_png(image, MCAP_PREVIEW_MAX_SIDE);
if !preview_ok return m, "", false, preview_error;
poster, _, _, poster_ok, poster_error := image_preview_png(image, max(image.width, image.height)); // at its own size
if !poster_ok return m, "", false, poster_error;
// When it was taken: what the phone wrote (local time where it was), else a date in the file's name
// (local time too, as recorders write it), else when the container was made (UTC, so turned into
// this computer's time zone, which may not be where it was), else when the file was last changed.
date, offset := import_video_local_date(video.creation_date);
date_from := "metadata";
if !date { date = import_name_date(name); date_from = "file name"; }
if !date { date = import_video_utc_date(video.creation_time); date_from = "metadata"; }
if !date { date = import_calendar_text(to_calendar(modified, .LOCAL)); date_from = "modified time"; }
make := import_text(video.make);
doc, info := import_document(path, name, import_device(make, video.model), date, size, modified, hash, video_format_name(head, name), image.width, image.height, date_from);
import_add_number(doc, info, "Duration", import_milliseconds(video.duration));
if video.frame_rate > 0 import_add_number(doc, info, "FrameRate", video.frame_rate);
import_add(doc, info, "VideoCodec", import_text(video.codec));
import_add_number(doc, info, "Rotation", xx video.rotation);
import_add_number(doc, info, "PosterTime", import_milliseconds(video.poster_time));
capture_object_add(doc, *info.root, "Audio", capture_boolean(video.has_audio));
if make import_add(doc, info, "CameraMake", make);
if video.model import_add(doc, info, "CameraModel", import_text(video.model));
if video.software && !begins_with(video.software, "Lavf") import_add(doc, info, "Software", import_text(video.software)); // not FFmpeg's own tag
if offset import_add(doc, info, "TimeZoneOffset", offset);
latitude, longitude, altitude, has_altitude, located := iso6709_parse(video.location);
if located {
import_add_number(doc, info, "Latitude", latitude);
import_add_number(doc, info, "Longitude", longitude);
if has_altitude && altitude != 0 import_add_number(doc, info, "Altitude", altitude);
}
text, text_ok := import_document_text(doc, path);
if !text_ok return m, "", false, "its Capture Text does not read back cleanly (see above)";
m.text = text;
m.preview = .{width = cast(u16) preview_w, height = cast(u16) preview_h, png = png};
m.original_name = name;
m.rendered_kind = .OWN;
m.rendered = poster;
return m, import_out_name(date, hash), true, "";
}
// An imported file's Capture Text so far (temp): metadata.json, and import.json with the keys every
// import has; the caller adds its own to info (import_add), then import_document_text.
import_document :: (path: string, name: string, device: string, date: string, size: s64, modified: Apollo_Time, hash: u64, format: string, width: s64, height: s64, date_from: string) -> *Capture_Document, info: *Capture_File {
doc := New(Capture_Document,, temp);
capture_document_init(doc, temp);
doc.comment = tprint("Imported by % % from %.", APP_NAME, VERSION, import_text(path_filename(path)));
metadata := capture_add_file(doc, "metadata.json");
import_add(doc, metadata, "AppName", device);
import_add(doc, metadata, "CaptureDate", date);
import_add(doc, metadata, "WindowName", import_text(name));
info := capture_add_file(doc, "import.json");
import_add(doc, info, "SourcePath", import_text(path));
import_add_number(doc, info, "SourceSize", xx size);
import_add(doc, info, "SourceModified", import_calendar_text(to_calendar(modified, .LOCAL)));
import_add(doc, info, "ContentHash", import_hash_text(hash));
import_add(doc, info, "Format", format);
import_add_number(doc, info, "Width", xx width);
import_add_number(doc, info, "Height", xx height);
import_add(doc, info, "DateFrom", date_from);
import_add(doc, info, "ImportDate", import_calendar_text(to_calendar(current_time_consensus(), .LOCAL)));
return doc, info;
}
import_add :: (doc: *Capture_Document, file: *Capture_File, key: string, value: string) {
capture_object_add(doc, *file.root, key, capture_string(doc, value));
}
import_add_number :: (doc: *Capture_Document, file: *Capture_File, key: string, value: float64) {
capture_object_add(doc, *file.root, key, capture_number(value));
}
// Seconds to the millisecond: 1.667, not 1.6666666666666668.
import_milliseconds :: (seconds: float64) -> float64 {
return floor(seconds * 1000 + 0.5) / 1000;
}
// The document as Capture Text (temp), once it reads back as written.
import_document_text :: (doc: *Capture_Document, path: string) -> string, ok: bool {
text := capture_text_write(doc);
back: Capture_Document;
capture_document_init(*back, temp);
if !capture_text_parse(*back, path, text) || back.mistakes || back.files.count != 2 return "", false;
return text, true;
}
// The camera or phone, as AppName: "Apple iPhone 15 Pro", or its model alone when that starts with
// its make.
import_device :: (make: string, model: string) -> string {
device := import_text(model);
if make && !begins_with_nocase(device, make) device = tprint("% %", make, device);
return trim(device);
}
// 2024-05-12_10-15-30_1a2b3c4d.mcap: newest last, as Snagit names its captures.
import_out_name :: (date: string, hash: u64) -> string {
stamp := replace(replace(date, " ", "_",, temp), ":", "-",, temp);
out_name := tprint("%_%.%", stamp, slice(import_hash_text(hash), 0, 8), MCAP_EXTENSION);
return m, out_name, true, "";
return tprint("%_%.%", stamp, slice(import_hash_text(hash), 0, 8), MCAP_EXTENSION);
}
// Apple's creationdate, "2024-05-12T10:15:30+0100": the local time it was taken, as Snagit writes
// CaptureDate, and its offset from UTC, "+01:00" ("" when it has none). "" when it is not a date.
import_video_local_date :: (text: string) -> date: string, offset: string {
date := import_exif_date(text);
if !date return "", "";
rest := advance(text, 19);
while rest.count && (rest[0] == #char "." || is_digit(rest[0])) rest = advance(rest, 1); // fractions of a second
offset := "";
if rest.count >= 5 && (rest[0] == #char "+" || rest[0] == #char "-") && is_digit(rest[1]) && is_digit(rest[2]) {
minutes := ifx rest[3] == #char ":" then slice(rest, 4, 2) else slice(rest, 3, 2);
if minutes.count == 2 && is_digit(minutes[0]) && is_digit(minutes[1]) offset = tprint("%:%", slice(rest, 0, 3), minutes);
} else if rest.count && rest[0] == #char "Z" {
offset = "+00:00";
}
return date, offset;
}
// A container's creation_time, "2024-05-12T09:15:30.000000Z", which is UTC, as local time in the form
// Snagit writes CaptureDate. "" when it is not a date, or is the zero date a camera with no clock
// writes (1904 or 1970).
import_video_utc_date :: (text: string) -> string {
date := import_exif_date(text);
if !date return "";
year := string_to_int(slice(date, 0, 4));
if year <= 1970 return "";
utc: Calendar_Time;
utc.year = cast(s32) year;
utc.month_starting_at_0 = cast(s8) (string_to_int(slice(date, 5, 2)) - 1);
utc.day_of_month_starting_at_0 = cast(s8) (string_to_int(slice(date, 8, 2)) - 1);
utc.hour = cast(s8) string_to_int(slice(date, 11, 2));
utc.minute = cast(s8) string_to_int(slice(date, 14, 2));
utc.second = cast(s8) string_to_int(slice(date, 17, 2));
utc.time_zone = .UTC;
return import_calendar_text(to_calendar(calendar_to_apollo(utc), .LOCAL));
}
import_hash :: (bytes: [] u8) -> u64 {
return fnv1a_hash(bytes.data, bytes.count);
}
// import_hash of the bytes ref points at, read a few MB at a time: the same hash, as FNV-1a carries
// on from where it left off.
import_hash_file :: (ref: Mcap_Original_Ref) -> u64, ok: bool {
CHUNK :: 4 * 1024 * 1024;
file, opened := file_open(ref.path, log_errors = false);
if !opened return 0, false;
defer file_close(*file);
if !file_set_position(file, ref.offset) return 0, false;
buffer := NewArray(min(ref.size, CHUNK), u8, initialized = false,, temp);
hash := FNV_64_OFFSET_BIAS;
left := ref.size;
while left > 0 {
count := min(left, CHUNK);
ok, read := file_read(file, buffer.data, count);
if !ok || read != count return 0, false;
hash = fnv1a_hash(buffer.data, count, hash);
left -= count;
}
return hash, true;
}
import_hash_text :: (hash: u64) -> string {
return tprint("%", formatInt(hash, base = 16, minimum_digits = 16));
}
@ -642,17 +834,17 @@ import_index_worker :: (thread: *Thread) -> s64 {
if n >= index_stale.count break;
item := index_stale[n];
path := tprint("%/%", importer.library, item.relative);
data, ok := read_entire_file(path, log_errors = false,, temp);
if ok {
m, parsed, error := mcap_parse(cast([] u8) data);
if parsed {
hash := import_hash(m.original);
lock(*importer.mutex);
import_record_set(item.relative, .{item.size, item.modified, hash});
unlock(*importer.mutex);
} else {
log_error("[Import] % is not a readable .mcap (%); it is left out of the index.", path, error);
}
// The original is hashed where it lies, a few MB at a time: it may be a video larger than memory.
m, original, parsed, error := mcap_read_without_original(path,, temp);
hash: u64;
hashed: bool;
if parsed hash, hashed = import_hash_file(original);
if hashed {
lock(*importer.mutex);
import_record_set(item.relative, .{item.size, item.modified, hash});
unlock(*importer.mutex);
} else {
log_error("[Import] % is not a readable .mcap (%); it is left out of the index.", path, ifx parsed then "its original cannot be read" else error);
}
atomic_add(*importer.index_done, 1);
reset_temporary_storage();
@ -773,7 +965,7 @@ import_flag_frame :: () {
import_window :: () { using ImGui;
running := importer.running;
TextColored(app_theme.palette.dim, "Imports every image under a folder (PNG, JPEG, HEIC, AVIF, WebP, GIF, BMP, TIFF, camera RAW) and Snagit .snagx captures into a folder of its own in %. The files there are only read. Each image is stored once in the whole library; videos are skipped.", viewer_folder());
TextColored(app_theme.palette.dim, "Imports every image (PNG, JPEG, HEIC, AVIF, WebP, GIF, BMP, TIFF, camera RAW), video (MP4, MOV, MKV, WebM, AVI, WMV, MPEG) and Snagit .snagx capture under a folder into a folder of its own in %. The files there are only read. Each is stored once in the whole library.", viewer_folder());
Spacing();
if running BeginDisabled();

View File

@ -293,18 +293,20 @@ ocr_read_and_store :: (items: [] Scan_Item, set: *Label_Set) -> [] string {
label_of: [..] s64; // which item each image to label is
label_of.allocator = temp;
for item, n: items {
data, read_ok := read_entire_file(item.path, log_errors = false,, temp);
if !read_ok { errors[n] = copy_string("cannot be read"); continue; }
m, ok, error := mcap_parse(cast([] u8) data);
// Only the front when the image is not wanted; never a video's original, which may be large.
m: Mcap;
bytes: [] u8;
kind: string;
ok: bool;
error: string;
if item.ocr || item.embed m, bytes, kind, ok, error = mcap_read_shown_image(item.path,, temp);
else m, ok, error = mcap_read_front(item.path, preview = false,, temp);
if !ok { errors[n] = copy_string(error); continue; }
if item.relabel {
chunk := mcap_find_chunk(m.front_chunks, mcap_tag(MCAP_EMBEDDING_TAG), LABELS_MODEL);
if chunk prepared[n].stored_vector = mcap_embedding_parse(chunk.data,, temp);
}
if !item.ocr && !item.embed continue;
bytes := mcap_rendered(m);
kind := "rendered";
if !bytes.count { bytes = m.original; kind = "original"; }
// Turned the right way up, as the Capture window shows it, so the boxes land on the text.
decoded, decoded_ok, decode_error := image_decode(bytes);
if !decoded_ok { errors[n] = sprint("its image cannot be decoded: %", decode_error); continue; }

View File

@ -132,6 +132,15 @@ viewer: struct {
full_failed: bool;
full_fit := Viewer_Fit.WINDOW;
full_zoom: float = 100; // percent of the size it was captured at; set by the fit, unless OFF
// A video or animation's original, written out to open in the default player (viewer_open_original).
open_thread: Thread;
open_started: bool;
open_busy: bool; // set by the main thread, cleared by the open thread when it ends
open_source: string; // the .mcap; heap, main thread's until the thread starts
open_name: string;
open_result: string; // the file written, or ""; heap, the open thread's until it ends
open_error: string;
}
// ---------------------------------------------------------------- startup and shutdown
@ -182,6 +191,7 @@ viewer_shutdown :: () {
unlock(*viewer.mutex);
for 1..viewer.workers_started signal(*viewer.wake);
for 0..viewer.workers_started - 1 thread_is_done(*viewer.workers[it], -1);
if viewer.open_started thread_is_done(*viewer.open_thread, -1);
background_shutdown();
}
@ -261,19 +271,16 @@ viewer_decode_preview :: (path: string) -> Decoded {
return viewer_decode_image(path, m.preview.png, .PNG);
}
// The rendered image, annotations and all, or the original when there is none.
// The rendered image, annotations and all (a video's: a frame from it), or the original when there
// is none. A video itself is never read.
viewer_decode_full :: (path: string) -> Decoded {
result: Decoded;
result.failed = true;
push_allocator(temp);
defer reset_temporary_storage();
data, read_ok := read_entire_file(path, log_errors = false);
if !read_ok { log_error("[viewer] Cannot read %.", path); return result; }
m, ok, error := mcap_parse(cast([] u8) data);
m, image, which, ok, error := mcap_read_shown_image(path);
if !ok { log_error("[viewer] % %", path, error); return result; }
image := mcap_rendered(m);
if !image.count image = m.original;
return viewer_decode_image(path, image);
}
@ -388,6 +395,7 @@ viewer_frame_gpu_work :: () {
viewer_evict();
viewer_apply_catalog_updates();
viewer_open_frame();
background_frame(); // takes back search results, and starts a search when visible_dirty (app/background.jai)
}
@ -912,6 +920,84 @@ viewer_draw_trash :: (draw: *ImGui.ImDrawList, p0: ImGui.ImVec2, p1: ImGui.ImVec
draw.AddLine(draw, .{middle, lid + w * 0.14}, .{middle, p1.y - w * 0.28}, ink, thick); // rib
}
// A video's play mark and how long it runs, or "GIF", at the bottom left of its thumbnail.
viewer_draw_media_badge :: (draw: *ImGui.ImDrawList, top_left: ImGui.ImVec2, cell: float, entry: *Catalog_Entry) {
using ImGui;
text := ifx entry.media == .VIDEO then viewer_duration_text(entry.duration) else "GIF";
text_size := CalcTextSize(text);
pad: float = 4;
mark := ifx entry.media == .VIDEO then text_size.y * 0.8 else 0;
width := pad + mark + (ifx mark > 0 then pad else 0) + text_size.x + pad;
height := text_size.y + pad;
p0 := ImVec2.{top_left.x + pad, top_left.y + cell - height - pad};
p1 := ImVec2.{p0.x + width, p0.y + height};
draw.AddRectFilled(draw, p0, p1, ImGui.ColorConvertFloat4ToU32(.{0, 0, 0, 0.65}), 4);
ink := ImGui.ColorConvertFloat4ToU32(.{1, 1, 1, 1});
x := p0.x + pad;
if mark > 0 { // a play triangle
y := p0.y + (height - mark) / 2;
draw.AddTriangleFilled(draw, .{x, y}, .{x, y + mark}, .{x + mark * 0.87, y + mark / 2}, ink);
x += mark + pad;
}
draw.AddText(draw, .{x, p0.y + pad / 2}, ink, text);
}
// 0:42, 12:05, 1:02:03. Temp.
viewer_duration_text :: (seconds: float64) -> string {
total := cast(s64) (seconds + 0.5);
two :: (n: s64) -> string { return tprint("%", formatInt(n, minimum_digits = 2)); }
if total >= 3600 return tprint("%:%:%", total / 3600, two(total / 60 % 60), two(total % 60));
return tprint("%:%", total / 60, two(total % 60));
}
// Writes the capture's original (a video, an animated GIF) as it was imported to a file in the
// temporary folder, on a thread of its own, as it may be large; viewer_open_frame opens it.
viewer_open_original :: (index: s32) {
if viewer.open_started return;
capture := *viewer.captures[index];
push_allocator(context.default_allocator);
free(viewer.open_source); free(viewer.open_name); free(viewer.open_result); free(viewer.open_error);
viewer.open_source = copy_string(capture.path);
viewer.open_name = copy_string(capture.name);
viewer.open_result = "";
viewer.open_error = "";
viewer.open_busy = true;
viewer.open_started = thread_init(*viewer.open_thread, viewer_open_proc, temporary_storage_size = 256 * 1024, name = "open original");
if !viewer.open_started { viewer.open_busy = false; toast_show(.Error, "Could not open %", capture.name); return; }
thread_start(*viewer.open_thread);
}
viewer_open_proc :: (thread: *Thread) -> s64 {
context.allocator = context.default_allocator;
defer atomic_swap(*viewer.open_busy, false);
m, original, ok, error := mcap_read_without_original(viewer.open_source,, temp);
if !ok { viewer.open_error = copy_string(error); return 0; }
#if OS == .WINDOWS base := windows_expand_environment("%TEMP%");
else base := "/tmp";
folder := tprint("%/%", base, APP_FOLDER_NAME);
make_directory_if_it_does_not_exist(folder, recursive = true);
extension := path_extension(m.original_name);
path := tprint("%/%.%", folder, viewer.open_name, ifx extension then extension else "bin");
_, size, exists := file_modtime_and_size(path);
if !exists || size != original.size { // written out before, it is used again
ok, error = mcap_extract_original(original, path);
if !ok { viewer.open_error = copy_string(error); return 0; }
}
viewer.open_result = copy_string(path);
return 0;
}
// Each frame: opens what viewer_open_proc wrote, once it is done.
viewer_open_frame :: () {
if !viewer.open_started || viewer.open_busy return;
thread_is_done(*viewer.open_thread, -1);
thread_deinit(*viewer.open_thread);
viewer.open_started = false;
if viewer.open_result platform_open_file_in_default_program(viewer.open_result, path_strip_filename(viewer.open_result));
else toast_show(.Error, "Could not write % out: %", viewer.open_name, viewer.open_error);
}
viewer_grid :: () { using ImGui;
style := GetStyle();
cell := viewer.thumb_size;
@ -963,6 +1049,7 @@ viewer_grid :: () { using ImGui;
draw.AddRectFilled(draw, top_left, bottom_right, placeholder);
if capture.failed draw.AddText(draw, .{top_left.x + 6, top_left.y + 6}, text_color, "Cannot read");
}
if capture.catalog && capture.catalog.media != .IMAGE viewer_draw_media_badge(draw, top_left, cell, capture.catalog);
if index == viewer.full_index draw.AddRect(draw, top_left, bottom_right, selected, 0, 3);
else if hovered draw.AddRect(draw, top_left, bottom_right, outline, 0, 2);
@ -1028,6 +1115,23 @@ viewer_full_window :: () { using ImGui;
if SliderFloat("Zoom", *viewer.full_zoom, VIEWER_MIN_ZOOM, VIEWER_MAX_ZOOM, "%.0f%%", .Logarithmic) viewer.full_fit = .OFF;
SameLine();
if SmallButton("100%") { viewer.full_fit = .OFF; viewer.full_zoom = 100; }
// A video or animation: what it is, and its original in the default player.
if capture.catalog && capture.catalog.media != .IMAGE {
entry := capture.catalog;
if entry.media == .VIDEO {
TextColored(app_theme.palette.dim, "Video, %%%", viewer_duration_text(entry.duration), ifx entry.video_codec then tprint(", %", entry.video_codec) else "",
ifx entry.frame_rate > 0 then tprint(", % fps", FF(entry.frame_rate, ifx entry.frame_rate == floor(entry.frame_rate) then 0 else 2)) else "");
} else {
TextColored(app_theme.palette.dim, "Animated GIF, %", viewer_duration_text(entry.duration));
}
SameLine();
busy := viewer.open_started;
if busy BeginDisabled();
if SmallButton(temp_c_string(ifx entry.media == .VIDEO then "Play in the default player" else "Open in the default viewer")) viewer_open_original(viewer.full_index);
if busy EndDisabled();
if IsItemHovered(.AllowWhenDisabled) SetTooltip("Writes the file as it was imported to the temporary folder, and opens it");
if busy { SameLine(); TextColored(app_theme.palette.dim, "Writing it out..."); }
}
ocr_ui_capture_controls(viewer.full_index, capture.path); // app/ocr_ui.jai
labels_ui_capture_line(viewer.full_index); // app/labels_ui.jai

View File

@ -1,6 +1,6 @@
// The importer's pieces on their own (app/import.jai, modules/Image_Decode): what kind of file bytes
// are, EXIF, turning an image by its orientation, the dates found in EXIF and in file names, and the
// folder: search.
// folder: search; videos and animations, and the .mcap file read and written with its original left on disk.
#scope_file
@ -62,6 +62,15 @@ kind_tests :: () {
expect(kind("\0\0\0\x18ftypisom\0\0\0\0isommp41") == .VIDEO, "MP4");
expect(kind("\0\0\0\x14ftypqt \0\0\0\0qt ") == .VIDEO, "QuickTime");
expect(kind("\0\0\0\x18ftypcrx \0\0\0\x01crx isom") == .RAW, "Canon CR3");
expect(kind("\x1A\x45\xDF\xA3\x01\0\0\0") == .VIDEO, "Matroska / WebM");
expect(kind("RIFF\0\0\0\0AVI LIST") == .VIDEO, "AVI");
expect(kind("\x30\x26\xB2\x75\x8E\x66\xCF\x11\xA6\xD9") == .VIDEO, "WMV");
expect(kind("\0\0\x01\xBA\x44\0\x04") == .VIDEO, "MPEG program stream");
expect(kind("\0\0\0\x08wide\0\0\0\0mdat") == .VIDEO, "old QuickTime");
expect(video_format_name(cast([] u8) "\0\0\0\x14ftypqt \0\0\0\0qt ", "IMG_7011.MOV") == "QuickTime", "MOV's format");
expect(video_format_name(cast([] u8) "\0\0\0\x18ftypisom\0\0\0\0isommp41", "a.mp4") == "MP4", "MP4's format");
expect(video_format_name(cast([] u8) "\x1A\x45\xDF\xA3", "clip.webm") == "WebM", "WebM's format, by its name");
expect(video_format_name(cast([] u8) "\x1A\x45\xDF\xA3", "clip.mkv") == "Matroska", "Matroska's format");
expect(kind("hello") == .UNKNOWN, "text");
expect(kind("") == .UNKNOWN, "nothing");
expect(image_kind_from_name(.TIFF, "IMG_0001.DNG") == .RAW, "a DNG is RAW");
@ -217,6 +226,149 @@ folder_tests :: () {
expect(!search_matches(*entry, search_parse("snagitlibrarywork")), "a plain word does not match the folder");
}
video_tests :: () {
group("video and animation");
// Rotation from a display matrix (16.16 fixed point): an iPhone held upright stores its frames a
// quarter turn anticlockwise, so they are turned 90 degrees clockwise.
ONE :: 0x10000;
expect(display_matrix_rotation(.[ONE, 0, 0, 0, ONE, 0, 0, 0, 0x40000000]) == 0, "no turn");
expect(display_matrix_rotation(.[0, ONE, 0, -ONE, 0, 0, 0, 0, 0x40000000]) == 90, "portrait: 90 clockwise");
expect(display_matrix_rotation(.[-ONE, 0, 0, 0, -ONE, 0, 0, 0, 0x40000000]) == 180, "upside down");
expect(display_matrix_rotation(.[0, -ONE, 0, ONE, 0, 0, 0, 0, 0x40000000]) == 270, "270 clockwise");
expect(display_matrix_rotation(.[0, 0, 0, 0, 0, 0, 0, 0, 0]) == 0, "an empty matrix");
latitude, longitude, altitude, has_altitude, ok := iso6709_parse("+51.5072-000.1276+011.000/");
expect(ok && abs(latitude - 51.5072) < 0.000001 && abs(longitude + 0.1276) < 0.000001 && has_altitude && abs(altitude - 11) < 0.000001, "Apple's location: % % %", latitude, longitude, altitude);
latitude, longitude, altitude, has_altitude, ok = iso6709_parse("+37.7749-122.4194/");
expect(ok && abs(latitude - 37.7749) < 0.000001 && abs(longitude + 122.4194) < 0.000001 && !has_altitude, "Android's location, with no altitude");
_, _, _, _, ok = iso6709_parse("");
expect(!ok, "no location");
_, _, _, _, ok = iso6709_parse("+95.0+10.0/");
expect(!ok, "a latitude past the pole");
date, offset := import_video_local_date("2024-05-12T10:15:30+0100");
expect(date == "2024-05-12 10:15:30" && offset == "+01:00", "Apple's creationdate: % %", date, offset);
date, offset = import_video_local_date("2024-05-12T10:15:30-07:00");
expect(date == "2024-05-12 10:15:30" && offset == "-07:00", "with a colon in its offset: % %", date, offset);
date, offset = import_video_local_date("nonsense");
expect(date == "" && offset == "", "not a date");
expect(import_video_utc_date("1904-01-01T00:00:00.000000Z") == "", "a camera with no clock");
utc := import_video_utc_date("2024-05-12T09:15:30.000000Z");
expect(utc.count == 19 && begins_with(utc, "2024-05-"), "creation_time as local time: %", utc);
// A GIF of three frames, 20, 0 (shown for 100 ms) and 50 hundredths of a second, with no colour tables.
gif: [..] u8;
gif.allocator = temp;
add :: (gif: *[..] u8, bytes: string) { array_add(gif, ..cast([] u8) bytes); }
add(*gif, "GIF89a\x02\0\x02\0\0\0\0");
for delay: u8.[20, 0, 50] {
add(*gif, "\x21\xF9\x04\0");
array_add(*gif, delay, 0, 0, 0);
add(*gif, "\x2C\0\0\0\0\x02\0\x02\0\0"); // a frame, 2 x 2
add(*gif, "\x02\x02\x44\x01\0"); // LZW code size, one data block, the end
}
add(*gif, "\x3B");
frames, duration_ms, gif_ok := gif_info(gif);
expect(gif_ok && frames == 3 && duration_ms == 200 + 100 + 500, "an animated GIF: % frames, % ms", frames, duration_ms);
frames, duration_ms, gif_ok = gif_info(array_view(gif, 0, 13 + 8 + 10 + 5));
expect(gif_ok && frames == 1, "one cut short counts its whole frames: %", frames);
_, _, gif_ok = gif_info(cast([] u8) "\x89PNG\r\n\x1A\n");
expect(!gif_ok, "not a GIF");
}
// The .mcap file, version 4, and reading and writing it with its original left on disk.
mcap_tests :: () {
group("mcap version 4");
#if OS == .WINDOWS folder := tprint("%/mcap-tests", windows_expand_environment("%TEMP%"));
else folder := "/tmp/mcap-tests";
make_directory_if_it_does_not_exist(folder, recursive = true);
m: Mcap;
m.text = "[[file \"metadata.json\"]]\nWindowName: \"clip.mp4\"\n";
m.preview = .{width = 2, height = 1, png = cast([] u8) "preview"};
m.original_name = "clip.mp4";
original := NewArray(3 * 1024 * 1024 + 17, u8,, temp); // across more than one read of the copy
for * original it.* = cast,trunc(u8) (it_index * 7 + 3);
m.original = original;
m.rendered_kind = .OWN;
m.rendered = cast([] u8) "a frame";
m.back_chunks = .[.{tag = mcap_tag("mask"), key = "model", data = cast([] u8) "large"}];
bytes, serialized := mcap_serialize(m,, temp);
expect(serialized, "serialized");
back, parsed, error := mcap_parse(bytes);
expect(parsed && back.version == 4, "parses as version 4: %", error);
expect(cast(string) back.original == cast(string) original && cast(string) back.rendered == "a frame" && back.back_chunks.count == 1, "every part back");
// Version 3: the original's size a u32.
v3: [..] u8;
v3.allocator = temp;
array_add(*v3, ..array_view(bytes, 0, 8));
array_add(*v3, 3, 0, 0, 0);
head_end := bytes.count - original.count - 8 - (1 + 4 + 7 + mcap_chunks_size_for_test(back.back_chunks));
array_add(*v3, ..array_view(bytes, 12, head_end - 12));
size := cast(u32) original.count;
array_add(*v3, ..cast([] u8) string.{4, cast(*u8) *size});
array_add(*v3, ..array_view(bytes, head_end + 8));
old, old_parsed, old_error := mcap_parse(v3);
expect(old_parsed && old.version == 3 && cast(string) old.original == cast(string) original, "version 3 still reads: %", old_error);
// Streamed: the same bytes as serialized, with the original copied from a file.
source := tprint("%/source.bin", folder);
write_entire_file(source, original.data, original.count);
path := tprint("%/streamed.mcap", folder);
without := m;
without.original = .[];
written, write_error := mcap_write_streamed(path, without, .{source, 0, original.count});
data, read_ok := read_entire_file(path,, temp);
expect(written && read_ok && data == cast(string) bytes, "streamed is byte for byte serialized: %", write_error);
// Read without the original: the ref finds it.
m2, ref, ok, read_error := mcap_read_without_original(path,, temp);
expect(ok && m2.original_name == "clip.mp4" && ref.size == original.count && cast(string) m2.rendered == "a frame" && m2.back_chunks.count == 1, "read without the original: %", read_error);
read_back, range_ok := mcap_read_original(ref,, temp);
expect(range_ok && cast(string) read_back == cast(string) original, "its original where the ref says");
_, shown, which, shown_ok := mcap_read_shown_image(path,, temp);
expect(shown_ok && which == "rendered" && cast(string) shown == "a frame", "shown: the rendered image");
// Storing a scan's chunk rewrites the front and copies the original across unchanged.
stored, store_error := mcap_store(path, front = .[.{tag = mcap_tag("labl"), key = "model", data = cast([] u8) "labels"}]);
after, after_parsed := mcap_parse(cast([] u8) read_entire_file(path,, temp));
expect(stored && after_parsed && cast(string) after.original == cast(string) original && after.front_chunks.count == 1 && after.back_chunks.count == 1, "stored, the original kept: %", store_error);
erased, erase_error, changed := mcap_erase(path, tags = .["labl"]);
expect(erased && changed, "erased: %", erase_error);
// An image with no rendered one is shown as its original.
image := m;
image.rendered_kind = .NONE;
image.original = cast([] u8) "an image";
image_bytes := mcap_serialize(image,, temp);
image_path := tprint("%/image.mcap", folder);
mcap_write(image_path, image_bytes);
_, shown, which, shown_ok = mcap_read_shown_image(image_path,, temp);
expect(shown_ok && which == "original" && cast(string) shown == "an image", "shown: the original, when there is no rendered image");
// Extracted, as the Capture window does to open it.
extracted := tprint("%/extracted.mp4", folder);
extract_ok := mcap_extract_original(ref, extracted);
extracted_data := read_entire_file(extracted,, temp);
expect(extract_ok && extracted_data == cast(string) original, "extracted as it was");
// import_hash_file is import_hash, read a piece at a time.
hash, hashed := import_hash_file(.{source, 0, original.count});
expect(hashed && hash == import_hash(original), "a file's hash is its bytes' hash");
for string.[source, path, image_path, extracted] file_delete(it);
}
// What mcap_chunks_size counts, here where it cannot be seen.
mcap_chunks_size_for_test :: (chunks: [] Mcap_Chunk) -> s64 {
size: s64 = 4;
for chunks size += 4 + 4 + 2 + it.key.count + it.data.count;
return size;
}
#scope_export
run_import_tests :: (builder: *String_Builder = null) -> bool {
@ -228,5 +380,7 @@ run_import_tests :: (builder: *String_Builder = null) -> bool {
orientation_tests();
date_tests();
folder_tests();
video_tests();
mcap_tests();
return report();
}

View File

@ -1,4 +1,4 @@
// image-probe: what modules/Image_Decode makes of image files. For each FILE, its kind, EXIF and the
// image-probe: what modules/Image_Decode makes of image and video files. For each FILE, its kind, EXIF and the
// decoded size; with -out FOLDER, the decoded image (turned the right way up) and its .mcap preview
// written there as PNGs, to look at.
//
@ -19,12 +19,42 @@ main :: () {
failures := 0;
for path: files {
// A video: what video_poster finds, and its poster frame.
head: [256] u8;
head_count := 0;
file, opened := file_open(path, log_errors = false);
if opened {
_, head_count = file_read(file, head.data, head.count);
file_close(*file);
}
head_view := array_view(head, 0, head_count);
if image_kind(head_view) == .VIDEO || array_find(string.["mts", "m2ts", "ts"], to_lower_copy(path_extension(path),, temp)) {
start := current_time_monotonic();
image, info, decoded, error := video_poster(path);
print("%\n video, %\n", path, video_format_name(head_view, path));
if !decoded { print(" cannot decode: %\n", error); failures += 1; continue; }
print(" % x % as shown, % s, % fps, %, rotation %, audio %; poster at % s, in % ms\n", info.width, info.height, FF(info.duration, 2), FF(info.frame_rate, 2), info.codec, info.rotation, info.has_audio, FF(info.poster_time, 2), FF(to_float64_seconds(current_time_monotonic() - start) * 1000, 1));
print(" creationdate \"%\", creation_time \"%\", make \"%\", model \"%\", software \"%\", location \"%\"\n", info.creation_date, info.creation_time, info.make, info.model, info.software, info.location);
if out_folder {
name := path_strip_extension(path_filename(path));
stbi_write_png(temp_c_string(tprint("%/%.poster.png", out_folder, name)), image.width, image.height, 4, image.pixels, image.width * 4);
print(" wrote %/%.poster.png\n", out_folder, name);
}
image_free(*image);
reset_temporary_storage();
continue;
}
data, ok := read_entire_file(path, log_errors = false);
if !ok { print("%: cannot be read\n", path); failures += 1; continue; }
bytes := cast([] u8) data;
kind := image_kind_from_name(image_kind(bytes), path);
exif := exif_read(bytes, kind);
print("%\n kind %, % bytes\n", path, image_kind_name(kind), bytes.count);
if kind == .GIF {
frames, duration_ms := gif_info(bytes);
print(" GIF: % frames, % ms\n", frames, duration_ms);
}
if exif.found {
print(" EXIF: orientation %, date \"%\" %, make \"%\", model \"%\", software \"%\"\n", exif.orientation, exif.date_original, exif.offset_original, exif.make, exif.model, exif.software);
if exif.has_gps print(" GPS: %, %, % m\n", exif.latitude, exif.longitude, exif.altitude);

View File

@ -59,21 +59,26 @@ main :: () {
for models array_add(*model_names, it.name);
}
// The captures, decoded once: the original image of each, as the camera took it.
// The captures, decoded once: the original image of each, as the camera took it; a video's frame.
Capture :: struct { name: string; pixels: *u8; width, height: s32; }
captures: [..] Capture;
decode_start := current_time_monotonic();
for file_list(folder, recursive = true) {
if captures.count >= limit break;
if !ends_with_nocase(it, ".mcap") continue;
data, ok := read_entire_file(it);
if !ok continue;
m, parsed, error := mcap_parse(cast([] u8) data);
m, original, parsed, error := mcap_read_without_original(it,, temp);
if !parsed { print("Skipping %: %\n", it, error); continue; }
image, decoded, decode_error := image_decode(m.original);
head, head_ok := mcap_read_original(.{original.path, original.offset, min(original.size, 256)},, temp);
bytes := m.rendered;
if !head_ok || image_kind(head) != .VIDEO {
ok: bool;
bytes, ok = mcap_read_original(original,, temp);
if !ok { print("Skipping %: it cannot be read.\n", it); continue; }
}
image, decoded, decode_error := image_decode(bytes);
if !decoded { print("Skipping %: its image cannot be decoded (%).\n", it, decode_error); continue; }
array_add(*captures, .{copy_string(path_strip_extension(path_filename(it))), image.pixels, image.width, image.height});
free(data);
reset_temporary_storage();
}
if !captures.count { print("No .mcap files in %.\n", folder); exit(1); }
quick_sort(captures, (a: Capture, b: Capture) -> s64 { return compare(a.name, b.name); });