[0.0.0] .mcap version 2: OCR stored in each capture, search, a library scan, sorting by date; turned text

This commit is contained in:
Musa Mahmood 2026-10-07 21:51:18 +01:00
parent bb94f03b94
commit ea9f055abc
23 changed files with 2776 additions and 251 deletions

View File

@ -29,6 +29,25 @@ and everything around them, ready for an app to be written into `src/app/app_ui.
progress, so it never looks hung. progress, so it never looks hung.
- **Asynchronous tests**, compiled into debug builds and run with `run_tests`. - **Asynchronous tests**, compiled into debug builds and run with `run_tests`.
## The capture viewer
`src/app/viewer.jai`, `catalog.jai` and `ocr_ui.jai`: every `.mcap` in `D:\SnagitIndex\mcap` as a grid
of previews, and one capture at full size in the Capture window.
- **Search** above the grid: every word must match, anywhere in the OCR text, window title, URL, app
name or annotation text; `"a phrase"` matches as one; `ocr:`, `title:`, `url:`, `app:` and `note:`
limit a word to one field. In the Capture window the matching OCR lines are highlighted, with Next
and Previous.
- **Order:** newest first by capture date, once the catalog (what search needs from every file) has
loaded; by name until then.
- **OCR** (`modules/OCR`): the OCR window shows whether it runs on the GPU and, if not, what to install,
and switches model (kept as `model` under `[ocr]` in the settings file). **Scan the library** reads
every capture shown that the model has not read yet, storing each reading in its file as it goes,
so it can pause, cancel and pick up again. `-library FOLDER` shows another folder of `.mcap` files;
`-scan` scans it with the chosen model once it has loaded and exits (0, or 1 if any capture
failed), for a scan without the UI, such as on a copy. The Capture window's **OCR** button reads one capture;
a capture read by several models keeps each reading, and the window can show any of them.
## Building ## Building
Requires the Jai compiler, beta 0.2.030 or later. Requires the Jai compiler, beta 0.2.030 or later.

View File

@ -0,0 +1,93 @@
# The .mcap file
One capture in one file: a fixed magic, a file version stamp, the capture's data as
[Capture Text](capture_text.md), what OCR read in it, a small preview, then the images. There is no ZIP
and no compression around it; the images keep their own PNG compression.
The parts a library browser needs (header, text, OCR and preview) come first, so it reads the start of
each file and never touches the images; a search, which needs no preview, reads less still.
## Layout, version 2
All integers are little-endian. Sizes count bytes.
| Field | Type | |
| --- | --- | --- |
| magic | 8 bytes | `89 4D 43 41 50 0D 0A 1A`, which is `\x89MCAP\r\n\x1A`. Says this is an .mcap, in every version. |
| file version | u32 | `2`. Version 1 files are still read; see below. |
| text size, text | u32, bytes | Capture Text, UTF-8: every JSON file of the capture, as `[[file]]` sections. |
| OCR size, OCR | u32, bytes | OCR text, UTF-8 (below): what each OCR model read. Size 0 when none has. |
| preview width, height | u16, u16 | In pixels. |
| preview size, preview | u32, bytes | PNG. Size 0 when there is none. |
| original name size, name | u16, bytes | The original image's name in the .snagx, such as `{GUID}.png`; the page's `CaptureBackgroundImage` names it too. |
| original size, original | u32, bytes | The capture as taken: PNG, exactly as it was in the .snagx. |
| rendered kind | u8 | 0: none. 1: the same as the original (stored once). 2: its own, below. |
| rendered size, rendered | u32, bytes | Only for kind 2. The capture with its annotations drawn in (Snagit's `thumbnail.png`): PNG, exactly as it was. |
The file ends there: anything after the last part is a mistake.
The magic borrows PNG's: the first byte has its high bit set, so a channel that strips it shows; `\r\n`
shows line endings being converted; and `\x1A` stops `type` on Windows.
**Version 1** is the same without the OCR size and OCR. A reader takes both; anything written is
version 2, so a version 1 file becomes version 2 the first time OCR is stored in it.
## The parts
- **Text.** Everything the capture's JSON held: `index.json`, `metadata.json`, the page's `{GUID}.json`,
and `{GUID}.backup.json` when there is one.
- **OCR.** One section per OCR model, so a capture read by several models keeps every reading;
reading it again with a model replaces only that model's section. See *OCR text* below.
- **Preview.** The rendered image (or the original, when there is no rendered one) scaled so its longest
side is at most 320 pixels, as a PNG. An image already that small is used as it is, never scaled up.
- **Original and rendered.** A capture with no annotations has a rendered image identical to the original,
byte for byte; it is then stored once, as kind 1.
- **One page.** An .mcap holds one page, as every capture so far does; a .snagx with more is not converted.
- **Left out:** `{GUID}.backup.png`, Snagit's image from before an edit. Its `{GUID}.backup.json` is kept,
as plenty of captures have one without the image.
## Versions
A reader checks the magic, then the file version. It refuses a version above its own, never writing
over a file from a later version, and reads every version up to its own. Version 0 does not exist.
## Converting
`snagit-converter mcap` converts .snagx files (or extracted captures) to .mcap, and `snagit-converter
snagx` converts .mcap files back to a .snagx Snagit can open. See the README for what each checks.
Back in a .snagx, the JSON is laid out as Snagit writes it, the images are the same bytes, and only the
backup image is missing. JSON numbers come back as values: `3.0` is written `3`.
## OCR text
In the style of the Ex4 text formats:
```
@version 2
[ocr ppocr-v5-mobile]
engine: "onnxruntime 1.30.0, CUDA"
image: rendered 1920 1080
date: 2026-10-08T21:04:11Z
line: 412 96 640 18 412 96 1052 96 1052 114 412 114 0.993 "Assign Footprints - KiCad"
line: 20 140 220 14 20 140 240 140 240 154 20 154 0.981 "Symbol : Footprint Assignments"
line: 1700 300 20 160 1720 300 1720 460 1700 460 1700 300 0.952 "Read downwards"
```
| Key | Value |
| --- | --- |
| `[ocr NAME]` | A section: the model's name, as in its `models.config`. |
| `engine` | What ran it, quoted. |
| `image` | Which image was read (`rendered` or `original`) and its width and height; the lines' boxes are in its pixels. |
| `date` | When it was read, UTC. |
| `line` | A text line: x, y, width and height, then its four corners as x y pairs, its confidence (0 to 1), and its text, quoted with `""` for a quote. A control character in the text is written as a space. |
The corners go clockwise from the text's top left: top left, top right, bottom right, bottom left, as
the text reads. Level text has the box's corners; a turned line's say which way it reads (the third
line above reads downwards, so its top left is the box's top right). x, y, width and height bound them.
A line that cannot be read is skipped, as are keys this version does not know. **Version 1** OCR text
had no corners (`line: x y width height confidence "text"`); it is still read, with the box's corners.
The OCR text carries its own version, so this did not change the .mcap file version. `mcap_store_ocr`
(`modules/Capture_File/ocr_text.jai`) stores a section in a file.

View File

@ -1,10 +1,13 @@
// .mcap: one capture in one file. A fixed magic, a file version stamp, the capture's Capture Text, // .mcap: one capture in one file. A fixed magic, a file version stamp, the capture's Capture Text,
// a small preview, then the images. See docs/mcap.md for the layout. // its OCR text (version 2), a small preview, then the images. See docs/mcap.md for the layout.
// //
// Reading gives views into the bytes it was given; nothing is copied. Images stay PNG. // Reading gives views into the bytes it was given; nothing is copied. Images stay PNG.
//
// The main copy is mcap's (D:/jai_projects/active/mcap/modules/Capture_File); SnagitConverter has
// the same files.
MCAP_MAGIC :: u8.[0x89, #char "M", #char "C", #char "A", #char "P", 0x0D, 0x0A, 0x1A]; MCAP_MAGIC :: u8.[0x89, #char "M", #char "C", #char "A", #char "P", 0x0D, 0x0A, 0x1A];
MCAP_VERSION :: 1; MCAP_VERSION :: 2; // what is written; every version from 1 up is read
MCAP_EXTENSION :: "mcap"; MCAP_EXTENSION :: "mcap";
MCAP_PREVIEW_MAX_SIDE :: 320; // the preview's longest side, in pixels MCAP_PREVIEW_MAX_SIDE :: 320; // the preview's longest side, in pixels
@ -23,6 +26,7 @@ Mcap_Image :: struct {
Mcap :: struct { Mcap :: struct {
version: u32 = MCAP_VERSION; // as read; always MCAP_VERSION when written version: u32 = MCAP_VERSION; // as read; always MCAP_VERSION when written
text: string; // Capture Text text: string; // Capture Text
ocr: string; // OCR text (ocr_text.jai), one section per model; "" when none, and always in version 1
preview: Mcap_Image; preview: Mcap_Image;
original_name: string; // the original image's name in the .snagx, such as {GUID}.png original_name: string; // the original image's name in the .snagx, such as {GUID}.png
original: [] u8; // PNG original: [] u8; // PNG
@ -44,7 +48,7 @@ mcap_rendered :: (m: Mcap) -> [] u8 {
// The file's bytes. A part too large for its size field fails the whole file rather than writing one // The file's bytes. A part too large for its size field fails the whole file rather than writing one
// that cannot be read back. // that cannot be read back.
mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string { mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string {
size := MCAP_HEADER_SIZE + 4 + m.text.count + 8 + m.preview.png.count + 2 + m.original_name.count + 4 + m.original.count + 1; size := MCAP_HEADER_SIZE + 4 + m.text.count + 4 + m.ocr.count + 8 + m.preview.png.count + 2 + m.original_name.count + 4 + m.original.count + 1;
if m.rendered_kind == .OWN size += 4 + m.rendered.count; if m.rendered_kind == .OWN size += 4 + m.rendered.count;
s := new_serializer(size); s := new_serializer(size);
@ -53,6 +57,7 @@ mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string {
Add(*s, cast(u32) MCAP_VERSION); Add(*s, cast(u32) MCAP_VERSION);
if !AddString(*s, m.text, u32) return s, false, "the text is too large for its size field"; if !AddString(*s, m.text, u32) return s, false, "the text is too large for its size field";
if !AddString(*s, m.ocr, u32) return s, false, "the OCR text is too large for its size field";
Add(*s, m.preview.width); Add(*s, m.preview.width);
Add(*s, m.preview.height); Add(*s, m.preview.height);
if !AddSizedArray(*s, m.preview.png, u32) return s, false, "the preview is too large for its size field"; if !AddSizedArray(*s, m.preview.png, u32) return s, false, "the preview is too large for its size field";
@ -95,10 +100,10 @@ mcap_parse :: (data: [] u8) -> Mcap, ok: bool, error: string {
return m, true, ""; return m, true, "";
} }
// Only the header, text and preview, as a library browser needs them: one read of the start of the // Only the front of the file, as a library browser needs it: the header, text, OCR text and, with
// file, and at most two more when the text or preview runs past it. The images are left unread and // preview, the preview. One read of the start of the file, and a few more only when a part runs past
// empty. The parts are views into a buffer from the context allocator. // it. The images are left unread and empty. The parts are views into a buffer from the context allocator.
mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string { mcap_read_front :: (path: string, preview := true) -> Mcap, ok: bool, error: string {
m: Mcap; m: Mcap;
file, opened := file_open(path, log_errors = false); file, opened := file_open(path, log_errors = false);
if !opened return m, false, "cannot be opened"; if !opened return m, false, "cannot be opened";
@ -107,17 +112,17 @@ mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string {
if !length_ok return m, false, "cannot be read"; if !length_ok return m, false, "cannot be read";
wanted := min(length, MCAP_FRONT_READ_SIZE); wanted := min(length, MCAP_FRONT_READ_SIZE);
for attempt: 1..3 { for attempt: 1..5 { // each further read reaches at least one more size field
buffer := NewArray(wanted, u8, initialized = false); buffer := NewArray(wanted, u8, initialized = false);
if !file_read(file, buffer.data, wanted) { array_free(buffer); return m, false, "cannot be read"; } if !file_read(file, buffer.data, wanted) { array_free(buffer); return m, false, "cannot be read"; }
d := to_deserializer(buffer); d := to_deserializer(buffer);
ok, error := mcap_read_front_part(*d, *m); ok, error := mcap_read_front_part(*d, *m, preview);
if ok return m, true, ""; if ok return m, true, "";
if d.ok || wanted == length { array_free(buffer); return m, false, error; } // not a matter of too few bytes if d.ok || wanted == length { array_free(buffer); return m, false, error; } // not a matter of too few bytes
// Short: the sizes read so far say how much is needed. // Short: the sizes read so far say how much is needed.
needed := mcap_front_size(buffer); needed := mcap_front_size(buffer, preview);
array_free(buffer); array_free(buffer);
if needed <= wanted || needed > length return m, false, "the file ends early (it is truncated or damaged)"; if needed <= wanted || needed > length return m, false, "the file ends early (it is truncated or damaged)";
wanted = needed; wanted = needed;
@ -131,8 +136,8 @@ mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string {
MCAP_HEADER_SIZE :: 12; // magic and version MCAP_HEADER_SIZE :: 12; // magic and version
MCAP_FRONT_READ_SIZE :: 64 * 1024; // holds the header, text and preview of nearly every capture MCAP_FRONT_READ_SIZE :: 64 * 1024; // holds the header, text and preview of nearly every capture
// Header, text and preview. // Header, text, OCR text and, with preview, the preview.
mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string { mcap_read_front_part :: (d: *Deserializer, m: *Mcap, preview := true) -> ok: bool, error: string {
magic := ReadArrayView(d, MCAP_MAGIC.count, u8); magic := ReadArrayView(d, MCAP_MAGIC.count, u8);
if !d.ok return false, "is not an .mcap file (it is shorter than the magic)"; if !d.ok return false, "is not an .mcap file (it is shorter than the magic)";
for MCAP_MAGIC if magic[it_index] != it return false, "is not an .mcap file (its magic is wrong)"; for MCAP_MAGIC if magic[it_index] != it return false, "is not an .mcap file (its magic is wrong)";
@ -142,6 +147,11 @@ mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string
if m.version > MCAP_VERSION return false, tprint("is file version %, but this build reads up to %; it was written by a later version", m.version, MCAP_VERSION); if m.version > MCAP_VERSION return false, tprint("is file version %, but this build reads up to %; it was written by a later version", m.version, MCAP_VERSION);
m.text = ReadSizedString(d, u32); m.text = ReadSizedString(d, u32);
if m.version >= 2 m.ocr = ReadSizedString(d, u32); // version 1 has no OCR block
if !preview {
if !d.ok return false, "the file ends early (it is truncated or damaged)";
return true, "";
}
m.preview.width = Read(d, u16); m.preview.width = Read(d, u16);
m.preview.height = Read(d, u16); m.preview.height = Read(d, u16);
m.preview.png = ReadSizedArray(d, u8, u32); m.preview.png = ReadSizedArray(d, u8, u32);
@ -149,18 +159,31 @@ mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string
return true, ""; return true, "";
} }
// How many bytes the header, text and preview take, as far as the sizes in prefix tell: with only the // How many bytes the front takes, as far as the size fields in prefix tell: the whole front when
// text's size in it, enough to reach the preview's size; with that too, the whole front. // they are all in it, otherwise how far to read to reach the next size field.
mcap_front_size :: (prefix: [] u8) -> s64 { mcap_front_size :: (prefix: [] u8, preview := true) -> s64 {
d := to_deserializer(prefix); d := to_deserializer(prefix);
d.cursor = MCAP_HEADER_SIZE; d.cursor = MCAP_MAGIC.count;
text_size := cast(s64) Read(*d, u32); version := Read(*d, u32);
preview_header := MCAP_HEADER_SIZE + 4 + text_size + 4; // where the preview's size is if !d.ok return MCAP_HEADER_SIZE;
if !d.ok || preview_header + 4 > prefix.count return preview_header + 4; position: s64 = MCAP_HEADER_SIZE;
d.cursor = preview_header; skip_sized :: (prefix: [] u8, position: *s64) -> reached: bool { // a u32 size and what it counts
return preview_header + 4 + cast(s64) Read(*d, u32); if position.* + 4 > prefix.count { position.* += 4; return false; }
d := to_deserializer(prefix);
d.cursor = position.*;
position.* += 4 + cast(s64) Read(*d, u32);
return true;
}
if !skip_sized(prefix, *position) return position; // the text
if version >= 2 && !skip_sized(prefix, *position) return position; // the OCR text
if !preview return position;
position += 4; // the preview's width and height
skip_sized(prefix, *position); // the preview
return position;
} }
#load "ocr_text.jai";
#import "Basic"; #import "Basic";
#import "File"; #import "File";
#import "File_Utilities"; #import "File_Utilities";

View File

@ -0,0 +1,209 @@
// OCR text: the OCR block of an .mcap (version 2), in the style of the Ex4 text formats. One section
// per model, so a capture read by several models keeps every reading; reading it again with a model
// replaces only that model's section.
//
// @version 2
// [ocr ppocr-v5-mobile]
// engine: "onnxruntime 1.30.0, CUDA"
// image: rendered 1920 1080
// date: 2026-10-08T21:04:11Z
// line: 412 96 640 18 412 96 1052 96 1052 114 412 114 0.993 "Assign Footprints - KiCad"
//
// line: x y width height, the line's four corners as x y pairs, confidence, "text". All in the pixels
// of the image named by image: (rendered or original, at that size). x y width height bound the
// corners; the corners go clockwise from the text's top left, so a turned line says which way it
// reads. Text is quoted, with "" for a quote; a control character is written as a space. A line that
// cannot be read is skipped; keys this version does not know are skipped too.
//
// Version 1 had no corners (line: x y width height confidence "text"); it is still read, its corners
// taken from its box.
MCAP_OCR_TEXT_VERSION :: 2;
Mcap_Ocr_Line :: struct {
x, y, width, height: s32;
confidence: float;
text: string;
corners: [8] s32; // x0 y0 x1 y1 x2 y2 x3 y3: top left, top right, bottom right, bottom left of the text; all 0 means the box's
}
// The line's corners: its own, or its box's when it has none.
mcap_ocr_corners :: (line: Mcap_Ocr_Line) -> [8] s32 {
for line.corners if it return line.corners;
x0, y0, x1, y1 := line.x, line.y, line.x + line.width, line.y + line.height;
return .[x0, y0, x1, y0, x1, y1, x0, y1];
}
Mcap_Ocr_Section :: struct {
model: string; // the OCR model's name, as in its models.config
engine: string; // what ran it, such as "onnxruntime 1.30.0, CUDA"
image: string; // "rendered" or "original"
image_width, image_height: s32;
date: string; // when it was read, UTC: 2026-10-08T21:04:11Z
lines: [..] Mcap_Ocr_Line;
}
// The sections of an OCR block; strings and arrays from the context allocator. "" gives none.
mcap_ocr_parse :: (text: string) -> [..] Mcap_Ocr_Section, ok: bool, error: string {
sections: [..] Mcap_Ocr_Section;
if !text.count return sections, true, "";
section: *Mcap_Ocr_Section;
version_seen := false;
version: s64;
for raw: split(text, "\n",, temp) {
line := trim(raw);
if !line.count || line[0] == #char "#" continue;
if begins_with(line, "@version") {
rest := trim(advance(line, 8));
number, ok := string_to_int(rest);
if !ok || number < 1 || number > MCAP_OCR_TEXT_VERSION return sections, false, tprint("its OCR text is version %, but this build reads up to %", rest, MCAP_OCR_TEXT_VERSION);
version = number;
version_seen = true;
continue;
}
if !version_seen return sections, false, "its OCR text does not start with @version";
if begins_with(line, "[ocr ") && ends_with(line, "]") {
section = array_add(*sections);
section.* = .{};
section.model = copy_string(trim(slice(line, 5, line.count - 6)));
continue;
}
colon := find_index_from_left(line, #char ":");
if colon < 0 || !section continue;
key := trim(slice(line, 0, colon));
value := trim(advance(line, colon + 1));
if key == {
case "engine";
quoted, _, ok := ocr_read_quoted(value);
if ok section.engine = copy_string(quoted);
case "image";
parts := split(value, " ",, temp);
if parts.count == 3 {
section.image = copy_string(parts[0]);
section.image_width = cast(s32) string_to_int(parts[1]);
section.image_height = cast(s32) string_to_int(parts[2]);
}
case "date";
section.date = copy_string(value);
case "line";
ocr_line, ok := ocr_parse_line(value, version);
if ok array_add(*section.lines, ocr_line);
}
}
return sections, true, "";
}
// The OCR block for sections. Temp.
mcap_ocr_text :: (sections: [] Mcap_Ocr_Section) -> string {
if !sections.count return "";
builder: String_Builder;
builder.allocator = temp;
print_to_builder(*builder, "@version %\n", MCAP_OCR_TEXT_VERSION);
for sections {
print_to_builder(*builder, "\n[ocr %]\n", it.model);
print_to_builder(*builder, "engine: %\n", ocr_quote(it.engine));
print_to_builder(*builder, "image: % % %\n", it.image, it.image_width, it.image_height);
print_to_builder(*builder, "date: %\n", it.date);
for line: it.lines {
c := mcap_ocr_corners(line);
print_to_builder(*builder, "line: % % % % % % % % % % % % % %\n", line.x, line.y, line.width, line.height,
c[0], c[1], c[2], c[3], c[4], c[5], c[6], c[7], formatFloat(line.confidence, trailing_width = 3, zero_removal = .NO), ocr_quote(line.text));
}
}
return builder_to_string(*builder,, temp);
}
// Stores section in the .mcap at path, in place of any section by the same model, keeping the
// others, and upgrades the file to the current version. The file is rewritten beside itself and
// swapped in (mcap_write), so a failure leaves it as it was.
mcap_store_ocr :: (path: string, section: Mcap_Ocr_Section) -> ok: bool, error: string {
data, read_ok := read_entire_file(path, log_errors = false);
if !read_ok return false, "cannot be read";
defer free(data);
m, parsed, parse_error := mcap_parse(cast([] u8) data);
if !parsed return false, parse_error;
push_allocator(temp);
sections, ok, error := mcap_ocr_parse(m.ocr);
if !ok return false, error;
replaced := false;
for * sections if it.model == section.model { it.* = section; replaced = true; break; }
if !replaced array_add(*sections, section);
m.ocr = mcap_ocr_text(sections);
m.version = MCAP_VERSION;
serializer, serialized, serialize_error := mcap_serialize(m);
if !serialized return false, serialize_error;
written, write_error := mcap_write(path, serializer);
return written, write_error;
}
// The time now as the date: key writes it, UTC. Temp.
mcap_ocr_now :: () -> string {
t := to_calendar(current_time_consensus(), .UTC);
two :: (n: s64) -> string { return tprint("%", formatInt(n, minimum_digits = 2)); }
return tprint("%-%-%T%:%:%Z", t.year, two(t.month_starting_at_0 + 1), two(t.day_of_month_starting_at_0 + 1), two(t.hour), two(t.minute), two(t.second));
}
#scope_file
// Quoted, a quote written twice, a control character as a space. Temp.
ocr_quote :: (text: string) -> string {
builder: String_Builder;
builder.allocator = temp;
append(*builder, #char "\"");
for 0..text.count - 1 {
c := text[it];
if c == #char "\"" append(*builder, "\"\"");
else if c < 0x20 || c == 0x7F append(*builder, #char " ");
else append(*builder, c);
}
append(*builder, #char "\"");
return builder_to_string(*builder,, temp);
}
// One quoted string ("" for a quote) at the start of s: its text (context allocator) and what follows.
ocr_read_quoted :: (s: string) -> text: string, rest: string, ok: bool {
if !s.count || s[0] != #char "\"" return "", s, false;
builder: String_Builder;
i := 1;
while i < s.count {
if s[i] == #char "\"" {
if i + 1 < s.count && s[i + 1] == #char "\"" { append(*builder, #char "\""); i += 2; continue; }
return builder_to_string(*builder), advance(s, i + 1), true;
}
append(*builder, s[i]);
i += 1;
}
free_buffers(*builder);
return "", s, false;
}
// Version 2: x y width height x0 y0 x1 y1 x2 y2 x3 y3 confidence "text". Version 1: x y width height confidence "text".
ocr_parse_line :: (value: string, version: s64) -> Mcap_Ocr_Line, bool {
result: Mcap_Ocr_Line;
rest := value;
all_numbers: [13] float64;
numbers := array_view(all_numbers, 0, ifx version >= 2 then 13 else 5);
for * numbers {
rest = trim_left(rest);
space := find_index_from_left(rest, #char " ");
if space < 0 return result, false;
number, ok := string_to_float64(slice(rest, 0, space));
if !ok return result, false;
it.* = number;
rest = advance(rest, space);
}
text, after, ok := ocr_read_quoted(trim_left(rest));
if !ok || trim(after).count return result, false;
result.x, result.y = cast(s32) numbers[0], cast(s32) numbers[1];
result.width, result.height = cast(s32) numbers[2], cast(s32) numbers[3];
result.confidence = cast(float) numbers[numbers.count - 1];
result.text = text;
if numbers.count == 13 for 0..7 result.corners[it] = cast(s32) numbers[4 + it];
result.corners = mcap_ocr_corners(result);
return result, true;
}
#import "String";

View File

@ -0,0 +1,746 @@
// Capture Text: the text format for a capture's structured data, the JSON files of a .snagx in
// the style of the Ex4 text formats. See docs/capture_text.md for the format itself.
//
// A Capture_Document allocates everything through the allocator it is given, and frees nothing on
// its own: give it an arena or a pool, and drop that to drop the document.
CAPTURE_TEXT_VERSION :: 1;
CAPTURE_TEXT_EXTENSION :: "capture-text";
Capture_Value_Kind :: enum u8 { NULL; BOOLEAN; NUMBER; STRING; ARRAY; OBJECT; }
Capture_Value :: struct {
kind: Capture_Value_Kind;
boolean: bool;
number: float64;
text: string;
items: [..] Capture_Value; // ARRAY
fields: [..] Capture_Field; // OBJECT, in the order they were added
}
Capture_Field :: struct {
key: string;
value: Capture_Value;
}
// One [[file "NAME"]] section: one of the capture's JSON files.
Capture_File :: struct {
name: string;
root: Capture_Value; // always an OBJECT
}
Capture_Document :: struct {
allocator: Allocator;
files: [..] Capture_File;
comment: string; // written as # lines under @version; not read back
mistakes: s64; // lines skipped by the last parse
}
capture_document_init :: (doc: *Capture_Document, allocator: Allocator) {
doc.* = .{};
doc.allocator = allocator;
doc.files.allocator = allocator;
}
capture_add_file :: (doc: *Capture_Document, name: string) -> *Capture_File {
file := array_add(*doc.files);
file.name = copy_string(name,, doc.allocator);
file.root = capture_object(doc);
return file;
}
capture_find_file :: (doc: *Capture_Document, name: string) -> *Capture_File {
for * doc.files if it.name == name return it;
return null;
}
// ---------------------------------------------------------------- values
capture_null :: () -> Capture_Value {
return .{kind = .NULL};
}
capture_boolean :: (b: bool) -> Capture_Value {
return .{kind = .BOOLEAN, boolean = b};
}
capture_number :: (n: float64) -> Capture_Value {
return .{kind = .NUMBER, number = n};
}
capture_string :: (doc: *Capture_Document, text: string) -> Capture_Value {
return .{kind = .STRING, text = copy_string(text,, doc.allocator)};
}
capture_array :: (doc: *Capture_Document) -> Capture_Value {
result: Capture_Value;
result.kind = .ARRAY;
result.items.allocator = doc.allocator;
return result;
}
capture_object :: (doc: *Capture_Document) -> Capture_Value {
result: Capture_Value;
result.kind = .OBJECT;
result.fields.allocator = doc.allocator;
return result;
}
// Adds key to object without checking whether it is already there; the key is copied.
capture_object_add :: (doc: *Capture_Document, object: *Capture_Value, key: string, value: Capture_Value) -> *Capture_Value {
assert(object.kind == .OBJECT);
field := array_add(*object.fields);
field.key = copy_string(key,, doc.allocator);
field.value = value;
return *field.value;
}
capture_object_find :: (object: *Capture_Value, key: string) -> *Capture_Value {
if object.kind != .OBJECT return null;
for * object.fields if it.key == key return *it.value;
return null;
}
// Equal as JSON values: the order of an object's keys does not matter, the order of an array's
// elements does. Numbers are compared bit for bit, so -0 and 0 differ.
capture_values_equal :: (a: Capture_Value, b: Capture_Value) -> bool {
if a.kind != b.kind return false;
if #complete a.kind == {
case .NULL; return true;
case .BOOLEAN; return a.boolean == b.boolean;
case .NUMBER; return (cast(*u64) *a.number).* == (cast(*u64) *b.number).*;
case .STRING; return a.text == b.text;
case .ARRAY;
if a.items.count != b.items.count return false;
for a.items if !capture_values_equal(it, b.items[it_index]) return false;
return true;
case .OBJECT;
if a.fields.count != b.fields.count return false;
for a.fields {
other := capture_object_find(*b, it.key);
if !other || !capture_values_equal(it.value, other.*) return false;
}
return true;
}
return false;
}
// ---------------------------------------------------------------- reading
// Reads file_data into doc, which should be empty. A bad line, or a bad [[file]] with its lines, is
// logged and skipped, and counted in doc.mistakes. Returns false when the file was not read at all:
// no @version 1 before the first line that needs it.
capture_text_parse :: (doc: *Capture_Document, path: string, file_data: string) -> bool {
push_allocator(doc.allocator);
doc.mistakes = 0;
data := file_data;
if data.count >= 3 && data[0] == 0xEF && data[1] == 0xBB && data[2] == 0xBF data = advance(data, 3);
state: Capture_Parse_State;
state.doc = doc;
state.path = path;
// Every line stands alone: a value never continues onto the next line.
offset: s64;
line_number := 0;
while offset < data.count {
line_number += 1;
line, colon, valid := capture_scan_line(data, *offset);
if !valid { capture_mistake(*state, line_number, "Quote is unterminated. Ignored."); continue; }
body := trim(line);
if !body.count continue;
if colon >= 0 colon -= body.data - line.data; // from the start of body
if body[0] == #char "@" {
key, value := capture_split_directive(body);
if key == "version" {
rest := value;
version, version_ok := parse_int(*rest, s64);
if !version_ok || trim(rest).count || version < 1 || version > CAPTURE_TEXT_VERSION {
log_error("% is version %, but this build reads version %. It was not read.", path, value, CAPTURE_TEXT_VERSION);
return false;
}
state.version_seen = true;
}
continue;
}
if !state.version_seen {
log_error("% does not start with @version %. It was not read.", path, CAPTURE_TEXT_VERSION);
return false;
}
if begins_with(body, "[[") {
capture_open_file(*state, line_number, body);
} else if body[0] == #char "[" {
capture_open_section(*state, line_number, body);
} else {
capture_key_line(*state, line_number, body, colon);
}
}
return true;
}
// ---------------------------------------------------------------- writing
CAPTURE_MAX_KEY_WIDTH :: 40; // keys longer than this do not push the other values further right
// The document as its file holds it. Temp.
capture_text_write :: (doc: *Capture_Document) -> string {
builder: String_Builder;
builder.allocator = temp;
print_to_builder(*builder, "@version %\n", CAPTURE_TEXT_VERSION);
if doc.comment.count {
for split(doc.comment, "\n",, temp) print_to_builder(*builder, "# %\n", it);
}
for file: doc.files {
print_to_builder(*builder, "\n[[file %]]\n", capture_quote(file.name));
lines: [..] Capture_Line;
lines.allocator = temp;
for file.root.fields {
if capture_is_section_array(it.value) continue;
capture_collect_lines(*lines, capture_key_text(it.key), it.value);
}
capture_write_block(*builder, lines);
for field: file.root.fields {
if !capture_is_section_array(field.value) continue;
header := capture_key_text(field.key);
for item: field.value.items {
print_to_builder(*builder, "\n[%]\n", header);
array_reset_keeping_memory(*lines);
for item.fields capture_collect_lines(*lines, capture_key_text(it.key), it.value);
capture_write_block(*builder, lines);
}
}
}
return builder_to_string(*builder,, temp);
}
// Always quoted, a quote written twice. Control characters cannot be quoted; see capture_string_parts.
capture_quote :: (text: string) -> string {
return tprint("\"%\"", replace(text, "\"", "\"\"",, temp));
}
// A key as one segment of a key path: bare when it can be, quoted otherwise. A key cannot hold
// control characters (see capture_key_is_writable). Temp.
capture_key_text :: (key: string) -> string {
if capture_is_bare_key(key) return key;
return capture_quote(key);
}
// Whether a key can be written at all: a quoted key is one part, so it cannot hold the characters
// below space, or DEL, that a string writes as 0xNN bytes.
capture_key_is_writable :: (key: string) -> bool {
for 0..key.count - 1 if key[it] < 0x20 || key[it] == 0x7F return false;
return true;
}
// The shortest text that reads back as exactly n. Temp.
capture_number_text :: (n: float64) -> string {
bits := (cast(*u64) *n).*;
if n == 0 return ifx bits >> 63 then "-0" else "0";
// Whole numbers that a float64 holds exactly are written as integers: 100, not 100.0 or 1e+02.
if n > -9007199254740992.0 && n < 9007199254740992.0 && n == cast(float64) cast,trunc(s64) n {
return tprint("%", cast,trunc(s64) n);
}
// Basic's SHORTEST mode keeps only six significant digits, so the fewest digits that read back as
// exactly n are found by trying: plain decimals for ordinary magnitudes, an exponent otherwise.
// 17 significant digits always suffice for a float64.
magnitude := ifx n < 0 then -n else n;
if magnitude >= 0.000001 && magnitude < 1000000000000000000000.0 {
for digits: 1..24 {
text := trim(tprint("%", formatFloat(n, trailing_width = digits, zero_removal = .YES)));
if capture_number_reads_back(text, bits) return text;
}
}
for digits: 0..16 {
text := trim(tprint("%", formatFloat(n, trailing_width = digits, mode = .SCIENTIFIC, zero_removal = .YES)));
if capture_number_reads_back(text, bits) return text;
}
return tprint("%", formatFloat(n, trailing_width = 16, mode = .SCIENTIFIC, zero_removal = .NO));
}
capture_number_reads_back :: (text: string, bits: u64) -> bool {
if capture_number_length(text) != text.count return false;
value, ok, rest := string_to_float64(text);
return ok && !rest.count && (cast(*u64) *value).* == bits;
}
#scope_module
// ---------------------------------------------------------------- reading, in detail
Capture_Parse_State :: struct {
doc: *Capture_Document;
path: string;
version_seen: bool;
file: *Capture_File; // the [[file]] being read; null before the first, or after a bad one
base: *Capture_Value; // the object key lines go into: the file's, or a [KEY] section's
skipping: bool; // after a bad header, until the next good one
}
MAX_NESTING :: 64;
capture_mistake :: (state: *Capture_Parse_State, line_number: s64, format: string, args: ..Any) {
log_error("%:%: %", state.path, line_number, tprint(format, ..args));
state.doc.mistakes += 1;
} @PrintLike
capture_open_file :: (state: *Capture_Parse_State, line_number: s64, body: string) {
state.file = null;
state.base = null;
state.skipping = true;
if !ends_with(body, "]]") { capture_mistake(state, line_number, "Malformed header (expected `[[file \"NAME\"]]`); its lines are skipped."); return; }
inner := trim(slice(body, 2, body.count - 4));
if !begins_with(inner, "file") || inner.count < 5 || !is_space(inner[4]) {
capture_mistake(state, line_number, "Unknown header % (expected `[[file \"NAME\"]]`); its lines are skipped.", body);
return;
}
name_text := trim(advance(inner, 4));
name, rest, ok := capture_read_quoted(name_text);
if !ok || trim(rest).count { capture_mistake(state, line_number, "The file name in % must be one quoted string; its lines are skipped.", body); return; }
if capture_find_file(state.doc, name) { capture_mistake(state, line_number, "File \"%\" was already read; this section is skipped.", name); return; }
state.file = capture_add_file(state.doc, name);
state.base = *state.file.root;
state.skipping = false;
}
capture_open_section :: (state: *Capture_Parse_State, line_number: s64, body: string) {
state.base = null;
if !state.file { capture_mistake(state, line_number, "% comes before any [[file]] section; its lines are skipped.", body); state.skipping = true; return; }
state.skipping = true;
if body[body.count - 1] != #char "]" { capture_mistake(state, line_number, "Malformed header (expected `[KEY]`); its lines are skipped."); return; }
segments, ok := capture_parse_path(trim(slice(body, 1, body.count - 2)));
if !ok { capture_mistake(state, line_number, "% is not a key path; its lines are skipped.", body); return; }
array, error := capture_walk(state.doc, *state.file.root, segments, .ARRAY);
if !array { capture_mistake(state, line_number, "%: %; its lines are skipped.", body, error); return; }
element := array_add(*array.items);
element.* = capture_object(state.doc);
state.base = element;
state.skipping = false;
}
// colon: where in body the first : outside quotes is, or -1.
capture_key_line :: (state: *Capture_Parse_State, line_number: s64, body: string, colon: s64) {
if state.skipping return;
if !state.base { capture_mistake(state, line_number, "Key line before any [[file]] section. Ignored."); return; }
if colon < 0 { capture_mistake(state, line_number, "Expected `KEY: VALUE`. Ignored."); return; }
key_text := trim(slice(body, 0, colon));
value_text := trim(advance(body, colon + 1));
segments, path_ok := capture_parse_path(key_text);
if !path_ok { capture_mistake(state, line_number, "% is not a key path. Ignored.", key_text); return; }
rest := value_text;
value, value_ok := capture_parse_value(state.doc, *rest, 0);
if !value_ok || trim(rest).count { capture_mistake(state, line_number, "% has no value it can read in `%`. Ignored.", key_text, value_text); return; }
slot, error := capture_walk(state.doc, state.base, segments, value.kind, value_must_be_new = true);
if !slot { capture_mistake(state, line_number, "%: %. Ignored.", key_text, error); return; }
slot.* = value;
}
Capture_Path_Segment :: struct {
key: string;
index := -1; // an array index when 0 or more
}
// Key path: segments separated by dots, each a bare name, a quoted key, or digits for an array index.
capture_parse_path :: (text: string) -> [..] Capture_Path_Segment, bool {
segments: [..] Capture_Path_Segment;
segments.allocator = temp;
rest := text;
while true {
segment: Capture_Path_Segment;
if rest.count && rest[0] == #char "\"" {
key, after, ok := capture_read_quoted(rest);
if !ok return segments, false;
segment.key = key;
rest = after;
} else {
i := 0;
while i < rest.count && capture_is_bare_char(rest[i]) i += 1;
if i == 0 return segments, false;
word := slice(rest, 0, i);
rest = advance(rest, i);
if capture_is_digits(word) {
if word.count > 9 return segments, false; // no array is that long
index_text := word;
segment.index = parse_int(*index_text, s64);
} else {
segment.key = word;
}
}
array_add(*segments, segment);
if !rest.count return segments, true;
if rest[0] != #char "." return segments, false;
rest = advance(rest, 1);
}
return segments, false;
}
// Follows segments down from node, making the objects and arrays the path needs, and returns the
// slot at the end: a new one for a key line, which must not have a value yet, or for a [KEY] header
// the array that is there or a new one. On failure returns null and why.
capture_walk :: (doc: *Capture_Document, node: *Capture_Value, segments: [] Capture_Path_Segment, final_kind: Capture_Value_Kind, value_must_be_new := false) -> *Capture_Value, string {
current := node;
for segment, segment_index: segments {
last := segment_index == segments.count - 1;
// What a new slot here has to be: what the next segment needs, or what is being put at the end.
kind := final_kind;
if !last kind = ifx segments[segment_index + 1].index >= 0 then Capture_Value_Kind.ARRAY else .OBJECT;
child: *Capture_Value;
is_new := false;
if segment.index >= 0 {
if current.kind != .ARRAY return null, tprint("index % of something that is not an array", segment.index);
if segment.index > current.items.count return null, tprint("index % skips past the next free index %", segment.index, current.items.count);
if segment.index == current.items.count {
child = array_add(*current.items);
is_new = true;
} else {
child = *current.items[segment.index];
}
} else {
if current.kind != .OBJECT return null, tprint("key \"%\" of something that is not an object", segment.key);
child = capture_object_find(current, segment.key);
if !child {
child = capture_object_add(doc, current, segment.key, .{});
is_new = true;
}
}
if is_new {
if kind == .ARRAY child.* = capture_array(doc);
else if kind == .OBJECT child.* = capture_object(doc);
} else if last && value_must_be_new {
return null, "already has a value";
} else if child.kind != kind {
return null, tprint("already holds % where % is needed", capture_kind_name(child.kind), capture_kind_name(kind));
}
current = child;
}
return current, "";
}
capture_kind_name :: (kind: Capture_Value_Kind) -> string {
if #complete kind == {
case .NULL; return "null";
case .BOOLEAN; return "a boolean";
case .NUMBER; return "a number";
case .STRING; return "a string";
case .ARRAY; return "an array";
case .OBJECT; return "an object";
}
return "";
}
capture_parse_value :: (doc: *Capture_Document, s: *string, depth: s64) -> Capture_Value, bool {
fail: Capture_Value;
s.* = trim_left(s.*);
if !s.count || depth > MAX_NESTING return fail, false;
c := s.*[0];
if c == #char "\"" {
first_part, rest, ok := capture_read_quoted(s.*);
if !ok return fail, false;
s.* = rest;
// Nearly every string is one part, copied straight into the document. Further parts are joined in a builder.
builder: String_Builder;
builder.allocator = temp;
parts := 1;
while true {
after := trim_left(s.*, " \t"); // spaces or tabs between parts
if after.count && after[0] == #char "\"" {
text, rest, ok := capture_read_quoted(after);
if !ok return fail, false;
if parts == 1 append(*builder, first_part);
append(*builder, text);
s.* = rest;
} else if after.count >= 4 && after[0] == #char "0" && after[1] == #char "x" {
high := capture_hex_digit(after[2]);
low := capture_hex_digit(after[3]);
if high < 0 || low < 0 return fail, false;
if after.count > 4 && !capture_ends_token(after[4]) return fail, false;
if parts == 1 append(*builder, first_part);
append(*builder, cast(u8) (high * 16 + low));
s.* = advance(after, 4);
} else {
break;
}
parts += 1;
}
if parts == 1 return capture_string(doc, first_part), true;
return capture_string(doc, builder_to_string(*builder,, temp)), true;
}
if c == #char "[" {
result := capture_array(doc);
s.* = trim_left(advance(s.*, 1));
if s.count && s.*[0] == #char "]" { s.* = advance(s.*, 1); return result, true; }
while true {
item, ok := capture_parse_value(doc, s, depth + 1);
if !ok return fail, false;
array_add(*result.items, item);
s.* = trim_left(s.*);
if !s.count return fail, false;
if s.*[0] == #char "]" { s.* = advance(s.*, 1); return result, true; }
if s.*[0] != #char "," return fail, false;
s.* = advance(s.*, 1);
}
}
if c == #char "{" {
rest := trim_left(advance(s.*, 1));
if !rest.count || rest[0] != #char "}" return fail, false; // only {} is written inline
s.* = advance(rest, 1);
return capture_object(doc), true;
}
if c == #char "-" || (c >= #char "0" && c <= #char "9") {
length := capture_number_length(s.*);
if length <= 0 || (length < s.count && !capture_ends_token(s.*[length])) return fail, false;
n, ok, remainder := string_to_float64(slice(s.*, 0, length));
if !ok || remainder.count return fail, false;
s.* = advance(s.*, length);
return capture_number(n), true;
}
for word: string.["null", "true", "false"] {
if !begins_with(s.*, word) continue;
if s.count > word.count && !capture_ends_token(s.*[word.count]) continue;
s.* = advance(s.*, word.count);
if word == "null" return capture_null(), true;
return capture_boolean(word == "true"), true;
}
return fail, false;
}
// The length of a JSON number at the start of s: -?digits(.digits)?([eE][+-]?digits)?. 0 when there is none.
capture_number_length :: (s: string) -> s64 {
i := 0;
if i < s.count && s[i] == #char "-" i += 1;
digits_start := i;
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
if i == digits_start return 0;
if i < s.count && s[i] == #char "." {
i += 1;
fraction_start := i;
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
if i == fraction_start return 0;
}
if i < s.count && (s[i] == #char "e" || s[i] == #char "E") {
i += 1;
if i < s.count && (s[i] == #char "+" || s[i] == #char "-") i += 1;
exponent_start := i;
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
if i == exponent_start return 0;
}
return i;
}
capture_ends_token :: (c: u8) -> bool {
return is_space(c) || c == #char "," || c == #char "]";
}
capture_hex_digit :: (c: u8) -> s64 {
if c >= #char "0" && c <= #char "9" return c - #char "0";
if c >= #char "A" && c <= #char "F" return c - #char "A" + 10;
if c >= #char "a" && c <= #char "f" return c - #char "a" + 10;
return -1;
}
// One quoted string ("" for a quote) at the start of s. Returns the text and what follows. The text is
// a slice of s when it holds no doubled quote, as nearly every string does; otherwise it is temp.
capture_read_quoted :: (s: string) -> text: string, rest: string, ok: bool {
if !s.count || s[0] != #char "\"" return "", s, false;
i := 1;
while i < s.count && s[i] != #char "\"" i += 1;
if i >= s.count return "", s, false;
if i + 1 >= s.count || s[i + 1] != #char "\"" return slice(s, 1, i - 1), advance(s, i + 1), true;
// A doubled quote: the text has to be put together.
builder: String_Builder;
builder.allocator = temp;
run_start := 1;
while i < s.count {
if s[i] == #char "\"" {
append(*builder, slice(s, run_start, i - run_start));
if i + 1 < s.count && s[i + 1] == #char "\"" {
append(*builder, #char "\"");
i += 2;
run_start = i;
continue;
}
return builder_to_string(*builder,, temp), advance(s, i + 1), true;
}
i += 1;
}
return "", s, false;
}
// One line, from offset, in a single pass: the line without its comment or line ending, and where in it
// the first : outside quotes is (-1 when there is none). "#" opens a comment only outside quotes,
// and only at the start of the line or after whitespace. valid is false when the line ends inside quotes.
capture_scan_line :: (data: string, offset: *s64) -> line: string, colon: s64, valid: bool {
start := offset.*;
i := start;
colon := -1;
end := -1; // where a comment starts
while i < data.count {
c := data[i];
if c == #char "\"" {
// Straight to the closing quote: strings are most of the bytes. A doubled quote closes and opens.
i += 1;
while i < data.count && data[i] != #char "\"" && data[i] != #char "\n" i += 1;
if i >= data.count || data[i] == #char "\n" {
while i < data.count && data[i] != #char "\n" i += 1;
offset.* = ifx i < data.count then i + 1 else data.count;
return "", -1, false;
}
} else if c == #char "\n" {
break;
} else if c == #char "#" && (i == start || is_space(data[i - 1])) {
end = i;
while i < data.count && data[i] != #char "\n" i += 1;
break;
} else if c == #char ":" && colon < 0 {
colon = i - start;
}
i += 1;
}
offset.* = ifx i < data.count then i + 1 else data.count;
if end < 0 {
end = i;
if end > start && data[end - 1] == #char "\r" end -= 1; // CRLF
}
return string.{end - start, data.data + start}, colon, true;
}
capture_split_directive :: (body: string) -> key: string, value: string {
rest := advance(body, 1); // skip the @
i := 0;
while i < rest.count && !is_space(rest[i]) i += 1;
return slice(rest, 0, i), trim(advance(rest, i));
}
capture_is_bare_char :: (c: u8) -> bool {
return (c >= #char "a" && c <= #char "z") || (c >= #char "A" && c <= #char "Z") || (c >= #char "0" && c <= #char "9") || c == #char "_" || c == #char "-";
}
capture_is_digits :: (s: string) -> bool {
if !s.count return false;
for 0..s.count - 1 if s[it] < #char "0" || s[it] > #char "9" return false;
return true;
}
capture_is_bare_key :: (key: string) -> bool {
if !key.count || capture_is_digits(key) return false;
for 0..key.count - 1 if !capture_is_bare_char(key[it]) return false;
return true;
}
// ---------------------------------------------------------------- writing, in detail
Capture_Line :: struct {
path: string;
value: Capture_Value;
}
// A root array of objects is written as [KEY] sections, one per object.
capture_is_section_array :: (value: Capture_Value) -> bool {
if value.kind != .ARRAY || !value.items.count return false;
for value.items if it.kind != .OBJECT return false;
return true;
}
// An array that holds an object with keys, at any depth, cannot be written inline.
capture_needs_paths :: (value: Capture_Value) -> bool {
if value.kind == .OBJECT return value.fields.count > 0;
if value.kind == .ARRAY {
for value.items if capture_needs_paths(it) return true;
}
return false;
}
capture_collect_lines :: (lines: *[..] Capture_Line, path: string, value: Capture_Value) {
if capture_needs_paths(value) {
if value.kind == .OBJECT {
for value.fields capture_collect_lines(lines, tprint("%.%", path, capture_key_text(it.key)), it.value);
} else {
for value.items capture_collect_lines(lines, tprint("%.%", path, it_index), it);
}
return;
}
array_add(lines, .{path, value});
}
// Key lines with their values lined up, each on one line however long.
capture_write_block :: (builder: *String_Builder, lines: [] Capture_Line) {
key_width := 0;
for lines if it.path.count + 1 <= CAPTURE_MAX_KEY_WIDTH key_width = max(key_width, it.path.count + 1);
for lines {
head := tprint("%:", it.path);
append(builder, head);
for head.count..max(head.count, key_width) append(builder, #char " ");
capture_append_value(builder, it.value);
append(builder, #char "\n");
}
}
capture_append_value :: (builder: *String_Builder, value: Capture_Value) {
if #complete value.kind == {
case .NULL; append(builder, "null");
case .BOOLEAN; append(builder, ifx value.boolean then "true" else "false");
case .NUMBER; append(builder, capture_number_text(value.number));
case .STRING; append(builder, capture_string_text(value.text));
case .OBJECT; append(builder, "{}"); // only an empty one reaches here
case .ARRAY;
append(builder, #char "[");
for value.items {
if it_index > 0 append(builder, ", ");
capture_append_value(builder, it);
}
append(builder, #char "]");
}
}
// A string on one line: quoted, with control characters and DEL, which quotes cannot hold, written
// as 0xNN parts between quoted ones. Temp.
capture_string_text :: (text: string) -> string {
if !text.count return "\"\"";
builder: String_Builder;
builder.allocator = temp;
i := 0;
while i < text.count {
if i > 0 append(*builder, #char " ");
c := text[i];
if c < 0x20 || c == 0x7F {
if i == 0 append(*builder, "\"\" "); // the first part is always quoted
print_to_builder(*builder, "0x%", formatInt(c, base = 16, minimum_digits = 2));
i += 1;
continue;
}
end := i;
while end < text.count && text[end] >= 0x20 && text[end] != 0x7F end += 1;
append(*builder, capture_quote(slice(text, i, end - i)));
i = end;
}
return builder_to_string(*builder,, temp);
}
#import "Basic";
#import "String";

View File

@ -1,7 +1,8 @@
# OCR # OCR
Text in images, with where each line is: a batch of RGBA images in, each image's text lines out, Text in images, with where each line is: a batch of RGBA images in, each image's text lines out,
each with a box and a confidence. It runs ONNX models through onnxruntime each with its corners, its box and a confidence. Lines at an angle, read downwards or upwards, or
upside down are read too. It runs ONNX models through onnxruntime
([`modules/onnxruntime`](../onnxruntime)): on an NVIDIA GPU through CUDA when every library that ([`modules/onnxruntime`](../onnxruntime)): on an NVIDIA GPU through CUDA when every library that
needs is present, and on the CPU otherwise. needs is present, and on the CPU otherwise.
@ -11,7 +12,7 @@ The binaries and the models are downloaded, not committed:
``` ```
cd modules/onnxruntime && jai fetch.jai # onnxruntime 1.30.0, GPU build (CUDA 13); `- cpu` for the CPU-only build cd modules/onnxruntime && jai fetch.jai # onnxruntime 1.30.0, GPU build (CUDA 13); `- cpu` for the CPU-only build
cd modules/OCR && jai fetch_models.jai # both models (about 195 MB); `- mobile` for the light one only cd modules/OCR && jai fetch_models.jai # both models (about 196 MB); `- mobile` for the light one only
jai generate.jai # in modules/onnxruntime, only when changing onnxruntime versions jai generate.jai # in modules/onnxruntime, only when changing onnxruntime versions
``` ```
@ -31,7 +32,16 @@ ocr_destroy(engine);
``` ```
`ocr_process` shares a batch out over worker threads: detection per image, then recognition of `ocr_process` shares a batch out over worker threads: detection per image, then recognition of
every line of every image in batches of similar width. The results come from the context every line of every image in batches of similar width.
**Turned text.** Detection gives each line the smallest rectangle around it at any angle (rotating
calipers over its convex hull), keeping a level one unless a turned one is clearly smaller; a level
box takes in every pixel it touches, as recognition reads best. Recognition samples each rectangle
along its own sides into a level crop. A box at least 1.5 times as tall as it is wide is taken as a
line read downwards, and also read level, since a narrow word such as "1" is as tall. The
orientation classifier marks lines that look upside down, and those are read both ways. Of each
line's readings, the more confident is kept. `Ocr_Line.corners` go clockwise from the text's top
left, so a turned line says which way it reads. The results come from the context
allocator, or the one passed. `ocr_stats` gives the time spent in each stage. allocator, or the one passed. `ocr_stats` gives the time spent in each stage.
## Models ## Models
@ -39,10 +49,13 @@ allocator, or the one passed. `ocr_stats` gives the time spent in each stage.
`models.config` lists them; `ocr_create` takes a name. Both are PaddleOCR's PP-OCRv5 (Apache 2.0), `models.config` lists them; `ocr_create` takes a name. Both are PaddleOCR's PP-OCRv5 (Apache 2.0),
as converted to ONNX by RapidOCR, and read Chinese, English and Japanese: as converted to ONNX by RapidOCR, and read Chinese, English and Japanese:
| Name | Files | On the 20 sample captures, CPU | | Name | Files | On the 20 sample captures: RTX 5080 laptop / CPU |
| --- | --- | --- | | --- | --- | --- |
| `ppocr-v5-mobile` | detector 4.8 MB, recognizer 16.6 MB | about 0.5 s per capture | | `ppocr-v5-mobile` | detector 4.8 MB, recognizer 16.6 MB | 0.13 s / 0.5 to 0.65 s per capture |
| `ppocr-v5-server` | detector 88 MB, recognizer 85 MB | about 2.5 s per capture | | `ppocr-v5-server` | detector 88 MB, recognizer 85 MB | 2.3 s / 2.5 s per capture (0.36 s on the GPU with `rec_batch: 4`) |
Both also use the text line orientation classifier `ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx`
(1 MB, `textline_ori:` in `models.config`).
Another model with the same two-stage design is a new `[model NAME]` section; a different design Another model with the same two-stage design is a new `[model NAME]` section; a different design
needs its own detection or recognition code beside `detect.jai` and `recognize.jai`. needs its own detection or recognition code beside `detect.jai` and `recognize.jai`.
@ -55,7 +68,9 @@ onnxruntime's CUDA build needs, on PATH or beside the executable:
- the CUDA Toolkit 13.x (`cudart64_13`, `cublas64_13`, `cublasLt64_13`, `cufft64_12`), from - the CUDA Toolkit 13.x (`cudart64_13`, `cublas64_13`, `cublasLt64_13`, `cufft64_12`), from
[developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads), whose installer puts its bin folder on PATH; [developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads), whose installer puts its bin folder on PATH;
- cuDNN 9.x for CUDA 13 (`cudnn64_9` and its `cudnn_*64_9` companions), from - cuDNN 9.x for CUDA 13 (`cudnn64_9` and its `cudnn_*64_9` companions), from
[developer.nvidia.com/cudnn-downloads](https://developer.nvidia.com/cudnn-downloads), whose bin folder for CUDA 13 you add to PATH yourself. [developer.nvidia.com/cudnn-downloads](https://developer.nvidia.com/cudnn-downloads). Its installer does not touch PATH;
when cuDNN is not on it, the newest `C:\Program Files\NVIDIA\CUDNN\v9.*\bin\13.*\x64` is added to the
program's own PATH, so an install in the usual place needs nothing more.
`ocr_create` looks for each of these first (`cuda_check.jai`) and says which are missing, rather `ocr_create` looks for each of these first (`cuda_check.jai`) and says which are missing, rather
than letting onnxruntime fail later and obscurely. If they are all there but CUDA still cannot than letting onnxruntime fail later and obscurely. If they are all there but CUDA still cannot

View File

@ -2,6 +2,10 @@
// The provider needs the NVIDIA driver, parts of the CUDA 13 toolkit and cuDNN 9 for CUDA 13; it // The provider needs the NVIDIA driver, parts of the CUDA 13 toolkit and cuDNN 9 for CUDA 13; it
// fails late and obscurely without them, so each library is looked for first, by the name the // fails late and obscurely without them, so each library is looked for first, by the name the
// provider loads it by (from onnxruntime_providers_cuda.dll's imports and the names it loads). // provider loads it by (from onnxruntime_providers_cuda.dll's imports and the names it loads).
//
// cuDNN's Windows installer does not put its DLLs on PATH. When they are not found, the newest
// cuDNN 9 for CUDA 13 under Program Files\NVIDIA\CUDNN is added to this process's PATH, so the
// provider finds it without anyone editing PATH.
Cuda_Library :: struct { Cuda_Library :: struct {
name: string; name: string;
@ -53,6 +57,8 @@ ocr_cuda_check :: () -> available: bool, explanation: string {
return false, tprint("OCR runs on the CPU: the onnxruntime beside the program is the CPU-only build (% is not there).\nTo use an NVIDIA GPU, run `jai fetch.jai` in modules/onnxruntime (without `cpu`) and rebuild.", CUDA_PROVIDER_LIBRARY); return false, tprint("OCR runs on the CPU: the onnxruntime beside the program is the CPU-only build (% is not there).\nTo use an NVIDIA GPU, run `jai fetch.jai` in modules/onnxruntime (without `cpu`) and rebuild.", CUDA_PROVIDER_LIBRARY);
} }
#if OS == .WINDOWS ocr_add_cudnn_to_path();
missing_driver, missing_toolkit, missing_cudnn: [..] string; missing_driver, missing_toolkit, missing_cudnn: [..] string;
missing_driver.allocator, missing_toolkit.allocator, missing_cudnn.allocator = temp, temp, temp; missing_driver.allocator, missing_toolkit.allocator, missing_cudnn.allocator = temp, temp, temp;
for CUDA_LIBRARIES { for CUDA_LIBRARIES {
@ -87,8 +93,8 @@ ocr_cuda_check :: () -> available: bool, explanation: string {
print_to_builder(*builder, "\n%. cuDNN 9.x for CUDA 13, which has % that are missing.\n", step, join(..missing_cudnn, ", ",, temp)); print_to_builder(*builder, "\n%. cuDNN 9.x for CUDA 13, which has % that are missing.\n", step, join(..missing_cudnn, ", ",, temp));
append(*builder, " https://developer.nvidia.com/cudnn-downloads\n"); append(*builder, " https://developer.nvidia.com/cudnn-downloads\n");
#if OS == .WINDOWS { #if OS == .WINDOWS {
append(*builder, " Its installer does not change PATH: add the folder with these DLLs to it yourself,\n"); append(*builder, " Its installer does not change PATH. Installed in its usual place, C:\\Program Files\\NVIDIA\\CUDNN,\n");
append(*builder, " such as C:\\Program Files\\NVIDIA\\CUDNN\\v9.x\\bin\\13.x\\x64.\n"); append(*builder, " it is found there; elsewhere, add the folder with these DLLs to PATH yourself.\n");
} else { } else {
append(*builder, " Then make sure its lib folder is on LD_LIBRARY_PATH.\n"); append(*builder, " Then make sure its lib folder is on LD_LIBRARY_PATH.\n");
} }
@ -115,3 +121,75 @@ ocr_library_loads :: (name: string) -> bool {
return true; return true;
} }
} }
#if OS == .WINDOWS {
// When cuDNN does not load, puts the newest Program Files\NVIDIA\CUDNN\v9.*\bin\13.*\x64 (or
// bin\13.*, as older releases lay it out) holding cudnn64_9.dll at the front of this process's PATH.
ocr_add_cudnn_to_path :: () {
if ocr_library_loads("cudnn64_9.dll") return;
program_files := ocr_environment_variable("ProgramW6432");
if !program_files program_files = ocr_environment_variable("ProgramFiles");
if !program_files return;
versions := ocr_subfolders_newest_first(tprint("%\\NVIDIA\\CUDNN", program_files), "v9.");
for version: versions {
for cuda: ocr_subfolders_newest_first(tprint("%\\bin", version), "13.") {
for folder: string.[tprint("%\\x64", cuda), cuda] {
if !file_exists(tprint("%\\cudnn64_9.dll", folder)) continue;
found := replace(folder, "/", "\\",, temp);
path := ocr_environment_variable("PATH");
OcrSetEnvironmentVariableW(utf8_to_wide("PATH",, temp), utf8_to_wide(tprint("%;%", found, path),, temp));
log("[OCR] cuDNN is not on PATH; using the one in %.", found);
return;
}
}
}
}
// The folders in parent whose names start with prefix, the highest version first ("v9.26" above
// "v9.3"). Temp.
ocr_subfolders_newest_first :: (parent: string, prefix: string) -> [..] string {
Found :: struct { names: [..] string; prefix: string; }
found: Found;
found.names.allocator = temp;
found.prefix = prefix;
visit_files(parent, recursive = false, *found, (info: *File_Visit_Info, found: *Found) {
if begins_with(info.short_name, found.prefix) array_add(*found.names, copy_string(info.full_name,, temp));
}, visit_files = false, visit_directories = true);
quick_sort(found.names, (a: string, b: string) -> s64 { return ocr_compare_versions(path_filename(b), path_filename(a)); });
return found.names;
}
// Compares the numbers in two names one by one: "v9.26" > "v9.3".
ocr_compare_versions :: (a: string, b: string) -> s64 {
next_number :: (s: *string) -> s64, found: bool {
while s.count && (s.*[0] < #char "0" || s.*[0] > #char "9") advance(s);
if !s.count return 0, false;
n: s64;
while s.count && s.*[0] >= #char "0" && s.*[0] <= #char "9" { n = n * 10 + s.*[0] - #char "0"; advance(s); }
return n, true;
}
x, y := a, b;
while true {
p, p_found := next_number(*x);
q, q_found := next_number(*y);
if !p_found || !q_found return ifx p_found then 1 else ifx q_found then -1 else 0;
if p != q return ifx p > q then 1 else -1;
}
return 0;
}
// Temp; "" when it is not set.
ocr_environment_variable :: (name: string) -> string {
wide_name := utf8_to_wide(name,, temp);
size := OcrGetEnvironmentVariableW(wide_name, null, 0);
if !size return "";
buffer := NewArray(size, u16,, temp);
written := OcrGetEnvironmentVariableW(wide_name, buffer.data, size);
if !written || written >= size return "";
return wide_to_utf8(buffer.data, cast(s32) written,, temp);
}
OcrGetEnvironmentVariableW :: (name: *u16, buffer: *u16, size: u32) -> u32 #foreign kernel32 "GetEnvironmentVariableW";
OcrSetEnvironmentVariableW :: (name: *u16, value: *u16) -> s32 #foreign kernel32 "SetEnvironmentVariableW";
}

View File

@ -1,19 +1,26 @@
// Detection: where the text lines are. The detector (a DB network) gives each pixel a probability // Detection: where the text lines are. The detector (a DB network) gives each pixel a probability
// of being text. Pixels above det_threshold are grouped into connected regions, each kept region // of being text. Pixels above det_threshold are grouped into connected regions, and each kept region
// becomes a box, and the box is grown by the DB unclip distance, since the network is trained to // becomes the smallest rectangle around it, at whatever angle, grown by the DB unclip distance, since
// mark a shrunken core of each line. // the network is trained to mark a shrunken core of each line.
// //
// Images wider than det_max_side are scaled down for detection. Images taller than det_tile_height // Images wider than det_max_side are scaled down for detection. Images taller than det_tile_height
// (after scaling) are detected in overlapping strips, so a long scrolling capture keeps its text at // (after scaling) are detected in overlapping strips, so a long scrolling capture keeps its text at
// full size; each box is kept by the strip whose own band holds its centre. // full size; each box is kept by the strip whose own band holds its centre.
//
// Version 1 finds horizontal text only, as screenshots have: boxes are axis-aligned.
Detected_Box :: struct { Detected_Box :: struct {
x0, y0, x1, y1: s32; // in the image, x1 and y1 exclusive // In the image's pixels, clockwise: the text's top left, top right, bottom right, bottom left, as
// far as its shape tells. A box at least TURN_RATIO times as tall as it is wide is taken as a line
// read downwards, so its first corner is its top right. Recognition may still turn it by 180°.
corners: [4] Vector2;
turned: bool; // taken as a line read downwards: started from its top right
score: float; score: float;
} }
TURN_RATIO :: 1.5;
// Screenshots are mostly level: a region keeps a level box unless a turned one is this much smaller.
LEVEL_PREFERENCE :: 1.2; // a long line tilted by 1° already has a level box ~30% bigger
DET_TILE_OVERLAP : s32 : 128; // px of each strip shared with the next, after scaling DET_TILE_OVERLAP : s32 : 128; // px of each strip shared with the next, after scaling
// The image's text boxes, from the default allocator. // The image's text boxes, from the default allocator.
@ -84,48 +91,180 @@ ocr_detect_tile :: (engine: *Ocr_Engine, image: Ocr_Image, scale: float, scaled_
} }
regions := ocr_find_regions(out.data, w, valid_w, valid_h, model.det_threshold); regions := ocr_find_regions(out.data, w, valid_w, valid_h, model.det_threshold);
defer array_free(regions,, context.default_allocator); defer {
for regions array_free(it.outline);
array_free(regions,, context.default_allocator);
}
for regions { for regions {
if it.pixels == 0 continue; if it.pixels == 0 continue;
score := cast(float) (it.probability_sum / it.pixels); score := cast(float) (it.probability_sum / it.pixels);
if score < model.box_threshold continue; if score < model.box_threshold continue;
bw := it.x1 - it.x0;
bh := it.y1 - it.y0; rect := ocr_min_area_rect(it.outline);
if min(bw, bh) < 3 continue; // specks rect_w := rect.u_max - rect.u_min;
rect_h := rect.v_max - rect.v_min;
if min(rect_w, rect_h) < 3 continue; // specks
// Grown as DB's unclip does: by area * ratio / perimeter on every side. // Grown as DB's unclip does: by area * ratio / perimeter on every side.
distance := (cast(float) bw * bh * model.unclip_ratio) / (2.0 * (bw + bh)); distance := (rect_w * rect_h * model.unclip_ratio) / (2.0 * (rect_w + rect_h));
gx0 := it.x0 - distance; rect.u_min -= distance; rect.u_max += distance;
gy0 := it.y0 - distance; rect.v_min -= distance; rect.v_max += distance;
gx1 := it.x1 + distance; corners := ocr_rect_corners(rect);
gy1 := it.y1 + distance;
centre_y := tile_top + (it.y0 + it.y1) / 2.0; centre_y := tile_top + (corners[0].y + corners[2].y) / 2;
if centre_y < cast(float) band_top || centre_y >= cast(float) band_bottom continue; // the neighbouring strip's if centre_y < cast(float) band_top || centre_y >= cast(float) band_bottom continue; // the neighbouring strip's
box: Detected_Box; box: Detected_Box;
box.x0 = clamp(cast(s32) (gx0 * inverse), 0, image.width);
box.x1 = clamp(cast(s32) (gx1 * inverse + 0.999), 0, image.width);
box.y0 = clamp(cast(s32) ((gy0 + tile_top) * inverse), 0, image.height);
box.y1 = clamp(cast(s32) ((gy1 + tile_top) * inverse + 0.999), 0, image.height);
box.score = score; box.score = score;
if box.x1 - box.x0 >= 2 && box.y1 - box.y0 >= 2 array_add(boxes, box); for corners {
box.corners[it_index].x = clamp(it.x * inverse, 0, cast(float) image.width);
box.corners[it_index].y = clamp((it.y + tile_top) * inverse, 0, cast(float) image.height);
}
if rect.u.x == 1 && rect.u.y == 0 {
// A level box takes in every pixel it touches: the recognizers read these crops best.
x0, y0, x1, y1 := ocr_corner_bounds(box.corners, image.width, image.height);
box.corners = .[.{xx x0, xx y0}, .{xx x1, xx y0}, .{xx x1, xx y1}, .{xx x0, xx y1}];
}
box.corners, box.turned = ocr_reading_order(box.corners);
x0, y0, x1, y1 := ocr_corner_bounds(box.corners, image.width, image.height);
if x1 - x0 >= 2 && y1 - y0 >= 2 array_add(boxes, box);
} }
return true, ""; return true, "";
} }
// The pixels the corners cover, clipped to the image: x0 and y0 inclusive, x1 and y1 exclusive.
ocr_corner_bounds :: (corners: [4] Vector2, image_width: s32, image_height: s32) -> x0: s32, y0: s32, x1: s32, y1: s32 {
low := corners[0];
high := corners[0];
for corners {
low.x = min(low.x, it.x); low.y = min(low.y, it.y);
high.x = max(high.x, it.x); high.y = max(high.y, it.y);
}
return clamp(cast(s32) floor(low.x), 0, image_width), clamp(cast(s32) floor(low.y), 0, image_height),
clamp(cast(s32) ceil(high.x), 0, image_width), clamp(cast(s32) ceil(high.y), 0, image_height);
}
// The same rectangle's corners started from another one, keeping them clockwise: by 1, turned a
// quarter (what was the right side becomes the top), by 2, upside down.
ocr_corners_turned :: (corners: [4] Vector2, quarters: s64) -> [4] Vector2 {
result: [4] Vector2;
for 0..3 result[it] = corners[(it + quarters) % 4];
return result;
}
// A rectangle as ranges along two perpendicular unit axes: u, and v = u turned a quarter clockwise
// on the screen (y down).
Rotated_Rect :: struct {
u: Vector2;
u_min, u_max, v_min, v_max: float;
}
ocr_rect_corners :: (rect: Rotated_Rect) -> [4] Vector2 {
u := rect.u;
v := Vector2.{-u.y, u.x};
return .[u * rect.u_min + v * rect.v_min, u * rect.u_max + v * rect.v_min,
u * rect.u_max + v * rect.v_max, u * rect.u_min + v * rect.v_max];
}
// The corners clockwise from the text's top left, as PaddleOCR orders them: of the two leftmost, the
// higher is the top left; of the two rightmost, the higher is the top right. A box at least
// TURN_RATIO times as tall as it is wide is then started from its top right, so a line read
// downwards comes out level.
ocr_reading_order :: (corners: [4] Vector2) -> [4] Vector2, turned: bool {
sorted := corners;
quick_sort(sorted, (a: Vector2, b: Vector2) -> s64 { return ifx a.x < b.x then -1 else ifx a.x > b.x then 1 else 0; });
top_left, bottom_left := sorted[0], sorted[1];
if bottom_left.y < top_left.y swap(*top_left, *bottom_left);
top_right, bottom_right := sorted[2], sorted[3];
if bottom_right.y < top_right.y swap(*top_right, *bottom_right);
result := Vector2.[top_left, top_right, bottom_right, bottom_left];
width := length(result[1] - result[0]);
height := length(result[3] - result[0]);
if height >= width * TURN_RATIO return ocr_corners_turned(result, 1), true;
return result, false;
}
// The smallest rectangle holding the points, at any angle, found by rotating calipers: one side of
// it lies along an edge of the points' convex hull. A level rectangle is kept unless a turned one is
// smaller by more than LEVEL_PREFERENCE.
ocr_min_area_rect :: (points: [] Vector2) -> Rotated_Rect {
level: Rotated_Rect;
level.u = .{1, 0};
level.u_min, level.v_min = FLOAT32_MAX, FLOAT32_MAX;
level.u_max, level.v_max = -FLOAT32_MAX, -FLOAT32_MAX;
for points {
level.u_min = min(level.u_min, it.x); level.u_max = max(level.u_max, it.x);
level.v_min = min(level.v_min, it.y); level.v_max = max(level.v_max, it.y);
}
if points.count < 3 return level;
level_area := (level.u_max - level.u_min) * (level.v_max - level.v_min);
hull := ocr_convex_hull(points);
best := level;
best_area := level_area;
for i: 0..hull.count - 1 {
edge := hull[(i + 1) % hull.count] - hull[i];
edge_length := length(edge);
if edge_length < 0.0001 continue;
rect: Rotated_Rect;
rect.u = edge / edge_length;
v := Vector2.{-rect.u.y, rect.u.x};
rect.u_min, rect.v_min = FLOAT32_MAX, FLOAT32_MAX;
rect.u_max, rect.v_max = -FLOAT32_MAX, -FLOAT32_MAX;
for hull {
a := dot(it, rect.u);
b := dot(it, v);
rect.u_min = min(rect.u_min, a); rect.u_max = max(rect.u_max, a);
rect.v_min = min(rect.v_min, b); rect.v_max = max(rect.v_max, b);
}
area := (rect.u_max - rect.u_min) * (rect.v_max - rect.v_min);
if area < best_area { best = rect; best_area = area; }
}
if level_area <= best_area * LEVEL_PREFERENCE return level;
return best;
}
// The convex hull of the points, counter-clockwise (Andrew's monotone chain). Temp.
ocr_convex_hull :: (points: [] Vector2) -> [] Vector2 {
sorted := array_copy(points,, temp);
quick_sort(sorted, (a: Vector2, b: Vector2) -> s64 {
if a.x != b.x return ifx a.x < b.x then -1 else 1;
return ifx a.y < b.y then -1 else ifx a.y > b.y then 1 else 0;
});
cross :: (o: Vector2, a: Vector2, b: Vector2) -> float { return (a.x - o.x) * (b.y - o.y) - (a.y - o.y) * (b.x - o.x); }
hull := NewArray(sorted.count * 2, Vector2,, temp);
k := 0;
for sorted { // lower hull
while k >= 2 && cross(hull[k - 2], hull[k - 1], it) <= 0 k -= 1;
hull[k] = it; k += 1;
}
lower := k + 1;
for < i: 0..sorted.count - 2 { // upper hull
p := sorted[i];
while k >= lower && cross(hull[k - 2], hull[k - 1], p) <= 0 k -= 1;
hull[k] = p; k += 1;
}
hull.count = max(k - 1, 1); // the last point is the first again
return hull;
}
align_up_32 :: (n: s32) -> s32 { align_up_32 :: (n: s32) -> s32 {
return (n + 31) & cast(s32) ~31; return (n + 31) & cast(s32) ~31;
} }
Region :: struct { Region :: struct {
x0, y0, x1, y1: s32; // x1 and y1 exclusive
pixels: s64; pixels: s64;
probability_sum: float64; probability_sum: float64;
// The corners of the first and last pixel in each of its rows: what its convex hull needs.
outline: [..] Vector2; // default allocator
row: s32 = -1; // the row being scanned, and its first and last pixel
row_first, row_last: s32;
} }
// Groups the pixels above threshold into 8-connected regions: one pass assigning provisional labels // Groups the pixels above threshold into 8-connected regions: one pass assigning provisional labels
// and recording which touch (union-find), then one summing each region's extent and probability. // and recording which touch (union-find), then one summing each region's probability and outline.
ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32, threshold: float) -> [..] Region { ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32, threshold: float) -> [..] Region {
regions: [..] Region; regions: [..] Region;
regions.allocator = context.default_allocator; regions.allocator = context.default_allocator;
@ -149,6 +288,14 @@ ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32,
else if rb < ra parent[ra] = rb; else if rb < ra parent[ra] = rb;
} }
flush_row :: (r: *Region) {
if r.row < 0 return;
top, bottom := cast(float) r.row, cast(float) r.row + 1;
array_add(*r.outline, .{cast(float) r.row_first, top}, .{cast(float) r.row_first, bottom},
.{cast(float) r.row_last + 1, top}, .{cast(float) r.row_last + 1, bottom});
r.row = -1;
}
for y: 0..height - 1 { for y: 0..height - 1 {
for x: 0..width - 1 { for x: 0..width - 1 {
if probability[cast(s64) y * stride + x] <= threshold continue; if probability[cast(s64) y * stride + x] <= threshold continue;
@ -184,16 +331,21 @@ ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32,
index := region_of[root] - 1; index := region_of[root] - 1;
if index < 0 { if index < 0 {
region := array_add(*regions); region := array_add(*regions);
region.* = .{x0 = x, y0 = y, x1 = x + 1, y1 = y + 1}; region.* = .{};
region.outline.allocator = context.default_allocator;
index = cast(s32) regions.count - 1; index = cast(s32) regions.count - 1;
region_of[root] = index + 1; region_of[root] = index + 1;
} }
r := *regions[index]; r := *regions[index];
r.x0 = min(r.x0, x); r.x1 = max(r.x1, x + 1); if r.row != y {
r.y0 = min(r.y0, y); r.y1 = max(r.y1, y + 1); flush_row(r);
r.row, r.row_first = y, x;
}
r.row_last = x; // rows are scanned left to right
r.pixels += 1; r.pixels += 1;
r.probability_sum += probability[cast(s64) y * stride + x]; r.probability_sum += probability[cast(s64) y * stride + x];
} }
} }
for * regions flush_row(it);
return regions; return regions;
} }

View File

@ -2,7 +2,7 @@
// are not in git; run this after cloning. Files already there with the right hash are skipped. // are not in git; run this after cloning. Files already there with the right hash are skipped.
// //
// jai fetch_models.jai every model // jai fetch_models.jai every model
// jai fetch_models.jai - mobile only the files ppocr-v5-mobile needs (about 21 MB) // jai fetch_models.jai - mobile only the files ppocr-v5-mobile needs (about 22 MB)
// //
// From RapidOCR's conversions of PaddleOCR's PP-OCRv5 (both Apache 2.0): // From RapidOCR's conversions of PaddleOCR's PP-OCRv5 (both Apache 2.0):
// https://www.modelscope.cn/models/RapidAI/RapidOCR // https://www.modelscope.cn/models/RapidAI/RapidOCR
@ -15,6 +15,7 @@ FILES :: Model_File.[
.{"ch_PP-OCRv5_det_mobile.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_mobile.onnx", "4d97c44a20d30a81aad087d6a396b08f786c4635742afc391f6621f5c6ae78ae", false}, .{"ch_PP-OCRv5_det_mobile.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_mobile.onnx", "4d97c44a20d30a81aad087d6a396b08f786c4635742afc391f6621f5c6ae78ae", false},
.{"ch_PP-OCRv5_rec_mobile.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile.onnx", "5825fc7ebf84ae7a412be049820b4d86d77620f204a041697b0494669b1742c5", false}, .{"ch_PP-OCRv5_rec_mobile.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile.onnx", "5825fc7ebf84ae7a412be049820b4d86d77620f204a041697b0494669b1742c5", false},
.{"ppocrv5_dict.txt", "paddle/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile/ppocrv5_dict.txt", "d1979e9f794c464c0d2e0b70a7fe14dd978e9dc644c0e71f14158cdf8342af1b", false}, .{"ppocrv5_dict.txt", "paddle/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile/ppocrv5_dict.txt", "d1979e9f794c464c0d2e0b70a7fe14dd978e9dc644c0e71f14158cdf8342af1b", false},
.{"ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx", "onnx/PP-OCRv5/cls/ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx", "54379ae5174d026780215fc748a7f31910dee36818e63d49e17dc598ecc82df7", false},
.{"ch_PP-OCRv5_det_server.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_server.onnx", "0f8846b1d4bba223a2a2f9d9b44022fbc22cc019051a602b41a7fda9667e4cad", true}, .{"ch_PP-OCRv5_det_server.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_server.onnx", "0f8846b1d4bba223a2a2f9d9b44022fbc22cc019051a602b41a7fda9667e4cad", true},
.{"ch_PP-OCRv5_rec_server.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_server.onnx", "e09385400eaaaef34ceff54aeb7c4f0f1fe014c27fa8b9905d4709b65746562a", true}, .{"ch_PP-OCRv5_rec_server.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_server.onnx", "e09385400eaaaef34ceff54aeb7c4f0f1fe014c27fa8b9905d4709b65746562a", true},
]; ];

View File

@ -5,6 +5,8 @@
# #
# title shown to people # title shown to people
# detector, recognizer, dictionary file names, quoted # detector, recognizer, dictionary file names, quoted
# textline_ori optional: a text line orientation classifier's file name, quoted; lines it
# finds upside down are read both ways and the more confident reading kept
# det_max_side wider images are scaled down to this width for detection (px) # det_max_side wider images are scaled down to this width for detection (px)
# det_tile_height taller images are detected in overlapping strips this high (px, after scaling) # det_tile_height taller images are detected in overlapping strips this high (px, after scaling)
# det_threshold a pixel is text when its probability is above this # det_threshold a pixel is text when its probability is above this
@ -18,6 +20,7 @@ title: "PP-OCRv5 mobile (light)"
detector: "ch_PP-OCRv5_det_mobile.onnx" detector: "ch_PP-OCRv5_det_mobile.onnx"
recognizer: "ch_PP-OCRv5_rec_mobile.onnx" recognizer: "ch_PP-OCRv5_rec_mobile.onnx"
dictionary: "ppocrv5_dict.txt" dictionary: "ppocrv5_dict.txt"
textline_ori: "ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx"
det_max_side: 2560 det_max_side: 2560
det_tile_height: 1536 det_tile_height: 1536
det_threshold: 0.3 det_threshold: 0.3
@ -31,6 +34,7 @@ title: "PP-OCRv5 server (heavy)"
detector: "ch_PP-OCRv5_det_server.onnx" detector: "ch_PP-OCRv5_det_server.onnx"
recognizer: "ch_PP-OCRv5_rec_server.onnx" recognizer: "ch_PP-OCRv5_rec_server.onnx"
dictionary: "ppocrv5_dict.txt" dictionary: "ppocrv5_dict.txt"
textline_ori: "ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx"
det_max_side: 2560 det_max_side: 2560
det_tile_height: 1536 det_tile_height: 1536
det_threshold: 0.3 det_threshold: 0.3

View File

@ -5,6 +5,7 @@ Ocr_Model_Info :: struct {
name: string; name: string;
title: string; title: string;
detector, recognizer, dictionary: string; // file names in the models folder detector, recognizer, dictionary: string; // file names in the models folder
textline_ori: string; // the orientation classifier's file name; "" for none
det_max_side: s32 = 2560; det_max_side: s32 = 2560;
det_tile_height: s32 = 1536; det_tile_height: s32 = 1536;
det_threshold: float = 0.3; det_threshold: float = 0.3;
@ -53,6 +54,7 @@ ocr_list_models :: (models_folder: string) -> [] Ocr_Model_Info, error: string {
case "detector"; model.detector = copy_string(value); case "detector"; model.detector = copy_string(value);
case "recognizer"; model.recognizer = copy_string(value); case "recognizer"; model.recognizer = copy_string(value);
case "dictionary"; model.dictionary = copy_string(value); case "dictionary"; model.dictionary = copy_string(value);
case "textline_ori"; model.textline_ori = copy_string(value);
case "det_max_side"; if number_ok model.det_max_side = cast(s32) number; case "det_max_side"; if number_ok model.det_max_side = cast(s32) number;
case "det_tile_height"; if number_ok model.det_tile_height = cast(s32) number; case "det_tile_height"; if number_ok model.det_tile_height = cast(s32) number;
case "det_threshold"; if number_ok model.det_threshold = number; case "det_threshold"; if number_ok model.det_threshold = number;
@ -82,6 +84,35 @@ ocr_strip_comment :: (line: string) -> string {
Ocr_Session :: struct { Ocr_Session :: struct {
session: *OrtSession; session: *OrtSession;
input_name, output_name: *u8; // heap C strings input_name, output_name: *u8; // heap C strings
input_shape: [4] s64; // as the model declares it: -1 for a size it leaves open
input_rank: s64;
}
// The model's sessions on the GPU or the CPU: detector, recognizer and, when it has one, orientation
// classifier. On failure, the reason; the sessions made so far are left for ocr_release_sessions.
ocr_create_sessions :: (engine: *Ocr_Engine, models_folder: string, gpu: bool, intra_op_threads: s32) -> ok: bool, error: string {
model := *engine.model;
ok: bool;
error: string;
engine.det, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.detector), gpu, intra_op_threads);
if !ok return false, error;
engine.rec, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.recognizer), gpu, intra_op_threads);
if !ok return false, error;
if model.textline_ori {
engine.orientation, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.textline_ori), gpu, intra_op_threads);
if !ok return false, error;
// PP-LCNet text line orientation takes 160 by 80; a model that says otherwise is taken at its word.
shape := engine.orientation.input_shape;
engine.orientation_height = ifx engine.orientation.input_rank == 4 && shape[2] > 0 then cast(s32) shape[2] else 80;
engine.orientation_width = ifx engine.orientation.input_rank == 4 && shape[3] > 0 then cast(s32) shape[3] else 160;
}
return true, "";
}
ocr_release_sessions :: (engine: *Ocr_Engine) {
ocr_release_session(*engine.det);
ocr_release_session(*engine.rec);
ocr_release_session(*engine.orientation);
} }
// A session for the model file; with CUDA when gpu is set. On failure, the reason. // A session for the model file; with CUDA when gpu is set. On failure, the reason.
@ -124,6 +155,18 @@ ocr_create_session :: (env: *OrtEnv, path: string, gpu: bool, intra_op_threads:
ort_check(api.SessionGetOutputName(result.session, 0, allocator, *name)); ort_check(api.SessionGetOutputName(result.session, 0, allocator, *name));
result.output_name = to_c_string(to_string(name),, context.default_allocator); result.output_name = to_c_string(to_string(name),, context.default_allocator);
ort_check(api.AllocatorFree(allocator, name)); ort_check(api.AllocatorFree(allocator, name));
input_type: *OrtTypeInfo;
if ort_check(api.SessionGetInputTypeInfo(result.session, 0, *input_type)) {
tensor_info: *OrtTensorTypeAndShapeInfo; // belongs to input_type
if ort_check(api.CastTypeInfoToTensorInfo(input_type, *tensor_info)) && tensor_info {
rank: u64;
ort_check(api.GetDimensionsCount(tensor_info, *rank));
result.input_rank = min(cast(s64) rank, result.input_shape.count);
ort_check(api.GetDimensions(tensor_info, result.input_shape.data, xx result.input_rank));
}
api.ReleaseTypeInfo(input_type);
}
return result, true, ""; return result, true, "";
} }

View File

@ -8,8 +8,9 @@
// with onnxruntime (modules/onnxruntime), on an NVIDIA GPU through CUDA when it can, else on the // with onnxruntime (modules/onnxruntime), on an NVIDIA GPU through CUDA when it can, else on the
// CPU. ocr_gpu_status says why the GPU is not in use, and what to install so it can be. // CPU. ocr_gpu_status says why the GPU is not in use, and what to install so it can be.
// //
// Version 1 runs PP-OCRv5's two-stage pipeline: detection (detect.jai), then recognition // It runs PP-OCRv5's pipeline: detection (detect.jai) finds each line as a rectangle at any angle,
// (recognize.jai). Text is expected horizontal, as in screenshots. // then recognition (recognize.jai) reads it level. When the model lists an orientation classifier,
// lines that look upside down are read both ways; lines read downwards or upwards are read too.
Ocr_Provider :: enum { CPU; CUDA; } Ocr_Provider :: enum { CPU; CUDA; }
@ -20,9 +21,10 @@ Ocr_Image :: struct {
} }
Ocr_Line :: struct { Ocr_Line :: struct {
x, y, width, height: s32; // in the image x, y, width, height: s32; // in the image: the bounds of corners
text: string; text: string;
confidence: float; // 0 to 1: the mean probability of its characters confidence: float; // 0 to 1: the mean probability of its characters
corners: [4] Vector2; // in the image's pixels: the text's top left, top right, bottom right, bottom left
} }
Ocr_Result :: struct { Ocr_Result :: struct {
@ -38,6 +40,8 @@ Ocr_Engine :: struct {
env: *OrtEnv; env: *OrtEnv;
memory_info: *OrtMemoryInfo; memory_info: *OrtMemoryInfo;
det, rec: Ocr_Session; det, rec: Ocr_Session;
orientation: Ocr_Session; // the text line orientation classifier; none when its session is null
orientation_width, orientation_height: s32; // its input size
dictionary: [] string; // class n is dictionary[n - 1] dictionary: [] string; // class n is dictionary[n - 1]
workers: s32; // threads ocr_process uses workers: s32; // threads ocr_process uses
stats: Ocr_Stats; // since creation, or the last ocr_reset_stats stats: Ocr_Stats; // since creation, or the last ocr_reset_stats
@ -49,6 +53,8 @@ Ocr_Stats :: struct {
lines: s64; lines: s64;
detect_us: s64; // preparing images, the detector, and finding the boxes detect_us: s64; // preparing images, the detector, and finding the boxes
recognize_us: s64; // cropping lines, the recognizer, and decoding the text recognize_us: s64; // cropping lines, the recognizer, and decoding the text
orientation_us: s64; // cropping lines and the orientation classifier
turned: s64; // lines that read better the other way up, or level as found
} }
ocr_stats :: (engine: *Ocr_Engine) -> Ocr_Stats { return engine.stats; } ocr_stats :: (engine: *Ocr_Engine) -> Ocr_Stats { return engine.stats; }
@ -84,8 +90,6 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
ort_check(api.CreateCpuMemoryInfo(.OrtArenaAllocator, .OrtMemTypeDefault, *engine.memory_info)); ort_check(api.CreateCpuMemoryInfo(.OrtArenaAllocator, .OrtMemTypeDefault, *engine.memory_info));
cores := cast(s32) max(get_number_of_processors(.ALL_LOGICAL), 1); // efficiency cores included cores := cast(s32) max(get_number_of_processors(.ALL_LOGICAL), 1); // efficiency cores included
detector_path := tprint("%/%", models_folder, model.detector);
recognizer_path := tprint("%/%", models_folder, model.recognizer);
use_gpu := false; use_gpu := false;
if prefer_gpu { if prefer_gpu {
@ -99,16 +103,12 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
if use_gpu { if use_gpu {
// A few CPU threads prepare images and decode results while the GPU runs the networks. // A few CPU threads prepare images and decode results while the GPU runs the networks.
engine.workers = 4; engine.workers = 4;
det_ok, rec_ok: bool; sessions_ok, sessions_error := ocr_create_sessions(engine, models_folder, true, 2);
det_error, rec_error: string; if sessions_ok {
engine.det, det_ok, det_error = ocr_create_session(engine.env, detector_path, true, 2);
if det_ok engine.rec, rec_ok, rec_error = ocr_create_session(engine.env, recognizer_path, true, 2);
if det_ok && rec_ok {
engine.provider = .CUDA; engine.provider = .CUDA;
} else { } else {
ocr_release_session(*engine.det); ocr_release_sessions(engine);
ocr_release_session(*engine.rec); engine.gpu_explanation = sprint("OCR runs on the CPU: the CUDA libraries were found, but onnxruntime could not start on the GPU:\n%", sessions_error);
engine.gpu_explanation = sprint("OCR runs on the CPU: the CUDA libraries were found, but onnxruntime could not start on the GPU:\n%", ifx det_ok then rec_error else det_error);
use_gpu = false; use_gpu = false;
} }
} }
@ -116,12 +116,8 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
// Several images at once, each network run spread over the rest of the cores. // Several images at once, each network run spread over the rest of the cores.
engine.workers = clamp(cores / 4, 1, 8); engine.workers = clamp(cores / 4, 1, 8);
threads := max(cores / engine.workers, 1); threads := max(cores / engine.workers, 1);
det_ok, rec_ok: bool; sessions_ok, sessions_error := ocr_create_sessions(engine, models_folder, false, threads);
det_error, rec_error: string; if !sessions_ok { ocr_destroy(engine); return null, false, sessions_error; }
engine.det, det_ok, det_error = ocr_create_session(engine.env, detector_path, false, threads);
if !det_ok { ocr_destroy(engine); return null, false, det_error; }
engine.rec, rec_ok, rec_error = ocr_create_session(engine.env, recognizer_path, false, threads);
if !rec_ok { ocr_destroy(engine); return null, false, rec_error; }
engine.provider = .CPU; engine.provider = .CPU;
} }
return engine, true, ""; return engine, true, "";
@ -138,8 +134,7 @@ ocr_gpu_status :: (engine: *Ocr_Engine) -> in_use: bool, explanation: string {
ocr_destroy :: (engine: *Ocr_Engine) { ocr_destroy :: (engine: *Ocr_Engine) {
if !engine return; if !engine return;
ocr_release_session(*engine.det); ocr_release_sessions(engine);
ocr_release_session(*engine.rec);
if engine.memory_info ort.api.ReleaseMemoryInfo(engine.memory_info); if engine.memory_info ort.api.ReleaseMemoryInfo(engine.memory_info);
if engine.env ort.api.ReleaseEnv(engine.env); if engine.env ort.api.ReleaseEnv(engine.env);
free(engine.gpu_explanation,, context.default_allocator); free(engine.gpu_explanation,, context.default_allocator);
@ -174,10 +169,49 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
for jobs free(it.text,, context.default_allocator); for jobs free(it.text,, context.default_allocator);
array_free(jobs); array_free(jobs);
} }
rec_height := engine.model.rec_height;
turned: [..] bool; // per first job: whether detection turned its box a quarter
turned.allocator = context.default_allocator;
defer array_free(turned);
for work: per_image { for work: per_image {
if !work.ok continue; if !work.ok continue;
for work.boxes array_add(*jobs, .{image = work.image, box = it, width = ocr_line_width(it, engine.model.rec_height)}); for work.boxes {
array_add(*jobs, .{image = work.image, corners = it.corners, width = ocr_line_width(it.corners, rec_height)});
array_add(*turned, it.turned);
}
} }
// Other ways to read a line, recognized beside it, the more confident reading kept: upside down,
// where the orientation classifier finds it more likely than not; and level as it was found, for a
// box detection turned a quarter, since a narrow word such as "1" is as tall as a line read downwards.
first_count := jobs.count;
upside_down := NewArray(first_count, bool,, context.default_allocator);
defer array_free(upside_down,, context.default_allocator);
Orientation_Context :: struct { jobs: [] Line_Job; upside_down: [] bool; size: s64; failed: bool; error: string; }
classify := Orientation_Context.{jobs = jobs, upside_down = upside_down, size = max(engine.model.rec_batch, 1)};
if engine.orientation.session && first_count {
run_on_workers(engine, (first_count + classify.size - 1) / classify.size, (engine: *Ocr_Engine, index: s64, data: *void) {
c := cast(*Orientation_Context) data;
first := index * c.size;
count := min(c.size, c.jobs.count - first);
batch := NewArray(count, *Line_Job,, temp);
for * batch it.* = *c.jobs[first + it_index];
start := current_time_monotonic();
ok, error := ocr_classify_batch(engine, batch, array_view(c.upside_down, first, count));
atomic_add(*engine.stats.orientation_us, to_microseconds(current_time_monotonic() - start));
if !ok { c.failed = true; c.error = copy_string(error,, context.default_allocator); } // the last error wins
}, *classify);
}
defer free(classify.error,, context.default_allocator);
for n: 0..first_count - 1 {
job := jobs[n];
if upside_down[n] array_add(*jobs, .{image = job.image, corners = ocr_corners_turned(job.corners, 2), width = job.width, primary = n});
if turned[n] {
level := ocr_corners_turned(job.corners, 3);
array_add(*jobs, .{image = job.image, corners = level, width = ocr_line_width(level, rec_height), primary = n});
}
}
order := NewArray(jobs.count, *Line_Job,, context.default_allocator); order := NewArray(jobs.count, *Line_Job,, context.default_allocator);
defer array_free(order,, context.default_allocator); defer array_free(order,, context.default_allocator);
for * jobs order[it_index] = it; for * jobs order[it_index] = it;
@ -196,6 +230,20 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
atomic_add(*engine.stats.recognize_us, to_microseconds(current_time_monotonic() - start)); atomic_add(*engine.stats.recognize_us, to_microseconds(current_time_monotonic() - start));
if !ok { b.failed = true; b.error = copy_string(error,, context.default_allocator); } // the last error wins; any is enough to say why if !ok { b.failed = true; b.error = copy_string(error,, context.default_allocator); } // the last error wins; any is enough to say why
}, *batches); }, *batches);
if classify.failed && !batches.failed {
batches.failed = true;
batches.error = copy_string(classify.error,, context.default_allocator);
}
// The more confident of each line's readings takes the first job's place.
for * other: array_view(jobs, first_count, jobs.count - first_count) {
first := *jobs[other.primary];
if other.confidence > first.confidence {
swap(*first.text, *other.text);
first.corners, first.confidence = other.corners, other.confidence;
engine.stats.turned += 1;
}
}
// Each image's lines, in reading order. // Each image's lines, in reading order.
for work, image_index: per_image { for work, image_index: per_image {
@ -213,10 +261,11 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
} }
lines: [..] Ocr_Line; lines: [..] Ocr_Line;
lines.allocator = allocator; lines.allocator = allocator;
for job: jobs { for job: array_view(jobs, 0, first_count) {
if job.image != work.image || !job.text.count continue; if job.image != work.image || !job.text.count continue;
array_add(*lines, .{x = job.box.x0, y = job.box.y0, width = job.box.x1 - job.box.x0, height = job.box.y1 - job.box.y0, x0, y0, x1, y1 := ocr_corner_bounds(job.corners, job.image.width, job.image.height);
text = copy_string(job.text,, allocator), confidence = job.confidence}); array_add(*lines, .{x = x0, y = y0, width = x1 - x0, height = y1 - y0, text = copy_string(job.text,, allocator),
confidence = job.confidence, corners = job.corners});
} }
// Rows by the middle of each line, then left to right within a row. // Rows by the middle of each line, then left to right within a row.
quick_sort(lines, (a: Ocr_Line, b: Ocr_Line) -> s64 { quick_sort(lines, (a: Ocr_Line, b: Ocr_Line) -> s64 {

View File

@ -1,13 +1,18 @@
// Recognition: the text of each detected line. Each box is cropped from the image at full size, // Recognition: the text of each detected line. Each box is sampled along its own sides into a level
// scaled to rec_height keeping its shape, and normalized to -1..1 in BGR order, as PaddleOCR's // crop rec_height high, keeping its shape, and normalized to -1..1 in BGR order, as PaddleOCR's
// recognizers expect. Lines of similar width are batched, padded to the widest in the batch. // recognizers expect. Lines of similar width are batched, padded to the widest in the batch.
// The output is a probability per character class per step; greedy CTC decoding takes the best // The output is a probability per character class per step; greedy CTC decoding takes the best
// class at each step, merges repeats and drops blanks. // class at each step, merges repeats and drops blanks.
//
// A box's corners only say which way its text runs up to a half turn. The orientation classifier,
// when the model has one, says which lines look upside down; such a line is read both ways, as is a
// box taken as a line read downwards, and the more confident reading is kept.
Line_Job :: struct { Line_Job :: struct {
image: *Ocr_Image; image: *Ocr_Image;
box: Detected_Box; corners: [4] Vector2; // as Detected_Box's
width: s32; // after scaling to rec_height width: s32; // after scaling to rec_height
primary := -1; // for another way of reading a line: the index of the line's first job
// Filled in by recognition: // Filled in by recognition:
text: string; // default allocator text: string; // default allocator
confidence: float; confidence: float;
@ -24,38 +29,8 @@ ocr_recognize_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job) -> ok: bool, er
plane := cast(s64) height * width; plane := cast(s64) height * width;
input := NewArray(plane * 3 * jobs.count, float, initialized = true,, context.default_allocator); // 0 is the padding input := NewArray(plane * 3 * jobs.count, float, initialized = true,, context.default_allocator); // 0 is the padding
defer array_free(input,, context.default_allocator); defer array_free(input,, context.default_allocator);
for job, n: jobs { for job, n: jobs {
base := input.data + plane * 3 * n; ocr_sample_crop(job.image, job.corners, job.width, height, input.data + plane * 3 * n, width, plane, REC_MEAN, REC_STD);
box := job.box;
bw := box.x1 - box.x0;
bh := box.y1 - box.y0;
image := job.image;
// Bilinear: the crops are mostly enlarged (small UI text up to 48 px), where nearest would be blocky.
x_scale := cast(float) bw / job.width;
y_scale := cast(float) bh / height;
for y: 0..height - 1 {
sy := (y + 0.5) * y_scale - 0.5 + box.y0;
y0 := clamp(cast(s32) floor(sy), 0, image.height - 1);
y1 := min(y0 + 1, image.height - 1);
fy := clamp(sy - y0, 0, 1);
row0 := image.pixels + cast(s64) y0 * image.stride;
row1 := image.pixels + cast(s64) y1 * image.stride;
for x: 0..job.width - 1 {
sx := (x + 0.5) * x_scale - 0.5 + box.x0;
x0 := clamp(cast(s32) floor(sx), 0, image.width - 1);
x1 := min(x0 + 1, image.width - 1);
fx := clamp(sx - x0, 0, 1);
i := cast(s64) y * width + x;
for c: 0..2 {
channel := 2 - c; // B, G, R from RGBA
top := row0[x0 * 4 + channel] * (1 - fx) + row0[x1 * 4 + channel] * fx;
bottom := row1[x0 * 4 + channel] * (1 - fx) + row1[x1 * 4 + channel] * fx;
value := top * (1 - fy) + bottom * fy;
base[i + plane * c] = (value / 255.0 - 0.5) / 0.5;
}
}
}
} }
shape := s64.[jobs.count, 3, height, width]; shape := s64.[jobs.count, 3, height, width];
@ -95,11 +70,68 @@ ocr_recognize_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job) -> ok: bool, er
return true, ""; return true, "";
} }
// The width a box is recognized at: rec_height high, keeping its shape. // The width corners are recognized at: rec_height high, keeping their shape.
ocr_line_width :: (box: Detected_Box, rec_height: s32) -> s32 { ocr_line_width :: (corners: [4] Vector2, rec_height: s32) -> s32 {
bw := box.x1 - box.x0; w := length(corners[1] - corners[0]);
bh := max(box.y1 - box.y0, 1); h := max(length(corners[3] - corners[0]), 1);
return clamp(cast(s32) (cast(float) bw * rec_height / bh + 0.5), 1, REC_MAX_WIDTH); return clamp(cast(s32) (w * rec_height / h + 0.5), 1, REC_MAX_WIDTH);
} }
REC_MAX_WIDTH :: 4096; // a line longer than this, at rec_height, is squeezed to fit REC_MAX_WIDTH :: 4096; // a line longer than this, at rec_height, is squeezed to fit
// The normalization the PP-OCR recognizers were trained with.
REC_MEAN :: float.[0.5, 0.5, 0.5];
REC_STD :: float.[0.5, 0.5, 0.5];
// Samples the rectangle the corners make (clockwise from its top left, at any angle) into a level
// crop width by height, bilinearly: crops are mostly enlarged (small UI text up to 48 px), where
// nearest would be blocky. Written planar in BGR order at dest, row_stride floats from one row to the
// next and plane from one channel to the next, as (value / 255 - mean) / deviation.
ocr_sample_crop :: (image: *Ocr_Image, corners: [4] Vector2, width: s32, height: s32, dest: *float, row_stride: s64, plane: s64, mean: [3] float, deviation: [3] float) {
step_x := (corners[1] - corners[0]) / cast(float) width; // one crop pixel along the line
step_y := (corners[3] - corners[0]) / cast(float) height; // and across it
// Pixel centres: an image pixel's centre is at +0.5.
origin := corners[0] + step_x * 0.5 + step_y * 0.5 - Vector2.{0.5, 0.5};
for y: 0..height - 1 {
row_start := origin + step_y * cast(float) y;
for x: 0..width - 1 {
s := row_start + step_x * cast(float) x;
x0 := clamp(cast(s32) floor(s.x), 0, image.width - 1);
y0 := clamp(cast(s32) floor(s.y), 0, image.height - 1);
x1 := min(x0 + 1, image.width - 1);
y1 := min(y0 + 1, image.height - 1);
fx := clamp(s.x - x0, 0, 1);
fy := clamp(s.y - y0, 0, 1);
row0 := image.pixels + cast(s64) y0 * image.stride;
row1 := image.pixels + cast(s64) y1 * image.stride;
i := cast(s64) y * row_stride + x;
for c: 0..2 {
channel := 2 - c; // B, G, R from RGBA
top := row0[x0 * 4 + channel] * (1 - fx) + row0[x1 * 4 + channel] * fx;
bottom := row1[x0 * 4 + channel] * (1 - fx) + row1[x1 * 4 + channel] * fx;
value := top * (1 - fy) + bottom * fy;
dest[i + plane * c] = (value / 255.0 - mean[c]) / deviation[c];
}
}
}
}
// The orientation classifier (PP-LCNet text line orientation) on one batch: sets upside_down[n] when
// it finds jobs[n] more likely upside down than not. Its input is a fixed size, the line stretched to
// fit, normalized as the detector's input is.
ocr_classify_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job, upside_down: [] bool) -> ok: bool, error: string {
if !jobs.count return true, "";
width, height := engine.orientation_width, engine.orientation_height;
plane := cast(s64) width * height;
input := NewArray(plane * 3 * jobs.count, float,, context.default_allocator);
defer array_free(input,, context.default_allocator);
for job, n: jobs ocr_sample_crop(job.image, job.corners, width, height, input.data + plane * 3 * n, width, plane, DET_MEAN, DET_STD);
shape := s64.[jobs.count, 3, height, width];
out, ok, error := ocr_run(*engine.orientation, engine.memory_info, input, shape);
if !ok return false, error;
defer ocr_release_output(*out);
if out.dim_count != 2 || out.dims[0] != jobs.count || out.dims[1] != 2 return false, tprint("The orientation classifier gave an output of shape %, not [%, 2].", array_view(out.dims, 0, out.dim_count), jobs.count);
for 0..jobs.count - 1 upside_down[it] = out.data[it * 2 + 1] > out.data[it * 2]; // classes: 0°, 180°
return true, "";
}

View File

@ -9,12 +9,15 @@ USING_WINDOWS :: OS == .WINDOWS;
#if COMPILE_TESTS { #if COMPILE_TESTS {
#load "test/testing.jai"; #load "test/testing.jai";
#load "test/logger_tests.jai"; #load "test/logger_tests.jai";
#load "test/catalog_tests.jai";
#load "test/ocr_tests.jai";
} }
#load "app/main.jai"; // entry point #load "app/main.jai"; // entry point
#load "app/app_ui.jai"; #load "app/app_ui.jai";
#load "app/viewer.jai"; #load "app/viewer.jai";
#load "app/ocr_ui.jai"; #load "app/ocr_ui.jai";
#load "app/catalog.jai";
#load "app/exit_shutdown.jai"; #load "app/exit_shutdown.jai";
#load "base/circle_buffer.jai"; #load "base/circle_buffer.jai";
@ -50,6 +53,8 @@ Shared_Channel :: #import "Shared_Memory_Channel";
#import "stb_image_resize"; #import "stb_image_resize";
#import "Capture_File"; // the .mcap format, copied from D:\SnagitIndex\SnagitConverter #import "Capture_File"; // the .mcap format, copied from D:\SnagitIndex\SnagitConverter
#import "OCR"; #import "OCR";
#import "Capture_Text";
#import "Base64";
#import "onnxruntime"; #import "onnxruntime";
// Basic's arenas import Secure_Memory at its defaults; this import adds its test suite. // Basic's arenas import Secure_Memory at its defaults; this import adds its test suite.
@ -167,6 +172,8 @@ App :: struct {
run_tests: bool; run_tests: bool;
headless: bool; // -headless (or a no_crash_dialog build): no message boxes on failures, and run_tests exits when done headless: bool; // -headless (or a no_crash_dialog build): no message boxes on failures, and run_tests exits when done
exit_code: s32; // the process's exit code when the main loop ends exit_code: s32; // the process's exit code when the main loop ends
library_folder: string; // -library FOLDER: the .mcap folder the viewer shows, in place of VIEWER_FOLDER
scan_and_exit: bool; // -scan: scan the library with the chosen OCR model once it has loaded, then exit (0, or 1 if any capture failed)
last_window_restore_time: Apollo_Time; last_window_restore_time: Apollo_Time;
imgui_context: *ImGui.ImGuiContext; imgui_context: *ImGui.ImGuiContext;

289
src/app/catalog.jai Normal file
View File

@ -0,0 +1,289 @@
// The catalog: what search and sorting need from every capture, loaded once from the front of each
// file (its Capture Text and OCR text, not its images) on the viewer's worker threads.
//
// Entries are made on the workers and handed over through viewer.catalog_done; only the main thread
// sets or frees viewer.captures[i].catalog, and only it reads them, so no entry is freed while in use.
Catalog_Entry :: struct {
date: string; // CaptureDate, "2026-10-07 08:11:15" (sorts as text); "" when unknown
title, url, app: string; // WindowName, WebURL, AppName
notes: string; // the annotations' text, one per line
ocr: [..] Mcap_Ocr_Section; // one per model that read it
// Lowercased copies, for matching without case.
find_title, find_url, find_app, find_notes, find_ocr: string;
}
// Everything the entry holds is from the default allocator.
catalog_entry_free :: (entry: *Catalog_Entry) {
if !entry return;
push_allocator(context.default_allocator);
free(entry.date); free(entry.title); free(entry.url); free(entry.app); free(entry.notes);
for entry.ocr {
free(it.model); free(it.engine); free(it.image); free(it.date);
for line: it.lines free(line.text);
array_free(it.lines);
}
array_free(entry.ocr);
free(entry.find_title); free(entry.find_url); free(entry.find_app); free(entry.find_notes); free(entry.find_ocr);
free(entry);
}
// The entry for the .mcap at path, or null with why. On any thread.
catalog_load :: (path: string) -> *Catalog_Entry, error: string {
front: Mcap;
ok: bool;
error: string;
{
push_allocator(temp);
front, ok, error = mcap_read_front(path, preview = false);
}
if !ok return null, error;
push_allocator(context.default_allocator);
entry := New(Catalog_Entry);
// The metadata and the annotations, from the Capture Text.
doc: Capture_Document;
capture_document_init(*doc, temp);
if capture_text_parse(*doc, path, front.text) {
metadata := capture_find_file(*doc, "metadata.json");
if metadata {
entry.date = catalog_string(metadata.root, "CaptureDate");
entry.title = catalog_string(metadata.root, "WindowName");
entry.url = catalog_string(metadata.root, "WebURL");
entry.app = catalog_string(metadata.root, "AppName");
}
notes: String_Builder;
notes.allocator = temp;
for file: doc.files {
if file.name == "index.json" || file.name == "metadata.json" || ends_with(file.name, ".backup.json") continue;
objects := capture_object_find(*file.root, "CaptureObjects");
if !objects || objects.kind != .ARRAY continue;
for object: objects.items {
encoded := capture_object_find(*object, "RTFEncodedText");
if !encoded || encoded.kind != .STRING || !encoded.text.count continue;
rtf, decoded := base64_decode(encoded.text,, temp);
if !decoded continue;
text := rtf_to_text(rtf);
if !text.count || text == "?" continue; // "?": an empty text box
if builder_string_length(*notes) append(*notes, "\n");
append(*notes, text);
}
}
entry.notes = builder_to_string(*notes);
}
sections, ocr_ok := mcap_ocr_parse(front.ocr);
if ocr_ok entry.ocr = sections;
entry.find_title = to_lower_copy(entry.title);
entry.find_url = to_lower_copy(entry.url);
entry.find_app = to_lower_copy(entry.app);
entry.find_notes = to_lower_copy(entry.notes);
all_ocr: String_Builder;
all_ocr.allocator = temp;
for entry.ocr for line: it.lines { append(*all_ocr, line.text); append(*all_ocr, "\n"); }
entry.find_ocr = to_lower_copy(builder_to_string(*all_ocr,, temp));
return entry, "";
}
catalog_section :: (entry: *Catalog_Entry, model: string) -> *Mcap_Ocr_Section {
if !entry return null;
for * entry.ocr if it.model == model return it;
return null;
}
// ---------------------------------------------------------------- search
Search_Field :: enum u8 { ANY; OCR; TITLE; URL; APP; NOTE; }
Search_Term :: struct {
field: Search_Field;
text: string; // lowercased
}
// Words, "quoted phrases", and either with a field prefix: ocr: title: url: app: note:. Temp.
search_parse :: (query: string) -> [..] Search_Term {
terms: [..] Search_Term;
terms.allocator = temp;
rest := trim(query);
while rest.count {
term: Search_Term;
colon := find_index_from_left(rest, #char ":");
space := find_index_from_left(rest, #char " ");
if colon > 0 && (space < 0 || colon < space) {
prefix := to_lower_copy(slice(rest, 0, colon),, temp);
field := Search_Field.ANY;
if prefix == {
case "ocr"; field = .OCR;
case "title"; field = .TITLE;
case "url"; field = .URL;
case "app"; field = .APP;
case "note"; field = .NOTE;
}
if field != .ANY { term.field = field; rest = advance(rest, colon + 1); } // "http:..." and the like stay words
}
if rest.count && rest[0] == #char "\"" {
close := find_index_from_left(advance(rest, 1), #char "\"");
if close < 0 { term.text = advance(rest, 1); rest = ""; }
else { term.text = slice(rest, 1, close); rest = advance(rest, close + 2); }
} else {
space = find_index_from_left(rest, #char " ");
if space < 0 { term.text = rest; rest = ""; }
else { term.text = slice(rest, 0, space); rest = advance(rest, space); }
}
term.text = to_lower_copy(term.text,, temp);
if term.text.count array_add(*terms, term);
rest = trim_left(rest);
}
return terms;
}
// Whether entry holds every term.
search_matches :: (entry: *Catalog_Entry, terms: [] Search_Term) -> bool {
if !terms.count return true;
if !entry return false;
for terms {
found: bool;
if it.field == {
case .ANY; found = contains(entry.find_ocr, it.text) || contains(entry.find_title, it.text) || contains(entry.find_url, it.text) ||
contains(entry.find_app, it.text) || contains(entry.find_notes, it.text);
case .OCR; found = contains(entry.find_ocr, it.text);
case .TITLE; found = contains(entry.find_title, it.text);
case .URL; found = contains(entry.find_url, it.text);
case .APP; found = contains(entry.find_app, it.text);
case .NOTE; found = contains(entry.find_notes, it.text);
}
if !found return false;
}
return true;
}
// Whether an OCR line holds one of the terms that can be in OCR text: the ones it should be shown for.
search_line_matches :: (text: string, terms: [] Search_Term) -> bool {
lowered := to_lower_copy(text,, temp);
for terms if (it.field == .ANY || it.field == .OCR) && contains(lowered, it.text) return true;
return false;
}
// ---------------------------------------------------------------- RTF
// An annotation's RTF reduced to its text: control words and their groups dropped, \par and \line as
// line breaks, \uN and \'hh as the characters they stand for. Temp.
rtf_to_text :: (rtf: string) -> string {
out: String_Builder;
out.allocator = temp;
skip_depth := -1; // the group depth at which skipping began; -1 when not skipping
depth := 0;
unicode_skip := 0; // the fallback characters left to drop after a \uN
i := 0;
while i < rtf.count {
c := rtf[i];
if c == #char "{" { depth += 1; i += 1; continue; }
if c == #char "}" {
if skip_depth == depth skip_depth = -1;
depth -= 1; i += 1; continue;
}
if c == #char "\r" || c == #char "\n" { i += 1; continue; }
if c != #char "\\" {
if skip_depth < 0 {
if unicode_skip > 0 unicode_skip -= 1;
else append(*out, c);
}
i += 1;
continue;
}
// A control: \word[-123][ ], \'hh, or \ followed by one character.
i += 1;
if i >= rtf.count break;
c = rtf[i];
if c == #char "'" && i + 2 < rtf.count {
byte := rtf_hex(rtf[i + 1]) * 16 + rtf_hex(rtf[i + 2]);
i += 3;
if skip_depth >= 0 continue;
if unicode_skip > 0 { unicode_skip -= 1; continue; }
rtf_append_code_point(*out, cp1252_to_unicode(cast(u8) byte));
continue;
}
if !is_alpha(c) {
i += 1;
if skip_depth >= 0 continue;
if c == #char "*" skip_depth = depth; // \* starts a destination this reader does not know
else if c == #char "\\" || c == #char "{" || c == #char "}" append(*out, c);
else if c == #char "~" append(*out, " ");
continue;
}
word_start := i;
while i < rtf.count && is_alpha(rtf[i]) i += 1;
word := slice(rtf, word_start, i - word_start);
number_start := i;
if i < rtf.count && rtf[i] == #char "-" i += 1;
while i < rtf.count && is_digit(rtf[i]) i += 1;
number_text := slice(rtf, number_start, i - number_start);
if i < rtf.count && rtf[i] == #char " " i += 1; // the space ending a control word is part of it
if skip_depth >= 0 continue;
if word == {
case "fonttbl"; #through;
case "colortbl"; #through;
case "stylesheet"; #through;
case "info"; #through;
case "pict"; #through;
case "header"; #through;
case "footer"; #through;
case "generator";
skip_depth = depth;
case "par"; #through;
case "line";
append(*out, "\n");
case "tab";
append(*out, "\t");
case "u";
value, ok := string_to_int(number_text);
if ok {
if value < 0 value += 65536;
rtf_append_code_point(*out, cast(u32) value);
unicode_skip = 1;
}
}
}
return trim(builder_to_string(*out,, temp));
}
#scope_file
catalog_string :: (object: Capture_Value, key: string) -> string {
value := capture_object_find(*object, key);
if !value || value.kind != .STRING return "";
return copy_string(value.text);
}
rtf_hex :: (c: u8) -> s64 {
if c >= #char "0" && c <= #char "9" return c - #char "0";
if c >= #char "a" && c <= #char "f" return c - #char "a" + 10;
if c >= #char "A" && c <= #char "F" return c - #char "A" + 10;
return 0;
}
// Windows-1252, which Snagit's RTF uses for \'hh: Latin-1 except 0x80 to 0x9F.
cp1252_to_unicode :: (byte: u8) -> u32 {
HIGH :: u32.[0x20AC, 0x81, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x8D, 0x017D, 0x8F,
0x90, 0x2018, 0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x9D, 0x017E, 0x0178];
if byte >= 0x80 && byte <= 0x9F return HIGH[byte - 0x80];
return byte;
}
rtf_append_code_point :: (out: *String_Builder, code_point: u32) {
if code_point < 0x80 {
append(out, cast(u8) code_point);
} else if code_point < 0x800 {
append(out, cast(u8) (0xC0 | (code_point >> 6)));
append(out, cast(u8) (0x80 | (code_point & 0x3F)));
} else {
append(out, cast(u8) (0xE0 | (code_point >> 12)));
append(out, cast(u8) (0x80 | ((code_point >> 6) & 0x3F)));
append(out, cast(u8) (0x80 | (code_point & 0x3F)));
}
}

View File

@ -17,6 +17,8 @@ main :: () {
app.headless = array_find(app.args, "-headless") || NO_CRASH_DIALOG; app.headless = array_find(app.args, "-headless") || NO_CRASH_DIALOG;
runtime_support_headless = app.headless; runtime_support_headless = app.headless;
app.allow_multiple_instances = array_find(app.args, "-allow-multiple-instances"); app.allow_multiple_instances = array_find(app.args, "-allow-multiple-instances");
for app.args if it == "-library" && it_index + 1 < app.args.count app.library_folder = app.args[it_index + 1];
app.scan_and_exit = array_find(app.args, "-scan");
defer report_memory_leaks(); defer report_memory_leaks();

View File

@ -1,65 +1,127 @@
// OCR in the app: an engine made on a background thread at startup (modules/OCR), an OCR window that // OCR in the app (modules/OCR): an engine made on a background thread, the OCR window (whether it runs
// says whether it runs on the GPU and, if not, what to install, and the OCR button in the Capture // on the GPU and, if not, what to install; the model, kept in the settings file; the library scan),
// window, which reads the capture shown and draws its lines over it. // and in the Capture window the OCR button, the lines read drawn over the image, and the search's
// matches among them.
// //
// Engines take a moment to make and OCR takes a moment to run, so both happen on one background // Every reading is stored in its .mcap (mcap_store_ocr: one section per model) and its catalog entry
// thread; the UI only looks at what it has finished. // is loaded again, so the overlay, the text shown and search all come from what is stored.
//
// Making an engine, reading one capture and the scan all run on one background thread, one at a time;
// the UI only looks at what it has finished, and at the scan's progress counters.
OCR_WINDOW :: "OCR"; OCR_WINDOW :: "OCR";
OCR_DEFAULT_MODEL :: "ppocr-v5-mobile"; OCR_DEFAULT_MODEL :: "ppocr-v5-mobile";
OCR_SCAN_BATCH :: 8; // captures per ocr_process call in a scan
OCR_STORE_ATTEMPTS :: 4; // a file the viewer has open at that moment is tried again, 250 ms apart
ocr_ui: struct { ocr_ui: struct {
models: [] Ocr_Model_Info; models: [] Ocr_Model_Info;
models_error: string; models_error: string;
model_index: s32; // the model chosen in the window model_index: s32; // the model chosen in the OCR window
// Owned by the main thread while thread_busy is false; by the background thread while it is true. // Owned by the main thread while thread_busy is false; by the background thread while it is true.
engine: *Ocr_Engine; engine: *Ocr_Engine;
engine_model: string; // the model engine runs
engine_error: string; // why the last engine could not be made engine_error: string; // why the last engine could not be made
thread: Thread; thread: Thread;
thread_started: bool; thread_started: bool;
thread_busy: bool; // atomic: set by the main thread, cleared by the background one thread_busy: bool; // set by the main thread, cleared by the background one
task: Ocr_Task; task: Ocr_Task;
// The last capture read, for the Capture window. read_error: string; // why the last single read failed
lines: [] Ocr_Line; read_index: s32 = -1;
lines_index: s32 = -1; // the capture they are for read_seconds: float64;
lines_error: string;
lines_seconds: float64; scan: Ocr_Scan;
scan_requested: bool; // -scan has started its scan
// The Capture window.
overlay_model: string; // the section shown over the image; "" for the chosen model's
show_boxes := true; show_boxes := true;
match_cursor: s64 = -1; // which match Next and Previous last went to
scroll_to_match: bool;
window_open := false; window_open := false;
toast_shown := false; toast_shown := false;
} }
Ocr_Task_Kind :: enum { NONE; CREATE; READ; } Ocr_Task_Kind :: enum { NONE; CREATE; READ; SCAN; }
Ocr_Task :: struct { Ocr_Task :: struct {
kind: Ocr_Task_Kind; kind: Ocr_Task_Kind;
model: string; model: string; // CREATE
path: string; // READ: the .mcap path: string; // READ
index: s32; // READ: its capture index index: s32; // READ
} }
// The library scan. The counters and flags are shared with the background thread.
Ocr_Scan :: struct {
items: [..] Scan_Item; // the captures it reads, in display order
model: string;
total: s64;
done: s64; // read and stored, or failed
failed: s64;
paused: bool;
cancel: bool;
running: bool;
start: Apollo_Time;
paused_seconds: float64; // time spent paused, left out of the rate
pause_start: Apollo_Time;
}
Scan_Item :: struct { index: s32; path: string; }
ocr_models_folder :: () -> string { ocr_models_folder :: () -> string {
return tprint("%ocr-models", path_strip_filename(get_path_of_running_executable())); return tprint("%ocr-models", path_strip_filename(get_path_of_running_executable()));
} }
// ---------------------------------------------------------------- startup and shutdown
ocr_ui_startup :: () { ocr_ui_startup :: () {
push_allocator(context.default_allocator); push_allocator(context.default_allocator);
models, error := ocr_list_models(ocr_models_folder()); models, error := ocr_list_models(ocr_models_folder());
ocr_ui.models = models; ocr_ui.models = models;
if error ocr_ui.models_error = copy_string(error); if error ocr_ui.models_error = copy_string(error);
for ocr_ui.models if it.name == OCR_DEFAULT_MODEL ocr_ui.model_index = cast(s32) it_index; ocr_ui.model_index = ocr_model_index(app_user_settings.ocr_model);
if ocr_ui.models.count ocr_ui_start(.{kind = .CREATE, model = ocr_ui.models[ocr_ui.model_index].name}); if ocr_ui.models.count ocr_ui_start(.{kind = .CREATE, model = ocr_ui.models[ocr_ui.model_index].name});
} }
ocr_ui_shutdown :: () { ocr_ui_shutdown :: () {
if ocr_ui.thread_started thread_is_done(*ocr_ui.thread, -1); // finishes what it is doing atomic_swap(*ocr_ui.scan.cancel, true); // a scan stops after the capture it is on
atomic_swap(*ocr_ui.scan.paused, false);
if ocr_ui.thread_started thread_is_done(*ocr_ui.thread, -1);
ocr_destroy(ocr_ui.engine); ocr_destroy(ocr_ui.engine);
} }
// The model the settings name, or the default one, or the first; with a log line when the settings
// name one that is not in models.config.
ocr_model_index :: (name: string) -> s32 {
wanted := ifx name.count then name else OCR_DEFAULT_MODEL;
for ocr_ui.models if it.name == wanted return cast(s32) it_index;
if name.count log_error("[OCR] The settings name model %, which models.config does not have; using %.", name, OCR_DEFAULT_MODEL);
for ocr_ui.models if it.name == OCR_DEFAULT_MODEL return cast(s32) it_index;
return 0;
}
// The settings file named another model (edited by hand): switch to it.
ocr_ui_settings_changed :: () {
if !ocr_ui.models.count return;
index := ocr_model_index(app_user_settings.ocr_model);
if index == ocr_ui.model_index return;
ocr_ui_choose_model(index, save = false);
}
ocr_ui_choose_model :: (index: s32, save := true) {
if ocr_ui.thread_busy return;
ocr_ui.model_index = index;
ocr_ui.toast_shown = true; // said once already, or not needed
ocr_ui_start(.{kind = .CREATE, model = ocr_ui.models[index].name});
if save {
free(app_user_settings.ocr_model,, context.default_allocator);
app_user_settings.ocr_model = copy_string(ocr_ui.models[index].name,, context.default_allocator);
app_save_user_settings_file();
}
}
// Starts task on the background thread; false while it is still busy with the last one. // Starts task on the background thread; false while it is still busy with the last one.
ocr_ui_start :: (task: Ocr_Task) -> bool { ocr_ui_start :: (task: Ocr_Task) -> bool {
if ocr_ui.thread_busy return false; if ocr_ui.thread_busy return false;
@ -76,6 +138,8 @@ ocr_ui_start :: (task: Ocr_Task) -> bool {
return true; return true;
} }
// ---------------------------------------------------------------- the background thread
ocr_ui_thread_proc :: (thread: *Thread) -> s64 { ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
context.allocator = context.default_allocator; context.allocator = context.default_allocator;
task := ocr_ui.task; task := ocr_ui.task;
@ -85,18 +149,35 @@ ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
ocr_ui.engine = null; ocr_ui.engine = null;
engine, ok, error := ocr_create(ocr_models_folder(), task.model); engine, ok, error := ocr_create(ocr_models_folder(), task.model);
ocr_ui.engine = engine; ocr_ui.engine = engine;
ocr_ui.engine_model = task.model; free(ocr_ui.engine_error);
ocr_ui.engine_error = ifx ok then "" else copy_string(error); ocr_ui.engine_error = ifx ok then "" else copy_string(error);
if ok log("[OCR] % on %.%", task.model, engine.provider, ifx engine.gpu_explanation then tprint(" %", engine.gpu_explanation) else ""); if ok log("[OCR] % on %.%", task.model, engine.provider, ifx engine.gpu_explanation then tprint(" %", engine.gpu_explanation) else "");
else log_error("[OCR] Could not start %: %", task.model, error); else log_error("[OCR] Could not start %: %", task.model, error);
case .READ; case .READ;
start := current_time_monotonic(); start := current_time_monotonic();
lines, error := ocr_ui_read_capture(task.path); items := Scan_Item.[.{task.index, task.path}];
ocr_ui.lines = lines; errors := ocr_read_and_store(items);
ocr_ui.lines_error = error; free(ocr_ui.read_error);
ocr_ui.lines_index = task.index; ocr_ui.read_error = ifx errors[0] then errors[0] else "";
ocr_ui.lines_seconds = to_float64_seconds(current_time_monotonic() - start); ocr_ui.read_index = task.index;
ocr_ui.read_seconds = to_float64_seconds(current_time_monotonic() - start);
case .SCAN;
scan := *ocr_ui.scan;
position := 0;
while position < scan.items.count {
while scan.paused && !scan.cancel sleep_milliseconds(100);
if scan.cancel break;
count := min(OCR_SCAN_BATCH, scan.items.count - position);
errors := ocr_read_and_store(array_view(scan.items, position, count));
for errors if it { atomic_add(*scan.failed, 1); log_error("[OCR scan] % %", scan.items[position + it_index].path, it); free(it); }
atomic_add(*scan.done, count);
position += count;
reset_temporary_storage();
}
log("[OCR scan] % of % captures read with % (% failed)%.", scan.done, scan.total, scan.model, scan.failed, ifx scan.cancel then ", cancelled" else "");
atomic_swap(*scan.running, false);
} }
free(task.model); free(task.model);
free(task.path); free(task.path);
@ -104,41 +185,170 @@ ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
return 0; return 0;
} }
// The lines in a capture's image as the Capture window shows it: the rendered one, annotations and // Reads items with the engine, stores each reading in its file, and hands each new catalog entry to
// all, or the original when there is none. Heap. // the viewer. An error message (heap) for each item that failed, "" for the others. Background thread.
ocr_ui_read_capture :: (path: string) -> [] Ocr_Line, error: string { ocr_read_and_store :: (items: [] Scan_Item) -> [] string {
if !ocr_ui.engine return .[], copy_string("No OCR engine is running."); errors := NewArray(items.count, string);
data, read_ok := read_entire_file(path, log_errors = false,, temp); if !ocr_ui.engine { for * errors it.* = copy_string("No OCR engine is running."); return errors; }
if !read_ok return .[], sprint("Cannot read %.", path);
m, ok, error := mcap_parse(cast([] u8) data); // The image the Capture window shows: the rendered one, annotations and all, or the original.
if !ok return .[], copy_string(error); Prepared :: struct { pixels: *u8; width, height: s32; image: string; }
png := mcap_rendered(m); prepared := NewArray(items.count, Prepared,, temp);
if !png.count png = m.original; images: [..] Ocr_Image;
w, h, channels: s32; images.allocator = temp;
pixels := stbi_load_from_memory(png.data, cast(s32) png.count, *w, *h, *channels, 4); image_of: [..] s64; // which item each image is
if !pixels return .[], copy_string("Its image cannot be decoded."); image_of.allocator = temp;
defer stbi_image_free(pixels); for item, n: items {
images := Ocr_Image.[.{pixels, w, h, w * 4}]; data, read_ok := read_entire_file(item.path, log_errors = false,, temp);
results := ocr_process(ocr_ui.engine, images); if !read_ok { errors[n] = copy_string("cannot be read"); continue; }
defer array_free(results); m, ok, error := mcap_parse(cast([] u8) data);
if results[0].failed return .[], results[0].error; if !ok { errors[n] = copy_string(error); continue; }
return results[0].lines, ""; png := mcap_rendered(m);
kind := "rendered";
if !png.count { png = m.original; kind = "original"; }
w, h, channels: s32;
pixels := stbi_load_from_memory(png.data, cast(s32) png.count, *w, *h, *channels, 4);
if !pixels { errors[n] = copy_string("its image cannot be decoded"); continue; }
prepared[n] = .{pixels, w, h, kind};
array_add(*images, .{pixels, w, h, w * 4});
array_add(*image_of, n);
}
defer for prepared if it.pixels stbi_image_free(it.pixels);
results := ocr_process(ocr_ui.engine, images,, temp);
engine_text := tprint("onnxruntime %, %", ort.version, ifx ocr_ui.engine.provider == .CUDA then "CUDA" else "CPU");
for result, r: results {
n := image_of[r];
if result.failed { errors[n] = copy_string(result.error); continue; }
section: Mcap_Ocr_Section;
section.model = ocr_ui.engine.model.name;
section.engine = engine_text;
section.image = prepared[n].image;
section.image_width, section.image_height = prepared[n].width, prepared[n].height;
section.date = mcap_ocr_now();
section.lines.allocator = temp;
for result.lines {
line := array_add(*section.lines);
line.* = .{x = it.x, y = it.y, width = it.width, height = it.height, confidence = it.confidence, text = it.text};
for corner: it.corners {
line.corners[it_index * 2] = cast(s32) round(corner.x);
line.corners[it_index * 2 + 1] = cast(s32) round(corner.y);
}
}
stored: bool;
error: string;
for 1..OCR_STORE_ATTEMPTS {
stored, error = mcap_store_ocr(items[n].path, section);
if stored break;
sleep_milliseconds(250);
}
if !stored { errors[n] = copy_string(error); continue; }
entry, load_error := catalog_load(items[n].path);
if !entry { errors[n] = sprint("stored, but could not be read back: %", load_error); continue; }
lock(*viewer.mutex);
array_add(*viewer.catalog_updates, .{items[n].index, entry});
unlock(*viewer.mutex);
}
return errors;
} }
ocr_ui_free_lines :: () { // ---------------------------------------------------------------- the scan
for ocr_ui.lines free(it.text);
array_free(ocr_ui.lines); // -scan: starts the scan once the engine and the catalog are ready, and ends the app when it is done.
free(ocr_ui.lines_error); ocr_ui_scan_and_exit :: (busy: bool) {
ocr_ui.lines = .[]; finish :: (code: s32, format: string, args: ..Any) {
ocr_ui.lines_error = ""; if code log_error(format, ..args);
ocr_ui.lines_index = -1; else log(format, ..args);
app.exit_code = code;
app.should_exit = true;
app.scan_and_exit = false;
}
if ocr_ui.models_error { finish(1, "[OCR scan] %", ocr_ui.models_error); return; }
if viewer.scan_error { finish(1, "[OCR scan] %", viewer.scan_error); return; }
if !ocr_ui.scan_requested {
if busy || !ocr_ui.engine || viewer.catalog_loaded < viewer.captures.count return;
ocr_ui.scan_requested = true;
ocr_scan_start();
return;
}
if busy || ocr_ui.scan.running return;
finish(ifx ocr_ui.scan.failed then cast(s32) 1 else 0, "[OCR scan] -scan is done: % read, % failed.", ocr_ui.scan.done, ocr_ui.scan.failed);
} }
// ---------------------------------------------------------------- drawing ocr_scan_start :: () {
if ocr_ui.thread_busy || !ocr_ui.engine return;
scan := *ocr_ui.scan;
for scan.items free(it.path);
array_reset(*scan.items);
free(scan.model);
scan.* = .{};
scan.items.allocator = context.default_allocator;
scan.model = copy_string(ocr_ui.engine.model.name,, context.default_allocator);
// Every capture without a reading by this model, in the order the grid shows them.
for viewer.visible {
capture := *viewer.captures[it];
if capture.catalog && catalog_section(capture.catalog, scan.model) continue;
array_add(*scan.items, .{it, copy_string(capture.path,, context.default_allocator)});
}
scan.total = scan.items.count;
if !scan.total { toast_show(.Success, "Every capture shown has been read with %", scan.model); return; }
scan.running = true;
scan.start = current_time_monotonic();
ocr_ui_start(.{kind = .SCAN});
}
ocr_scan_ui :: () { using ImGui;
scan := *ocr_ui.scan;
busy := ocr_ui.thread_busy;
catalog_ready := viewer.catalog_loaded >= viewer.captures.count;
if !scan.running {
can_scan := ocr_ui.engine && !busy && catalog_ready;
if !can_scan BeginDisabled();
label := ifx viewer.search_terms.count then "Scan the captures shown" else "Scan the library";
if Button(temp_c_string(label)) ocr_scan_start();
if !can_scan EndDisabled();
SameLine();
if !catalog_ready TextColored(app_theme.palette.dim, "(once the catalog has loaded)");
else TextColored(app_theme.palette.dim, "Reads every capture without a reading by this model. Each is stored as it is read, so a scan can stop and start again.");
if scan.total && scan.done {
TextColored(app_theme.palette.dim, "Last scan: % of % read with %, % failed.", scan.done, scan.total, scan.model, scan.failed);
}
return;
}
done, total, failed := scan.done, scan.total, scan.failed;
elapsed := to_float64_seconds(current_time_monotonic() - scan.start) - scan.paused_seconds;
if scan.paused elapsed -= to_float64_seconds(current_time_monotonic() - scan.pause_start);
rate := ifx elapsed > 0 then done / elapsed else 0;
left := ifx rate > 0 then (total - done) / rate else 0;
ProgressBar(cast(float) done / max(total, 1), .{-1, 0}, temp_c_string(tprint("% / %", done, total)));
Text("% captures/s, % left, % failed, on %", FF(rate, 1), ocr_duration_text(left), failed, ifx ocr_ui.engine && ocr_ui.engine.provider == .CUDA then "the GPU" else "the CPU");
if scan.paused {
if Button("Resume") { scan.paused_seconds += to_float64_seconds(current_time_monotonic() - scan.pause_start); atomic_swap(*scan.paused, false); }
} else {
if Button("Pause") { scan.pause_start = current_time_monotonic(); atomic_swap(*scan.paused, true); }
}
SameLine();
if Button("Cancel") { atomic_swap(*scan.cancel, true); atomic_swap(*scan.paused, false); }
}
ocr_duration_text :: (seconds: float64) -> string {
s := cast(s64) seconds;
if s >= 3600 return tprint("% h % min", s / 3600, (s % 3600) / 60);
if s >= 60 return tprint("% min % s", s / 60, s % 60);
return tprint("% s", s);
}
// ---------------------------------------------------------------- the OCR window
// Each frame: the toast once the engine is up on the CPU though a GPU could run it, and the OCR window. // Each frame: the toast once the engine is up on the CPU though a GPU could run it, and the OCR window.
ocr_ui_frame :: () { using ImGui; ocr_ui_frame :: () { using ImGui;
busy := ocr_ui.thread_busy; busy := ocr_ui.thread_busy;
if app.scan_and_exit ocr_ui_scan_and_exit(busy);
if !busy && ocr_ui.engine && !ocr_ui.toast_shown { if !busy && ocr_ui.engine && !ocr_ui.toast_shown {
ocr_ui.toast_shown = true; ocr_ui.toast_shown = true;
in_use, explanation := ocr_gpu_status(ocr_ui.engine); in_use, explanation := ocr_gpu_status(ocr_ui.engine);
@ -149,46 +359,46 @@ ocr_ui_frame :: () { using ImGui;
} }
if !ocr_ui.window_open return; if !ocr_ui.window_open return;
SetNextWindowSize(.{640, 360}, .FirstUseEver); SetNextWindowSize(.{680, 420}, .FirstUseEver);
if Begin(OCR_WINDOW, *ocr_ui.window_open) { if Begin(OCR_WINDOW, *ocr_ui.window_open) {
if ocr_ui.models_error { if ocr_ui.models_error {
TextColored(app_theme.palette.warning, "%", ocr_ui.models_error); TextColored(app_theme.palette.warning, "%", ocr_ui.models_error);
TextWrapped("Run `jai fetch_models.jai` in modules/OCR and `jai fetch.jai` in modules/onnxruntime, then rebuild."); TextWrapped("Run `jai fetch_models.jai` in modules/OCR and `jai fetch.jai` in modules/onnxruntime, then rebuild.");
} else { } else {
// The model: changing it makes a new engine on the background thread. // The model: changing it makes a new engine, and is kept in the settings file.
preview := ifx ocr_ui.models.count then ocr_ui.models[ocr_ui.model_index].title else ""; preview := ifx ocr_ui.models.count then ocr_ui.models[ocr_ui.model_index].title else "";
if busy BeginDisabled();
SetNextItemWidth(320); SetNextItemWidth(320);
if BeginCombo("Model", temp_c_string(preview)) { if BeginCombo("Model", temp_c_string(preview)) {
for ocr_ui.models { for ocr_ui.models {
if Selectable(temp_c_string(it.title), it_index == ocr_ui.model_index) && it_index != ocr_ui.model_index && !busy { if Selectable(temp_c_string(it.title), it_index == ocr_ui.model_index) && it_index != ocr_ui.model_index ocr_ui_choose_model(cast(s32) it_index);
ocr_ui.model_index = cast(s32) it_index;
ocr_ui.toast_shown = true; // said once already, or not needed
ocr_ui_free_lines();
ocr_ui_start(.{kind = .CREATE, model = it.name});
}
} }
EndCombo(); EndCombo();
} }
if busy EndDisabled();
if busy && ocr_ui.task.kind == .CREATE { if busy && ocr_ui.task.kind == .CREATE {
TextColored(app_theme.palette.dim, "Starting % ...", ocr_ui.task.model); TextColored(app_theme.palette.dim, "Starting % ...", ocr_ui.task.model);
} else if ocr_ui.engine_error { } else if ocr_ui.engine_error {
TextColored(app_theme.palette.warning, "OCR could not start:"); TextColored(app_theme.palette.warning, "OCR could not start:");
TextWrapped("%", ocr_ui.engine_error); TextWrapped("%", ocr_ui.engine_error);
} else if ocr_ui.engine && !busy { } else if ocr_ui.engine {
in_use, explanation := ocr_gpu_status(ocr_ui.engine); in_use, explanation := ocr_gpu_status(ocr_ui.engine);
Text("Running on: %", ifx in_use then "the GPU (CUDA)" else "the CPU"); Text("Running on: %", ifx in_use then "the GPU (CUDA)" else "the CPU");
SameLine(); SameLine();
TextColored(app_theme.palette.dim, "onnxruntime %, % worker threads", ort.version, ocr_ui.engine.workers); TextColored(app_theme.palette.dim, "onnxruntime %, % worker threads", ort.version, ocr_ui.engine.workers);
Separator();
ocr_scan_ui();
if explanation { Separator(); text_with_links(explanation); } if explanation { Separator(); text_with_links(explanation); }
} }
// The text of the capture last read, in reading order. // The text of the capture shown in the Capture window, as the overlay's model read it.
if !busy && ocr_ui.lines_index >= 0 && ocr_ui.lines_index < viewer.captures.count { section := ocr_overlay_section();
if section && viewer.full_index >= 0 {
Separator(); Separator();
Text("Text of %:", viewer.captures[ocr_ui.lines_index].name); Text("Text of % (% , % lines):", viewer.captures[viewer.full_index].name, section.model, section.lines.count);
BeginChild("ocr text"); BeginChild("ocr text");
ocr_ui_capture_text(ocr_ui.lines_index); for section.lines TextUnformatted(it.text);
EndChild(); EndChild();
} }
} }
@ -196,45 +406,6 @@ ocr_ui_frame :: () { using ImGui;
End(); End();
} }
// The OCR row of the Capture window, above its image: the button, and how the last read went.
ocr_ui_capture_controls :: (index: s32, path: string) { using ImGui;
busy := ocr_ui.thread_busy;
can_read := ocr_ui.engine && !busy;
if !can_read BeginDisabled();
if SmallButton("OCR") {
ocr_ui_free_lines();
ocr_ui_start(.{kind = .READ, path = path, index = index});
}
if !can_read EndDisabled();
SameLine();
if busy && ocr_ui.task.kind == .READ && ocr_ui.task.index == index {
TextColored(app_theme.palette.dim, "Reading...");
} else if !busy && ocr_ui.lines_index == index {
if ocr_ui.lines_error TextColored(app_theme.palette.warning, "%", ocr_ui.lines_error);
else TextColored(app_theme.palette.dim, "% lines in % s", ocr_ui.lines.count, FF(ocr_ui.lines_seconds, 2));
SameLine();
Checkbox("Boxes", *ocr_ui.show_boxes);
} else if !ocr_ui.engine {
TextColored(app_theme.palette.dim, "(OCR is not running; see the OCR window)");
}
SameLine();
if SmallButton("OCR window") ocr_ui.window_open = true;
}
// The lines read from this capture, as boxes over its image, drawn at image_min with scale; and their
// text in a list below the image area.
ocr_ui_capture_overlay :: (index: s32, image_min: ImGui.ImVec2, scale: float) { using ImGui;
if ocr_ui.thread_busy || ocr_ui.lines_index != index || !ocr_ui.show_boxes return;
draw := GetWindowDrawList();
colour := GetColorU32(.PlotHistogram);
for ocr_ui.lines {
p0 := ImVec2.{image_min.x + it.x * scale, image_min.y + it.y * scale};
p1 := ImVec2.{p0.x + it.width * scale, p0.y + it.height * scale};
draw.AddRect(draw, p0, p1, colour, 0, 1.5);
if IsMouseHoveringRect(p0, p1) SetTooltip("% (%)", it.text, FF(it.confidence, 2));
}
}
// Text a line at a time, each https:// address in it drawn as a link that opens in the browser. // Text a line at a time, each https:// address in it drawn as a link that opens in the browser.
text_with_links :: (text: string) { using ImGui; text_with_links :: (text: string) { using ImGui;
for line: split(text, "\n",, temp) { for line: split(text, "\n",, temp) {
@ -250,7 +421,113 @@ text_with_links :: (text: string) { using ImGui;
} }
} }
ocr_ui_capture_text :: (index: s32) { using ImGui; // ---------------------------------------------------------------- the Capture window
if ocr_ui.thread_busy || ocr_ui.lines_index != index || !ocr_ui.lines.count return;
for ocr_ui.lines TextUnformatted(it.text); // The reading shown over the capture in the Capture window: the overlay's choice, else the chosen
// model's, else the first there is.
ocr_overlay_section :: () -> *Mcap_Ocr_Section {
if viewer.full_index < 0 || viewer.full_index >= viewer.captures.count return null;
entry := viewer.captures[viewer.full_index].catalog;
if !entry || !entry.ocr.count return null;
if ocr_ui.overlay_model.count {
section := catalog_section(entry, ocr_ui.overlay_model);
if section return section;
}
if ocr_ui.models.count {
section := catalog_section(entry, ocr_ui.models[ocr_ui.model_index].name);
if section return section;
}
return *entry.ocr[0];
}
// The OCR row of the Capture window, above its image.
ocr_ui_capture_controls :: (index: s32, path: string) { using ImGui;
busy := ocr_ui.thread_busy;
entry := viewer.captures[index].catalog;
model := ifx ocr_ui.models.count then ocr_ui.models[ocr_ui.model_index].name else "";
has_reading := entry && catalog_section(entry, model) != null;
can_read := ocr_ui.engine && !busy;
if !can_read BeginDisabled();
if SmallButton(temp_c_string(ifx has_reading then "Read again" else "OCR")) ocr_ui_start(.{kind = .READ, path = path, index = index});
if !can_read EndDisabled();
SameLine();
if busy && ocr_ui.task.kind == .READ && ocr_ui.task.index == index {
TextColored(app_theme.palette.dim, "Reading...");
SameLine();
} else if !busy && ocr_ui.read_index == index {
if ocr_ui.read_error { TextColored(app_theme.palette.warning, "%", ocr_ui.read_error); SameLine(); }
else { TextColored(app_theme.palette.dim, "read in % s", FF(ocr_ui.read_seconds, 2)); SameLine(); }
}
section := ocr_overlay_section();
if section {
// A capture read by several models: which reading to show.
if entry.ocr.count > 1 {
SetNextItemWidth(170);
if BeginCombo("##reading", temp_c_string(section.model)) {
for entry.ocr if Selectable(temp_c_string(it.model), it.model == section.model) {
free(ocr_ui.overlay_model,, context.default_allocator);
ocr_ui.overlay_model = copy_string(it.model,, context.default_allocator);
}
EndCombo();
}
SameLine();
}
Checkbox("Boxes", *ocr_ui.show_boxes);
if viewer.search_terms.count {
matches := 0;
for section.lines if search_line_matches(it.text, viewer.search_terms) matches += 1;
SameLine();
Text("Matches: %", matches);
if matches {
SameLine(); if SmallButton("Previous") { ocr_ui.match_cursor = (ocr_ui.match_cursor - 1 + matches) % matches; ocr_ui.scroll_to_match = true; }
SameLine(); if SmallButton("Next") { ocr_ui.match_cursor = (ocr_ui.match_cursor + 1) % matches; ocr_ui.scroll_to_match = true; }
}
}
SameLine();
} else if !ocr_ui.engine {
TextColored(app_theme.palette.dim, "(OCR is not running; see the OCR window)");
SameLine();
}
if SmallButton("OCR window") ocr_ui.window_open = true;
}
// The reading's lines as outlines over the image (turned with the text, for a turned line), drawn at
// image_min with zoom (the image's size as shown over its captured size): those matching the search
// highlighted, the others dimmer. Inside the image's scrolling child, which Next and Previous scroll.
ocr_ui_capture_overlay :: (index: s32, image_min: ImGui.ImVec2, zoom: float) { using ImGui;
section := ocr_overlay_section();
if !section || !ocr_ui.show_boxes return;
// The section's pixels may be another image's (the original, where the rendered one differs).
scale_x := zoom * ifx section.image_width then cast(float) viewer.full_source_width / section.image_width else 1;
scale_y := zoom * ifx section.image_height then cast(float) viewer.full_source_height / section.image_height else 1;
draw := GetWindowDrawList();
searching := viewer.search_terms.count > 0;
plain := GetColorU32(.PlotHistogram, ifx searching then 0.35 else 1.0);
matching := GetColorU32(.PlotLinesHovered);
match_number := 0;
for section.lines {
p0 := ImVec2.{image_min.x + it.x * scale_x, image_min.y + it.y * scale_y};
p1 := ImVec2.{p0.x + it.width * scale_x, p0.y + it.height * scale_y};
c := mcap_ocr_corners(it);
q: [4] ImVec2;
for * q it.* = .{image_min.x + c[it_index * 2] * scale_x, image_min.y + c[it_index * 2 + 1] * scale_y};
is_match := searching && search_line_matches(it.text, viewer.search_terms);
if is_match {
draw.AddQuadFilled(draw, q[0], q[1], q[2], q[3], GetColorU32(.PlotLinesHovered, 0.25));
draw.AddQuad(draw, q[0], q[1], q[2], q[3], matching, 2.5);
if ocr_ui.scroll_to_match && match_number == ocr_ui.match_cursor {
// Into view, a third of the way down.
SetScrollX(it.x * scale_x - GetWindowWidth() / 3);
SetScrollY(it.y * scale_y - GetWindowHeight() / 3);
ocr_ui.scroll_to_match = false;
}
match_number += 1;
} else {
draw.AddQuad(draw, q[0], q[1], q[2], q[3], plain, 1.5);
}
if IsMouseHoveringRect(p0, p1) SetTooltip("% (%)", it.text, FF(it.confidence, 2));
}
} }

View File

@ -2,8 +2,16 @@
// its own. Scrolling never waits on the disk: previews are read and decoded on worker threads, // its own. Scrolling never waits on the disk: previews are read and decoded on worker threads,
// those in view first and then the rows around them, and are uploaded a few per frame. Textures // those in view first and then the rows around them, and are uploaded a few per frame. Textures
// for previews far out of view are released once there are more than VIEWER_MAX_TEXTURES. // for previews far out of view are released once there are more than VIEWER_MAX_TEXTURES.
//
// When the workers have no previews to decode they load the catalog (app/catalog.jai), what search
// and sorting need from each capture. The grid shows viewer.visible: the captures in display order,
// newest first once the catalog is complete, and only those matching the search.
VIEWER_FOLDER :: "D:/SnagitIndex/mcap"; // hardcoded for now VIEWER_FOLDER :: "D:/SnagitIndex/mcap"; // hardcoded for now; -library FOLDER shows another
viewer_folder :: () -> string {
return ifx app.library_folder then app.library_folder else VIEWER_FOLDER;
}
VIEWER_WORKER_COUNT :: 4; VIEWER_WORKER_COUNT :: 4;
VIEWER_MAX_TEXTURES :: 1200; // previews on the GPU at once, about 200 KB each VIEWER_MAX_TEXTURES :: 1200; // previews on the GPU at once, about 200 KB each
VIEWER_PREFETCH_ROWS :: 8; // most rows decoded ahead of the view, above and below VIEWER_PREFETCH_ROWS :: 8; // most rows decoded ahead of the view, above and below
@ -36,6 +44,13 @@ Viewer_Capture :: struct {
last_wanted_frame: u64; // last in the rows viewer_request_previews asked for last_wanted_frame: u64; // last in the rows viewer_request_previews asked for
failed: bool; failed: bool;
in_flight: bool; // under viewer.mutex: queued for a worker, or being decoded in_flight: bool; // under viewer.mutex: queued for a worker, or being decoded
catalog: *Catalog_Entry; // main thread only; null until loaded
}
// A catalog entry made off the main thread, for the main thread to put in place.
Catalog_Update :: struct {
index: s32;
entry: *Catalog_Entry; // null when it could not be loaded
} }
// How the full-size window sizes its image. // How the full-size window sizes its image.
@ -78,6 +93,17 @@ viewer: struct {
quit: bool; quit: bool;
workers: [VIEWER_WORKER_COUNT] Thread; workers: [VIEWER_WORKER_COUNT] Thread;
workers_started: s64; workers_started: s64;
catalog_next: s64; // the next capture whose catalog entry a worker loads
catalog_updates: [..] Catalog_Update; // made, waiting for the main thread
// Display order and search; main thread only.
visible: [..] s32; // indices into captures, as the grid shows them
visible_dirty := true;
visible_rebuilt_at: Apollo_Time;
catalog_loaded: s64; // entries put in place, or found unreadable
sorted_by_date: bool;
search_buffer: [256] u8;
search_terms: [..] Search_Term;
// The full view; main thread only. // The full view; main thread only.
full_open: bool; full_open: bool;
@ -98,12 +124,15 @@ viewer_startup :: () {
viewer.with_texture.allocator = context.default_allocator; viewer.with_texture.allocator = context.default_allocator;
viewer.queue.allocator = context.default_allocator; viewer.queue.allocator = context.default_allocator;
viewer.done.allocator = context.default_allocator; viewer.done.allocator = context.default_allocator;
viewer.catalog_updates.allocator = context.default_allocator;
viewer.visible.allocator = context.default_allocator;
viewer.search_terms.allocator = context.default_allocator;
init(*viewer.mutex, "viewer"); init(*viewer.mutex, "viewer");
init(*viewer.wake); init(*viewer.wake);
start := current_time_monotonic(); start := current_time_monotonic();
files := file_list(VIEWER_FOLDER,, temp); files := file_list(viewer_folder(),, temp);
for files { for files {
name := path_filename(it); name := path_filename(it);
if !ends_with_nocase(name, tprint(".%", MCAP_EXTENSION)) continue; if !ends_with_nocase(name, tprint(".%", MCAP_EXTENSION)) continue;
@ -113,8 +142,8 @@ viewer_startup :: () {
} }
// Newest first, roughly: dated names (2026-...) sort after the older {GUID} ones. // Newest first, roughly: dated names (2026-...) sort after the older {GUID} ones.
quick_sort(viewer.captures, (a: Viewer_Capture, b: Viewer_Capture) -> s64 { return compare(b.name, a.name); }); quick_sort(viewer.captures, (a: Viewer_Capture, b: Viewer_Capture) -> s64 { return compare(b.name, a.name); });
if !viewer.captures.count viewer.scan_error = sprint("No .% files in %.", MCAP_EXTENSION, VIEWER_FOLDER); if !viewer.captures.count viewer.scan_error = sprint("No .% files in %.", MCAP_EXTENSION, viewer_folder());
log("[viewer] Found % captures in % in % ms.", viewer.captures.count, VIEWER_FOLDER, FF(to_float64_seconds(current_time_monotonic() - start) * 1000.0, 1)); log("[viewer] Found % captures in % in % ms.", viewer.captures.count, viewer_folder(), FF(to_float64_seconds(current_time_monotonic() - start) * 1000.0, 1));
for * viewer.workers { for * viewer.workers {
// Reading a front takes 64 KB of temporary storage, and a full image several MB. // Reading a front takes 64 KB of temporary storage, and a full image several MB.
@ -144,6 +173,7 @@ viewer_worker :: (thread: *Thread) -> s64 {
while true { while true {
index: s32 = -1; index: s32 = -1;
full: bool; full: bool;
catalog: bool;
request: u64; request: u64;
path: string; path: string;
@ -157,11 +187,26 @@ viewer_worker :: (thread: *Thread) -> s64 {
} else if viewer.queue_next < viewer.queue.count { } else if viewer.queue_next < viewer.queue.count {
index = viewer.queue[viewer.queue_next]; index = viewer.queue[viewer.queue_next];
viewer.queue_next += 1; viewer.queue_next += 1;
} else if viewer.catalog_next < viewer.captures.count {
// Nothing to show is waiting: load the next catalog entry.
catalog = true;
index = cast(s32) viewer.catalog_next;
viewer.catalog_next += 1;
} }
if index >= 0 path = viewer.captures[index].path; // the array no longer grows after startup if index >= 0 path = viewer.captures[index].path; // the array no longer grows after startup
unlock(*viewer.mutex); unlock(*viewer.mutex);
if index < 0 break; // nothing to do: wait to be woken if index < 0 break; // nothing to do: wait to be woken
if catalog {
entry, error := catalog_load(path);
if !entry log_error("[catalog] % %", path, error);
reset_temporary_storage();
lock(*viewer.mutex);
array_add(*viewer.catalog_updates, .{index, entry});
unlock(*viewer.mutex);
continue;
}
result := ifx full then viewer_decode_full(path) else viewer_decode_preview(path); result := ifx full then viewer_decode_full(path) else viewer_decode_preview(path);
result.index = index; result.index = index;
@ -318,6 +363,8 @@ viewer_frame_gpu_work :: () {
} }
viewer_evict(); viewer_evict();
viewer_apply_catalog_updates();
if viewer.visible_dirty viewer_rebuild_visible();
} }
// When a preview was last in view or wanted ahead of it. A prefetched preview has not been drawn // When a preview was last in view or wanted ahead of it. A prefetched preview has not been drawn
@ -326,6 +373,69 @@ viewer_last_used :: (capture: Viewer_Capture) -> u64 {
return max(capture.last_drawn_frame, capture.last_wanted_frame); return max(capture.last_drawn_frame, capture.last_wanted_frame);
} }
// Catalog entries made by the workers (or by an OCR scan) go in place here, on the main thread,
// which is the only one that reads them; a replaced entry is freed now, before anything draws it.
viewer_apply_catalog_updates :: () {
lock(*viewer.mutex);
updates := array_copy(viewer.catalog_updates,, temp);
array_reset_keeping_memory(*viewer.catalog_updates);
more_to_load := viewer.catalog_next < viewer.captures.count;
unlock(*viewer.mutex);
for updates {
capture := *viewer.captures[it.index];
if !capture.catalog viewer.catalog_loaded += 1;
if it.entry || !capture.catalog {
catalog_entry_free(capture.catalog);
capture.catalog = it.entry;
}
}
if updates.count && viewer.search_terms.count viewer.visible_dirty = true;
if !viewer.sorted_by_date && viewer.catalog_loaded >= viewer.captures.count {
viewer.sorted_by_date = true;
viewer.visible_dirty = true;
}
// The workers sleep when there is nothing to show; wake them while there is catalog to load.
if more_to_load signal(*viewer.wake);
}
// The captures in display order: by date, newest first, once every date is known (by name until
// then), and only those matching the search.
viewer_rebuild_visible :: () {
// While the catalog loads under a search, matches arrive every frame: a few rebuilds a second do.
if viewer.search_terms.count && !viewer.sorted_by_date && to_float64_seconds(current_time_monotonic() - viewer.visible_rebuilt_at) < 0.25 return;
viewer.visible_dirty = false;
viewer.visible_rebuilt_at = current_time_monotonic();
array_reset_keeping_memory(*viewer.visible);
for viewer.captures {
if viewer.search_terms.count && !search_matches(it.catalog, viewer.search_terms) continue;
array_add(*viewer.visible, cast(s32) it_index);
}
if viewer.sorted_by_date {
quick_sort(viewer.visible, (a: s32, b: s32) -> s64 {
ca, cb := *viewer.captures[a], *viewer.captures[b];
return viewer_display_order(ifx ca.catalog then ca.catalog.date else "", ca.name, ifx cb.catalog then cb.catalog.date else "", cb.name);
});
}
}
// The grid's order: newest CaptureDate first, undated captures last, ties by name, newest first.
viewer_display_order :: (date_a: string, name_a: string, date_b: string, name_b: string) -> s64 {
if date_a != date_b {
if !date_a.count return 1;
if !date_b.count return -1;
return compare(date_b, date_a);
}
return compare(name_b, name_a);
}
// Where index is in the display order, or -1.
viewer_visible_position :: (index: s32) -> s64 {
for viewer.visible if it == index return it_index;
return -1;
}
// Past VIEWER_MAX_TEXTURES, the previews used longest ago are released, never one in view or // Past VIEWER_MAX_TEXTURES, the previews used longest ago are released, never one in view or
// wanted around it as of the last frame. // wanted around it as of the last frame.
viewer_evict :: () { viewer_evict :: () {
@ -350,8 +460,8 @@ viewer_evict :: () {
// What the workers should decode next: the rows in view, then the rows around them, nearest first. // What the workers should decode next: the rows in view, then the rows around them, nearest first.
viewer_request_previews :: (first_row: s64, last_row: s64, columns: s64) { viewer_request_previews :: (first_row: s64, last_row: s64, columns: s64) {
if !viewer.captures.count || columns <= 0 return; if !viewer.visible.count || columns <= 0 return;
rows := (viewer.captures.count + columns - 1) / columns; rows := (viewer.visible.count + columns - 1) / columns;
// Rows ahead only as far as the textures they need leave half of VIEWER_MAX_TEXTURES spare, so // Rows ahead only as far as the textures they need leave half of VIEWER_MAX_TEXTURES spare, so
// what is wanted always fits and nothing wanted is ever released to make room. // what is wanted always fits and nothing wanted is ever released to make room.
@ -363,8 +473,9 @@ viewer_request_previews :: (first_row: s64, last_row: s64, columns: s64) {
wanted.allocator = temp; wanted.allocator = temp;
add_row :: (wanted: *[..] s32, row: s64, columns: s64) { add_row :: (wanted: *[..] s32, row: s64, columns: s64) {
for column: 0..columns - 1 { for column: 0..columns - 1 {
index := row * columns + column; position := row * columns + column;
if index >= viewer.captures.count break; if position >= viewer.visible.count break;
index := viewer.visible[position];
capture := *viewer.captures[index]; capture := *viewer.captures[index];
capture.last_wanted_frame = viewer.frame; // kept by viewer_evict, texture or not capture.last_wanted_frame = viewer.frame; // kept by viewer_evict, texture or not
if !capture.texture.handle && !capture.failed array_add(wanted, cast(s32) index); if !capture.texture.handle && !capture.failed array_add(wanted, cast(s32) index);
@ -413,11 +524,25 @@ viewer_ui :: () { using ImGui;
if viewer.scan_error { TextColored(app_theme.palette.warning, "%", viewer.scan_error); return; } if viewer.scan_error { TextColored(app_theme.palette.warning, "%", viewer.scan_error); return; }
loaded := viewer.with_texture.count; SetNextItemWidth(min(GetContentRegionAvail().x * 0.5, 520));
Text("% captures in %", viewer.captures.count, VIEWER_FOLDER); if InputTextWithHint("##search", "Search text, titles, URLs, apps, notes. \"a phrase\" ocr: title: url: app: note:", viewer.search_buffer.data, viewer.search_buffer.count) {
query := to_string(viewer.search_buffer.data);
terms := search_parse(query);
for viewer.search_terms free(it.text);
array_reset_keeping_memory(*viewer.search_terms);
for terms array_add(*viewer.search_terms, .{it.field, copy_string(it.text)});
viewer.visible_dirty = true;
viewer.visible_rebuilt_at = .{}; // at once, not throttled
}
SameLine(); SameLine();
TextColored(app_theme.palette.dim, "(% previews loaded)", loaded); if viewer.search_terms.count Text("% of % captures", viewer.visible.count, viewer.captures.count);
SetNextItemWidth(200); else Text("% captures", viewer.captures.count);
if viewer.catalog_loaded < viewer.captures.count {
SameLine();
TextColored(app_theme.palette.dim, "(loading the catalog: % of %; sorted by name until then)", viewer.catalog_loaded, viewer.captures.count);
}
SameLine();
SetNextItemWidth(160);
SliderFloat("Size", *viewer.thumb_size, 64, MCAP_PREVIEW_MAX_SIDE, "%.0f px"); SliderFloat("Size", *viewer.thumb_size, 64, MCAP_PREVIEW_MAX_SIDE, "%.0f px");
BeginChild("grid"); BeginChild("grid");
@ -433,7 +558,7 @@ viewer_grid :: () { using ImGui;
label := GetTextLineHeight(); label := GetTextLineHeight();
available := GetContentRegionAvail().x; available := GetContentRegionAvail().x;
columns := max(1, cast(s64) ((available + style.ItemSpacing.x) / (cell + style.ItemSpacing.x))); columns := max(1, cast(s64) ((available + style.ItemSpacing.x) / (cell + style.ItemSpacing.x)));
rows := (viewer.captures.count + columns - 1) / columns; rows := (viewer.visible.count + columns - 1) / columns;
row_height := cell + label + style.ItemSpacing.y; row_height := cell + label + style.ItemSpacing.y;
draw := GetWindowDrawList(); draw := GetWindowDrawList();
@ -452,8 +577,9 @@ viewer_grid :: () { using ImGui;
last_row = max(last_row, cast(s64) clipper.DisplayEnd - 1); last_row = max(last_row, cast(s64) clipper.DisplayEnd - 1);
for row: clipper.DisplayStart..clipper.DisplayEnd - 1 { for row: clipper.DisplayStart..clipper.DisplayEnd - 1 {
for column: 0..columns - 1 { for column: 0..columns - 1 {
index := row * columns + column; position := row * columns + column;
if index >= viewer.captures.count break; if position >= viewer.visible.count break;
index := viewer.visible[position];
capture := *viewer.captures[index]; capture := *viewer.captures[index];
if column > 0 SameLine(); if column > 0 SameLine();
@ -530,8 +656,9 @@ viewer_full_window :: () { using ImGui;
// Left and Right step through the captures in grid order while this window has focus. // Left and Right step through the captures in grid order while this window has focus.
if IsWindowFocused(.RootAndChildWindows) { if IsWindowFocused(.RootAndChildWindows) {
if IsKeyPressed(.LeftArrow) && viewer.full_index > 0 viewer_request_full(viewer.full_index - 1); position := viewer_visible_position(viewer.full_index);
if IsKeyPressed(.RightArrow) && viewer.full_index < viewer.captures.count - 1 viewer_request_full(viewer.full_index + 1); if IsKeyPressed(.LeftArrow) && position > 0 viewer_request_full(viewer.visible[position - 1]);
if IsKeyPressed(.RightArrow) && position >= 0 && position < viewer.visible.count - 1 viewer_request_full(viewer.visible[position + 1]);
} }
if viewer.full_failed { if viewer.full_failed {

121
src/test/catalog_tests.jai Normal file
View File

@ -0,0 +1,121 @@
// The catalog's pieces on their own (app/catalog.jai, app/viewer.jai): the search query's parsing and
// matching, the RTF stripper on Snagit's annotations, and the grid's order by date.
#scope_file
checks_run := 0;
checks_failed := 0;
MAX_GROUPS :: 8;
group_names : [MAX_GROUPS] string;
group_checks : [MAX_GROUPS] int;
group_fails : [MAX_GROUPS] int;
group_count := 0;
output_builder: *String_Builder;
emit :: (format_string: string, args: ..Any) {
if output_builder print_to_builder(output_builder, format_string, ..args);
else print(format_string, ..args);
} @PrintLike
group :: (name: string) {
assert(group_count < MAX_GROUPS, "raise MAX_GROUPS");
group_names[group_count] = name;
group_count += 1;
emit("\n-- % --\n", name);
}
expect :: (condition: bool, format: string, args: ..Any, loc := #caller_location) {
checks_run += 1;
if group_count group_checks[group_count-1] += 1;
if condition return;
checks_failed += 1;
if group_count group_fails[group_count-1] += 1;
emit(" FAIL [%:%] %\n", group_names[max(group_count-1, 0)], loc.line_number, tprint(format, ..args));
}
report :: () -> bool {
emit("\n");
for 0..group_count-1 {
if group_fails[it] emit(" % -- % checks, % FAILED\n", group_names[it], group_checks[it], group_fails[it]);
else emit(" % -- % checks\n", group_names[it], group_checks[it]);
}
if checks_failed emit("\n% OF % CATALOG CHECKS FAILED\n", checks_failed, checks_run);
else emit("\nALL CATALOG TESTS PASSED (% checks over % groups)\n", checks_run, group_count);
return checks_failed == 0;
}
query_tests :: () {
group("query");
terms := search_parse(" KiCad \"Footprint Assignments\" url:digikey note:late http://x.y ");
expect(terms.count == 5, "five terms (got %)", terms.count);
if terms.count != 5 return;
expect(terms[0].field == .ANY && terms[0].text == "kicad", "a word, lowercased: %", terms[0]);
expect(terms[1].field == .ANY && terms[1].text == "footprint assignments", "a phrase: %", terms[1]);
expect(terms[2].field == .URL && terms[2].text == "digikey", "a field prefix: %", terms[2]);
expect(terms[3].field == .NOTE && terms[3].text == "late", "note: %", terms[3]);
expect(terms[4].field == .ANY && terms[4].text == "http://x.y", "an unknown prefix stays part of the word: %", terms[4]);
phrase := search_parse("title:\"two words\"");
expect(phrase.count == 1 && phrase[0].field == .TITLE && phrase[0].text == "two words", "a prefixed phrase: %", phrase);
expect(search_parse(" ").count == 0, "blank is no terms");
}
match_tests :: () {
group("matching");
entry: Catalog_Entry;
entry.find_title = "assign footprints - kicad";
entry.find_url = "https://www.digikey.com/x";
entry.find_app = "kicad eda 7.0.2";
entry.find_notes = "late";
entry.find_ocr = "0.1uf : capacitor_smd:c_0201_0603metric\nc1 -\n";
expect(search_matches(*entry, search_parse("capacitor kicad")), "words from different fields all match");
expect(!search_matches(*entry, search_parse("capacitor resistor")), "every word must match");
expect(search_matches(*entry, search_parse("url:digikey")), "a URL word");
expect(!search_matches(*entry, search_parse("title:digikey")), "a prefix limits the word to its field");
expect(search_matches(*entry, search_parse("ocr:\"c1 -\"")), "an OCR phrase");
expect(search_matches(*entry, search_parse("note:LATE")), "matching ignores case");
expect(search_matches(*entry, search_parse("")), "no terms match everything");
expect(!search_matches(null, search_parse("x")), "an entry not loaded matches no search");
expect(search_line_matches("0.1uF : Capacitor_SMD", search_parse("capacitor")), "a line holding a word");
expect(!search_line_matches("0.1uF : Capacitor_SMD", search_parse("title:capacitor")), "a title word is not an OCR line's");
}
rtf_tests :: () {
group("rtf");
// A Snagit text annotation, as stored (base64 RTF), whose text is "late".
encoded :: "e1xydGYxXGFuc2lcYW5zaWNwZzEyNTJcZGVmZjBcZGVmbGFuZzEwMzN7XGZvbnR0Ymx7XGYwXGZzd2lzc1xmcHJxMlxmY2hhcnNldDAgQXJpYWw7fX0NCntcY29sb3J0YmwgO1xyZWQyNTJcZ3JlZW42NlxibHVlNjY7fQ0KXHZpZXdraW5kNFx1YzFccGFyZFxxY1x0eDcyMFxjZjFcZjBcZnMyNCBsYXRlXHBhcg0KfQ0K";
rtf, decoded := base64_decode(encoded,, temp);
expect(decoded, "the sample decodes");
text := rtf_to_text(rtf);
expect(text == "late", "a Snagit annotation reads as its text: \"%\"", text);
text = rtf_to_text("{\\rtf1{\\fonttbl{\\f0 Arial;}}{\\*\\generator Riched20;}one\\par two\\tab three caf\\'e9 \\u8364? \\{x\\}}");
expect(text == "one\ntwo\tthree caf\xC3\xA9 \xE2\x82\xAC {x}", "breaks, tabs, \\'hh, \\uN and escaped braces: \"%\"", text);
expect(rtf_to_text("{\\rtf1\\ansi\\'93quoted\\'94}") == "\xE2\x80\x9Cquoted\xE2\x80\x9D", "Windows-1252 quotes");
}
order_tests :: () {
group("order");
expect(viewer_display_order("2026-10-07 08:00:00", "a", "2025-01-01 00:00:00", "b") < 0, "newer first");
expect(viewer_display_order("", "a", "2025-01-01 00:00:00", "b") > 0, "undated after dated");
expect(viewer_display_order("2025-01-01 00:00:00", "a", "", "b") < 0, "dated before undated");
expect(viewer_display_order("2025-01-01 00:00:00", "b", "2025-01-01 00:00:00", "a") < 0, "the same date: by name, newest first");
}
#scope_export
run_catalog_tests :: (builder: *String_Builder = null) -> bool {
output_builder = builder;
emit("=== catalog test suite ===\n");
push_allocator(context.default_allocator);
query_tests();
match_tests();
rtf_tests();
order_tests();
return report();
}

213
src/test/ocr_tests.jai Normal file
View File

@ -0,0 +1,213 @@
// OCR on turned text (modules/OCR): two lines drawn with Windows' GDI, read level, then turned by 15°,
// 90° (read downwards), 180° and 270° (read upwards). Each turn must give back the same text, and the
// line's corners must run the way the text does. Needs the models beside the program (the build copies
// them); without them, or off Windows, the suite says so and passes.
#scope_file
checks_run := 0;
checks_failed := 0;
MAX_GROUPS :: 8;
group_names : [MAX_GROUPS] string;
group_checks : [MAX_GROUPS] int;
group_fails : [MAX_GROUPS] int;
group_count := 0;
output_builder: *String_Builder;
emit :: (format_string: string, args: ..Any) {
if output_builder print_to_builder(output_builder, format_string, ..args);
else print(format_string, ..args);
} @PrintLike
group :: (name: string) {
assert(group_count < MAX_GROUPS, "raise MAX_GROUPS");
group_names[group_count] = name;
group_count += 1;
emit("\n-- % --\n", name);
}
expect :: (condition: bool, format: string, args: ..Any, loc := #caller_location) {
checks_run += 1;
if group_count group_checks[group_count-1] += 1;
if condition return;
checks_failed += 1;
if group_count group_fails[group_count-1] += 1;
emit(" FAIL [%:%] %\n", group_names[max(group_count-1, 0)], loc.line_number, tprint(format, ..args));
}
report :: () -> bool {
emit("\n");
for 0..group_count-1 {
if group_fails[it] emit(" % -- % checks, % FAILED\n", group_names[it], group_checks[it], group_fails[it]);
else emit(" % -- % checks\n", group_names[it], group_checks[it]);
}
if checks_failed emit("\n% OF % OCR CHECKS FAILED\n", checks_failed, checks_run);
else emit("\nALL OCR TESTS PASSED (% checks over % groups)\n", checks_run, group_count);
return checks_failed == 0;
}
LINES :: string.["Assign Footprints to Symbols", "Capacitor 100nF 0402"];
TURNS :: float.[0, 15, 90, 180, 270]; // degrees, clockwise on the screen; 0 is the text as drawn
// Lowercase, without spaces: recognition may split or join words differently once turned.
squeezed :: (s: string) -> string {
builder: String_Builder;
builder.allocator = temp;
for 0..s.count - 1 if s[it] != #char " " append(*builder, to_lower(s[it]));
return builder_to_string(*builder,, temp);
}
// The line found holding wanted, or null.
find_line :: (lines: [] Ocr_Line, wanted: string) -> *Ocr_Line {
for * lines if contains(squeezed(it.text), squeezed(wanted)) return it;
return null;
}
// The image turned clockwise by degrees about its centre onto a white canvas just big enough. Temp.
turned_image :: (source: Ocr_Image, degrees: float) -> Ocr_Image {
radians := degrees * PI / 180;
c, s := cos(radians), sin(radians);
if abs(c) < 0.000001 c = 0;
if abs(s) < 0.000001 s = 0;
result: Ocr_Image;
result.width = cast(s32) ceil(abs(source.width * c) + abs(source.height * s));
result.height = cast(s32) ceil(abs(source.width * s) + abs(source.height * c));
result.stride = result.width * 4;
result.pixels = alloc(cast(s64) result.stride * result.height,, temp);
source_centre := Vector2.{source.width / 2.0, source.height / 2.0};
centre := Vector2.{result.width / 2.0, result.height / 2.0};
for y: 0..result.height - 1 for x: 0..result.width - 1 {
d := Vector2.{x + 0.5, y + 0.5} - centre;
p := Vector2.{c * d.x + s * d.y, -s * d.x + c * d.y} + source_centre - Vector2.{0.5, 0.5}; // turned back
out := result.pixels + cast(s64) y * result.stride + x * 4;
x0, y0 := cast(s32) floor(p.x), cast(s32) floor(p.y);
fx, fy := p.x - x0, p.y - y0;
for channel: 0..3 {
sample :: (image: Ocr_Image, x: s32, y: s32, channel: s64) -> float {
if x < 0 || y < 0 || x >= image.width || y >= image.height return 255;
return cast(float) (image.pixels + cast(s64) y * image.stride + x * 4)[channel];
}
top := sample(source, x0, y0, channel) * (1 - fx) + sample(source, x0 + 1, y0, channel) * fx;
bottom := sample(source, x0, y0 + 1, channel) * (1 - fx) + sample(source, x0 + 1, y0 + 1, channel) * fx;
out[channel] = cast(u8) clamp(top * (1 - fy) + bottom * fy + 0.5, 0, 255);
}
}
return result;
}
#if OS == .WINDOWS {
test_gdi :: #library,system "Gdi32";
TestCreateFontW :: (height: s32, width: s32, escapement: s32, orientation: s32, weight: s32, italic: u32, underline: u32, strike_out: u32, char_set: u32, out_precision: u32, clip_precision: u32, quality: u32, pitch_and_family: u32, face_name: *u16) -> *void #foreign test_gdi "CreateFontW";
TestCreateDIBSection :: (hdc: HDC, info: *BITMAPINFO, usage: u32, bits: **void, section: *void, offset: u32) -> *void #foreign test_gdi "CreateDIBSection";
TestSetTextColor :: (hdc: HDC, color: u32) -> u32 #foreign test_gdi "SetTextColor";
TestSetBkColor :: (hdc: HDC, color: u32) -> u32 #foreign test_gdi "SetBkColor";
TestTextOutW :: (hdc: HDC, x: s32, y: s32, text: *u16, count: s32) -> s32 #foreign test_gdi "TextOutW";
TestGdiFlush :: () -> s32 #foreign test_gdi "GdiFlush";
ANTIALIASED_QUALITY :: 4;
// LINES, black on white, in Segoe UI. Temp.
drawn_lines :: () -> Ocr_Image, ok: bool {
image: Ocr_Image;
image.width, image.height = 720, 170;
image.stride = image.width * 4;
dc := CreateCompatibleDC(null);
if !dc return image, false;
defer DeleteDC(dc);
info: BITMAPINFO;
info.bmiHeader = .{biSize = size_of(BITMAPINFOHEADER), biWidth = image.width, biHeight = -image.height, biPlanes = 1, biBitCount = 32}; // top-down
bits: *void;
bitmap := TestCreateDIBSection(dc, *info, 0, *bits, null, 0);
if !bitmap || !bits return image, false;
defer DeleteObject(bitmap);
font := TestCreateFontW(-34, 0, 0, 0, 400, 0, 0, 0, 1, 0, 0, ANTIALIASED_QUALITY, 0, utf8_to_wide("Segoe UI",, temp));
if !font return image, false;
defer DeleteObject(font);
old_bitmap := SelectObject(dc, bitmap);
old_font := SelectObject(dc, font);
memset(bits, 0xFF, cast(s64) image.stride * image.height);
TestSetTextColor(dc, 0x000000);
TestSetBkColor(dc, 0xFFFFFF);
for LINES {
wide, _, bytes := utf8_to_wide(it,, temp);
TestTextOutW(dc, 40, cast(s32) (30 + it_index * 64), wide, bytes / 2);
}
TestGdiFlush();
// BGRA to RGBA.
image.pixels = alloc(cast(s64) image.stride * image.height,, temp);
source := cast(*u8) bits;
for i: 0..cast(s64) image.width * image.height - 1 {
image.pixels[i * 4] = source[i * 4 + 2];
image.pixels[i * 4 + 1] = source[i * 4 + 1];
image.pixels[i * 4 + 2] = source[i * 4];
image.pixels[i * 4 + 3] = 255;
}
SelectObject(dc, old_font);
SelectObject(dc, old_bitmap);
return image, true;
}
}
turn_tests :: () {
#if OS != .WINDOWS {
emit(" (skipped: the test text is drawn with GDI, on Windows only)\n");
return;
} else {
group("turned text");
folder := ocr_models_folder();
if !file_exists(tprint("%/models.config", folder)) {
emit(" (skipped: no OCR models in %)\n", folder);
return;
}
engine, created, error := ocr_create(folder, OCR_DEFAULT_MODEL);
expect(created, "the OCR engine starts: %", error);
if !created return;
defer ocr_destroy(engine);
expect(engine.orientation.session != null, "% has an orientation classifier", OCR_DEFAULT_MODEL);
emit(" running on %; classifier input % x %\n", ifx engine.provider == .CUDA then "CUDA" else "the CPU", engine.orientation_width, engine.orientation_height);
level, drawn := drawn_lines();
expect(drawn, "the test text is drawn");
if !drawn return;
for degrees: TURNS {
image := ifx degrees == 0 then level else turned_image(level, degrees);
results := ocr_process(engine, Ocr_Image.[image],, temp);
expect(results.count == 1 && !results[0].failed, "% degrees: OCR runs (%)", degrees, ifx results.count then results[0].error else "");
if results.count != 1 || results[0].failed continue;
lines := results[0].lines;
radians := degrees * PI / 180;
direction := Vector2.{cos(radians), sin(radians)}; // the way turned text reads
for wanted: LINES {
line := find_line(lines, wanted);
if !line {
found: [..] string;
found.allocator = temp;
for lines array_add(*found, it.text);
expect(false, "% degrees: \"%\" is read (found %)", degrees, wanted, found);
continue;
}
along := line.corners[1] - line.corners[0];
along_length := length(along);
cosine := ifx along_length > 0 then dot(along, direction) / along_length else 0;
expect(cosine > 0.95, "% degrees: \"%\" has corners that read its way (cosine %, corners %)", degrees, wanted, cosine, line.corners);
}
}
}
}
#scope_export
run_ocr_tests :: (builder: *String_Builder = null) -> bool {
output_builder = builder;
emit("=== OCR test suite ===\n");
push_allocator(context.default_allocator);
turn_tests();
return report();
}

View File

@ -38,6 +38,8 @@ Test_Type :: enum s32 {
fixed_arena_suite; fixed_arena_suite;
secure_memory_suite; secure_memory_suite;
logger_suite; logger_suite;
catalog_suite;
ocr_suite;
} }
testing_started :: () -> bool { testing_started :: () -> bool {
@ -78,6 +80,8 @@ run_all_tests :: () {
add_suite_test(.fixed_arena_suite, "fixed_arena"); add_suite_test(.fixed_arena_suite, "fixed_arena");
add_suite_test(.secure_memory_suite, "secure_memory"); add_suite_test(.secure_memory_suite, "secure_memory");
add_suite_test(.logger_suite, "logger"); add_suite_test(.logger_suite, "logger");
add_suite_test(.catalog_suite, "catalog");
add_suite_test(.ocr_suite, "ocr");
} }
// One work item per suite. Each suite keeps its tallies in file-scope globals, so the whole // One work item per suite. Each suite keeps its tallies in file-scope globals, so the whole
@ -182,6 +186,10 @@ test_thread_group_proc :: (group: *Thread_Group, thread: *Thread, work: *void) -
task.pass = run_secure_memory_tests(task.output); task.pass = run_secure_memory_tests(task.output);
case .logger_suite; case .logger_suite;
task.pass = run_logger_tests(task.output); task.pass = run_logger_tests(task.output);
case .catalog_suite;
task.pass = run_catalog_tests(task.output);
case .ocr_suite;
task.pass = run_ocr_tests(task.output);
} }
return .CONTINUE; return .CONTINUE;

View File

@ -292,6 +292,7 @@ app_theme_set_status :: (message: string, is_error: bool) {
app_user_settings: App_User_Settings; app_user_settings: App_User_Settings;
App_User_Settings :: struct { App_User_Settings :: struct {
theme: string; // "" means the default theme; heap-allocated theme: string; // "" means the default theme; heap-allocated
ocr_model: string; // [ocr] model: the OCR model's name in models.config; "" means the default; heap-allocated
// Of the monitor the window first opens on. // Of the monitor the window first opens on.
window_width_fraction: float = 0.80; window_width_fraction: float = 0.80;
@ -313,6 +314,7 @@ App_User_Settings :: struct {
settings_free :: (s: *App_User_Settings) { settings_free :: (s: *App_User_Settings) {
free(s.theme,, context.default_allocator); free(s.theme,, context.default_allocator);
free(s.ocr_model,, context.default_allocator);
array_free(s.hotkey_bindings); array_free(s.hotkey_bindings);
array_free(s.global_hotkey_bindings); array_free(s.global_hotkey_bindings);
} }
@ -368,11 +370,11 @@ app_load_user_settings_file :: () -> write_back: bool {
if hotkeys_missing.count || global_missing.count return true; if hotkeys_missing.count || global_missing.count return true;
#if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet #if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys); return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model);
} }
Settings_Seen :: struct { Settings_Seen :: struct {
theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys: bool; theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys, ocr_model: bool;
} }
// Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an // Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an
@ -382,6 +384,7 @@ settings_file_hash: u64;
// The settings the app itself changes, as it holds them now. // The settings the app itself changes, as it holds them now.
Settings_App_Values :: struct { Settings_App_Values :: struct {
theme: string; theme: string;
ocr_model: string;
font_size_base: float; font_size_base: float;
font_scale_main: float; font_scale_main: float;
} }
@ -395,6 +398,7 @@ settings_app_values :: () -> Settings_App_Values {
entry := app_theme_current(); entry := app_theme_current();
values.theme = ifx entry then entry.name else app_user_settings.theme; values.theme = ifx entry then entry.name else app_user_settings.theme;
if !values.theme.count values.theme = APP_DEFAULT_THEME_NAME; if !values.theme.count values.theme = APP_DEFAULT_THEME_NAME;
values.ocr_model = ifx app_user_settings.ocr_model.count then app_user_settings.ocr_model else OCR_DEFAULT_MODEL;
values.font_size_base = app_user_settings.font_size_base; values.font_size_base = app_user_settings.font_size_base;
values.font_scale_main = app_user_settings.font_scale_main; values.font_scale_main = app_user_settings.font_scale_main;
@ -410,8 +414,10 @@ settings_app_values :: () -> Settings_App_Values {
settings_note_synced :: () { settings_note_synced :: () {
free(settings_synced.theme,, context.default_allocator); free(settings_synced.theme,, context.default_allocator);
free(settings_synced.ocr_model,, context.default_allocator);
settings_synced = settings_app_values(); settings_synced = settings_app_values();
settings_synced.theme = copy_string(settings_synced.theme,, context.default_allocator); settings_synced.theme = copy_string(settings_synced.theme,, context.default_allocator);
settings_synced.ocr_model = copy_string(settings_synced.ocr_model,, context.default_allocator);
} }
// Parses the file's contents into out, replacing its theme and hotkey bindings. ok is false when // Parses the file's contents into out, replacing its theme and hotkey bindings. ok is false when
@ -483,7 +489,11 @@ settings_parse :: (path: string, file_data: string, out: *App_User_Settings) ->
key, value, split_ok := theme_split_key(body); key, value, split_ok := theme_split_key(body);
if !split_ok continue; if !split_ok continue;
if section == "[ui]" && key == "theme" { if section == "[ocr]" && key == "model" {
free(out.ocr_model,, context.default_allocator);
out.ocr_model = copy_string(settings_unquote(value),, context.default_allocator);
seen.ocr_model = true;
} else if section == "[ui]" && key == "theme" {
free(out.theme,, context.default_allocator); free(out.theme,, context.default_allocator);
out.theme = copy_string(settings_unquote(value),, context.default_allocator); out.theme = copy_string(settings_unquote(value),, context.default_allocator);
seen.theme = true; seen.theme = true;
@ -592,6 +602,7 @@ app_save_user_settings_file :: () -> bool {
values := settings_app_values(); values := settings_app_values();
theme := values.theme; theme := values.theme;
ocr_model := values.ocr_model;
font_size_base := values.font_size_base; font_size_base := values.font_size_base;
font_scale_main := values.font_scale_main; font_scale_main := values.font_scale_main;
@ -621,6 +632,7 @@ app_save_user_settings_file :: () -> bool {
} }
if disk_ok { if disk_ok {
if seen.theme && values.theme == settings_synced.theme && on_disk.theme.count theme = on_disk.theme; if seen.theme && values.theme == settings_synced.theme && on_disk.theme.count theme = on_disk.theme;
if seen.ocr_model && values.ocr_model == settings_synced.ocr_model && on_disk.ocr_model.count ocr_model = on_disk.ocr_model;
if seen.font_size && values.font_size_base == settings_synced.font_size_base font_size_base = on_disk.font_size_base; if seen.font_size && values.font_size_base == settings_synced.font_size_base font_size_base = on_disk.font_size_base;
if seen.font_scale && values.font_scale_main == settings_synced.font_scale_main font_scale_main = on_disk.font_scale_main; if seen.font_scale && values.font_scale_main == settings_synced.font_scale_main font_scale_main = on_disk.font_scale_main;
if seen.width app_user_settings.window_width_fraction = on_disk.window_width_fraction; if seen.width app_user_settings.window_width_fraction = on_disk.window_width_fraction;
@ -643,6 +655,9 @@ app_save_user_settings_file :: () -> bool {
append(*builder, "# Text size is size_base * scale_main * the monitor's DPI scale. The font size hotkeys change scale_main.\n"); append(*builder, "# Text size is size_base * scale_main * the monitor's DPI scale. The font size hotkeys change scale_main.\n");
print_to_builder(*builder, "size_base: %\n", FF(font_size_base, 1)); print_to_builder(*builder, "size_base: %\n", FF(font_size_base, 1));
print_to_builder(*builder, "scale_main: %\n\n", FF(font_scale_main, 2)); print_to_builder(*builder, "scale_main: %\n\n", FF(font_scale_main, 2));
append(*builder, "[ocr]\n");
append(*builder, "# The OCR model, by its name in ocr-models/models.config: ppocr-v5-mobile (light) or ppocr-v5-server (heavy).\n");
print_to_builder(*builder, "model: %\n\n", settings_quote_if_needed(ocr_model));
append(*builder, "[logging]\n"); append(*builder, "[logging]\n");
append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n"); append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n");
print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity); print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity);
@ -766,6 +781,7 @@ app_reload_user_settings_if_changed :: () {
} }
if fresh.event_log_capacity != old.event_log_capacity logger_set_event_log_capacity(fresh.event_log_capacity); if fresh.event_log_capacity != old.event_log_capacity logger_set_event_log_capacity(fresh.event_log_capacity);
if fresh.ocr_model != old.ocr_model ocr_ui_settings_changed(); // app/ocr_ui.jai
bindings: [] Hotkey_Binding = hotkey_defaults(); bindings: [] Hotkey_Binding = hotkey_defaults();
if seen.hotkeys bindings = app_user_settings.hotkey_bindings; if seen.hotkeys bindings = app_user_settings.hotkey_bindings;

View File

@ -124,7 +124,8 @@ main :: () {
print(" first image alone: % s (warm-up)\n", FF(first_seconds, 3)); print(" first image alone: % s (warm-up)\n", FF(first_seconds, 3));
print(" all % images: % s, % ms per image, % megapixels/s, % failed\n", images.count, FF(seconds, 2), FF(1000 * seconds / images.count, 1), FF(megapixels / seconds, 2), failures); print(" all % images: % s, % ms per image, % megapixels/s, % failed\n", images.count, FF(seconds, 2), FF(1000 * seconds / images.count, 1), FF(megapixels / seconds, 2), failures);
print(" found: % lines, % bytes of text\n", lines, characters); print(" found: % lines, % bytes of text\n", lines, characters);
print(" stage time (summed over threads): detection % s, recognition % s\n", FF(stats.detect_us / 1_000_000.0, 2), FF(stats.recognize_us / 1_000_000.0, 2)); print(" stage time (summed over threads): detection % s, orientation % s, recognition % s\n", FF(stats.detect_us / 1_000_000.0, 2), FF(stats.orientation_us / 1_000_000.0, 2), FF(stats.recognize_us / 1_000_000.0, 2));
print(" lines read better turned: %\n", stats.turned);
print(" text in %\n\n", text_folder); print(" text in %\n\n", text_folder);
ocr_destroy(engine); ocr_destroy(engine);
} }