[0.0.0] .mcap version 2: OCR stored in each capture, search, a library scan, sorting by date; turned text
This commit is contained in:
parent
bb94f03b94
commit
ea9f055abc
19
README.md
19
README.md
@ -29,6 +29,25 @@ and everything around them, ready for an app to be written into `src/app/app_ui.
|
||||
progress, so it never looks hung.
|
||||
- **Asynchronous tests**, compiled into debug builds and run with `run_tests`.
|
||||
|
||||
## The capture viewer
|
||||
|
||||
`src/app/viewer.jai`, `catalog.jai` and `ocr_ui.jai`: every `.mcap` in `D:\SnagitIndex\mcap` as a grid
|
||||
of previews, and one capture at full size in the Capture window.
|
||||
|
||||
- **Search** above the grid: every word must match, anywhere in the OCR text, window title, URL, app
|
||||
name or annotation text; `"a phrase"` matches as one; `ocr:`, `title:`, `url:`, `app:` and `note:`
|
||||
limit a word to one field. In the Capture window the matching OCR lines are highlighted, with Next
|
||||
and Previous.
|
||||
- **Order:** newest first by capture date, once the catalog (what search needs from every file) has
|
||||
loaded; by name until then.
|
||||
- **OCR** (`modules/OCR`): the OCR window shows whether it runs on the GPU and, if not, what to install,
|
||||
and switches model (kept as `model` under `[ocr]` in the settings file). **Scan the library** reads
|
||||
every capture shown that the model has not read yet, storing each reading in its file as it goes,
|
||||
so it can pause, cancel and pick up again. `-library FOLDER` shows another folder of `.mcap` files;
|
||||
`-scan` scans it with the chosen model once it has loaded and exits (0, or 1 if any capture
|
||||
failed), for a scan without the UI, such as on a copy. The Capture window's **OCR** button reads one capture;
|
||||
a capture read by several models keeps each reading, and the window can show any of them.
|
||||
|
||||
## Building
|
||||
|
||||
Requires the Jai compiler, beta 0.2.030 or later.
|
||||
|
||||
93
modules/Capture_File/README.md
Normal file
93
modules/Capture_File/README.md
Normal file
@ -0,0 +1,93 @@
|
||||
# The .mcap file
|
||||
|
||||
One capture in one file: a fixed magic, a file version stamp, the capture's data as
|
||||
[Capture Text](capture_text.md), what OCR read in it, a small preview, then the images. There is no ZIP
|
||||
and no compression around it; the images keep their own PNG compression.
|
||||
|
||||
The parts a library browser needs (header, text, OCR and preview) come first, so it reads the start of
|
||||
each file and never touches the images; a search, which needs no preview, reads less still.
|
||||
|
||||
## Layout, version 2
|
||||
|
||||
All integers are little-endian. Sizes count bytes.
|
||||
|
||||
| Field | Type | |
|
||||
| --- | --- | --- |
|
||||
| magic | 8 bytes | `89 4D 43 41 50 0D 0A 1A`, which is `\x89MCAP\r\n\x1A`. Says this is an .mcap, in every version. |
|
||||
| file version | u32 | `2`. Version 1 files are still read; see below. |
|
||||
| text size, text | u32, bytes | Capture Text, UTF-8: every JSON file of the capture, as `[[file]]` sections. |
|
||||
| OCR size, OCR | u32, bytes | OCR text, UTF-8 (below): what each OCR model read. Size 0 when none has. |
|
||||
| preview width, height | u16, u16 | In pixels. |
|
||||
| preview size, preview | u32, bytes | PNG. Size 0 when there is none. |
|
||||
| original name size, name | u16, bytes | The original image's name in the .snagx, such as `{GUID}.png`; the page's `CaptureBackgroundImage` names it too. |
|
||||
| original size, original | u32, bytes | The capture as taken: PNG, exactly as it was in the .snagx. |
|
||||
| rendered kind | u8 | 0: none. 1: the same as the original (stored once). 2: its own, below. |
|
||||
| rendered size, rendered | u32, bytes | Only for kind 2. The capture with its annotations drawn in (Snagit's `thumbnail.png`): PNG, exactly as it was. |
|
||||
|
||||
The file ends there: anything after the last part is a mistake.
|
||||
|
||||
The magic borrows PNG's: the first byte has its high bit set, so a channel that strips it shows; `\r\n`
|
||||
shows line endings being converted; and `\x1A` stops `type` on Windows.
|
||||
|
||||
**Version 1** is the same without the OCR size and OCR. A reader takes both; anything written is
|
||||
version 2, so a version 1 file becomes version 2 the first time OCR is stored in it.
|
||||
|
||||
## The parts
|
||||
|
||||
- **Text.** Everything the capture's JSON held: `index.json`, `metadata.json`, the page's `{GUID}.json`,
|
||||
and `{GUID}.backup.json` when there is one.
|
||||
- **OCR.** One section per OCR model, so a capture read by several models keeps every reading;
|
||||
reading it again with a model replaces only that model's section. See *OCR text* below.
|
||||
- **Preview.** The rendered image (or the original, when there is no rendered one) scaled so its longest
|
||||
side is at most 320 pixels, as a PNG. An image already that small is used as it is, never scaled up.
|
||||
- **Original and rendered.** A capture with no annotations has a rendered image identical to the original,
|
||||
byte for byte; it is then stored once, as kind 1.
|
||||
- **One page.** An .mcap holds one page, as every capture so far does; a .snagx with more is not converted.
|
||||
- **Left out:** `{GUID}.backup.png`, Snagit's image from before an edit. Its `{GUID}.backup.json` is kept,
|
||||
as plenty of captures have one without the image.
|
||||
|
||||
## Versions
|
||||
|
||||
A reader checks the magic, then the file version. It refuses a version above its own, never writing
|
||||
over a file from a later version, and reads every version up to its own. Version 0 does not exist.
|
||||
|
||||
## Converting
|
||||
|
||||
`snagit-converter mcap` converts .snagx files (or extracted captures) to .mcap, and `snagit-converter
|
||||
snagx` converts .mcap files back to a .snagx Snagit can open. See the README for what each checks.
|
||||
|
||||
Back in a .snagx, the JSON is laid out as Snagit writes it, the images are the same bytes, and only the
|
||||
backup image is missing. JSON numbers come back as values: `3.0` is written `3`.
|
||||
|
||||
## OCR text
|
||||
|
||||
In the style of the Ex4 text formats:
|
||||
|
||||
```
|
||||
@version 2
|
||||
|
||||
[ocr ppocr-v5-mobile]
|
||||
engine: "onnxruntime 1.30.0, CUDA"
|
||||
image: rendered 1920 1080
|
||||
date: 2026-10-08T21:04:11Z
|
||||
line: 412 96 640 18 412 96 1052 96 1052 114 412 114 0.993 "Assign Footprints - KiCad"
|
||||
line: 20 140 220 14 20 140 240 140 240 154 20 154 0.981 "Symbol : Footprint Assignments"
|
||||
line: 1700 300 20 160 1720 300 1720 460 1700 460 1700 300 0.952 "Read downwards"
|
||||
```
|
||||
|
||||
| Key | Value |
|
||||
| --- | --- |
|
||||
| `[ocr NAME]` | A section: the model's name, as in its `models.config`. |
|
||||
| `engine` | What ran it, quoted. |
|
||||
| `image` | Which image was read (`rendered` or `original`) and its width and height; the lines' boxes are in its pixels. |
|
||||
| `date` | When it was read, UTC. |
|
||||
| `line` | A text line: x, y, width and height, then its four corners as x y pairs, its confidence (0 to 1), and its text, quoted with `""` for a quote. A control character in the text is written as a space. |
|
||||
|
||||
The corners go clockwise from the text's top left: top left, top right, bottom right, bottom left, as
|
||||
the text reads. Level text has the box's corners; a turned line's say which way it reads (the third
|
||||
line above reads downwards, so its top left is the box's top right). x, y, width and height bound them.
|
||||
|
||||
A line that cannot be read is skipped, as are keys this version does not know. **Version 1** OCR text
|
||||
had no corners (`line: x y width height confidence "text"`); it is still read, with the box's corners.
|
||||
The OCR text carries its own version, so this did not change the .mcap file version. `mcap_store_ocr`
|
||||
(`modules/Capture_File/ocr_text.jai`) stores a section in a file.
|
||||
@ -1,10 +1,13 @@
|
||||
// .mcap: one capture in one file. A fixed magic, a file version stamp, the capture's Capture Text,
|
||||
// a small preview, then the images. See docs/mcap.md for the layout.
|
||||
// its OCR text (version 2), a small preview, then the images. See docs/mcap.md for the layout.
|
||||
//
|
||||
// Reading gives views into the bytes it was given; nothing is copied. Images stay PNG.
|
||||
//
|
||||
// The main copy is mcap's (D:/jai_projects/active/mcap/modules/Capture_File); SnagitConverter has
|
||||
// the same files.
|
||||
|
||||
MCAP_MAGIC :: u8.[0x89, #char "M", #char "C", #char "A", #char "P", 0x0D, 0x0A, 0x1A];
|
||||
MCAP_VERSION :: 1;
|
||||
MCAP_VERSION :: 2; // what is written; every version from 1 up is read
|
||||
MCAP_EXTENSION :: "mcap";
|
||||
MCAP_PREVIEW_MAX_SIDE :: 320; // the preview's longest side, in pixels
|
||||
|
||||
@ -23,6 +26,7 @@ Mcap_Image :: struct {
|
||||
Mcap :: struct {
|
||||
version: u32 = MCAP_VERSION; // as read; always MCAP_VERSION when written
|
||||
text: string; // Capture Text
|
||||
ocr: string; // OCR text (ocr_text.jai), one section per model; "" when none, and always in version 1
|
||||
preview: Mcap_Image;
|
||||
original_name: string; // the original image's name in the .snagx, such as {GUID}.png
|
||||
original: [] u8; // PNG
|
||||
@ -44,7 +48,7 @@ mcap_rendered :: (m: Mcap) -> [] u8 {
|
||||
// The file's bytes. A part too large for its size field fails the whole file rather than writing one
|
||||
// that cannot be read back.
|
||||
mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string {
|
||||
size := MCAP_HEADER_SIZE + 4 + m.text.count + 8 + m.preview.png.count + 2 + m.original_name.count + 4 + m.original.count + 1;
|
||||
size := MCAP_HEADER_SIZE + 4 + m.text.count + 4 + m.ocr.count + 8 + m.preview.png.count + 2 + m.original_name.count + 4 + m.original.count + 1;
|
||||
if m.rendered_kind == .OWN size += 4 + m.rendered.count;
|
||||
s := new_serializer(size);
|
||||
|
||||
@ -53,6 +57,7 @@ mcap_serialize :: (m: Mcap) -> Serializer, ok: bool, error: string {
|
||||
Add(*s, cast(u32) MCAP_VERSION);
|
||||
|
||||
if !AddString(*s, m.text, u32) return s, false, "the text is too large for its size field";
|
||||
if !AddString(*s, m.ocr, u32) return s, false, "the OCR text is too large for its size field";
|
||||
Add(*s, m.preview.width);
|
||||
Add(*s, m.preview.height);
|
||||
if !AddSizedArray(*s, m.preview.png, u32) return s, false, "the preview is too large for its size field";
|
||||
@ -95,10 +100,10 @@ mcap_parse :: (data: [] u8) -> Mcap, ok: bool, error: string {
|
||||
return m, true, "";
|
||||
}
|
||||
|
||||
// Only the header, text and preview, as a library browser needs them: one read of the start of the
|
||||
// file, and at most two more when the text or preview runs past it. The images are left unread and
|
||||
// empty. The parts are views into a buffer from the context allocator.
|
||||
mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string {
|
||||
// Only the front of the file, as a library browser needs it: the header, text, OCR text and, with
|
||||
// preview, the preview. One read of the start of the file, and a few more only when a part runs past
|
||||
// it. The images are left unread and empty. The parts are views into a buffer from the context allocator.
|
||||
mcap_read_front :: (path: string, preview := true) -> Mcap, ok: bool, error: string {
|
||||
m: Mcap;
|
||||
file, opened := file_open(path, log_errors = false);
|
||||
if !opened return m, false, "cannot be opened";
|
||||
@ -107,17 +112,17 @@ mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string {
|
||||
if !length_ok return m, false, "cannot be read";
|
||||
|
||||
wanted := min(length, MCAP_FRONT_READ_SIZE);
|
||||
for attempt: 1..3 {
|
||||
for attempt: 1..5 { // each further read reaches at least one more size field
|
||||
buffer := NewArray(wanted, u8, initialized = false);
|
||||
if !file_read(file, buffer.data, wanted) { array_free(buffer); return m, false, "cannot be read"; }
|
||||
|
||||
d := to_deserializer(buffer);
|
||||
ok, error := mcap_read_front_part(*d, *m);
|
||||
ok, error := mcap_read_front_part(*d, *m, preview);
|
||||
if ok return m, true, "";
|
||||
if d.ok || wanted == length { array_free(buffer); return m, false, error; } // not a matter of too few bytes
|
||||
|
||||
// Short: the sizes read so far say how much is needed.
|
||||
needed := mcap_front_size(buffer);
|
||||
needed := mcap_front_size(buffer, preview);
|
||||
array_free(buffer);
|
||||
if needed <= wanted || needed > length return m, false, "the file ends early (it is truncated or damaged)";
|
||||
wanted = needed;
|
||||
@ -131,8 +136,8 @@ mcap_read_front :: (path: string) -> Mcap, ok: bool, error: string {
|
||||
MCAP_HEADER_SIZE :: 12; // magic and version
|
||||
MCAP_FRONT_READ_SIZE :: 64 * 1024; // holds the header, text and preview of nearly every capture
|
||||
|
||||
// Header, text and preview.
|
||||
mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string {
|
||||
// Header, text, OCR text and, with preview, the preview.
|
||||
mcap_read_front_part :: (d: *Deserializer, m: *Mcap, preview := true) -> ok: bool, error: string {
|
||||
magic := ReadArrayView(d, MCAP_MAGIC.count, u8);
|
||||
if !d.ok return false, "is not an .mcap file (it is shorter than the magic)";
|
||||
for MCAP_MAGIC if magic[it_index] != it return false, "is not an .mcap file (its magic is wrong)";
|
||||
@ -142,6 +147,11 @@ mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string
|
||||
if m.version > MCAP_VERSION return false, tprint("is file version %, but this build reads up to %; it was written by a later version", m.version, MCAP_VERSION);
|
||||
|
||||
m.text = ReadSizedString(d, u32);
|
||||
if m.version >= 2 m.ocr = ReadSizedString(d, u32); // version 1 has no OCR block
|
||||
if !preview {
|
||||
if !d.ok return false, "the file ends early (it is truncated or damaged)";
|
||||
return true, "";
|
||||
}
|
||||
m.preview.width = Read(d, u16);
|
||||
m.preview.height = Read(d, u16);
|
||||
m.preview.png = ReadSizedArray(d, u8, u32);
|
||||
@ -149,18 +159,31 @@ mcap_read_front_part :: (d: *Deserializer, m: *Mcap) -> ok: bool, error: string
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// How many bytes the header, text and preview take, as far as the sizes in prefix tell: with only the
|
||||
// text's size in it, enough to reach the preview's size; with that too, the whole front.
|
||||
mcap_front_size :: (prefix: [] u8) -> s64 {
|
||||
// How many bytes the front takes, as far as the size fields in prefix tell: the whole front when
|
||||
// they are all in it, otherwise how far to read to reach the next size field.
|
||||
mcap_front_size :: (prefix: [] u8, preview := true) -> s64 {
|
||||
d := to_deserializer(prefix);
|
||||
d.cursor = MCAP_HEADER_SIZE;
|
||||
text_size := cast(s64) Read(*d, u32);
|
||||
preview_header := MCAP_HEADER_SIZE + 4 + text_size + 4; // where the preview's size is
|
||||
if !d.ok || preview_header + 4 > prefix.count return preview_header + 4;
|
||||
d.cursor = preview_header;
|
||||
return preview_header + 4 + cast(s64) Read(*d, u32);
|
||||
d.cursor = MCAP_MAGIC.count;
|
||||
version := Read(*d, u32);
|
||||
if !d.ok return MCAP_HEADER_SIZE;
|
||||
position: s64 = MCAP_HEADER_SIZE;
|
||||
skip_sized :: (prefix: [] u8, position: *s64) -> reached: bool { // a u32 size and what it counts
|
||||
if position.* + 4 > prefix.count { position.* += 4; return false; }
|
||||
d := to_deserializer(prefix);
|
||||
d.cursor = position.*;
|
||||
position.* += 4 + cast(s64) Read(*d, u32);
|
||||
return true;
|
||||
}
|
||||
if !skip_sized(prefix, *position) return position; // the text
|
||||
if version >= 2 && !skip_sized(prefix, *position) return position; // the OCR text
|
||||
if !preview return position;
|
||||
position += 4; // the preview's width and height
|
||||
skip_sized(prefix, *position); // the preview
|
||||
return position;
|
||||
}
|
||||
|
||||
#load "ocr_text.jai";
|
||||
|
||||
#import "Basic";
|
||||
#import "File";
|
||||
#import "File_Utilities";
|
||||
|
||||
209
modules/Capture_File/ocr_text.jai
Normal file
209
modules/Capture_File/ocr_text.jai
Normal file
@ -0,0 +1,209 @@
|
||||
// OCR text: the OCR block of an .mcap (version 2), in the style of the Ex4 text formats. One section
|
||||
// per model, so a capture read by several models keeps every reading; reading it again with a model
|
||||
// replaces only that model's section.
|
||||
//
|
||||
// @version 2
|
||||
// [ocr ppocr-v5-mobile]
|
||||
// engine: "onnxruntime 1.30.0, CUDA"
|
||||
// image: rendered 1920 1080
|
||||
// date: 2026-10-08T21:04:11Z
|
||||
// line: 412 96 640 18 412 96 1052 96 1052 114 412 114 0.993 "Assign Footprints - KiCad"
|
||||
//
|
||||
// line: x y width height, the line's four corners as x y pairs, confidence, "text". All in the pixels
|
||||
// of the image named by image: (rendered or original, at that size). x y width height bound the
|
||||
// corners; the corners go clockwise from the text's top left, so a turned line says which way it
|
||||
// reads. Text is quoted, with "" for a quote; a control character is written as a space. A line that
|
||||
// cannot be read is skipped; keys this version does not know are skipped too.
|
||||
//
|
||||
// Version 1 had no corners (line: x y width height confidence "text"); it is still read, its corners
|
||||
// taken from its box.
|
||||
|
||||
MCAP_OCR_TEXT_VERSION :: 2;
|
||||
|
||||
Mcap_Ocr_Line :: struct {
|
||||
x, y, width, height: s32;
|
||||
confidence: float;
|
||||
text: string;
|
||||
corners: [8] s32; // x0 y0 x1 y1 x2 y2 x3 y3: top left, top right, bottom right, bottom left of the text; all 0 means the box's
|
||||
}
|
||||
|
||||
// The line's corners: its own, or its box's when it has none.
|
||||
mcap_ocr_corners :: (line: Mcap_Ocr_Line) -> [8] s32 {
|
||||
for line.corners if it return line.corners;
|
||||
x0, y0, x1, y1 := line.x, line.y, line.x + line.width, line.y + line.height;
|
||||
return .[x0, y0, x1, y0, x1, y1, x0, y1];
|
||||
}
|
||||
|
||||
Mcap_Ocr_Section :: struct {
|
||||
model: string; // the OCR model's name, as in its models.config
|
||||
engine: string; // what ran it, such as "onnxruntime 1.30.0, CUDA"
|
||||
image: string; // "rendered" or "original"
|
||||
image_width, image_height: s32;
|
||||
date: string; // when it was read, UTC: 2026-10-08T21:04:11Z
|
||||
lines: [..] Mcap_Ocr_Line;
|
||||
}
|
||||
|
||||
// The sections of an OCR block; strings and arrays from the context allocator. "" gives none.
|
||||
mcap_ocr_parse :: (text: string) -> [..] Mcap_Ocr_Section, ok: bool, error: string {
|
||||
sections: [..] Mcap_Ocr_Section;
|
||||
if !text.count return sections, true, "";
|
||||
section: *Mcap_Ocr_Section;
|
||||
version_seen := false;
|
||||
version: s64;
|
||||
for raw: split(text, "\n",, temp) {
|
||||
line := trim(raw);
|
||||
if !line.count || line[0] == #char "#" continue;
|
||||
if begins_with(line, "@version") {
|
||||
rest := trim(advance(line, 8));
|
||||
number, ok := string_to_int(rest);
|
||||
if !ok || number < 1 || number > MCAP_OCR_TEXT_VERSION return sections, false, tprint("its OCR text is version %, but this build reads up to %", rest, MCAP_OCR_TEXT_VERSION);
|
||||
version = number;
|
||||
version_seen = true;
|
||||
continue;
|
||||
}
|
||||
if !version_seen return sections, false, "its OCR text does not start with @version";
|
||||
|
||||
if begins_with(line, "[ocr ") && ends_with(line, "]") {
|
||||
section = array_add(*sections);
|
||||
section.* = .{};
|
||||
section.model = copy_string(trim(slice(line, 5, line.count - 6)));
|
||||
continue;
|
||||
}
|
||||
colon := find_index_from_left(line, #char ":");
|
||||
if colon < 0 || !section continue;
|
||||
key := trim(slice(line, 0, colon));
|
||||
value := trim(advance(line, colon + 1));
|
||||
if key == {
|
||||
case "engine";
|
||||
quoted, _, ok := ocr_read_quoted(value);
|
||||
if ok section.engine = copy_string(quoted);
|
||||
case "image";
|
||||
parts := split(value, " ",, temp);
|
||||
if parts.count == 3 {
|
||||
section.image = copy_string(parts[0]);
|
||||
section.image_width = cast(s32) string_to_int(parts[1]);
|
||||
section.image_height = cast(s32) string_to_int(parts[2]);
|
||||
}
|
||||
case "date";
|
||||
section.date = copy_string(value);
|
||||
case "line";
|
||||
ocr_line, ok := ocr_parse_line(value, version);
|
||||
if ok array_add(*section.lines, ocr_line);
|
||||
}
|
||||
}
|
||||
return sections, true, "";
|
||||
}
|
||||
|
||||
// The OCR block for sections. Temp.
|
||||
mcap_ocr_text :: (sections: [] Mcap_Ocr_Section) -> string {
|
||||
if !sections.count return "";
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
print_to_builder(*builder, "@version %\n", MCAP_OCR_TEXT_VERSION);
|
||||
for sections {
|
||||
print_to_builder(*builder, "\n[ocr %]\n", it.model);
|
||||
print_to_builder(*builder, "engine: %\n", ocr_quote(it.engine));
|
||||
print_to_builder(*builder, "image: % % %\n", it.image, it.image_width, it.image_height);
|
||||
print_to_builder(*builder, "date: %\n", it.date);
|
||||
for line: it.lines {
|
||||
c := mcap_ocr_corners(line);
|
||||
print_to_builder(*builder, "line: % % % % % % % % % % % % % %\n", line.x, line.y, line.width, line.height,
|
||||
c[0], c[1], c[2], c[3], c[4], c[5], c[6], c[7], formatFloat(line.confidence, trailing_width = 3, zero_removal = .NO), ocr_quote(line.text));
|
||||
}
|
||||
}
|
||||
return builder_to_string(*builder,, temp);
|
||||
}
|
||||
|
||||
// Stores section in the .mcap at path, in place of any section by the same model, keeping the
|
||||
// others, and upgrades the file to the current version. The file is rewritten beside itself and
|
||||
// swapped in (mcap_write), so a failure leaves it as it was.
|
||||
mcap_store_ocr :: (path: string, section: Mcap_Ocr_Section) -> ok: bool, error: string {
|
||||
data, read_ok := read_entire_file(path, log_errors = false);
|
||||
if !read_ok return false, "cannot be read";
|
||||
defer free(data);
|
||||
m, parsed, parse_error := mcap_parse(cast([] u8) data);
|
||||
if !parsed return false, parse_error;
|
||||
|
||||
push_allocator(temp);
|
||||
sections, ok, error := mcap_ocr_parse(m.ocr);
|
||||
if !ok return false, error;
|
||||
replaced := false;
|
||||
for * sections if it.model == section.model { it.* = section; replaced = true; break; }
|
||||
if !replaced array_add(*sections, section);
|
||||
m.ocr = mcap_ocr_text(sections);
|
||||
m.version = MCAP_VERSION;
|
||||
|
||||
serializer, serialized, serialize_error := mcap_serialize(m);
|
||||
if !serialized return false, serialize_error;
|
||||
written, write_error := mcap_write(path, serializer);
|
||||
return written, write_error;
|
||||
}
|
||||
|
||||
// The time now as the date: key writes it, UTC. Temp.
|
||||
mcap_ocr_now :: () -> string {
|
||||
t := to_calendar(current_time_consensus(), .UTC);
|
||||
two :: (n: s64) -> string { return tprint("%", formatInt(n, minimum_digits = 2)); }
|
||||
return tprint("%-%-%T%:%:%Z", t.year, two(t.month_starting_at_0 + 1), two(t.day_of_month_starting_at_0 + 1), two(t.hour), two(t.minute), two(t.second));
|
||||
}
|
||||
|
||||
#scope_file
|
||||
|
||||
// Quoted, a quote written twice, a control character as a space. Temp.
|
||||
ocr_quote :: (text: string) -> string {
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
append(*builder, #char "\"");
|
||||
for 0..text.count - 1 {
|
||||
c := text[it];
|
||||
if c == #char "\"" append(*builder, "\"\"");
|
||||
else if c < 0x20 || c == 0x7F append(*builder, #char " ");
|
||||
else append(*builder, c);
|
||||
}
|
||||
append(*builder, #char "\"");
|
||||
return builder_to_string(*builder,, temp);
|
||||
}
|
||||
|
||||
// One quoted string ("" for a quote) at the start of s: its text (context allocator) and what follows.
|
||||
ocr_read_quoted :: (s: string) -> text: string, rest: string, ok: bool {
|
||||
if !s.count || s[0] != #char "\"" return "", s, false;
|
||||
builder: String_Builder;
|
||||
i := 1;
|
||||
while i < s.count {
|
||||
if s[i] == #char "\"" {
|
||||
if i + 1 < s.count && s[i + 1] == #char "\"" { append(*builder, #char "\""); i += 2; continue; }
|
||||
return builder_to_string(*builder), advance(s, i + 1), true;
|
||||
}
|
||||
append(*builder, s[i]);
|
||||
i += 1;
|
||||
}
|
||||
free_buffers(*builder);
|
||||
return "", s, false;
|
||||
}
|
||||
|
||||
// Version 2: x y width height x0 y0 x1 y1 x2 y2 x3 y3 confidence "text". Version 1: x y width height confidence "text".
|
||||
ocr_parse_line :: (value: string, version: s64) -> Mcap_Ocr_Line, bool {
|
||||
result: Mcap_Ocr_Line;
|
||||
rest := value;
|
||||
all_numbers: [13] float64;
|
||||
numbers := array_view(all_numbers, 0, ifx version >= 2 then 13 else 5);
|
||||
for * numbers {
|
||||
rest = trim_left(rest);
|
||||
space := find_index_from_left(rest, #char " ");
|
||||
if space < 0 return result, false;
|
||||
number, ok := string_to_float64(slice(rest, 0, space));
|
||||
if !ok return result, false;
|
||||
it.* = number;
|
||||
rest = advance(rest, space);
|
||||
}
|
||||
text, after, ok := ocr_read_quoted(trim_left(rest));
|
||||
if !ok || trim(after).count return result, false;
|
||||
result.x, result.y = cast(s32) numbers[0], cast(s32) numbers[1];
|
||||
result.width, result.height = cast(s32) numbers[2], cast(s32) numbers[3];
|
||||
result.confidence = cast(float) numbers[numbers.count - 1];
|
||||
result.text = text;
|
||||
if numbers.count == 13 for 0..7 result.corners[it] = cast(s32) numbers[4 + it];
|
||||
result.corners = mcap_ocr_corners(result);
|
||||
return result, true;
|
||||
}
|
||||
|
||||
#import "String";
|
||||
746
modules/Capture_Text/module.jai
Normal file
746
modules/Capture_Text/module.jai
Normal file
@ -0,0 +1,746 @@
|
||||
// Capture Text: the text format for a capture's structured data, the JSON files of a .snagx in
|
||||
// the style of the Ex4 text formats. See docs/capture_text.md for the format itself.
|
||||
//
|
||||
// A Capture_Document allocates everything through the allocator it is given, and frees nothing on
|
||||
// its own: give it an arena or a pool, and drop that to drop the document.
|
||||
|
||||
CAPTURE_TEXT_VERSION :: 1;
|
||||
CAPTURE_TEXT_EXTENSION :: "capture-text";
|
||||
|
||||
Capture_Value_Kind :: enum u8 { NULL; BOOLEAN; NUMBER; STRING; ARRAY; OBJECT; }
|
||||
|
||||
Capture_Value :: struct {
|
||||
kind: Capture_Value_Kind;
|
||||
boolean: bool;
|
||||
number: float64;
|
||||
text: string;
|
||||
items: [..] Capture_Value; // ARRAY
|
||||
fields: [..] Capture_Field; // OBJECT, in the order they were added
|
||||
}
|
||||
|
||||
Capture_Field :: struct {
|
||||
key: string;
|
||||
value: Capture_Value;
|
||||
}
|
||||
|
||||
// One [[file "NAME"]] section: one of the capture's JSON files.
|
||||
Capture_File :: struct {
|
||||
name: string;
|
||||
root: Capture_Value; // always an OBJECT
|
||||
}
|
||||
|
||||
Capture_Document :: struct {
|
||||
allocator: Allocator;
|
||||
files: [..] Capture_File;
|
||||
comment: string; // written as # lines under @version; not read back
|
||||
mistakes: s64; // lines skipped by the last parse
|
||||
}
|
||||
|
||||
capture_document_init :: (doc: *Capture_Document, allocator: Allocator) {
|
||||
doc.* = .{};
|
||||
doc.allocator = allocator;
|
||||
doc.files.allocator = allocator;
|
||||
}
|
||||
|
||||
capture_add_file :: (doc: *Capture_Document, name: string) -> *Capture_File {
|
||||
file := array_add(*doc.files);
|
||||
file.name = copy_string(name,, doc.allocator);
|
||||
file.root = capture_object(doc);
|
||||
return file;
|
||||
}
|
||||
|
||||
capture_find_file :: (doc: *Capture_Document, name: string) -> *Capture_File {
|
||||
for * doc.files if it.name == name return it;
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- values
|
||||
|
||||
capture_null :: () -> Capture_Value {
|
||||
return .{kind = .NULL};
|
||||
}
|
||||
|
||||
capture_boolean :: (b: bool) -> Capture_Value {
|
||||
return .{kind = .BOOLEAN, boolean = b};
|
||||
}
|
||||
|
||||
capture_number :: (n: float64) -> Capture_Value {
|
||||
return .{kind = .NUMBER, number = n};
|
||||
}
|
||||
|
||||
capture_string :: (doc: *Capture_Document, text: string) -> Capture_Value {
|
||||
return .{kind = .STRING, text = copy_string(text,, doc.allocator)};
|
||||
}
|
||||
|
||||
capture_array :: (doc: *Capture_Document) -> Capture_Value {
|
||||
result: Capture_Value;
|
||||
result.kind = .ARRAY;
|
||||
result.items.allocator = doc.allocator;
|
||||
return result;
|
||||
}
|
||||
|
||||
capture_object :: (doc: *Capture_Document) -> Capture_Value {
|
||||
result: Capture_Value;
|
||||
result.kind = .OBJECT;
|
||||
result.fields.allocator = doc.allocator;
|
||||
return result;
|
||||
}
|
||||
|
||||
// Adds key to object without checking whether it is already there; the key is copied.
|
||||
capture_object_add :: (doc: *Capture_Document, object: *Capture_Value, key: string, value: Capture_Value) -> *Capture_Value {
|
||||
assert(object.kind == .OBJECT);
|
||||
field := array_add(*object.fields);
|
||||
field.key = copy_string(key,, doc.allocator);
|
||||
field.value = value;
|
||||
return *field.value;
|
||||
}
|
||||
|
||||
capture_object_find :: (object: *Capture_Value, key: string) -> *Capture_Value {
|
||||
if object.kind != .OBJECT return null;
|
||||
for * object.fields if it.key == key return *it.value;
|
||||
return null;
|
||||
}
|
||||
|
||||
// Equal as JSON values: the order of an object's keys does not matter, the order of an array's
|
||||
// elements does. Numbers are compared bit for bit, so -0 and 0 differ.
|
||||
capture_values_equal :: (a: Capture_Value, b: Capture_Value) -> bool {
|
||||
if a.kind != b.kind return false;
|
||||
if #complete a.kind == {
|
||||
case .NULL; return true;
|
||||
case .BOOLEAN; return a.boolean == b.boolean;
|
||||
case .NUMBER; return (cast(*u64) *a.number).* == (cast(*u64) *b.number).*;
|
||||
case .STRING; return a.text == b.text;
|
||||
case .ARRAY;
|
||||
if a.items.count != b.items.count return false;
|
||||
for a.items if !capture_values_equal(it, b.items[it_index]) return false;
|
||||
return true;
|
||||
case .OBJECT;
|
||||
if a.fields.count != b.fields.count return false;
|
||||
for a.fields {
|
||||
other := capture_object_find(*b, it.key);
|
||||
if !other || !capture_values_equal(it.value, other.*) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- reading
|
||||
|
||||
// Reads file_data into doc, which should be empty. A bad line, or a bad [[file]] with its lines, is
|
||||
// logged and skipped, and counted in doc.mistakes. Returns false when the file was not read at all:
|
||||
// no @version 1 before the first line that needs it.
|
||||
capture_text_parse :: (doc: *Capture_Document, path: string, file_data: string) -> bool {
|
||||
push_allocator(doc.allocator);
|
||||
doc.mistakes = 0;
|
||||
|
||||
data := file_data;
|
||||
if data.count >= 3 && data[0] == 0xEF && data[1] == 0xBB && data[2] == 0xBF data = advance(data, 3);
|
||||
|
||||
state: Capture_Parse_State;
|
||||
state.doc = doc;
|
||||
state.path = path;
|
||||
|
||||
// Every line stands alone: a value never continues onto the next line.
|
||||
offset: s64;
|
||||
line_number := 0;
|
||||
while offset < data.count {
|
||||
line_number += 1;
|
||||
line, colon, valid := capture_scan_line(data, *offset);
|
||||
if !valid { capture_mistake(*state, line_number, "Quote is unterminated. Ignored."); continue; }
|
||||
body := trim(line);
|
||||
if !body.count continue;
|
||||
if colon >= 0 colon -= body.data - line.data; // from the start of body
|
||||
|
||||
if body[0] == #char "@" {
|
||||
key, value := capture_split_directive(body);
|
||||
if key == "version" {
|
||||
rest := value;
|
||||
version, version_ok := parse_int(*rest, s64);
|
||||
if !version_ok || trim(rest).count || version < 1 || version > CAPTURE_TEXT_VERSION {
|
||||
log_error("% is version %, but this build reads version %. It was not read.", path, value, CAPTURE_TEXT_VERSION);
|
||||
return false;
|
||||
}
|
||||
state.version_seen = true;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
if !state.version_seen {
|
||||
log_error("% does not start with @version %. It was not read.", path, CAPTURE_TEXT_VERSION);
|
||||
return false;
|
||||
}
|
||||
|
||||
if begins_with(body, "[[") {
|
||||
capture_open_file(*state, line_number, body);
|
||||
} else if body[0] == #char "[" {
|
||||
capture_open_section(*state, line_number, body);
|
||||
} else {
|
||||
capture_key_line(*state, line_number, body, colon);
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- writing
|
||||
|
||||
CAPTURE_MAX_KEY_WIDTH :: 40; // keys longer than this do not push the other values further right
|
||||
|
||||
// The document as its file holds it. Temp.
|
||||
capture_text_write :: (doc: *Capture_Document) -> string {
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
print_to_builder(*builder, "@version %\n", CAPTURE_TEXT_VERSION);
|
||||
if doc.comment.count {
|
||||
for split(doc.comment, "\n",, temp) print_to_builder(*builder, "# %\n", it);
|
||||
}
|
||||
|
||||
for file: doc.files {
|
||||
print_to_builder(*builder, "\n[[file %]]\n", capture_quote(file.name));
|
||||
|
||||
lines: [..] Capture_Line;
|
||||
lines.allocator = temp;
|
||||
for file.root.fields {
|
||||
if capture_is_section_array(it.value) continue;
|
||||
capture_collect_lines(*lines, capture_key_text(it.key), it.value);
|
||||
}
|
||||
capture_write_block(*builder, lines);
|
||||
|
||||
for field: file.root.fields {
|
||||
if !capture_is_section_array(field.value) continue;
|
||||
header := capture_key_text(field.key);
|
||||
for item: field.value.items {
|
||||
print_to_builder(*builder, "\n[%]\n", header);
|
||||
array_reset_keeping_memory(*lines);
|
||||
for item.fields capture_collect_lines(*lines, capture_key_text(it.key), it.value);
|
||||
capture_write_block(*builder, lines);
|
||||
}
|
||||
}
|
||||
}
|
||||
return builder_to_string(*builder,, temp);
|
||||
}
|
||||
|
||||
// Always quoted, a quote written twice. Control characters cannot be quoted; see capture_string_parts.
|
||||
capture_quote :: (text: string) -> string {
|
||||
return tprint("\"%\"", replace(text, "\"", "\"\"",, temp));
|
||||
}
|
||||
|
||||
// A key as one segment of a key path: bare when it can be, quoted otherwise. A key cannot hold
|
||||
// control characters (see capture_key_is_writable). Temp.
|
||||
capture_key_text :: (key: string) -> string {
|
||||
if capture_is_bare_key(key) return key;
|
||||
return capture_quote(key);
|
||||
}
|
||||
|
||||
// Whether a key can be written at all: a quoted key is one part, so it cannot hold the characters
|
||||
// below space, or DEL, that a string writes as 0xNN bytes.
|
||||
capture_key_is_writable :: (key: string) -> bool {
|
||||
for 0..key.count - 1 if key[it] < 0x20 || key[it] == 0x7F return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// The shortest text that reads back as exactly n. Temp.
|
||||
capture_number_text :: (n: float64) -> string {
|
||||
bits := (cast(*u64) *n).*;
|
||||
if n == 0 return ifx bits >> 63 then "-0" else "0";
|
||||
// Whole numbers that a float64 holds exactly are written as integers: 100, not 100.0 or 1e+02.
|
||||
if n > -9007199254740992.0 && n < 9007199254740992.0 && n == cast(float64) cast,trunc(s64) n {
|
||||
return tprint("%", cast,trunc(s64) n);
|
||||
}
|
||||
|
||||
// Basic's SHORTEST mode keeps only six significant digits, so the fewest digits that read back as
|
||||
// exactly n are found by trying: plain decimals for ordinary magnitudes, an exponent otherwise.
|
||||
// 17 significant digits always suffice for a float64.
|
||||
magnitude := ifx n < 0 then -n else n;
|
||||
if magnitude >= 0.000001 && magnitude < 1000000000000000000000.0 {
|
||||
for digits: 1..24 {
|
||||
text := trim(tprint("%", formatFloat(n, trailing_width = digits, zero_removal = .YES)));
|
||||
if capture_number_reads_back(text, bits) return text;
|
||||
}
|
||||
}
|
||||
for digits: 0..16 {
|
||||
text := trim(tprint("%", formatFloat(n, trailing_width = digits, mode = .SCIENTIFIC, zero_removal = .YES)));
|
||||
if capture_number_reads_back(text, bits) return text;
|
||||
}
|
||||
return tprint("%", formatFloat(n, trailing_width = 16, mode = .SCIENTIFIC, zero_removal = .NO));
|
||||
}
|
||||
|
||||
capture_number_reads_back :: (text: string, bits: u64) -> bool {
|
||||
if capture_number_length(text) != text.count return false;
|
||||
value, ok, rest := string_to_float64(text);
|
||||
return ok && !rest.count && (cast(*u64) *value).* == bits;
|
||||
}
|
||||
|
||||
#scope_module
|
||||
|
||||
// ---------------------------------------------------------------- reading, in detail
|
||||
|
||||
Capture_Parse_State :: struct {
|
||||
doc: *Capture_Document;
|
||||
path: string;
|
||||
version_seen: bool;
|
||||
|
||||
file: *Capture_File; // the [[file]] being read; null before the first, or after a bad one
|
||||
base: *Capture_Value; // the object key lines go into: the file's, or a [KEY] section's
|
||||
skipping: bool; // after a bad header, until the next good one
|
||||
}
|
||||
|
||||
MAX_NESTING :: 64;
|
||||
|
||||
capture_mistake :: (state: *Capture_Parse_State, line_number: s64, format: string, args: ..Any) {
|
||||
log_error("%:%: %", state.path, line_number, tprint(format, ..args));
|
||||
state.doc.mistakes += 1;
|
||||
} @PrintLike
|
||||
|
||||
capture_open_file :: (state: *Capture_Parse_State, line_number: s64, body: string) {
|
||||
state.file = null;
|
||||
state.base = null;
|
||||
state.skipping = true;
|
||||
|
||||
if !ends_with(body, "]]") { capture_mistake(state, line_number, "Malformed header (expected `[[file \"NAME\"]]`); its lines are skipped."); return; }
|
||||
inner := trim(slice(body, 2, body.count - 4));
|
||||
if !begins_with(inner, "file") || inner.count < 5 || !is_space(inner[4]) {
|
||||
capture_mistake(state, line_number, "Unknown header % (expected `[[file \"NAME\"]]`); its lines are skipped.", body);
|
||||
return;
|
||||
}
|
||||
name_text := trim(advance(inner, 4));
|
||||
name, rest, ok := capture_read_quoted(name_text);
|
||||
if !ok || trim(rest).count { capture_mistake(state, line_number, "The file name in % must be one quoted string; its lines are skipped.", body); return; }
|
||||
if capture_find_file(state.doc, name) { capture_mistake(state, line_number, "File \"%\" was already read; this section is skipped.", name); return; }
|
||||
|
||||
state.file = capture_add_file(state.doc, name);
|
||||
state.base = *state.file.root;
|
||||
state.skipping = false;
|
||||
}
|
||||
|
||||
capture_open_section :: (state: *Capture_Parse_State, line_number: s64, body: string) {
|
||||
state.base = null;
|
||||
if !state.file { capture_mistake(state, line_number, "% comes before any [[file]] section; its lines are skipped.", body); state.skipping = true; return; }
|
||||
state.skipping = true;
|
||||
|
||||
if body[body.count - 1] != #char "]" { capture_mistake(state, line_number, "Malformed header (expected `[KEY]`); its lines are skipped."); return; }
|
||||
segments, ok := capture_parse_path(trim(slice(body, 1, body.count - 2)));
|
||||
if !ok { capture_mistake(state, line_number, "% is not a key path; its lines are skipped.", body); return; }
|
||||
|
||||
array, error := capture_walk(state.doc, *state.file.root, segments, .ARRAY);
|
||||
if !array { capture_mistake(state, line_number, "%: %; its lines are skipped.", body, error); return; }
|
||||
|
||||
element := array_add(*array.items);
|
||||
element.* = capture_object(state.doc);
|
||||
state.base = element;
|
||||
state.skipping = false;
|
||||
}
|
||||
|
||||
// colon: where in body the first : outside quotes is, or -1.
|
||||
capture_key_line :: (state: *Capture_Parse_State, line_number: s64, body: string, colon: s64) {
|
||||
if state.skipping return;
|
||||
if !state.base { capture_mistake(state, line_number, "Key line before any [[file]] section. Ignored."); return; }
|
||||
|
||||
if colon < 0 { capture_mistake(state, line_number, "Expected `KEY: VALUE`. Ignored."); return; }
|
||||
key_text := trim(slice(body, 0, colon));
|
||||
value_text := trim(advance(body, colon + 1));
|
||||
|
||||
segments, path_ok := capture_parse_path(key_text);
|
||||
if !path_ok { capture_mistake(state, line_number, "% is not a key path. Ignored.", key_text); return; }
|
||||
|
||||
rest := value_text;
|
||||
value, value_ok := capture_parse_value(state.doc, *rest, 0);
|
||||
if !value_ok || trim(rest).count { capture_mistake(state, line_number, "% has no value it can read in `%`. Ignored.", key_text, value_text); return; }
|
||||
|
||||
slot, error := capture_walk(state.doc, state.base, segments, value.kind, value_must_be_new = true);
|
||||
if !slot { capture_mistake(state, line_number, "%: %. Ignored.", key_text, error); return; }
|
||||
slot.* = value;
|
||||
}
|
||||
|
||||
Capture_Path_Segment :: struct {
|
||||
key: string;
|
||||
index := -1; // an array index when 0 or more
|
||||
}
|
||||
|
||||
// Key path: segments separated by dots, each a bare name, a quoted key, or digits for an array index.
|
||||
capture_parse_path :: (text: string) -> [..] Capture_Path_Segment, bool {
|
||||
segments: [..] Capture_Path_Segment;
|
||||
segments.allocator = temp;
|
||||
rest := text;
|
||||
while true {
|
||||
segment: Capture_Path_Segment;
|
||||
if rest.count && rest[0] == #char "\"" {
|
||||
key, after, ok := capture_read_quoted(rest);
|
||||
if !ok return segments, false;
|
||||
segment.key = key;
|
||||
rest = after;
|
||||
} else {
|
||||
i := 0;
|
||||
while i < rest.count && capture_is_bare_char(rest[i]) i += 1;
|
||||
if i == 0 return segments, false;
|
||||
word := slice(rest, 0, i);
|
||||
rest = advance(rest, i);
|
||||
if capture_is_digits(word) {
|
||||
if word.count > 9 return segments, false; // no array is that long
|
||||
index_text := word;
|
||||
segment.index = parse_int(*index_text, s64);
|
||||
} else {
|
||||
segment.key = word;
|
||||
}
|
||||
}
|
||||
array_add(*segments, segment);
|
||||
if !rest.count return segments, true;
|
||||
if rest[0] != #char "." return segments, false;
|
||||
rest = advance(rest, 1);
|
||||
}
|
||||
return segments, false;
|
||||
}
|
||||
|
||||
// Follows segments down from node, making the objects and arrays the path needs, and returns the
|
||||
// slot at the end: a new one for a key line, which must not have a value yet, or for a [KEY] header
|
||||
// the array that is there or a new one. On failure returns null and why.
|
||||
capture_walk :: (doc: *Capture_Document, node: *Capture_Value, segments: [] Capture_Path_Segment, final_kind: Capture_Value_Kind, value_must_be_new := false) -> *Capture_Value, string {
|
||||
current := node;
|
||||
for segment, segment_index: segments {
|
||||
last := segment_index == segments.count - 1;
|
||||
// What a new slot here has to be: what the next segment needs, or what is being put at the end.
|
||||
kind := final_kind;
|
||||
if !last kind = ifx segments[segment_index + 1].index >= 0 then Capture_Value_Kind.ARRAY else .OBJECT;
|
||||
|
||||
child: *Capture_Value;
|
||||
is_new := false;
|
||||
if segment.index >= 0 {
|
||||
if current.kind != .ARRAY return null, tprint("index % of something that is not an array", segment.index);
|
||||
if segment.index > current.items.count return null, tprint("index % skips past the next free index %", segment.index, current.items.count);
|
||||
if segment.index == current.items.count {
|
||||
child = array_add(*current.items);
|
||||
is_new = true;
|
||||
} else {
|
||||
child = *current.items[segment.index];
|
||||
}
|
||||
} else {
|
||||
if current.kind != .OBJECT return null, tprint("key \"%\" of something that is not an object", segment.key);
|
||||
child = capture_object_find(current, segment.key);
|
||||
if !child {
|
||||
child = capture_object_add(doc, current, segment.key, .{});
|
||||
is_new = true;
|
||||
}
|
||||
}
|
||||
|
||||
if is_new {
|
||||
if kind == .ARRAY child.* = capture_array(doc);
|
||||
else if kind == .OBJECT child.* = capture_object(doc);
|
||||
} else if last && value_must_be_new {
|
||||
return null, "already has a value";
|
||||
} else if child.kind != kind {
|
||||
return null, tprint("already holds % where % is needed", capture_kind_name(child.kind), capture_kind_name(kind));
|
||||
}
|
||||
current = child;
|
||||
}
|
||||
return current, "";
|
||||
}
|
||||
|
||||
capture_kind_name :: (kind: Capture_Value_Kind) -> string {
|
||||
if #complete kind == {
|
||||
case .NULL; return "null";
|
||||
case .BOOLEAN; return "a boolean";
|
||||
case .NUMBER; return "a number";
|
||||
case .STRING; return "a string";
|
||||
case .ARRAY; return "an array";
|
||||
case .OBJECT; return "an object";
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
capture_parse_value :: (doc: *Capture_Document, s: *string, depth: s64) -> Capture_Value, bool {
|
||||
fail: Capture_Value;
|
||||
s.* = trim_left(s.*);
|
||||
if !s.count || depth > MAX_NESTING return fail, false;
|
||||
|
||||
c := s.*[0];
|
||||
if c == #char "\"" {
|
||||
first_part, rest, ok := capture_read_quoted(s.*);
|
||||
if !ok return fail, false;
|
||||
s.* = rest;
|
||||
// Nearly every string is one part, copied straight into the document. Further parts are joined in a builder.
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
parts := 1;
|
||||
while true {
|
||||
after := trim_left(s.*, " \t"); // spaces or tabs between parts
|
||||
if after.count && after[0] == #char "\"" {
|
||||
text, rest, ok := capture_read_quoted(after);
|
||||
if !ok return fail, false;
|
||||
if parts == 1 append(*builder, first_part);
|
||||
append(*builder, text);
|
||||
s.* = rest;
|
||||
} else if after.count >= 4 && after[0] == #char "0" && after[1] == #char "x" {
|
||||
high := capture_hex_digit(after[2]);
|
||||
low := capture_hex_digit(after[3]);
|
||||
if high < 0 || low < 0 return fail, false;
|
||||
if after.count > 4 && !capture_ends_token(after[4]) return fail, false;
|
||||
if parts == 1 append(*builder, first_part);
|
||||
append(*builder, cast(u8) (high * 16 + low));
|
||||
s.* = advance(after, 4);
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
parts += 1;
|
||||
}
|
||||
if parts == 1 return capture_string(doc, first_part), true;
|
||||
return capture_string(doc, builder_to_string(*builder,, temp)), true;
|
||||
}
|
||||
|
||||
if c == #char "[" {
|
||||
result := capture_array(doc);
|
||||
s.* = trim_left(advance(s.*, 1));
|
||||
if s.count && s.*[0] == #char "]" { s.* = advance(s.*, 1); return result, true; }
|
||||
while true {
|
||||
item, ok := capture_parse_value(doc, s, depth + 1);
|
||||
if !ok return fail, false;
|
||||
array_add(*result.items, item);
|
||||
s.* = trim_left(s.*);
|
||||
if !s.count return fail, false;
|
||||
if s.*[0] == #char "]" { s.* = advance(s.*, 1); return result, true; }
|
||||
if s.*[0] != #char "," return fail, false;
|
||||
s.* = advance(s.*, 1);
|
||||
}
|
||||
}
|
||||
|
||||
if c == #char "{" {
|
||||
rest := trim_left(advance(s.*, 1));
|
||||
if !rest.count || rest[0] != #char "}" return fail, false; // only {} is written inline
|
||||
s.* = advance(rest, 1);
|
||||
return capture_object(doc), true;
|
||||
}
|
||||
|
||||
if c == #char "-" || (c >= #char "0" && c <= #char "9") {
|
||||
length := capture_number_length(s.*);
|
||||
if length <= 0 || (length < s.count && !capture_ends_token(s.*[length])) return fail, false;
|
||||
n, ok, remainder := string_to_float64(slice(s.*, 0, length));
|
||||
if !ok || remainder.count return fail, false;
|
||||
s.* = advance(s.*, length);
|
||||
return capture_number(n), true;
|
||||
}
|
||||
|
||||
for word: string.["null", "true", "false"] {
|
||||
if !begins_with(s.*, word) continue;
|
||||
if s.count > word.count && !capture_ends_token(s.*[word.count]) continue;
|
||||
s.* = advance(s.*, word.count);
|
||||
if word == "null" return capture_null(), true;
|
||||
return capture_boolean(word == "true"), true;
|
||||
}
|
||||
return fail, false;
|
||||
}
|
||||
|
||||
// The length of a JSON number at the start of s: -?digits(.digits)?([eE][+-]?digits)?. 0 when there is none.
|
||||
capture_number_length :: (s: string) -> s64 {
|
||||
i := 0;
|
||||
if i < s.count && s[i] == #char "-" i += 1;
|
||||
digits_start := i;
|
||||
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
|
||||
if i == digits_start return 0;
|
||||
if i < s.count && s[i] == #char "." {
|
||||
i += 1;
|
||||
fraction_start := i;
|
||||
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
|
||||
if i == fraction_start return 0;
|
||||
}
|
||||
if i < s.count && (s[i] == #char "e" || s[i] == #char "E") {
|
||||
i += 1;
|
||||
if i < s.count && (s[i] == #char "+" || s[i] == #char "-") i += 1;
|
||||
exponent_start := i;
|
||||
while i < s.count && s[i] >= #char "0" && s[i] <= #char "9" i += 1;
|
||||
if i == exponent_start return 0;
|
||||
}
|
||||
return i;
|
||||
}
|
||||
|
||||
capture_ends_token :: (c: u8) -> bool {
|
||||
return is_space(c) || c == #char "," || c == #char "]";
|
||||
}
|
||||
|
||||
capture_hex_digit :: (c: u8) -> s64 {
|
||||
if c >= #char "0" && c <= #char "9" return c - #char "0";
|
||||
if c >= #char "A" && c <= #char "F" return c - #char "A" + 10;
|
||||
if c >= #char "a" && c <= #char "f" return c - #char "a" + 10;
|
||||
return -1;
|
||||
}
|
||||
|
||||
// One quoted string ("" for a quote) at the start of s. Returns the text and what follows. The text is
|
||||
// a slice of s when it holds no doubled quote, as nearly every string does; otherwise it is temp.
|
||||
capture_read_quoted :: (s: string) -> text: string, rest: string, ok: bool {
|
||||
if !s.count || s[0] != #char "\"" return "", s, false;
|
||||
i := 1;
|
||||
while i < s.count && s[i] != #char "\"" i += 1;
|
||||
if i >= s.count return "", s, false;
|
||||
if i + 1 >= s.count || s[i + 1] != #char "\"" return slice(s, 1, i - 1), advance(s, i + 1), true;
|
||||
|
||||
// A doubled quote: the text has to be put together.
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
run_start := 1;
|
||||
while i < s.count {
|
||||
if s[i] == #char "\"" {
|
||||
append(*builder, slice(s, run_start, i - run_start));
|
||||
if i + 1 < s.count && s[i + 1] == #char "\"" {
|
||||
append(*builder, #char "\"");
|
||||
i += 2;
|
||||
run_start = i;
|
||||
continue;
|
||||
}
|
||||
return builder_to_string(*builder,, temp), advance(s, i + 1), true;
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
return "", s, false;
|
||||
}
|
||||
|
||||
// One line, from offset, in a single pass: the line without its comment or line ending, and where in it
|
||||
// the first : outside quotes is (-1 when there is none). "#" opens a comment only outside quotes,
|
||||
// and only at the start of the line or after whitespace. valid is false when the line ends inside quotes.
|
||||
capture_scan_line :: (data: string, offset: *s64) -> line: string, colon: s64, valid: bool {
|
||||
start := offset.*;
|
||||
i := start;
|
||||
colon := -1;
|
||||
end := -1; // where a comment starts
|
||||
while i < data.count {
|
||||
c := data[i];
|
||||
if c == #char "\"" {
|
||||
// Straight to the closing quote: strings are most of the bytes. A doubled quote closes and opens.
|
||||
i += 1;
|
||||
while i < data.count && data[i] != #char "\"" && data[i] != #char "\n" i += 1;
|
||||
if i >= data.count || data[i] == #char "\n" {
|
||||
while i < data.count && data[i] != #char "\n" i += 1;
|
||||
offset.* = ifx i < data.count then i + 1 else data.count;
|
||||
return "", -1, false;
|
||||
}
|
||||
} else if c == #char "\n" {
|
||||
break;
|
||||
} else if c == #char "#" && (i == start || is_space(data[i - 1])) {
|
||||
end = i;
|
||||
while i < data.count && data[i] != #char "\n" i += 1;
|
||||
break;
|
||||
} else if c == #char ":" && colon < 0 {
|
||||
colon = i - start;
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
offset.* = ifx i < data.count then i + 1 else data.count;
|
||||
if end < 0 {
|
||||
end = i;
|
||||
if end > start && data[end - 1] == #char "\r" end -= 1; // CRLF
|
||||
}
|
||||
return string.{end - start, data.data + start}, colon, true;
|
||||
}
|
||||
|
||||
capture_split_directive :: (body: string) -> key: string, value: string {
|
||||
rest := advance(body, 1); // skip the @
|
||||
i := 0;
|
||||
while i < rest.count && !is_space(rest[i]) i += 1;
|
||||
return slice(rest, 0, i), trim(advance(rest, i));
|
||||
}
|
||||
|
||||
capture_is_bare_char :: (c: u8) -> bool {
|
||||
return (c >= #char "a" && c <= #char "z") || (c >= #char "A" && c <= #char "Z") || (c >= #char "0" && c <= #char "9") || c == #char "_" || c == #char "-";
|
||||
}
|
||||
|
||||
capture_is_digits :: (s: string) -> bool {
|
||||
if !s.count return false;
|
||||
for 0..s.count - 1 if s[it] < #char "0" || s[it] > #char "9" return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
capture_is_bare_key :: (key: string) -> bool {
|
||||
if !key.count || capture_is_digits(key) return false;
|
||||
for 0..key.count - 1 if !capture_is_bare_char(key[it]) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- writing, in detail
|
||||
|
||||
Capture_Line :: struct {
|
||||
path: string;
|
||||
value: Capture_Value;
|
||||
}
|
||||
|
||||
// A root array of objects is written as [KEY] sections, one per object.
|
||||
capture_is_section_array :: (value: Capture_Value) -> bool {
|
||||
if value.kind != .ARRAY || !value.items.count return false;
|
||||
for value.items if it.kind != .OBJECT return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
// An array that holds an object with keys, at any depth, cannot be written inline.
|
||||
capture_needs_paths :: (value: Capture_Value) -> bool {
|
||||
if value.kind == .OBJECT return value.fields.count > 0;
|
||||
if value.kind == .ARRAY {
|
||||
for value.items if capture_needs_paths(it) return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
capture_collect_lines :: (lines: *[..] Capture_Line, path: string, value: Capture_Value) {
|
||||
if capture_needs_paths(value) {
|
||||
if value.kind == .OBJECT {
|
||||
for value.fields capture_collect_lines(lines, tprint("%.%", path, capture_key_text(it.key)), it.value);
|
||||
} else {
|
||||
for value.items capture_collect_lines(lines, tprint("%.%", path, it_index), it);
|
||||
}
|
||||
return;
|
||||
}
|
||||
array_add(lines, .{path, value});
|
||||
}
|
||||
|
||||
// Key lines with their values lined up, each on one line however long.
|
||||
capture_write_block :: (builder: *String_Builder, lines: [] Capture_Line) {
|
||||
key_width := 0;
|
||||
for lines if it.path.count + 1 <= CAPTURE_MAX_KEY_WIDTH key_width = max(key_width, it.path.count + 1);
|
||||
|
||||
for lines {
|
||||
head := tprint("%:", it.path);
|
||||
append(builder, head);
|
||||
for head.count..max(head.count, key_width) append(builder, #char " ");
|
||||
capture_append_value(builder, it.value);
|
||||
append(builder, #char "\n");
|
||||
}
|
||||
}
|
||||
|
||||
capture_append_value :: (builder: *String_Builder, value: Capture_Value) {
|
||||
if #complete value.kind == {
|
||||
case .NULL; append(builder, "null");
|
||||
case .BOOLEAN; append(builder, ifx value.boolean then "true" else "false");
|
||||
case .NUMBER; append(builder, capture_number_text(value.number));
|
||||
case .STRING; append(builder, capture_string_text(value.text));
|
||||
case .OBJECT; append(builder, "{}"); // only an empty one reaches here
|
||||
case .ARRAY;
|
||||
append(builder, #char "[");
|
||||
for value.items {
|
||||
if it_index > 0 append(builder, ", ");
|
||||
capture_append_value(builder, it);
|
||||
}
|
||||
append(builder, #char "]");
|
||||
}
|
||||
}
|
||||
|
||||
// A string on one line: quoted, with control characters and DEL, which quotes cannot hold, written
|
||||
// as 0xNN parts between quoted ones. Temp.
|
||||
capture_string_text :: (text: string) -> string {
|
||||
if !text.count return "\"\"";
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
i := 0;
|
||||
while i < text.count {
|
||||
if i > 0 append(*builder, #char " ");
|
||||
c := text[i];
|
||||
if c < 0x20 || c == 0x7F {
|
||||
if i == 0 append(*builder, "\"\" "); // the first part is always quoted
|
||||
print_to_builder(*builder, "0x%", formatInt(c, base = 16, minimum_digits = 2));
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
end := i;
|
||||
while end < text.count && text[end] >= 0x20 && text[end] != 0x7F end += 1;
|
||||
append(*builder, capture_quote(slice(text, i, end - i)));
|
||||
i = end;
|
||||
}
|
||||
return builder_to_string(*builder,, temp);
|
||||
}
|
||||
|
||||
#import "Basic";
|
||||
#import "String";
|
||||
@ -1,7 +1,8 @@
|
||||
# OCR
|
||||
|
||||
Text in images, with where each line is: a batch of RGBA images in, each image's text lines out,
|
||||
each with a box and a confidence. It runs ONNX models through onnxruntime
|
||||
each with its corners, its box and a confidence. Lines at an angle, read downwards or upwards, or
|
||||
upside down are read too. It runs ONNX models through onnxruntime
|
||||
([`modules/onnxruntime`](../onnxruntime)): on an NVIDIA GPU through CUDA when every library that
|
||||
needs is present, and on the CPU otherwise.
|
||||
|
||||
@ -11,7 +12,7 @@ The binaries and the models are downloaded, not committed:
|
||||
|
||||
```
|
||||
cd modules/onnxruntime && jai fetch.jai # onnxruntime 1.30.0, GPU build (CUDA 13); `- cpu` for the CPU-only build
|
||||
cd modules/OCR && jai fetch_models.jai # both models (about 195 MB); `- mobile` for the light one only
|
||||
cd modules/OCR && jai fetch_models.jai # both models (about 196 MB); `- mobile` for the light one only
|
||||
jai generate.jai # in modules/onnxruntime, only when changing onnxruntime versions
|
||||
```
|
||||
|
||||
@ -31,7 +32,16 @@ ocr_destroy(engine);
|
||||
```
|
||||
|
||||
`ocr_process` shares a batch out over worker threads: detection per image, then recognition of
|
||||
every line of every image in batches of similar width. The results come from the context
|
||||
every line of every image in batches of similar width.
|
||||
|
||||
**Turned text.** Detection gives each line the smallest rectangle around it at any angle (rotating
|
||||
calipers over its convex hull), keeping a level one unless a turned one is clearly smaller; a level
|
||||
box takes in every pixel it touches, as recognition reads best. Recognition samples each rectangle
|
||||
along its own sides into a level crop. A box at least 1.5 times as tall as it is wide is taken as a
|
||||
line read downwards, and also read level, since a narrow word such as "1" is as tall. The
|
||||
orientation classifier marks lines that look upside down, and those are read both ways. Of each
|
||||
line's readings, the more confident is kept. `Ocr_Line.corners` go clockwise from the text's top
|
||||
left, so a turned line says which way it reads. The results come from the context
|
||||
allocator, or the one passed. `ocr_stats` gives the time spent in each stage.
|
||||
|
||||
## Models
|
||||
@ -39,10 +49,13 @@ allocator, or the one passed. `ocr_stats` gives the time spent in each stage.
|
||||
`models.config` lists them; `ocr_create` takes a name. Both are PaddleOCR's PP-OCRv5 (Apache 2.0),
|
||||
as converted to ONNX by RapidOCR, and read Chinese, English and Japanese:
|
||||
|
||||
| Name | Files | On the 20 sample captures, CPU |
|
||||
| Name | Files | On the 20 sample captures: RTX 5080 laptop / CPU |
|
||||
| --- | --- | --- |
|
||||
| `ppocr-v5-mobile` | detector 4.8 MB, recognizer 16.6 MB | about 0.5 s per capture |
|
||||
| `ppocr-v5-server` | detector 88 MB, recognizer 85 MB | about 2.5 s per capture |
|
||||
| `ppocr-v5-mobile` | detector 4.8 MB, recognizer 16.6 MB | 0.13 s / 0.5 to 0.65 s per capture |
|
||||
| `ppocr-v5-server` | detector 88 MB, recognizer 85 MB | 2.3 s / 2.5 s per capture (0.36 s on the GPU with `rec_batch: 4`) |
|
||||
|
||||
Both also use the text line orientation classifier `ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx`
|
||||
(1 MB, `textline_ori:` in `models.config`).
|
||||
|
||||
Another model with the same two-stage design is a new `[model NAME]` section; a different design
|
||||
needs its own detection or recognition code beside `detect.jai` and `recognize.jai`.
|
||||
@ -55,7 +68,9 @@ onnxruntime's CUDA build needs, on PATH or beside the executable:
|
||||
- the CUDA Toolkit 13.x (`cudart64_13`, `cublas64_13`, `cublasLt64_13`, `cufft64_12`), from
|
||||
[developer.nvidia.com/cuda-downloads](https://developer.nvidia.com/cuda-downloads), whose installer puts its bin folder on PATH;
|
||||
- cuDNN 9.x for CUDA 13 (`cudnn64_9` and its `cudnn_*64_9` companions), from
|
||||
[developer.nvidia.com/cudnn-downloads](https://developer.nvidia.com/cudnn-downloads), whose bin folder for CUDA 13 you add to PATH yourself.
|
||||
[developer.nvidia.com/cudnn-downloads](https://developer.nvidia.com/cudnn-downloads). Its installer does not touch PATH;
|
||||
when cuDNN is not on it, the newest `C:\Program Files\NVIDIA\CUDNN\v9.*\bin\13.*\x64` is added to the
|
||||
program's own PATH, so an install in the usual place needs nothing more.
|
||||
|
||||
`ocr_create` looks for each of these first (`cuda_check.jai`) and says which are missing, rather
|
||||
than letting onnxruntime fail later and obscurely. If they are all there but CUDA still cannot
|
||||
|
||||
@ -2,6 +2,10 @@
|
||||
// The provider needs the NVIDIA driver, parts of the CUDA 13 toolkit and cuDNN 9 for CUDA 13; it
|
||||
// fails late and obscurely without them, so each library is looked for first, by the name the
|
||||
// provider loads it by (from onnxruntime_providers_cuda.dll's imports and the names it loads).
|
||||
//
|
||||
// cuDNN's Windows installer does not put its DLLs on PATH. When they are not found, the newest
|
||||
// cuDNN 9 for CUDA 13 under Program Files\NVIDIA\CUDNN is added to this process's PATH, so the
|
||||
// provider finds it without anyone editing PATH.
|
||||
|
||||
Cuda_Library :: struct {
|
||||
name: string;
|
||||
@ -53,6 +57,8 @@ ocr_cuda_check :: () -> available: bool, explanation: string {
|
||||
return false, tprint("OCR runs on the CPU: the onnxruntime beside the program is the CPU-only build (% is not there).\nTo use an NVIDIA GPU, run `jai fetch.jai` in modules/onnxruntime (without `cpu`) and rebuild.", CUDA_PROVIDER_LIBRARY);
|
||||
}
|
||||
|
||||
#if OS == .WINDOWS ocr_add_cudnn_to_path();
|
||||
|
||||
missing_driver, missing_toolkit, missing_cudnn: [..] string;
|
||||
missing_driver.allocator, missing_toolkit.allocator, missing_cudnn.allocator = temp, temp, temp;
|
||||
for CUDA_LIBRARIES {
|
||||
@ -87,8 +93,8 @@ ocr_cuda_check :: () -> available: bool, explanation: string {
|
||||
print_to_builder(*builder, "\n%. cuDNN 9.x for CUDA 13, which has % that are missing.\n", step, join(..missing_cudnn, ", ",, temp));
|
||||
append(*builder, " https://developer.nvidia.com/cudnn-downloads\n");
|
||||
#if OS == .WINDOWS {
|
||||
append(*builder, " Its installer does not change PATH: add the folder with these DLLs to it yourself,\n");
|
||||
append(*builder, " such as C:\\Program Files\\NVIDIA\\CUDNN\\v9.x\\bin\\13.x\\x64.\n");
|
||||
append(*builder, " Its installer does not change PATH. Installed in its usual place, C:\\Program Files\\NVIDIA\\CUDNN,\n");
|
||||
append(*builder, " it is found there; elsewhere, add the folder with these DLLs to PATH yourself.\n");
|
||||
} else {
|
||||
append(*builder, " Then make sure its lib folder is on LD_LIBRARY_PATH.\n");
|
||||
}
|
||||
@ -115,3 +121,75 @@ ocr_library_loads :: (name: string) -> bool {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
#if OS == .WINDOWS {
|
||||
// When cuDNN does not load, puts the newest Program Files\NVIDIA\CUDNN\v9.*\bin\13.*\x64 (or
|
||||
// bin\13.*, as older releases lay it out) holding cudnn64_9.dll at the front of this process's PATH.
|
||||
ocr_add_cudnn_to_path :: () {
|
||||
if ocr_library_loads("cudnn64_9.dll") return;
|
||||
program_files := ocr_environment_variable("ProgramW6432");
|
||||
if !program_files program_files = ocr_environment_variable("ProgramFiles");
|
||||
if !program_files return;
|
||||
|
||||
versions := ocr_subfolders_newest_first(tprint("%\\NVIDIA\\CUDNN", program_files), "v9.");
|
||||
for version: versions {
|
||||
for cuda: ocr_subfolders_newest_first(tprint("%\\bin", version), "13.") {
|
||||
for folder: string.[tprint("%\\x64", cuda), cuda] {
|
||||
if !file_exists(tprint("%\\cudnn64_9.dll", folder)) continue;
|
||||
found := replace(folder, "/", "\\",, temp);
|
||||
path := ocr_environment_variable("PATH");
|
||||
OcrSetEnvironmentVariableW(utf8_to_wide("PATH",, temp), utf8_to_wide(tprint("%;%", found, path),, temp));
|
||||
log("[OCR] cuDNN is not on PATH; using the one in %.", found);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The folders in parent whose names start with prefix, the highest version first ("v9.26" above
|
||||
// "v9.3"). Temp.
|
||||
ocr_subfolders_newest_first :: (parent: string, prefix: string) -> [..] string {
|
||||
Found :: struct { names: [..] string; prefix: string; }
|
||||
found: Found;
|
||||
found.names.allocator = temp;
|
||||
found.prefix = prefix;
|
||||
visit_files(parent, recursive = false, *found, (info: *File_Visit_Info, found: *Found) {
|
||||
if begins_with(info.short_name, found.prefix) array_add(*found.names, copy_string(info.full_name,, temp));
|
||||
}, visit_files = false, visit_directories = true);
|
||||
quick_sort(found.names, (a: string, b: string) -> s64 { return ocr_compare_versions(path_filename(b), path_filename(a)); });
|
||||
return found.names;
|
||||
}
|
||||
|
||||
// Compares the numbers in two names one by one: "v9.26" > "v9.3".
|
||||
ocr_compare_versions :: (a: string, b: string) -> s64 {
|
||||
next_number :: (s: *string) -> s64, found: bool {
|
||||
while s.count && (s.*[0] < #char "0" || s.*[0] > #char "9") advance(s);
|
||||
if !s.count return 0, false;
|
||||
n: s64;
|
||||
while s.count && s.*[0] >= #char "0" && s.*[0] <= #char "9" { n = n * 10 + s.*[0] - #char "0"; advance(s); }
|
||||
return n, true;
|
||||
}
|
||||
x, y := a, b;
|
||||
while true {
|
||||
p, p_found := next_number(*x);
|
||||
q, q_found := next_number(*y);
|
||||
if !p_found || !q_found return ifx p_found then 1 else ifx q_found then -1 else 0;
|
||||
if p != q return ifx p > q then 1 else -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Temp; "" when it is not set.
|
||||
ocr_environment_variable :: (name: string) -> string {
|
||||
wide_name := utf8_to_wide(name,, temp);
|
||||
size := OcrGetEnvironmentVariableW(wide_name, null, 0);
|
||||
if !size return "";
|
||||
buffer := NewArray(size, u16,, temp);
|
||||
written := OcrGetEnvironmentVariableW(wide_name, buffer.data, size);
|
||||
if !written || written >= size return "";
|
||||
return wide_to_utf8(buffer.data, cast(s32) written,, temp);
|
||||
}
|
||||
|
||||
OcrGetEnvironmentVariableW :: (name: *u16, buffer: *u16, size: u32) -> u32 #foreign kernel32 "GetEnvironmentVariableW";
|
||||
OcrSetEnvironmentVariableW :: (name: *u16, value: *u16) -> s32 #foreign kernel32 "SetEnvironmentVariableW";
|
||||
}
|
||||
|
||||
@ -1,19 +1,26 @@
|
||||
// Detection: where the text lines are. The detector (a DB network) gives each pixel a probability
|
||||
// of being text. Pixels above det_threshold are grouped into connected regions, each kept region
|
||||
// becomes a box, and the box is grown by the DB unclip distance, since the network is trained to
|
||||
// mark a shrunken core of each line.
|
||||
// of being text. Pixels above det_threshold are grouped into connected regions, and each kept region
|
||||
// becomes the smallest rectangle around it, at whatever angle, grown by the DB unclip distance, since
|
||||
// the network is trained to mark a shrunken core of each line.
|
||||
//
|
||||
// Images wider than det_max_side are scaled down for detection. Images taller than det_tile_height
|
||||
// (after scaling) are detected in overlapping strips, so a long scrolling capture keeps its text at
|
||||
// full size; each box is kept by the strip whose own band holds its centre.
|
||||
//
|
||||
// Version 1 finds horizontal text only, as screenshots have: boxes are axis-aligned.
|
||||
|
||||
Detected_Box :: struct {
|
||||
x0, y0, x1, y1: s32; // in the image, x1 and y1 exclusive
|
||||
// In the image's pixels, clockwise: the text's top left, top right, bottom right, bottom left, as
|
||||
// far as its shape tells. A box at least TURN_RATIO times as tall as it is wide is taken as a line
|
||||
// read downwards, so its first corner is its top right. Recognition may still turn it by 180°.
|
||||
corners: [4] Vector2;
|
||||
turned: bool; // taken as a line read downwards: started from its top right
|
||||
score: float;
|
||||
}
|
||||
|
||||
TURN_RATIO :: 1.5;
|
||||
|
||||
// Screenshots are mostly level: a region keeps a level box unless a turned one is this much smaller.
|
||||
LEVEL_PREFERENCE :: 1.2; // a long line tilted by 1° already has a level box ~30% bigger
|
||||
|
||||
DET_TILE_OVERLAP : s32 : 128; // px of each strip shared with the next, after scaling
|
||||
|
||||
// The image's text boxes, from the default allocator.
|
||||
@ -84,48 +91,180 @@ ocr_detect_tile :: (engine: *Ocr_Engine, image: Ocr_Image, scale: float, scaled_
|
||||
}
|
||||
|
||||
regions := ocr_find_regions(out.data, w, valid_w, valid_h, model.det_threshold);
|
||||
defer array_free(regions,, context.default_allocator);
|
||||
defer {
|
||||
for regions array_free(it.outline);
|
||||
array_free(regions,, context.default_allocator);
|
||||
}
|
||||
for regions {
|
||||
if it.pixels == 0 continue;
|
||||
score := cast(float) (it.probability_sum / it.pixels);
|
||||
if score < model.box_threshold continue;
|
||||
bw := it.x1 - it.x0;
|
||||
bh := it.y1 - it.y0;
|
||||
if min(bw, bh) < 3 continue; // specks
|
||||
|
||||
rect := ocr_min_area_rect(it.outline);
|
||||
rect_w := rect.u_max - rect.u_min;
|
||||
rect_h := rect.v_max - rect.v_min;
|
||||
if min(rect_w, rect_h) < 3 continue; // specks
|
||||
|
||||
// Grown as DB's unclip does: by area * ratio / perimeter on every side.
|
||||
distance := (cast(float) bw * bh * model.unclip_ratio) / (2.0 * (bw + bh));
|
||||
gx0 := it.x0 - distance;
|
||||
gy0 := it.y0 - distance;
|
||||
gx1 := it.x1 + distance;
|
||||
gy1 := it.y1 + distance;
|
||||
distance := (rect_w * rect_h * model.unclip_ratio) / (2.0 * (rect_w + rect_h));
|
||||
rect.u_min -= distance; rect.u_max += distance;
|
||||
rect.v_min -= distance; rect.v_max += distance;
|
||||
corners := ocr_rect_corners(rect);
|
||||
|
||||
centre_y := tile_top + (it.y0 + it.y1) / 2.0;
|
||||
centre_y := tile_top + (corners[0].y + corners[2].y) / 2;
|
||||
if centre_y < cast(float) band_top || centre_y >= cast(float) band_bottom continue; // the neighbouring strip's
|
||||
|
||||
box: Detected_Box;
|
||||
box.x0 = clamp(cast(s32) (gx0 * inverse), 0, image.width);
|
||||
box.x1 = clamp(cast(s32) (gx1 * inverse + 0.999), 0, image.width);
|
||||
box.y0 = clamp(cast(s32) ((gy0 + tile_top) * inverse), 0, image.height);
|
||||
box.y1 = clamp(cast(s32) ((gy1 + tile_top) * inverse + 0.999), 0, image.height);
|
||||
box.score = score;
|
||||
if box.x1 - box.x0 >= 2 && box.y1 - box.y0 >= 2 array_add(boxes, box);
|
||||
for corners {
|
||||
box.corners[it_index].x = clamp(it.x * inverse, 0, cast(float) image.width);
|
||||
box.corners[it_index].y = clamp((it.y + tile_top) * inverse, 0, cast(float) image.height);
|
||||
}
|
||||
if rect.u.x == 1 && rect.u.y == 0 {
|
||||
// A level box takes in every pixel it touches: the recognizers read these crops best.
|
||||
x0, y0, x1, y1 := ocr_corner_bounds(box.corners, image.width, image.height);
|
||||
box.corners = .[.{xx x0, xx y0}, .{xx x1, xx y0}, .{xx x1, xx y1}, .{xx x0, xx y1}];
|
||||
}
|
||||
box.corners, box.turned = ocr_reading_order(box.corners);
|
||||
x0, y0, x1, y1 := ocr_corner_bounds(box.corners, image.width, image.height);
|
||||
if x1 - x0 >= 2 && y1 - y0 >= 2 array_add(boxes, box);
|
||||
}
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// The pixels the corners cover, clipped to the image: x0 and y0 inclusive, x1 and y1 exclusive.
|
||||
ocr_corner_bounds :: (corners: [4] Vector2, image_width: s32, image_height: s32) -> x0: s32, y0: s32, x1: s32, y1: s32 {
|
||||
low := corners[0];
|
||||
high := corners[0];
|
||||
for corners {
|
||||
low.x = min(low.x, it.x); low.y = min(low.y, it.y);
|
||||
high.x = max(high.x, it.x); high.y = max(high.y, it.y);
|
||||
}
|
||||
return clamp(cast(s32) floor(low.x), 0, image_width), clamp(cast(s32) floor(low.y), 0, image_height),
|
||||
clamp(cast(s32) ceil(high.x), 0, image_width), clamp(cast(s32) ceil(high.y), 0, image_height);
|
||||
}
|
||||
|
||||
// The same rectangle's corners started from another one, keeping them clockwise: by 1, turned a
|
||||
// quarter (what was the right side becomes the top), by 2, upside down.
|
||||
ocr_corners_turned :: (corners: [4] Vector2, quarters: s64) -> [4] Vector2 {
|
||||
result: [4] Vector2;
|
||||
for 0..3 result[it] = corners[(it + quarters) % 4];
|
||||
return result;
|
||||
}
|
||||
|
||||
// A rectangle as ranges along two perpendicular unit axes: u, and v = u turned a quarter clockwise
|
||||
// on the screen (y down).
|
||||
Rotated_Rect :: struct {
|
||||
u: Vector2;
|
||||
u_min, u_max, v_min, v_max: float;
|
||||
}
|
||||
|
||||
ocr_rect_corners :: (rect: Rotated_Rect) -> [4] Vector2 {
|
||||
u := rect.u;
|
||||
v := Vector2.{-u.y, u.x};
|
||||
return .[u * rect.u_min + v * rect.v_min, u * rect.u_max + v * rect.v_min,
|
||||
u * rect.u_max + v * rect.v_max, u * rect.u_min + v * rect.v_max];
|
||||
}
|
||||
|
||||
// The corners clockwise from the text's top left, as PaddleOCR orders them: of the two leftmost, the
|
||||
// higher is the top left; of the two rightmost, the higher is the top right. A box at least
|
||||
// TURN_RATIO times as tall as it is wide is then started from its top right, so a line read
|
||||
// downwards comes out level.
|
||||
ocr_reading_order :: (corners: [4] Vector2) -> [4] Vector2, turned: bool {
|
||||
sorted := corners;
|
||||
quick_sort(sorted, (a: Vector2, b: Vector2) -> s64 { return ifx a.x < b.x then -1 else ifx a.x > b.x then 1 else 0; });
|
||||
top_left, bottom_left := sorted[0], sorted[1];
|
||||
if bottom_left.y < top_left.y swap(*top_left, *bottom_left);
|
||||
top_right, bottom_right := sorted[2], sorted[3];
|
||||
if bottom_right.y < top_right.y swap(*top_right, *bottom_right);
|
||||
result := Vector2.[top_left, top_right, bottom_right, bottom_left];
|
||||
|
||||
width := length(result[1] - result[0]);
|
||||
height := length(result[3] - result[0]);
|
||||
if height >= width * TURN_RATIO return ocr_corners_turned(result, 1), true;
|
||||
return result, false;
|
||||
}
|
||||
|
||||
// The smallest rectangle holding the points, at any angle, found by rotating calipers: one side of
|
||||
// it lies along an edge of the points' convex hull. A level rectangle is kept unless a turned one is
|
||||
// smaller by more than LEVEL_PREFERENCE.
|
||||
ocr_min_area_rect :: (points: [] Vector2) -> Rotated_Rect {
|
||||
level: Rotated_Rect;
|
||||
level.u = .{1, 0};
|
||||
level.u_min, level.v_min = FLOAT32_MAX, FLOAT32_MAX;
|
||||
level.u_max, level.v_max = -FLOAT32_MAX, -FLOAT32_MAX;
|
||||
for points {
|
||||
level.u_min = min(level.u_min, it.x); level.u_max = max(level.u_max, it.x);
|
||||
level.v_min = min(level.v_min, it.y); level.v_max = max(level.v_max, it.y);
|
||||
}
|
||||
if points.count < 3 return level;
|
||||
level_area := (level.u_max - level.u_min) * (level.v_max - level.v_min);
|
||||
|
||||
hull := ocr_convex_hull(points);
|
||||
best := level;
|
||||
best_area := level_area;
|
||||
for i: 0..hull.count - 1 {
|
||||
edge := hull[(i + 1) % hull.count] - hull[i];
|
||||
edge_length := length(edge);
|
||||
if edge_length < 0.0001 continue;
|
||||
rect: Rotated_Rect;
|
||||
rect.u = edge / edge_length;
|
||||
v := Vector2.{-rect.u.y, rect.u.x};
|
||||
rect.u_min, rect.v_min = FLOAT32_MAX, FLOAT32_MAX;
|
||||
rect.u_max, rect.v_max = -FLOAT32_MAX, -FLOAT32_MAX;
|
||||
for hull {
|
||||
a := dot(it, rect.u);
|
||||
b := dot(it, v);
|
||||
rect.u_min = min(rect.u_min, a); rect.u_max = max(rect.u_max, a);
|
||||
rect.v_min = min(rect.v_min, b); rect.v_max = max(rect.v_max, b);
|
||||
}
|
||||
area := (rect.u_max - rect.u_min) * (rect.v_max - rect.v_min);
|
||||
if area < best_area { best = rect; best_area = area; }
|
||||
}
|
||||
if level_area <= best_area * LEVEL_PREFERENCE return level;
|
||||
return best;
|
||||
}
|
||||
|
||||
// The convex hull of the points, counter-clockwise (Andrew's monotone chain). Temp.
|
||||
ocr_convex_hull :: (points: [] Vector2) -> [] Vector2 {
|
||||
sorted := array_copy(points,, temp);
|
||||
quick_sort(sorted, (a: Vector2, b: Vector2) -> s64 {
|
||||
if a.x != b.x return ifx a.x < b.x then -1 else 1;
|
||||
return ifx a.y < b.y then -1 else ifx a.y > b.y then 1 else 0;
|
||||
});
|
||||
cross :: (o: Vector2, a: Vector2, b: Vector2) -> float { return (a.x - o.x) * (b.y - o.y) - (a.y - o.y) * (b.x - o.x); }
|
||||
|
||||
hull := NewArray(sorted.count * 2, Vector2,, temp);
|
||||
k := 0;
|
||||
for sorted { // lower hull
|
||||
while k >= 2 && cross(hull[k - 2], hull[k - 1], it) <= 0 k -= 1;
|
||||
hull[k] = it; k += 1;
|
||||
}
|
||||
lower := k + 1;
|
||||
for < i: 0..sorted.count - 2 { // upper hull
|
||||
p := sorted[i];
|
||||
while k >= lower && cross(hull[k - 2], hull[k - 1], p) <= 0 k -= 1;
|
||||
hull[k] = p; k += 1;
|
||||
}
|
||||
hull.count = max(k - 1, 1); // the last point is the first again
|
||||
return hull;
|
||||
}
|
||||
|
||||
align_up_32 :: (n: s32) -> s32 {
|
||||
return (n + 31) & cast(s32) ~31;
|
||||
}
|
||||
|
||||
Region :: struct {
|
||||
x0, y0, x1, y1: s32; // x1 and y1 exclusive
|
||||
pixels: s64;
|
||||
probability_sum: float64;
|
||||
// The corners of the first and last pixel in each of its rows: what its convex hull needs.
|
||||
outline: [..] Vector2; // default allocator
|
||||
row: s32 = -1; // the row being scanned, and its first and last pixel
|
||||
row_first, row_last: s32;
|
||||
}
|
||||
|
||||
// Groups the pixels above threshold into 8-connected regions: one pass assigning provisional labels
|
||||
// and recording which touch (union-find), then one summing each region's extent and probability.
|
||||
// and recording which touch (union-find), then one summing each region's probability and outline.
|
||||
ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32, threshold: float) -> [..] Region {
|
||||
regions: [..] Region;
|
||||
regions.allocator = context.default_allocator;
|
||||
@ -149,6 +288,14 @@ ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32,
|
||||
else if rb < ra parent[ra] = rb;
|
||||
}
|
||||
|
||||
flush_row :: (r: *Region) {
|
||||
if r.row < 0 return;
|
||||
top, bottom := cast(float) r.row, cast(float) r.row + 1;
|
||||
array_add(*r.outline, .{cast(float) r.row_first, top}, .{cast(float) r.row_first, bottom},
|
||||
.{cast(float) r.row_last + 1, top}, .{cast(float) r.row_last + 1, bottom});
|
||||
r.row = -1;
|
||||
}
|
||||
|
||||
for y: 0..height - 1 {
|
||||
for x: 0..width - 1 {
|
||||
if probability[cast(s64) y * stride + x] <= threshold continue;
|
||||
@ -184,16 +331,21 @@ ocr_find_regions :: (probability: *float, stride: s32, width: s32, height: s32,
|
||||
index := region_of[root] - 1;
|
||||
if index < 0 {
|
||||
region := array_add(*regions);
|
||||
region.* = .{x0 = x, y0 = y, x1 = x + 1, y1 = y + 1};
|
||||
region.* = .{};
|
||||
region.outline.allocator = context.default_allocator;
|
||||
index = cast(s32) regions.count - 1;
|
||||
region_of[root] = index + 1;
|
||||
}
|
||||
r := *regions[index];
|
||||
r.x0 = min(r.x0, x); r.x1 = max(r.x1, x + 1);
|
||||
r.y0 = min(r.y0, y); r.y1 = max(r.y1, y + 1);
|
||||
if r.row != y {
|
||||
flush_row(r);
|
||||
r.row, r.row_first = y, x;
|
||||
}
|
||||
r.row_last = x; // rows are scanned left to right
|
||||
r.pixels += 1;
|
||||
r.probability_sum += probability[cast(s64) y * stride + x];
|
||||
}
|
||||
}
|
||||
for * regions flush_row(it);
|
||||
return regions;
|
||||
}
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
// are not in git; run this after cloning. Files already there with the right hash are skipped.
|
||||
//
|
||||
// jai fetch_models.jai every model
|
||||
// jai fetch_models.jai - mobile only the files ppocr-v5-mobile needs (about 21 MB)
|
||||
// jai fetch_models.jai - mobile only the files ppocr-v5-mobile needs (about 22 MB)
|
||||
//
|
||||
// From RapidOCR's conversions of PaddleOCR's PP-OCRv5 (both Apache 2.0):
|
||||
// https://www.modelscope.cn/models/RapidAI/RapidOCR
|
||||
@ -15,6 +15,7 @@ FILES :: Model_File.[
|
||||
.{"ch_PP-OCRv5_det_mobile.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_mobile.onnx", "4d97c44a20d30a81aad087d6a396b08f786c4635742afc391f6621f5c6ae78ae", false},
|
||||
.{"ch_PP-OCRv5_rec_mobile.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile.onnx", "5825fc7ebf84ae7a412be049820b4d86d77620f204a041697b0494669b1742c5", false},
|
||||
.{"ppocrv5_dict.txt", "paddle/PP-OCRv5/rec/ch_PP-OCRv5_rec_mobile/ppocrv5_dict.txt", "d1979e9f794c464c0d2e0b70a7fe14dd978e9dc644c0e71f14158cdf8342af1b", false},
|
||||
.{"ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx", "onnx/PP-OCRv5/cls/ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx", "54379ae5174d026780215fc748a7f31910dee36818e63d49e17dc598ecc82df7", false},
|
||||
.{"ch_PP-OCRv5_det_server.onnx", "onnx/PP-OCRv5/det/ch_PP-OCRv5_det_server.onnx", "0f8846b1d4bba223a2a2f9d9b44022fbc22cc019051a602b41a7fda9667e4cad", true},
|
||||
.{"ch_PP-OCRv5_rec_server.onnx", "onnx/PP-OCRv5/rec/ch_PP-OCRv5_rec_server.onnx", "e09385400eaaaef34ceff54aeb7c4f0f1fe014c27fa8b9905d4709b65746562a", true},
|
||||
];
|
||||
|
||||
@ -5,6 +5,8 @@
|
||||
#
|
||||
# title shown to people
|
||||
# detector, recognizer, dictionary file names, quoted
|
||||
# textline_ori optional: a text line orientation classifier's file name, quoted; lines it
|
||||
# finds upside down are read both ways and the more confident reading kept
|
||||
# det_max_side wider images are scaled down to this width for detection (px)
|
||||
# det_tile_height taller images are detected in overlapping strips this high (px, after scaling)
|
||||
# det_threshold a pixel is text when its probability is above this
|
||||
@ -18,6 +20,7 @@ title: "PP-OCRv5 mobile (light)"
|
||||
detector: "ch_PP-OCRv5_det_mobile.onnx"
|
||||
recognizer: "ch_PP-OCRv5_rec_mobile.onnx"
|
||||
dictionary: "ppocrv5_dict.txt"
|
||||
textline_ori: "ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx"
|
||||
det_max_side: 2560
|
||||
det_tile_height: 1536
|
||||
det_threshold: 0.3
|
||||
@ -31,6 +34,7 @@ title: "PP-OCRv5 server (heavy)"
|
||||
detector: "ch_PP-OCRv5_det_server.onnx"
|
||||
recognizer: "ch_PP-OCRv5_rec_server.onnx"
|
||||
dictionary: "ppocrv5_dict.txt"
|
||||
textline_ori: "ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx"
|
||||
det_max_side: 2560
|
||||
det_tile_height: 1536
|
||||
det_threshold: 0.3
|
||||
|
||||
@ -5,6 +5,7 @@ Ocr_Model_Info :: struct {
|
||||
name: string;
|
||||
title: string;
|
||||
detector, recognizer, dictionary: string; // file names in the models folder
|
||||
textline_ori: string; // the orientation classifier's file name; "" for none
|
||||
det_max_side: s32 = 2560;
|
||||
det_tile_height: s32 = 1536;
|
||||
det_threshold: float = 0.3;
|
||||
@ -53,6 +54,7 @@ ocr_list_models :: (models_folder: string) -> [] Ocr_Model_Info, error: string {
|
||||
case "detector"; model.detector = copy_string(value);
|
||||
case "recognizer"; model.recognizer = copy_string(value);
|
||||
case "dictionary"; model.dictionary = copy_string(value);
|
||||
case "textline_ori"; model.textline_ori = copy_string(value);
|
||||
case "det_max_side"; if number_ok model.det_max_side = cast(s32) number;
|
||||
case "det_tile_height"; if number_ok model.det_tile_height = cast(s32) number;
|
||||
case "det_threshold"; if number_ok model.det_threshold = number;
|
||||
@ -82,6 +84,35 @@ ocr_strip_comment :: (line: string) -> string {
|
||||
Ocr_Session :: struct {
|
||||
session: *OrtSession;
|
||||
input_name, output_name: *u8; // heap C strings
|
||||
input_shape: [4] s64; // as the model declares it: -1 for a size it leaves open
|
||||
input_rank: s64;
|
||||
}
|
||||
|
||||
// The model's sessions on the GPU or the CPU: detector, recognizer and, when it has one, orientation
|
||||
// classifier. On failure, the reason; the sessions made so far are left for ocr_release_sessions.
|
||||
ocr_create_sessions :: (engine: *Ocr_Engine, models_folder: string, gpu: bool, intra_op_threads: s32) -> ok: bool, error: string {
|
||||
model := *engine.model;
|
||||
ok: bool;
|
||||
error: string;
|
||||
engine.det, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.detector), gpu, intra_op_threads);
|
||||
if !ok return false, error;
|
||||
engine.rec, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.recognizer), gpu, intra_op_threads);
|
||||
if !ok return false, error;
|
||||
if model.textline_ori {
|
||||
engine.orientation, ok, error = ocr_create_session(engine.env, tprint("%/%", models_folder, model.textline_ori), gpu, intra_op_threads);
|
||||
if !ok return false, error;
|
||||
// PP-LCNet text line orientation takes 160 by 80; a model that says otherwise is taken at its word.
|
||||
shape := engine.orientation.input_shape;
|
||||
engine.orientation_height = ifx engine.orientation.input_rank == 4 && shape[2] > 0 then cast(s32) shape[2] else 80;
|
||||
engine.orientation_width = ifx engine.orientation.input_rank == 4 && shape[3] > 0 then cast(s32) shape[3] else 160;
|
||||
}
|
||||
return true, "";
|
||||
}
|
||||
|
||||
ocr_release_sessions :: (engine: *Ocr_Engine) {
|
||||
ocr_release_session(*engine.det);
|
||||
ocr_release_session(*engine.rec);
|
||||
ocr_release_session(*engine.orientation);
|
||||
}
|
||||
|
||||
// A session for the model file; with CUDA when gpu is set. On failure, the reason.
|
||||
@ -124,6 +155,18 @@ ocr_create_session :: (env: *OrtEnv, path: string, gpu: bool, intra_op_threads:
|
||||
ort_check(api.SessionGetOutputName(result.session, 0, allocator, *name));
|
||||
result.output_name = to_c_string(to_string(name),, context.default_allocator);
|
||||
ort_check(api.AllocatorFree(allocator, name));
|
||||
|
||||
input_type: *OrtTypeInfo;
|
||||
if ort_check(api.SessionGetInputTypeInfo(result.session, 0, *input_type)) {
|
||||
tensor_info: *OrtTensorTypeAndShapeInfo; // belongs to input_type
|
||||
if ort_check(api.CastTypeInfoToTensorInfo(input_type, *tensor_info)) && tensor_info {
|
||||
rank: u64;
|
||||
ort_check(api.GetDimensionsCount(tensor_info, *rank));
|
||||
result.input_rank = min(cast(s64) rank, result.input_shape.count);
|
||||
ort_check(api.GetDimensions(tensor_info, result.input_shape.data, xx result.input_rank));
|
||||
}
|
||||
api.ReleaseTypeInfo(input_type);
|
||||
}
|
||||
return result, true, "";
|
||||
}
|
||||
|
||||
|
||||
@ -8,8 +8,9 @@
|
||||
// with onnxruntime (modules/onnxruntime), on an NVIDIA GPU through CUDA when it can, else on the
|
||||
// CPU. ocr_gpu_status says why the GPU is not in use, and what to install so it can be.
|
||||
//
|
||||
// Version 1 runs PP-OCRv5's two-stage pipeline: detection (detect.jai), then recognition
|
||||
// (recognize.jai). Text is expected horizontal, as in screenshots.
|
||||
// It runs PP-OCRv5's pipeline: detection (detect.jai) finds each line as a rectangle at any angle,
|
||||
// then recognition (recognize.jai) reads it level. When the model lists an orientation classifier,
|
||||
// lines that look upside down are read both ways; lines read downwards or upwards are read too.
|
||||
|
||||
Ocr_Provider :: enum { CPU; CUDA; }
|
||||
|
||||
@ -20,9 +21,10 @@ Ocr_Image :: struct {
|
||||
}
|
||||
|
||||
Ocr_Line :: struct {
|
||||
x, y, width, height: s32; // in the image
|
||||
x, y, width, height: s32; // in the image: the bounds of corners
|
||||
text: string;
|
||||
confidence: float; // 0 to 1: the mean probability of its characters
|
||||
corners: [4] Vector2; // in the image's pixels: the text's top left, top right, bottom right, bottom left
|
||||
}
|
||||
|
||||
Ocr_Result :: struct {
|
||||
@ -38,6 +40,8 @@ Ocr_Engine :: struct {
|
||||
env: *OrtEnv;
|
||||
memory_info: *OrtMemoryInfo;
|
||||
det, rec: Ocr_Session;
|
||||
orientation: Ocr_Session; // the text line orientation classifier; none when its session is null
|
||||
orientation_width, orientation_height: s32; // its input size
|
||||
dictionary: [] string; // class n is dictionary[n - 1]
|
||||
workers: s32; // threads ocr_process uses
|
||||
stats: Ocr_Stats; // since creation, or the last ocr_reset_stats
|
||||
@ -49,6 +53,8 @@ Ocr_Stats :: struct {
|
||||
lines: s64;
|
||||
detect_us: s64; // preparing images, the detector, and finding the boxes
|
||||
recognize_us: s64; // cropping lines, the recognizer, and decoding the text
|
||||
orientation_us: s64; // cropping lines and the orientation classifier
|
||||
turned: s64; // lines that read better the other way up, or level as found
|
||||
}
|
||||
|
||||
ocr_stats :: (engine: *Ocr_Engine) -> Ocr_Stats { return engine.stats; }
|
||||
@ -84,8 +90,6 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
|
||||
ort_check(api.CreateCpuMemoryInfo(.OrtArenaAllocator, .OrtMemTypeDefault, *engine.memory_info));
|
||||
|
||||
cores := cast(s32) max(get_number_of_processors(.ALL_LOGICAL), 1); // efficiency cores included
|
||||
detector_path := tprint("%/%", models_folder, model.detector);
|
||||
recognizer_path := tprint("%/%", models_folder, model.recognizer);
|
||||
|
||||
use_gpu := false;
|
||||
if prefer_gpu {
|
||||
@ -99,16 +103,12 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
|
||||
if use_gpu {
|
||||
// A few CPU threads prepare images and decode results while the GPU runs the networks.
|
||||
engine.workers = 4;
|
||||
det_ok, rec_ok: bool;
|
||||
det_error, rec_error: string;
|
||||
engine.det, det_ok, det_error = ocr_create_session(engine.env, detector_path, true, 2);
|
||||
if det_ok engine.rec, rec_ok, rec_error = ocr_create_session(engine.env, recognizer_path, true, 2);
|
||||
if det_ok && rec_ok {
|
||||
sessions_ok, sessions_error := ocr_create_sessions(engine, models_folder, true, 2);
|
||||
if sessions_ok {
|
||||
engine.provider = .CUDA;
|
||||
} else {
|
||||
ocr_release_session(*engine.det);
|
||||
ocr_release_session(*engine.rec);
|
||||
engine.gpu_explanation = sprint("OCR runs on the CPU: the CUDA libraries were found, but onnxruntime could not start on the GPU:\n%", ifx det_ok then rec_error else det_error);
|
||||
ocr_release_sessions(engine);
|
||||
engine.gpu_explanation = sprint("OCR runs on the CPU: the CUDA libraries were found, but onnxruntime could not start on the GPU:\n%", sessions_error);
|
||||
use_gpu = false;
|
||||
}
|
||||
}
|
||||
@ -116,12 +116,8 @@ ocr_create :: (models_folder: string, model_name: string, prefer_gpu := true) ->
|
||||
// Several images at once, each network run spread over the rest of the cores.
|
||||
engine.workers = clamp(cores / 4, 1, 8);
|
||||
threads := max(cores / engine.workers, 1);
|
||||
det_ok, rec_ok: bool;
|
||||
det_error, rec_error: string;
|
||||
engine.det, det_ok, det_error = ocr_create_session(engine.env, detector_path, false, threads);
|
||||
if !det_ok { ocr_destroy(engine); return null, false, det_error; }
|
||||
engine.rec, rec_ok, rec_error = ocr_create_session(engine.env, recognizer_path, false, threads);
|
||||
if !rec_ok { ocr_destroy(engine); return null, false, rec_error; }
|
||||
sessions_ok, sessions_error := ocr_create_sessions(engine, models_folder, false, threads);
|
||||
if !sessions_ok { ocr_destroy(engine); return null, false, sessions_error; }
|
||||
engine.provider = .CPU;
|
||||
}
|
||||
return engine, true, "";
|
||||
@ -138,8 +134,7 @@ ocr_gpu_status :: (engine: *Ocr_Engine) -> in_use: bool, explanation: string {
|
||||
|
||||
ocr_destroy :: (engine: *Ocr_Engine) {
|
||||
if !engine return;
|
||||
ocr_release_session(*engine.det);
|
||||
ocr_release_session(*engine.rec);
|
||||
ocr_release_sessions(engine);
|
||||
if engine.memory_info ort.api.ReleaseMemoryInfo(engine.memory_info);
|
||||
if engine.env ort.api.ReleaseEnv(engine.env);
|
||||
free(engine.gpu_explanation,, context.default_allocator);
|
||||
@ -174,10 +169,49 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
|
||||
for jobs free(it.text,, context.default_allocator);
|
||||
array_free(jobs);
|
||||
}
|
||||
rec_height := engine.model.rec_height;
|
||||
turned: [..] bool; // per first job: whether detection turned its box a quarter
|
||||
turned.allocator = context.default_allocator;
|
||||
defer array_free(turned);
|
||||
for work: per_image {
|
||||
if !work.ok continue;
|
||||
for work.boxes array_add(*jobs, .{image = work.image, box = it, width = ocr_line_width(it, engine.model.rec_height)});
|
||||
for work.boxes {
|
||||
array_add(*jobs, .{image = work.image, corners = it.corners, width = ocr_line_width(it.corners, rec_height)});
|
||||
array_add(*turned, it.turned);
|
||||
}
|
||||
}
|
||||
|
||||
// Other ways to read a line, recognized beside it, the more confident reading kept: upside down,
|
||||
// where the orientation classifier finds it more likely than not; and level as it was found, for a
|
||||
// box detection turned a quarter, since a narrow word such as "1" is as tall as a line read downwards.
|
||||
first_count := jobs.count;
|
||||
upside_down := NewArray(first_count, bool,, context.default_allocator);
|
||||
defer array_free(upside_down,, context.default_allocator);
|
||||
Orientation_Context :: struct { jobs: [] Line_Job; upside_down: [] bool; size: s64; failed: bool; error: string; }
|
||||
classify := Orientation_Context.{jobs = jobs, upside_down = upside_down, size = max(engine.model.rec_batch, 1)};
|
||||
if engine.orientation.session && first_count {
|
||||
run_on_workers(engine, (first_count + classify.size - 1) / classify.size, (engine: *Ocr_Engine, index: s64, data: *void) {
|
||||
c := cast(*Orientation_Context) data;
|
||||
first := index * c.size;
|
||||
count := min(c.size, c.jobs.count - first);
|
||||
batch := NewArray(count, *Line_Job,, temp);
|
||||
for * batch it.* = *c.jobs[first + it_index];
|
||||
start := current_time_monotonic();
|
||||
ok, error := ocr_classify_batch(engine, batch, array_view(c.upside_down, first, count));
|
||||
atomic_add(*engine.stats.orientation_us, to_microseconds(current_time_monotonic() - start));
|
||||
if !ok { c.failed = true; c.error = copy_string(error,, context.default_allocator); } // the last error wins
|
||||
}, *classify);
|
||||
}
|
||||
defer free(classify.error,, context.default_allocator);
|
||||
for n: 0..first_count - 1 {
|
||||
job := jobs[n];
|
||||
if upside_down[n] array_add(*jobs, .{image = job.image, corners = ocr_corners_turned(job.corners, 2), width = job.width, primary = n});
|
||||
if turned[n] {
|
||||
level := ocr_corners_turned(job.corners, 3);
|
||||
array_add(*jobs, .{image = job.image, corners = level, width = ocr_line_width(level, rec_height), primary = n});
|
||||
}
|
||||
}
|
||||
|
||||
order := NewArray(jobs.count, *Line_Job,, context.default_allocator);
|
||||
defer array_free(order,, context.default_allocator);
|
||||
for * jobs order[it_index] = it;
|
||||
@ -196,6 +230,20 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
|
||||
atomic_add(*engine.stats.recognize_us, to_microseconds(current_time_monotonic() - start));
|
||||
if !ok { b.failed = true; b.error = copy_string(error,, context.default_allocator); } // the last error wins; any is enough to say why
|
||||
}, *batches);
|
||||
if classify.failed && !batches.failed {
|
||||
batches.failed = true;
|
||||
batches.error = copy_string(classify.error,, context.default_allocator);
|
||||
}
|
||||
|
||||
// The more confident of each line's readings takes the first job's place.
|
||||
for * other: array_view(jobs, first_count, jobs.count - first_count) {
|
||||
first := *jobs[other.primary];
|
||||
if other.confidence > first.confidence {
|
||||
swap(*first.text, *other.text);
|
||||
first.corners, first.confidence = other.corners, other.confidence;
|
||||
engine.stats.turned += 1;
|
||||
}
|
||||
}
|
||||
|
||||
// Each image's lines, in reading order.
|
||||
for work, image_index: per_image {
|
||||
@ -213,10 +261,11 @@ ocr_process :: (engine: *Ocr_Engine, images: [] Ocr_Image, allocator := context.
|
||||
}
|
||||
lines: [..] Ocr_Line;
|
||||
lines.allocator = allocator;
|
||||
for job: jobs {
|
||||
for job: array_view(jobs, 0, first_count) {
|
||||
if job.image != work.image || !job.text.count continue;
|
||||
array_add(*lines, .{x = job.box.x0, y = job.box.y0, width = job.box.x1 - job.box.x0, height = job.box.y1 - job.box.y0,
|
||||
text = copy_string(job.text,, allocator), confidence = job.confidence});
|
||||
x0, y0, x1, y1 := ocr_corner_bounds(job.corners, job.image.width, job.image.height);
|
||||
array_add(*lines, .{x = x0, y = y0, width = x1 - x0, height = y1 - y0, text = copy_string(job.text,, allocator),
|
||||
confidence = job.confidence, corners = job.corners});
|
||||
}
|
||||
// Rows by the middle of each line, then left to right within a row.
|
||||
quick_sort(lines, (a: Ocr_Line, b: Ocr_Line) -> s64 {
|
||||
|
||||
@ -1,13 +1,18 @@
|
||||
// Recognition: the text of each detected line. Each box is cropped from the image at full size,
|
||||
// scaled to rec_height keeping its shape, and normalized to -1..1 in BGR order, as PaddleOCR's
|
||||
// Recognition: the text of each detected line. Each box is sampled along its own sides into a level
|
||||
// crop rec_height high, keeping its shape, and normalized to -1..1 in BGR order, as PaddleOCR's
|
||||
// recognizers expect. Lines of similar width are batched, padded to the widest in the batch.
|
||||
// The output is a probability per character class per step; greedy CTC decoding takes the best
|
||||
// class at each step, merges repeats and drops blanks.
|
||||
//
|
||||
// A box's corners only say which way its text runs up to a half turn. The orientation classifier,
|
||||
// when the model has one, says which lines look upside down; such a line is read both ways, as is a
|
||||
// box taken as a line read downwards, and the more confident reading is kept.
|
||||
|
||||
Line_Job :: struct {
|
||||
image: *Ocr_Image;
|
||||
box: Detected_Box;
|
||||
corners: [4] Vector2; // as Detected_Box's
|
||||
width: s32; // after scaling to rec_height
|
||||
primary := -1; // for another way of reading a line: the index of the line's first job
|
||||
// Filled in by recognition:
|
||||
text: string; // default allocator
|
||||
confidence: float;
|
||||
@ -24,38 +29,8 @@ ocr_recognize_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job) -> ok: bool, er
|
||||
plane := cast(s64) height * width;
|
||||
input := NewArray(plane * 3 * jobs.count, float, initialized = true,, context.default_allocator); // 0 is the padding
|
||||
defer array_free(input,, context.default_allocator);
|
||||
|
||||
for job, n: jobs {
|
||||
base := input.data + plane * 3 * n;
|
||||
box := job.box;
|
||||
bw := box.x1 - box.x0;
|
||||
bh := box.y1 - box.y0;
|
||||
image := job.image;
|
||||
// Bilinear: the crops are mostly enlarged (small UI text up to 48 px), where nearest would be blocky.
|
||||
x_scale := cast(float) bw / job.width;
|
||||
y_scale := cast(float) bh / height;
|
||||
for y: 0..height - 1 {
|
||||
sy := (y + 0.5) * y_scale - 0.5 + box.y0;
|
||||
y0 := clamp(cast(s32) floor(sy), 0, image.height - 1);
|
||||
y1 := min(y0 + 1, image.height - 1);
|
||||
fy := clamp(sy - y0, 0, 1);
|
||||
row0 := image.pixels + cast(s64) y0 * image.stride;
|
||||
row1 := image.pixels + cast(s64) y1 * image.stride;
|
||||
for x: 0..job.width - 1 {
|
||||
sx := (x + 0.5) * x_scale - 0.5 + box.x0;
|
||||
x0 := clamp(cast(s32) floor(sx), 0, image.width - 1);
|
||||
x1 := min(x0 + 1, image.width - 1);
|
||||
fx := clamp(sx - x0, 0, 1);
|
||||
i := cast(s64) y * width + x;
|
||||
for c: 0..2 {
|
||||
channel := 2 - c; // B, G, R from RGBA
|
||||
top := row0[x0 * 4 + channel] * (1 - fx) + row0[x1 * 4 + channel] * fx;
|
||||
bottom := row1[x0 * 4 + channel] * (1 - fx) + row1[x1 * 4 + channel] * fx;
|
||||
value := top * (1 - fy) + bottom * fy;
|
||||
base[i + plane * c] = (value / 255.0 - 0.5) / 0.5;
|
||||
}
|
||||
}
|
||||
}
|
||||
ocr_sample_crop(job.image, job.corners, job.width, height, input.data + plane * 3 * n, width, plane, REC_MEAN, REC_STD);
|
||||
}
|
||||
|
||||
shape := s64.[jobs.count, 3, height, width];
|
||||
@ -95,11 +70,68 @@ ocr_recognize_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job) -> ok: bool, er
|
||||
return true, "";
|
||||
}
|
||||
|
||||
// The width a box is recognized at: rec_height high, keeping its shape.
|
||||
ocr_line_width :: (box: Detected_Box, rec_height: s32) -> s32 {
|
||||
bw := box.x1 - box.x0;
|
||||
bh := max(box.y1 - box.y0, 1);
|
||||
return clamp(cast(s32) (cast(float) bw * rec_height / bh + 0.5), 1, REC_MAX_WIDTH);
|
||||
// The width corners are recognized at: rec_height high, keeping their shape.
|
||||
ocr_line_width :: (corners: [4] Vector2, rec_height: s32) -> s32 {
|
||||
w := length(corners[1] - corners[0]);
|
||||
h := max(length(corners[3] - corners[0]), 1);
|
||||
return clamp(cast(s32) (w * rec_height / h + 0.5), 1, REC_MAX_WIDTH);
|
||||
}
|
||||
|
||||
REC_MAX_WIDTH :: 4096; // a line longer than this, at rec_height, is squeezed to fit
|
||||
|
||||
// The normalization the PP-OCR recognizers were trained with.
|
||||
REC_MEAN :: float.[0.5, 0.5, 0.5];
|
||||
REC_STD :: float.[0.5, 0.5, 0.5];
|
||||
|
||||
// Samples the rectangle the corners make (clockwise from its top left, at any angle) into a level
|
||||
// crop width by height, bilinearly: crops are mostly enlarged (small UI text up to 48 px), where
|
||||
// nearest would be blocky. Written planar in BGR order at dest, row_stride floats from one row to the
|
||||
// next and plane from one channel to the next, as (value / 255 - mean) / deviation.
|
||||
ocr_sample_crop :: (image: *Ocr_Image, corners: [4] Vector2, width: s32, height: s32, dest: *float, row_stride: s64, plane: s64, mean: [3] float, deviation: [3] float) {
|
||||
step_x := (corners[1] - corners[0]) / cast(float) width; // one crop pixel along the line
|
||||
step_y := (corners[3] - corners[0]) / cast(float) height; // and across it
|
||||
// Pixel centres: an image pixel's centre is at +0.5.
|
||||
origin := corners[0] + step_x * 0.5 + step_y * 0.5 - Vector2.{0.5, 0.5};
|
||||
for y: 0..height - 1 {
|
||||
row_start := origin + step_y * cast(float) y;
|
||||
for x: 0..width - 1 {
|
||||
s := row_start + step_x * cast(float) x;
|
||||
x0 := clamp(cast(s32) floor(s.x), 0, image.width - 1);
|
||||
y0 := clamp(cast(s32) floor(s.y), 0, image.height - 1);
|
||||
x1 := min(x0 + 1, image.width - 1);
|
||||
y1 := min(y0 + 1, image.height - 1);
|
||||
fx := clamp(s.x - x0, 0, 1);
|
||||
fy := clamp(s.y - y0, 0, 1);
|
||||
row0 := image.pixels + cast(s64) y0 * image.stride;
|
||||
row1 := image.pixels + cast(s64) y1 * image.stride;
|
||||
i := cast(s64) y * row_stride + x;
|
||||
for c: 0..2 {
|
||||
channel := 2 - c; // B, G, R from RGBA
|
||||
top := row0[x0 * 4 + channel] * (1 - fx) + row0[x1 * 4 + channel] * fx;
|
||||
bottom := row1[x0 * 4 + channel] * (1 - fx) + row1[x1 * 4 + channel] * fx;
|
||||
value := top * (1 - fy) + bottom * fy;
|
||||
dest[i + plane * c] = (value / 255.0 - mean[c]) / deviation[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The orientation classifier (PP-LCNet text line orientation) on one batch: sets upside_down[n] when
|
||||
// it finds jobs[n] more likely upside down than not. Its input is a fixed size, the line stretched to
|
||||
// fit, normalized as the detector's input is.
|
||||
ocr_classify_batch :: (engine: *Ocr_Engine, jobs: [] *Line_Job, upside_down: [] bool) -> ok: bool, error: string {
|
||||
if !jobs.count return true, "";
|
||||
width, height := engine.orientation_width, engine.orientation_height;
|
||||
plane := cast(s64) width * height;
|
||||
input := NewArray(plane * 3 * jobs.count, float,, context.default_allocator);
|
||||
defer array_free(input,, context.default_allocator);
|
||||
for job, n: jobs ocr_sample_crop(job.image, job.corners, width, height, input.data + plane * 3 * n, width, plane, DET_MEAN, DET_STD);
|
||||
|
||||
shape := s64.[jobs.count, 3, height, width];
|
||||
out, ok, error := ocr_run(*engine.orientation, engine.memory_info, input, shape);
|
||||
if !ok return false, error;
|
||||
defer ocr_release_output(*out);
|
||||
if out.dim_count != 2 || out.dims[0] != jobs.count || out.dims[1] != 2 return false, tprint("The orientation classifier gave an output of shape %, not [%, 2].", array_view(out.dims, 0, out.dim_count), jobs.count);
|
||||
for 0..jobs.count - 1 upside_down[it] = out.data[it * 2 + 1] > out.data[it * 2]; // classes: 0°, 180°
|
||||
return true, "";
|
||||
}
|
||||
|
||||
@ -9,12 +9,15 @@ USING_WINDOWS :: OS == .WINDOWS;
|
||||
#if COMPILE_TESTS {
|
||||
#load "test/testing.jai";
|
||||
#load "test/logger_tests.jai";
|
||||
#load "test/catalog_tests.jai";
|
||||
#load "test/ocr_tests.jai";
|
||||
}
|
||||
|
||||
#load "app/main.jai"; // entry point
|
||||
#load "app/app_ui.jai";
|
||||
#load "app/viewer.jai";
|
||||
#load "app/ocr_ui.jai";
|
||||
#load "app/catalog.jai";
|
||||
#load "app/exit_shutdown.jai";
|
||||
|
||||
#load "base/circle_buffer.jai";
|
||||
@ -50,6 +53,8 @@ Shared_Channel :: #import "Shared_Memory_Channel";
|
||||
#import "stb_image_resize";
|
||||
#import "Capture_File"; // the .mcap format, copied from D:\SnagitIndex\SnagitConverter
|
||||
#import "OCR";
|
||||
#import "Capture_Text";
|
||||
#import "Base64";
|
||||
#import "onnxruntime";
|
||||
|
||||
// Basic's arenas import Secure_Memory at its defaults; this import adds its test suite.
|
||||
@ -167,6 +172,8 @@ App :: struct {
|
||||
run_tests: bool;
|
||||
headless: bool; // -headless (or a no_crash_dialog build): no message boxes on failures, and run_tests exits when done
|
||||
exit_code: s32; // the process's exit code when the main loop ends
|
||||
library_folder: string; // -library FOLDER: the .mcap folder the viewer shows, in place of VIEWER_FOLDER
|
||||
scan_and_exit: bool; // -scan: scan the library with the chosen OCR model once it has loaded, then exit (0, or 1 if any capture failed)
|
||||
last_window_restore_time: Apollo_Time;
|
||||
|
||||
imgui_context: *ImGui.ImGuiContext;
|
||||
|
||||
289
src/app/catalog.jai
Normal file
289
src/app/catalog.jai
Normal file
@ -0,0 +1,289 @@
|
||||
// The catalog: what search and sorting need from every capture, loaded once from the front of each
|
||||
// file (its Capture Text and OCR text, not its images) on the viewer's worker threads.
|
||||
//
|
||||
// Entries are made on the workers and handed over through viewer.catalog_done; only the main thread
|
||||
// sets or frees viewer.captures[i].catalog, and only it reads them, so no entry is freed while in use.
|
||||
|
||||
Catalog_Entry :: struct {
|
||||
date: string; // CaptureDate, "2026-10-07 08:11:15" (sorts as text); "" when unknown
|
||||
title, url, app: string; // WindowName, WebURL, AppName
|
||||
notes: string; // the annotations' text, one per line
|
||||
ocr: [..] Mcap_Ocr_Section; // one per model that read it
|
||||
|
||||
// Lowercased copies, for matching without case.
|
||||
find_title, find_url, find_app, find_notes, find_ocr: string;
|
||||
}
|
||||
|
||||
// Everything the entry holds is from the default allocator.
|
||||
catalog_entry_free :: (entry: *Catalog_Entry) {
|
||||
if !entry return;
|
||||
push_allocator(context.default_allocator);
|
||||
free(entry.date); free(entry.title); free(entry.url); free(entry.app); free(entry.notes);
|
||||
for entry.ocr {
|
||||
free(it.model); free(it.engine); free(it.image); free(it.date);
|
||||
for line: it.lines free(line.text);
|
||||
array_free(it.lines);
|
||||
}
|
||||
array_free(entry.ocr);
|
||||
free(entry.find_title); free(entry.find_url); free(entry.find_app); free(entry.find_notes); free(entry.find_ocr);
|
||||
free(entry);
|
||||
}
|
||||
|
||||
// The entry for the .mcap at path, or null with why. On any thread.
|
||||
catalog_load :: (path: string) -> *Catalog_Entry, error: string {
|
||||
front: Mcap;
|
||||
ok: bool;
|
||||
error: string;
|
||||
{
|
||||
push_allocator(temp);
|
||||
front, ok, error = mcap_read_front(path, preview = false);
|
||||
}
|
||||
if !ok return null, error;
|
||||
|
||||
push_allocator(context.default_allocator);
|
||||
entry := New(Catalog_Entry);
|
||||
|
||||
// The metadata and the annotations, from the Capture Text.
|
||||
doc: Capture_Document;
|
||||
capture_document_init(*doc, temp);
|
||||
if capture_text_parse(*doc, path, front.text) {
|
||||
metadata := capture_find_file(*doc, "metadata.json");
|
||||
if metadata {
|
||||
entry.date = catalog_string(metadata.root, "CaptureDate");
|
||||
entry.title = catalog_string(metadata.root, "WindowName");
|
||||
entry.url = catalog_string(metadata.root, "WebURL");
|
||||
entry.app = catalog_string(metadata.root, "AppName");
|
||||
}
|
||||
notes: String_Builder;
|
||||
notes.allocator = temp;
|
||||
for file: doc.files {
|
||||
if file.name == "index.json" || file.name == "metadata.json" || ends_with(file.name, ".backup.json") continue;
|
||||
objects := capture_object_find(*file.root, "CaptureObjects");
|
||||
if !objects || objects.kind != .ARRAY continue;
|
||||
for object: objects.items {
|
||||
encoded := capture_object_find(*object, "RTFEncodedText");
|
||||
if !encoded || encoded.kind != .STRING || !encoded.text.count continue;
|
||||
rtf, decoded := base64_decode(encoded.text,, temp);
|
||||
if !decoded continue;
|
||||
text := rtf_to_text(rtf);
|
||||
if !text.count || text == "?" continue; // "?": an empty text box
|
||||
if builder_string_length(*notes) append(*notes, "\n");
|
||||
append(*notes, text);
|
||||
}
|
||||
}
|
||||
entry.notes = builder_to_string(*notes);
|
||||
}
|
||||
|
||||
sections, ocr_ok := mcap_ocr_parse(front.ocr);
|
||||
if ocr_ok entry.ocr = sections;
|
||||
|
||||
entry.find_title = to_lower_copy(entry.title);
|
||||
entry.find_url = to_lower_copy(entry.url);
|
||||
entry.find_app = to_lower_copy(entry.app);
|
||||
entry.find_notes = to_lower_copy(entry.notes);
|
||||
all_ocr: String_Builder;
|
||||
all_ocr.allocator = temp;
|
||||
for entry.ocr for line: it.lines { append(*all_ocr, line.text); append(*all_ocr, "\n"); }
|
||||
entry.find_ocr = to_lower_copy(builder_to_string(*all_ocr,, temp));
|
||||
return entry, "";
|
||||
}
|
||||
|
||||
catalog_section :: (entry: *Catalog_Entry, model: string) -> *Mcap_Ocr_Section {
|
||||
if !entry return null;
|
||||
for * entry.ocr if it.model == model return it;
|
||||
return null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- search
|
||||
|
||||
Search_Field :: enum u8 { ANY; OCR; TITLE; URL; APP; NOTE; }
|
||||
|
||||
Search_Term :: struct {
|
||||
field: Search_Field;
|
||||
text: string; // lowercased
|
||||
}
|
||||
|
||||
// Words, "quoted phrases", and either with a field prefix: ocr: title: url: app: note:. Temp.
|
||||
search_parse :: (query: string) -> [..] Search_Term {
|
||||
terms: [..] Search_Term;
|
||||
terms.allocator = temp;
|
||||
rest := trim(query);
|
||||
while rest.count {
|
||||
term: Search_Term;
|
||||
colon := find_index_from_left(rest, #char ":");
|
||||
space := find_index_from_left(rest, #char " ");
|
||||
if colon > 0 && (space < 0 || colon < space) {
|
||||
prefix := to_lower_copy(slice(rest, 0, colon),, temp);
|
||||
field := Search_Field.ANY;
|
||||
if prefix == {
|
||||
case "ocr"; field = .OCR;
|
||||
case "title"; field = .TITLE;
|
||||
case "url"; field = .URL;
|
||||
case "app"; field = .APP;
|
||||
case "note"; field = .NOTE;
|
||||
}
|
||||
if field != .ANY { term.field = field; rest = advance(rest, colon + 1); } // "http:..." and the like stay words
|
||||
}
|
||||
if rest.count && rest[0] == #char "\"" {
|
||||
close := find_index_from_left(advance(rest, 1), #char "\"");
|
||||
if close < 0 { term.text = advance(rest, 1); rest = ""; }
|
||||
else { term.text = slice(rest, 1, close); rest = advance(rest, close + 2); }
|
||||
} else {
|
||||
space = find_index_from_left(rest, #char " ");
|
||||
if space < 0 { term.text = rest; rest = ""; }
|
||||
else { term.text = slice(rest, 0, space); rest = advance(rest, space); }
|
||||
}
|
||||
term.text = to_lower_copy(term.text,, temp);
|
||||
if term.text.count array_add(*terms, term);
|
||||
rest = trim_left(rest);
|
||||
}
|
||||
return terms;
|
||||
}
|
||||
|
||||
// Whether entry holds every term.
|
||||
search_matches :: (entry: *Catalog_Entry, terms: [] Search_Term) -> bool {
|
||||
if !terms.count return true;
|
||||
if !entry return false;
|
||||
for terms {
|
||||
found: bool;
|
||||
if it.field == {
|
||||
case .ANY; found = contains(entry.find_ocr, it.text) || contains(entry.find_title, it.text) || contains(entry.find_url, it.text) ||
|
||||
contains(entry.find_app, it.text) || contains(entry.find_notes, it.text);
|
||||
case .OCR; found = contains(entry.find_ocr, it.text);
|
||||
case .TITLE; found = contains(entry.find_title, it.text);
|
||||
case .URL; found = contains(entry.find_url, it.text);
|
||||
case .APP; found = contains(entry.find_app, it.text);
|
||||
case .NOTE; found = contains(entry.find_notes, it.text);
|
||||
}
|
||||
if !found return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Whether an OCR line holds one of the terms that can be in OCR text: the ones it should be shown for.
|
||||
search_line_matches :: (text: string, terms: [] Search_Term) -> bool {
|
||||
lowered := to_lower_copy(text,, temp);
|
||||
for terms if (it.field == .ANY || it.field == .OCR) && contains(lowered, it.text) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- RTF
|
||||
|
||||
// An annotation's RTF reduced to its text: control words and their groups dropped, \par and \line as
|
||||
// line breaks, \uN and \'hh as the characters they stand for. Temp.
|
||||
rtf_to_text :: (rtf: string) -> string {
|
||||
out: String_Builder;
|
||||
out.allocator = temp;
|
||||
skip_depth := -1; // the group depth at which skipping began; -1 when not skipping
|
||||
depth := 0;
|
||||
unicode_skip := 0; // the fallback characters left to drop after a \uN
|
||||
i := 0;
|
||||
while i < rtf.count {
|
||||
c := rtf[i];
|
||||
if c == #char "{" { depth += 1; i += 1; continue; }
|
||||
if c == #char "}" {
|
||||
if skip_depth == depth skip_depth = -1;
|
||||
depth -= 1; i += 1; continue;
|
||||
}
|
||||
if c == #char "\r" || c == #char "\n" { i += 1; continue; }
|
||||
if c != #char "\\" {
|
||||
if skip_depth < 0 {
|
||||
if unicode_skip > 0 unicode_skip -= 1;
|
||||
else append(*out, c);
|
||||
}
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
// A control: \word[-123][ ], \'hh, or \ followed by one character.
|
||||
i += 1;
|
||||
if i >= rtf.count break;
|
||||
c = rtf[i];
|
||||
if c == #char "'" && i + 2 < rtf.count {
|
||||
byte := rtf_hex(rtf[i + 1]) * 16 + rtf_hex(rtf[i + 2]);
|
||||
i += 3;
|
||||
if skip_depth >= 0 continue;
|
||||
if unicode_skip > 0 { unicode_skip -= 1; continue; }
|
||||
rtf_append_code_point(*out, cp1252_to_unicode(cast(u8) byte));
|
||||
continue;
|
||||
}
|
||||
if !is_alpha(c) {
|
||||
i += 1;
|
||||
if skip_depth >= 0 continue;
|
||||
if c == #char "*" skip_depth = depth; // \* starts a destination this reader does not know
|
||||
else if c == #char "\\" || c == #char "{" || c == #char "}" append(*out, c);
|
||||
else if c == #char "~" append(*out, " ");
|
||||
continue;
|
||||
}
|
||||
word_start := i;
|
||||
while i < rtf.count && is_alpha(rtf[i]) i += 1;
|
||||
word := slice(rtf, word_start, i - word_start);
|
||||
number_start := i;
|
||||
if i < rtf.count && rtf[i] == #char "-" i += 1;
|
||||
while i < rtf.count && is_digit(rtf[i]) i += 1;
|
||||
number_text := slice(rtf, number_start, i - number_start);
|
||||
if i < rtf.count && rtf[i] == #char " " i += 1; // the space ending a control word is part of it
|
||||
if skip_depth >= 0 continue;
|
||||
|
||||
if word == {
|
||||
case "fonttbl"; #through;
|
||||
case "colortbl"; #through;
|
||||
case "stylesheet"; #through;
|
||||
case "info"; #through;
|
||||
case "pict"; #through;
|
||||
case "header"; #through;
|
||||
case "footer"; #through;
|
||||
case "generator";
|
||||
skip_depth = depth;
|
||||
case "par"; #through;
|
||||
case "line";
|
||||
append(*out, "\n");
|
||||
case "tab";
|
||||
append(*out, "\t");
|
||||
case "u";
|
||||
value, ok := string_to_int(number_text);
|
||||
if ok {
|
||||
if value < 0 value += 65536;
|
||||
rtf_append_code_point(*out, cast(u32) value);
|
||||
unicode_skip = 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
return trim(builder_to_string(*out,, temp));
|
||||
}
|
||||
|
||||
#scope_file
|
||||
|
||||
catalog_string :: (object: Capture_Value, key: string) -> string {
|
||||
value := capture_object_find(*object, key);
|
||||
if !value || value.kind != .STRING return "";
|
||||
return copy_string(value.text);
|
||||
}
|
||||
|
||||
rtf_hex :: (c: u8) -> s64 {
|
||||
if c >= #char "0" && c <= #char "9" return c - #char "0";
|
||||
if c >= #char "a" && c <= #char "f" return c - #char "a" + 10;
|
||||
if c >= #char "A" && c <= #char "F" return c - #char "A" + 10;
|
||||
return 0;
|
||||
}
|
||||
|
||||
// Windows-1252, which Snagit's RTF uses for \'hh: Latin-1 except 0x80 to 0x9F.
|
||||
cp1252_to_unicode :: (byte: u8) -> u32 {
|
||||
HIGH :: u32.[0x20AC, 0x81, 0x201A, 0x0192, 0x201E, 0x2026, 0x2020, 0x2021, 0x02C6, 0x2030, 0x0160, 0x2039, 0x0152, 0x8D, 0x017D, 0x8F,
|
||||
0x90, 0x2018, 0x2019, 0x201C, 0x201D, 0x2022, 0x2013, 0x2014, 0x02DC, 0x2122, 0x0161, 0x203A, 0x0153, 0x9D, 0x017E, 0x0178];
|
||||
if byte >= 0x80 && byte <= 0x9F return HIGH[byte - 0x80];
|
||||
return byte;
|
||||
}
|
||||
|
||||
rtf_append_code_point :: (out: *String_Builder, code_point: u32) {
|
||||
if code_point < 0x80 {
|
||||
append(out, cast(u8) code_point);
|
||||
} else if code_point < 0x800 {
|
||||
append(out, cast(u8) (0xC0 | (code_point >> 6)));
|
||||
append(out, cast(u8) (0x80 | (code_point & 0x3F)));
|
||||
} else {
|
||||
append(out, cast(u8) (0xE0 | (code_point >> 12)));
|
||||
append(out, cast(u8) (0x80 | ((code_point >> 6) & 0x3F)));
|
||||
append(out, cast(u8) (0x80 | (code_point & 0x3F)));
|
||||
}
|
||||
}
|
||||
@ -17,6 +17,8 @@ main :: () {
|
||||
app.headless = array_find(app.args, "-headless") || NO_CRASH_DIALOG;
|
||||
runtime_support_headless = app.headless;
|
||||
app.allow_multiple_instances = array_find(app.args, "-allow-multiple-instances");
|
||||
for app.args if it == "-library" && it_index + 1 < app.args.count app.library_folder = app.args[it_index + 1];
|
||||
app.scan_and_exit = array_find(app.args, "-scan");
|
||||
|
||||
defer report_memory_leaks();
|
||||
|
||||
|
||||
@ -1,65 +1,127 @@
|
||||
// OCR in the app: an engine made on a background thread at startup (modules/OCR), an OCR window that
|
||||
// says whether it runs on the GPU and, if not, what to install, and the OCR button in the Capture
|
||||
// window, which reads the capture shown and draws its lines over it.
|
||||
// OCR in the app (modules/OCR): an engine made on a background thread, the OCR window (whether it runs
|
||||
// on the GPU and, if not, what to install; the model, kept in the settings file; the library scan),
|
||||
// and in the Capture window the OCR button, the lines read drawn over the image, and the search's
|
||||
// matches among them.
|
||||
//
|
||||
// Engines take a moment to make and OCR takes a moment to run, so both happen on one background
|
||||
// thread; the UI only looks at what it has finished.
|
||||
// Every reading is stored in its .mcap (mcap_store_ocr: one section per model) and its catalog entry
|
||||
// is loaded again, so the overlay, the text shown and search all come from what is stored.
|
||||
//
|
||||
// Making an engine, reading one capture and the scan all run on one background thread, one at a time;
|
||||
// the UI only looks at what it has finished, and at the scan's progress counters.
|
||||
|
||||
OCR_WINDOW :: "OCR";
|
||||
OCR_DEFAULT_MODEL :: "ppocr-v5-mobile";
|
||||
OCR_SCAN_BATCH :: 8; // captures per ocr_process call in a scan
|
||||
OCR_STORE_ATTEMPTS :: 4; // a file the viewer has open at that moment is tried again, 250 ms apart
|
||||
|
||||
ocr_ui: struct {
|
||||
models: [] Ocr_Model_Info;
|
||||
models_error: string;
|
||||
model_index: s32; // the model chosen in the window
|
||||
model_index: s32; // the model chosen in the OCR window
|
||||
|
||||
// Owned by the main thread while thread_busy is false; by the background thread while it is true.
|
||||
engine: *Ocr_Engine;
|
||||
engine_model: string; // the model engine runs
|
||||
engine_error: string; // why the last engine could not be made
|
||||
thread: Thread;
|
||||
thread_started: bool;
|
||||
thread_busy: bool; // atomic: set by the main thread, cleared by the background one
|
||||
thread_busy: bool; // set by the main thread, cleared by the background one
|
||||
task: Ocr_Task;
|
||||
|
||||
// The last capture read, for the Capture window.
|
||||
lines: [] Ocr_Line;
|
||||
lines_index: s32 = -1; // the capture they are for
|
||||
lines_error: string;
|
||||
lines_seconds: float64;
|
||||
read_error: string; // why the last single read failed
|
||||
read_index: s32 = -1;
|
||||
read_seconds: float64;
|
||||
|
||||
scan: Ocr_Scan;
|
||||
scan_requested: bool; // -scan has started its scan
|
||||
|
||||
// The Capture window.
|
||||
overlay_model: string; // the section shown over the image; "" for the chosen model's
|
||||
show_boxes := true;
|
||||
match_cursor: s64 = -1; // which match Next and Previous last went to
|
||||
scroll_to_match: bool;
|
||||
|
||||
window_open := false;
|
||||
toast_shown := false;
|
||||
}
|
||||
|
||||
Ocr_Task_Kind :: enum { NONE; CREATE; READ; }
|
||||
Ocr_Task_Kind :: enum { NONE; CREATE; READ; SCAN; }
|
||||
|
||||
Ocr_Task :: struct {
|
||||
kind: Ocr_Task_Kind;
|
||||
model: string;
|
||||
path: string; // READ: the .mcap
|
||||
index: s32; // READ: its capture index
|
||||
model: string; // CREATE
|
||||
path: string; // READ
|
||||
index: s32; // READ
|
||||
}
|
||||
|
||||
// The library scan. The counters and flags are shared with the background thread.
|
||||
Ocr_Scan :: struct {
|
||||
items: [..] Scan_Item; // the captures it reads, in display order
|
||||
model: string;
|
||||
total: s64;
|
||||
done: s64; // read and stored, or failed
|
||||
failed: s64;
|
||||
paused: bool;
|
||||
cancel: bool;
|
||||
running: bool;
|
||||
start: Apollo_Time;
|
||||
paused_seconds: float64; // time spent paused, left out of the rate
|
||||
pause_start: Apollo_Time;
|
||||
}
|
||||
|
||||
Scan_Item :: struct { index: s32; path: string; }
|
||||
|
||||
ocr_models_folder :: () -> string {
|
||||
return tprint("%ocr-models", path_strip_filename(get_path_of_running_executable()));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- startup and shutdown
|
||||
|
||||
ocr_ui_startup :: () {
|
||||
push_allocator(context.default_allocator);
|
||||
models, error := ocr_list_models(ocr_models_folder());
|
||||
ocr_ui.models = models;
|
||||
if error ocr_ui.models_error = copy_string(error);
|
||||
for ocr_ui.models if it.name == OCR_DEFAULT_MODEL ocr_ui.model_index = cast(s32) it_index;
|
||||
ocr_ui.model_index = ocr_model_index(app_user_settings.ocr_model);
|
||||
if ocr_ui.models.count ocr_ui_start(.{kind = .CREATE, model = ocr_ui.models[ocr_ui.model_index].name});
|
||||
}
|
||||
|
||||
ocr_ui_shutdown :: () {
|
||||
if ocr_ui.thread_started thread_is_done(*ocr_ui.thread, -1); // finishes what it is doing
|
||||
atomic_swap(*ocr_ui.scan.cancel, true); // a scan stops after the capture it is on
|
||||
atomic_swap(*ocr_ui.scan.paused, false);
|
||||
if ocr_ui.thread_started thread_is_done(*ocr_ui.thread, -1);
|
||||
ocr_destroy(ocr_ui.engine);
|
||||
}
|
||||
|
||||
// The model the settings name, or the default one, or the first; with a log line when the settings
|
||||
// name one that is not in models.config.
|
||||
ocr_model_index :: (name: string) -> s32 {
|
||||
wanted := ifx name.count then name else OCR_DEFAULT_MODEL;
|
||||
for ocr_ui.models if it.name == wanted return cast(s32) it_index;
|
||||
if name.count log_error("[OCR] The settings name model %, which models.config does not have; using %.", name, OCR_DEFAULT_MODEL);
|
||||
for ocr_ui.models if it.name == OCR_DEFAULT_MODEL return cast(s32) it_index;
|
||||
return 0;
|
||||
}
|
||||
|
||||
// The settings file named another model (edited by hand): switch to it.
|
||||
ocr_ui_settings_changed :: () {
|
||||
if !ocr_ui.models.count return;
|
||||
index := ocr_model_index(app_user_settings.ocr_model);
|
||||
if index == ocr_ui.model_index return;
|
||||
ocr_ui_choose_model(index, save = false);
|
||||
}
|
||||
|
||||
ocr_ui_choose_model :: (index: s32, save := true) {
|
||||
if ocr_ui.thread_busy return;
|
||||
ocr_ui.model_index = index;
|
||||
ocr_ui.toast_shown = true; // said once already, or not needed
|
||||
ocr_ui_start(.{kind = .CREATE, model = ocr_ui.models[index].name});
|
||||
if save {
|
||||
free(app_user_settings.ocr_model,, context.default_allocator);
|
||||
app_user_settings.ocr_model = copy_string(ocr_ui.models[index].name,, context.default_allocator);
|
||||
app_save_user_settings_file();
|
||||
}
|
||||
}
|
||||
|
||||
// Starts task on the background thread; false while it is still busy with the last one.
|
||||
ocr_ui_start :: (task: Ocr_Task) -> bool {
|
||||
if ocr_ui.thread_busy return false;
|
||||
@ -76,6 +138,8 @@ ocr_ui_start :: (task: Ocr_Task) -> bool {
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- the background thread
|
||||
|
||||
ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
|
||||
context.allocator = context.default_allocator;
|
||||
task := ocr_ui.task;
|
||||
@ -85,18 +149,35 @@ ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
|
||||
ocr_ui.engine = null;
|
||||
engine, ok, error := ocr_create(ocr_models_folder(), task.model);
|
||||
ocr_ui.engine = engine;
|
||||
ocr_ui.engine_model = task.model;
|
||||
free(ocr_ui.engine_error);
|
||||
ocr_ui.engine_error = ifx ok then "" else copy_string(error);
|
||||
if ok log("[OCR] % on %.%", task.model, engine.provider, ifx engine.gpu_explanation then tprint(" %", engine.gpu_explanation) else "");
|
||||
else log_error("[OCR] Could not start %: %", task.model, error);
|
||||
|
||||
case .READ;
|
||||
start := current_time_monotonic();
|
||||
lines, error := ocr_ui_read_capture(task.path);
|
||||
ocr_ui.lines = lines;
|
||||
ocr_ui.lines_error = error;
|
||||
ocr_ui.lines_index = task.index;
|
||||
ocr_ui.lines_seconds = to_float64_seconds(current_time_monotonic() - start);
|
||||
items := Scan_Item.[.{task.index, task.path}];
|
||||
errors := ocr_read_and_store(items);
|
||||
free(ocr_ui.read_error);
|
||||
ocr_ui.read_error = ifx errors[0] then errors[0] else "";
|
||||
ocr_ui.read_index = task.index;
|
||||
ocr_ui.read_seconds = to_float64_seconds(current_time_monotonic() - start);
|
||||
|
||||
case .SCAN;
|
||||
scan := *ocr_ui.scan;
|
||||
position := 0;
|
||||
while position < scan.items.count {
|
||||
while scan.paused && !scan.cancel sleep_milliseconds(100);
|
||||
if scan.cancel break;
|
||||
count := min(OCR_SCAN_BATCH, scan.items.count - position);
|
||||
errors := ocr_read_and_store(array_view(scan.items, position, count));
|
||||
for errors if it { atomic_add(*scan.failed, 1); log_error("[OCR scan] % %", scan.items[position + it_index].path, it); free(it); }
|
||||
atomic_add(*scan.done, count);
|
||||
position += count;
|
||||
reset_temporary_storage();
|
||||
}
|
||||
log("[OCR scan] % of % captures read with % (% failed)%.", scan.done, scan.total, scan.model, scan.failed, ifx scan.cancel then ", cancelled" else "");
|
||||
atomic_swap(*scan.running, false);
|
||||
}
|
||||
free(task.model);
|
||||
free(task.path);
|
||||
@ -104,41 +185,170 @@ ocr_ui_thread_proc :: (thread: *Thread) -> s64 {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// The lines in a capture's image as the Capture window shows it: the rendered one, annotations and
|
||||
// all, or the original when there is none. Heap.
|
||||
ocr_ui_read_capture :: (path: string) -> [] Ocr_Line, error: string {
|
||||
if !ocr_ui.engine return .[], copy_string("No OCR engine is running.");
|
||||
data, read_ok := read_entire_file(path, log_errors = false,, temp);
|
||||
if !read_ok return .[], sprint("Cannot read %.", path);
|
||||
// Reads items with the engine, stores each reading in its file, and hands each new catalog entry to
|
||||
// the viewer. An error message (heap) for each item that failed, "" for the others. Background thread.
|
||||
ocr_read_and_store :: (items: [] Scan_Item) -> [] string {
|
||||
errors := NewArray(items.count, string);
|
||||
if !ocr_ui.engine { for * errors it.* = copy_string("No OCR engine is running."); return errors; }
|
||||
|
||||
// The image the Capture window shows: the rendered one, annotations and all, or the original.
|
||||
Prepared :: struct { pixels: *u8; width, height: s32; image: string; }
|
||||
prepared := NewArray(items.count, Prepared,, temp);
|
||||
images: [..] Ocr_Image;
|
||||
images.allocator = temp;
|
||||
image_of: [..] s64; // which item each image is
|
||||
image_of.allocator = temp;
|
||||
for item, n: items {
|
||||
data, read_ok := read_entire_file(item.path, log_errors = false,, temp);
|
||||
if !read_ok { errors[n] = copy_string("cannot be read"); continue; }
|
||||
m, ok, error := mcap_parse(cast([] u8) data);
|
||||
if !ok return .[], copy_string(error);
|
||||
if !ok { errors[n] = copy_string(error); continue; }
|
||||
png := mcap_rendered(m);
|
||||
if !png.count png = m.original;
|
||||
kind := "rendered";
|
||||
if !png.count { png = m.original; kind = "original"; }
|
||||
w, h, channels: s32;
|
||||
pixels := stbi_load_from_memory(png.data, cast(s32) png.count, *w, *h, *channels, 4);
|
||||
if !pixels return .[], copy_string("Its image cannot be decoded.");
|
||||
defer stbi_image_free(pixels);
|
||||
images := Ocr_Image.[.{pixels, w, h, w * 4}];
|
||||
results := ocr_process(ocr_ui.engine, images);
|
||||
defer array_free(results);
|
||||
if results[0].failed return .[], results[0].error;
|
||||
return results[0].lines, "";
|
||||
if !pixels { errors[n] = copy_string("its image cannot be decoded"); continue; }
|
||||
prepared[n] = .{pixels, w, h, kind};
|
||||
array_add(*images, .{pixels, w, h, w * 4});
|
||||
array_add(*image_of, n);
|
||||
}
|
||||
defer for prepared if it.pixels stbi_image_free(it.pixels);
|
||||
|
||||
results := ocr_process(ocr_ui.engine, images,, temp);
|
||||
engine_text := tprint("onnxruntime %, %", ort.version, ifx ocr_ui.engine.provider == .CUDA then "CUDA" else "CPU");
|
||||
for result, r: results {
|
||||
n := image_of[r];
|
||||
if result.failed { errors[n] = copy_string(result.error); continue; }
|
||||
|
||||
section: Mcap_Ocr_Section;
|
||||
section.model = ocr_ui.engine.model.name;
|
||||
section.engine = engine_text;
|
||||
section.image = prepared[n].image;
|
||||
section.image_width, section.image_height = prepared[n].width, prepared[n].height;
|
||||
section.date = mcap_ocr_now();
|
||||
section.lines.allocator = temp;
|
||||
for result.lines {
|
||||
line := array_add(*section.lines);
|
||||
line.* = .{x = it.x, y = it.y, width = it.width, height = it.height, confidence = it.confidence, text = it.text};
|
||||
for corner: it.corners {
|
||||
line.corners[it_index * 2] = cast(s32) round(corner.x);
|
||||
line.corners[it_index * 2 + 1] = cast(s32) round(corner.y);
|
||||
}
|
||||
}
|
||||
|
||||
stored: bool;
|
||||
error: string;
|
||||
for 1..OCR_STORE_ATTEMPTS {
|
||||
stored, error = mcap_store_ocr(items[n].path, section);
|
||||
if stored break;
|
||||
sleep_milliseconds(250);
|
||||
}
|
||||
if !stored { errors[n] = copy_string(error); continue; }
|
||||
|
||||
entry, load_error := catalog_load(items[n].path);
|
||||
if !entry { errors[n] = sprint("stored, but could not be read back: %", load_error); continue; }
|
||||
lock(*viewer.mutex);
|
||||
array_add(*viewer.catalog_updates, .{items[n].index, entry});
|
||||
unlock(*viewer.mutex);
|
||||
}
|
||||
return errors;
|
||||
}
|
||||
|
||||
ocr_ui_free_lines :: () {
|
||||
for ocr_ui.lines free(it.text);
|
||||
array_free(ocr_ui.lines);
|
||||
free(ocr_ui.lines_error);
|
||||
ocr_ui.lines = .[];
|
||||
ocr_ui.lines_error = "";
|
||||
ocr_ui.lines_index = -1;
|
||||
// ---------------------------------------------------------------- the scan
|
||||
|
||||
// -scan: starts the scan once the engine and the catalog are ready, and ends the app when it is done.
|
||||
ocr_ui_scan_and_exit :: (busy: bool) {
|
||||
finish :: (code: s32, format: string, args: ..Any) {
|
||||
if code log_error(format, ..args);
|
||||
else log(format, ..args);
|
||||
app.exit_code = code;
|
||||
app.should_exit = true;
|
||||
app.scan_and_exit = false;
|
||||
}
|
||||
if ocr_ui.models_error { finish(1, "[OCR scan] %", ocr_ui.models_error); return; }
|
||||
if viewer.scan_error { finish(1, "[OCR scan] %", viewer.scan_error); return; }
|
||||
if !ocr_ui.scan_requested {
|
||||
if busy || !ocr_ui.engine || viewer.catalog_loaded < viewer.captures.count return;
|
||||
ocr_ui.scan_requested = true;
|
||||
ocr_scan_start();
|
||||
return;
|
||||
}
|
||||
if busy || ocr_ui.scan.running return;
|
||||
finish(ifx ocr_ui.scan.failed then cast(s32) 1 else 0, "[OCR scan] -scan is done: % read, % failed.", ocr_ui.scan.done, ocr_ui.scan.failed);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- drawing
|
||||
ocr_scan_start :: () {
|
||||
if ocr_ui.thread_busy || !ocr_ui.engine return;
|
||||
scan := *ocr_ui.scan;
|
||||
for scan.items free(it.path);
|
||||
array_reset(*scan.items);
|
||||
free(scan.model);
|
||||
scan.* = .{};
|
||||
scan.items.allocator = context.default_allocator;
|
||||
scan.model = copy_string(ocr_ui.engine.model.name,, context.default_allocator);
|
||||
// Every capture without a reading by this model, in the order the grid shows them.
|
||||
for viewer.visible {
|
||||
capture := *viewer.captures[it];
|
||||
if capture.catalog && catalog_section(capture.catalog, scan.model) continue;
|
||||
array_add(*scan.items, .{it, copy_string(capture.path,, context.default_allocator)});
|
||||
}
|
||||
scan.total = scan.items.count;
|
||||
if !scan.total { toast_show(.Success, "Every capture shown has been read with %", scan.model); return; }
|
||||
scan.running = true;
|
||||
scan.start = current_time_monotonic();
|
||||
ocr_ui_start(.{kind = .SCAN});
|
||||
}
|
||||
|
||||
ocr_scan_ui :: () { using ImGui;
|
||||
scan := *ocr_ui.scan;
|
||||
busy := ocr_ui.thread_busy;
|
||||
catalog_ready := viewer.catalog_loaded >= viewer.captures.count;
|
||||
|
||||
if !scan.running {
|
||||
can_scan := ocr_ui.engine && !busy && catalog_ready;
|
||||
if !can_scan BeginDisabled();
|
||||
label := ifx viewer.search_terms.count then "Scan the captures shown" else "Scan the library";
|
||||
if Button(temp_c_string(label)) ocr_scan_start();
|
||||
if !can_scan EndDisabled();
|
||||
SameLine();
|
||||
if !catalog_ready TextColored(app_theme.palette.dim, "(once the catalog has loaded)");
|
||||
else TextColored(app_theme.palette.dim, "Reads every capture without a reading by this model. Each is stored as it is read, so a scan can stop and start again.");
|
||||
if scan.total && scan.done {
|
||||
TextColored(app_theme.palette.dim, "Last scan: % of % read with %, % failed.", scan.done, scan.total, scan.model, scan.failed);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
done, total, failed := scan.done, scan.total, scan.failed;
|
||||
elapsed := to_float64_seconds(current_time_monotonic() - scan.start) - scan.paused_seconds;
|
||||
if scan.paused elapsed -= to_float64_seconds(current_time_monotonic() - scan.pause_start);
|
||||
rate := ifx elapsed > 0 then done / elapsed else 0;
|
||||
left := ifx rate > 0 then (total - done) / rate else 0;
|
||||
ProgressBar(cast(float) done / max(total, 1), .{-1, 0}, temp_c_string(tprint("% / %", done, total)));
|
||||
Text("% captures/s, % left, % failed, on %", FF(rate, 1), ocr_duration_text(left), failed, ifx ocr_ui.engine && ocr_ui.engine.provider == .CUDA then "the GPU" else "the CPU");
|
||||
if scan.paused {
|
||||
if Button("Resume") { scan.paused_seconds += to_float64_seconds(current_time_monotonic() - scan.pause_start); atomic_swap(*scan.paused, false); }
|
||||
} else {
|
||||
if Button("Pause") { scan.pause_start = current_time_monotonic(); atomic_swap(*scan.paused, true); }
|
||||
}
|
||||
SameLine();
|
||||
if Button("Cancel") { atomic_swap(*scan.cancel, true); atomic_swap(*scan.paused, false); }
|
||||
}
|
||||
|
||||
ocr_duration_text :: (seconds: float64) -> string {
|
||||
s := cast(s64) seconds;
|
||||
if s >= 3600 return tprint("% h % min", s / 3600, (s % 3600) / 60);
|
||||
if s >= 60 return tprint("% min % s", s / 60, s % 60);
|
||||
return tprint("% s", s);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- the OCR window
|
||||
|
||||
// Each frame: the toast once the engine is up on the CPU though a GPU could run it, and the OCR window.
|
||||
ocr_ui_frame :: () { using ImGui;
|
||||
busy := ocr_ui.thread_busy;
|
||||
if app.scan_and_exit ocr_ui_scan_and_exit(busy);
|
||||
if !busy && ocr_ui.engine && !ocr_ui.toast_shown {
|
||||
ocr_ui.toast_shown = true;
|
||||
in_use, explanation := ocr_gpu_status(ocr_ui.engine);
|
||||
@ -149,46 +359,46 @@ ocr_ui_frame :: () { using ImGui;
|
||||
}
|
||||
|
||||
if !ocr_ui.window_open return;
|
||||
SetNextWindowSize(.{640, 360}, .FirstUseEver);
|
||||
SetNextWindowSize(.{680, 420}, .FirstUseEver);
|
||||
if Begin(OCR_WINDOW, *ocr_ui.window_open) {
|
||||
if ocr_ui.models_error {
|
||||
TextColored(app_theme.palette.warning, "%", ocr_ui.models_error);
|
||||
TextWrapped("Run `jai fetch_models.jai` in modules/OCR and `jai fetch.jai` in modules/onnxruntime, then rebuild.");
|
||||
} else {
|
||||
// The model: changing it makes a new engine on the background thread.
|
||||
// The model: changing it makes a new engine, and is kept in the settings file.
|
||||
preview := ifx ocr_ui.models.count then ocr_ui.models[ocr_ui.model_index].title else "";
|
||||
if busy BeginDisabled();
|
||||
SetNextItemWidth(320);
|
||||
if BeginCombo("Model", temp_c_string(preview)) {
|
||||
for ocr_ui.models {
|
||||
if Selectable(temp_c_string(it.title), it_index == ocr_ui.model_index) && it_index != ocr_ui.model_index && !busy {
|
||||
ocr_ui.model_index = cast(s32) it_index;
|
||||
ocr_ui.toast_shown = true; // said once already, or not needed
|
||||
ocr_ui_free_lines();
|
||||
ocr_ui_start(.{kind = .CREATE, model = it.name});
|
||||
}
|
||||
if Selectable(temp_c_string(it.title), it_index == ocr_ui.model_index) && it_index != ocr_ui.model_index ocr_ui_choose_model(cast(s32) it_index);
|
||||
}
|
||||
EndCombo();
|
||||
}
|
||||
if busy EndDisabled();
|
||||
|
||||
if busy && ocr_ui.task.kind == .CREATE {
|
||||
TextColored(app_theme.palette.dim, "Starting % ...", ocr_ui.task.model);
|
||||
} else if ocr_ui.engine_error {
|
||||
TextColored(app_theme.palette.warning, "OCR could not start:");
|
||||
TextWrapped("%", ocr_ui.engine_error);
|
||||
} else if ocr_ui.engine && !busy {
|
||||
} else if ocr_ui.engine {
|
||||
in_use, explanation := ocr_gpu_status(ocr_ui.engine);
|
||||
Text("Running on: %", ifx in_use then "the GPU (CUDA)" else "the CPU");
|
||||
SameLine();
|
||||
TextColored(app_theme.palette.dim, "onnxruntime %, % worker threads", ort.version, ocr_ui.engine.workers);
|
||||
Separator();
|
||||
ocr_scan_ui();
|
||||
if explanation { Separator(); text_with_links(explanation); }
|
||||
}
|
||||
|
||||
// The text of the capture last read, in reading order.
|
||||
if !busy && ocr_ui.lines_index >= 0 && ocr_ui.lines_index < viewer.captures.count {
|
||||
// The text of the capture shown in the Capture window, as the overlay's model read it.
|
||||
section := ocr_overlay_section();
|
||||
if section && viewer.full_index >= 0 {
|
||||
Separator();
|
||||
Text("Text of %:", viewer.captures[ocr_ui.lines_index].name);
|
||||
Text("Text of % (% , % lines):", viewer.captures[viewer.full_index].name, section.model, section.lines.count);
|
||||
BeginChild("ocr text");
|
||||
ocr_ui_capture_text(ocr_ui.lines_index);
|
||||
for section.lines TextUnformatted(it.text);
|
||||
EndChild();
|
||||
}
|
||||
}
|
||||
@ -196,45 +406,6 @@ ocr_ui_frame :: () { using ImGui;
|
||||
End();
|
||||
}
|
||||
|
||||
// The OCR row of the Capture window, above its image: the button, and how the last read went.
|
||||
ocr_ui_capture_controls :: (index: s32, path: string) { using ImGui;
|
||||
busy := ocr_ui.thread_busy;
|
||||
can_read := ocr_ui.engine && !busy;
|
||||
if !can_read BeginDisabled();
|
||||
if SmallButton("OCR") {
|
||||
ocr_ui_free_lines();
|
||||
ocr_ui_start(.{kind = .READ, path = path, index = index});
|
||||
}
|
||||
if !can_read EndDisabled();
|
||||
SameLine();
|
||||
if busy && ocr_ui.task.kind == .READ && ocr_ui.task.index == index {
|
||||
TextColored(app_theme.palette.dim, "Reading...");
|
||||
} else if !busy && ocr_ui.lines_index == index {
|
||||
if ocr_ui.lines_error TextColored(app_theme.palette.warning, "%", ocr_ui.lines_error);
|
||||
else TextColored(app_theme.palette.dim, "% lines in % s", ocr_ui.lines.count, FF(ocr_ui.lines_seconds, 2));
|
||||
SameLine();
|
||||
Checkbox("Boxes", *ocr_ui.show_boxes);
|
||||
} else if !ocr_ui.engine {
|
||||
TextColored(app_theme.palette.dim, "(OCR is not running; see the OCR window)");
|
||||
}
|
||||
SameLine();
|
||||
if SmallButton("OCR window") ocr_ui.window_open = true;
|
||||
}
|
||||
|
||||
// The lines read from this capture, as boxes over its image, drawn at image_min with scale; and their
|
||||
// text in a list below the image area.
|
||||
ocr_ui_capture_overlay :: (index: s32, image_min: ImGui.ImVec2, scale: float) { using ImGui;
|
||||
if ocr_ui.thread_busy || ocr_ui.lines_index != index || !ocr_ui.show_boxes return;
|
||||
draw := GetWindowDrawList();
|
||||
colour := GetColorU32(.PlotHistogram);
|
||||
for ocr_ui.lines {
|
||||
p0 := ImVec2.{image_min.x + it.x * scale, image_min.y + it.y * scale};
|
||||
p1 := ImVec2.{p0.x + it.width * scale, p0.y + it.height * scale};
|
||||
draw.AddRect(draw, p0, p1, colour, 0, 1.5);
|
||||
if IsMouseHoveringRect(p0, p1) SetTooltip("% (%)", it.text, FF(it.confidence, 2));
|
||||
}
|
||||
}
|
||||
|
||||
// Text a line at a time, each https:// address in it drawn as a link that opens in the browser.
|
||||
text_with_links :: (text: string) { using ImGui;
|
||||
for line: split(text, "\n",, temp) {
|
||||
@ -250,7 +421,113 @@ text_with_links :: (text: string) { using ImGui;
|
||||
}
|
||||
}
|
||||
|
||||
ocr_ui_capture_text :: (index: s32) { using ImGui;
|
||||
if ocr_ui.thread_busy || ocr_ui.lines_index != index || !ocr_ui.lines.count return;
|
||||
for ocr_ui.lines TextUnformatted(it.text);
|
||||
// ---------------------------------------------------------------- the Capture window
|
||||
|
||||
// The reading shown over the capture in the Capture window: the overlay's choice, else the chosen
|
||||
// model's, else the first there is.
|
||||
ocr_overlay_section :: () -> *Mcap_Ocr_Section {
|
||||
if viewer.full_index < 0 || viewer.full_index >= viewer.captures.count return null;
|
||||
entry := viewer.captures[viewer.full_index].catalog;
|
||||
if !entry || !entry.ocr.count return null;
|
||||
if ocr_ui.overlay_model.count {
|
||||
section := catalog_section(entry, ocr_ui.overlay_model);
|
||||
if section return section;
|
||||
}
|
||||
if ocr_ui.models.count {
|
||||
section := catalog_section(entry, ocr_ui.models[ocr_ui.model_index].name);
|
||||
if section return section;
|
||||
}
|
||||
return *entry.ocr[0];
|
||||
}
|
||||
|
||||
// The OCR row of the Capture window, above its image.
|
||||
ocr_ui_capture_controls :: (index: s32, path: string) { using ImGui;
|
||||
busy := ocr_ui.thread_busy;
|
||||
entry := viewer.captures[index].catalog;
|
||||
model := ifx ocr_ui.models.count then ocr_ui.models[ocr_ui.model_index].name else "";
|
||||
has_reading := entry && catalog_section(entry, model) != null;
|
||||
|
||||
can_read := ocr_ui.engine && !busy;
|
||||
if !can_read BeginDisabled();
|
||||
if SmallButton(temp_c_string(ifx has_reading then "Read again" else "OCR")) ocr_ui_start(.{kind = .READ, path = path, index = index});
|
||||
if !can_read EndDisabled();
|
||||
SameLine();
|
||||
if busy && ocr_ui.task.kind == .READ && ocr_ui.task.index == index {
|
||||
TextColored(app_theme.palette.dim, "Reading...");
|
||||
SameLine();
|
||||
} else if !busy && ocr_ui.read_index == index {
|
||||
if ocr_ui.read_error { TextColored(app_theme.palette.warning, "%", ocr_ui.read_error); SameLine(); }
|
||||
else { TextColored(app_theme.palette.dim, "read in % s", FF(ocr_ui.read_seconds, 2)); SameLine(); }
|
||||
}
|
||||
|
||||
section := ocr_overlay_section();
|
||||
if section {
|
||||
// A capture read by several models: which reading to show.
|
||||
if entry.ocr.count > 1 {
|
||||
SetNextItemWidth(170);
|
||||
if BeginCombo("##reading", temp_c_string(section.model)) {
|
||||
for entry.ocr if Selectable(temp_c_string(it.model), it.model == section.model) {
|
||||
free(ocr_ui.overlay_model,, context.default_allocator);
|
||||
ocr_ui.overlay_model = copy_string(it.model,, context.default_allocator);
|
||||
}
|
||||
EndCombo();
|
||||
}
|
||||
SameLine();
|
||||
}
|
||||
Checkbox("Boxes", *ocr_ui.show_boxes);
|
||||
if viewer.search_terms.count {
|
||||
matches := 0;
|
||||
for section.lines if search_line_matches(it.text, viewer.search_terms) matches += 1;
|
||||
SameLine();
|
||||
Text("Matches: %", matches);
|
||||
if matches {
|
||||
SameLine(); if SmallButton("Previous") { ocr_ui.match_cursor = (ocr_ui.match_cursor - 1 + matches) % matches; ocr_ui.scroll_to_match = true; }
|
||||
SameLine(); if SmallButton("Next") { ocr_ui.match_cursor = (ocr_ui.match_cursor + 1) % matches; ocr_ui.scroll_to_match = true; }
|
||||
}
|
||||
}
|
||||
SameLine();
|
||||
} else if !ocr_ui.engine {
|
||||
TextColored(app_theme.palette.dim, "(OCR is not running; see the OCR window)");
|
||||
SameLine();
|
||||
}
|
||||
if SmallButton("OCR window") ocr_ui.window_open = true;
|
||||
}
|
||||
|
||||
// The reading's lines as outlines over the image (turned with the text, for a turned line), drawn at
|
||||
// image_min with zoom (the image's size as shown over its captured size): those matching the search
|
||||
// highlighted, the others dimmer. Inside the image's scrolling child, which Next and Previous scroll.
|
||||
ocr_ui_capture_overlay :: (index: s32, image_min: ImGui.ImVec2, zoom: float) { using ImGui;
|
||||
section := ocr_overlay_section();
|
||||
if !section || !ocr_ui.show_boxes return;
|
||||
// The section's pixels may be another image's (the original, where the rendered one differs).
|
||||
scale_x := zoom * ifx section.image_width then cast(float) viewer.full_source_width / section.image_width else 1;
|
||||
scale_y := zoom * ifx section.image_height then cast(float) viewer.full_source_height / section.image_height else 1;
|
||||
|
||||
draw := GetWindowDrawList();
|
||||
searching := viewer.search_terms.count > 0;
|
||||
plain := GetColorU32(.PlotHistogram, ifx searching then 0.35 else 1.0);
|
||||
matching := GetColorU32(.PlotLinesHovered);
|
||||
match_number := 0;
|
||||
for section.lines {
|
||||
p0 := ImVec2.{image_min.x + it.x * scale_x, image_min.y + it.y * scale_y};
|
||||
p1 := ImVec2.{p0.x + it.width * scale_x, p0.y + it.height * scale_y};
|
||||
c := mcap_ocr_corners(it);
|
||||
q: [4] ImVec2;
|
||||
for * q it.* = .{image_min.x + c[it_index * 2] * scale_x, image_min.y + c[it_index * 2 + 1] * scale_y};
|
||||
is_match := searching && search_line_matches(it.text, viewer.search_terms);
|
||||
if is_match {
|
||||
draw.AddQuadFilled(draw, q[0], q[1], q[2], q[3], GetColorU32(.PlotLinesHovered, 0.25));
|
||||
draw.AddQuad(draw, q[0], q[1], q[2], q[3], matching, 2.5);
|
||||
if ocr_ui.scroll_to_match && match_number == ocr_ui.match_cursor {
|
||||
// Into view, a third of the way down.
|
||||
SetScrollX(it.x * scale_x - GetWindowWidth() / 3);
|
||||
SetScrollY(it.y * scale_y - GetWindowHeight() / 3);
|
||||
ocr_ui.scroll_to_match = false;
|
||||
}
|
||||
match_number += 1;
|
||||
} else {
|
||||
draw.AddQuad(draw, q[0], q[1], q[2], q[3], plain, 1.5);
|
||||
}
|
||||
if IsMouseHoveringRect(p0, p1) SetTooltip("% (%)", it.text, FF(it.confidence, 2));
|
||||
}
|
||||
}
|
||||
|
||||
@ -2,8 +2,16 @@
|
||||
// its own. Scrolling never waits on the disk: previews are read and decoded on worker threads,
|
||||
// those in view first and then the rows around them, and are uploaded a few per frame. Textures
|
||||
// for previews far out of view are released once there are more than VIEWER_MAX_TEXTURES.
|
||||
//
|
||||
// When the workers have no previews to decode they load the catalog (app/catalog.jai), what search
|
||||
// and sorting need from each capture. The grid shows viewer.visible: the captures in display order,
|
||||
// newest first once the catalog is complete, and only those matching the search.
|
||||
|
||||
VIEWER_FOLDER :: "D:/SnagitIndex/mcap"; // hardcoded for now
|
||||
VIEWER_FOLDER :: "D:/SnagitIndex/mcap"; // hardcoded for now; -library FOLDER shows another
|
||||
|
||||
viewer_folder :: () -> string {
|
||||
return ifx app.library_folder then app.library_folder else VIEWER_FOLDER;
|
||||
}
|
||||
VIEWER_WORKER_COUNT :: 4;
|
||||
VIEWER_MAX_TEXTURES :: 1200; // previews on the GPU at once, about 200 KB each
|
||||
VIEWER_PREFETCH_ROWS :: 8; // most rows decoded ahead of the view, above and below
|
||||
@ -36,6 +44,13 @@ Viewer_Capture :: struct {
|
||||
last_wanted_frame: u64; // last in the rows viewer_request_previews asked for
|
||||
failed: bool;
|
||||
in_flight: bool; // under viewer.mutex: queued for a worker, or being decoded
|
||||
catalog: *Catalog_Entry; // main thread only; null until loaded
|
||||
}
|
||||
|
||||
// A catalog entry made off the main thread, for the main thread to put in place.
|
||||
Catalog_Update :: struct {
|
||||
index: s32;
|
||||
entry: *Catalog_Entry; // null when it could not be loaded
|
||||
}
|
||||
|
||||
// How the full-size window sizes its image.
|
||||
@ -78,6 +93,17 @@ viewer: struct {
|
||||
quit: bool;
|
||||
workers: [VIEWER_WORKER_COUNT] Thread;
|
||||
workers_started: s64;
|
||||
catalog_next: s64; // the next capture whose catalog entry a worker loads
|
||||
catalog_updates: [..] Catalog_Update; // made, waiting for the main thread
|
||||
|
||||
// Display order and search; main thread only.
|
||||
visible: [..] s32; // indices into captures, as the grid shows them
|
||||
visible_dirty := true;
|
||||
visible_rebuilt_at: Apollo_Time;
|
||||
catalog_loaded: s64; // entries put in place, or found unreadable
|
||||
sorted_by_date: bool;
|
||||
search_buffer: [256] u8;
|
||||
search_terms: [..] Search_Term;
|
||||
|
||||
// The full view; main thread only.
|
||||
full_open: bool;
|
||||
@ -98,12 +124,15 @@ viewer_startup :: () {
|
||||
viewer.with_texture.allocator = context.default_allocator;
|
||||
viewer.queue.allocator = context.default_allocator;
|
||||
viewer.done.allocator = context.default_allocator;
|
||||
viewer.catalog_updates.allocator = context.default_allocator;
|
||||
viewer.visible.allocator = context.default_allocator;
|
||||
viewer.search_terms.allocator = context.default_allocator;
|
||||
|
||||
init(*viewer.mutex, "viewer");
|
||||
init(*viewer.wake);
|
||||
|
||||
start := current_time_monotonic();
|
||||
files := file_list(VIEWER_FOLDER,, temp);
|
||||
files := file_list(viewer_folder(),, temp);
|
||||
for files {
|
||||
name := path_filename(it);
|
||||
if !ends_with_nocase(name, tprint(".%", MCAP_EXTENSION)) continue;
|
||||
@ -113,8 +142,8 @@ viewer_startup :: () {
|
||||
}
|
||||
// Newest first, roughly: dated names (2026-...) sort after the older {GUID} ones.
|
||||
quick_sort(viewer.captures, (a: Viewer_Capture, b: Viewer_Capture) -> s64 { return compare(b.name, a.name); });
|
||||
if !viewer.captures.count viewer.scan_error = sprint("No .% files in %.", MCAP_EXTENSION, VIEWER_FOLDER);
|
||||
log("[viewer] Found % captures in % in % ms.", viewer.captures.count, VIEWER_FOLDER, FF(to_float64_seconds(current_time_monotonic() - start) * 1000.0, 1));
|
||||
if !viewer.captures.count viewer.scan_error = sprint("No .% files in %.", MCAP_EXTENSION, viewer_folder());
|
||||
log("[viewer] Found % captures in % in % ms.", viewer.captures.count, viewer_folder(), FF(to_float64_seconds(current_time_monotonic() - start) * 1000.0, 1));
|
||||
|
||||
for * viewer.workers {
|
||||
// Reading a front takes 64 KB of temporary storage, and a full image several MB.
|
||||
@ -144,6 +173,7 @@ viewer_worker :: (thread: *Thread) -> s64 {
|
||||
while true {
|
||||
index: s32 = -1;
|
||||
full: bool;
|
||||
catalog: bool;
|
||||
request: u64;
|
||||
path: string;
|
||||
|
||||
@ -157,11 +187,26 @@ viewer_worker :: (thread: *Thread) -> s64 {
|
||||
} else if viewer.queue_next < viewer.queue.count {
|
||||
index = viewer.queue[viewer.queue_next];
|
||||
viewer.queue_next += 1;
|
||||
} else if viewer.catalog_next < viewer.captures.count {
|
||||
// Nothing to show is waiting: load the next catalog entry.
|
||||
catalog = true;
|
||||
index = cast(s32) viewer.catalog_next;
|
||||
viewer.catalog_next += 1;
|
||||
}
|
||||
if index >= 0 path = viewer.captures[index].path; // the array no longer grows after startup
|
||||
unlock(*viewer.mutex);
|
||||
if index < 0 break; // nothing to do: wait to be woken
|
||||
|
||||
if catalog {
|
||||
entry, error := catalog_load(path);
|
||||
if !entry log_error("[catalog] % %", path, error);
|
||||
reset_temporary_storage();
|
||||
lock(*viewer.mutex);
|
||||
array_add(*viewer.catalog_updates, .{index, entry});
|
||||
unlock(*viewer.mutex);
|
||||
continue;
|
||||
}
|
||||
|
||||
result := ifx full then viewer_decode_full(path) else viewer_decode_preview(path);
|
||||
result.index = index;
|
||||
|
||||
@ -318,6 +363,8 @@ viewer_frame_gpu_work :: () {
|
||||
}
|
||||
|
||||
viewer_evict();
|
||||
viewer_apply_catalog_updates();
|
||||
if viewer.visible_dirty viewer_rebuild_visible();
|
||||
}
|
||||
|
||||
// When a preview was last in view or wanted ahead of it. A prefetched preview has not been drawn
|
||||
@ -326,6 +373,69 @@ viewer_last_used :: (capture: Viewer_Capture) -> u64 {
|
||||
return max(capture.last_drawn_frame, capture.last_wanted_frame);
|
||||
}
|
||||
|
||||
// Catalog entries made by the workers (or by an OCR scan) go in place here, on the main thread,
|
||||
// which is the only one that reads them; a replaced entry is freed now, before anything draws it.
|
||||
viewer_apply_catalog_updates :: () {
|
||||
lock(*viewer.mutex);
|
||||
updates := array_copy(viewer.catalog_updates,, temp);
|
||||
array_reset_keeping_memory(*viewer.catalog_updates);
|
||||
more_to_load := viewer.catalog_next < viewer.captures.count;
|
||||
unlock(*viewer.mutex);
|
||||
|
||||
for updates {
|
||||
capture := *viewer.captures[it.index];
|
||||
if !capture.catalog viewer.catalog_loaded += 1;
|
||||
if it.entry || !capture.catalog {
|
||||
catalog_entry_free(capture.catalog);
|
||||
capture.catalog = it.entry;
|
||||
}
|
||||
}
|
||||
if updates.count && viewer.search_terms.count viewer.visible_dirty = true;
|
||||
if !viewer.sorted_by_date && viewer.catalog_loaded >= viewer.captures.count {
|
||||
viewer.sorted_by_date = true;
|
||||
viewer.visible_dirty = true;
|
||||
}
|
||||
// The workers sleep when there is nothing to show; wake them while there is catalog to load.
|
||||
if more_to_load signal(*viewer.wake);
|
||||
}
|
||||
|
||||
// The captures in display order: by date, newest first, once every date is known (by name until
|
||||
// then), and only those matching the search.
|
||||
viewer_rebuild_visible :: () {
|
||||
// While the catalog loads under a search, matches arrive every frame: a few rebuilds a second do.
|
||||
if viewer.search_terms.count && !viewer.sorted_by_date && to_float64_seconds(current_time_monotonic() - viewer.visible_rebuilt_at) < 0.25 return;
|
||||
viewer.visible_dirty = false;
|
||||
viewer.visible_rebuilt_at = current_time_monotonic();
|
||||
|
||||
array_reset_keeping_memory(*viewer.visible);
|
||||
for viewer.captures {
|
||||
if viewer.search_terms.count && !search_matches(it.catalog, viewer.search_terms) continue;
|
||||
array_add(*viewer.visible, cast(s32) it_index);
|
||||
}
|
||||
if viewer.sorted_by_date {
|
||||
quick_sort(viewer.visible, (a: s32, b: s32) -> s64 {
|
||||
ca, cb := *viewer.captures[a], *viewer.captures[b];
|
||||
return viewer_display_order(ifx ca.catalog then ca.catalog.date else "", ca.name, ifx cb.catalog then cb.catalog.date else "", cb.name);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// The grid's order: newest CaptureDate first, undated captures last, ties by name, newest first.
|
||||
viewer_display_order :: (date_a: string, name_a: string, date_b: string, name_b: string) -> s64 {
|
||||
if date_a != date_b {
|
||||
if !date_a.count return 1;
|
||||
if !date_b.count return -1;
|
||||
return compare(date_b, date_a);
|
||||
}
|
||||
return compare(name_b, name_a);
|
||||
}
|
||||
|
||||
// Where index is in the display order, or -1.
|
||||
viewer_visible_position :: (index: s32) -> s64 {
|
||||
for viewer.visible if it == index return it_index;
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Past VIEWER_MAX_TEXTURES, the previews used longest ago are released, never one in view or
|
||||
// wanted around it as of the last frame.
|
||||
viewer_evict :: () {
|
||||
@ -350,8 +460,8 @@ viewer_evict :: () {
|
||||
|
||||
// What the workers should decode next: the rows in view, then the rows around them, nearest first.
|
||||
viewer_request_previews :: (first_row: s64, last_row: s64, columns: s64) {
|
||||
if !viewer.captures.count || columns <= 0 return;
|
||||
rows := (viewer.captures.count + columns - 1) / columns;
|
||||
if !viewer.visible.count || columns <= 0 return;
|
||||
rows := (viewer.visible.count + columns - 1) / columns;
|
||||
|
||||
// Rows ahead only as far as the textures they need leave half of VIEWER_MAX_TEXTURES spare, so
|
||||
// what is wanted always fits and nothing wanted is ever released to make room.
|
||||
@ -363,8 +473,9 @@ viewer_request_previews :: (first_row: s64, last_row: s64, columns: s64) {
|
||||
wanted.allocator = temp;
|
||||
add_row :: (wanted: *[..] s32, row: s64, columns: s64) {
|
||||
for column: 0..columns - 1 {
|
||||
index := row * columns + column;
|
||||
if index >= viewer.captures.count break;
|
||||
position := row * columns + column;
|
||||
if position >= viewer.visible.count break;
|
||||
index := viewer.visible[position];
|
||||
capture := *viewer.captures[index];
|
||||
capture.last_wanted_frame = viewer.frame; // kept by viewer_evict, texture or not
|
||||
if !capture.texture.handle && !capture.failed array_add(wanted, cast(s32) index);
|
||||
@ -413,11 +524,25 @@ viewer_ui :: () { using ImGui;
|
||||
|
||||
if viewer.scan_error { TextColored(app_theme.palette.warning, "%", viewer.scan_error); return; }
|
||||
|
||||
loaded := viewer.with_texture.count;
|
||||
Text("% captures in %", viewer.captures.count, VIEWER_FOLDER);
|
||||
SetNextItemWidth(min(GetContentRegionAvail().x * 0.5, 520));
|
||||
if InputTextWithHint("##search", "Search text, titles, URLs, apps, notes. \"a phrase\" ocr: title: url: app: note:", viewer.search_buffer.data, viewer.search_buffer.count) {
|
||||
query := to_string(viewer.search_buffer.data);
|
||||
terms := search_parse(query);
|
||||
for viewer.search_terms free(it.text);
|
||||
array_reset_keeping_memory(*viewer.search_terms);
|
||||
for terms array_add(*viewer.search_terms, .{it.field, copy_string(it.text)});
|
||||
viewer.visible_dirty = true;
|
||||
viewer.visible_rebuilt_at = .{}; // at once, not throttled
|
||||
}
|
||||
SameLine();
|
||||
TextColored(app_theme.palette.dim, "(% previews loaded)", loaded);
|
||||
SetNextItemWidth(200);
|
||||
if viewer.search_terms.count Text("% of % captures", viewer.visible.count, viewer.captures.count);
|
||||
else Text("% captures", viewer.captures.count);
|
||||
if viewer.catalog_loaded < viewer.captures.count {
|
||||
SameLine();
|
||||
TextColored(app_theme.palette.dim, "(loading the catalog: % of %; sorted by name until then)", viewer.catalog_loaded, viewer.captures.count);
|
||||
}
|
||||
SameLine();
|
||||
SetNextItemWidth(160);
|
||||
SliderFloat("Size", *viewer.thumb_size, 64, MCAP_PREVIEW_MAX_SIDE, "%.0f px");
|
||||
|
||||
BeginChild("grid");
|
||||
@ -433,7 +558,7 @@ viewer_grid :: () { using ImGui;
|
||||
label := GetTextLineHeight();
|
||||
available := GetContentRegionAvail().x;
|
||||
columns := max(1, cast(s64) ((available + style.ItemSpacing.x) / (cell + style.ItemSpacing.x)));
|
||||
rows := (viewer.captures.count + columns - 1) / columns;
|
||||
rows := (viewer.visible.count + columns - 1) / columns;
|
||||
row_height := cell + label + style.ItemSpacing.y;
|
||||
|
||||
draw := GetWindowDrawList();
|
||||
@ -452,8 +577,9 @@ viewer_grid :: () { using ImGui;
|
||||
last_row = max(last_row, cast(s64) clipper.DisplayEnd - 1);
|
||||
for row: clipper.DisplayStart..clipper.DisplayEnd - 1 {
|
||||
for column: 0..columns - 1 {
|
||||
index := row * columns + column;
|
||||
if index >= viewer.captures.count break;
|
||||
position := row * columns + column;
|
||||
if position >= viewer.visible.count break;
|
||||
index := viewer.visible[position];
|
||||
capture := *viewer.captures[index];
|
||||
if column > 0 SameLine();
|
||||
|
||||
@ -530,8 +656,9 @@ viewer_full_window :: () { using ImGui;
|
||||
|
||||
// Left and Right step through the captures in grid order while this window has focus.
|
||||
if IsWindowFocused(.RootAndChildWindows) {
|
||||
if IsKeyPressed(.LeftArrow) && viewer.full_index > 0 viewer_request_full(viewer.full_index - 1);
|
||||
if IsKeyPressed(.RightArrow) && viewer.full_index < viewer.captures.count - 1 viewer_request_full(viewer.full_index + 1);
|
||||
position := viewer_visible_position(viewer.full_index);
|
||||
if IsKeyPressed(.LeftArrow) && position > 0 viewer_request_full(viewer.visible[position - 1]);
|
||||
if IsKeyPressed(.RightArrow) && position >= 0 && position < viewer.visible.count - 1 viewer_request_full(viewer.visible[position + 1]);
|
||||
}
|
||||
|
||||
if viewer.full_failed {
|
||||
|
||||
121
src/test/catalog_tests.jai
Normal file
121
src/test/catalog_tests.jai
Normal file
@ -0,0 +1,121 @@
|
||||
// The catalog's pieces on their own (app/catalog.jai, app/viewer.jai): the search query's parsing and
|
||||
// matching, the RTF stripper on Snagit's annotations, and the grid's order by date.
|
||||
|
||||
#scope_file
|
||||
|
||||
checks_run := 0;
|
||||
checks_failed := 0;
|
||||
|
||||
MAX_GROUPS :: 8;
|
||||
group_names : [MAX_GROUPS] string;
|
||||
group_checks : [MAX_GROUPS] int;
|
||||
group_fails : [MAX_GROUPS] int;
|
||||
group_count := 0;
|
||||
|
||||
output_builder: *String_Builder;
|
||||
|
||||
emit :: (format_string: string, args: ..Any) {
|
||||
if output_builder print_to_builder(output_builder, format_string, ..args);
|
||||
else print(format_string, ..args);
|
||||
} @PrintLike
|
||||
|
||||
group :: (name: string) {
|
||||
assert(group_count < MAX_GROUPS, "raise MAX_GROUPS");
|
||||
group_names[group_count] = name;
|
||||
group_count += 1;
|
||||
emit("\n-- % --\n", name);
|
||||
}
|
||||
|
||||
expect :: (condition: bool, format: string, args: ..Any, loc := #caller_location) {
|
||||
checks_run += 1;
|
||||
if group_count group_checks[group_count-1] += 1;
|
||||
if condition return;
|
||||
|
||||
checks_failed += 1;
|
||||
if group_count group_fails[group_count-1] += 1;
|
||||
emit(" FAIL [%:%] %\n", group_names[max(group_count-1, 0)], loc.line_number, tprint(format, ..args));
|
||||
}
|
||||
|
||||
report :: () -> bool {
|
||||
emit("\n");
|
||||
for 0..group_count-1 {
|
||||
if group_fails[it] emit(" % -- % checks, % FAILED\n", group_names[it], group_checks[it], group_fails[it]);
|
||||
else emit(" % -- % checks\n", group_names[it], group_checks[it]);
|
||||
}
|
||||
if checks_failed emit("\n% OF % CATALOG CHECKS FAILED\n", checks_failed, checks_run);
|
||||
else emit("\nALL CATALOG TESTS PASSED (% checks over % groups)\n", checks_run, group_count);
|
||||
return checks_failed == 0;
|
||||
}
|
||||
|
||||
query_tests :: () {
|
||||
group("query");
|
||||
terms := search_parse(" KiCad \"Footprint Assignments\" url:digikey note:late http://x.y ");
|
||||
expect(terms.count == 5, "five terms (got %)", terms.count);
|
||||
if terms.count != 5 return;
|
||||
expect(terms[0].field == .ANY && terms[0].text == "kicad", "a word, lowercased: %", terms[0]);
|
||||
expect(terms[1].field == .ANY && terms[1].text == "footprint assignments", "a phrase: %", terms[1]);
|
||||
expect(terms[2].field == .URL && terms[2].text == "digikey", "a field prefix: %", terms[2]);
|
||||
expect(terms[3].field == .NOTE && terms[3].text == "late", "note: %", terms[3]);
|
||||
expect(terms[4].field == .ANY && terms[4].text == "http://x.y", "an unknown prefix stays part of the word: %", terms[4]);
|
||||
|
||||
phrase := search_parse("title:\"two words\"");
|
||||
expect(phrase.count == 1 && phrase[0].field == .TITLE && phrase[0].text == "two words", "a prefixed phrase: %", phrase);
|
||||
expect(search_parse(" ").count == 0, "blank is no terms");
|
||||
}
|
||||
|
||||
match_tests :: () {
|
||||
group("matching");
|
||||
entry: Catalog_Entry;
|
||||
entry.find_title = "assign footprints - kicad";
|
||||
entry.find_url = "https://www.digikey.com/x";
|
||||
entry.find_app = "kicad eda 7.0.2";
|
||||
entry.find_notes = "late";
|
||||
entry.find_ocr = "0.1uf : capacitor_smd:c_0201_0603metric\nc1 -\n";
|
||||
|
||||
expect(search_matches(*entry, search_parse("capacitor kicad")), "words from different fields all match");
|
||||
expect(!search_matches(*entry, search_parse("capacitor resistor")), "every word must match");
|
||||
expect(search_matches(*entry, search_parse("url:digikey")), "a URL word");
|
||||
expect(!search_matches(*entry, search_parse("title:digikey")), "a prefix limits the word to its field");
|
||||
expect(search_matches(*entry, search_parse("ocr:\"c1 -\"")), "an OCR phrase");
|
||||
expect(search_matches(*entry, search_parse("note:LATE")), "matching ignores case");
|
||||
expect(search_matches(*entry, search_parse("")), "no terms match everything");
|
||||
expect(!search_matches(null, search_parse("x")), "an entry not loaded matches no search");
|
||||
|
||||
expect(search_line_matches("0.1uF : Capacitor_SMD", search_parse("capacitor")), "a line holding a word");
|
||||
expect(!search_line_matches("0.1uF : Capacitor_SMD", search_parse("title:capacitor")), "a title word is not an OCR line's");
|
||||
}
|
||||
|
||||
rtf_tests :: () {
|
||||
group("rtf");
|
||||
// A Snagit text annotation, as stored (base64 RTF), whose text is "late".
|
||||
encoded :: "e1xydGYxXGFuc2lcYW5zaWNwZzEyNTJcZGVmZjBcZGVmbGFuZzEwMzN7XGZvbnR0Ymx7XGYwXGZzd2lzc1xmcHJxMlxmY2hhcnNldDAgQXJpYWw7fX0NCntcY29sb3J0YmwgO1xyZWQyNTJcZ3JlZW42NlxibHVlNjY7fQ0KXHZpZXdraW5kNFx1YzFccGFyZFxxY1x0eDcyMFxjZjFcZjBcZnMyNCBsYXRlXHBhcg0KfQ0K";
|
||||
rtf, decoded := base64_decode(encoded,, temp);
|
||||
expect(decoded, "the sample decodes");
|
||||
text := rtf_to_text(rtf);
|
||||
expect(text == "late", "a Snagit annotation reads as its text: \"%\"", text);
|
||||
|
||||
text = rtf_to_text("{\\rtf1{\\fonttbl{\\f0 Arial;}}{\\*\\generator Riched20;}one\\par two\\tab three caf\\'e9 \\u8364? \\{x\\}}");
|
||||
expect(text == "one\ntwo\tthree caf\xC3\xA9 \xE2\x82\xAC {x}", "breaks, tabs, \\'hh, \\uN and escaped braces: \"%\"", text);
|
||||
expect(rtf_to_text("{\\rtf1\\ansi\\'93quoted\\'94}") == "\xE2\x80\x9Cquoted\xE2\x80\x9D", "Windows-1252 quotes");
|
||||
}
|
||||
|
||||
order_tests :: () {
|
||||
group("order");
|
||||
expect(viewer_display_order("2026-10-07 08:00:00", "a", "2025-01-01 00:00:00", "b") < 0, "newer first");
|
||||
expect(viewer_display_order("", "a", "2025-01-01 00:00:00", "b") > 0, "undated after dated");
|
||||
expect(viewer_display_order("2025-01-01 00:00:00", "a", "", "b") < 0, "dated before undated");
|
||||
expect(viewer_display_order("2025-01-01 00:00:00", "b", "2025-01-01 00:00:00", "a") < 0, "the same date: by name, newest first");
|
||||
}
|
||||
|
||||
#scope_export
|
||||
|
||||
run_catalog_tests :: (builder: *String_Builder = null) -> bool {
|
||||
output_builder = builder;
|
||||
emit("=== catalog test suite ===\n");
|
||||
push_allocator(context.default_allocator);
|
||||
query_tests();
|
||||
match_tests();
|
||||
rtf_tests();
|
||||
order_tests();
|
||||
return report();
|
||||
}
|
||||
213
src/test/ocr_tests.jai
Normal file
213
src/test/ocr_tests.jai
Normal file
@ -0,0 +1,213 @@
|
||||
// OCR on turned text (modules/OCR): two lines drawn with Windows' GDI, read level, then turned by 15°,
|
||||
// 90° (read downwards), 180° and 270° (read upwards). Each turn must give back the same text, and the
|
||||
// line's corners must run the way the text does. Needs the models beside the program (the build copies
|
||||
// them); without them, or off Windows, the suite says so and passes.
|
||||
|
||||
#scope_file
|
||||
|
||||
checks_run := 0;
|
||||
checks_failed := 0;
|
||||
|
||||
MAX_GROUPS :: 8;
|
||||
group_names : [MAX_GROUPS] string;
|
||||
group_checks : [MAX_GROUPS] int;
|
||||
group_fails : [MAX_GROUPS] int;
|
||||
group_count := 0;
|
||||
|
||||
output_builder: *String_Builder;
|
||||
|
||||
emit :: (format_string: string, args: ..Any) {
|
||||
if output_builder print_to_builder(output_builder, format_string, ..args);
|
||||
else print(format_string, ..args);
|
||||
} @PrintLike
|
||||
|
||||
group :: (name: string) {
|
||||
assert(group_count < MAX_GROUPS, "raise MAX_GROUPS");
|
||||
group_names[group_count] = name;
|
||||
group_count += 1;
|
||||
emit("\n-- % --\n", name);
|
||||
}
|
||||
|
||||
expect :: (condition: bool, format: string, args: ..Any, loc := #caller_location) {
|
||||
checks_run += 1;
|
||||
if group_count group_checks[group_count-1] += 1;
|
||||
if condition return;
|
||||
|
||||
checks_failed += 1;
|
||||
if group_count group_fails[group_count-1] += 1;
|
||||
emit(" FAIL [%:%] %\n", group_names[max(group_count-1, 0)], loc.line_number, tprint(format, ..args));
|
||||
}
|
||||
|
||||
report :: () -> bool {
|
||||
emit("\n");
|
||||
for 0..group_count-1 {
|
||||
if group_fails[it] emit(" % -- % checks, % FAILED\n", group_names[it], group_checks[it], group_fails[it]);
|
||||
else emit(" % -- % checks\n", group_names[it], group_checks[it]);
|
||||
}
|
||||
if checks_failed emit("\n% OF % OCR CHECKS FAILED\n", checks_failed, checks_run);
|
||||
else emit("\nALL OCR TESTS PASSED (% checks over % groups)\n", checks_run, group_count);
|
||||
return checks_failed == 0;
|
||||
}
|
||||
|
||||
LINES :: string.["Assign Footprints to Symbols", "Capacitor 100nF 0402"];
|
||||
TURNS :: float.[0, 15, 90, 180, 270]; // degrees, clockwise on the screen; 0 is the text as drawn
|
||||
|
||||
// Lowercase, without spaces: recognition may split or join words differently once turned.
|
||||
squeezed :: (s: string) -> string {
|
||||
builder: String_Builder;
|
||||
builder.allocator = temp;
|
||||
for 0..s.count - 1 if s[it] != #char " " append(*builder, to_lower(s[it]));
|
||||
return builder_to_string(*builder,, temp);
|
||||
}
|
||||
|
||||
// The line found holding wanted, or null.
|
||||
find_line :: (lines: [] Ocr_Line, wanted: string) -> *Ocr_Line {
|
||||
for * lines if contains(squeezed(it.text), squeezed(wanted)) return it;
|
||||
return null;
|
||||
}
|
||||
|
||||
// The image turned clockwise by degrees about its centre onto a white canvas just big enough. Temp.
|
||||
turned_image :: (source: Ocr_Image, degrees: float) -> Ocr_Image {
|
||||
radians := degrees * PI / 180;
|
||||
c, s := cos(radians), sin(radians);
|
||||
if abs(c) < 0.000001 c = 0;
|
||||
if abs(s) < 0.000001 s = 0;
|
||||
result: Ocr_Image;
|
||||
result.width = cast(s32) ceil(abs(source.width * c) + abs(source.height * s));
|
||||
result.height = cast(s32) ceil(abs(source.width * s) + abs(source.height * c));
|
||||
result.stride = result.width * 4;
|
||||
result.pixels = alloc(cast(s64) result.stride * result.height,, temp);
|
||||
source_centre := Vector2.{source.width / 2.0, source.height / 2.0};
|
||||
centre := Vector2.{result.width / 2.0, result.height / 2.0};
|
||||
for y: 0..result.height - 1 for x: 0..result.width - 1 {
|
||||
d := Vector2.{x + 0.5, y + 0.5} - centre;
|
||||
p := Vector2.{c * d.x + s * d.y, -s * d.x + c * d.y} + source_centre - Vector2.{0.5, 0.5}; // turned back
|
||||
out := result.pixels + cast(s64) y * result.stride + x * 4;
|
||||
x0, y0 := cast(s32) floor(p.x), cast(s32) floor(p.y);
|
||||
fx, fy := p.x - x0, p.y - y0;
|
||||
for channel: 0..3 {
|
||||
sample :: (image: Ocr_Image, x: s32, y: s32, channel: s64) -> float {
|
||||
if x < 0 || y < 0 || x >= image.width || y >= image.height return 255;
|
||||
return cast(float) (image.pixels + cast(s64) y * image.stride + x * 4)[channel];
|
||||
}
|
||||
top := sample(source, x0, y0, channel) * (1 - fx) + sample(source, x0 + 1, y0, channel) * fx;
|
||||
bottom := sample(source, x0, y0 + 1, channel) * (1 - fx) + sample(source, x0 + 1, y0 + 1, channel) * fx;
|
||||
out[channel] = cast(u8) clamp(top * (1 - fy) + bottom * fy + 0.5, 0, 255);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
#if OS == .WINDOWS {
|
||||
test_gdi :: #library,system "Gdi32";
|
||||
TestCreateFontW :: (height: s32, width: s32, escapement: s32, orientation: s32, weight: s32, italic: u32, underline: u32, strike_out: u32, char_set: u32, out_precision: u32, clip_precision: u32, quality: u32, pitch_and_family: u32, face_name: *u16) -> *void #foreign test_gdi "CreateFontW";
|
||||
TestCreateDIBSection :: (hdc: HDC, info: *BITMAPINFO, usage: u32, bits: **void, section: *void, offset: u32) -> *void #foreign test_gdi "CreateDIBSection";
|
||||
TestSetTextColor :: (hdc: HDC, color: u32) -> u32 #foreign test_gdi "SetTextColor";
|
||||
TestSetBkColor :: (hdc: HDC, color: u32) -> u32 #foreign test_gdi "SetBkColor";
|
||||
TestTextOutW :: (hdc: HDC, x: s32, y: s32, text: *u16, count: s32) -> s32 #foreign test_gdi "TextOutW";
|
||||
TestGdiFlush :: () -> s32 #foreign test_gdi "GdiFlush";
|
||||
ANTIALIASED_QUALITY :: 4;
|
||||
|
||||
// LINES, black on white, in Segoe UI. Temp.
|
||||
drawn_lines :: () -> Ocr_Image, ok: bool {
|
||||
image: Ocr_Image;
|
||||
image.width, image.height = 720, 170;
|
||||
image.stride = image.width * 4;
|
||||
|
||||
dc := CreateCompatibleDC(null);
|
||||
if !dc return image, false;
|
||||
defer DeleteDC(dc);
|
||||
info: BITMAPINFO;
|
||||
info.bmiHeader = .{biSize = size_of(BITMAPINFOHEADER), biWidth = image.width, biHeight = -image.height, biPlanes = 1, biBitCount = 32}; // top-down
|
||||
bits: *void;
|
||||
bitmap := TestCreateDIBSection(dc, *info, 0, *bits, null, 0);
|
||||
if !bitmap || !bits return image, false;
|
||||
defer DeleteObject(bitmap);
|
||||
font := TestCreateFontW(-34, 0, 0, 0, 400, 0, 0, 0, 1, 0, 0, ANTIALIASED_QUALITY, 0, utf8_to_wide("Segoe UI",, temp));
|
||||
if !font return image, false;
|
||||
defer DeleteObject(font);
|
||||
|
||||
old_bitmap := SelectObject(dc, bitmap);
|
||||
old_font := SelectObject(dc, font);
|
||||
memset(bits, 0xFF, cast(s64) image.stride * image.height);
|
||||
TestSetTextColor(dc, 0x000000);
|
||||
TestSetBkColor(dc, 0xFFFFFF);
|
||||
for LINES {
|
||||
wide, _, bytes := utf8_to_wide(it,, temp);
|
||||
TestTextOutW(dc, 40, cast(s32) (30 + it_index * 64), wide, bytes / 2);
|
||||
}
|
||||
TestGdiFlush();
|
||||
|
||||
// BGRA to RGBA.
|
||||
image.pixels = alloc(cast(s64) image.stride * image.height,, temp);
|
||||
source := cast(*u8) bits;
|
||||
for i: 0..cast(s64) image.width * image.height - 1 {
|
||||
image.pixels[i * 4] = source[i * 4 + 2];
|
||||
image.pixels[i * 4 + 1] = source[i * 4 + 1];
|
||||
image.pixels[i * 4 + 2] = source[i * 4];
|
||||
image.pixels[i * 4 + 3] = 255;
|
||||
}
|
||||
SelectObject(dc, old_font);
|
||||
SelectObject(dc, old_bitmap);
|
||||
return image, true;
|
||||
}
|
||||
}
|
||||
|
||||
turn_tests :: () {
|
||||
#if OS != .WINDOWS {
|
||||
emit(" (skipped: the test text is drawn with GDI, on Windows only)\n");
|
||||
return;
|
||||
} else {
|
||||
group("turned text");
|
||||
folder := ocr_models_folder();
|
||||
if !file_exists(tprint("%/models.config", folder)) {
|
||||
emit(" (skipped: no OCR models in %)\n", folder);
|
||||
return;
|
||||
}
|
||||
engine, created, error := ocr_create(folder, OCR_DEFAULT_MODEL);
|
||||
expect(created, "the OCR engine starts: %", error);
|
||||
if !created return;
|
||||
defer ocr_destroy(engine);
|
||||
expect(engine.orientation.session != null, "% has an orientation classifier", OCR_DEFAULT_MODEL);
|
||||
emit(" running on %; classifier input % x %\n", ifx engine.provider == .CUDA then "CUDA" else "the CPU", engine.orientation_width, engine.orientation_height);
|
||||
|
||||
level, drawn := drawn_lines();
|
||||
expect(drawn, "the test text is drawn");
|
||||
if !drawn return;
|
||||
|
||||
for degrees: TURNS {
|
||||
image := ifx degrees == 0 then level else turned_image(level, degrees);
|
||||
results := ocr_process(engine, Ocr_Image.[image],, temp);
|
||||
expect(results.count == 1 && !results[0].failed, "% degrees: OCR runs (%)", degrees, ifx results.count then results[0].error else "");
|
||||
if results.count != 1 || results[0].failed continue;
|
||||
lines := results[0].lines;
|
||||
|
||||
radians := degrees * PI / 180;
|
||||
direction := Vector2.{cos(radians), sin(radians)}; // the way turned text reads
|
||||
for wanted: LINES {
|
||||
line := find_line(lines, wanted);
|
||||
if !line {
|
||||
found: [..] string;
|
||||
found.allocator = temp;
|
||||
for lines array_add(*found, it.text);
|
||||
expect(false, "% degrees: \"%\" is read (found %)", degrees, wanted, found);
|
||||
continue;
|
||||
}
|
||||
along := line.corners[1] - line.corners[0];
|
||||
along_length := length(along);
|
||||
cosine := ifx along_length > 0 then dot(along, direction) / along_length else 0;
|
||||
expect(cosine > 0.95, "% degrees: \"%\" has corners that read its way (cosine %, corners %)", degrees, wanted, cosine, line.corners);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#scope_export
|
||||
|
||||
run_ocr_tests :: (builder: *String_Builder = null) -> bool {
|
||||
output_builder = builder;
|
||||
emit("=== OCR test suite ===\n");
|
||||
push_allocator(context.default_allocator);
|
||||
turn_tests();
|
||||
return report();
|
||||
}
|
||||
@ -38,6 +38,8 @@ Test_Type :: enum s32 {
|
||||
fixed_arena_suite;
|
||||
secure_memory_suite;
|
||||
logger_suite;
|
||||
catalog_suite;
|
||||
ocr_suite;
|
||||
}
|
||||
|
||||
testing_started :: () -> bool {
|
||||
@ -78,6 +80,8 @@ run_all_tests :: () {
|
||||
add_suite_test(.fixed_arena_suite, "fixed_arena");
|
||||
add_suite_test(.secure_memory_suite, "secure_memory");
|
||||
add_suite_test(.logger_suite, "logger");
|
||||
add_suite_test(.catalog_suite, "catalog");
|
||||
add_suite_test(.ocr_suite, "ocr");
|
||||
}
|
||||
|
||||
// One work item per suite. Each suite keeps its tallies in file-scope globals, so the whole
|
||||
@ -182,6 +186,10 @@ test_thread_group_proc :: (group: *Thread_Group, thread: *Thread, work: *void) -
|
||||
task.pass = run_secure_memory_tests(task.output);
|
||||
case .logger_suite;
|
||||
task.pass = run_logger_tests(task.output);
|
||||
case .catalog_suite;
|
||||
task.pass = run_catalog_tests(task.output);
|
||||
case .ocr_suite;
|
||||
task.pass = run_ocr_tests(task.output);
|
||||
}
|
||||
|
||||
return .CONTINUE;
|
||||
|
||||
@ -292,6 +292,7 @@ app_theme_set_status :: (message: string, is_error: bool) {
|
||||
app_user_settings: App_User_Settings;
|
||||
App_User_Settings :: struct {
|
||||
theme: string; // "" means the default theme; heap-allocated
|
||||
ocr_model: string; // [ocr] model: the OCR model's name in models.config; "" means the default; heap-allocated
|
||||
|
||||
// Of the monitor the window first opens on.
|
||||
window_width_fraction: float = 0.80;
|
||||
@ -313,6 +314,7 @@ App_User_Settings :: struct {
|
||||
|
||||
settings_free :: (s: *App_User_Settings) {
|
||||
free(s.theme,, context.default_allocator);
|
||||
free(s.ocr_model,, context.default_allocator);
|
||||
array_free(s.hotkey_bindings);
|
||||
array_free(s.global_hotkey_bindings);
|
||||
}
|
||||
@ -368,11 +370,11 @@ app_load_user_settings_file :: () -> write_back: bool {
|
||||
if hotkeys_missing.count || global_missing.count return true;
|
||||
|
||||
#if OS == .LINUX seen.global_hotkeys = true; // not written there: Linux has no global hotkeys yet
|
||||
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys);
|
||||
return !(seen.theme && seen.width && seen.height && seen.font_size && seen.font_scale && seen.event_log && seen.hotkeys && seen.global_hotkeys && seen.ocr_model);
|
||||
}
|
||||
|
||||
Settings_Seen :: struct {
|
||||
theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys: bool;
|
||||
theme, width, height, font_size, font_scale, event_log, hotkeys, global_hotkeys, ocr_model: bool;
|
||||
}
|
||||
|
||||
// Of the file's bytes as last loaded or written, so a watcher event for our own save, or for an
|
||||
@ -382,6 +384,7 @@ settings_file_hash: u64;
|
||||
// The settings the app itself changes, as it holds them now.
|
||||
Settings_App_Values :: struct {
|
||||
theme: string;
|
||||
ocr_model: string;
|
||||
font_size_base: float;
|
||||
font_scale_main: float;
|
||||
}
|
||||
@ -395,6 +398,7 @@ settings_app_values :: () -> Settings_App_Values {
|
||||
entry := app_theme_current();
|
||||
values.theme = ifx entry then entry.name else app_user_settings.theme;
|
||||
if !values.theme.count values.theme = APP_DEFAULT_THEME_NAME;
|
||||
values.ocr_model = ifx app_user_settings.ocr_model.count then app_user_settings.ocr_model else OCR_DEFAULT_MODEL;
|
||||
|
||||
values.font_size_base = app_user_settings.font_size_base;
|
||||
values.font_scale_main = app_user_settings.font_scale_main;
|
||||
@ -410,8 +414,10 @@ settings_app_values :: () -> Settings_App_Values {
|
||||
|
||||
settings_note_synced :: () {
|
||||
free(settings_synced.theme,, context.default_allocator);
|
||||
free(settings_synced.ocr_model,, context.default_allocator);
|
||||
settings_synced = settings_app_values();
|
||||
settings_synced.theme = copy_string(settings_synced.theme,, context.default_allocator);
|
||||
settings_synced.ocr_model = copy_string(settings_synced.ocr_model,, context.default_allocator);
|
||||
}
|
||||
|
||||
// Parses the file's contents into out, replacing its theme and hotkey bindings. ok is false when
|
||||
@ -483,7 +489,11 @@ settings_parse :: (path: string, file_data: string, out: *App_User_Settings) ->
|
||||
key, value, split_ok := theme_split_key(body);
|
||||
if !split_ok continue;
|
||||
|
||||
if section == "[ui]" && key == "theme" {
|
||||
if section == "[ocr]" && key == "model" {
|
||||
free(out.ocr_model,, context.default_allocator);
|
||||
out.ocr_model = copy_string(settings_unquote(value),, context.default_allocator);
|
||||
seen.ocr_model = true;
|
||||
} else if section == "[ui]" && key == "theme" {
|
||||
free(out.theme,, context.default_allocator);
|
||||
out.theme = copy_string(settings_unquote(value),, context.default_allocator);
|
||||
seen.theme = true;
|
||||
@ -592,6 +602,7 @@ app_save_user_settings_file :: () -> bool {
|
||||
|
||||
values := settings_app_values();
|
||||
theme := values.theme;
|
||||
ocr_model := values.ocr_model;
|
||||
font_size_base := values.font_size_base;
|
||||
font_scale_main := values.font_scale_main;
|
||||
|
||||
@ -621,6 +632,7 @@ app_save_user_settings_file :: () -> bool {
|
||||
}
|
||||
if disk_ok {
|
||||
if seen.theme && values.theme == settings_synced.theme && on_disk.theme.count theme = on_disk.theme;
|
||||
if seen.ocr_model && values.ocr_model == settings_synced.ocr_model && on_disk.ocr_model.count ocr_model = on_disk.ocr_model;
|
||||
if seen.font_size && values.font_size_base == settings_synced.font_size_base font_size_base = on_disk.font_size_base;
|
||||
if seen.font_scale && values.font_scale_main == settings_synced.font_scale_main font_scale_main = on_disk.font_scale_main;
|
||||
if seen.width app_user_settings.window_width_fraction = on_disk.window_width_fraction;
|
||||
@ -643,6 +655,9 @@ app_save_user_settings_file :: () -> bool {
|
||||
append(*builder, "# Text size is size_base * scale_main * the monitor's DPI scale. The font size hotkeys change scale_main.\n");
|
||||
print_to_builder(*builder, "size_base: %\n", FF(font_size_base, 1));
|
||||
print_to_builder(*builder, "scale_main: %\n\n", FF(font_scale_main, 2));
|
||||
append(*builder, "[ocr]\n");
|
||||
append(*builder, "# The OCR model, by its name in ocr-models/models.config: ppocr-v5-mobile (light) or ppocr-v5-server (heavy).\n");
|
||||
print_to_builder(*builder, "model: %\n\n", settings_quote_if_needed(ocr_model));
|
||||
append(*builder, "[logging]\n");
|
||||
append(*builder, "# Entries the Event Log keeps in memory. Past it the oldest are dropped; the log file keeps every line.\n");
|
||||
print_to_builder(*builder, "event_log_capacity: %\n", app_user_settings.event_log_capacity);
|
||||
@ -766,6 +781,7 @@ app_reload_user_settings_if_changed :: () {
|
||||
}
|
||||
|
||||
if fresh.event_log_capacity != old.event_log_capacity logger_set_event_log_capacity(fresh.event_log_capacity);
|
||||
if fresh.ocr_model != old.ocr_model ocr_ui_settings_changed(); // app/ocr_ui.jai
|
||||
|
||||
bindings: [] Hotkey_Binding = hotkey_defaults();
|
||||
if seen.hotkeys bindings = app_user_settings.hotkey_bindings;
|
||||
|
||||
@ -124,7 +124,8 @@ main :: () {
|
||||
print(" first image alone: % s (warm-up)\n", FF(first_seconds, 3));
|
||||
print(" all % images: % s, % ms per image, % megapixels/s, % failed\n", images.count, FF(seconds, 2), FF(1000 * seconds / images.count, 1), FF(megapixels / seconds, 2), failures);
|
||||
print(" found: % lines, % bytes of text\n", lines, characters);
|
||||
print(" stage time (summed over threads): detection % s, recognition % s\n", FF(stats.detect_us / 1_000_000.0, 2), FF(stats.recognize_us / 1_000_000.0, 2));
|
||||
print(" stage time (summed over threads): detection % s, orientation % s, recognition % s\n", FF(stats.detect_us / 1_000_000.0, 2), FF(stats.orientation_us / 1_000_000.0, 2), FF(stats.recognize_us / 1_000_000.0, 2));
|
||||
print(" lines read better turned: %\n", stats.turned);
|
||||
print(" text in %\n\n", text_folder);
|
||||
ocr_destroy(engine);
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user