[0.0.14] Segmentation phase 5: has: search, and what is in the images in Stats

- has: searches what segmentation found, by the whole class name: has:dog, has:"dining table"
  (has:table is a table, not a dining table). Plain words do not match segments.
- The catalog keeps each capture's segments (class, score, area) and its classes between newlines,
  so a has: term is one contains per capture.
- Stats: "What is in the images", how many of the segmented images hold each class, each one a
  has: search; shown even when nothing has been read by OCR. -stats logs it.
- Tests: has: (whole names, quoted names, case, combinations, plain words not matching).

README.md

@@ -133,7 +133,9 @@ works only in the library folders you choose, and never on videos.
 What OCR found in the captures shown (the whole library, or a search's matches), by the chosen OCR
 model: captures with text, lines, words, characters, confidence, words per capture with a breakdown,
 a table by year, the captures with the most words (click to open), each capture's best label
-counted (click one to search for it), and the commonest words (click one to search for it).
+counted (click one to search for it), and the commonest words (click one to search for it). When any
+of them are segmented, it also shows **What is in the images**: how many images hold each class
+(person, sky, dog ...). Click one to search `has:` for it.
 
 ## Importing
 
@@ -201,7 +203,7 @@ A prefix limits a word, or a quoted phrase, to one field: `title:kicad`, `title:
 
 | Prefix | Looks in |
 | --- | --- |
-| *(none)* | everything below except `like:` |
+| *(none)* | everything below except `like:`, `folder:` and `has:` |
 | `ocr:` | the text read from the image |
 | `title:` | the window title when it was captured; for an imported image, its file name |
 | `url:` | the web page's address, for browser captures |
@@ -210,9 +212,22 @@ A prefix limits a word, or a quoted phrase, to one field: `title:kicad`, `title:
 | `label:` | the capture's labels: its best label, and any scoring at least half as much |
 | `like:` | the image's meaning (below) |
 | `folder:` | the library folder it is in: `folder:work` |
+| `has:` | what segmentation found in the image (below): `has:dog` |
 
 A word with a colon that is not one of these stays an ordinary word, so `http://` and `C:\` work.
 
+### What is in the image: `has:`
+
+`has:dog` finds images that segmentation (the Segmentation window) found a dog in. Only images in
+folders set to segment, and already segmented, take part.
+
+- **The whole class name:** `has:table` is a table, not a dining table; `has:"dining table"` is that.
+- **The 133 COCO classes:** 80 kinds of thing (person, bicycle, car, dog, cat, chair, laptop, cell
+  phone, book ...) and 53 of stuff (sky, wall, grass, road, floor wood, ...). The Stats window lists
+  those in the images shown.
+- **Combining:** `has:person has:dog` finds images with both; `has:cat label:"photo"` mixes it with
+  labels.
+
 ### Searching by meaning: `like:`
 
 `like:"a circuit board under a microscope"` ranks every labelled capture by how close its image is
@@ -239,6 +254,8 @@ then in that order instead of by date.
 | `note:todo` | captures with an annotation saying TODO |
 | `like:"a photo of a soldering iron"` | captures that look like that, best first |
 | `like:"a spreadsheet" app:excel` | Excel captures, the most spreadsheet-like first |
+| `has:dog` | images with a dog in them (segmented folders) |
+| `has:person has:"cell phone"` | images with a person and a phone |
 
 ## Command line
 

check_targets.jai

@@ -29,7 +29,7 @@ APP_NAME        :: "mcap";
 APP_FOLDER_NAME :: "mcap";
 GIT_BRANCH    :: "check";
 GIT_HASH      :: "-";
-VERSION       :: "0.0.13";
+VERSION       :: "0.0.14";
 BUILD_DATE    :: "-";
 DEBUG         :: true;
 COMPILE_TESTS :: true;

first.jai

@@ -8,7 +8,7 @@ APP_NAME              :: "mcap";
 APP_FOLDER_NAME       :: "mcap";
 APP_BUNDLE_IDENTIFIER :: "com.example.mcap";
 
-VERSION :: "0.0.13";
+VERSION :: "0.0.14";
 // Where everything goes: bin/, or bin-headless/ for builds made with no_crash_dialog, so automated
 // builds never write over the ones you run by hand.
 build_directory := "bin";

segmentation-plan.html

@@ -78,7 +78,7 @@
     <li><span class="tag decided">Decided</span> <strong>Images only</strong>, and only in the <strong>library folders you opt in</strong>. The setting is per folder, kept in a file in the library root, so it travels with the library.</li>
     <li><span class="tag decided">Decided</span> <strong>No .mcap version bump.</strong> Results go in two new chunk kinds: a small <code>segs</code> summary at the front for search, and a <code>mask</code> at the back for drawing.</li>
     <li>It is built in six phases. The first proves the ONNX export matches PyTorch before anything else depends on it.</li>
-    <li><span class="tag decided">Progress</span> <strong>Phases 1 to 4 are done</strong> (2026-10-10). The model is exported and checked, and <code>modules/Segment</code> runs it in mcap at about 57 ms per image on the RTX 5080. Results are stored in .mcap files at about 5 KB per image. Library folders opt in through <code>.library-folders</code>, and the Segmentation window and <code>-segment</code> scan them. Next is phase 5: <code>has:</code> search and Stats.</li>
+    <li><span class="tag decided">Progress</span> <strong>Phases 1 to 5 are done</strong> (2026-10-10). The model is exported and checked, and <code>modules/Segment</code> runs it in mcap at about 57 ms per image on the RTX 5080. Results are stored in .mcap files at about 5 KB per image. Library folders opt in through <code>.library-folders</code>, and the Segmentation window and <code>-segment</code> scan them. <code>has:</code> search and Stats' "What is in the images" are in (phase 5). Next is phase 6: the overlay in the Capture window.</li>
     <li><span class="tag later">Later</span> Video, by treating sampled frames as images, is shelved.</li>
   </ul>
 </div>
@@ -399,8 +399,27 @@ then runs:         (id, length) pairs over the whole map, row by row, wrapping a
 </div>
 
 <div class="phase">
-<h3>Phase 5: Catalog, <code>has:</code> and Stats</h3>
+<h3>Phase 5: Catalog, <code>has:</code> and Stats <span class="tag decided">Done</span></h3>
 <p><strong>Done when:</strong> <code>has:person</code> finds the right captures in the scratch library, catalog tests cover the new field, and Stats shows the top classes, each clickable.</p>
+<div class="note"><strong>Result (2026-10-10).</strong> The test library is phase 4's: 18 segmented images.
+<ul>
+  <li><strong><code>has:</code> searches</strong> (headless <code>-stats</code> with each search):
+    <ul>
+      <li><code>has:person</code>: 9 images.</li>
+      <li><code>has:cat</code>: 3, the COCO cats photo and the 2 GIFs.</li>
+      <li><code>has:table</code>: 1, the only plain "table", not the "dining table" images.</li>
+      <li>A plain <code>person</code>: none, since plain words don't search segments.</li>
+    </ul></li>
+  <li><strong>Stats</strong> lists person 9, wall 7, sky 4, tree 4, cat 3 and so on, each class counted once per image.</li>
+  <li><strong>Tests:</strong> 10 new catalog checks cover <code>has:</code>: whole names, quoted names with spaces, case, combinations, and a plain word not matching.</li>
+</ul>
+<p>As built:</p>
+<ul>
+  <li><strong>The catalog</strong> keeps each segment's class, score and area, and a lookup string of the classes between newlines. <code>search_parse</code> wraps a <code>has:</code> term in newlines once, so matching a whole name is one <code>contains</code> per capture.</li>
+  <li><strong>The Stats section</strong> "What is in the images" shows even when nothing has been OCR'd.</li>
+</ul>
+<p><strong>Not yet checked by eye:</strong> the Stats section's layout.</p>
+<p><strong>Left for later:</strong> counts and sizes (<code>has:person:3+</code>, <code>has:dog&gt;20%</code>).</p></div>
 </div>
 
 <div class="phase">

src/app/catalog.jai

@@ -23,9 +23,18 @@ Catalog_Entry :: struct {
   // The segmentation model's (SEGMENT_MODEL, app/segment_ui.jai) segs chunk, when it has one.
   segment_classes: string;            // its class list's hash; "" when not segmented
   segment_count: s32;
+  segments: [..] Catalog_Segment;     // largest first
 
   // Lowercased copies, for matching without case.
   find_title, find_url, find_app, find_notes, find_ocr, find_labels, find_folder: string;
+  find_segments: string;              // each class in it once, between newlines: "|person|dog|" with | a newline, for has:
+}
+
+// A segment of the capture's image (its segs chunk): what, how sure, and how much of the image.
+Catalog_Segment :: struct {
+  class: string;      // "person", "wall"
+  score: float;
+  area: float;        // a fraction of the image
 }
 
 Catalog_Media :: enum u8 {
@@ -50,6 +59,9 @@ catalog_entry_free :: (entry: *Catalog_Entry) {
   free(entry.label_list);
   free(entry.video_codec);
   free(entry.segment_classes);
+  for entry.segments  free(it.class);
+  array_free(entry.segments);
+  free(entry.find_segments);
   array_free(entry.embedding);
   free(entry.find_title); free(entry.find_url); free(entry.find_app); free(entry.find_notes); free(entry.find_ocr); free(entry.find_labels); free(entry.find_folder);
   free(entry);
@@ -132,6 +144,15 @@ catalog_load :: (path: string) -> *Catalog_Entry, error: string {
     if segments_ok {
       entry.segment_classes = copy_string(segments.classes);
       entry.segment_count = cast(s32) segments.segments.count;
+      entry.segments.allocator = context.default_allocator;
+      // Each class once, lowercased, between newlines, for has: (search_matches).
+      find := "\n";
+      for segments.segments {
+        array_add(*entry.segments, .{class = copy_string(it.class), score = it.score, area = it.area});
+        lower := to_lower_copy(it.class,, temp);
+        if !contains(find, tprint("\n%\n", lower))  find = tprint("%%\n", find, lower);
+      }
+      entry.find_segments = copy_string(find);
     }
   }
 
@@ -172,14 +193,15 @@ catalog_section :: (entry: *Catalog_Entry, model: string) -> *Mcap_Ocr_Section {
 
 // LIKE is by meaning: the captures whose image is nearest the phrase, ranked by the label model
 // (app/background.jai does the ranking); here it only asks for a labelled capture.
-Search_Field :: enum u8 { ANY; OCR; TITLE; URL; APP; NOTE; LABEL; LIKE; FOLDER; }
+// HAS is a segment's class, the whole name: has:dog, has:"traffic light" (app/segment_ui.jai).
+Search_Field :: enum u8 { ANY; OCR; TITLE; URL; APP; NOTE; LABEL; LIKE; FOLDER; HAS; }
 
 Search_Term :: struct {
   field: Search_Field;
   text: string;                // lowercased
 }
 
-// Words, "quoted phrases", and either with a field prefix: ocr: title: url: app: note: label: like: folder:. Temp.
+// Words, "quoted phrases", and either with a field prefix: ocr: title: url: app: note: label: like: folder: has:. Temp.
 search_parse :: (query: string) -> [..] Search_Term {
   terms: [..] Search_Term;
   terms.allocator = temp;
@@ -200,6 +222,7 @@ search_parse :: (query: string) -> [..] Search_Term {
         case "label";  field = .LABEL;
         case "like";   field = .LIKE;
         case "folder"; field = .FOLDER;
+        case "has";    field = .HAS;
       }
       if field != .ANY { term.field = field; rest = advance(rest, colon + 1); }   // "http:..." and the like stay words
     }
@@ -213,6 +236,8 @@ search_parse :: (query: string) -> [..] Search_Term {
       else          { term.text = slice(rest, 0, space); rest = advance(rest, space); }
     }
     term.text = to_lower_copy(term.text,, temp);
+    // has: matches a whole class name, as find_segments holds them: between newlines.
+    if term.field == .HAS && term.text.count  term.text = tprint("\n%\n", trim(term.text));
     if term.text.count  array_add(*terms, term);
     rest = trim_left(rest);
   }
@@ -236,6 +261,7 @@ search_matches :: (entry: *Catalog_Entry, terms: [] Search_Term) -> bool {
       case .LABEL;  found = contains(entry.find_labels, it.text);
       case .LIKE;   found = entry.embedding.count > 0;
       case .FOLDER; found = contains(entry.find_folder, it.text);
+      case .HAS;    found = contains(entry.find_segments, it.text);   // it.text is the class between newlines: a whole name
     }
     if !found  return false;
   }

src/app/stats_ui.jai

@@ -6,6 +6,7 @@
 STATS_WINDOW :: "Statistics";
 STATS_TOP_CAPTURES :: 10;
 STATS_TOP_WORDS    :: 40;
+STATS_TOP_CLASSES  :: 40;   // segment classes listed
 STATS_LOW_CONFIDENCE :: 0.8;
 
 // Words per capture, in these ranges: 0, 1-10, 11-50, 51-200, 201-1000, more.
@@ -30,6 +31,8 @@ Stats_Result :: struct {
   common_words: [..] Stats_Word;       // commonest first
   labelled: s64;                       // captures with labels (app/labels_ui.jai)
   top_labels: [..] Stats_Word;         // how many captures have each label as their best, most first
+  segmented: s64;                      // captures with segments (app/segment_ui.jai)
+  top_classes: [..] Stats_Word;        // how many captures hold each segment class, most first
 }
 
 stats_ui: struct {
@@ -64,6 +67,8 @@ stats_result_free :: (s: *Stats_Result) {
   for s.common_words  free(it.word);
   for s.top_labels  free(it.word);
   array_free(s.top_labels);
+  for s.top_classes  free(it.word);
+  array_free(s.top_classes);
   array_free(s.years);
   array_free(s.most_words);
   array_free(s.common_words);
@@ -131,6 +136,10 @@ stats_log :: () {
   labels.allocator = temp;
   for s.top_labels  print_to_builder(*labels, "% %, ", it.word, it.count);
   log("[stats] % labelled; best labels: %", s.labelled, builder_to_string(*labels,, temp));
+  classes: String_Builder;
+  classes.allocator = temp;
+  for s.top_classes  print_to_builder(*classes, "% %, ", it.word, it.count);
+  log("[stats] % segmented; images holding each class: %", s.segmented, builder_to_string(*classes,, temp));
   if viewer.search_terms.count {
     shown: String_Builder;
     shown.allocator = temp;
@@ -155,6 +164,27 @@ stats_window_contents :: () { using ImGui;
   if s.read < s.captures {
     TextColored(app_theme.palette.dim, "% have not been read with this model; the OCR window's scan reads them.", stats_number(s.captures - s.read));
   }
+
+  // What segmentation found: shown whether or not OCR has read anything.
+  if s.segmented {
+    SeparatorText("What is in the images");
+    TextColored(app_theme.palette.dim, "How many of the % segmented images hold each thing (%). Click one to search for it.", stats_number(s.segmented), SEGMENT_MODEL);
+    if BeginTable("##classes", 3, .SizingFixedFit) {
+      TableSetupColumn("class");
+      TableSetupColumn("images");
+      TableSetupColumn("bar", .WidthStretch);
+      for s.top_classes {
+        if it_index >= STATS_TOP_CLASSES  break;
+        TableNextRow();
+        TableNextColumn();
+        if SmallButton(temp_c_string(tprint("%##class%", it.word, it_index)))  viewer_set_search(tprint("has:\"%\"", it.word));
+        TableNextColumn(); TextUnformatted(stats_number(it.count));
+        TableNextColumn(); ProgressBar(cast(float) it.count / max(s.segmented, 1), .{-FLOAT32_MIN, 0}, temp_c_string(stats_percent(it.count, s.segmented)));
+      }
+      EndTable();
+    }
+  }
+
   if !s.read { TextColored(app_theme.palette.dim, "No readings yet."); return; }
 
   SeparatorText("Text found");
@@ -266,8 +296,11 @@ stats_compute :: (items: [] Search_Item, model: string) -> *Stats_Result {
   s.most_words.allocator = context.default_allocator;
   s.common_words.allocator = context.default_allocator;
   s.top_labels.allocator = context.default_allocator;
+  s.top_classes.allocator = context.default_allocator;
   top: Table(string, s64);   // keys borrow the entries' label names
   top.allocator = temp;
+  classes: Table(string, s64);   // keys borrow the entries' segment classes
+  classes.allocator = temp;
 
   per_capture: [..] Stats_Capture;
   per_capture.allocator = temp;
@@ -289,6 +322,17 @@ stats_compute :: (items: [] Search_Item, model: string) -> *Stats_Result {
       count := find_or_add(*top, entry.labels[0].name);
       count.* += 1;
     }
+    if entry && entry.segment_classes {
+      s.segmented += 1;
+      // Each class once per capture, however many of it there are.
+      for segment, n: entry.segments {
+        seen := false;
+        for 0..n - 1  if entry.segments[it].class == segment.class { seen = true; break; }
+        if seen  continue;
+        count := find_or_add(*classes, segment.class);
+        count.* += 1;
+      }
+    }
 
     section := catalog_section(entry, model);
     if !section  continue;
@@ -337,6 +381,8 @@ stats_compute :: (items: [] Search_Item, model: string) -> *Stats_Result {
 
   for count, name: top  array_add(*s.top_labels, .{copy_string(name,, context.default_allocator), count});
   quick_sort(s.top_labels, (a: Stats_Word, b: Stats_Word) -> s64 { return ifx a.count != b.count then b.count - a.count else compare(a.word, b.word); });
+  for count, name: classes  array_add(*s.top_classes, .{copy_string(name,, context.default_allocator), count});
+  quick_sort(s.top_classes, (a: Stats_Word, b: Stats_Word) -> s64 { return ifx a.count != b.count then b.count - a.count else compare(a.word, b.word); });
 
   s.compute_ms = to_float64_seconds(current_time_monotonic() - start) * 1000;
   return s;

src/app/viewer.jai

@@ -826,7 +826,7 @@ viewer_ui :: () { using ImGui;
   }
 
   SetNextItemWidth(min(GetContentRegionAvail().x * 0.5, 520));
-  if InputTextWithHint("##search", "Search text, titles, URLs, apps, notes, labels.  \"a phrase\"  ocr: title: url: app: note: label: like: folder:", viewer.search_buffer.data, viewer.search_buffer.count) {
+  if InputTextWithHint("##search", "Search text, titles, URLs, apps, notes, labels.  \"a phrase\"  ocr: title: url: app: note: label: like: folder: has:", viewer.search_buffer.data, viewer.search_buffer.count) {
     viewer_search_changed();
   }
   SameLine();

src/test/catalog_tests.jai

@@ -95,6 +95,21 @@ match_tests :: () {
   expect(search_matches(*entry, search_parse("like:anything")), "like: takes a labelled capture (the ranking is the background's)");
   entry.embedding = .[];
 
+  // has: a segment's class, the whole name (find_segments, as catalog_load makes it).
+  entry.find_segments = "\nperson\ndog\ndining table\n";
+  expect(search_matches(*entry, search_parse("has:dog")), "has: a class");
+  expect(search_matches(*entry, search_parse("has:DOG has:person")), "has: twice, ignoring case");
+  expect(search_matches(*entry, search_parse("has:\"dining table\"")), "has: a quoted name with a space");
+  expect(!search_matches(*entry, search_parse("has:table")), "has: is the whole name: table is not dining table");
+  expect(!search_matches(*entry, search_parse("has:do")), "nor part of a name");
+  expect(!search_matches(*entry, search_parse("has:cat")), "a class it does not hold");
+  expect(!search_matches(*entry, search_parse("dog")), "a plain word does not match segments");
+  expect(search_matches(*entry, search_parse("has:dog kicad")), "has: with other words");
+  terms := search_parse("has:dog");
+  expect(terms.count == 1 && terms[0].field == .HAS, "the has: prefix is read");
+  entry.find_segments = "";
+  expect(!search_matches(*entry, search_parse("has:dog")), "nothing segmented: no has: matches");
+
   expect(search_line_matches("0.1uF : Capacitor_SMD", search_parse("capacitor")), "a line holding a word");
   expect(!search_line_matches("0.1uF : Capacitor_SMD", search_parse("title:capacitor")), "a title word is not an OCR line's");
 }