# Dorayaki: shelf-intelligence decision

Measured 2026-09-29 · server `product-recognition-demo/recognization` at
`fbf31d7` plus the coverage-first diff described in §F · evidence in
[`evidence/`](evidence/) · every number below is read from a JSON in that
directory, named inline.

Scope: the live decision path (group inference, OCR scheduling, decision
levels), the catalog the server matches against, and the two native overlays.
Not in scope: detector quality, tag/planogram pipeline, on-device runtime.

The previous phase ([`../2026-09-28-recognition-architecture/DECISION.md`](../2026-09-28-recognition-architecture/DECISION.md))
settled *what* matches a product: a local-feature instance matcher plus a text
lane behind a calibrated open-set gate. This phase settles *how much of a shelf
to label, how fast, and how to spend compute doing it*.

## Verdict

**Label every detected product with its best candidate immediately, mark it
provisional, and refine it as stronger evidence arrives.** Keep the accepted
line where it is; stop treating abstention as quality. Group adjacent facings
so one strong observation can answer for a run, and spend OCR on group
representatives and ambiguous tracks instead of every crop.

The measured case for it, on `add/image6.jpg` at live resolution
(`ocr-ablation.json`, `anytime-policies.json`):

| Policy | Shown | Correct | Precision | Coverage of 64 in-catalog facings | Blank |
|---|---|---|---|---|---|
| strict / individual | 39 | 38 | 97.4 % | 59.4 % | 64.2 % |
| **coverage-first (shipped)** | **63** | **55** | **87.3 %** | **86.0 %** | **42.2 %** |

The shipped policy shows 24 more products and gets 17 more of them right, at
the cost of 7 more wrong labels. That is the trade the product asked for: a
shelf where 86 % of products carry a name reads as working; one where 59 % do
reads as broken.

## A. Where the live time goes

`replay.py` on the running server, `image6` at 8 fps (`replay-image6-8fps-*.json`):

| Stage | p50 | p95 |
|---|---|---|
| decode | 8–12 ms | 20 ms |
| detect | 67–107 ms | 131–278 ms |
| track | 1–3 ms | 6 ms |
| total | 90–140 ms | 200–300 ms |

Detection dominates frame cost by an order of magnitude; tracking is free.
Identity work is off the frame path (a worker pool), so it shows up as
*label latency*, not frame latency. `scheduler-benchmark.json` prices it:
218 classification tasks, worker p50 147 ms with OCR on every attempt versus
19.6 ms with OCR once per track. **OCR is ~87 % of per-task identity cost.**

Ranked bottlenecks: (1) OCR invocations, (2) matcher invocations per facing,
(3) detector. Only the first two are ours to schedule.

## B. Compute and evidence schedule

`scheduler-benchmark.json`, same 218 tasks:

| Schedule | OCR calls | Worker p50 | First facing | All first facings | Wall | CPU |
|---|---|---|---|---|---|---|
| every attempt | 218 | 147 ms | 1177 ms | 1890 ms | 4.32 s | 4.48 s |
| once per track | 109 | 20 ms | 1235 ms | 2680 ms | 2.77 s | 3.42 s |
| ambiguous track only | 83 | 16 ms | 806 ms | 1951 ms | 2.05 s | 2.97 s |
| **group representative** | **49** | **16 ms** | **454 ms** | **1168 ms** | **1.30 s** | **2.64 s** |

Group representatives cut OCR calls 4.4× and wall time 3.3×, and they make the
first label appear 2.6× sooner. The accuracy cost is small and measured
(`ocr-ablation.json`, group metrics): full-resolution coverage 57.8 % → 56.0 %,
precision 87.3 % → 80.3 %; at live resolution coverage *rises* 56.0 % → 58.7 %
and precision 90.2 % → 87.5 %. Ship the group-representative schedule; the
`ambiguous_track` schedule is the fallback when grouping is unavailable.

The shipped live path already does the cheap half of this: OCR runs once per
track (`live.py` caches `text_read` on the track), and the local lane re-runs
on every attempt.

## C. Repeated-facing group detection

`analysis.json` (`resolutions.full` and `.live`) scores eleven pairwise signals
on 83 adjacent pairs from `image6` (72 same-SKU, 3 sibling, 8 unrelated), with
leave-one-row-out thresholds:

| Signal | AUC same-vs-sibling | AUC same-vs-unrelated | Precision | Recall | Boundary acc. |
|---|---|---|---|---|---|
| hue histogram | 0.977 | 0.967 | 98.3 % | 80.6 % | 81.9 % |
| logo pixels | 0.968 | 0.981 | 100 % | 93.1 % | 94.0 % |
| pixels | 0.944 | 0.979 | 100 % | 88.9 % | 90.4 % |
| inliers | 0.944 | 0.944 | 100 % | 88.9 % | 90.4 % |
| height | 0.972 | 0.775 | 92.6 % | 87.5 % | 83.1 % |
| aspect | 0.403 | 0.691 | 93.9 % | 43.1 % | 48.2 % |
| ocr_jaccard | 0.667 | 0.806 | 97.7 % | 58.3 % | 62.7 % |
| **combined** | **0.991** | **0.990** | **98.5 %** | **91.7 %** | **91.6 %** |

Use the **combined** signal: it is the only one that is simultaneously strong on
siblings (the hard case) and on unrelated pairs. Hue alone is nearly as good and
is the cheapest single signal; logo pixels is the best single signal but needs a
logo detector. Aspect ratio and OCR similarity are actively misleading on
siblings (AUC 0.40 and 0.67) — do not gate a group on either.

At live resolution the combined signal holds: AUC 0.991 / 0.984, precision
96.9 %, recall 87.5 %, boundary accuracy 86.7 %.

## D. Group-to-individual inference

`representative.json` prices propagating a group identity to its members, on
`image6` (109 facings, 64 in-catalog):

| Arm | Matcher calls | Total ms | Shown | Correct | Precision | Coverage | Rescued | Damaged |
|---|---|---|---|---|---|---|---|---|
| sharpest-one | 47 | 1114 | 50 | 46 | 92.0 % | 45.9 % | 1 | 8 |
| sharpest-two | 70 | 1413 | 52 | 48 | 92.3 % | 47.7 % | 1 | 6 |
| **all (no grouping)** | **109** | **2024** | **63** | **55** | **87.3 %** | **57.8 %** | **2** | **0** |

At full resolution, recognising only the sharpest member of each group **damages
8 correct labels** to rescue 1. At live resolution the same arm rescues 3 and
damages 8. Group propagation is therefore **not** a coverage win on this shelf:
the members are individually recognisable, and the group's answer is only as
good as its weakest member when the group is wrong.

Decision: **use groups to schedule compute (C, B), not to overwrite individual
labels.** Propagate a group identity only to members whose own evidence is
empty, and never over a member that produced a confident individual answer.
The `rescued_correct` column is the ceiling for that rule; it is small here and
should be re-measured on a shelf with more blur and occlusion.

## E. Sibling boundaries

The combined signal's boundary accuracy is 91.6 % at full resolution, 86.7 % at
live (`analysis.json`). The failure mode is a boundary missed, not a boundary
invented: precision 98.5 % vs recall 91.7 %. A missed boundary merges two
sibling runs into one group.

Because §D says groups do not overwrite individual labels, a missed boundary is
now a *scheduling* error (one representative OCR'd for two SKUs), not a
labelling error. That is the safe direction to fail. Two rules keep it safe:

- A member whose own individual answer disagrees with the group's is a boundary
  candidate, not an outlier to be smoothed away.
- A group never spans a detected price-tag change or a gap wider than the
  shelf's own facing pitch.

## F. Provisional and confirmed transitions

Shipped in `live.py` `_on_classified`, `identity.py`, `identity_lanes.py`:

- `accepted` — the decision cleared its gate. `state: recognized`,
  `identity_level: accepted`, `confirmed = True`, never re-classified.
- `provisional` — the decision produced a candidate but did not clear the gate.
  `state: recognized`, `identity_level: provisional`, `confirmed = False`, and
  the track **stays eligible for re-classification** so a later, sharper attempt
  can upgrade it to accepted or correct it.
- `unrecognized` — no candidate at all, or the attempt budget is spent with no
  provisional label on screen. `state: unrecognized`, no chip.

A provisional label is never blanked once shown: when the budget runs out the
track keeps its guess and simply stops being re-classified. An inherited
provisional label stays re-classifiable; an inherited accepted label does not.

Measured on the wire (`replay-image6-8fps-0.5x-0.json`): 65 of 79 tracks
labelled, 742 accepted + 309 provisional observations, first label at 1.27 s.
`identity_level` is the only new wire field; an older client that ignores it
renders a provisional label as a normal one, which is the intended degradation.

Both overlays render provisional distinctly: amber chip and dashed box
(`LiveScanOverlayPolicy.isProvisional(identity)`, `ScanScreen.kt`,
`LiveScanOverlay.swift`).

## G. Operating recommendation

`far-unknown-policies.json` prices the accept floor on 4580 crops from 64
unrelated Wikimedia shelf photos:

| Policy | local_accept | Accepts | Rate |
|---|---|---|---|
| strict | 4.0 | 3 | 0.066 % |
| balanced | 2.0 | 8 | 0.175 % |
| coverage-first | 0.5 | 83 | 1.81 % |
| near-argmax | 0.0 + fallback | 119 | 2.60 % |

The shipped `LOCAL_ACCEPT` stays at **4.0** for the *accepted* level: it is the
line that keeps far-unknown false accepts at 0.066 %, and the accepted level is
what the product treats as settled. Coverage comes from the **provisional**
level, which is a separate, much looser rule and is not gated by this number.

This is the key structural choice: **do not lower the accept line to raise
coverage.** Lowering it to 0.5 would raise far-unknown false accepts 27× and
would still leave the shelf's own hard cases (siblings) unresolved. Showing the
best candidate as provisional gets the coverage without moving the line that
protects precision.

`anytime-policies.json` confirms the shape: the strict policy reaches 59.4 %
coverage and stays there at every time budget; coverage-first reaches 86 % and
is already at 28 % of facings by 250 ms.

## H. Measured improvement

| Metric | Before | After |
|---|---|---|
| Catalog the server matches | 7 SKUs / 13 views | **60 SKUs / 86 views** |
| Tracks labelled, `image6` 8 fps 0.5× | 57 / 80 | **65 / 79** |
| In-catalog facings shown | 59.4 % | **86.0 %** |
| Precision of shown labels | 97.4 % | 87.3 % |
| First label | 623 ms | 1.27 s |
| OCR calls per 218 tasks | 218 | 49 (group-representative schedule) |
| Wall time per 218 tasks | 4.32 s | 1.30 s |

The catalog change is `code/build_stock_all.py`, which merges `stock`,
`stock-lopia`, `stock-mediya` and `stock-milk150` into `stock-all` (111 images,
60 SKUs; `stock-demo` is byte-identical to mediya+milk150 and is not a source).
SKU ids are unique across the folders, so the merge is a copy plus a CSV concat.

## Limitations

- Every shelf number is one photo, `add/image6.jpg`, and its labels are the
  corrected ones from the previous phase. The group signals and the
  representative arms are measured on that photo's own facings.
- The far-unknown rates come from a cache that records no catalog, so the
  catalog match is read off the worktree, not attested by the cache.
- `scheduler-benchmark.json` and `anytime-policies.json` are diagnostic
  counterfactuals: they price schedules against frozen evidence, not against a
  live deployment. Transport and grouping time are excluded.
- The native overlays were changed but not compiled: no JDK is installed on
  this machine, so `./gradlew` cannot run. The shared signature change is
  mechanical and every caller was updated, but this is unverified by a compiler.
- Group propagation (§D) is deliberately not shipped as a labelling rule. Its
  ceiling on this shelf is 1–3 rescued labels against 8 damaged; re-measure on a
  shelf with real blur and occlusion before revisiting.