feat: add Van's real baseline lane and stop size tokens naming products
Some checks failed
tests / model-free (pull_request) Has been cancelled

Two review findings, both confirmed by measurement.

1. The evaluator had no always-argmax lane, so every "vs Van" claim compared
   against the *gated* matcher, which is not Van's contract - the original demo
   answered every crop with its best-scoring SKU and had no accept line. The new
   'van' lane is that contract: same scores, no gate. Against it the local lane
   wins both axes at both resolutions (full 61%/77% vs 52%/29%; live 59%/95% vs
   57%/32%), but the live coverage margin is one facing, not seven, and the 95%
   figure is live-resolution only - at full resolution the same lane reads 77%
   with 11 false accepts and a hub alarm.

2. A crop reading only "400g" scored 0.667 on three SKUs, above TEXT_ACCEPT, so
   the text lane named a product from a size line. The fill-in job now requires
   two non-digit bigrams; the veto is unchanged, so "65%" still contradicts a
   local answer. The raw OCR read travels with the ranking through
   score_crop_lanes for that check.

Tests: 3 for the size gate (policy + live) and the van lane's own row. Removing
the gate fails all three. Suite 276 -> 280.
This commit is contained in:
tngo 2026-09-28 23:12:44 +09:00
parent 13ee7426e2
commit fd72251160
7 changed files with 180 additions and 22 deletions

View file

@ -589,17 +589,57 @@ own" job unreachable. Live now decides per crop on the lane's own order.
Measured after the change, through the real server: `/v1/recognize` names 41 of
112 detections at 90% precision with no view-suffixed ids; the live WebSocket
names 12 at 100% precision with 0 unknown accepts, first label on frame 3.
Offline on the corrected labels: 36/61 (59%) at 95% precision, 0/48 unknowns,
11 of those labels sourced from the text lane alone.
Those are detector-box figures, not label-matched ones: the 41 are detections
whose box overlaps a labelled facing, so they measure the endpoint rather than
the identity lane. The label-matched number is the evaluator's, below.
## The baseline is Van's contract, not the gated matcher
The `matcher` lane applies `decide_identity`, which is not what the original
demo did - it answered every crop with its best-scoring SKU and had no accept
line. Comparing against the gated matcher understates the baseline's coverage
and overstates its precision, so the evaluator now has a `van` lane: the same
scores, always answered. Every "vs Van" claim in this record uses that row.
.venv/bin/python code/eval_identity.py --stock stock-lopia-clean \
--labels docs/research/2026-09-28-recognition-architecture/evidence/image6_v2.json \
--image add/image6.jpg --lanes van,matcher,local --live
full resolution — 109 scored facings (61 in-catalog, 48 unknown)
lane catalog identity decision end-to-end precision false acc
van 69% 76% 76% 52% 29% 77
matcher 69% 76% 71% 49% 39% 46 HUB!
local 69% 95% 88% 61% 77% 11 HUB!
live resolution — 109 scored facings (61 in-catalog, 48 unknown)
lane catalog identity decision end-to-end precision false acc
van 69% 83% 83% 57% 32% 74
matcher 69% 83% 76% 52% 43% 43
local 69% 86% 86% 59% 95% 2
Against Van's real contract the local lane wins both axes at both resolutions:
61% at 77% against 52% at 29% (full), and 59% at 95% against 57% at 32% (live).
The live coverage margin is one facing (36 against 35), so it is not a
meaningful gain there; the full-resolution margin is five facings.
**The 95% figure is live-resolution only.** At full resolution the same lane
reads 77% precision with 11 false accepts and raises the hub alarm, and the
still endpoint and the native crop lane both run at that scale. Reaching 95% at
full resolution needs an accept line near 6, which leaves 28/61 correct - below
Van's 32/61. So the design does not yet beat Van on both axes at full
resolution, and the accept line is provisional: it is tuned on Tier 1 and needs
per-input-path calibration on Tier 2.
The live count is pacing-dependent, not a fixed property: a track needs two
attempts before it settles, so a harness that pushes frames faster than the
classifier drains them reports fewer labels. Re-run at four pacings on the same
12-frame clip: 12, 8, 2 and 6 named at full speed, 14 named with 0.1 s between
frames, and 38 named of 30 frames at 0.1 s. Precision stays 100% throughout and
0 unknowns are accepted at every pacing, which is the property that matters;
coverage is a function of how long the phone holds still, so it belongs to the
device measurement (G4), not to this record.
0 unknowns are accepted at every pacing. Coverage is a gate (G1), not a
footnote: 12 of 61 streamed is a regression against Van's 57% until device data
says otherwise, and the likely cause is measurable - OCR runs on every crop at
every attempt, taking a crop from 5 ms to 25 ms and throughput from 789 to 60
crops/s with 8 workers.
The shipped default scale now has test coverage. `make_session` builds only
`matcher`-backend engines, so `scale == "local"` in `_on_classified` never ran