vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

docs(decision): re-measure the stream and live rows after the calibration

Browse source 
The stream row was recorded before the accept lines moved. Three fresh 20 s pan
replays through the shipped lane: 70-74% coverage at 77-85% precision, 0 wrong
labels in all three (0-1 before), 8-14 unknown false accepts - the near-unknown
case, which the size gate and catalog completeness own rather than the accept
line.

Live resolution on the shipped evaluator: 56% end-to-end at 97% precision with
1 false accept, so the calibration is not a full-resolution artifact.

The recordings are committed under stream-gated/ so the row is reproducible.
This commit is contained in:
kiyo9w 2026-09-29 04:22:46 +09:00
parent 5eb4bdf6c8
commit 01d9a5e614
4 changed files with 6228 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

16
recognization/docs/research/2026-09-28-recognition-architecture/DECISION.md
View file

@ -728,10 +728,14 @@ Two live-path bugs surfaced while measuring this, both fixed:
Coverage is still a gate (G1). Four 20 s pan replays of `add/image6.jpg`
through the shipped lane name 63-70% of the in-catalog facings a window can
show, at 76-84% precision (`score_stream.py` over
`data/evaluations/2026-09-29-session/stream/fin_*.json`). Van's contract on the
same stream answers more and is wrong far more often, which is the trade the
decision makes. The earlier "53 of 131 tracks" figure was a harness artifact
rather than a recognition limit.
`data/evaluations/2026-09-29-session/stream/fin_*.json`). Re-recorded
2026-09-29 with the calibrated accept lines
(`stream-gated/tracks_*.json`): 70-74% at 77-85%, with 0 wrong labels in all
three runs against 0-1 before, and 8-14 unknown false accepts - the
near-unknown case, which is the size gate's and the catalog's job rather than
the accept line's. Van's contract on the same stream answers more and is wrong
far more often, which is the trade the decision makes. The earlier "53 of 131
tracks" figure was a harness artifact rather than a recognition limit.
The shipped default scale now has test coverage. `make_session` builds only
`matcher`-backend engines, so `scale == "local"` in `_on_classified` never ran
@ -885,7 +889,9 @@ on 2650 distractor crops and 109 image6 facings:
far-unknown) with coverage left. On the shipped evaluator that is 66 %
end-to-end at 95 % precision with 2 false accepts, against 75 % / 92 % / 4 at
the old 2.0 line - the coverage is what the precision costs, and G2 is the gate
that says to pay it.
that says to pay it. At live resolution the same lines give 56 % end-to-end at
97 % precision with 1 false accept, so the calibration holds on the path the
phone actually uses and not only at full resolution.
Reproduce: `.venv/bin/python code/eval_distractors.py --stock stock-lopia-clean
--write data/external-shelves/wikimedia-commons data/external-shelves/wikimedia-commons-2`.