vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat(eval): add --no-text so the E-3 claim is reproducible

Browse source 
E-3 asks what the text lane is worth, and the record answered it with numbers
that no committed command could produce - the evaluator had no way to run the
local lane without the text lane. --no-text is that way, and it keeps the
comparison to one variable by building the lane either way, so the OCR cost is
paid in both arms.

Measured: full resolution 77% end-to-end at 91% precision with text against 61%
at 66% without (5 false accepts against 20); live 62% at 93% against 56% at 84%
(3 against 7). The lane adds 16 pp and 6 pp of coverage while raising precision,
against E-3's +5 pp bar.

Also refreshes the re-measured block above it, which still printed the
pre-correction 61/48 label counts and a stray 'HUB!' marker.
This commit is contained in:
kiyo9w 2026-09-29 03:32:38 +09:00
parent 56cff09292
commit a1c69f72fe
2 changed files with 25 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

25
recognization/docs/research/2026-09-28-recognition-architecture/DECISION.md
View file

@ -758,23 +758,24 @@ unanswerable before and after.
The text lane's evidence floor (shared bigrams between the read and the winning
row's name) and the local lane's signature pass-through landed, and the
decision ceiling was corrected: it divided a numerator that counted unenrolled
facings by the enrolled count, so it read 117%. It now reads 86%.
facings by the enrolled count, so it read 117%.
full resolution — 109 scored facings (61 in-catalog, 48 unknown)
full resolution — 109 scored facings (64 in-catalog, 45 unknown)
lane catalog identity decision end-to-end precision false acc
van 69% 76% 76% 52% 29% 77
matcher 69% 76% 71% 49% 39% 46 HUB!
local 69% 93% 86% 80% 91% 5
van 83% 77% 77% 64% 38% 68
matcher 83% 77% 74% 61% 51% 38
local 83% 92% 83% 77% 91% 5
live resolution — 109 scored facings (61 in-catalog, 48 unknown)
live resolution — 109 scored facings (64 in-catalog, 45 unknown)
lane catalog identity decision end-to-end precision false acc
local 69% 86% 86% 66% 95% 2
local 83% 91% 64% 62% 93% 3
**E-3 answered.** Removing the text lane from the decision, same crops and
catalog, drops the local lane from 80% to 49% coverage at full resolution
(91% → 79% precision, 5 → 8 false accepts) and from 66% to 43% at live
(95% → 87%, 2 → 4). The lane adds 31 pp and 23 pp of coverage while *raising*
precision, against E-3's +5 pp bar. It is not a veto-only lane.
**E-3 answered.** `--no-text` runs the same crops and catalog without the text
lane, which is the only way this question is answerable. Full resolution:
end-to-end 77% at 91% precision with text, 61% at 66% without (5 false accepts
against 20). Live: 62% at 93% against 56% at 84% (3 against 7). The lane adds
16 pp and 6 pp of coverage while *raising* precision, against E-3's +5 pp bar.
It is not a veto-only lane.
**E-2 answered.** `LocalLane` is RootSIFT-only; ORB and BRISK survive only in
the `matcher` baseline lane, so there is nothing to drop from the shipped path.