56cff09292
19 scored rather than 20: sku_l06_1.jpg is now the enrolled l06 view, so it is reported as leaked instead of as a held-out correct accept. 12 correct, 7 abstain, 0 false accepts. Both leak checks are named in the record and in the external README, since the enrollment that closed F-12 is exactly the kind of change that silently converts a held-out row into training data.
60 lines
3.1 KiB
Markdown
60 lines
3.1 KiB
Markdown
# External evidence (2026-09-29)
|
||
|
||
Independent-source media used to stress the identity pipeline. **None of this is
|
||
Tier 2.** DECISION.md's Tier 2 is a second labelled store visit; these are
|
||
`external_proxy` artifacts, and every manifest here says so.
|
||
|
||
| Directory | What it holds |
|
||
|---|---|
|
||
| `other-mayo/` | 8 accepted images for `sku_l04`, `sku_l07`, `sku_l09` from 6 source domains, plus the scored result |
|
||
| `l12-reconciliation/` | What `sku_l12` actually is, from the shelf tag and package pixels |
|
||
| `kewpie/` | 12 accepted images for six SKUs from 6 source domains, plus the scored result. One is now enrolled, so it is excluded from scoring. |
|
||
| `tier3-sources/` | Two reusable pan clips and 11 rejected candidates for a future Tier-3 replay |
|
||
|
||
## Leak checks
|
||
|
||
Two, both in `code/eval_external.py`, both run before any number is reported:
|
||
|
||
- **Against the evaluation photo.** Every image is template-matched against
|
||
`add/image6.jpg`. The highest score across all 20 is 0.675 (`sku_l07_1`), and
|
||
the 0.95+ band that identifies a crop of the evaluation photo is empty.
|
||
- **Against the catalog.** An image that has since been enrolled is refused
|
||
rather than scored. `kewpie/sku_l06_1.jpg` is one: it is now the enrolled l06
|
||
view, so scoring it would be training accuracy wearing a held-out label.
|
||
|
||
## Result: 19 scored, 0 false accepts
|
||
|
||
.venv/bin/python code/eval_external.py --stock stock-lopia-clean \
|
||
data/external-eval/kewpie data/external-eval/other-mayo
|
||
|
||
| Set | Scored | Correct | Abstain | False accepts | Leaked |
|
||
|---|---|---|---|---|---|
|
||
| `kewpie/` | 11 | 8 | 3 | **0** | 1 |
|
||
| `other-mayo/` | 8 | 4 | 4 | **0** | 0 |
|
||
| **Total** | **19** | **12** | **7** | **0** | **1** |
|
||
|
||
Two things this establishes that the image6 shelf cannot:
|
||
|
||
- **`sku_l07` is answered with no enrolled image.** Both l07 images accept
|
||
through the text lane alone: `sku_l07_1` reads ラッキー フヨネーズ, and
|
||
`sku_l07_2` reads the rear label's 製造者：丸和油脂株式会社 with 内容量：200g.
|
||
The catalog row is 丸和油脂 ラッキーマヨネーズ 200g, so the manufacturer and
|
||
the size together name it. This is the text lane's coverage claim - that a
|
||
product with no reference image can still be named - holding on data that was
|
||
not used to build it.
|
||
- **The l04 abstentions are correct behaviour, not a miss.** l04 has no
|
||
enrolled image either, and its reads are weak: `sku_l04_2` reads only
|
||
コと旨 タ 味 (all rows score 0.0), and `sku_l04_4` reads the rear label with
|
||
やまや garbled to まや, scoring 0.379 - below the accept line. Abstaining on
|
||
a weak read is the designed outcome.
|
||
|
||
`sku_l07_2` accepts with a negative local score (-0.55) on text evidence alone.
|
||
That is the lane doing its job, but it is worth naming: the accept rests on two
|
||
matched tokens (manufacturer + size), not on geometry.
|
||
|
||
## What is still missing
|
||
|
||
No external source gives per-SKU identity for the whole catalog, and no
|
||
reusable clip combines the target category with shelf-distance text
|
||
resolution. The Tier-2 and Tier-3 gates in DECISION.md still need the
|
||
operator's own recorded walks.
|