product-recognition-demo/recognization/data/external-eval/README.md
kiyo9w 306ea9b3e8 test(external): score 8 independent-source images, 0 false accepts
Eight images for sku_l04, sku_l07 and sku_l09 from six source domains, scored
through the shipped LocalLaneAdapter and DecisionPolicy against
stock-lopia-clean: 4 correct accepts, 4 abstains, no false accepts.

Leak-tested first: the highest template match of any image against
add/image6.jpg is 0.675, so none is a crop of the evaluation photo.

The result that matters is sku_l07. It has no enrolled image, and both its
images accept through the text lane alone - one reads ラッキー フヨネーズ, the
other the rear label's 製造者:丸和油脂株式会社 with 内容量:200g, which
together name the row 丸和油脂 ラッキーマヨネーズ 200g. The text lane's
coverage claim holds on data that was not used to build it.

The four l04 abstentions are correct behaviour: l04 is also unenrolled and its
reads are weak (one reads only コと旨 タ 味, another scores 0.379 against a
0.45 accept line).

Also records the l12 reconciliation, the Kewpie 450 g research, and two
reusable pan clips with 11 rejected candidates for a future Tier-3 replay.
Nothing here is Tier 2 or Tier 3 and every manifest says so.
2026-09-29 03:13:16 +09:00

55 lines
2.6 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# External evidence (2026-09-29)
Independent-source media used to stress the identity pipeline. **None of this is
Tier 2.** DECISION.md's Tier 2 is a second labelled store visit; these are
`external_proxy` artifacts, and every manifest here says so.
| Directory | What it holds |
|---|---|
| `other-mayo/` | 8 accepted images for `sku_l04`, `sku_l07`, `sku_l09` from 6 source domains, plus the scored result |
| `l12-reconciliation/` | What `sku_l12` actually is, from the shelf tag and package pixels |
| `kewpie/` | Kewpie 450 g research (the near-unknown case) |
| `tier3-sources/` | Two reusable pan clips and 11 rejected candidates for a future Tier-3 replay |
## Leak test
Every accepted image was template-matched against `add/image6.jpg`. Highest
score across all 8 is 0.675 (`sku_l07_1`), and the 0.95+ band that identifies a
crop of the evaluation photo is empty. The images are genuinely external.
## Result: 8 images, 0 false accepts
Scored through the shipped `LocalLaneAdapter` + `DecisionPolicy` against
`stock-lopia-clean` (`other-mayo/eval_result.json`):
| Verdict | Count | Files |
|---|---|---|
| Correct accept | 4 | `sku_l07_1`, `sku_l07_2`, `sku_l09_1`, `sku_l09_2` |
| Abstain | 4 | all `sku_l04_*` |
| False accept | **0** | — |
Two things this establishes that the image6 shelf cannot:
- **`sku_l07` is answered with no enrolled image.** Both l07 images accept
through the text lane alone: `sku_l07_1` reads ラッキー フヨネーズ, and
`sku_l07_2` reads the rear label's 製造者:丸和油脂株式会社 with 内容量:200g.
The catalog row is 丸和油脂 ラッキーマヨネーズ 200g, so the manufacturer and
the size together name it. This is the text lane's coverage claim - that a
product with no reference image can still be named - holding on data that was
not used to build it.
- **The l04 abstentions are correct behaviour, not a miss.** l04 has no
enrolled image either, and its reads are weak: `sku_l04_2` reads only
コと旨 タ 味 (all rows score 0.0), and `sku_l04_4` reads the rear label with
やまや garbled to まや, scoring 0.379 - below the accept line. Abstaining on
a weak read is the designed outcome.
`sku_l07_2` accepts with a negative local score (-0.55) on text evidence alone.
That is the lane doing its job, but it is worth naming: the accept rests on two
matched tokens (manufacturer + size), not on geometry.
## What is still missing
No external source gives per-SKU identity for the whole catalog, and no
reusable clip combines the target category with shelf-distance text
resolution. The Tier-2 and Tier-3 gates in DECISION.md still need the
operator's own recorded walks.