Both CSVs called sku_l11 キューピー ハーフマヨネーズ 450g. The shelf tag under
the l11 row in add/image6.jpg reads キユーピー ハーフマヨネーズ〈400g〉, and the
enrolled packshot's own OCR reads 400g, so the catalog shipped a size the
product does not have - the same defect class as sku_l10.
Measured on the corrected image6 labels, local lane: full 86% precision (was
84%), live 95% (was 91%), false accepts 8 and 2. 303 tests pass.
Ajinomoto ピュアセレクト マヨネーズ has no 450g size: the official lineup is
250g/400g/600g/1kg, the shelf tag in add/image6.jpg reads 400g, and DECISION.md
already recorded the mismatch. Both CSVs carried 450g, so every l10 answer
shipped a wrong size.
The packshot is the exact 400g front (JAN 4901001114030, 1000x1000) with its
source URL and the 450g rejection recorded in
data/packshots/research/sku_l10/manifest.json.
Enrolling it moves the catalog ceiling from 69% to 87% of in-catalog facings
answerable by image (l10 was 11 unenrolled facings). Measured on the corrected
image6 labels: local lane 92% identity at 84% precision full, 85% at 91% live.
Also points external-eval defaults at the repository-local data path; the
licensed Wikimedia corpus remains uncommitted pending manifest review.
Stage 2 of the plan is "identity as retrieval", so the first thing measured was
whether retrieval can even work here. It cannot, on its own: an off-the-shelf
DINOv2 embedding does group identical products (two orange juices, two apple
juices, two almond drinks) but the fine-grained cases collapse - 32 cartons into
one cluster, and nine different bottled drinks grouped by *container shape*. That
matches the literature (RP2K, RPC all fine-tune; nobody ships off-the-shelf
features for SKU identity).
Meanwhile the shelf has been carrying the answer. Every product has a tag printed
under it, and the tag spells the product's name out in clean text, once per
product - the store's own label. This adds that as an identity channel:
- a tag with a readable name is matched against the catalog by name similarity;
- facings are assigned to the *nearest tag along their row* (the tags mark where
products sit), not by geometric grouping, which was measured to merge different
products into a 945 px "product" that then swallowed the wrong tag;
- rows are clustered first, because the same column on six shelves would
otherwise become one product.
Then the demo catalog itself had to be fixed: it listed another store's products
(senbei, Tropicana) while the shelf holds mayonnaise, so nothing could match. The
catalog for this shelf was read off the tags - 12 products with names and prices -
and the products were enrolled as packaging templates by cropping the facings the
tags had just named.
Measured through the real still path on add/image6.jpg (111 detections):
tag named the product 28 facings (all 28 verified correct by eye)
packaging named the product 74 facings
both spoke 23 facings, agreeing 16 (69 %)
facings with any identity 79
So packaging recognition makes a ~30 % error rate on the facings the tag can
check, and every disagreement carried a low packaging score. The tag therefore
wins a conflict, and the conflict is recorded (`identity_conflict`) instead of
hidden - a product whose tag says one thing and whose packaging says another is a
real finding, not noise to suppress.
144 tests pass; test_tag_identity.py adds 6.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
