feat: let the shelf's own price tag name the product

Stage 2 of the plan is "identity as retrieval", so the first thing measured was
whether retrieval can even work here. It cannot, on its own: an off-the-shelf
DINOv2 embedding does group identical products (two orange juices, two apple
juices, two almond drinks) but the fine-grained cases collapse - 32 cartons into
one cluster, and nine different bottled drinks grouped by *container shape*. That
matches the literature (RP2K, RPC all fine-tune; nobody ships off-the-shelf
features for SKU identity).

Meanwhile the shelf has been carrying the answer. Every product has a tag printed
under it, and the tag spells the product's name out in clean text, once per
product - the store's own label. This adds that as an identity channel:

- a tag with a readable name is matched against the catalog by name similarity;
- facings are assigned to the *nearest tag along their row* (the tags mark where
  products sit), not by geometric grouping, which was measured to merge different
  products into a 945 px "product" that then swallowed the wrong tag;
- rows are clustered first, because the same column on six shelves would
  otherwise become one product.

Then the demo catalog itself had to be fixed: it listed another store's products
(senbei, Tropicana) while the shelf holds mayonnaise, so nothing could match. The
catalog for this shelf was read off the tags - 12 products with names and prices -
and the products were enrolled as packaging templates by cropping the facings the
tags had just named.

Measured through the real still path on add/image6.jpg (111 detections):

  tag named the product          28 facings   (all 28 verified correct by eye)
  packaging named the product    74 facings
  both spoke                     23 facings, agreeing 16 (69 %)
  facings with any identity      79

So packaging recognition makes a ~30 % error rate on the facings the tag can
check, and every disagreement carried a low packaging score. The tag therefore
wins a conflict, and the conflict is recorded (`identity_conflict`) instead of
hidden - a product whose tag says one thing and whose packaging says another is a
real finding, not noise to suppress.

144 tests pass; test_tag_identity.py adds 6.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
kiyo9w 2026-09-24 23:28:36 +09:00
parent cf7e8a5a88
commit 52905e73f6
11 changed files with 507 additions and 1 deletions

View file

@ -0,0 +1,13 @@
SKU_ID,product_name,receipt_date,expiration_date,number_in,number_out,number_on,image_link,price
sku_l01,キューピー エッグケア 305g,2026-09-18,2026-12-20,36,12,24,'',349
sku_l02,キューピー アマニ油マヨネーズ 200g,2026-09-18,2026-12-20,24,9,15,'',329
sku_l03,キューピー からしマヨネーズ 200g,2026-09-18,2026-12-20,30,14,16,'',219
sku_l04,やまや めんたいマヨネーズタイプ 200g,2026-09-18,2026-11-30,18,6,12,'',399
sku_l05,キューピー ゼロ ノンコレステロール 310g,2026-09-20,2026-12-25,24,11,13,'',269
sku_l06,キューピー フィッテ 210g,2026-09-20,2026-12-25,18,7,11,'',379
sku_l07,丸和油脂 ラッキーマヨネーズ 200g,2026-09-20,2026-12-25,24,10,14,'',199
sku_l08,スパイシーストーム ペリペリミックス 30g,2026-09-15,2027-03-15,12,4,8,'',699
sku_l09,ピュアセレクト コクうま カロリーカット65% 360g,2026-09-22,2026-12-28,30,13,17,'',209
sku_l10,ピュアセレクト マヨネーズ 450g,2026-09-22,2026-12-28,36,15,21,'',222
sku_l11,キューピー ハーフマヨネーズ 450g,2026-09-19,2026-12-22,48,22,26,'',269
sku_l12,焼肉のたれ 中辛,2026-09-10,2027-02-10,24,8,16,'',299
1 SKU_ID product_name receipt_date expiration_date number_in number_out number_on image_link price
2 sku_l01 キューピー エッグケア 305g 2026-09-18 2026-12-20 36 12 24 '' 349
3 sku_l02 キューピー アマニ油マヨネーズ 200g 2026-09-18 2026-12-20 24 9 15 '' 329
4 sku_l03 キューピー からしマヨネーズ 200g 2026-09-18 2026-12-20 30 14 16 '' 219
5 sku_l04 やまや めんたいマヨネーズタイプ 200g 2026-09-18 2026-11-30 18 6 12 '' 399
6 sku_l05 キューピー ゼロ ノンコレステロール 310g 2026-09-20 2026-12-25 24 11 13 '' 269
7 sku_l06 キューピー フィッテ 210g 2026-09-20 2026-12-25 18 7 11 '' 379
8 sku_l07 丸和油脂 ラッキーマヨネーズ 200g 2026-09-20 2026-12-25 24 10 14 '' 199
9 sku_l08 スパイシーストーム ペリペリミックス 30g 2026-09-15 2027-03-15 12 4 8 '' 699
10 sku_l09 ピュアセレクト コクうま カロリーカット65% 360g 2026-09-22 2026-12-28 30 13 17 '' 209
11 sku_l10 ピュアセレクト マヨネーズ 450g 2026-09-22 2026-12-28 36 15 21 '' 222
12 sku_l11 キューピー ハーフマヨネーズ 450g 2026-09-19 2026-12-22 48 22 26 '' 269
13 sku_l12 焼肉のたれ 中辛 2026-09-10 2027-02-10 24 8 16 '' 299