tngo/product-recognition-demo
1
0
Fork 0
forked from vnguyen/product-recognition-demo
Code Pull requests Activity

fix(catalog): re-source sku_l09 and enforce the pyramid floor

Browse source 
sku_l09.jpg was 407x305, so its 400 px pyramid level was a 305->400 upscale: a
blur with no high-frequency detail to disagree with, which won 31 of 54 crops
on the shelf it was cut from. Re-sourced at 1000x1000.

The floor is now a hard gate in rebuild_catalog.py - a view under 800 px tall
is refused, not reported - because reporting it is what let the 305 px view
ship. data/packshots/source/sku_l07.jpg (600x600) is refused on this rule.

README records the real sizes and the provenance for every row.
This commit is contained in:
kiyo9w 2026-09-29 09:28:16 +09:00
parent 5094b5f6ca
commit 730a5e782b
4 changed files with 274 additions and 102 deletions
Download patch file Download diff file Expand all files Collapse all files

165
recognization/stock-lopia-clean/README.md
View file

@ -3,73 +3,150 @@
Official manufacturer pack shots, one front view per SKU. This replaces the
views in `../stock-lopia/`, which were 71-119 px crops cut from the test shelf
`add/image6.jpg` itself: one 18 px sliver, duplicate crops, a wrong product on
`sku_l09`, and five SKUs with no view at all. Enrolling from the test shelf is
`sku_l09`, and four SKUs with no view at all. Enrolling from the test shelf is
why every dark package collapsed onto one SKU and why the identity head was
trained on its own exam.
Rebuild or extend with:
Eight of the twelve rows are enrolled; `sku_l04`, `sku_l07`, `sku_l08` and
`sku_l12` stay text-only (see *Known gaps*). Every enrolled view is at least
1000 px tall.
python code/rebuild_catalog.py --source stock-lopia-clean --output stock-lopia-clean \
--replace --min-views 1 --csv stock-lopia/stock.csv
Rebuild from the staged packshots with:
python code/rebuild_catalog.py --source data/packshots/source \
--output stock-lopia-clean --csv stock-lopia/stock.csv \
--replace --min-views 1
`--min-views 1` because official pages publish a single front view per JAN. The
default floor of 2 exists to catch accidental single-view enrollment from shelf
crops, where several views are available.
To re-run the audit without touching the catalog, point `--output` at a
scratch path inside the repo and keep the JSON:
python code/rebuild_catalog.py --source data/packshots/source \
--output data/packshots/audit-check --csv stock-lopia/stock.csv \
--min-views 1 --json data/packshots/audit-check.json
## Provenance
| SKU | Product | Source | Image |
|---|---|---|---|
| sku_l01 | キユーピー エッグケア 305g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577090301/) | `4901577090301L.jpg` |
| sku_l02 | キユーピー 血圧ケア アマニ油マヨネーズ 200g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577060052/) | `4901577060052L.jpg` |
| sku_l03 | キユーピー からしマヨネーズ 200g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577338779/) | `4901577338779L.jpg` |
| sku_l05 | キユーピー ゼロ ノンコレステロール 310g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577092381/) | `4901577092381L.jpg` |
| sku_l06 | キユーピー 内臓脂肪ケア フィッテ 210g | [kewpie.co.jp](https://www.kewpie.co.jp/products/close/) (discontinued 2026秋) | `4901577083600L.jpg` |
| sku_l09 | ピュアセレクト コクうま カロリーカット65% 360g | [ajinomoto.co.jp](https://www.ajinomoto.co.jp/products/detail/?ProductName=pureselect_2) | `A140600321` |
| sku_l11 | キユーピー ハーフ 400g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577031083/) | `4901577031083L.jpg` |
Product names are the catalog's, verbatim from `stock.csv`; sizes are measured.
Kewpie images follow `https://img.kewpie.co.jp/products_src/g3/imgs/<JAN13>L.jpg`;
Ajinomoto images follow
`https://www.ajinomoto.co.jp/products/Image.aspx?PicType=0&PIMNO=<PIMNO>`.
| SKU | Catalog name | View | Recorded source |
|---|---|---|---|
| sku_l01 | キューピー エッグケア 305g | 455x1000 | not recorded |
| sku_l02 | キューピー アマニ油マヨネーズ 200g | 468x1000 | not recorded |
| sku_l03 | キューピー からしマヨネーズ 200g | 450x1000 | not recorded |
| sku_l05 | キューピー ゼロ ノンコレステロール 310g | 461x1000 | not recorded |
| sku_l06 | キューピー フィッテ 210g | 654x1500 | [amazon.co.jp](https://www.amazon.co.jp/dp/B08V2676LK) |
| sku_l09 | ピュアセレクト コクうま カロリーカット65% 360g | 1000x1000 | not recorded |
| sku_l10 | ピュアセレクト マヨネーズ 400g | 1000x1000 | [rakuten.co.jp](https://item.rakuten.co.jp/matsukiyo/4901001114030/) (JAN 4901001114030) |
| sku_l11 | キューピー ハーフマヨネーズ 400g | 482x1000 | not recorded |
`data/packshots/source/ATTRIBUTION.json` records a URL for `sku_l06` and
`sku_l10` only. Its `_provenance_gap` note names the other six - l01, l02, l03,
l05, l09, l11 - as enrolled before any provenance was tracked, and the audit
reports them under `unattributed_skus` on every run. What *is* verified about
them, per that note: each is independent of the evaluation photo (template
match against `add/image6.jpg` peaks at 0.166, against 0.95 for a crop of it),
each is 1000 px or taller, and each one's own OCR matches its catalog row (see
*Name/OCR gate*). That is a traceability gap, not a correctness one.
`sku_l06` is the current package (`内臓脂肪を減らす`), which replaced the
superseded `内臓脂肪ケア` view on 2026-09-29. The catalog name was shortened to
match; the shelf tags were not part of that change.
## Name/OCR gate
A packshot's OCR has to explain its own catalog row's name, or it is refused.
Measured over the shipped packshots - every image read in all four rotations,
every read scored against all 12 catalog names:
| Class | n | Score |
|---|---|---|
| Correct packshot vs its own row | 8 | 0.9167 - 1.0000 |
| Wrong product vs any row | 1 image x 12 names | 0.0476 - 0.5833 |
`MIN_NAME_OCR = 0.75` is the midpoint of those two bands: 0.167 of headroom
above the worst correct packshot (l11, whose read omits `マヨネーズ`) and 0.167
below the wrong-product ceiling. The wrong-product image is
`references/rejected/kewpie-mayonnaise-450g.jpg`; its best score anywhere is
0.5833, tied against `キューピー からしマヨネーズ 200g` and
`キューピー ハーフマヨネーズ 400g`, and it bottoms out at 0.0476 against
`ピュアセレクト コクうま カロリーカット65% 360g`.
Reproduce the band with (run from the recognition root; unindented so it runs
verbatim):
```python
import csv, os, sys, cv2
sys.path.insert(0, 'code')
from ocr import build_ocr_engine
from rebuild_catalog import best_orientation_text, name_ocr_similarity
clean = 'stock-lopia-clean'
names = {r['SKU_ID']: r['product_name'] for r in csv.DictReader(open(clean + '/stock.csv'))}
engine = build_ocr_engine(None)
for n in sorted(os.listdir(clean)):
if n.endswith('.jpg'):
sku = n[:-4]
read = best_orientation_text(engine, cv2.imread(os.path.join(clean, n)), names[sku])
print(sku, round(name_ocr_similarity(names[sku], read), 4))
rej = 'references/rejected/kewpie-mayonnaise-450g.jpg'
read = best_orientation_text(engine, cv2.imread(os.path.join(clean, rej)), names['sku_l01'])
print('rejected max', max((round(name_ocr_similarity(v, read), 4), k) for k, v in names.items()))
```
The gate cannot separate siblings, and does not try: the `sku_l09` packshot
scores 0.8462 against the `sku_l10` name, because both are Ajinomoto
ピュアセレクト mayonnaise. That case is what `text_discriminated_siblings` flags
(l09 ~ l10, cosine 0.9817) - it is a cross-SKU geometry problem, not a gap a
threshold closes.
## Pyramid floor
Every enrolled view must be at least 800 px tall, or it is refused. This is a
hard gate, not a report. It exists because of `sku_l09`: the view enrolled at
305 px could not be stored as a 200/400/800 px pyramid, so its local lane
upscaled 305->400 px and scored 85-129 inliers against 10-40 for every other
reference, winning facings it should not. It was re-sourced at 1000x1000 on
2026-09-29. Reporting the defect instead of refusing it is what let it ship.
`data/packshots/source/sku_l07.jpg` is 600x600 and is refused on this rule
before OCR even runs.
## Known gaps
Five SKUs in `../stock-lopia/stock.csv` have no official pack shot here, so no
image lane can match them. They stay in `stock.csv` as text-only rows: the text
lane builds a signature from the printed name, which is how it can name or veto
a near-unknown the matcher cannot see. They are the current image-coverage
ceiling, not a recognition failure:
Four rows in `../stock-lopia/stock.csv` have no enrolled image, so no image
lane can match them. They stay in `stock.csv` as text-only rows: the text lane
builds a signature from the printed name, which is how it can name or veto a
near-unknown the matcher cannot see. They are the current image-coverage
ceiling, not a recognition failure.
| SKU | Catalog name | Why it is missing |
| SKU | Catalog name | Why there is no view |
|---|---|---|
| sku_l04 | やまや めんたいマヨネーズタイプ 200g | Yamaya's current lineup has no 200 g mentaiko mayo |
| sku_l07 | 丸和油脂 ラッキーマヨネーズ 200g | Maruwa's current catalog has no 200 g Lucky Mayonnaise |
| sku_l04 | やまや めんたいマヨネーズタイプ 200g | A packshot is staged and refused at name/OCR 0.00: the image reads only `明太手屋さんが つくつ た コ`, brand text that runs vertically. A correctly oriented or higher-resolution shot is what would clear it. |
| sku_l07 | 丸和油脂 ラッキーマヨネーズ 200g | A packshot is staged and refused twice: it is 600x600, under the pyramid floor, and scores name/OCR 0.5714. Every product bigram reads (`ラッキー マヨネーズ`); the six it misses are the maker prefix `丸和`/`和油`/`油脂` and the size `20`/`0G`/`00`, which the package does not print. |
| sku_l08 | スパイシーストーム ペリペリミックス 30g | No official 30 g Peri-Peri Mix found |
| sku_l10 | ピュアセレクト マヨネーズ 450g | Ajinomoto's current lineup is 400/250/600 g/1 kg; no 450 g |
| sku_l12 | 焼肉のたれ 中辛 | Brand not recorded in the catalog; ambiguous |
| sku_l12 | ユーラス 野菜をおいしく食べる旨たれ 380g | No pack shot sourced |
The catalog names for these five disagree with the manufacturers' current
lineups, so the shelf tags and the CSV need reconciling before enrollment.
`sku_l10` left this table on 2026-09-29. Its catalog name said 450 g, which
Ajinomoto does not make; the shelf tag in `add/image6.jpg` and the
manufacturer's lineup both say 400 g, so `stock.csv` was corrected and a
1000x1000 packshot (JAN 4901001114030) enrolled. See
`data/packshots/source/ATTRIBUTION.json`.
Keep the text-only rows when rebuilding: the CSV carry-over must preserve every
source row, not just the SKUs with a kept view. Dropping these five costs the
source row, not just the SKUs with a kept view. Dropping these four costs the
text lane the signatures that cover them.
`sku_l09.jpg` is enrolled but only 305 px tall, below the 800 px source floor
the enrollment protocol wants (§8 of the architecture decision). Its matcher
scores dwarf the other references (85-129 inliers against 10-40), so it wins
facings it should not; re-source it before trusting l09 precision. Measured on
the corrected image6 labels, l09 is the largest single-SKU share of accepts at
both resolutions: 18 of 48 accepts (37.5%) at full, 11 of 38 (28.9%) at live,
against a true share of 13 of 61 facings (21.3%). That is a hubness ratio of
1.76 and 1.36 against the completion gate's limit of 2.0, so G7 passes; the
stricter internal 30% share heuristic is what raises the alarm at full
resolution. Re-sourcing the reference is the fix, not a threshold change.
## Rejected
`references/rejected/kewpie-mayonnaise-450g.jpg` is Kewpie Mayonnaise 450g (JAN
4901577042072). It was briefly filed as `sku_l09` and then as `sku_l10` because
the catalog's names pointed there, but it is neither product: l09 is Ajinomoto
Pure Select コクうま カロリーカット 65% 360g and l10 is Pure Select マヨネーズ
450g. It is kept as provenance for that mix-up and is never enrolled.
4901577042072). It was filed as `sku_l09` and then as `sku_l10` because the
catalog's names pointed there, but it is neither: l09 is Ajinomoto Pure Select
コクうま カロリーカット 65% 360g and l10 is Pure Select マヨネーズ 400g. Both now
hold their own correct views. The rejected image is kept as provenance for that
mix-up and is never enrolled; the name/OCR gate scores it at most 0.5833
against any catalog name, below the 0.75 threshold.