Product recognition demo for supermarket shelf images
f74366d6a4
Everything else here is measured on two photographs. This collects real store shelves from Wikimedia Commons (scriptable API, explicit licences: CC0, CC BY, CC BY-SA, public domain, each recorded in a manifest) and evaluates the pipeline on them. 46 of 54 images carry a shelf; the rest are book scans and a shop front. What it found, and it is not what I expected: - **Detection transfers.** 2,258 detections across dairy cases, soy-sauce shelves, spice racks, a confectionery aisle, an Okinawan supermarket and a Hong Kong grocery - tight boxes on images the model has never seen, and correctly nothing on the book scans. - **Identity transfers when the product is genuinely there.** On a 2011 milk-case photo it labelled six facings of 明治 おいしい牛乳 correctly, and 野菜生活100 and R-1 elsewhere. The matcher does reach new photographs of products it has templates for. - **Precision is the failure, and rejection is missing.** In that same photograph it labelled 低脂肪殺菌牛乳, 有機豆乳, 特濃4.5, 無調整豆乳 and 北海道牛乳 as おいしい牛乳 - they are all white milk cartons. Across 24 shelves whose products are not in the catalog it accepted 451 identities, and the score cannot be thresholded to fix it: the real shelf's median is 18.2 with a p10 of 5.2, while the foreign shelves' median is 6.0 and the worst offenders score 50.5 (a bento), 80.6 (a shop interior) and 189.9 (deer cookies). A threshold that catches the foreigners also throws away the real matches. That confirms on real data what the literature and the review predicted: the matcher's raw inlier score is not a confidence, and fine-grained look-alikes are the failure mode. Step 4 of the plan - shortlist, local re-rank, calibrated rejection - is the fix, not a refinement. docs/EXTERNAL-EVAL.md records the numbers, the method and the attribution; the images themselves are not redistributed. 175 tests pass. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| recognization | ||
| .gitignore | ||