383ff9cf2d
Two real store photographs, chosen for density and for products that look different
from each other, each with its own enrolled store:
demo-images/demo-image-1-mediya-dairy.jpg CC0 MEDIYA dairy case
21 products kept (13 price cards dropped), 16 recognised, all 14 products named
demo-images/demo-image-2-milk-case.jpg CC BY 2.0 supermarket milk case
56 detections, 49 recognised, all 13 products named
Both were enrolled the way a store would be: the products were read off a contact
sheet of the detections, their crops became the store's gallery (stock-mediya,
stock-milk150), and a head was trained on them. `build_store_from_shelf.py` is that
flow.
The filter came out of the first image. Every Japanese supermarket photo has the same
artifact: the detector boxes the shelf-edge price cards as products, because a card is
roughly the shape of a pack - 13 of 34 detections on image 1 were cards. Their text
separates them cleanly: a card's characters are 30-44% digits and it prints the
tax-excluded price (本体), while a product's are 3-19% digits. Cards are now dropped
before classification, and the count is reported as `price_cards_dropped` rather than
hidden.
demo-images/README.md records how to run both, what the demo shows, and the honest
caveat: both shelves are enrolled, so the recognition rate is in-distribution - what
the app does on a shelf it has been onboarded to, not on one it has never seen.
175 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
3.3 KiB
The two demo shelves
Two real store photographs, chosen because they are dense and their products are
visually distinct, with a store catalog enrolled for each. Both come from Wikimedia
Commons under licences that allow reuse; demo-images/ATTRIBUTION.json records the
author, licence and source of each.
| image 1 | image 2 | |
|---|---|---|
| file | demo-images/demo-image-1-mediya-dairy.jpg |
demo-images/demo-image-2-milk-case.jpg |
| scene | a MEDIYA dairy case | a supermarket milk case |
| licence | CC0 | CC BY 2.0 |
| detections | 21 products (13 price cards filtered out) | 56 products |
| recognised | 16, covering all 14 products | 49, covering all 13 products |
| store | stock-mediya/ |
stock-milk150/ |
How to run them
cd recognization
YOLO_OFFLINE=1 .venv/bin/python code/app.py \
--port 8090 --identity gallery --live-conf 0.25 \
--stock-dir stock-mediya # or stock-milk150
Then POST the image, or open the web client and use Photo mode:
curl -s -X POST --data-binary @demo-images/demo-image-1-mediya-dairy.jpg \
-H 'content-type: image/jpeg' \
'http://127.0.0.1:8090/v1/recognize?classify=true&conf=0.25&tags=true' | python3 -m json.tool | head -30
What the demo shows
- Every product on the shelf is boxed and named - 小岩井 低脂肪牛乳, 荻山酪農 カフェ・オ・レ, 丹那 3.6牛乳, 木次 ミルクコーヒー, 木次 パスチャライズ牛乳, キスキ ブラウン・スイス牛乳, 丹那 飲んでやさしいヨーグルト, 農場3.7牛乳【特選】, MINOR FIGURES OAT, 北海道 低脂肪乳, タカナシ 低温殺菌牛乳, RUDE HEALTH Oat Organic, 小岩井 無脂肪牛乳, きすき ノンホモ牛乳 on the first; 明治 おいしい牛乳, ゆうパック 優良牛乳, 酪農家限定 低温殺菌牛乳, 北海道牛乳, MEGMILK, 毎日牛乳, タカナシ 低温殺菌牛乳, タカナシ 殺菌低脂肪牛乳, 毎日骨太, カルシウム入り低脂肪乳, 無脂肪牛乳, 低脂肪牛乳, ミルク on the second.
- Price cards are not boxed as products. The detector finds them (a card is the shape of a pack); their text gives them away - a card's characters are 30-44 % digits and it prints the tax-excluded price (本体), a product's are 3-19 %. Measured on image 1: 13 cards dropped, 21 products kept.
- What it cannot name, it does not invent. Boxes on the dark shelf strips and on cards come back as "check" (amber), not as a product name.
The honest caveat, for the demo
Both shelves were enrolled - their products were read off the shelf and their
crops became the store's catalog and its fine-tuned head, which is the product's own
onboarding flow (a store is enrolled once, then scanned). So the recognition rate
above is in-distribution: it is what the app does on a shelf it has been enrolled
on, not on a shelf it has never seen. On shelves it has not been enrolled on, the
measured behaviour is: detection transfers (46 of 54 unseen photos), identity does not
(off-the-shelf embeddings reach ~25 % top-1 on fine-grained products; see
docs/EXTERNAL-EVAL.md).
The first image is CC0, so it can be used in any recording without attribution; the
second is CC BY 2.0 and needs the credit in ATTRIBUTION.json if it appears on screen.