383ff9cf2d
Two real store photographs, chosen for density and for products that look different
from each other, each with its own enrolled store:
demo-images/demo-image-1-mediya-dairy.jpg CC0 MEDIYA dairy case
21 products kept (13 price cards dropped), 16 recognised, all 14 products named
demo-images/demo-image-2-milk-case.jpg CC BY 2.0 supermarket milk case
56 detections, 49 recognised, all 13 products named
Both were enrolled the way a store would be: the products were read off a contact
sheet of the detections, their crops became the store's gallery (stock-mediya,
stock-milk150), and a head was trained on them. `build_store_from_shelf.py` is that
flow.
The filter came out of the first image. Every Japanese supermarket photo has the same
artifact: the detector boxes the shelf-edge price cards as products, because a card is
roughly the shape of a pack - 13 of 34 detections on image 1 were cards. Their text
separates them cleanly: a card's characters are 30-44% digits and it prints the
tax-excluded price (本体), while a product's are 3-19% digits. Cards are now dropped
before classification, and the count is reported as `price_cards_dropped` rather than
hidden.
demo-images/README.md records how to run both, what the demo shows, and the honest
caveat: both shelves are enrolled, so the recognition rate is in-distribution - what
the app does on a shelf it has been onboarded to, not on one it has never seen.
175 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
63 lines
3.3 KiB
Markdown
63 lines
3.3 KiB
Markdown
# The two demo shelves
|
|
|
|
Two real store photographs, chosen because they are dense and their products are
|
|
visually distinct, with a store catalog enrolled for each. Both come from Wikimedia
|
|
Commons under licences that allow reuse; `demo-images/ATTRIBUTION.json` records the
|
|
author, licence and source of each.
|
|
|
|
| | image 1 | image 2 |
|
|
| --- | --- | --- |
|
|
| file | `demo-images/demo-image-1-mediya-dairy.jpg` | `demo-images/demo-image-2-milk-case.jpg` |
|
|
| scene | a MEDIYA dairy case | a supermarket milk case |
|
|
| licence | **CC0** | **CC BY 2.0** |
|
|
| detections | 21 products (13 price cards filtered out) | 56 products |
|
|
| recognised | **16, covering all 14 products** | **49, covering all 13 products** |
|
|
| store | `stock-mediya/` | `stock-milk150/` |
|
|
|
|
## How to run them
|
|
|
|
```bash
|
|
cd recognization
|
|
YOLO_OFFLINE=1 .venv/bin/python code/app.py \
|
|
--port 8090 --identity gallery --live-conf 0.25 \
|
|
--stock-dir stock-mediya # or stock-milk150
|
|
```
|
|
|
|
Then POST the image, or open the web client and use **Photo** mode:
|
|
|
|
```bash
|
|
curl -s -X POST --data-binary @demo-images/demo-image-1-mediya-dairy.jpg \
|
|
-H 'content-type: image/jpeg' \
|
|
'http://127.0.0.1:8090/v1/recognize?classify=true&conf=0.25&tags=true' | python3 -m json.tool | head -30
|
|
```
|
|
|
|
## What the demo shows
|
|
|
|
* **Every product on the shelf is boxed and named** - 小岩井 低脂肪牛乳,
|
|
荻山酪農 カフェ・オ・レ, 丹那 3.6牛乳, 木次 ミルクコーヒー, 木次 パスチャライズ牛乳,
|
|
キスキ ブラウン・スイス牛乳, 丹那 飲んでやさしいヨーグルト, 農場3.7牛乳【特選】,
|
|
MINOR FIGURES OAT, 北海道 低脂肪乳, タカナシ 低温殺菌牛乳, RUDE HEALTH Oat Organic,
|
|
小岩井 無脂肪牛乳, きすき ノンホモ牛乳 on the first; 明治 おいしい牛乳, ゆうパック
|
|
優良牛乳, 酪農家限定 低温殺菌牛乳, 北海道牛乳, MEGMILK, 毎日牛乳, タカナシ 低温殺菌牛乳,
|
|
タカナシ 殺菌低脂肪牛乳, 毎日骨太, カルシウム入り低脂肪乳, 無脂肪牛乳, 低脂肪牛乳, ミルク
|
|
on the second.
|
|
* **Price cards are not boxed as products.** The detector finds them (a card is the
|
|
shape of a pack); their text gives them away - a card's characters are 30-44 %
|
|
digits and it prints the tax-excluded price (本体), a product's are 3-19 %. Measured
|
|
on image 1: 13 cards dropped, 21 products kept.
|
|
* **What it cannot name, it does not invent.** Boxes on the dark shelf strips and on
|
|
cards come back as "check" (amber), not as a product name.
|
|
|
|
## The honest caveat, for the demo
|
|
|
|
Both shelves were **enrolled** - their products were read off the shelf and their
|
|
crops became the store's catalog and its fine-tuned head, which is the product's own
|
|
onboarding flow (a store is enrolled once, then scanned). So the recognition rate
|
|
above is **in-distribution**: it is what the app does on a shelf it has been enrolled
|
|
on, not on a shelf it has never seen. On shelves it has not been enrolled on, the
|
|
measured behaviour is: detection transfers (46 of 54 unseen photos), identity does not
|
|
(off-the-shelf embeddings reach ~25 % top-1 on fine-grained products; see
|
|
`docs/EXTERNAL-EVAL.md`).
|
|
|
|
The first image is CC0, so it can be used in any recording without attribution; the
|
|
second is CC BY 2.0 and needs the credit in `ATTRIBUTION.json` if it appears on screen.
|