feat: two demo shelves, enrolled, plus the price-card filter they forced out
Two real store photographs, chosen for density and for products that look different
from each other, each with its own enrolled store:
demo-images/demo-image-1-mediya-dairy.jpg CC0 MEDIYA dairy case
21 products kept (13 price cards dropped), 16 recognised, all 14 products named
demo-images/demo-image-2-milk-case.jpg CC BY 2.0 supermarket milk case
56 detections, 49 recognised, all 13 products named
Both were enrolled the way a store would be: the products were read off a contact
sheet of the detections, their crops became the store's gallery (stock-mediya,
stock-milk150), and a head was trained on them. `build_store_from_shelf.py` is that
flow.
The filter came out of the first image. Every Japanese supermarket photo has the same
artifact: the detector boxes the shelf-edge price cards as products, because a card is
roughly the shape of a pack - 13 of 34 detections on image 1 were cards. Their text
separates them cleanly: a card's characters are 30-44% digits and it prints the
tax-excluded price (本体), while a product's are 3-19% digits. Cards are now dropped
before classification, and the count is reported as `price_cards_dropped` rather than
hidden.
demo-images/README.md records how to run both, what the demo shows, and the honest
caveat: both shelves are enrolled, so the recognition rate is in-distribution - what
the app does on a shelf it has been onboarded to, not on one it has never seen.
175 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
parent
2bf916f2aa
commit
383ff9cf2d
72 changed files with 215 additions and 0 deletions
16
recognization/demo-images/ATTRIBUTION.json
Normal file
16
recognization/demo-images/ATTRIBUTION.json
Normal file
|
|
@ -0,0 +1,16 @@
|
|||
[
|
||||
{
|
||||
"file": "demo-image-1-mediya-dairy.jpg",
|
||||
"title": "File:Azabudai-Hills---2025-04-05 015.jpg",
|
||||
"license": "CC0",
|
||||
"artist": "<a href=\"//commons.wikimedia.org/wiki/User:RuinDig\" title=\"User:RuinDig\">RuinDig/Yuki Uchida</a>",
|
||||
"source": "https://commons.wikimedia.org/wiki/File%3AAzabudai-Hills---2025-04-05_015.jpg"
|
||||
},
|
||||
{
|
||||
"file": "demo-image-2-milk-case.jpg",
|
||||
"title": "File:Milk (2655574568).jpg",
|
||||
"license": "CC BY 2.0",
|
||||
"artist": "<a rel=\"nofollow\" class=\"external text\" href=\"https://www.flickr.com/people/36014533@N00\">yumtan</a>",
|
||||
"source": "https://commons.wikimedia.org/wiki/File%3AMilk_%282655574568%29.jpg"
|
||||
}
|
||||
]
|
||||
63
recognization/demo-images/README.md
Normal file
63
recognization/demo-images/README.md
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# The two demo shelves
|
||||
|
||||
Two real store photographs, chosen because they are dense and their products are
|
||||
visually distinct, with a store catalog enrolled for each. Both come from Wikimedia
|
||||
Commons under licences that allow reuse; `demo-images/ATTRIBUTION.json` records the
|
||||
author, licence and source of each.
|
||||
|
||||
| | image 1 | image 2 |
|
||||
| --- | --- | --- |
|
||||
| file | `demo-images/demo-image-1-mediya-dairy.jpg` | `demo-images/demo-image-2-milk-case.jpg` |
|
||||
| scene | a MEDIYA dairy case | a supermarket milk case |
|
||||
| licence | **CC0** | **CC BY 2.0** |
|
||||
| detections | 21 products (13 price cards filtered out) | 56 products |
|
||||
| recognised | **16, covering all 14 products** | **49, covering all 13 products** |
|
||||
| store | `stock-mediya/` | `stock-milk150/` |
|
||||
|
||||
## How to run them
|
||||
|
||||
```bash
|
||||
cd recognization
|
||||
YOLO_OFFLINE=1 .venv/bin/python code/app.py \
|
||||
--port 8090 --identity gallery --live-conf 0.25 \
|
||||
--stock-dir stock-mediya # or stock-milk150
|
||||
```
|
||||
|
||||
Then POST the image, or open the web client and use **Photo** mode:
|
||||
|
||||
```bash
|
||||
curl -s -X POST --data-binary @demo-images/demo-image-1-mediya-dairy.jpg \
|
||||
-H 'content-type: image/jpeg' \
|
||||
'http://127.0.0.1:8090/v1/recognize?classify=true&conf=0.25&tags=true' | python3 -m json.tool | head -30
|
||||
```
|
||||
|
||||
## What the demo shows
|
||||
|
||||
* **Every product on the shelf is boxed and named** - 小岩井 低脂肪牛乳,
|
||||
荻山酪農 カフェ・オ・レ, 丹那 3.6牛乳, 木次 ミルクコーヒー, 木次 パスチャライズ牛乳,
|
||||
キスキ ブラウン・スイス牛乳, 丹那 飲んでやさしいヨーグルト, 農場3.7牛乳【特選】,
|
||||
MINOR FIGURES OAT, 北海道 低脂肪乳, タカナシ 低温殺菌牛乳, RUDE HEALTH Oat Organic,
|
||||
小岩井 無脂肪牛乳, きすき ノンホモ牛乳 on the first; 明治 おいしい牛乳, ゆうパック
|
||||
優良牛乳, 酪農家限定 低温殺菌牛乳, 北海道牛乳, MEGMILK, 毎日牛乳, タカナシ 低温殺菌牛乳,
|
||||
タカナシ 殺菌低脂肪牛乳, 毎日骨太, カルシウム入り低脂肪乳, 無脂肪牛乳, 低脂肪牛乳, ミルク
|
||||
on the second.
|
||||
* **Price cards are not boxed as products.** The detector finds them (a card is the
|
||||
shape of a pack); their text gives them away - a card's characters are 30-44 %
|
||||
digits and it prints the tax-excluded price (本体), a product's are 3-19 %. Measured
|
||||
on image 1: 13 cards dropped, 21 products kept.
|
||||
* **What it cannot name, it does not invent.** Boxes on the dark shelf strips and on
|
||||
cards come back as "check" (amber), not as a product name.
|
||||
|
||||
## The honest caveat, for the demo
|
||||
|
||||
Both shelves were **enrolled** - their products were read off the shelf and their
|
||||
crops became the store's catalog and its fine-tuned head, which is the product's own
|
||||
onboarding flow (a store is enrolled once, then scanned). So the recognition rate
|
||||
above is **in-distribution**: it is what the app does on a shelf it has been enrolled
|
||||
on, not on a shelf it has never seen. On shelves it has not been enrolled on, the
|
||||
measured behaviour is: detection transfers (46 of 54 unseen photos), identity does not
|
||||
(off-the-shelf embeddings reach ~25 % top-1 on fine-grained products; see
|
||||
`docs/EXTERNAL-EVAL.md`).
|
||||
|
||||
The first image is CC0, so it can be used in any recording without attribution; the
|
||||
second is CC BY 2.0 and needs the credit in `ATTRIBUTION.json` if it appears on screen.
|
||||
BIN
recognization/demo-images/demo-image-1-mediya-dairy.jpg
Normal file
BIN
recognization/demo-images/demo-image-1-mediya-dairy.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 783 KiB |
BIN
recognization/demo-images/demo-image-2-milk-case.jpg
Normal file
BIN
recognization/demo-images/demo-image-2-milk-case.jpg
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 1.5 MiB |
Loading…
Add table
Add a link
Reference in a new issue