Product recognition demo for supermarket shelf images
4fd5059ef0
Part 2 ended on the number that matters: enrolling a real shelf from itself leaves off-the-shelf DINOv2 at ~25% top-1, because real milk cartons differ by small text and a colour band. This trains a small projection head on frozen features - 384 -> 128, cosine classifier, angular margin - with augmentation standing in for the second visit we do not have, and measures it on 72 held-out facings over 8 products. raw DINOv2 cosine top-1 2/72 (2%) balanced 1% raw, rejecting below 0.55 top-1 2/72 (2%) balanced 1% (5% of answered right) fine-tuned + margin top-1 28/72 (38%) balanced 73% fine-tuned, rejecting below 0.4 top-1 17/72 (23%) balanced 58% (94% of answered right) So fine-tuning is the difference between working and not (balanced 1% -> 73%), and with a rejection threshold the model becomes precise - it answers 18 of 72 facings and is right on 94% of them, which is the operating point a store wants. The raw model cannot do that at any threshold. Where it fails is where my labels are coarse: MEGMILK, which the contact sheet shows is probably several products under one brand, scores 1/32 because the class is internally inconsistent and the model correctly refuses most of it. Honest limit, stated in the doc: enrolment and held-out facings come from one photograph, so these are upper bounds; only a recorded second visit settles the real number. What is settled is the mechanism - a head trained on labels the pipeline can produce itself moves fine-grained identity from useless to usable. 175 tests pass. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|---|---|---|
| recognization | ||
| .gitignore | ||