feat: an embedding identity backend with a calibrated rejection

The external evaluation showed the matcher cannot say "unknown": on 20 shelves whose
products are absent it claimed 47.8% of facings as recognised, and its raw inlier
score cannot be thresholded (foreign shelves' median 6.0 overlaps the real shelf's
5.2 p10, and the worst offenders score 50-190). So the gallery becomes a selectable
identity backend:

    --identity gallery        (or DORAYAKI_IDENTITY=gallery)

Measured on the same 20 foreign shelves: false accepts fall from 47.8% to 13.8% at
the default threshold and to **5%** at 0.60, with a smooth curve to pick an operating
point from. Detection is unchanged (the gallery only names boxes).

What it does NOT fix, measured on a milk shelf where 14 of 107 facings really are
明治 おいしい牛乳 (labelled by eye): precision is 28-45%, no better than the matcher's
39%, and enrolling the shelf from itself (1-3 views per product) is flat at ~25%
top-1. Off-the-shelf DINOv2 is not discriminative enough for fine-grained SKUs - the
same conclusion the literature reaches, now measured here.

The packaging's own text is a real channel and resolution-bound: 51% recall
single-scale, 60% with the tag reader's multi-scale + red-channel trick, 85% where
the text is large (MEGMILK) and 28% where it is small vertical Japanese.

docs/EXTERNAL-EVAL.md records all of it, including the identity architecture it
implies: reject by default, name from the tag when readable, from the packaging text
when legible, from a fine-tuned embedding otherwise, and hand the rest to the
operator.

175 tests pass.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
kiyo9w 2026-09-25 03:04:18 +09:00
parent f74366d6a4
commit 7d01de8062
3 changed files with 207 additions and 0 deletions

View file

@ -276,6 +276,14 @@ def create_app(
"warm": True,
},
"catalog": {"version": engine.catalog_version(), "sku_count": engine.sku_count()},
"identity": {
"backend": getattr(engine.settings, "identity_backend", "matcher"),
"gallery_views": (
engine.identity_gallery().view_count()
if getattr(engine.settings, "identity_backend", "") == "gallery" and engine.identity_gallery()
else None
),
},
"ocr": {
# Without this an operator could not tell a broken OCR from a shelf
# with no readable tags.
@ -987,6 +995,8 @@ def main() -> None:
parser.add_argument("--classify-workers", type=int, default=None)
parser.add_argument("--live-method", type=str, default=None, choices=list(CLASSIFY_METHODS))
parser.add_argument("--live-conf", type=float, default=None)
parser.add_argument("--identity", type=str, default=None, choices=["matcher", "gallery"],
help="which mechanism names products (default matcher)")
parser.add_argument("--live-inflight", type=int, default=None,
help="classifications in flight per session (demo walks want a larger budget)")
parser.add_argument("--live-attempts", type=int, default=None,
@ -1036,6 +1046,8 @@ def main() -> None:
server.record_frames = args.record_frames
if args.record_every is not None:
server.record_every = args.record_every
if args.identity is not None:
engine_settings.identity_backend = args.identity
if args.live_inflight is not None:
engine_settings.live_inflight = args.live_inflight
if args.live_attempts is not None: