feat: an embedding identity backend with a calibrated rejection
The external evaluation showed the matcher cannot say "unknown": on 20 shelves whose
products are absent it claimed 47.8% of facings as recognised, and its raw inlier
score cannot be thresholded (foreign shelves' median 6.0 overlaps the real shelf's
5.2 p10, and the worst offenders score 50-190). So the gallery becomes a selectable
identity backend:
--identity gallery (or DORAYAKI_IDENTITY=gallery)
Measured on the same 20 foreign shelves: false accepts fall from 47.8% to 13.8% at
the default threshold and to **5%** at 0.60, with a smooth curve to pick an operating
point from. Detection is unchanged (the gallery only names boxes).
What it does NOT fix, measured on a milk shelf where 14 of 107 facings really are
明治 おいしい牛乳 (labelled by eye): precision is 28-45%, no better than the matcher's
39%, and enrolling the shelf from itself (1-3 views per product) is flat at ~25%
top-1. Off-the-shelf DINOv2 is not discriminative enough for fine-grained SKUs - the
same conclusion the literature reaches, now measured here.
The packaging's own text is a real channel and resolution-bound: 51% recall
single-scale, 60% with the tag reader's multi-scale + red-channel trick, 85% where
the text is large (MEGMILK) and 28% where it is small vertical Japanese.
docs/EXTERNAL-EVAL.md records all of it, including the identity architecture it
implies: reject by default, name from the tag when readable, from the packaging text
when legible, from a fine-tuned embedding otherwise, and hand the rest to the
operator.
175 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
parent
f74366d6a4
commit
7d01de8062
3 changed files with 207 additions and 0 deletions
|
|
@ -276,6 +276,14 @@ def create_app(
|
|||
"warm": True,
|
||||
},
|
||||
"catalog": {"version": engine.catalog_version(), "sku_count": engine.sku_count()},
|
||||
"identity": {
|
||||
"backend": getattr(engine.settings, "identity_backend", "matcher"),
|
||||
"gallery_views": (
|
||||
engine.identity_gallery().view_count()
|
||||
if getattr(engine.settings, "identity_backend", "") == "gallery" and engine.identity_gallery()
|
||||
else None
|
||||
),
|
||||
},
|
||||
"ocr": {
|
||||
# Without this an operator could not tell a broken OCR from a shelf
|
||||
# with no readable tags.
|
||||
|
|
@ -987,6 +995,8 @@ def main() -> None:
|
|||
parser.add_argument("--classify-workers", type=int, default=None)
|
||||
parser.add_argument("--live-method", type=str, default=None, choices=list(CLASSIFY_METHODS))
|
||||
parser.add_argument("--live-conf", type=float, default=None)
|
||||
parser.add_argument("--identity", type=str, default=None, choices=["matcher", "gallery"],
|
||||
help="which mechanism names products (default matcher)")
|
||||
parser.add_argument("--live-inflight", type=int, default=None,
|
||||
help="classifications in flight per session (demo walks want a larger budget)")
|
||||
parser.add_argument("--live-attempts", type=int, default=None,
|
||||
|
|
@ -1036,6 +1046,8 @@ def main() -> None:
|
|||
server.record_frames = args.record_frames
|
||||
if args.record_every is not None:
|
||||
server.record_every = args.record_every
|
||||
if args.identity is not None:
|
||||
engine_settings.identity_backend = args.identity
|
||||
if args.live_inflight is not None:
|
||||
engine_settings.live_inflight = args.live_inflight
|
||||
if args.live_attempts is not None:
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue