vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions
product-recognition-demo/recognization/docs/research/2026-09-28-recognition-architecture/evidence
History
tngo aaed5d388f feat(identity): local-feature matcher plus text lane as the default backend 
Replaces the DINO gallery as the recognition path with the measured design
from docs/research/2026-09-28-recognition-architecture/DECISION.md: RootSIFT
over a per-SKU reference pyramid, similarity-constrained RANSAC and a hue
check, fused with an OCR text lane that vetoes a match whose package text
names a different catalog row and answers on its own when geometry finds
nothing.

- identity_lanes.py: LocalLane, TextLane, DecisionPolicy. The text lane
  builds each row's signature from the catalog name plus its packshot OCR, so
  a row with no enrolled image is still answerable.
- engine.py: every backend groups scores by canonical SKU, so no
  view-suffixed id reaches the wire and a name and price always resolve.
  decide_crop fuses the lanes; score_crop_lanes feeds the live path. Both
  lanes are built in warm_up (non-fatally) so the first frame is not the slow
  one, and the text lane shares the engine's OCR lock.
- live.py: decides per crop on the lane's own rank order. The lane's order
  comes from its rank key and its value is only the gate, so re-sorting by
  value named a different SKU than the lane chose; averaging in an empty
  observation also diluted a good confidence below the accept line.
- identity.py: a `local` score scale whose gate is one absolute line, because
  the value is already a margin over the runner-up.
- app.py, run-demo-server.sh: `--identity local` is the default; the demo
  server no longer starts on the leaked `stock-lopia` catalog.
- eval_live.py: the replay vocabulary accepts `local`, so a recording from
  the shipped server can be scored.
- README.md: documents the three backends and what each is for.

Measured on the corrected image6 labels at live resolution: 36/61 in-catalog
facings named at 95% precision, 0 of 48 unknowns accepted, 11 labels sourced
from the text lane alone. Through the real server: /v1/recognize 90%
precision, /v1/stream 100% precision, no view-suffixed ids on either.
2026-09-28 22:27:21 +09:00
..
an2.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
analyze.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
diag.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
dino_scale.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
distract.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
fuse_eval.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
image6-v1-labels-overlay.jpg feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
image6_v2.json feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
image6_v2_lfo.json feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
lfo_setup.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
ocr_exp.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
ocr_fuse.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
README.md feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
scale.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
sheet.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00
variants.py feat(identity): local-feature matcher plus text lane as the default backend 2026-09-28 22:27:21 +09:00

README.md

Evidence scripts (2026-09-28 audit)

Read-only against product code: every script imports from code/ and writes only into this folder. Run from recognization/ with .venv/bin/python.

image6_v2.json is the corrected image6 label set (26 of 111 v1 labels changed; every row carries v1_label, source, and a note where changed). image6-v1-labels-overlay.jpg shows the v1 labels on the shelf; green = catalog label, red = unknown. sheet.py regenerates per-SKU contact sheets.

Step Command (prefix E=docs/research/2026-09-28-recognition-architecture/evidence) Produces
Original matcher, both catalogs, full/live/live-with-unscaled-boxes .venv/bin/python $E/diag.py --stock stock-lopia-clean --labels $E/image6_v2.json [--live] [--bug] --out $E/run_stock-lopia-clean_<full|live|livebug>.pkl (repeat with --stock stock-lopia) run_*.pkl
Score them under v1 or v2 labels .venv/bin/python $E/analyze.py <labels.json> $E/run_*.pkl identity top-1, argmax, current gate
Matcher variants .venv/bin/python $E/variants.py --stock stock-lopia-clean --labels $E/image6_v2.json [--live] --scales 200,400 --root --geom affine --out $E/v_ms_root_aff_<full|live>.pkl (also --tmpl-h 250/400, no flags = raw) v_*.pkl
Variant ranking + open-set frontier .venv/bin/python $E/an2.py $E $(basename -a $E/v_*.pkl) (LABELS=image6_v2_lfo.json for the leave-facing-out runs) correct at 100/95/90 % precision
Leave-facing-out catalog .venv/bin/python $E/lfo_setup.py, then variants.py --stock $E/cat_<pack|full> --labels $E/image6_v2_lfo.json --scales 200,400 --root --geom affine [--live] --out $E/lfo_cat_<pack|full>_<full|live>.pkl lfo_*.pkl
Unrelated-shelf distractors YOLO_OFFLINE=1 .venv/bin/python $E/distract.py stock-lopia-clean $E/cat_full distract.pkl
Catalog growth, exhaustive vs descriptor index YOLO_OFFLINE=1 .venv/bin/python $E/scale.py stdout
DINO retrieval under growth YOLO_OFFLINE=1 .venv/bin/python $E/dino_scale.py stdout
OCR read-rate .venv/bin/python $E/ocr_exp.py ocr.pkl
OCR signatures, then fusion .venv/bin/python $E/ocr_fuse.py && .venv/bin/python $E/fuse_eval.py sig.pkl, stdout
Per-SKU contact sheets + the labelled shelf .venv/bin/python $E/sheet.py <outdir> gt_<label>.jpg, shelf_labels.jpg

Timings in DECISION.md come from scale.py (single process). The other scripts ran up to ten processes at once, so their ms figures are relative only.

Verified 2026-09-28

Every command above was run from recognization/ and produced its listed output. Two rows were wrong as written:

  • The table said python; the python on this machine's PATH has no cv2, so every row failed on import. It is .venv/bin/python.
  • an2.py takes the folder and then file names inside it, so v_*.pkl expanded in the working directory, where those files do not live. It is $(basename -a $E/v_*.pkl).

sheet.py was missing from the table. It takes its output directory as a positional argument, not a flag: sheet.py --help reads --help as a directory name, writes nothing, and still exits 0. analyze.py and an2.py with no arguments raise IndexError; neither validates its own usage.

v1 labels are no longer a file on disk — eval/labels/image6.json is the v2 set. To score a run under v1, rebuild it from the v1_label field:

.venv/bin/python -c "import json;d=json.load(open('$E/image6_v2.json'));\
[f.update(label=f.pop('v1_label')) for f in d['facings']];\
json.dump(d,open('/tmp/image6_v1.json','w'),ensure_ascii=False)"

That reconstruction reproduces DECISION.md's v1 numbers exactly: analyze.py gives ungated top-1 26/35 at both resolutions and the current gate 24/35 full, 23/35 live (§2 row 15, F-01).