From 05cf32e1d90e88dfaedaba092cdcdfff60ec1014 Mon Sep 17 00:00:00 2001 From: kiyo9w Date: Mon, 28 Sep 2026 14:17:26 +0900 Subject: [PATCH] docs: describe cross-visit gallery replay --- recognization/docs/DATA.md | 66 ++++++++++++++++++++++++-------------- 1 file changed, 42 insertions(+), 24 deletions(-) diff --git a/recognization/docs/DATA.md b/recognization/docs/DATA.md index 81e1f60..667eff2 100644 --- a/recognization/docs/DATA.md +++ b/recognization/docs/DATA.md @@ -9,22 +9,31 @@ the device that will do the scanning. ## 1. Record a walk -Start the server with recording on. Frames are written every Nth processed frame, -and each one is written with a line describing what the server saw in it. +From `recognization`, run `./run-demo-server.sh stock-lopia` once to provision +the ignored `config/local.env` token and the native apps' local LAN settings, +then stop that server. Rebuild the app after provisioning. The token is a secret: +keep `.demo-token`, `config/local.env`, and app-local settings out of commits and +shared logs. For a recorded walk, start the same gallery backend on the +launcher's port; `code/app.py` reads the token from `config/local.env`: ```bash -cd recognization +cd product-recognition-demo/recognization # from the app repository root YOLO_OFFLINE=1 .venv/bin/python code/app.py \ - --port 8090 --host 0.0.0.0 \ - --classify-workers 8 \ - --record-frames ~/dorayaki-data/2026-09-25-storeA \ + --port 8099 --host 0.0.0.0 --stock-dir stock-lopia --identity gallery \ + --live-conf 0.15 --classify-workers 8 \ + --record-frames ~/dorayaki-data/storeA-visit1 \ --record-every 3 ``` +This requires the local pinned DINOv2 file described in `../README.md` +(`weights/dinov2_vits14_lvd142m.safetensors`, or +`DORAYAKI_DINOV2_WEIGHTS`). A LAN bind without a token is refused. Record the +second visit to a separate directory with the same catalog and model. + Then run the app and scan. Each scan session becomes its own directory: ``` -~/dorayaki-data/2026-09-25-storeA// +~/dorayaki-data/storeA-visit1// 000000.jpg the frame, exactly as the client sent it 000001.jpg index.jsonl one line per recorded frame: @@ -60,33 +69,42 @@ The tag channel seeds the labels. A facing whose tag named a product, with a pri the tag's own arithmetic corroborates, is a candidate label rather than a guess: ```bash -python -c " +python -c ' import json, sys -sys.path.insert(0, 'code') +sys.path.insert(0, "code") from record import labelling_candidates -seeds = labelling_candidates('~/dorayaki-data/2026-09-25-storeA//index.jsonl') -print(len(seeds), 'candidate labels') +seeds = labelling_candidates(sys.argv[1]) +print(len(seeds), "candidate labels") print(json.dumps(seeds[:3], ensure_ascii=False, indent=1)) -" +' "$HOME/dorayaki-data/storeA-visit1/REPLACE_WITH_SESSION/index.jsonl" ``` A human then confirms or corrects them in the app's confirmation screen — the same screen that closes the flywheel. Nothing here is a training set on its own; it is the raw material for one. -## 4. Freeze a benchmark +## 4. Replay a cross-visit benchmark -Write the confirmed labels into a manifest that the evaluation reads, and keep it in -the repository so results are reproducible: +Enroll the gallery from visit 1 only, then freeze that catalog. Confirm labels +for visit 2 independently; `labels.json` beside its `index.jsonl` (or supplied +with `--labels`) is a JSON list, or `{"labels": [...]}`. Each entry has `frame` +(the number in `index.jsonl`), `box` (pixel coordinates `[x1, y1, x2, y2]`), +and `sku_id` (a catalog SKU). These are confirmed annotations, not the server's +predictions. For the second visit, run from `recognization` with the same frozen +`stock-lopia` catalog: -``` -benchmarks/-/ - visit1//.jpg the gallery: 2-3 views per SKU - visit2/*.jpg the test frames - labels.json {"visit2/0007.jpg": [{"box": [...], "sku_id": "..."}]} +```bash +SESSION="$HOME/dorayaki-data/storeA-visit2/REPLACE_WITH_SESSION" +YOLO_OFFLINE=1 .venv/bin/python code/eval_live.py "$SESSION" \ + --stock-dir stock-lopia --labels "$SESSION/labels.json" \ + --replay --identity gallery --device cpu ``` -Then the identity benchmark (see `code/bench_identity.py`) can be run with a -cross-visit split, and every number it reports is a number worth acting on. Until -that exists, treat the current figures as what they are: measurements on a -photograph. +Replace `REPLACE_WITH_SESSION` with the actual session name. The evaluator replays +recorded JPEGs through `LiveSession`, writes `eval_live.json` in that directory, +and reports backend, loaded head, weight/catalog fingerprints, decision counts, +and held-out labelled top-1 when valid labels exist. It rejects missing or +unpinned DINOv2 weights rather than substituting matcher scores; labels copied +from catalog templates are excluded as leakage. Without confirmed held-out +visit-2 labels, counts are diagnostic only, **not gallery walk accuracy**. No +held-out cross-visit result is available here yet.