feat: record a live session so it can become data
The deep review's first recommendation was real data, because every accuracy number so far comes from one or two photographs - the identity benchmark even takes its gallery and its queries from the same photo. This is the mechanism for replacing that. `--record-frames DIR` writes each processed frame, sampled every Nth, with a JSON line describing what the server saw in it: the boxes, the identity channels and the tag reading. The lines are the labelling input - `labelling_candidates()` returns the facings the tag channel named with a corroborated price, which are seeds for the confirmation screen rather than guesses. Nothing here is a training set on its own. Sampling keeps each frame's own number in the index, so a sampled recording says so instead of hiding it, and `/v1/health` reports how much has been written. Measured on a 163-frame walk: 32 frames and their lines. `docs/DATA.md` is the runbook: what to capture (a slow walk per rail, a close pass for the barcode, and the same shelf on a second visit), how to turn it into labels, and how to freeze a benchmark with a cross-visit split. 167 tests pass. Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
parent
dc83e75a34
commit
a718acdc15
4 changed files with 329 additions and 0 deletions
92
recognization/docs/DATA.md
Normal file
92
recognization/docs/DATA.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Capturing real data
|
||||
|
||||
Everything measured in this project so far comes from one or two photographs. That
|
||||
is its largest weakness: the identity benchmark takes its gallery and its queries
|
||||
from the same photo, so every accuracy number is an in-distribution number.
|
||||
|
||||
This is the procedure for replacing that with data from a real store, recorded on
|
||||
the device that will do the scanning.
|
||||
|
||||
## 1. Record a walk
|
||||
|
||||
Start the server with recording on. Frames are written every Nth processed frame,
|
||||
and each one is written with a line describing what the server saw in it.
|
||||
|
||||
```bash
|
||||
cd recognization
|
||||
YOLO_OFFLINE=1 .venv/bin/python code/app.py \
|
||||
--port 8090 --host 0.0.0.0 \
|
||||
--classify-workers 8 \
|
||||
--record-frames ~/dorayaki-data/2026-09-25-storeA \
|
||||
--record-every 3
|
||||
```
|
||||
|
||||
Then run the app and scan. Each scan session becomes its own directory:
|
||||
|
||||
```
|
||||
~/dorayaki-data/2026-09-25-storeA/<session>/
|
||||
000000.jpg the frame, exactly as the client sent it
|
||||
000001.jpg
|
||||
index.jsonl one line per recorded frame:
|
||||
{"frame": 123, "path": "000001.jpg", "at": 1699...,
|
||||
"summary": {...},
|
||||
"tracks": [{"box": [...], "identity": {...},
|
||||
"tag_identity": {...}, "tag": {...}}]}
|
||||
```
|
||||
|
||||
Frames are sampled, not continuous, and the index keeps each frame's own number so
|
||||
the sampling is visible rather than silent. `/v1/health` reports how many frames
|
||||
have been written.
|
||||
|
||||
## 2. What to capture, and why
|
||||
|
||||
Two visits to the same shelf are worth more than one long visit, because the
|
||||
benchmark needs a **cross-visit split**: the gallery comes from visit 1 and the
|
||||
test set from visit 2. That is the difference between "it works on this photo" and
|
||||
"it works on this shelf".
|
||||
|
||||
Per shelf, capture:
|
||||
|
||||
- a **slow walk along each rail** at working distance (about 1 m) — the live loop's
|
||||
real input;
|
||||
- a **close pass** (about 30 cm) over a few tags — the only distance where the
|
||||
barcode is decodable (measured: 0.63 px per module at 1 m needs ~3× more pixels);
|
||||
- the same shelf again **on a second visit**, ideally with something changed: a gap,
|
||||
a moved facing, a new product.
|
||||
|
||||
## 3. Turn a recording into labels
|
||||
|
||||
The tag channel seeds the labels. A facing whose tag named a product, with a price
|
||||
the tag's own arithmetic corroborates, is a candidate label rather than a guess:
|
||||
|
||||
```bash
|
||||
python -c "
|
||||
import json, sys
|
||||
sys.path.insert(0, 'code')
|
||||
from record import labelling_candidates
|
||||
seeds = labelling_candidates('~/dorayaki-data/2026-09-25-storeA/<session>/index.jsonl')
|
||||
print(len(seeds), 'candidate labels')
|
||||
print(json.dumps(seeds[:3], ensure_ascii=False, indent=1))
|
||||
"
|
||||
```
|
||||
|
||||
A human then confirms or corrects them in the app's confirmation screen — the same
|
||||
screen that closes the flywheel. Nothing here is a training set on its own; it is
|
||||
the raw material for one.
|
||||
|
||||
## 4. Freeze a benchmark
|
||||
|
||||
Write the confirmed labels into a manifest that the evaluation reads, and keep it in
|
||||
the repository so results are reproducible:
|
||||
|
||||
```
|
||||
benchmarks/<store>-<date>/
|
||||
visit1/<sku_id>/<view>.jpg the gallery: 2-3 views per SKU
|
||||
visit2/*.jpg the test frames
|
||||
labels.json {"visit2/0007.jpg": [{"box": [...], "sku_id": "..."}]}
|
||||
```
|
||||
|
||||
Then the identity benchmark (see `code/bench_identity.py`) can be run with a
|
||||
cross-visit split, and every number it reports is a number worth acting on. Until
|
||||
that exists, treat the current figures as what they are: measurements on a
|
||||
photograph.
|
||||
Loading…
Add table
Add a link
Reference in a new issue