# Capturing real data

Everything measured in this project so far comes from one or two photographs. That
is its largest weakness: the identity benchmark takes its gallery and its queries
from the same photo, so every accuracy number is an in-distribution number.

This is the procedure for replacing that with data from a real store, recorded on
the device that will do the scanning.

## 1. Record a walk

Start the server with recording on. Frames are written every Nth processed frame,
and each one is written with a line describing what the server saw in it.

```bash
cd recognization
YOLO_OFFLINE=1 .venv/bin/python code/app.py \
  --port 8090 --host 0.0.0.0 \
  --classify-workers 8 \
  --record-frames ~/dorayaki-data/2026-09-25-storeA \
  --record-every 3
```

Then run the app and scan. Each scan session becomes its own directory:

```
~/dorayaki-data/2026-09-25-storeA/<session>/
    000000.jpg        the frame, exactly as the client sent it
    000001.jpg
    index.jsonl       one line per recorded frame:
                      {"frame": 123, "path": "000001.jpg", "at": 1699...,
                       "summary": {...},
                       "tracks": [{"box": [...], "identity": {...},
                                   "tag_identity": {...}, "tag": {...}}]}
```

Frames are sampled, not continuous, and the index keeps each frame's own number so
the sampling is visible rather than silent. `/v1/health` reports how many frames
have been written.

## 2. What to capture, and why

Two visits to the same shelf are worth more than one long visit, because the
benchmark needs a **cross-visit split**: the gallery comes from visit 1 and the
test set from visit 2. That is the difference between "it works on this photo" and
"it works on this shelf".

Per shelf, capture:

- a **slow walk along each rail** at working distance (about 1 m) — the live loop's
  real input;
- a **close pass** (about 30 cm) over a few tags — the only distance where the
  barcode is decodable (measured: 0.63 px per module at 1 m needs ~3× more pixels);
- the same shelf again **on a second visit**, ideally with something changed: a gap,
  a moved facing, a new product.

## 3. Turn a recording into labels

The tag channel seeds the labels. A facing whose tag named a product, with a price
the tag's own arithmetic corroborates, is a candidate label rather than a guess:

```bash
python -c "
import json, sys
sys.path.insert(0, 'code')
from record import labelling_candidates
seeds = labelling_candidates('~/dorayaki-data/2026-09-25-storeA/<session>/index.jsonl')
print(len(seeds), 'candidate labels')
print(json.dumps(seeds[:3], ensure_ascii=False, indent=1))
"
```

A human then confirms or corrects them in the app's confirmation screen — the same
screen that closes the flywheel. Nothing here is a training set on its own; it is
the raw material for one.

## 4. Freeze a benchmark

Write the confirmed labels into a manifest that the evaluation reads, and keep it in
the repository so results are reproducible:

```
benchmarks/<store>-<date>/
    visit1/<sku_id>/<view>.jpg      the gallery: 2-3 views per SKU
    visit2/*.jpg                    the test frames
    labels.json                     {"visit2/0007.jpg": [{"box": [...], "sku_id": "..."}]}
```

Then the identity benchmark (see `code/bench_identity.py`) can be run with a
cross-visit split, and every number it reports is a number worth acting on. Until
that exists, treat the current figures as what they are: measurements on a
photograph.