tngo
aaed5d388f
Replaces the DINO gallery as the recognition path with the measured design from docs/research/2026-09-28-recognition-architecture/DECISION.md: RootSIFT over a per-SKU reference pyramid, similarity-constrained RANSAC and a hue check, fused with an OCR text lane that vetoes a match whose package text names a different catalog row and answers on its own when geometry finds nothing. - identity_lanes.py: LocalLane, TextLane, DecisionPolicy. The text lane builds each row's signature from the catalog name plus its packshot OCR, so a row with no enrolled image is still answerable. - engine.py: every backend groups scores by canonical SKU, so no view-suffixed id reaches the wire and a name and price always resolve. decide_crop fuses the lanes; score_crop_lanes feeds the live path. Both lanes are built in warm_up (non-fatally) so the first frame is not the slow one, and the text lane shares the engine's OCR lock. - live.py: decides per crop on the lane's own rank order. The lane's order comes from its rank key and its value is only the gate, so re-sorting by value named a different SKU than the lane chose; averaging in an empty observation also diluted a good confidence below the accept line. - identity.py: a `local` score scale whose gate is one absolute line, because the value is already a margin over the runner-up. - app.py, run-demo-server.sh: `--identity local` is the default; the demo server no longer starts on the leaked `stock-lopia` catalog. - eval_live.py: the replay vocabulary accepts `local`, so a recording from the shipped server can be scored. - README.md: documents the three backends and what each is for. Measured on the corrected image6 labels at live resolution: 36/61 in-catalog facings named at 95% precision, 0 of 48 unknowns accepted, 11 labels sourced from the text lane alone. Through the real server: /v1/recognize 90% precision, /v1/stream 100% precision, no view-suffixed ids on either.
51 lines
3.1 KiB
Markdown
51 lines
3.1 KiB
Markdown
# Shelf label files
|
|
|
|
Ground truth for the identity evaluators (`code/eval_identity.py`,
|
|
`code/eval_live.py`) and for the evidence scripts in
|
|
`docs/research/2026-09-28-recognition-architecture/evidence/`.
|
|
|
|
{"image": "add/image6.jpg", "version": 2, "facings": [ ... ]}
|
|
|
|
`image` is the photo the boxes index into. `facings` is one row per detected
|
|
product facing, in that photo's own pixel coordinates.
|
|
|
|
## Row schema
|
|
|
|
| Field | Required | Meaning |
|
|
|---|---|---|
|
|
| `index` | yes | Row identity. Stable across label revisions, so a run's per-facing log joins to truth by `index`, never by array position. |
|
|
| `box` | yes | `[x1, y1, x2, y2]` in `image` pixels. |
|
|
| `label` | yes | Canonical SKU id (`sku_l01`), `unknown` for a facing that is not in the catalog, or `exclude` for a facing no lane can be scored on. |
|
|
| `source` | yes | How the label was decided. `v1-confirmed-visual` = the v1 label was checked against the package and kept; `visual-audit-2026-09-28` = the label was re-read from the package in the 2026-09-28 audit. |
|
|
| `v1_label` | yes | The label this row carried in v1. Kept so claims computed on v1 stay reproducible and the 26 corrections stay auditable. |
|
|
| `note` | when `label != v1_label` | Why the label changed. Also on a few unchanged rows that need a remark (e.g. `#46-48`, the tare bottles that may be what `stock.csv` calls l12). |
|
|
| `match` | no | v1 provenance only: the field DECISION.md §3 reads as a match IoU, and as evidence that v1 labels were propagated from tag or box matching rather than read off packages. Values here are 0.0, 0.911, 1.0. No scorer reads it; it is carried through unchanged for traceability. |
|
|
|
|
`label` takes a canonical SKU id, `unknown`, or `exclude`. This file uses
|
|
`sku_l01`-`sku_l11`: l12 has no real facing on the shelf, so no row carries it.
|
|
`exclude` rows stay in the file so the facing inventory is complete; every
|
|
scorer drops them before computing a rate. The evaluators read `index`, `box`,
|
|
and `label`; the provenance fields are additive.
|
|
|
|
`image6.json` is the corrected v2 set: 111 rows, 109 scored (2 `exclude`), 61
|
|
in-catalog across 11 SKUs, 48 `unknown`, 26 labels changed from v1. It is a copy
|
|
of `docs/research/2026-09-28-recognition-architecture/evidence/image6_v2.json`;
|
|
edit the evidence copy and re-copy, so the two cannot drift.
|
|
|
|
## Schema check
|
|
|
|
DECISION.md §7 sets the rule for every label file:
|
|
|
|
> Every label row carries `source`, `confidence`, and `labeler`. A label file
|
|
> without them fails a schema check.
|
|
|
|
The current file does not meet it. All 111 rows carry `source` and `v1_label`,
|
|
but none carries `confidence` or `labeler`: the file records where each label
|
|
came from and what it replaced, not a per-row certainty or a labeler name.
|
|
Inventing confidence values after the fact would be worse than the gap, so they
|
|
are absent rather than guessed.
|
|
|
|
Tier-2 labels must add both. `confidence` is the labeler's own certainty for
|
|
that row; `labeler` names the person or pass that produced it, because Tier 2 is
|
|
double-labelled and a disagreement has to be traceable to a pass. Tier 1
|
|
(`image6_v2.json`) is dev-only and its numbers never ship, so it stays as it is.
|