e25fcda67d
Enrolling a packshot from an external set is normal - it is how F-12 was closed - and it silently turns one of those rows into training data. The external sets were scored by an ad-hoc script that never checked, so sku_l06_1.jpg was being reported as a held-out correct accept while being byte-identical to the enrolled view. code/eval_external.py now scores those sets with the same leaked() check the shelf evaluator runs, and refuses a row that has since been enrolled. That check needed fixing first. Its 64x64 signature fires on two different photographs of the same product: the l09 rear label from a retailer page sits 19.5 from the enrolled front packshot, under the 20.0 line, while being a different photograph (resized MAD 26.3). A signature hit is now confirmed by correlation - a genuine re-cut, re-encode or rescale correlates at 0.99+, that pair at 0.28 - so the two signals together are decisive where either alone is not. Verified against both directions: identical, q55 re-encode and 700px rescale are caught; the different-photo pair is not. Also adds the van backend (always-argmax, no gate) so the stream comparison in DECISION.md is reproducible rather than asserted, with two tests pinning that it answers where the shipped gate abstains and still refuses an empty score set. External result on the corrected basis: 19 scored, 12 correct, 7 abstain, 0 false accepts, 1 leak excluded. Suite 307 -> 309. |
||
|---|---|---|
| .. | ||
| evaluations/2026-09-29-session | ||
| external-eval | ||
| packshots | ||
| README.md | ||
Recognition data
Everything the identity work needs to be reproducible: sourced packshots, evaluation runs, and the audits that produced them. Nothing here is ephemeral; a run that matters is written here, not to a temp directory.
packshots/source/
Front packshots as downloaded, before enrollment. ATTRIBUTION.json records the
source URL, the product as printed, and the manufacturer for each. ocr-reads.json
holds the OCR text each image produced, so a name/OCR refusal can be re-checked
without re-running Vision.
rebuild_catalog.py reads this directory and writes a catalog:
.venv/bin/python code/rebuild_catalog.py --source data/packshots/source \
--output stock-lopia-clean --csv stock-lopia/stock.csv --min-views 1 --replace
evaluations/
One directory per session. Each holds the raw artifacts a claim was computed from, so a number in DECISION.md can be traced back to the run that produced it.
| Directory | Contents |
|---|---|
stream/ |
WebSocket track dumps from stream_client.py, scored by score_stream.py. |
still/ |
/v1/recognize responses for the still-image path. |
catalog-audit/ |
rebuild_catalog.py --json reports. |
scratch/ |
Intermediate images (rotations, crops) kept because they document a finding. |
Scoring a stream dump:
.venv/bin/python code/score_stream.py data/evaluations/<session>/stream/<file>.json \
--labels eval/labels/image6.json --stock-dir stock-lopia-clean