docs(decision): date the header and re-measure the OCR cost
The header said 'Audited 2026-09-28' while the document now carries 2026-09-29 implementation results; it says both, and that audit-time figures are kept beside the later ones rather than replaced. The OCR cost row claimed 18.0 ms on 40 production crops with no committed script behind it. code/bench_ocr.py is that measurement: 23.3 ms p50, 37.2 ms p95, mean 23.1 - inside the recorded 18-26 ms band, at its upper half.
This commit is contained in:
parent
81a2ab10e7
commit
4d1c17f15d
2 changed files with 46 additions and 2 deletions
42
recognization/code/bench_ocr.py
Normal file
42
recognization/code/bench_ocr.py
Normal file
|
|
@ -0,0 +1,42 @@
|
|||
"""OCR cost per crop, on production-shaped crops.
|
||||
|
||||
The decision record claims ~18-26 ms per crop on Mac and re-measured 18.0 ms on
|
||||
40 production crops. This is that measurement, so the number is reproducible
|
||||
rather than remembered.
|
||||
|
||||
.venv/bin/python code/bench_ocr.py
|
||||
"""
|
||||
import json, statistics, sys, time
|
||||
sys.path.insert(0, 'code')
|
||||
import cv2
|
||||
from eval_identity import LocalLaneAdapter, crop_of
|
||||
|
||||
lane = LocalLaneAdapter('stock-lopia-clean')
|
||||
labels = json.load(open('eval/labels/image6.json'))
|
||||
image = cv2.imread('add/image6.jpg')
|
||||
|
||||
crops = []
|
||||
for facing in labels['facings']:
|
||||
if facing.get('label') in (None, 'exclude'):
|
||||
continue
|
||||
crop = crop_of(image, facing['box'])
|
||||
if crop is not None and crop.size:
|
||||
crops.append(crop)
|
||||
if len(crops) >= 40:
|
||||
break
|
||||
|
||||
# Warm the recognizer: the first call loads the Vision model.
|
||||
lane.text_lane.read(crops[0])
|
||||
|
||||
times = []
|
||||
for crop in crops:
|
||||
started = time.perf_counter()
|
||||
lane.text_lane.read(crop)
|
||||
times.append((time.perf_counter() - started) * 1000)
|
||||
|
||||
times.sort()
|
||||
print(f'{len(times)} crops')
|
||||
print(f' p50 {statistics.median(times):.1f} ms')
|
||||
print(f' mean {statistics.mean(times):.1f} ms')
|
||||
print(f' p95 {times[int(len(times) * 0.95)]:.1f} ms')
|
||||
print(f' min {times[0]:.1f} max {times[-1]:.1f}')
|
||||
|
|
@ -2,7 +2,9 @@
|
|||
|
||||
Audited 2026-09-28 · submodule `25dbf13` on `fix/live-recognition-quality` plus
|
||||
its uncommitted diff (snapshot: `git stash create` → `2785c07`) · auditor:
|
||||
Claude Opus 5.5
|
||||
Claude Opus 5.5. Implemented and re-measured 2026-09-29; every section that
|
||||
carries a later number says so, and the audit-time figures are kept beside them
|
||||
rather than replaced.
|
||||
Scope: identity pipeline (matcher, gallery, decision, evaluator, catalogs,
|
||||
labels), live track/label logic, the uncommitted diff. Not audited: detector
|
||||
quality, tag/planogram pipeline, iOS/Android rendering, on-device runtime.
|
||||
|
|
@ -214,7 +216,7 @@ Ordered by how much of the gap to a good product each class explains.
|
|||
| D | Retrieval → Van's matcher | yes (DINO) | DINO recall collapses with scale (7/42 @10 at 493 SKUs) | DINO adds ~41 ms per crop | **Reject DINO retrieval.** |
|
||||
| D′ | RootSIFT descriptor index (FLANN) → vote → verify top-10 | yes | 35–37/42, equal to or slightly better than exhaustive | 1.7–3.9 ms per crop flat from 7 to 493 SKUs; build 171 ms for 64k descriptors | **Adopt past ~150 SKUs.** |
|
||||
| E | Retrieval → learned verifier | no | — | — | Only if C wins. |
|
||||
| F | OCR text lane against catalog text signatures | yes (macOS Vision) | Veto: precision 62 → 82 % at τ=0.5 for −1 correct. As a lane: +10 correct at live, including unenrolled SKUs. **Audit-time figures on v1 labels**; the current measurement is in § Re-measured 2026-09-29 (E-3): the lane adds 16 pp of coverage at full resolution while raising precision. | ~18–26 ms per crop on Mac (re-measured 18.0 ms on 40 production crops); on-device on iOS (Vision) and Android (ML Kit) | **Adopt.** |
|
||||
| F | OCR text lane against catalog text signatures | yes (macOS Vision) | Veto: precision 62 → 82 % at τ=0.5 for −1 correct. As a lane: +10 correct at live, including unenrolled SKUs. **Audit-time figures on v1 labels**; the current measurement is in § Re-measured 2026-09-29 (E-3): the lane adds 16 pp of coverage at full resolution while raising precision. | ~18–26 ms per crop on Mac; re-measured 23.3 ms p50 / 37.2 ms p95 on 40 production crops (`code/bench_ocr.py`); on-device on iOS (Vision) and Android (ML Kit) | **Adopt.** |
|
||||
| G | Fusion (geometry + colour + text + temporal + tag prior) | partly (geometry + colour + text) | Largest measured gain | — | **Adopt**; the text and geometry errors are complementary (text reads siblings apart; geometry covers text-poor faces). |
|
||||
| H | Store-trained identity head on DINO | no (it was trained on the eval shelf) | Leaked; no valid measurement exists | — | Retire with DINO. |
|
||||
| J | Shelf-tag / planogram prior | no (`tags.py` exists) | Every facing run on image6 sits above a readable tag with name and price | — | Strong hypothesis, H-02. |
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue