Replaces the DINO gallery as the recognition path with the measured design
from docs/research/2026-09-28-recognition-architecture/DECISION.md: RootSIFT
over a per-SKU reference pyramid, similarity-constrained RANSAC and a hue
check, fused with an OCR text lane that vetoes a match whose package text
names a different catalog row and answers on its own when geometry finds
nothing.
- identity_lanes.py: LocalLane, TextLane, DecisionPolicy. The text lane
builds each row's signature from the catalog name plus its packshot OCR, so
a row with no enrolled image is still answerable.
- engine.py: every backend groups scores by canonical SKU, so no
view-suffixed id reaches the wire and a name and price always resolve.
decide_crop fuses the lanes; score_crop_lanes feeds the live path. Both
lanes are built in warm_up (non-fatally) so the first frame is not the slow
one, and the text lane shares the engine's OCR lock.
- live.py: decides per crop on the lane's own rank order. The lane's order
comes from its rank key and its value is only the gate, so re-sorting by
value named a different SKU than the lane chose; averaging in an empty
observation also diluted a good confidence below the accept line.
- identity.py: a `local` score scale whose gate is one absolute line, because
the value is already a margin over the runner-up.
- app.py, run-demo-server.sh: `--identity local` is the default; the demo
server no longer starts on the leaked `stock-lopia` catalog.
- eval_live.py: the replay vocabulary accepts `local`, so a recording from
the shipped server can be scored.
- README.md: documents the three backends and what each is for.
Measured on the corrected image6 labels at live resolution: 36/61 in-catalog
facings named at 95% precision, 0 of 48 unknowns accepted, 11 labels sourced
from the text lane alone. Through the real server: /v1/recognize 90%
precision, /v1/stream 100% precision, no view-suffixed ids on either.
The previous commit stopped the live path re-gating gallery scores at
COSINE_ACCEPT, but it passed zero accept lines. `aggregate_votes` divides by the
whole observation window, so one rejected crop beside one accepted 0.45 averages
to 0.225 - and a zero gate labelled that, at a score the gallery itself calls
unknown.
The aggregate now clears the gallery's own line: `head_threshold` (0.40) when a
fine-tuned head is loaded, `threshold` (0.55) otherwise. The margin gate is
still the only extra gate, so a head match between 0.40 and 0.60 is labelled.
Regression test: a rejected crop followed by an accepted 0.45 must not label the
track (fails on the zero-gate version).
Verified: `python -m unittest discover -s code -p 'test_*.py'` - 258 tests, OK,
2 skipped.
`score_crop` returns only scores the gallery already accepted: the raw 0.55
line, or the fine-tuned head's own 0.40. The live path then ranked that
aggregate with `decide_identity(scale="cosine")`, whose defaults rewrite to
COSINE_ACCEPT 0.60 / COSINE_AMBIGUOUS 0.50 - a second, higher line. Every head
match between 0.40 and 0.60 was therefore refused or downgraded to ambiguous,
which is the live regression: the head's own training views score 0.57-0.77.
`engine.inspect` and `_classify_with_gallery` already avoid this by ranking
through `decision_for_match`, which applies only the published margin. The live
path now does the same: on the cosine scale it passes explicit zero accept
lines, so the margin gate is the only extra gate.
Regression tests: a 0.45 head-scale match is labelled (fails on the previous
code), and a 0.45/0.44 pair inside the margin stays unnamed.
Verified: `python -m unittest discover -s code -p 'test_*.py'` - 257 tests, OK,
2 skipped.
The crop lane asked the phone for a full-resolution crop of a facing the
frame could not decide. Four defects made that unsafe or useless:
* A capability was half-applied. `crop: true` without a usable
`max_crop_bytes` enabled the lane on the client's say-so, leaving the two
sides believing different limits. A capability is now believed whole or
refused with BAD_MESSAGE, and the limit must be an integer in 1..1048576
(bool is an int in Python and is not a limit).
* A small JPEG could claim enormous dimensions. The header is now read
without decoding and anything over 4 megapixels is refused before the
decode allocates it.
* A request was single-use, so an expired one stranded a facing only the
crop lane could decide. The lane now spends the same max_attempts budget
the frame path spends, refilled on method change and on settled-label
drift, and a phone that never answers still cannot loop.
* A candidate could be asked about from a box the frame path would refuse:
clipped by the frame edge, carried over from an earlier frame, or measured
at a score the tracker does not trust. Both lanes now share one edge gate,
and only a track whose box this frame actually measured is asked about.
Drift and method change also cleared `identity` but left `confirmed = True`,
which permanently blocked both the crop lane and label inheritance for that
track; both now clear it, without which the budget reset was dead code.
Blur and glare are refused as whole-capture faults, while `too_far` is
exempt on purpose: a distant shelf is the one frame a native-resolution crop
exists to decide.
The live session scored crops with the matcher regardless of the configured identity
backend, so a live scan used the old mechanism while the still path used the enrolled
gallery - measured, the matcher accepts look-alikes and cannot say "unknown", which is
exactly what a real shelf needs it to do. score_crop now routes to the gallery, and
the decision uses a similarity scale (accept at 0.60 with a 0.04 margin) rather than
inlier-count thresholds that mean nothing on a cosine.
The synthetic walk is gone: demo.py, demo_replay.py, the /v1/demo endpoints and the
walk benchmark are deleted, and the app's frame source is the camera on a device (the
single-image replay stays for the simulator, which has no camera).
stock-demo merges both demo shelves (27 products, 62 views, trained head) so one
server recognises either image, and run-demo-server.sh starts it bound to 0.0.0.0 and
prints the exact URL to put in Config.xcconfig.
Verified over the LAN with 1280x960 frames: 40 frames in 5.3s, 9 products recognised
on demo image 1.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
From the deep review's hygiene list:
- The demo replay (a still recognition replayed as if it were live tracking) moves
out of app.py into demo_replay.py, and every payload it produces is marked
`synthetic: true`. It had been committed inside an unrelated feature commit, which
made a film-making device look like part of the server.
- `/v1/health` reports the OCR engine and whether it is available. Without it an
operator could not tell a broken OCR from a shelf with no readable tags.
- A failed OCR pass is counted and logged instead of swallowed: `TagService.stats()`
now carries `passes_failed` and `last_pass_error`.
- The live classification budget goes back to the measured production defaults
(16 in flight, 2 attempts); a demo walk over a nearly static shelf asks for more
through `--live-inflight` / `--live-attempts` rather than silently changing the
default for everyone.
162 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/v1/demo/plan + /v1/demo/frame give a demo the one thing a still image cannot:
motion. The virtual camera pans a shelf row left to right, steps up, and pans
back - a store employee sweeping a shelf - over a real shelf photo, ~1 ms per
frame, deterministic, and wrapping so an unattended demo loops. The client drives
only `step`, so the walk's speed follows whatever frame rate it can sustain.
Three real defects came out of measuring the walk rather than a still:
- Bands were keyed by on-screen position, so a walking camera minted a new key
every few pixels of pan and re-read the same shelf continuously. Keyed by row
now; OCR stops chasing the window.
- The classification budget was consumed by retries of products already leaving
the frame. Retries now skip anything near the edge, and a track gets 2 attempts.
- The pan advance was 0.22 of the window per step: a facing was in view for one
or two frames, so nothing could ever be classified (9 labelled tracks in 20 s).
0.06 keeps a product in view for about a second.
Measured on the walk (20 s, 1280x720 frames, one client): orb labels 296 tracks
across 21 SKUs with the first label at 0.2 s, while hybrid manages 13 tracks /
4 SKUs at 5.4 s - so the live default for a demo is the fast method, with hybrid
kept for tap-to-inspect.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The engine owns the OCR backend, a shared OCR lock and a dedicated tag worker, so
a tag pass can never occupy a classification worker or slow a frame. A live
session plans rail bands on the frame loop (cheap), runs them on that worker, and
attaches whatever the cache already knows; a still runs one synchronous pass.
Tracks and detections now carry price_tag and reconciliation, and the summary
carries price_tags plus compliance. Config can turn tags off per session.
Compliance now fails closed on evidence quality. The first real run produced 106
"missing tag" violations on a shelf where only 8 tags were readable - our tag
*detection* was incomplete, not the shelf. A row is only judged when at least
half its facings carry tag evidence; otherwise it reports "tags detected for 14
of 91 facings in this row" as a hint. Missing-tag findings are one per row with a
count, not one per facing, and price_mismatch still reports every disagreement.
On add/image6.jpg the result is now 7 real price mismatches and no false
violations.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
app.py becomes the recognition server: /v1 REST (health, catalog + template
images, samples, recognize, classify, inspect, stock) plus /v1/stream over
WebSocket, with token auth, size limits, one consistent error shape, and the
legacy /api/sessions routes kept but now backed by the real engine instead of
the fake adapter. Default bind is 127.0.0.1; LAN and TLS are opt-in.
stream_client.py is the reference non-browser client (camera, video file, or a
pan across a still) and prints results/s, RTT, detect time and label stability.
Live quality work driven by measurements:
- Labels are remembered per place: a fresh track covering a recently recognized
product inherits the label instead of queueing for classification again. On a
continuous pan this raised recognized products in view from 9/66 to 17-21/66
and cut time-to-first-label to ~1.6 s.
- Classification priority is newest-first, because sorting by area spends the
budget on products that are already leaving the frame.
- Per-session classification concurrency raised to 12.
hybrid_fast (ORB shortlist then hybrid) is implemented but NOT the default: it
is 3.4x faster (13 ms vs 44 ms p50) yet agreed with hybrid on only 47% of
accepted crops across 553 crops, far below the 98% adoption rule. It stays
available for experiments; the numbers are in the commit and the benchmark.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
One LiveSession per streaming client. Each frame runs detection and tracking,
then classification is scheduled for products that are new, big enough, fully in
frame and not already settled; results are cached on the track so detection
never waits for classification. Empty-slot gaps are smoothed over frames and
quality hints (glare, blur, too far) come from a 320 px downscale.
Correctness work that the acceptance run forced out:
- Classification submissions happen OUTSIDE the session lock. add_done_callback
runs the callback inline when the future already completed, so attaching it
while holding the lock deadlocked the session on the second frame.
- A label is dropped only when its box has drifted off the product for two
consecutive frames (panning slides boxes onto neighbours), and the anchor is
the box the crop came from, not wherever the track moved by the time
classification finished.
- An id that disappears and comes back is a restart, not the same object: the
old label is cleared instead of silently labelling a neighbour. relabels and
restarts counters are monotonic per id so clients can tell a re-verification
from a silent flip.
- Track ids are tracked per session (tracking.py) and counters survive pruning.
Measured on the synthetic pan (30 frames, 248 label observations): 20
re-verifications, 0 unexplained label changes, detection 20 ms/frame.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
