The engine owns the OCR backend, a shared OCR lock and a dedicated tag worker, so
a tag pass can never occupy a classification worker or slow a frame. A live
session plans rail bands on the frame loop (cheap), runs them on that worker, and
attaches whatever the cache already knows; a still runs one synchronous pass.
Tracks and detections now carry price_tag and reconciliation, and the summary
carries price_tags plus compliance. Config can turn tags off per session.
Compliance now fails closed on evidence quality. The first real run produced 106
"missing tag" violations on a shelf where only 8 tags were readable - our tag
*detection* was incomplete, not the shelf. A row is only judged when at least
half its facings carry tag evidence; otherwise it reports "tags detected for 14
of 91 facings in this row" as a hint. Missing-tag findings are one per row with a
count, not one per facing, and price_mismatch still reports every disagreement.
On add/image6.jpg the result is now 7 real price mismatches and no false
violations.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Detection and classification share the CPU, and leaving both uncapped made them
fight: uncapped torch plus 8 workers measured 11.1 results/s single-client and
dropped to 6.3/s each with two clients. Capping torch at 4 threads and
classifying with 5 workers gives 12.2 results/s single-client (46 ms detection)
and 8.5/s each for two clients, with zero unexplained label changes.
Detection is the part that must stay real-time; classification fills in behind
it, so the defaults now protect detection's share of the CPU. Both remain
overridable (--classify-workers, DORAYAKI_TORCH_THREADS).
docs/API.md records the shipped numbers and says which configuration produced
them, including the honest gap: detection during a live pan is ~46-48 ms, not
the 28 ms of an idle measurement.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The client is a pure API consumer: live frames go out over /v1/stream, stills
over /v1/recognize, and it can be pointed at another server with
?server=&token=. It keeps the team's visual identity (green palette, logo, EN/JA
toggle) and adds the live surface: state-coloured boxes with name/price chips,
interpolated track boxes, a results/s + latency HUD, quality hints, and an
in-view list that highlights a SKU's boxes.
Photo mode recognises the whole shelf at once instead of one click per product:
boxes paint immediately, identities fill in, tapping a product inspects it
(crop + feature-match visualization), the algorithm selector re-classifies, and
dispense updates stock.
Fixes found by browser verification with a synthetic camera feed:
- The WebSocket handler referenced its processing loop before definition, so the
socket died with NameError and live mode never received a result. Helpers are
now defined inside the handler.
- The client never kept a reference to the video element, so frame capture
returned early and nothing was ever sent.
- /v1/images/{id}/classify passed detection dicts where the engine expected box
dicts (KeyError: 'width').
- Placeholder chrome (thumb, dashes, hidden groups) leaked on first paint.
Verified in Chrome with a fake camera: live 8-12 results/s, ~60 products in
view, product card and feature matching populated, photo 105/170 recognised on
image.jpg and 59/111 on add/image6.jpg, 0 console errors.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
app.py becomes the recognition server: /v1 REST (health, catalog + template
images, samples, recognize, classify, inspect, stock) plus /v1/stream over
WebSocket, with token auth, size limits, one consistent error shape, and the
legacy /api/sessions routes kept but now backed by the real engine instead of
the fake adapter. Default bind is 127.0.0.1; LAN and TLS are opt-in.
stream_client.py is the reference non-browser client (camera, video file, or a
pan across a still) and prints results/s, RTT, detect time and label stability.
Live quality work driven by measurements:
- Labels are remembered per place: a fresh track covering a recently recognized
product inherits the label instead of queueing for classification again. On a
continuous pan this raised recognized products in view from 9/66 to 17-21/66
and cut time-to-first-label to ~1.6 s.
- Classification priority is newest-first, because sorting by area spends the
budget on products that are already leaving the frame.
- Per-session classification concurrency raised to 12.
hybrid_fast (ORB shortlist then hybrid) is implemented but NOT the default: it
is 3.4x faster (13 ms vs 44 ms p50) yet agreed with hybrid on only 47% of
accepted crops across 553 crops, far below the 98% adoption rule. It stays
available for experiments; the numbers are in the commit and the benchmark.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
engine.py is the process-wide, thread-safe core the server is built on: one
YOLO instance behind a lock, thread-local matcher access, a classification pool
so detection is never starved, the stock catalog under its own lock, empty-slot
geometry normalized to a reference width (the detector's gap thresholds are
absolute pixels), and a small LRU of recent still images for later
classify/inspect calls. Detection output is normalized so every client sees the
same box shape.
tracking.py wraps ultralytics BYTETracker. The important part: ultralytics
mints track ids from a process-global counter and resets it when a tracker is
constructed, so with two live clients one client's ids would be re-issued while
the other still held cached labels. SessionTracker gives each tracker its own
counter and never touches the global one.
Measured defaults (bench_engine.py, M5): detection p50 26-29 ms on CPU vs
112 ms on MPS with byte-identical boxes, so device defaults to cpu; hybrid
classifies at ~51 ms/crop, 92 crops/s across 8 threads.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
