feat: web client with live and photo modes, served by the server

The client is a pure API consumer: live frames go out over /v1/stream, stills
over /v1/recognize, and it can be pointed at another server with
?server=&token=. It keeps the team's visual identity (green palette, logo, EN/JA
toggle) and adds the live surface: state-coloured boxes with name/price chips,
interpolated track boxes, a results/s + latency HUD, quality hints, and an
in-view list that highlights a SKU's boxes.

Photo mode recognises the whole shelf at once instead of one click per product:
boxes paint immediately, identities fill in, tapping a product inspects it
(crop + feature-match visualization), the algorithm selector re-classifies, and
dispense updates stock.

Fixes found by browser verification with a synthetic camera feed:
- The WebSocket handler referenced its processing loop before definition, so the
  socket died with NameError and live mode never received a result. Helpers are
  now defined inside the handler.
- The client never kept a reference to the video element, so frame capture
  returned early and nothing was ever sent.
- /v1/images/{id}/classify passed detection dicts where the engine expected box
  dicts (KeyError: 'width').
- Placeholder chrome (thumb, dashes, hidden groups) leaked on first paint.

Verified in Chrome with a fake camera: live 8-12 results/s, ~60 products in
view, product card and feature matching populated, photo 105/170 recognised on
image.jpg and 59/111 on add/image6.jpg, 0 console errors.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
kiyo9w 2026-09-24 12:42:02 +09:00
parent 5f84f4f2a7
commit 6902067322
5 changed files with 1454 additions and 50 deletions

View file

@ -374,7 +374,14 @@ class RecognitionEngine:
raise UnknownMethod(method)
if not boxes:
return []
ordered = sorted(boxes, key=lambda b: b["width"] * b["height"], reverse=True)
def area_of(box: Dict[str, Any]) -> int:
if "width" in box and "height" in box:
return int(box["width"]) * int(box["height"])
x1, y1, x2, y2 = [int(v) for v in box["box"]]
return max(0, x2 - x1) * max(0, y2 - y1)
ordered = sorted(boxes, key=area_of, reverse=True)
crops = [self.crop(image, box["box"]) for box in ordered]
if method == "hybrid_fast":
futures = [self._executor.submit(self.matcher.score_crop_fast, crop) for crop in crops]