From 2de574092289b29370ac1494fefe692d6cfe0a0f Mon Sep 17 00:00:00 2001 From: kiyo9w Date: Mon, 28 Sep 2026 15:49:42 +0900 Subject: [PATCH] docs: describe optional crop protocol and gallery cache --- recognization/README.md | 14 ++++++++++++++ recognization/docs/API.md | 35 ++++++++++++++++++++++++++++++++++- 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/recognization/README.md b/recognization/README.md index bcff2d4..20e4f23 100644 --- a/recognization/README.md +++ b/recognization/README.md @@ -64,6 +64,15 @@ cp "$HOME/.cache/huggingface/hub/models--timm--vit_small_patch14_dinov2.lvd142m/ `DORAYAKI_DINOV2_WEIGHTS` overrides that path. A file with any other sha256 is refused. +With the gallery identity backend, enrolled views are embedded into an ignored +`stock-dir/.gallery_cache_.npz` feature cache. The cache is reused +only when its key matches the preprocessing version, image size, pinned DINOv2 +SHA-256, catalog CSV, enrolled SKU/view order and image bytes; its matrix shape, +numeric finite values and row identities are also checked. A stale or malformed +cache is rebuilt. A cache hit still checks the local model file's SHA-256, and +gallery startup refuses readiness if the pinned weights or usable gallery are +missing. OCR continues on the separate tag worker (`tag_executor`). + (`requirements-dev.txt` = `requirements.txt` + `httpx` cho test.) ### Bước 4: Khởi chạy recognition server @@ -100,6 +109,11 @@ Camera trên điện thoại chỉ hoạt động khi trang mở bằng **https* Server có REST + WebSocket đầy đủ, tài liệu chi tiết trong [`docs/API.md`](docs/API.md). Swagger UI: **http://127.0.0.1:8080/docs**. +Live WebSocket protocol `1` also supports an optional full-resolution crop lane: +clients advertise `capabilities`, then answer per-track `crop_request` messages +from their buffered capture. Clients without the capability are unchanged; wire +examples, byte limits, expiry and errors are in [`docs/API.md`](docs/API.md#websocket-v1stream). + ```bash # Nhận diện một ảnh kệ curl -X POST --data-binary @image.jpg -H 'Content-Type: image/jpeg' \ diff --git a/recognization/docs/API.md b/recognization/docs/API.md index 989c2c0..6a37ee7 100644 --- a/recognization/docs/API.md +++ b/recognization/docs/API.md @@ -224,7 +224,7 @@ Live frames in, tracked products out. One session per connection. {"type": "hello", "protocol": 1, "session_id": "...", "catalog_version": "...", "model": {"version": "...", "device": "cpu", "imgsz": 640}, "config": {"method": "hybrid", "conf": 0.1, "empty_slots": true}, - "limits": {"max_frame_bytes": 4194304}} + "limits": {"max_frame_bytes": 4194304, "max_crop_bytes": 1048576}} ``` **Frames (binary):** `4-byte big-endian frame id` + JPEG bytes. The id is echoed back @@ -236,6 +236,8 @@ so the client can pair a result with the frame it sent. | --- | --- | | `{"type":"config","method":"orb","conf":0.15,"empty_slots":false}` | `{"type":"config_ack","config":{...}}` | | `{"type":"inspect","track_id":12,"method":"hybrid","request_id":"r1"}` | `{"type":"inspect_result","request_id":"r1",...}` or an error | +| `{"type":"capabilities","max_crop_bytes":131072}` | `{"type":"capabilities_ack","capabilities":{"crop":true,"max_crop_bytes":131072}}` | +| `{"type":"crop","request_id":"c1","track_id":12,"frame_id":41,"jpeg":""}` | `{"type":"crop_ack","request_id":"c1","track_id":12,"frame_id":41}` or `crop_error` | | `{"type":"reset"}` | `{"type":"reset_ack"}` | | `{"type":"ping","t":1}` | `{"type":"pong","t":1}` | @@ -257,6 +259,37 @@ so the client can pair a result with the frame it sent. "stats": {"frames_received": 264, "frames_processed": 264, "frames_dropped": 0, "server_fps": 13.2, "classify_pending": 3, "classify_ms_p50": 88.9}} ``` +The optional crop lane is additive to protocol `1`. Without `capabilities`, the +server sends no `crop_request` and existing clients keep their frame/control flow. +After opting in, a `result` may include `"crop_request":{"request_id":"c1", +"track_id":12,"frame_id":41,"box":[812,404,850,460]}` on an eligible +track. The box uses the **sent frame's** pixel coordinates (the raw detection), +not the camera's native resolution. Keep that exact frame's full-resolution +capture briefly, map its box into the capture, encode the crop as JPEG, and reply +in a text JSON `crop` message. Binary frames remain `4-byte id + JPEG`; the crop +does not take a frame id of its own. `crop_ack` means accepted for asynchronous +classification, not recognized; subsequent results carry the updated identity and +`identity.source` (`frame`, `crop`, or `inherited`). `stats.crops` has `requested`, +`accepted`, and `rejected` counts. + +The hello limit is 1 MiB of **decoded crop bytes**. The optional +`max_crop_bytes` in `capabilities` states the client's limit; the acknowledged +session limit is clamped to 1 KiB–1 MiB (default 128 KiB when omitted). Use the +acknowledged limit, not only hello's ceiling. At most four requests are outstanding +per session and each track object is requested at most once. A request expires +after more than six processed frames; clients keep a bounded recent-frame buffer +and may miss a crop if its capture has already been evicted. A late response +cannot label a recycled track. Changing method or resetting invalidates requests. + +Crop failures are recoverable text messages with `type: "crop_error"`, `code`, +and `message`. Pre-decode failures also echo `request_id`; session-level refusals +do not currently echo it. Codes: `CROP_BAD_MESSAGE` (missing/invalid base64 JPEG +or missing IDs), `CROP_TOO_LARGE` (empty or oversized decoded crop, or excessive +encoded length), `CROP_UNKNOWN_REQUEST` (no outstanding request, including one +already expired), `CROP_MISMATCH` (track/frame differs), `CROP_STALE` (late or +invalidated track/session), `CROP_UNDECODABLE` (unreadable image), and +`CROP_UNAVAILABLE` (classifier shutting down). A request is single-use, including +when its response is rejected; continue streaming frames rather than retrying it. **Backpressure:** the server keeps one pending frame per session and processes the newest, counting the ones it replaced in `frames_dropped`. A client should keep one