vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

docs: describe optional crop protocol and gallery cache

Browse source
This commit is contained in:
kiyo9w 2026-09-28 15:49:42 +09:00
parent 0d0e9f82b9
commit 2de5740922
2 changed files with 48 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

14
recognization/README.md
View file

@ -64,6 +64,15 @@ cp "$HOME/.cache/huggingface/hub/models--timm--vit_small_patch14_dinov2.lvd142m/
`DORAYAKI_DINOV2_WEIGHTS` overrides that path. A file with any other sha256 is refused.
With the gallery identity backend, enrolled views are embedded into an ignored
`stock-dir/.gallery_cache_<image_size>.npz` feature cache. The cache is reused
only when its key matches the preprocessing version, image size, pinned DINOv2
SHA-256, catalog CSV, enrolled SKU/view order and image bytes; its matrix shape,
numeric finite values and row identities are also checked. A stale or malformed
cache is rebuilt. A cache hit still checks the local model file's SHA-256, and
gallery startup refuses readiness if the pinned weights or usable gallery are
missing. OCR continues on the separate tag worker (`tag_executor`).
(`requirements-dev.txt` = `requirements.txt` + `httpx` cho test.)
### Bước 4: Khởi chạy recognition server
@ -100,6 +109,11 @@ Camera trên điện thoại chỉ hoạt động khi trang mở bằng **https*
Server có REST + WebSocket đầy đủ, tài liệu chi tiết trong [`docs/API.md`](docs/API.md). Swagger UI: **http://127.0.0.1:8080/docs**.
Live WebSocket protocol `1` also supports an optional full-resolution crop lane:
clients advertise `capabilities`, then answer per-track `crop_request` messages
from their buffered capture. Clients without the capability are unchanged; wire
examples, byte limits, expiry and errors are in [`docs/API.md`](docs/API.md#websocket-v1stream).
```bash
# Nhận diện một ảnh kệ
curl -X POST --data-binary @image.jpg -H 'Content-Type: image/jpeg' \

35
recognization/docs/API.md
View file

@ -224,7 +224,7 @@ Live frames in, tracked products out. One session per connection.
{"type": "hello", "protocol": 1, "session_id": "...", "catalog_version": "...",
"model": {"version": "...", "device": "cpu", "imgsz": 640},
"config": {"method": "hybrid", "conf": 0.1, "empty_slots": true},
"limits": {"max_frame_bytes": 4194304}}
"limits": {"max_frame_bytes": 4194304, "max_crop_bytes": 1048576}}
```
**Frames (binary):** `4-byte big-endian frame id` + JPEG bytes. The id is echoed back
@ -236,6 +236,8 @@ so the client can pair a result with the frame it sent.
| --- | --- |
| `{"type":"config","method":"orb","conf":0.15,"empty_slots":false}` | `{"type":"config_ack","config":{...}}` |
| `{"type":"inspect","track_id":12,"method":"hybrid","request_id":"r1"}` | `{"type":"inspect_result","request_id":"r1",...}` or an error |
| `{"type":"capabilities","max_crop_bytes":131072}` | `{"type":"capabilities_ack","capabilities":{"crop":true,"max_crop_bytes":131072}}` |
| `{"type":"crop","request_id":"c1","track_id":12,"frame_id":41,"jpeg":"<base64 JPEG>"}` | `{"type":"crop_ack","request_id":"c1","track_id":12,"frame_id":41}` or `crop_error` |
| `{"type":"reset"}` | `{"type":"reset_ack"}` |
| `{"type":"ping","t":1}` | `{"type":"pong","t":1}` |
@ -257,6 +259,37 @@ so the client can pair a result with the frame it sent.
"stats": {"frames_received": 264, "frames_processed": 264, "frames_dropped": 0,
"server_fps": 13.2, "classify_pending": 3, "classify_ms_p50": 88.9}}
```
The optional crop lane is additive to protocol `1`. Without `capabilities`, the
server sends no `crop_request` and existing clients keep their frame/control flow.
After opting in, a `result` may include `"crop_request":{"request_id":"c1",
"track_id":12,"frame_id":41,"box":[812,404,850,460]}` on an eligible
track. The box uses the **sent frame's** pixel coordinates (the raw detection),
not the camera's native resolution. Keep that exact frame's full-resolution
capture briefly, map its box into the capture, encode the crop as JPEG, and reply
in a text JSON `crop` message. Binary frames remain `4-byte id + JPEG`; the crop
does not take a frame id of its own. `crop_ack` means accepted for asynchronous
classification, not recognized; subsequent results carry the updated identity and
`identity.source` (`frame`, `crop`, or `inherited`). `stats.crops` has `requested`,
`accepted`, and `rejected` counts.
The hello limit is 1 MiB of **decoded crop bytes**. The optional
`max_crop_bytes` in `capabilities` states the client's limit; the acknowledged
session limit is clamped to 1 KiB–1 MiB (default 128 KiB when omitted). Use the
acknowledged limit, not only hello's ceiling. At most four requests are outstanding
per session and each track object is requested at most once. A request expires
after more than six processed frames; clients keep a bounded recent-frame buffer
and may miss a crop if its capture has already been evicted. A late response
cannot label a recycled track. Changing method or resetting invalidates requests.
Crop failures are recoverable text messages with `type: "crop_error"`, `code`,
and `message`. Pre-decode failures also echo `request_id`; session-level refusals
do not currently echo it. Codes: `CROP_BAD_MESSAGE` (missing/invalid base64 JPEG
or missing IDs), `CROP_TOO_LARGE` (empty or oversized decoded crop, or excessive
encoded length), `CROP_UNKNOWN_REQUEST` (no outstanding request, including one
already expired), `CROP_MISMATCH` (track/frame differs), `CROP_STALE` (late or
invalidated track/session), `CROP_UNDECODABLE` (unreadable image), and
`CROP_UNAVAILABLE` (classifier shutting down). A request is single-use, including
when its response is rejected; continue streaming frames rather than retrying it.
**Backpressure:** the server keeps one pending frame per session and processes the
newest, counting the ones it replaced in `frames_dropped`. A client should keep one