tngo/product-recognition-demo
1
0
Fork 0
forked from vnguyen/product-recognition-demo
Code Pull requests Activity

docs: describe optional crop protocol and gallery cache

Browse source
This commit is contained in:
kiyo9w 2026-09-28 15:49:42 +09:00
parent 0d0e9f82b9
commit 2de5740922
2 changed files with 48 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

35
recognization/docs/API.md
View file

@ -224,7 +224,7 @@ Live frames in, tracked products out. One session per connection.
{"type": "hello", "protocol": 1, "session_id": "...", "catalog_version": "...",
"model": {"version": "...", "device": "cpu", "imgsz": 640},
"config": {"method": "hybrid", "conf": 0.1, "empty_slots": true},
"limits": {"max_frame_bytes": 4194304}}
"limits": {"max_frame_bytes": 4194304, "max_crop_bytes": 1048576}}
```
**Frames (binary):** `4-byte big-endian frame id` + JPEG bytes. The id is echoed back
@ -236,6 +236,8 @@ so the client can pair a result with the frame it sent.
| --- | --- |
| `{"type":"config","method":"orb","conf":0.15,"empty_slots":false}` | `{"type":"config_ack","config":{...}}` |
| `{"type":"inspect","track_id":12,"method":"hybrid","request_id":"r1"}` | `{"type":"inspect_result","request_id":"r1",...}` or an error |
| `{"type":"capabilities","max_crop_bytes":131072}` | `{"type":"capabilities_ack","capabilities":{"crop":true,"max_crop_bytes":131072}}` |
| `{"type":"crop","request_id":"c1","track_id":12,"frame_id":41,"jpeg":"<base64 JPEG>"}` | `{"type":"crop_ack","request_id":"c1","track_id":12,"frame_id":41}` or `crop_error` |
| `{"type":"reset"}` | `{"type":"reset_ack"}` |
| `{"type":"ping","t":1}` | `{"type":"pong","t":1}` |
@ -257,6 +259,37 @@ so the client can pair a result with the frame it sent.
"stats": {"frames_received": 264, "frames_processed": 264, "frames_dropped": 0,
"server_fps": 13.2, "classify_pending": 3, "classify_ms_p50": 88.9}}
```
The optional crop lane is additive to protocol `1`. Without `capabilities`, the
server sends no `crop_request` and existing clients keep their frame/control flow.
After opting in, a `result` may include `"crop_request":{"request_id":"c1",
"track_id":12,"frame_id":41,"box":[812,404,850,460]}` on an eligible
track. The box uses the **sent frame's** pixel coordinates (the raw detection),
not the camera's native resolution. Keep that exact frame's full-resolution
capture briefly, map its box into the capture, encode the crop as JPEG, and reply
in a text JSON `crop` message. Binary frames remain `4-byte id + JPEG`; the crop
does not take a frame id of its own. `crop_ack` means accepted for asynchronous
classification, not recognized; subsequent results carry the updated identity and
`identity.source` (`frame`, `crop`, or `inherited`). `stats.crops` has `requested`,
`accepted`, and `rejected` counts.
The hello limit is 1 MiB of **decoded crop bytes**. The optional
`max_crop_bytes` in `capabilities` states the client's limit; the acknowledged
session limit is clamped to 1 KiB–1 MiB (default 128 KiB when omitted). Use the
acknowledged limit, not only hello's ceiling. At most four requests are outstanding
per session and each track object is requested at most once. A request expires
after more than six processed frames; clients keep a bounded recent-frame buffer
and may miss a crop if its capture has already been evicted. A late response
cannot label a recycled track. Changing method or resetting invalidates requests.
Crop failures are recoverable text messages with `type: "crop_error"`, `code`,
and `message`. Pre-decode failures also echo `request_id`; session-level refusals
do not currently echo it. Codes: `CROP_BAD_MESSAGE` (missing/invalid base64 JPEG
or missing IDs), `CROP_TOO_LARGE` (empty or oversized decoded crop, or excessive
encoded length), `CROP_UNKNOWN_REQUEST` (no outstanding request, including one
already expired), `CROP_MISMATCH` (track/frame differs), `CROP_STALE` (late or
invalidated track/session), `CROP_UNDECODABLE` (unreadable image), and
`CROP_UNAVAILABLE` (classifier shutting down). A request is single-use, including
when its response is rejected; continue streaming frames rather than retrying it.
**Backpressure:** the server keeps one pending frame per session and processes the
newest, counting the ones it replaced in `frames_dropped`. A client should keep one