From 25dbf132e728437e2fe14ae088e493935d5f77e5 Mon Sep 17 00:00:00 2001 From: kiyo9w Date: Mon, 28 Sep 2026 19:10:31 +0900 Subject: [PATCH] docs: describe the mobile lane as shipped, and fix the capabilities example Two things in API.md described a state that no longer exists. The mobile section called the app's scan screen "a timer-driven placeholder today" and said wiring it "is a separate lane". The app ships the real client (`data/shelf/LiveScanClient.kt`, `platform/CameraFrames`) and ADR-002 governs it, so the section now names what the client actually is and what it needs from this server, including the negotiated crop lane. The control-message table showed `{"type":"capabilities","max_crop_bytes":N}` as the opt-in. `handle_capabilities` refuses exactly that message - it requires `"crop": true` - so the documented example returned `BAD_MESSAGE` and left the lane off. Verified against a running server: the corrected message answers `capabilities_ack`, the old one answers `BAD_MESSAGE`. --- recognization/docs/API.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/recognization/docs/API.md b/recognization/docs/API.md index 95e291e..6f8c11d 100644 --- a/recognization/docs/API.md +++ b/recognization/docs/API.md @@ -236,7 +236,7 @@ so the client can pair a result with the frame it sent. | --- | --- | | `{"type":"config","method":"orb","conf":0.15,"empty_slots":false}` | `{"type":"config_ack","config":{...}}` | | `{"type":"inspect","track_id":12,"method":"hybrid","request_id":"r1"}` | `{"type":"inspect_result","request_id":"r1",...}` or an error | -| `{"type":"capabilities","max_crop_bytes":131072}` | `{"type":"capabilities_ack","capabilities":{"crop":true,"max_crop_bytes":131072}}` | +| `{"type":"capabilities","crop":true,"max_crop_bytes":131072}` | `{"type":"capabilities_ack","capabilities":{"crop":true,"max_crop_bytes":131072}}` | | `{"type":"crop","request_id":"c1","track_id":12,"frame_id":41,"jpeg":""}` | `{"type":"crop_ack","request_id":"c1","track_id":12,"frame_id":41}` or `crop_error` | | `{"type":"reset"}` | `{"type":"reset_ack"}` | | `{"type":"ping","t":1}` | `{"type":"pong","t":1}` | @@ -415,9 +415,12 @@ is matched against the catalog separately. - **Stock is in memory** and resets when the server restarts. - **`hybrid_fast` is experimental** and off by default. -## Mobile client notes (future lane) +## Mobile client -The protocol is capture-source agnostic; an iPad/phone client needs no server changes. +The protocol is capture-source agnostic. The dorayaki app ships a native client +against it: `shared/.../data/shelf/LiveScanClient.kt` holds the Ktor WebSocket, +`platform/CameraFrames` supplies AVFoundation/CameraX frames, and ADR-002 in the +app repo governs the client side. What the client needs from this server: - WebSocket client (Ktor `WebSockets` plugin), one frame in flight. - Capture JPEG frames from `AVCaptureVideoDataOutput` (iOS) or CameraX @@ -425,9 +428,12 @@ The protocol is capture-source agnostic; an iPad/phone client needs no server ch same object-fit transform the web client uses. - iOS ATS: allow local networking (`NSAllowsLocalNetworking`) or use HTTPS; Android needs cleartext permission for plain `http://`. -- The dorayaki app repo requires an ADR before adding a second API base URL, and its - scan screen is a timer-driven placeholder today - wiring it to this server is a - separate lane. +- The crop lane is negotiated, not assumed: announce + `{"type":"capabilities","crop":true,"max_crop_bytes":N}` after `hello`, wait for + `capabilities_ack`, then answer `track.crop_request` with the exact + `request_id`/`track_id`/`frame_id` and the crop's JPEG as base64. Silence is a + valid answer for a frame the camera no longer holds; the request expires after + `crop_ttl_frames`. ## Tests