fix(replay): walk the sample at the app's real frame geometry

The replayed walk and the harness that measures it disagreed with the
phone in two ways, so every live number taken from them was measuring a
different pipeline than the one that ships.

The server cut a window `PAN_LONG_EDGE` tall, which on image6 shows one
shelf row: a 720 px band reaches 52 of 66 enrolled facings and gives
every other SKU 0% coverage, which reads as a matcher failure when it is
only the crop. It now spans the sample's full height and pans
horizontally, and `PAN_STEP` is expressed in sent-frame pixels so the
pace survives the downscale.

`stream_client.py` had the same window bug plus a box-mapping bug: it
added the pan offset but never divided by the downscale, so a `--width`
below the source size shrank every box and the evaluator matched the
wrong facing. It now scales the window itself (so the frame it sends is
byte-for-byte the app's 720x1280), records that scale per frame, and
divides by it when mapping a label box back to source coordinates. The
pan accumulator is clamped at the reversal: a negative `pan_x` satisfied
`<= 0` on every later frame and flipped the direction each time, parking
the window at the left edge.

Measured through the real server at the app's geometry: 57 of 66
enrolled facings covered (86%) at 94% precision, 9 distinct SKUs.
This commit is contained in:
kiyo9w 2026-09-29 10:09:22 +09:00
parent 7736465bee
commit e52684c37e
2 changed files with 71 additions and 25 deletions

View file

@ -97,11 +97,12 @@ CATALOG_VERSION_FALLBACK = "unknown"
MAX_CROP_PIXELS = 4_000_000
# The replayed-sample pan: a portrait window at the phone's frame size, JPEG at
# the quality the device camera sends. `PAN_STEP` is pixels per frame, so at the
# scan's ~10 fps the shelf walks ~80 px/s - brisk enough to cover the shelf in
# about 9 s without jumping past a facing between frames.
# the quality the device camera sends. `PAN_STEP` is pixels per frame **in the
# sent frame**, so it is divided by the downscale before slicing: the window is
# the sample's full height scaled to `PAN_LONG_EDGE`, and a step measured in
# source pixels would shrink with that scale and silently change the pace.
PAN_LONG_EDGE = 1280
PAN_STEP = 8
PAN_STEP = 5.6
PAN_JPEG_QUALITY = 70
@ -412,8 +413,16 @@ def create_app(
def _panned_jpeg(path: str, index: int) -> Optional[bytes]:
"""One portrait window of `path`, slid `index` steps along the shelf.
The window is the shape the phone sends - long edge `PAN_LONG_EDGE`,
9:16 - so a simulator sees the frame size the live path really gets.
The window is the shape the phone sends - 9:16, long edge
`PAN_LONG_EDGE` - but it spans the sample's **full height** and pans
horizontally. A window that is only `PAN_LONG_EDGE` tall shows one
shelf row: on `image6` (1500x2000, rows at y 200/560/870/1150/1440) a
720 px band centred at 640-1360 reaches 52 of 66 enrolled facings and
gives every other SKU 0% coverage, which reads as a matcher failure
when it is only the crop. Full height reaches all of them, and the
downscale to `PAN_LONG_EDGE` is the same 0.64 the evaluator's live
resolution uses.
It steps `PAN_STEP` pixels per frame and reverses at each edge, so the
walk is continuous rather than a jump back to the start.
@ -426,19 +435,27 @@ def create_app(
if image is None:
return None
height, width = image.shape[:2]
window_h = min(PAN_LONG_EDGE, height)
window_h = height
window_w = min(int(window_h * 9 / 16), width)
max_x = max(0, width - window_w)
travel = index * PAN_STEP
# `PAN_STEP` is in sent-frame pixels; the slice is in source pixels.
scale = min(1.0, PAN_LONG_EDGE / float(window_h))
step = PAN_STEP / scale if scale > 0 else PAN_STEP
travel = index * step
if max_x == 0:
x = 0
else:
# Triangle wave: pan out to the edge, then back, forever.
period = 2 * max_x
phase = travel % period
x = phase if phase <= max_x else period - phase
y = max(0, (height - window_h) // 2)
window = image[y:y + window_h, x:x + window_w]
x = int(phase if phase <= max_x else period - phase)
window = image[:, x:x + window_w]
if scale < 1.0:
window = cv2.resize(
window,
(int(round(window.shape[1] * scale)), int(round(window.shape[0] * scale))),
interpolation=cv2.INTER_AREA,
)
ok, encoded = cv2.imencode(".jpg", window, [int(cv2.IMWRITE_JPEG_QUALITY), PAN_JPEG_QUALITY])
return encoded.tobytes() if ok else None