The server segfaulted a few seconds into every live scan that used the crop
lane. The crash report put the fault in libtorch's Metal shader library
(`MetalShaderLibrary::exec_unary_kernel` inside an unordered_map insert),
reached from `copy_cast_kernel_mps` under `clone`/`reshape`.
Reproduced deterministically: eight threads calling `embed()` on a fresh
process crash it, while the same eight threads after one warm forward are
fine. Metal's shader library is process-global and is not thread-safe while
kernels are still being compiled into it, which is exactly what the first
forward does. The detector already serializes its predict for this reason;
the gallery did not.
Two changes:
* A module-level `_MPS_INFER_LOCK` covers the whole MPS region of a forward -
model load, `.to(device)`, the forward, and the normalize that follows it.
Normalize dispatches its own Metal kernels, so leaving it outside the lock
left the same first-compile race open. Module-level because the state being
protected is the process's, not one gallery's.
* A cache hit skips the embedding forward, so the model used to be built and
the kernels compiled by whichever classify worker asked first - up to eight
racing into one process-wide cache. The load path now warms the backbone
single-threaded under the existing lock, on the cache-hit branch too. The
readiness path loads the gallery before the socket accepts, so the warmup
is free. It is Metal-only and best-effort: a CPU gallery has no shader
library to compile, and a cache hit is served without a usable backbone by
contract.
The regression test records overlap rather than timing: a fake backbone that
sleeps and counts concurrent entries, run from four threads, asserting the
count never exceeds one. It fails on the previous gallery.py and needs no MPS.
A cache key proves the store, not the rows inside the file. The old check
asked only that every cached row appear somewhere in the enrolled list, so a
row moved across SKUs still passed: the matrix that travelled with it would
then be filed under the wrong product, and a valid key would rename a SKU.
Rows now walk the enrolled list forward and never backwards, so a moved,
repeated or foreign row has nowhere left to match. Views that could not be
decoded stay absent without disturbing the order of the ones after them,
which is what keeps a shorter cache a hit rather than a re-embed.
The user's objection was right and it changed the design: the tag channel is
sparse in the real world (measured: 0 tags readable at ~2 m on the main shelf), so
packaging recognition has to carry identity. Measured on a 12-class, 57-facing
evaluation set read off the demo shelf, the packaging channel as shipped cannot:
views per SKU embedding top-1 shipped matcher top-1
1 57 % 33 %
2 72 % 27 %
3 75 % 20 %
4 76 % 5 %
Two things follow. First, two views per SKU is where most of the gain is - that is
the enrollment spec (a first scan already produces several facings per product).
Second, the matcher gets *worse* with more views, because its raw inlier score is
not calibrated to be aggregated and it cannot express two views of one SKU at all:
it keys templates by filename.
So identity becomes retrieval: `gallery.py` embeds every enrolled view with DINOv2
ViT-S/14, scores a SKU by its best view, and rejects when nothing is close enough
(an unknown product must come back unknown, not as the nearest stranger). Measured
with held-out views - the gallery never contains its own queries - 2 views per SKU
over 12 SKUs gives 69 % top-1 with rejection working, and the threshold moves the
rejection line.
It is affordable in the live loop only on the GPU: 159 ms per crop on CPU versus
3.6 ms on the M5's MPS, and a live frame only needs its new tracks embedded.
157 tests pass; test_gallery.py covers the retrieval contract (best view wins,
several views are one answer, strangers are rejected) with injected embeddings so
it needs no torch.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
