Compare commits

..

6 commits

Author SHA1 Message Date
kiyo9w
fbf31d74fe feat(decision): make the accept lines tunable from the environment
`LOCAL_ACCEPT` and `TEXT_MIN_SHARED` were module constants, so sweeping
them meant editing the source and restarting. Both lanes now read
`DORAYAKI_LOCAL_ACCEPT` / `DORAYAKI_TEXT_MIN_SHARED` with the constants
as defaults, which is what the calibration sweeps need and what a demo
host needs to loosen or tighten the gates without a code change.

Defaults are unchanged, so the shipped behaviour is identical.
2026-09-29 10:09:25 +09:00
kiyo9w
e52684c37e fix(replay): walk the sample at the app's real frame geometry
The replayed walk and the harness that measures it disagreed with the
phone in two ways, so every live number taken from them was measuring a
different pipeline than the one that ships.

The server cut a window `PAN_LONG_EDGE` tall, which on image6 shows one
shelf row: a 720 px band reaches 52 of 66 enrolled facings and gives
every other SKU 0% coverage, which reads as a matcher failure when it is
only the crop. It now spans the sample's full height and pans
horizontally, and `PAN_STEP` is expressed in sent-frame pixels so the
pace survives the downscale.

`stream_client.py` had the same window bug plus a box-mapping bug: it
added the pan offset but never divided by the downscale, so a `--width`
below the source size shrank every box and the evaluator matched the
wrong facing. It now scales the window itself (so the frame it sends is
byte-for-byte the app's 720x1280), records that scale per frame, and
divides by it when mapping a label box back to source coordinates. The
pan accumulator is clamped at the reversal: a negative `pan_x` satisfied
`<= 0` on every later frame and flipped the direction each time, parking
the window at the left edge.

Measured through the real server at the app's geometry: 57 of 66
enrolled facings covered (86%) at 94% precision, 9 distinct SKUs.
2026-09-29 10:09:22 +09:00
kiyo9w
7736465bee docs(decision): correct the record against the re-measured evidence
The audit's own numbers were right; several claims built on them were not, and
the advisors found each one:

- 'wins both axes at both resolutions' - false at live, where the lane trades
  59% coverage for 93% precision against Van's 66% at 39%. Stated as the trade
  it is.
- G1's bootstrap now exists: the local lane's correct set is a strict superset
  of Van's on the stream, so the 95% lower bound is a hard +0.0 pp - a floor of
  that pairing, not a margin. tracks_final.json is level with Van and does not
  clear it.
- The far-unknown rate is 0.066% on 4580 crops at the shipped configuration,
  not 0.30% on 2650 at the old one.
- The E-3 ablation compared text-presence *and* a different argmax; both arms
  now go through the same DecisionPolicy ordering. Text adds 9-10 pp of
  coverage at full resolution while raising precision.
- The sweep table swept one axis while pinning the other; replaced with the
  joint sweep.
- The two Re-measured tables are dated snapshots, labeled as such.
2026-09-29 09:28:19 +09:00
kiyo9w
730a5e782b fix(catalog): re-source sku_l09 and enforce the pyramid floor
sku_l09.jpg was 407x305, so its 400 px pyramid level was a 305->400 upscale: a
blur with no high-frequency detail to disagree with, which won 31 of 54 crops
on the shelf it was cut from. Re-sourced at 1000x1000.

The floor is now a hard gate in rebuild_catalog.py - a view under 800 px tall
is refused, not reported - because reporting it is what let the 305 px view
ship. data/packshots/source/sku_l07.jpg (600x600) is refused on this rule.

README records the real sizes and the provenance for every row.
2026-09-29 09:28:16 +09:00
kiyo9w
5094b5f6ca fix(recognition): calibrate the accept lines against unrelated shelves
Every threshold was fitted on image6, where each crop is a facing of a product
the shelf holds, so the set could not price a false accept. Measured on 4580
crops from 64 unrelated Wikimedia shelf photos at the shipped configuration:

  LOCAL_ACCEPT 2.0 -> 4.0   false accepts 60 (1.31%) -> 3 (0.07%)
  TEXT_MIN_SHARED 2 -> 4    the two-bigram reads that named sku_l12 via おい/いし

On image6 that is 44 shown / 42 correct at full resolution - 95.5% precision,
65.6% coverage - the highest-coverage cell of the joint sweep that passes both
G2 (>= 95%) and G3 (<= 1%). At live resolution 56% end-to-end at 97%.

The text lane's evidence floor gates its answer only: zeroing the row's score
would reorder the ranking and make the margin gate compare a different pair
than the one it was measured on.

code/sweep_accept_lines.py sweeps both axes jointly through the shipped
DecisionPolicy; the earlier table swept one axis while pinning the other, so
its baseline rows were not the old line.
2026-09-29 09:28:13 +09:00
kiyo9w
4ce41ecb17 feat(server): walk the replayed sample instead of freezing it
A still replayed as frames gave every frame the same boxes, so the live path
never saw a track move. /v1/samples/{id}?pan=N now returns a portrait 720x1280
window slid N*PAN_STEP pixels along the sample, reversing at each edge, at the
frame size and JPEG quality the phone sends.

The decoded image is cached by (mtime, size): re-reading a 1500x2000 JPEG per
frame cost more than the encode and made the walk slower than PAN_STEP says.
Measured after the cache: 3.3 ms mean per panned frame, 301 frames/s ceiling.

Two tests, both verified to fail before the change.
2026-09-29 09:28:07 +09:00
23 changed files with 1397 additions and 361 deletions

View file

@ -36,3 +36,6 @@ data/external-shelves/*/*.png
data/packshots/research/*/images/
data/packshots/research/*/pages/
data/packshots/research/*/*.jpg
# Rebuildable sweep cache (code/sweep_accept_lines.py --cache).
*.pkl

View file

@ -96,6 +96,15 @@ WS_IDLE_TIMEOUT_S = 120.0
CATALOG_VERSION_FALLBACK = "unknown"
MAX_CROP_PIXELS = 4_000_000
# The replayed-sample pan: a portrait window at the phone's frame size, JPEG at
# the quality the device camera sends. `PAN_STEP` is pixels per frame **in the
# sent frame**, so it is divided by the downscale before slicing: the window is
# the sample's full height scaled to `PAN_LONG_EDGE`, and a step measured in
# source pixels would shrink with that scale and silently change the pace.
PAN_LONG_EDGE = 1280
PAN_STEP = 5.6
PAN_JPEG_QUALITY = 70
def _jpeg_dimensions(data: bytes) -> Optional[Tuple[int, int]]:
"""Read JPEG SOF dimensions without allocating a decoded image."""
@ -299,6 +308,9 @@ def create_app(
# path -> (mtime_ns, size, width, height). Dropped when the file changes.
sample_dimensions: Dict[str, Tuple[int, int, int, int]] = {}
sample_dimensions_lock = threading.Lock()
# path -> ((mtime_ns, size), decoded image). The pan re-cuts this per frame.
sample_images: Dict[str, Any] = {}
sample_image_lock = threading.Lock()
sessions: Dict[str, LiveSession] = {}
sessions_lock = asyncio.Lock()
app.state.engine = engine
@ -398,6 +410,77 @@ def create_app(
sample_dimensions[path] = (key[0], key[1], width, height)
return width, height
def _panned_jpeg(path: str, index: int) -> Optional[bytes]:
"""One portrait window of `path`, slid `index` steps along the shelf.
The window is the shape the phone sends - 9:16, long edge
`PAN_LONG_EDGE` - but it spans the sample's **full height** and pans
horizontally. A window that is only `PAN_LONG_EDGE` tall shows one
shelf row: on `image6` (1500x2000, rows at y 200/560/870/1150/1440) a
720 px band centred at 640-1360 reaches 52 of 66 enrolled facings and
gives every other SKU 0% coverage, which reads as a matcher failure
when it is only the crop. Full height reaches all of them, and the
downscale to `PAN_LONG_EDGE` is the same 0.64 the evaluator's live
resolution uses.
It steps `PAN_STEP` pixels per frame and reverses at each edge, so the
walk is continuous rather than a jump back to the start.
The decoded image is cached: re-reading a 1500x2000 JPEG per frame cost
more than the encode, which made the walk slower than `PAN_STEP` says.
"""
import cv2
image = _decoded_sample(path)
if image is None:
return None
height, width = image.shape[:2]
window_h = height
window_w = min(int(window_h * 9 / 16), width)
max_x = max(0, width - window_w)
# `PAN_STEP` is in sent-frame pixels; the slice is in source pixels.
scale = min(1.0, PAN_LONG_EDGE / float(window_h))
step = PAN_STEP / scale if scale > 0 else PAN_STEP
travel = index * step
if max_x == 0:
x = 0
else:
# Triangle wave: pan out to the edge, then back, forever.
period = 2 * max_x
phase = travel % period
x = int(phase if phase <= max_x else period - phase)
window = image[:, x:x + window_w]
if scale < 1.0:
window = cv2.resize(
window,
(int(round(window.shape[1] * scale)), int(round(window.shape[0] * scale))),
interpolation=cv2.INTER_AREA,
)
ok, encoded = cv2.imencode(".jpg", window, [int(cv2.IMWRITE_JPEG_QUALITY), PAN_JPEG_QUALITY])
return encoded.tobytes() if ok else None
def _decoded_sample(path: str) -> Any:
"""The sample decoded once, re-read only when the file changes."""
try:
stat = os.stat(path)
except OSError:
with sample_image_lock:
sample_images.pop(path, None)
return None
key = (stat.st_mtime_ns, stat.st_size)
with sample_image_lock:
cached = sample_images.get(path)
if cached is not None and cached[0] == key:
return cached[1]
import cv2
image = cv2.imread(path)
if image is None:
return None
with sample_image_lock:
sample_images[path] = (key, image)
return image
def _sample_listing() -> List[Dict[str, Any]]:
listing = []
for item in samples.values():
@ -414,11 +497,22 @@ def create_app(
return {"samples": await loop.run_in_executor(None, _sample_listing)}
@app.get("/v1/samples/{sample_id}", dependencies=[Depends(authorize)])
async def get_sample(sample_id: str) -> Response:
async def get_sample(sample_id: str, pan: Optional[int] = None) -> Response:
item = samples.get(sample_id)
if not item or not await asyncio.get_running_loop().run_in_executor(None, os.path.exists, item["path"]):
raise _error("NOT_FOUND", f"no sample '{sample_id}'", 404)
return FileResponse(item["path"])
if pan is None:
return FileResponse(item["path"])
# A still replayed as frames is a frozen shelf: every frame carries the
# same boxes, so the live path never exercises a track that moves. `pan`
# is the frame index, and the window slides one step per frame and
# reverses at the edges - the same walk `stream_client.py` drives, so a
# simulator sees what a real slow pan produces.
loop = asyncio.get_running_loop()
jpeg = await loop.run_in_executor(None, _panned_jpeg, item["path"], max(0, pan))
if jpeg is None:
raise _error("NOT_FOUND", f"cannot read sample '{sample_id}'", 404)
return Response(content=jpeg, media_type="image/jpeg")
# ---------------------------------------------------------------- recognize
@app.post("/v1/recognize", dependencies=[Depends(authorize)])

View file

@ -507,7 +507,7 @@ class RecognitionEngine:
know about. Without OCR the local lane stands alone.
"""
from identity import decide_identity
from identity_lanes import DecisionPolicy
from identity_lanes import LOCAL_ACCEPT, TEXT_MIN_SHARED, DecisionPolicy
if self.identity_scale != "local":
return decide_identity(
@ -525,9 +525,12 @@ class RecognitionEngine:
text_rank = sorted(
text_lane.scores_from_text(read).items(), key=lambda kv: kv[1], reverse=True
)
return DecisionPolicy(signatures=text_lane.signatures if text_lane else None).decide(
lane.rank(crop), text_rank, text_read=read
policy = DecisionPolicy(
local_accept=float(os.environ.get("DORAYAKI_LOCAL_ACCEPT", LOCAL_ACCEPT)),
text_min_shared=int(os.environ.get("DORAYAKI_TEXT_MIN_SHARED", TEXT_MIN_SHARED)),
signatures=text_lane.signatures if text_lane else None,
)
return policy.decide(lane.rank(crop), text_rank, text_read=read)
def score_crop_lanes(
self, crop: Any, method: Optional[str] = None, skip_ocr: bool = False

View file

@ -5,8 +5,11 @@ that share nothing with the LOPIA catalog. Every accept here is a false accept
by construction, which is what makes the set useful: the image6 shelf can only
measure false accepts on the products it happens to contain.
The stored eval.json predates the open-set gate, so it is re-run rather than
trusted.
Measured the way the server measures: a 1280-long-side frame detected at
`LiveSettings.conf`. The sources are 4032-4512 px photos, and detecting them at
full resolution and at the default `detect_conf` instead measures a
configuration the phone never runs, which is how a threshold change reads as a
pass on evidence the product does not produce.
.venv/bin/python code/eval_distractors.py --stock stock-lopia-clean \
data/external-shelves/wikimedia-commons
@ -28,7 +31,13 @@ if current_dir not in sys.path:
import cv2 # noqa: E402
from eval_identity import LOCAL_ACCEPT, LocalLaneAdapter # noqa: E402
from eval_identity import ( # noqa: E402
LIVE_LONG_SIDE,
LOCAL_ACCEPT,
LocalLaneAdapter,
at_live_resolution,
)
from identity_lanes import TEXT_MIN_SHARED # noqa: E402
def main(argv: List[str] | None = None) -> int:
@ -40,9 +49,11 @@ def main(argv: List[str] | None = None) -> int:
args = parser.parse_args(argv)
from engine import build_default_engine, EngineSettings
from live import LiveSettings
engine = build_default_engine(EngineSettings(stock_dir=args.stock))
lane = LocalLaneAdapter(args.stock)
conf = LiveSettings().conf
for directory in args.directories:
manifest_path = os.path.join(directory, "manifest.json")
try:
@ -65,14 +76,15 @@ def main(argv: List[str] | None = None) -> int:
if image is None:
continue
started = time.perf_counter()
detections = engine.detect(image)
frame = at_live_resolution(image)
detections = engine.detect(frame, conf=conf)
accepted = 0
for detection in detections:
box = detection.get("box") or detection.get("bbox")
if not box:
continue
x1, y1, x2, y2 = [int(v) for v in box]
crop = image[max(0, y1):y2, max(0, x1):x2]
crop = frame[max(0, y1):y2, max(0, x1):x2]
if crop.size == 0:
continue
decision = lane.decide(crop, lane.rank(crop), accept=LOCAL_ACCEPT,
@ -93,6 +105,14 @@ def main(argv: List[str] | None = None) -> int:
"dataset": os.path.basename(directory.rstrip("/")),
"catalog": args.stock,
"note": "Unrelated shelves: every accept is a false accept by construction.",
# The configuration the numbers above were measured under, so the
# stored evidence is checkable against the constants it claims.
"config": {
"frame_long_side": LIVE_LONG_SIDE,
"detect_conf": conf,
"local_accept": LOCAL_ACCEPT,
"text_min_shared": TEXT_MIN_SHARED,
},
"images": len(rows),
"detections": detections,
"false_accepts": accepts,

View file

@ -418,11 +418,16 @@ class LocalLaneAdapter(Lane):
self.lane = LocalLane(stock_dir)
# E-3 asks what the text lane is worth, which is only answerable by
# running the same crops without it. Building the lane anyway keeps the
# comparison to one variable: the OCR cost is paid either way.
# running the same crops without it. `--no-text` therefore drops the
# lane whole: nothing in the decision reads it, and an ablation that
# still had to build an OCR engine could not be run on a machine
# without one.
self.use_text = use_text
self.text_lane = TextLane.from_catalog(
_catalog_rows(stock_dir), stock_dir, ocr if ocr is not None else _ocr_engine()
self.text_lane = (
TextLane.from_catalog(
_catalog_rows(stock_dir), stock_dir, ocr if ocr is not None else _ocr_engine()
)
if use_text else None
)
def rank(self, crop: Any) -> List[Tuple[str, float]]:
@ -433,9 +438,10 @@ class LocalLaneAdapter(Lane):
from identity_lanes import DecisionPolicy
if not self.use_text:
from identity import decide_identity
return decide_identity(dict(ranked), scale="local", min_score=accept)
# E-3's other arm: the same policy, over the same ranking, with no
# text evidence. `dict(ranked)` would not do - it drops the lane's
# order, and the value is not the order (see `LocalLane.rank`).
return DecisionPolicy(local_accept=accept).decide(ranked)
read = self.text_lane.read(crop)
text_rank = sorted(
self.text_lane.scores_from_text(read).items(),
@ -523,8 +529,6 @@ def score_lane(
the SKU is on the shelf but has no reference, so an accept there is wrong
and is the failure the enrollment protocol exists to prevent.
"""
from identity import decide_identity
in_catalog = [f for f in facings if f["label"] != UNKNOWN]
out_catalog = [f for f in facings if f["label"] == UNKNOWN]
enrolled_count = sum(1 for f in in_catalog if f["label"] in enrolled_skus)

View file

@ -11,7 +11,7 @@ store was enrolled; scoring them is training-set accuracy.
python code/eval_live.py RECORDING_DIR [--stock-dir DIR] [--labels PATH]
[--json PATH] [--iou 0.5]
[--replay] [--identity matcher|local|gallery] [--device cpu|mps|cuda]
[--replay] [--identity matcher|local|gallery|van] [--device cpu|mps|cuda]
Prints a human count table and writes the same report as JSON (default
<recording>/eval_live.json). Missing input exits 2 with {"error": ...} on
@ -66,7 +66,7 @@ def content_sha256(payload: bytes) -> str:
# Every backend the server can be started with, so a recording made by the
# shipped `run-demo-server.sh` (--identity local) can be replayed. `local` is
# the default; `matcher` is Van's scorer kept as the E-1 baseline lane.
IDENTITY_BACKENDS = ("matcher", "local", "gallery")
IDENTITY_BACKENDS = ("matcher", "local", "gallery", "van")
def _iou(a: Sequence[float], b: Sequence[float]) -> float:

View file

@ -45,28 +45,38 @@ from rebuild_catalog import bigrams, normalize_text, signature_bigrams, sku_of
# wrong label on a shelf is worse than a missing one.
#
# 4.0 since 2026-09-29: the line was tuned on image6 alone, where every crop is
# a facing of a product the shelf actually holds. On 2297 crops from unrelated
# Wikimedia shelves, 2.0 accepted 25 of them on geometry alone (scores 2.26-4.81
# with no text overlap at all), a 1.1% far-unknown rate against G3's 1%. At 4.0
# that falls to 8 (0.35%) while image6 keeps 42 of 48 correct labels at 95.5%
# precision, which is the only region passing both G2 and G3 with coverage left.
# a facing of a product the shelf actually holds, so that set could not price
# it. Re-measured with `code/sweep_accept_lines.py` on 4580 crops from 64
# unrelated Wikimedia shelf photos - 1280-long-side frames at
# `LiveSettings.conf`, the configuration the phone runs - against
# `TEXT_MIN_SHARED = 4`: 2.0 accepts 60 of them (1.31%) and 4.0 accepts 3
# (0.07%), against G3's 1%. On image6 at full resolution 4.0 shows 44 labels of
# which 42 are right - 95.5% precision, and 65.6% of the 64 facings that hold a
# catalog row - and it is the highest-coverage cell in the sweep that passes
# both G2 and G3.
LOCAL_ACCEPT = 4.0
# The text lane's own accept line: a bigram overlap this high, with this much
# margin over the runner-up, names the row on its own. Measured: the text lane
# picks the right SKU on 41-42 of the 55 facings where it reads anything.
TEXT_ACCEPT = 0.5
TEXT_MARGIN = 0.15
# How many of the read's bigrams the winning row must actually contain. The
# ratio alone is not evidence: a two-pair read the row happens to contain scores
# 1.0, and "サソース" named sku_l08 that way. Two pairs was the old floor, and it
# is too low once the lane answers on unrelated shelves: on 2297 crops from
# Wikimedia milk, soy and snack shelves, 20 of 26 text-lane answers rested on
# exactly two shared bigrams ("避くて おいしい" -> sku_l12 via おい/いし), an
# 11.4% false accept rate against G3's 1%. Four is where the two sides separate
# once the local line is also at 4.0: false accepts fall to 8 (0.35%) while
# image6 keeps 42 of its 48 correct labels. It is a floor on evidence, not on
# the ratio, so a long read of shared brand copy still scores low and is still
# refused by the margin.
# How many of the read's bigrams the winning row must actually contain before
# that read may name a product. A floor on the amount of shared evidence, not
# on the ratio: a two-pair read the row happens to contain scores 1.0, and
# "サソース" named sku_l08 that way.
#
# It gates the text lane's *answer* and nothing else. Zeroing the row's score
# would be a different policy under the same name: it reorders the ranking, so
# a zeroed top row promotes a runner-up and the margin gate ends up comparing a
# different pair than the one it was measured on. It would also gate the veto,
# which is backwards - the veto only ever refuses a local accept, so a floor
# under it would let a two-bigram read rescue a match the local lane earned.
# `code/sweep_accept_lines.py` measures the line by calling the policy below
# with this value, so the sweep and the shipped lane cannot disagree about what
# it means.
#
# The ratio stays the ranking's own gate, so a long read of shared brand copy
# still scores low and is still refused by the margin.
TEXT_MIN_SHARED = 4
# Reference pyramid heights. A ~1000 px packshot against a 110-250 px crop is a
# 4-8x ratio, and ORB's 8x1.2 pyramid tops out at 3.6x, which is why ORB and
@ -348,8 +358,15 @@ def text_similarity(
the most real text to match were punished hardest. Bounded this way the
same reads score 0.86 and 1.0 against their true rows, and the garbage
read ties across three rows and is refused by the margin.
`vocabulary` defaults to the weight table's own keys, which are by
construction the bigrams some signature produces. A second copy of that set
on the lane is a second thing to keep in step, and it is the drift the
module already paid for once with the folding rules.
"""
read_grams = bigrams(read)
if vocabulary is None and weights is not None:
vocabulary = set(weights)
if vocabulary is not None:
read_grams &= vocabulary
if not read_grams:
@ -404,12 +421,6 @@ class TextLane:
# Bigram weights over the catalog's own names, so the lane can tell
# copy every sibling prints from the words that name one product.
self.weights = bigram_weights(list(self.signatures.values()))
# Every pair any name can produce. A read's pairs outside this set are
# OCR noise or word-boundary artifacts, and counting them in the
# denominator charges a row for text it could never have explained.
self.vocabulary: Set[str] = set()
for text in self.signatures.values():
self.vocabulary |= signature_bigrams(text)
self.ocr_engine = ocr_engine
# The engine's OCR lock. Vision tolerates concurrency but the CPU does
# not, and this lane runs one OCR per crop on the classify pool, so it
@ -472,7 +483,7 @@ class TextLane:
if len(normalize_text(read)) < 4:
return {}
return {
sku: text_similarity(read, signature, self.weights, self.vocabulary)
sku: text_similarity(read, signature, self.weights)
for sku, signature in self.signatures.items()
}

View file

@ -306,6 +306,9 @@ class LiveSession:
self._frames_dropped = 0
self._process_times: "collections.deque[float]" = collections.deque(maxlen=STATS_WINDOW)
self._classify_times: "collections.deque[float]" = collections.deque(maxlen=STATS_WINDOW)
# A lane that raised is an operational failure, not "nothing matched".
# Counted so a broken run is never read as a coverage change.
self._classify_failures = 0
self._slot_history: "collections.deque[List[Dict[str, Any]]]" = collections.deque(
maxlen=EMPTY_SLOT_HISTORY
)
@ -1198,7 +1201,14 @@ class LiveSession:
) -> None:
try:
scores, text_rank, text_read = future.result()
except Exception:
except Exception as exc:
# A lane that raised is an operational failure, not a crop that
# matched nothing. Swallowing it made a broken run (tracks, no
# labels, clean log) indistinguishable from a coverage change, so
# it is counted and logged before the empty answer is used.
with self._lock:
self._classify_failures += 1
LOGGER.exception("classification failed for track %s: %s", track_id, exc)
scores, text_rank, text_read = {}, [], ""
finished_at = time.perf_counter()
# Resolved before the session lock is taken. `identity_text_lane` builds
@ -1270,7 +1280,7 @@ class LiveSession:
from identity import aggregate_votes, decide_identity
from identity_lanes import DecisionPolicy
from identity_lanes import LOCAL_ACCEPT, TEXT_MIN_SHARED, DecisionPolicy
mean_scores = aggregate_votes(track.observations)
scale = getattr(self.engine, "identity_scale", "inlier")
@ -1299,7 +1309,9 @@ class LiveSession:
# in an empty observation halves a good confidence below the
# accept line, which is the dilution the cosine branch guards too.
decision = DecisionPolicy(
signatures=text_lane.signatures if text_lane else None
local_accept=float(os.environ.get("DORAYAKI_LOCAL_ACCEPT", LOCAL_ACCEPT)),
text_min_shared=int(os.environ.get("DORAYAKI_TEXT_MIN_SHARED", TEXT_MIN_SHARED)),
signatures=text_lane.signatures if text_lane else None,
).decide(list(scores.items()), text_rank, text_read=text_read)
elif scale == "van":
# Van's contract on the stream: the aggregate's top SKU, always
@ -1457,6 +1469,7 @@ class LiveSession:
"frames_dropped": self._frames_dropped,
"server_fps": fps,
"classify_pending": sum(track.in_flight for track in self._tracks.values()),
"classify_failures": self._classify_failures,
"classify_ms_p50": classify_p50,
}

View file

@ -7,7 +7,9 @@ its own exam.
This rebuilds a catalog directory from named packshots and refuses, per view:
* a view shorter than MIN_HEIGHT or narrower than MIN_WIDTH,
* a view shorter than MIN_HEIGHT, narrower than MIN_WIDTH, or shorter than
PYRAMID_FLOOR (a source under it cannot be stored as the protocol's
200/400/800 px pyramid, so its scores would not be comparable with the rest),
* a view whose capture session appears in --eval-manifest,
* a packshot whose catalog name is not supported by its OCR text,
* a view that is a near-duplicate of another view of the same SKU,
@ -63,10 +65,16 @@ MIN_WIDTH = 120
# The enrollment protocol's floor: a reference view under 160 px tall carries no
# text geometry (facings under 60 px wide failed every measured lane).
MIN_HEIGHT = 160
# Correct packshots measured 0.79-1.00 of the catalog name's bigrams in their
# OCR text; the wrong-size Kewpie 450g packshot tops out at 0.64 against every
# name in the catalog. 0.70 sits between them with room for OCR noise.
MIN_NAME_OCR = 0.70
# The name/OCR gate, re-measured on the shipped packshots with the strict name
# side in place, each image read four ways and scored against all 12 catalog
# names: correct packshots 0.9167-1.0000 (seven read their whole name; l11's
# read omits "マヨネーズ" and is the floor), the wrong-product packshot in
# references/rejected/ 0.0476-0.5833. 0.75 is the midpoint: 0.167 of headroom
# above every correct packshot and 0.167 below the wrong-product ceiling, so
# OCR noise on either side does not move a verdict. Siblings are not this
# gate's job - the コクうま packshot scores 0.85 against the マヨネーズ name,
# which is what text_discriminated_siblings flags, not a gap a threshold closes.
MIN_NAME_OCR = 0.75
# Vision drops small print at native scale: the l11 packshot reads ハーフ only
# when upscaled (0.57 -> 0.79). Reading at 2x is what makes the check about the
# package rather than about Vision's recall.
@ -75,10 +83,12 @@ OCR_SCALE = 2.0
DUPLICATE_COSINE = 0.98
# The enrollment protocol's pyramid target: a source at least this tall can be
# stored as a 200/400/800 px pyramid. Below it the view still enrolls - the hard
# reject is MIN_HEIGHT - but its scores are not comparable with the rest, which
# is the documented l09 defect: at 305 px its inlier scores dwarf every other
# reference (85-129 against 10-40) and it wins facings it should not.
# stored as a 200/400/800 px pyramid, so a view under it cannot be scored
# against the rest. Refused, not reported: the l09 defect was a 305 px view
# whose inlier scores (85-129) dwarfed every other reference (10-40) and it won
# facings it should not, and a report is exactly the kind of thing that gets
# read once and then ignored. The fix is re-sourcing, which is a data task
# this tool should not let through the door in the first place.
PYRAMID_FLOOR = 800
# Official pack-shot pages publish a single front view per JAN, so a catalog
@ -317,6 +327,12 @@ def audit(
refused.append({"sku": sku, "view": name,
"reason": f"{image.shape[1]}px wide, below the {MIN_WIDTH}px floor"})
continue
if image.shape[0] < PYRAMID_FLOOR:
refused.append({"sku": sku, "view": name,
"reason": f"{image.shape[0]}px tall, below the {PYRAMID_FLOOR}px "
"pyramid floor; its scores would not be comparable "
"with the rest - re-source this view"})
continue
if ocr_engine is not None and names is not None:
try:
text = best_orientation_text(ocr_engine, image, names.get(sku, ""))
@ -453,27 +469,6 @@ def load_shelf_skus(path: Optional[str]) -> Optional[Set[str]]:
raise RebuildError(f"{path}: expected a JSON list or one SKU per line")
def below_pyramid_floor(kept: Dict[str, List[str]], source: str) -> List[Dict[str, Any]]:
"""Enrolled views too small to store as the protocol's pyramid.
Reported, not refused: the view is usable and dropping it would remove the
SKU from the catalog entirely, which is worse than an incomparable score.
The fix is re-sourcing the packshot, which is a data task.
"""
small: List[Dict[str, Any]] = []
for sku, filenames in sorted(kept.items()):
for filename in filenames:
image = cv2.imread(os.path.join(source, filename))
if image is not None and image.shape[0] < PYRAMID_FLOOR:
small.append({
"sku": sku,
"view": filename,
"height": int(image.shape[0]),
"floor": PYRAMID_FLOOR,
})
return small
def write_catalog(kept: Dict[str, List[str]], source: str, output: str, csv_path: Optional[str]) -> None:
"""Copy the kept views and the whole CSV into a staging dir, then swap it in."""
staging = tempfile.mkdtemp(prefix=".catalog-", dir=os.path.dirname(os.path.abspath(output)) or ".")
@ -582,7 +577,6 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
# and vetoes it from the CSV alone, so it is a coverage gap, not a refusal.
text_only = sorted(set(names) - set(kept))
siblings = text_discriminated_siblings(kept, args.source, names)
small = below_pyramid_floor(kept, args.source)
no_source = unattributed(args.source)
verdict = live_ready_verdict(kept, names, shelf_skus)
report = {
@ -596,7 +590,6 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
"ocr": engine.name if ocr_engine is not None else "skipped",
"text_only_skus": text_only,
"text_discriminated_siblings": siblings,
"below_pyramid_floor": small,
"unattributed_skus": no_source,
"completeness": verdict,
"skus": {sku: views for sku, views in sorted(kept.items())},
@ -616,11 +609,6 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
for row in siblings:
print(f" {row['sku']:<10} {row['view']:<20} ~ {row['sibling_sku']} "
f"({row['sibling_view']}) cosine {row['cosine']}")
if small:
print(f"\nbelow the {PYRAMID_FLOOR}px pyramid floor ({len(small)}):")
for row in small:
print(f" {row['sku']:<10} {row['view']:<20} {row['height']}px tall - "
"scores are not comparable with the rest; re-source this view")
if no_source:
print(f"\nno recorded source ({len(no_source)}): {', '.join(no_source)} - "
"usable, but provenance is missing; re-sourcing would close it")

View file

@ -15,6 +15,13 @@ Reported per run: correct / in-catalog facings, precision, unknown accepts, and
tracks that matched no labelled facing. A track whose box overlaps a facing the
catalog does not enrol is an unenrolled false accept - the dangerous kind, and
the one the evaluator's own precision denominator used to hide.
G1 has two halves - new covers at least as much as Van, and the lower bound of
the paired-bootstrap difference clears -2 pp - and the second half needs the
per-facing pairing, so every row carries the `facing` index it matched. Two
reports plus the labels then answer the gate on the same facings:
python code/score_stream.py NEW.json --van VAN.json
"""
from __future__ import annotations
@ -22,6 +29,7 @@ from __future__ import annotations
import argparse
import json
import os
import random
import sys
from typing import Any, Dict, List, Optional, Sequence, Tuple
@ -33,6 +41,14 @@ if current_dir not in sys.path:
# so the denominator keeps only facings at least this far inside the window.
VISIBLE_MIN = 0.5
# G1's second half: the lower bound of the paired difference must clear -2 pp.
G1_BOUND_MIN = -0.02
# Resampling draws for the paired bootstrap, and the seed that makes the bound
# a number this repo can print twice.
BOOTSTRAP_DRAWS = 10000
BOOTSTRAP_SEED = 0
def iou(a: Sequence[int], b: Sequence[int]) -> float:
ax1, ay1, ax2, ay2 = a
@ -72,6 +88,22 @@ def visible_fraction(box: Sequence[int], window: Sequence[int]) -> float:
return (inter_w * inter_h) / area if area > 0 else 0.0
def reachable(facings: Sequence[Dict[str, Any]],
windows: Optional[Sequence[Sequence[int]]]
) -> List[Dict[str, Any]]:
"""Facings a run's windows showed well enough for the run to have named."""
if not windows:
return list(facings)
return [f for f in facings
if max(visible_fraction(f["box"], w) for w in windows) >= VISIBLE_MIN]
def correct_facings(rows: Sequence[Dict[str, Any]]) -> set:
"""Facing indices at least one track got right - one hit each, not one per track."""
return {row["facing"] for row in rows
if row["outcome"] == "correct" and row["facing"] is not None}
def score(tracks: Sequence[Dict[str, Any]], facings: Sequence[Dict[str, Any]],
stock_dir: str, iou_min: float,
windows: Optional[Sequence[Sequence[int]]] = None
@ -90,11 +122,7 @@ def score(tracks: Sequence[Dict[str, Any]], facings: Sequence[Dict[str, Any]],
text lane names SKUs the image lane has no view for, which is the point of
it, and scoring that as a false accept punishes the lane for working.
"""
if windows:
facings = [
f for f in facings
if max(visible_fraction(f["box"], w) for w in windows) >= VISIBLE_MIN
]
facings = reachable(facings, windows)
in_catalog = [f for f in facings if f["label"] != "unknown"]
enrolled = enrolled_skus(stock_dir)
@ -122,7 +150,8 @@ def score(tracks: Sequence[Dict[str, Any]], facings: Sequence[Dict[str, Any]],
if best is None or best_iou < iou_min:
unmatched += 1
rows.append({"track_id": track["track_id"], "predicted": track["sku_id"],
"truth": None, "iou": round(best_iou, 3), "outcome": "unmatched"})
"facing": None, "truth": None, "iou": round(best_iou, 3),
"outcome": "unmatched"})
continue
truth = best["label"]
predicted = track["sku_id"]
@ -140,7 +169,8 @@ def score(tracks: Sequence[Dict[str, Any]], facings: Sequence[Dict[str, Any]],
wrong += 1
outcome = "wrong"
rows.append({"track_id": track["track_id"], "predicted": predicted,
"truth": truth, "iou": round(best_iou, 3), "outcome": outcome})
"facing": best["index"], "truth": truth,
"iou": round(best_iou, 3), "outcome": outcome})
shown = correct + wrong + unenrolled_false + unknown_false
return {
@ -155,10 +185,81 @@ def score(tracks: Sequence[Dict[str, Any]], facings: Sequence[Dict[str, Any]],
"shown": shown,
"coverage": round(len(named) / len(in_catalog), 4) if in_catalog else 0.0,
"precision": round(correct / shown, 4) if shown else 0.0,
# The window set travels with the report: a paired comparison has to
# know what each run was shown, not just what it named.
"windows": [list(w) for w in windows] if windows else None,
"rows": rows,
}
def paired_bootstrap(van: Dict[str, Any], new: Dict[str, Any],
facings: Sequence[Dict[str, Any]],
draws: int = BOOTSTRAP_DRAWS,
seed: int = BOOTSTRAP_SEED) -> Dict[str, Any]:
"""G1's second half: new minus Van, per facing, with the 95 % lower bound.
Both reports are graded on one facing set - the in-catalog facings either
run's windows showed - so a facing one run panned past cannot flatter the
other. Resampling draws facings with replacement rather than comparing two
independent rates, because the two runs saw the same photo: pairing removes
the facing-to-facing variance that otherwise swamps a 27-facing sample.
Percentile method on the sorted draw differences; the seed is fixed so the
number in a report is one this repo reproduces.
"""
windows = list(van.get("windows") or []) + list(new.get("windows") or [])
shared = sorted(f["index"] for f in reachable(facings, windows)
if f["label"] != "unknown")
if not shared:
return {"facings": 0, "difference": 0.0, "lower_95": 0.0, "upper_95": 0.0,
"van": 0, "new": 0, "draws": draws, "seed": seed}
van_hits = correct_facings(van["rows"])
new_hits = correct_facings(new["rows"])
van_vector = [1 if i in van_hits else 0 for i in shared]
new_vector = [1 if i in new_hits else 0 for i in shared]
n = len(shared)
rng = random.Random(seed)
spread = []
for _ in range(draws):
van_sum = new_sum = 0
for _ in range(n):
# The same draw index into both vectors: that is the pairing.
drawn = rng.randrange(n)
van_sum += van_vector[drawn]
new_sum += new_vector[drawn]
spread.append((new_sum - van_sum) / n)
spread.sort()
tail = max(1, int(0.025 * draws))
return {
"facings": n,
"van": sum(van_vector),
"new": sum(new_vector),
"van_rate": round(sum(van_vector) / n, 4),
"new_rate": round(sum(new_vector) / n, 4),
"difference": round((sum(new_vector) - sum(van_vector)) / n, 4),
"lower_95": round(spread[tail - 1], 4),
"upper_95": round(spread[min(draws - 1, int(0.975 * draws))], 4),
"draws": draws,
"seed": seed,
}
def load_run(path: str, window: Optional[str] = None
) -> Tuple[Dict[str, Any], Optional[List[Tuple[int, ...]]]]:
"""A stream_client --tracks dump and the source-image windows it showed."""
with open(path) as handle:
payload = json.load(handle)
if window:
windows = [tuple(int(part) for part in window.split(","))]
elif payload.get("windows"):
windows = [tuple(w) for w in payload["windows"]]
else:
windows = None
return payload, windows
def main(argv: Optional[Sequence[str]] = None) -> int:
parser = argparse.ArgumentParser(description="Score streamed labels against ground truth")
parser.add_argument("tracks", help="JSON written by stream_client.py --tracks")
@ -169,22 +270,20 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
parser.add_argument("--window", default=None,
help="x0,y0,x1,y1 the client showed; the denominator is "
"restricted to facings at least half inside it")
parser.add_argument("--van", default=None,
help="Van baseline track dump on the same clip; scores G1's "
"paired-bootstrap half against it")
parser.add_argument("--draws", type=int, default=BOOTSTRAP_DRAWS)
parser.add_argument("--seed", type=int, default=BOOTSTRAP_SEED)
parser.add_argument("--verbose", action="store_true", help="print every scored track")
args = parser.parse_args(argv)
with open(args.tracks) as handle:
payload = json.load(handle)
windows = None
if args.window:
windows = [tuple(int(part) for part in args.window.split(","))]
elif payload.get("windows"):
windows = [tuple(w) for w in payload["windows"]]
report = score(payload["tracks"], load_facings(args.labels), args.stock_dir,
args.iou, windows)
facings = load_facings(args.labels)
payload, windows = load_run(args.tracks, args.window)
report = score(payload["tracks"], facings, args.stock_dir, args.iou, windows)
report["source"] = payload.get("source")
report["pace"] = payload.get("pace")
report["seconds"] = payload.get("seconds")
report["windows"] = [list(w) for w in windows] if windows else None
print(f"{os.path.basename(args.tracks)} pace={report['pace']} "
f"{report['seconds']}s")
@ -197,7 +296,27 @@ def main(argv: Optional[Sequence[str]] = None) -> int:
if args.verbose:
for row in report["rows"]:
print(f" t{row['track_id']:4d} {row['outcome']:24s} "
f"pred={row['predicted']} truth={row['truth']} iou={row['iou']}")
f"facing={row['facing']} pred={row['predicted']} "
f"truth={row['truth']} iou={row['iou']}")
if args.van:
van_payload, van_windows = load_run(args.van, args.window)
van_report = score(van_payload["tracks"], facings, args.stock_dir,
args.iou, van_windows)
paired = paired_bootstrap(van_report, report, facings, args.draws, args.seed)
report["van"] = {k: van_report[k] for k in
("in_catalog_facings", "facings_named", "coverage",
"correct", "wrong", "precision", "shown")}
report["g1_paired"] = paired
print(f" G1 vs {os.path.basename(args.van)}: new {paired['new']}/"
f"{paired['facings']} ({paired['new_rate']:.0%}) - van {paired['van']}/"
f"{paired['facings']} ({paired['van_rate']:.0%}) = "
f"{paired['difference'] * 100:+.1f} pp")
print(f" paired bootstrap {paired['draws']} draws seed {paired['seed']}: "
f"95% bounds {paired['lower_95'] * 100:+.1f} pp to "
f"{paired['upper_95'] * 100:+.1f} pp; lower bound clears "
f"{G1_BOUND_MIN * 100:+.0f} pp: "
f"{'yes' if paired['lower_95'] >= G1_BOUND_MIN else 'NO'}")
if args.json_path:
with open(args.json_path, "w") as handle:

View file

@ -56,6 +56,10 @@ class FrameSource:
self.image = None
self.pan_x = 0
self.pan_direction = 1
# The downscale `frame()` applied to the window. A box the server
# measured is in the sent frame's pixels, so mapping it back to the
# source divides by this.
self.scale = 1.0
# Where the last frame was cut from the source image. (0, 0) for a
# camera or a video, whose frames are the whole image.
self.offset = (0, 0)
@ -83,17 +87,36 @@ class FrameSource:
"""
if self.image is not None:
height, width = self.image.shape[:2]
window_w = min(self.pan_width, width)
window_h = min(int(window_w * 9 / 16), height)
# Full height, 9:16 wide: the same window the app's replay source
# sends. A window only `pan_width` tall shows one shelf row, which
# reads as a matcher failure when it is only the crop.
window_h = height
window_w = min(int(window_h * 9 / 16), width)
max_x = max(0, width - window_w)
x = min(self.pan_x, max_x)
y = max(0, (height - window_h) // 2)
window = self.image[y:y + window_h, x:x + window_w]
self.offset = (x, y)
self.windows.append((x, y, x + window_w, y + window_h))
self.pan_x += self.pan_direction * self.pan_step
if self.pan_x >= max_x or self.pan_x <= 0:
self.pan_direction *= -1
x = int(max(0, min(self.pan_x, max_x)))
window = self.image[:, x:x + window_w]
# Scale to the send size here, so the frame this yields is exactly
# what the app sends and `encode` has nothing left to do. The pan
# step is in sent-frame pixels, so it is divided by this scale.
self.scale = min(1.0, self.width / float(window_w)) if self.width else 1.0
if self.scale < 1.0:
window = cv2.resize(
window,
(int(round(window.shape[1] * self.scale)), int(round(window.shape[0] * self.scale))),
interpolation=cv2.INTER_AREA,
)
self.offset = (x, 0)
self.windows.append((x, 0, x + window_w, window_h))
self.pan_x += self.pan_direction * self.pan_step / self.scale
# Clamp the accumulator, not just the slice origin: a negative
# `pan_x` would satisfy `<= 0` on every later frame and flip the
# direction each time, parking the window at the left edge.
if self.pan_x >= max_x:
self.pan_x = float(max_x)
self.pan_direction = -1
elif self.pan_x <= 0:
self.pan_x = 0.0
self.pan_direction = 1
return window
if self.capture is not None:
ok, frame = self.capture.read()
@ -126,7 +149,7 @@ def main() -> None:
parser.add_argument("--url", default="ws://127.0.0.1:8080/v1/stream")
parser.add_argument("--token", default=os.environ.get("DORAYAKI_API_TOKEN"))
parser.add_argument("--source", default="image.jpg", help="camera index, video path, or image path")
parser.add_argument("--pan", type=int, default=12, help="pixels per frame when panning a still image")
parser.add_argument("--pan", type=float, default=12.0, help="pixels per frame when panning a still image")
parser.add_argument("--seconds", type=float, default=20.0)
parser.add_argument("--json", action="store_true", help="print a JSON summary instead of the table")
parser.add_argument("--tracks", default=None,
@ -161,6 +184,11 @@ def main() -> None:
# --tracks so an evaluator can match labels to ground truth.
track_records: Dict[int, Dict[str, Any]] = {}
frame_offsets: Dict[int, Tuple[int, int]] = {}
# The scale `encode` applied to that frame. A box the server measured is in
# the sent frame's pixels, so mapping it back to the source needs both the
# pan offset and this factor; without it a `--width` below the source size
# shrinks every box and the evaluator matches the wrong facing.
frame_scales: Dict[int, float] = {}
label_changes = 0
unexplained_changes = 0
relabel_seen: Dict[int, int] = {}
@ -231,14 +259,15 @@ def main() -> None:
# offset comes from the frame the box was measured on, not the
# frame this result arrived with: the pan has moved on since.
offset = frame_offsets.get(track.get("label_frame_id"), (0, 0))
inv = 1.0 / frame_scales.get(track.get("label_frame_id"), 1.0)
label_box = track.get("label_box") or track.get("box") or []
if len(label_box) == 4:
track_records[track_id] = {
"track_id": track_id,
"sku_id": sku,
"label_box": [
label_box[0] + offset[0], label_box[1] + offset[1],
label_box[2] + offset[0], label_box[3] + offset[1],
label_box[0] * inv + offset[0], label_box[1] * inv + offset[1],
label_box[2] * inv + offset[0], label_box[3] * inv + offset[1],
],
"label_frame_id": track.get("label_frame_id"),
"result_frame_id": message["frame_id"],
@ -254,7 +283,6 @@ def main() -> None:
f"unconfirmed={last_summary.get('unconfirmed', 0):3d} "
f"detect={message['timing_ms']['detect']:5.1f}ms "
f"rtt={rtts[-1]:6.1f}ms fps={results / (now - started):5.2f}")
frame_id = 0
index = 0
while time.perf_counter() - started < args.seconds:
@ -264,6 +292,7 @@ def main() -> None:
index += 1
frame_id = (frame_id + 1) & 0xFFFFFFFF
frame_offsets[frame_id] = source.offset
frame_scales[frame_id] = source.scale
payload = encode(frame, args.width, args.quality)
if len(in_flight) >= args.max_inflight:
if pending_frame is not None:

View file

@ -0,0 +1,225 @@
"""Joint sweep of the two open-set gates: LOCAL_ACCEPT x TEXT_MIN_SHARED.
Both lines were fitted on image6, where every crop is a facing of a product the
shelf actually holds. That set can only measure false accepts on the products it
happens to contain, so it cannot price a line on its own - the far-unknown rate
comes from `data/external-shelves/*`: Wikimedia Commons shelf photos that share
nothing with the catalog, where every accept is a false accept by construction.
Each cell is produced by calling the shipped `DecisionPolicy` with that cell's
values and the catalog signatures, never by re-implementing the gate in the
sweep: a sweep that scores its own copy of the policy reports on a machine the
server does not run. `TEXT_MIN_SHARED` is a floor on the evidence behind the
text lane's *answer*. It is deliberately not a floor on the score - zeroing a
row reorders the ranking, so the margin gate would compare a different pair -
and deliberately not a floor on the veto, which only ever refuses. The constant
in `identity_lanes.py` says the same thing next to the code that does it.
The distractor side is measured the way the phone measures it - a 1280-long-side
frame at `LiveSettings.conf`, the path `code/eval_distractors.py` runs. The
shelf is scored at both resolutions: the full-resolution column is the one the
published table quotes, the live column is what the phone sees.
RootSIFT and OCR dominate the runtime and neither depends on the gates, so the
per-crop evidence is cached. Without `--cache` the sweep rebuilds it every run
and prints the same numbers, only slower.
.venv/bin/python code/sweep_accept_lines.py \\
--cache data/evaluations/2026-09-29-session/scratch/accept_line_cache.pkl
"""
from __future__ import annotations
import argparse
import glob
import os
import pickle
import sys
from typing import Any, Dict, List, Optional, Sequence, Tuple
current_dir = os.path.dirname(os.path.abspath(__file__))
if current_dir not in sys.path:
sys.path.insert(0, current_dir)
import cv2 # noqa: E402
from eval_identity import ( # noqa: E402
EXCLUDE,
LIVE_LONG_SIDE,
UNKNOWN,
LocalLaneAdapter,
at_live_resolution,
crop_of,
load_labels,
)
from identity_lanes import LOCAL_ACCEPT, TEXT_MIN_SHARED, DecisionPolicy # noqa: E402
SHELF_LABELS = "eval/labels/image6.json"
SHELF_IMAGE = "add/image6.jpg"
DISTRACTOR_DIRS = (
"data/external-shelves/wikimedia-commons",
"data/external-shelves/wikimedia-commons-2",
)
LOCAL_ACCEPTS = (2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0)
TEXT_MIN_SHAREDS = (2, 3, 4, 5, 6)
CACHE_VERSION = 3
# One crop's evidence: the local lane's ranking and the text lane's raw read.
Evidence = Tuple[Any, str]
# One facing: its ground-truth label plus that evidence.
Facing = Tuple[str, Any, str]
def _evidence(lane: LocalLaneAdapter, frame: Any, box: Sequence[int]) -> Optional[Evidence]:
"""Rank and read one crop, or None when the box is off the frame."""
crop = crop_of(frame, box)
if crop.size == 0:
return None
return lane.rank(crop), lane.text_lane.read(crop)
def build_cache(stock_dir: str) -> Dict[str, Any]:
"""Every crop's evidence, gathered once and swept many times."""
from engine import EngineSettings, build_default_engine
from live import LiveSettings
lane = LocalLaneAdapter(stock_dir)
image = cv2.imread(SHELF_IMAGE)
live = at_live_resolution(image)
factor = live.shape[1] / float(image.shape[1])
shelf_full: List[Facing] = []
shelf_live: List[Facing] = []
for facing in load_labels(SHELF_LABELS)["facings"]:
label = facing.get("label")
if label in (None, EXCLUDE):
continue
full = _evidence(lane, image, facing["box"])
if full is None:
continue
shelf_full.append((label, full[0], full[1]))
# A facing the live frame cuts to nothing is dropped rather than
# scored at full resolution: the two columns have to be the same crops
# at two sizes, or the resolution is not the only variable.
small = _evidence(lane, live, [int(round(v * factor)) for v in facing["box"]])
if small is not None:
shelf_live.append((label, small[0], small[1]))
engine = build_default_engine(EngineSettings(stock_dir=stock_dir))
conf = LiveSettings().conf
distractors: List[Evidence] = []
photos = 0
for directory in DISTRACTOR_DIRS:
paths = sorted(
path for path in glob.glob(os.path.join(directory, "*"))
if os.path.splitext(path)[1].lower() in (".jpg", ".jpeg", ".png")
)
for path in paths:
shelf_photo = cv2.imread(path)
if shelf_photo is None:
continue
photos += 1
frame = at_live_resolution(shelf_photo)
for detection in engine.detect(frame, conf=conf):
box = detection.get("box") or detection.get("bbox")
if not box:
continue
item = _evidence(lane, frame, [int(v) for v in box])
if item is not None:
distractors.append(item)
return {
"version": CACHE_VERSION,
"conf": conf,
"photos": photos,
"shelf_full": shelf_full,
"shelf_live": shelf_live,
"distractors": distractors,
}
def _score(lane: LocalLaneAdapter, policy: DecisionPolicy, facings: Sequence[Facing]) -> Tuple[int, int]:
"""Shown and correct labels for one shelf column under one policy."""
shown = correct = 0
for label, ranked, read in facings:
text = sorted(
lane.text_lane.scores_from_text(read).items(), key=lambda kv: kv[1], reverse=True
)
answer = policy.decide(ranked, text, text_read=read)["best_sku"]
if answer:
shown += 1
correct += answer == label
return shown, correct
def _false_accepts(lane: LocalLaneAdapter, policy: DecisionPolicy,
distractors: Sequence[Evidence]) -> int:
"""Every accept on an unrelated shelf is a false accept by construction."""
accepts = 0
for ranked, read in distractors:
text = sorted(
lane.text_lane.scores_from_text(read).items(), key=lambda kv: kv[1], reverse=True
)
if policy.decide(ranked, text, text_read=read)["best_sku"]:
accepts += 1
return accepts
def main(argv: Optional[List[str]] = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--stock", default="stock-lopia-clean")
parser.add_argument("--cache", default="", help="build once, reuse on later runs")
parser.add_argument("--rebuild", action="store_true", help="ignore an existing cache")
args = parser.parse_args(argv)
cache: Optional[Dict[str, Any]] = None
if args.cache and os.path.exists(args.cache) and not args.rebuild:
with open(args.cache, "rb") as handle:
cache = pickle.load(handle)
if cache.get("version") != CACHE_VERSION:
cache = None
if cache is None:
cache = build_cache(args.stock)
if args.cache:
os.makedirs(os.path.dirname(os.path.abspath(args.cache)), exist_ok=True)
with open(args.cache, "wb") as handle:
pickle.dump(cache, handle)
lane = LocalLaneAdapter(args.stock)
shelf_full: List[Facing] = cache["shelf_full"]
shelf_live: List[Facing] = cache["shelf_live"]
distractors: List[Evidence] = cache["distractors"]
known = sum(1 for label, _, _ in shelf_full if label != UNKNOWN)
print(f"catalog {args.stock}, {LIVE_LONG_SIDE}-long-side frames at detect_conf {cache['conf']}")
print(f"image6: {len(shelf_full)} facings, {known} holding a catalog row, "
f"{len(shelf_live)} of them cut from the live frame")
print(f"distractors: {len(distractors)} crops from {cache['photos']} unrelated shelf photos")
print()
print(f"{'accept':>6} {'floor':>5} | {'shown':>5} {'corr':>4} {'prec':>6} {'cover':>6} "
f"| {'shown':>5} {'corr':>4} {'prec':>6} | {'false':>5} {'rate':>6}")
print(f"{'':>6} {'':>5} | {'--- full resolution ---':^25} "
f"| {'---- live resolution ----':^25} | {'-- unknown shelves --':^14}")
for accept in LOCAL_ACCEPTS:
for floor in TEXT_MIN_SHAREDS:
policy = DecisionPolicy(
local_accept=accept, text_min_shared=floor,
signatures=lane.text_lane.signatures,
)
full_shown, full_correct = _score(lane, policy, shelf_full)
live_shown, live_correct = _score(lane, policy, shelf_live)
false = _false_accepts(lane, policy, distractors)
shown, correct = full_shown, full_correct
rate = false / len(distractors) * 100
shipped = " <- shipped" if (accept, floor) == (LOCAL_ACCEPT, TEXT_MIN_SHARED) else ""
print(
f"{accept:>6.1f} {floor:>5} | {shown:>5} {correct:>4} "
f"{correct / shown * 100 if shown else 0:>5.1f}% {correct / known * 100:>5.1f}% "
f"| {live_shown:>5} {live_correct:>4} "
f"{live_correct / live_shown * 100 if live_shown else 0:>5.1f}% "
f"| {false:>5} {rate:>5.2f}%{shipped}"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())

View file

@ -69,6 +69,30 @@ class RestContractTests(unittest.TestCase):
self.assertIn("version", body["model"])
self.assertEqual(body["catalog"]["sku_count"], 2)
def test_a_panned_sample_walks_and_stays_portrait(self):
# The debug replay source walks a shelf by asking for a frame index. A
# pan that returned the same window every time would freeze the stage
# while the boxes moved, which is the bug this pins.
first = self.client.get("/v1/samples/image6?pan=0")
self.assertEqual(first.status_code, 200)
self.assertEqual(first.headers["content-type"], "image/jpeg")
later = self.client.get("/v1/samples/image6?pan=40")
self.assertEqual(later.status_code, 200)
self.assertNotEqual(first.content, later.content, "a later pan must show a different window")
import cv2
import numpy as np
decoded = cv2.imdecode(np.frombuffer(later.content, dtype="uint8"), cv2.IMREAD_COLOR)
height, width = decoded.shape[:2]
self.assertEqual((width, height), (720, 1280), "the window must match the phone's portrait frame")
def test_a_sample_without_pan_is_the_whole_image(self):
response = self.client.get("/v1/samples/image6")
self.assertEqual(response.status_code, 200)
self.assertGreater(len(response.content), 1024)
def test_catalog_shape_has_no_local_paths(self):
body = self.client.get("/v1/catalog").json()
product = body["products"][0]

View file

@ -178,6 +178,63 @@ class TextLaneSignatureTests(unittest.TestCase):
"sku_l08",
)
def test_one_shared_pair_is_refused_by_the_floor_alone(self):
"""Everything but the evidence floor says answer, and it says no.
"アンース" holds one pair the catalog knows - "ース" - so the row scores
a perfect 1.0 with the whole margin over an empty runner-up, and the
read carries three pairs, so `carries_identity` passes as well. Only
`TEXT_MIN_SHARED` stands between a coincidental pair and a named
product, which is why the test asserts the score is 1.0: it pins the
floor as the only thing doing the refusing.
"""
from identity_lanes import DecisionPolicy, TextLane
lane = TextLane.from_catalog(
[{"sku_id": "sku_x", "product_name": "オイスターソース"}],
"no-such-dir",
ocr_engine=None,
)
read = "アンース"
self.assertEqual(lane.scores_from_text(read)["sku_x"], 1.0)
self.assertIsNone(
DecisionPolicy(signatures=lane.signatures).decide(
[], [("sku_x", 1.0)], text_read=read
)["best_sku"],
"one shared pair is not enough to name a row",
)
def test_a_thin_read_still_vetoes_even_though_it_cannot_name(self):
"""The floor is a floor on naming, not on contradicting.
"北海道塩" shares three bigrams with sku_b - one short of the floor -
so it cannot name that row. It is still the row the text prefers over
the local answer, and the local lane clears its own line at 5.0 against
a line of 4.0, so if the floor also sat under the veto this would come
back as sku_a: a three-bigram read would have rescued a match the
geometry earned.
"""
from identity_lanes import DecisionPolicy, TextLane
lane = TextLane.from_catalog(
[
{"sku_id": "sku_a", "product_name": "キューピー ハーフマヨネーズ 400g"},
{"sku_id": "sku_b", "product_name": "北海道塩"},
],
"no-such-dir",
ocr_engine=None,
)
read = "北海道塩"
ranked = sorted(lane.scores_from_text(read).items(),
key=lambda kv: kv[1], reverse=True)
self.assertEqual((ranked[0][0], ranked[0][1]), ("sku_b", 1.0))
self.assertIsNone(
DecisionPolicy(signatures=lane.signatures).decide(
[("sku_a", 5.0)], ranked, text_read=read
)["best_sku"],
"a read too thin to name a row still refuses the local answer",
)
class FakeDetector:
"""Records concurrency and returns fixed boxes."""

View file

@ -18,6 +18,7 @@ import cv2 # noqa: E402
import numpy as np # noqa: E402
from eval_live import ( # noqa: E402
IDENTITY_BACKENDS,
EvalInputError,
identity_level,
labelled_top1,
@ -296,5 +297,42 @@ class IdentityContractTests(unittest.TestCase):
{"identity_backend": "matcher"},
])
def test_van_is_accepted_by_the_cli_and_by_a_recording_that_names_it(self):
self.assertIn("van", IDENTITY_BACKENDS)
self.assertEqual(resolve_identity("van", None), "van")
self.assertEqual(recorded_identity([{"identity_backend": "van"}]), "van")
root = tempfile.mkdtemp()
self.addCleanup(shutil.rmtree, root, ignore_errors=True)
stock = os.path.join(root, "stock")
os.makedirs(stock)
recording = os.path.join(root, "session")
os.makedirs(recording)
write_jpeg(os.path.join(recording, "000000.jpg"), np.zeros((8, 8, 3), dtype=np.uint8))
line = {"frame": 1, "path": "000000.jpg", "tracks": [], "identity_backend": "van"}
with open(os.path.join(recording, "index.jsonl"), "w", encoding="utf-8") as handle:
handle.write(json.dumps(line) + "\n")
for extra in ([], ["--identity", "van"]):
report_path = os.path.join(root, "eval_live.json")
completed = subprocess.run(
[
sys.executable,
os.path.join(current_dir, "eval_live.py"),
recording,
"--stock-dir",
stock,
"--json",
report_path,
*extra,
],
capture_output=True,
text=True,
check=False,
)
self.assertEqual(completed.returncode, 0, completed.stdout + completed.stderr)
with open(report_path, "r", encoding="utf-8") as handle:
report = json.load(handle)
self.assertEqual(report["identity"]["backend"], "van")
self.assertEqual(report["fingerprints"]["model"]["identity"], "van")
if __name__ == "__main__":
unittest.main()

View file

@ -455,6 +455,35 @@ class LocalScaleTests(unittest.TestCase):
self.assertGreater(engine.ocr_calls, 1, "the size-only read must not be cached")
engine.shutdown()
def test_a_failing_lane_is_counted_and_logged(self):
"""A lane that raises is a failure, not a crop that matched nothing.
`_on_classified` substituted an empty answer for the exception, so a
run in which `score_crop_lanes` failed on every crop produced tracks,
zero labels and a clean log - indistinguishable from a real coverage
drop. The counter and the log line are what tell the two apart, and
the session has to keep serving frames either way.
"""
session, engine, _, _ = make_session(engine_cls=LocalScaleEngine)
def boom(crop, method=None, skip_ocr=False):
raise RuntimeError("lane unavailable")
engine.score_crop_lanes = boom
named = None
with self.assertLogs("dorayaki.live", level="ERROR") as logs:
for frame_id in range(1, 14):
tracks = process(session, frame_id)["tracks"]
self.assertTrue(tracks, "a failing lane must not take the session down")
named = named or next(
(t["identity"]["sku_id"] for t in tracks if t["identity"]["sku_id"]), None)
self.assertTrue(wait_for(lambda: bool(logs.output)),
"the exception must be logged, not swallowed")
self.assertGreaterEqual(session.snapshot()["stats"]["classify_failures"], 1,
"a lane failure must be counted in the session stats")
self.assertIsNone(named, "a failed lane must not name a product")
engine.shutdown()
class TagPassIntegrationTests(unittest.TestCase):
"""The tag pass must reach the live result without blocking frames."""

View file

@ -1,11 +1,13 @@
"""
Contract tests for the enrollment tool's reporting checks.
Contract tests for the enrollment tool's gates.
The refusal rules (size floors, session leakage, name/OCR, duplicates) are what
the tool exists for; these tests cover the two checks that report rather than
refuse, because both are properties of the catalog as a whole and neither is
visible from a single view: cross-SKU near-duplicates, and the completeness
rule that decides whether a bay is live-ready.
The refusal rules (size floors including the pyramid floor, session leakage,
name/OCR, duplicates) are what the tool exists for, and they are pinned here
against the measured distribution rather than against invented numbers. The
two checks that report rather than refuse are covered too, because both are
properties of the catalog as a whole and neither is visible from a single
view: cross-SKU near-duplicates, and the completeness rule that decides whether
a bay is live-ready.
"""
from __future__ import annotations
@ -23,11 +25,13 @@ import cv2 # noqa: E402
import numpy as np # noqa: E402
from rebuild_catalog import ( # noqa: E402
MIN_NAME_OCR,
PYRAMID_FLOOR,
RebuildError,
below_pyramid_floor,
audit,
live_ready_verdict,
load_shelf_skus,
name_ocr_similarity,
text_discriminated_siblings,
)
@ -75,8 +79,13 @@ class SiblingFlagTests(unittest.TestCase):
text_discriminated_siblings({"sku_x1": ["sku_x1.jpg", "sku_x1_2.jpg"]}, root), []
)
class PyramidFloorTests(unittest.TestCase):
"""The l09 defect: a view too small to store as the protocol's pyramid."""
"""The l09 defect: a 305 px view whose scores dwarf every other reference.
Refused, not reported. Reporting it is what let the defect ship - the view
enrolled anyway, because a line in a catalog README is not a gate.
"""
def make_source(self, height):
root = tempfile.mkdtemp(prefix="floor-")
@ -84,17 +93,112 @@ class PyramidFloorTests(unittest.TestCase):
cv2.imwrite(os.path.join(root, "sku_x1.jpg"), image)
return root
def test_a_small_view_is_reported(self):
root = self.make_source(305)
small = below_pyramid_floor({"sku_x1": ["sku_x1.jpg"]}, root)
self.assertEqual(len(small), 1, small)
self.assertEqual(small[0]["sku"], "sku_x1")
self.assertEqual(small[0]["height"], 305)
self.assertEqual(small[0]["floor"], PYRAMID_FLOOR)
def test_a_view_under_the_floor_is_refused(self):
kept, refused = audit(self.make_source(305), min_views=1)
self.assertEqual(kept, {}, refused)
self.assertEqual(len(refused), 1, refused)
self.assertIn("305px", refused[0]["reason"])
self.assertIn("pyramid floor", refused[0]["reason"])
def test_a_full_size_view_is_not_reported(self):
root = self.make_source(1000)
self.assertEqual(below_pyramid_floor({"sku_x1": ["sku_x1.jpg"]}, root), [])
def test_the_floor_is_exact(self):
"""799 refused, 800 kept: the boundary is the number, not a rounding."""
under = audit(self.make_source(PYRAMID_FLOOR - 1), min_views=1)
self.assertEqual(under[0], {}, under[1])
at = audit(self.make_source(PYRAMID_FLOOR), min_views=1)
self.assertEqual(at[0], {"sku_x1": ["sku_x1.jpg"]}, at[1])
self.assertEqual(at[1], [])
def test_a_full_size_view_still_enrolls(self):
kept, refused = audit(self.make_source(1000), min_views=1)
self.assertEqual(kept, {"sku_x1": ["sku_x1.jpg"]}, refused)
self.assertEqual(refused, [])
class NameOcrTests(unittest.TestCase):
"""The wrong-product gate: a packshot's OCR must explain its catalog row.
Pinned to the measured distribution, not to invented numbers. The reads are
what the vision backend returned for the shipped packshots and for the
rejected Kewpie 450g; recording them keeps the test on the pure function
instead of on Vision's recall, which moves with the macOS version.
"""
# Worst correct packshot in stock-lopia-clean: the l11 read carries
# "キューピー ハーフ" and "400g" but never "マヨネーズ".
WORST_CORRECT = (
"キューピー ハーフマヨネーズ 400g",
"2 キューピーハーフ New! 卵黄ならではのコクとうま味 "
"※写真はイメージです。 カロリー 50%ット 当社マヨネーズ比 400g",
0.9167,
)
# The recorded l09/l10 mix-up: Kewpie Mayonnaise 450g, filed under an
# Ajinomoto row. This is its best score against any of the 12 catalog names.
WRONG_PRODUCT = (
"キューピー からしマヨネーズ 200g",
"キューピーマヨネーズ KEWPIE MAYONNAISE ® 450g",
0.5833,
)
def test_a_correct_packshot_clears_the_gate(self):
name, text, measured = self.WORST_CORRECT
self.assertAlmostEqual(name_ocr_similarity(name, text), measured, places=3)
self.assertGreaterEqual(name_ocr_similarity(name, text), MIN_NAME_OCR)
def test_the_wrong_size_packshot_is_refused(self):
"""The check that exists to stop the l09/l10 mix-up keeps its teeth."""
name, text, measured = self.WRONG_PRODUCT
self.assertAlmostEqual(name_ocr_similarity(name, text), measured, places=3)
self.assertLess(name_ocr_similarity(name, text), MIN_NAME_OCR)
def test_the_threshold_sits_between_the_two_measured_bands(self):
"""The recalibration pin. Lower it and the wrong product enrolls."""
worst_correct = name_ocr_similarity(*self.WORST_CORRECT[:2])
best_wrong = name_ocr_similarity(*self.WRONG_PRODUCT[:2])
self.assertGreater(worst_correct, best_wrong, "the bands overlap; no threshold separates them")
self.assertGreaterEqual(MIN_NAME_OCR, best_wrong)
self.assertLessEqual(MIN_NAME_OCR, worst_correct)
self.assertGreaterEqual(MIN_NAME_OCR - best_wrong, 0.1, "no headroom below the wrong-product ceiling")
self.assertGreaterEqual(worst_correct - MIN_NAME_OCR, 0.1, "no headroom above the worst correct packshot")
def test_a_maker_prefix_gap_does_not_admit_a_wrong_row(self):
"""l07: its product tokens all read, its maker and size do not.
The catalog names l07 "丸和油脂 ラッキーマヨネーズ 200g" and the packshot
reads "ラッキー マヨネーズ 調味料(アミノ酸)不使用" - every product bigram,
none of the maker or the size. The 0.5714 it scores is inside the
wrong-product band and stays refused: the gap is a name-token problem,
not a threshold to move. Scoping the check to the product tokens would
admit it; that is a change to this gate, not a reason to lower it.
"""
score = name_ocr_similarity("丸和油脂 ラッキーマヨネーズ 200g",
"ラッキー マヨネーズ 調味料(アミノ酸)不使用")
self.assertAlmostEqual(score, 0.5714, places=3)
self.assertLess(score, MIN_NAME_OCR)
def test_audit_refuses_a_packshot_the_name_never_supports(self):
"""l04 at 0.00: the upright read is three stray characters."""
root = tempfile.mkdtemp(prefix="name-ocr-")
cv2.imwrite(os.path.join(root, "sku_x1.jpg"),
cv2.resize(artwork(), (300, 800), interpolation=cv2.INTER_NEAREST))
class FixedOcr:
name = "fixed"
def read(self, image):
class Line:
text = "明太手屋さんが つくつ た コ"
class Read:
lines = [Line()]
return Read()
kept, refused = audit(root, min_views=1,
names={"sku_x1": "やまや めんたいマヨネーズタイプ 200g"},
ocr_engine=FixedOcr())
self.assertEqual(kept, {}, refused)
self.assertEqual(len(refused), 1, refused)
self.assertIn("name/OCR", refused[0]["reason"])
class CompletenessVerdictTests(unittest.TestCase):
@ -257,7 +361,7 @@ class OrientationTests(unittest.TestCase):
root = tempfile.mkdtemp(prefix="orient-")
# Wider than tall, so the upright pass is the "wide" read (noise) and
# only a 90-degree rotation produces the "tall" read that matches.
cv2.imwrite(os.path.join(root, "sku_x1.jpg"), self.marked(300, 400))
cv2.imwrite(os.path.join(root, "sku_x1.jpg"), self.marked(800, 1000))
engine = self.FakeOcr({"wide": "つくつ|た", "tall": "キユーピー|エッグケア|305g"})
kept, refused = audit(root, min_views=1, names={"sku_x1": "キューピー エッグケア 305g"},
ocr_engine=engine)
@ -269,7 +373,7 @@ class OrientationTests(unittest.TestCase):
from rebuild_catalog import audit
root = tempfile.mkdtemp(prefix="orient-bad-")
cv2.imwrite(os.path.join(root, "sku_x1.jpg"), self.marked(400, 300))
cv2.imwrite(os.path.join(root, "sku_x1.jpg"), self.marked(800, 300))
engine = self.FakeOcr({"wide": "キューピー|マヨネーズ", "tall": "キューピー|マヨネーズ"})
kept, refused = audit(root, min_views=1, names={"sku_x1": "やまや めんたいマヨネーズタイプ 200g"},
ocr_engine=engine)

View file

@ -0,0 +1,137 @@
"""
Tests for the G1 scorer.
The gate has two halves - the new lane covers at least as much as Van, and the
lower bound of the paired-bootstrap difference clears -2 pp - and the second
half is only computable if a scored row still says which facing it matched.
These tests pin that pairing and the bound it produces.
"""
from __future__ import annotations
import os
import sys
import tempfile
import unittest
current_dir = os.path.dirname(os.path.abspath(__file__))
if current_dir not in sys.path:
sys.path.insert(0, current_dir)
from score_stream import ( # noqa: E402
G1_BOUND_MIN,
correct_facings,
paired_bootstrap,
score,
)
def facing(index, box, label="sku_a"):
return {"index": index, "box": list(box), "label": label}
def track(track_id, box, sku="sku_a"):
return {"track_id": track_id, "sku_id": sku, "label_box": list(box)}
def report(rows, windows=None):
"""A score report shaped the way `score` returns one, without the boxes."""
return {"rows": rows, "windows": windows}
def correct(track_id, matched):
return {"track_id": track_id, "predicted": "sku_a", "facing": matched,
"truth": "sku_a", "iou": 1.0, "outcome": "correct"}
def grid(count):
return [facing(i, [i * 20, 0, i * 20 + 10, 10]) for i in range(count)]
class ScoreRowTests(unittest.TestCase):
def setUp(self):
self.stock = tempfile.TemporaryDirectory()
open(os.path.join(self.stock.name, "sku_a.jpg"), "wb").close()
self.addCleanup(self.stock.cleanup)
def test_rows_carry_the_matched_facing(self):
facings = [facing(7, [0, 0, 100, 100]), facing(8, [900, 0, 1000, 100])]
rows = score([track(1, [0, 0, 100, 100]), track(2, [400, 400, 500, 500])],
facings, self.stock.name, 0.5)["rows"]
by_track = {row["track_id"]: row for row in rows}
self.assertEqual(by_track[1]["facing"], 7)
self.assertEqual(by_track[1]["outcome"], "correct")
# A track that hit nothing is unpaired, and says so instead of guessing.
self.assertIsNone(by_track[2]["facing"])
self.assertEqual(by_track[2]["outcome"], "unmatched")
def test_correct_facings_counts_a_facing_once(self):
rows = [correct(1, 7), correct(2, 7), correct(3, 9)]
self.assertEqual(correct_facings(rows), {7, 9})
class PairedBootstrapTests(unittest.TestCase):
"""27 facings is a small sample, so the pairing is what carries the gate."""
def test_identical_systems_sit_at_zero_everywhere(self):
rows = [correct(1, i) for i in range(10)]
result = paired_bootstrap(report(rows), report(rows), [])
self.assertEqual(result["difference"], 0.0)
self.assertEqual(result["lower_95"], 0.0)
self.assertEqual(result["upper_95"], 0.0)
self.assertEqual(result["van"], result["new"])
def test_a_superset_of_vans_hits_can_never_bound_below_zero(self):
# The image6 shape: the new lane gets everything Van got, plus a facing.
van = [correct(i, i) for i in range(17)]
new = van + [correct(99, 17)]
result = paired_bootstrap(report(van), report(new), grid(27))
self.assertEqual(result["facings"], 27)
self.assertEqual(result["van"], 17)
self.assertEqual(result["new"], 18)
self.assertAlmostEqual(result["difference"], 1 / 27, places=4)
# No resample of these 27 can put a facing in the new run that Van's run
# did not already have, so the 2.5th percentile is a hard zero. A bound
# printed as +0.0 pp is the floor of this pairing, not a margin.
self.assertEqual(result["lower_95"], 0.0)
self.assertGreaterEqual(result["lower_95"], G1_BOUND_MIN)
def test_a_swapped_facing_spreads_across_zero(self):
van = [correct(i, i) for i in range(17)]
new = van[:16] + [correct(99, 17)]
result = paired_bootstrap(report(van), report(new), grid(27))
self.assertEqual(result["difference"], 0.0)
self.assertLess(result["lower_95"], G1_BOUND_MIN)
self.assertGreater(result["upper_95"], 0.0)
def test_both_runs_are_graded_on_what_either_window_reached(self):
# Two facings at opposite edges; each run panned to one of them. Grading
# on the union keeps both on the same facings; the intersection would
# score each run on its own pan and the pairing would be empty.
facings = [facing(0, [0, 0, 100, 100]), facing(1, [900, 0, 1000, 100])]
left, right = [0, 0, 500, 1000], [500, 0, 1000, 1000]
result = paired_bootstrap(report([correct(1, 0)], [left]),
report([correct(2, 1)], [right]), facings)
self.assertEqual(result["facings"], 2)
self.assertEqual((result["van"], result["new"]), (1, 1))
self.assertEqual(result["difference"], 0.0)
def test_the_bound_repeats_for_the_same_seed(self):
van = [correct(i, i) for i in range(17)]
new = van[:5] + [correct(90 + i, 20 + i) for i in range(9)]
facings = grid(27)
first = paired_bootstrap(report(van), report(new), facings)
self.assertEqual(first, paired_bootstrap(report(van), report(new), facings))
def test_no_reachable_facing_scores_zero_rather_than_dividing_by_zero(self):
# Both runs panned to a corner the only facing is not in: the pairing
# set is empty, and a rate is not a number there.
elsewhere = [facing(0, [0, 0, 10, 10])]
corner = [[500, 500, 600, 600]]
result = paired_bootstrap(report([], corner), report([], corner), elsewhere)
self.assertEqual(result["facings"], 0)
self.assertEqual(result["lower_95"], 0.0)
if __name__ == "__main__":
unittest.main()

View file

@ -2,129 +2,142 @@
"dataset": "wikimedia-commons-2",
"catalog": "stock-lopia-clean",
"note": "Unrelated shelves: every accept is a false accept by construction.",
"images": 17,
"detections": 462,
"false_accepts": 1,
"false_accept_rate": 0.0022,
"config": {
"frame_long_side": 1280,
"detect_conf": 0.1,
"local_accept": 4.0,
"text_min_shared": 4
},
"images": 18,
"detections": 1076,
"false_accepts": 0,
"false_accept_rate": 0.0,
"rows": [
{
"file": "000_1920px-2011年4月の牛乳_(5588310773).jpg",
"title": "File:2011年4月の牛乳 (5588310773).jpg",
"detections": 78,
"false_accepts": 1,
"ms": 3504
"detections": 105,
"false_accepts": 0,
"ms": 3108
},
{
"file": "001_Candy_in_Japan_2005_(5256472).jpg",
"title": "File:Candy in Japan 2005 (5256472).jpg",
"detections": 12,
"detections": 41,
"false_accepts": 0,
"ms": 408
"ms": 1077
},
{
"file": "002_1920px-Empty_shelves_in_Tokyo_area_as_people_get_ready_for_t.jpg",
"title": "File:Empty shelves in Tokyo area as people get ready for typhoon Hagibis - Oct 11 2019 330pm.jpeg",
"detections": 8,
"detections": 55,
"false_accepts": 0,
"ms": 101
"ms": 637
},
{
"file": "003_1920px-HK_SW_上環_Sheung_Wan_德輔道西_88_Des_Voeux_Road_West_shop_.jpg",
"title": "File:HK SW 上環 Sheung Wan 德輔道西 88 Des Voeux Road West shop 惠康超級市場 Wellcome Supermarket goods August 2020 SS2 27.jpg",
"detections": 10,
"detections": 38,
"false_accepts": 0,
"ms": 1560
"ms": 952
},
{
"file": "005_1920px-Illuminated_wooden_shelf_with_many_glass_jars_contain.jpg",
"title": "File:Illuminated wooden shelf with many glass jars containing cookies for sale in Tokyo.jpg",
"detections": 41,
"detections": 47,
"false_accepts": 0,
"ms": 2946
"ms": 862
},
{
"file": "006_1920px-Interior_of_JUSCO_Shingu_Shopping_Center_20100213c.jpg",
"title": "File:Interior of JUSCO Shingu Shopping Center 20100213c.JPG",
"detections": 105,
"false_accepts": 0,
"ms": 1087
},
{
"file": "007_Japan_earthquake_store_shelves.jpg",
"title": "File:Japan earthquake store shelves.jpg",
"detections": 4,
"detections": 11,
"false_accepts": 0,
"ms": 298
"ms": 277
},
{
"file": "009_Milk_(2655574568).jpg",
"title": "File:Milk (2655574568).jpg",
"detections": 58,
"detections": 153,
"false_accepts": 0,
"ms": 2802
"ms": 2720
},
{
"file": "010_Natsuge-mikado_okashi.jpg",
"title": "File:Natsuge-mikado okashi.jpg",
"detections": 7,
"detections": 30,
"false_accepts": 0,
"ms": 704
"ms": 724
},
{
"file": "012_1920px-Supermarket_in_2006_(646332846).jpg",
"title": "File:Supermarket in 2006 (646332846).jpg",
"detections": 129,
"detections": 174,
"false_accepts": 0,
"ms": 6707
"ms": 1950
},
{
"file": "014_1920px-「ペヤング」予想通りの大売れ!_(18634213682).jpg",
"title": "File:「ペヤング」予想通りの大売れ! (18634213682).jpg",
"detections": 27,
"detections": 30,
"false_accepts": 0,
"ms": 762
"ms": 707
},
{
"file": "015_1920px-お客様各位_2008_国内産バター_(2429997079).jpg",
"title": "File:お客様各位 2008 国内産バター (2429997079).jpg",
"detections": 14,
"detections": 28,
"false_accepts": 0,
"ms": 1499
"ms": 1036
},
{
"file": "016_1920px-パン売り場_手動ドア式_2016_(28574754416).jpg",
"title": "File:パン売り場 手動ドア式 2016 (28574754416).jpg",
"detections": 21,
"detections": 63,
"false_accepts": 0,
"ms": 1842
"ms": 1248
},
{
"file": "017_1920px-プラレール_2015_(22339689873).jpg",
"title": "File:プラレール 2015 (22339689873).jpg",
"detections": 12,
"detections": 73,
"false_accepts": 0,
"ms": 864
"ms": 1324
},
{
"file": "018_一家族様一本限り_(5588902836).jpg",
"title": "File:一家族様一本限り (5588902836).jpg",
"detections": 27,
"detections": 41,
"false_accepts": 0,
"ms": 3880
"ms": 1208
},
{
"file": "019_1920px-洗濯洗剤_(10852819193).jpg",
"title": "File:洗濯洗剤 (10852819193).jpg",
"detections": 8,
"detections": 44,
"false_accepts": 0,
"ms": 704
"ms": 606
},
{
"file": "020_1920px-申し訳ございませんが_ごめんなさい_お魚本日の調理は終了いたしました。_どうぞよろしくお願いします_(11.jpg",
"title": "File:申し訳ございませんが ごめんなさい お魚本日の調理は終了いたしました。 どうぞよろしくお願いします (11331761984).jpg",
"detections": 6,
"detections": 25,
"false_accepts": 0,
"ms": 957
"ms": 571
},
{
"file": "021_1920px-要冷蔵_(4513315790).jpg",
"title": "File:要冷蔵 (4513315790).jpg",
"detections": 0,
"detections": 13,
"false_accepts": 0,
"ms": 190
"ms": 259
}
]
}

View file

@ -2,325 +2,338 @@
"dataset": "wikimedia-commons",
"catalog": "stock-lopia-clean",
"note": "Unrelated shelves: every accept is a false accept by construction.",
"images": 45,
"detections": 2133,
"false_accepts": 7,
"false_accept_rate": 0.0033,
"config": {
"frame_long_side": 1280,
"detect_conf": 0.1,
"local_accept": 4.0,
"text_min_shared": 4
},
"images": 46,
"detections": 3504,
"false_accepts": 3,
"false_accept_rate": 0.0009,
"rows": [
{
"file": "000_1920px-198円_中華弁当_2016_(25800617695).jpg",
"title": "File:198円 中華弁当 2016 (25800617695).jpg",
"detections": 2,
"detections": 14,
"false_accepts": 0,
"ms": 1774
"ms": 1669
},
{
"file": "001_1920px-2011年4月の牛乳_(5588310773).jpg",
"title": "File:2011年4月の牛乳 (5588310773).jpg",
"detections": 78,
"false_accepts": 1,
"ms": 4553
"detections": 105,
"false_accepts": 0,
"ms": 3225
},
{
"file": "002_1920px-20140909-0252_Costa_Mesa_Mitsuwa.JPG",
"title": "File:20140909-0252 Costa Mesa Mitsuwa.JPG",
"detections": 95,
"detections": 121,
"false_accepts": 0,
"ms": 3025
"ms": 2674
},
{
"file": "003_1920px-Aomori_Home_(4721758999).jpg",
"title": "File:Aomori Home (4721758999).jpg",
"detections": 76,
"detections": 97,
"false_accepts": 0,
"ms": 3634
"ms": 3333
},
{
"file": "004_1920px-Asakusa_15_-_making_rice_crackers_the_old_way_(155797.jpg",
"title": "File:Asakusa 15 - making rice crackers the old way (15579727827).jpg",
"detections": 14,
"detections": 40,
"false_accepts": 0,
"ms": 162
"ms": 456
},
{
"file": "005_1920px-Asakusa_16_-_making_rice_crackers_the_old_way_(155794.jpg",
"title": "File:Asakusa 16 - making rice crackers the old way (15579472088).jpg",
"detections": 3,
"detections": 30,
"false_accepts": 0,
"ms": 131
"ms": 537
},
{
"file": "006_1920px-Asakusa_17_-_making_rice_crackers_the_old_way_(155794.jpg",
"title": "File:Asakusa 17 - making rice crackers the old way (15579471588).jpg",
"detections": 4,
"detections": 30,
"false_accepts": 0,
"ms": 132
"ms": 485
},
{
"file": "007_1920px-Asakusa_19_-_the_woman_selling_the_rice_crackers_(157.jpg",
"title": "File:Asakusa 19 - the woman selling the rice crackers (15764889785).jpg",
"detections": 3,
"detections": 33,
"false_accepts": 0,
"ms": 67
"ms": 502
},
{
"file": "008_1920px-Azabudai-Hills---2025-04-05_015.jpg",
"title": "File:Azabudai-Hills---2025-04-05 015.jpg",
"detections": 37,
"detections": 54,
"false_accepts": 2,
"ms": 2113
"ms": 2525
},
{
"file": "010_1920px-Breathalyzers_in_a_MaxValu_supermarket.jpg",
"title": "File:Breathalyzers in a MaxValu supermarket.jpg",
"detections": 5,
"detections": 24,
"false_accepts": 0,
"ms": 368
"ms": 1508
},
{
"file": "011_1920px-Chocolate_in_Japan_(5137097169).jpg",
"title": "File:Chocolate in Japan (5137097169).jpg",
"detections": 65,
"false_accepts": 1,
"ms": 3829
"detections": 78,
"false_accepts": 0,
"ms": 1961
},
{
"file": "013_1920px-Confectionery_at_WCTC_Shopping_Center,_Koror_City,_Pa.jpg",
"title": "File:Confectionery at WCTC Shopping Center, Koror City, Palau.jpg",
"detections": 96,
"detections": 154,
"false_accepts": 0,
"ms": 6114
"ms": 3424
},
{
"file": "014_1920px-Deer_cookies_at_Nara,_Japan.jpg",
"title": "File:Deer cookies at Nara, Japan.jpg",
"detections": 10,
"detections": 24,
"false_accepts": 0,
"ms": 733
"ms": 465
},
{
"file": "015_1920px-ESL_Price_Tag.JPG",
"title": "File:ESL Price Tag.JPG",
"detections": 32,
"detections": 56,
"false_accepts": 0,
"ms": 1506
"ms": 1383
},
{
"file": "017_1920px-Food-with-physiological-energy-value-example.jpg",
"title": "File:Food-with-physiological-energy-value-example.jpg",
"detections": 10,
"detections": 32,
"false_accepts": 0,
"ms": 454
"ms": 1070
},
{
"file": "019_1920px-HK_CWB_Wellcome_Shop_Supermarket_goods_Carbonated_sof.jpg",
"title": "File:HK CWB Wellcome Shop Supermarket goods Carbonated soft drink codd-neck bottles 波子汽水 Apple Ramune Japanese products July 2017 IX1.jpg",
"detections": 18,
"detections": 42,
"false_accepts": 0,
"ms": 712
"ms": 1094
},
{
"file": "021_1920px-Interior_of_Supermarket_in_Japan_02.jpg",
"title": "File:Interior of Supermarket in Japan 02.jpg",
"detections": 5,
"detections": 18,
"false_accepts": 0,
"ms": 286
"ms": 491
},
{
"file": "022_1920px-JP_日本_Japan_沖繩_OKINAWA_浦添市_Urasoe-city_AEON_Supermark.jpg",
"title": "File:JP 日本 Japan 沖繩 OKINAWA 浦添市 Urasoe-city AEON Supermarket food drinks February 2026 N13p 02.jpg",
"detections": 119,
"detections": 165,
"false_accepts": 0,
"ms": 4603
"ms": 3522
},
{
"file": "023_1920px-thumbnail.jpg",
"title": "File:JP 日本 Japan 沖繩 OKINAWA 那霸 Naha Omoromachi Shintoshin 新都心 Naha Main Place 商場 Shopping Mall shop Aeon Supermarket drink 益力多 January 2025 R12S 01.jpg",
"detections": 117,
"detections": 197,
"false_accepts": 0,
"ms": 2581
"ms": 3389
},
{
"file": "024_1920px-JaSupermarket_Instant_noodles.JPG",
"title": "File:JaSupermarket Instant noodles.JPG",
"detections": 97,
"detections": 147,
"false_accepts": 0,
"ms": 2750
"ms": 2744
},
{
"file": "026_1920px-Kawasaki-Inryo-1.jpg",
"title": "File:Kawasaki-Inryo-1.jpg",
"detections": 12,
"detections": 45,
"false_accepts": 0,
"ms": 892
"ms": 2007
},
{
"file": "029_1920px-Miyazu_Mipple_1F_2021-04_ac.jpg",
"title": "File:Miyazu Mipple 1F 2021-04 ac.jpg",
"detections": 73,
"detections": 157,
"false_accepts": 0,
"ms": 852
"ms": 1782
},
{
"file": "030_1920px-Octopus_rice_crackers_No,1.JPG",
"title": "File:Octopus rice crackers No,1.JPG",
"detections": 5,
"detections": 22,
"false_accepts": 0,
"ms": 247
"ms": 556
},
{
"file": "031_1920px-Octopus_senbei.JPG",
"title": "File:Octopus senbei.JPG",
"detections": 4,
"detections": 23,
"false_accepts": 0,
"ms": 140
"ms": 612
},
{
"file": "032_1920px-Okaki_for_sale_in_Kawagoe.jpg",
"title": "File:Okaki for sale in Kawagoe.jpg",
"detections": 27,
"detections": 70,
"false_accepts": 0,
"ms": 267
"ms": 1042
},
{
"file": "033_Okonomiyaki_and_takoyaki_ingredients.jpg",
"title": "File:Okonomiyaki and takoyaki ingredients.jpg",
"detections": 39,
"detections": 55,
"false_accepts": 0,
"ms": 1228
"ms": 2553
},
{
"file": "034_1920px-Rice_toppings_&_additional_ingredients_(5846028686).jpg",
"title": "File:Rice toppings & additional ingredients (5846028686).jpg",
"detections": 59,
"false_accepts": 0,
"ms": 1871
},
{
"file": "035_1920px-SW_4963.jpg",
"title": "File:SW 4963.jpg",
"detections": 5,
"detections": 41,
"false_accepts": 0,
"ms": 200
"ms": 755
},
{
"file": "037_1920px-Soy_Sauce_selection_(6362318717).jpg",
"title": "File:Soy Sauce selection (6362318717).jpg",
"detections": 68,
"detections": 85,
"false_accepts": 0,
"ms": 1671
"ms": 1623
},
{
"file": "038_1920px-Soy_sauce_(5846027636).jpg",
"title": "File:Soy sauce (5846027636).jpg",
"detections": 103,
"false_accepts": 1,
"ms": 1626
"detections": 216,
"false_accepts": 0,
"ms": 2534
},
{
"file": "039_1920px-Soy_sauce_in_supermarket.JPG",
"title": "File:Soy sauce in supermarket.JPG",
"detections": 71,
"detections": 101,
"false_accepts": 0,
"ms": 2380
"ms": 1884
},
{
"file": "040_1920px-Spice_Supermarket_Japan.JPG",
"title": "File:Spice Supermarket Japan.JPG",
"detections": 254,
"detections": 300,
"false_accepts": 0,
"ms": 2544
"ms": 2994
},
{
"file": "041_1920px-Strong_Zero_4962.jpg",
"title": "File:Strong Zero 4962.jpg",
"detections": 18,
"detections": 45,
"false_accepts": 0,
"ms": 856
"ms": 1731
},
{
"file": "042_1920px-Supermarket_Digital_Displays_(2927051130).jpg",
"title": "File:Supermarket Digital Displays (2927051130).jpg",
"detections": 89,
"detections": 97,
"false_accepts": 1,
"ms": 3561
"ms": 2454
},
{
"file": "043_1920px-Tasty_Food_Abundance_in_Healthy_Europe.png",
"title": "File:Tasty Food Abundance in Healthy Europe.png",
"detections": 13,
"detections": 32,
"false_accepts": 0,
"ms": 1202
"ms": 982
},
{
"file": "047_1920px-Tokyo---Ebisu-and-Shibuya---2026-02-04_162.jpg",
"title": "File:Tokyo---Ebisu-and-Shibuya---2026-02-04 162.jpg",
"detections": 74,
"detections": 81,
"false_accepts": 0,
"ms": 4770
"ms": 1356
},
{
"file": "048_1920px-Tokyo---Ebisu-and-Shibuya---2026-02-04_167.jpg",
"title": "File:Tokyo---Ebisu-and-Shibuya---2026-02-04 167.jpg",
"detections": 46,
"detections": 51,
"false_accepts": 0,
"ms": 2557
"ms": 823
},
{
"file": "050_1920px-Umami_Snack_(41957534).jpeg",
"title": "File:Umami Snack (41957534).jpeg",
"detections": 33,
"detections": 55,
"false_accepts": 0,
"ms": 3096
"ms": 2518
},
{
"file": "051_1920px-このかねをならすのはだーれ_(3079692697).jpg",
"title": "File:このかねをならすのはだーれ (3079692697).jpg",
"detections": 0,
"detections": 3,
"false_accepts": 0,
"ms": 125
"ms": 170
},
{
"file": "052_1920px-スーパーの野菜コーナー_2008_(2785121488).jpg",
"title": "File:スーパーの野菜コーナー 2008 (2785121488).jpg",
"detections": 25,
"detections": 85,
"false_accepts": 0,
"ms": 1681
"ms": 1963
},
{
"file": "055_1920px-中央フード山手店_-_panoramio.jpg",
"title": "File:中央フード山手店 - panoramio.jpg",
"detections": 15,
"detections": 39,
"false_accepts": 0,
"ms": 263
"ms": 689
},
{
"file": "056_1920px-何このリポD人気⁈買い占めてしまおう。_(4025507198).jpg",
"title": "File:何このリポD人気⁈買い占めてしまおう。 (4025507198).jpg",
"detections": 38,
"detections": 72,
"false_accepts": 0,
"ms": 2912
"ms": 2286
},
{
"file": "057_1920px-味噌売り場_(16183474871).jpg",
"title": "File:味噌売り場 (16183474871).jpg",
"detections": 61,
"detections": 76,
"false_accepts": 0,
"ms": 3355
"ms": 2081
},
{
"file": "058_1920px-塩ひかえめ_(25347152904).jpg",
"title": "File:塩ひかえめ (25347152904).jpg",
"detections": 66,
"detections": 86,
"false_accepts": 0,
"ms": 3043
"ms": 2496
},
{
"file": "060_1920px-牛乳_豆乳_2016_(32081175436).jpg",
"title": "File:牛乳 豆乳 2016 (32081175436).jpg",
"detections": 103,
"false_accepts": 1,
"ms": 3677
"detections": 130,
"false_accepts": 0,
"ms": 4677
},
{
"file": "061_1920px-表示価格より100円引_(38057600585).jpg",
"title": "File:表示価格より100円引 (38057600585).jpg",
"detections": 5,
"detections": 17,
"false_accepts": 0,
"ms": 632
"ms": 899
}
]
}

View file

@ -257,7 +257,8 @@ so the client can pair a result with the frame it sent.
"by_sku": [{"sku_id": "sku_005", "facings": 3}]},
"timing_ms": {"decode": 3.1, "detect": 21.4, "track": 0.8, "total": 25.3},
"stats": {"frames_received": 264, "frames_processed": 264, "frames_dropped": 0,
"server_fps": 13.2, "classify_pending": 3, "classify_ms_p50": 88.9}}
"server_fps": 13.2, "classify_pending": 3, "classify_failures": 0,
"classify_ms_p50": 88.9}}
```
The optional crop lane is additive to protocol `1`. Without `capabilities`, the
server sends no `crop_request` and existing clients keep their frame/control flow.
@ -271,6 +272,8 @@ does not take a frame id of its own. `crop_ack` means accepted for asynchronous
classification, not recognized; subsequent results carry the updated identity and
`identity.source` (`frame`, `crop`, or `inherited`). `stats.crops` has `requested`,
`accepted`, and `rejected` counts.
`stats.classify_failures` counts classifications that raised. A non-zero value
means labels are missing for an operational reason, not because nothing matched.
The hello limit is 1 MiB of **decoded crop bytes**. The `max_crop_bytes` in
`capabilities` states the client's limit and must be an integer in 1..1048576; a

View file

@ -59,8 +59,8 @@ store-scoped, enrolled catalog, with live-video evidence accumulation**.
becomes retrieval plus verification (§6).
- **Open set is the dominant difficulty, and it comes in two kinds.** *Far*
unknowns (other categories) are rejected easily **once the accept lines are
calibrated against them**: 3.09 % at the old lines, 0.30 % at the shipped
ones, on 2650 crops from 60 unrelated Wikimedia supermarket photos, at a
calibrated against them**: 3.09 % at the old lines, 0.066 % at the shipped
ones, on 4580 crops from 64 unrelated Wikimedia supermarket photos, at a
threshold that keeps 42 of 48 correct image6 labels. The audit's 0.2-0.7 %
was measured on 602 crops from the `add/` photos with the audit's own variant
matcher; the shipped lane at the shipped lines was 3.09 % until 2026-09-29
@ -399,14 +399,17 @@ with always-argmax, run on that same catalog.
| G8 runtime | server identity latency p95 per crop; server memory | ≤ 60 ms; within the demo host's budget |
Status 2026-09-29, offline on image6 plus the Wikimedia distractor sets:
**G1 passes on the stream** (70-74 % coverage against Van's 63 % on the same
stream and catalog, 0 wrong against Van's 7). **G2 passes** (95 % precision
full, 97 % live). **G3's far-unknown half passes** (0.30 % on 2650 unrelated
crops); its near-unknown half is measured on image6 only, where the size gate
rejects the documented 450 g case. **G8 passes** (identity p95 well inside
60 ms). **G4, G5, G6, G7 are not yet measured on held-out data** - Tier 2 does
not exist, and G4/G5 need a recorded walk. Nothing here is a device-path
result.
**G1's point estimate passes on the stream** (70-74 % coverage against Van's
63 % on the same stream and catalog, 0 wrong against Van's 7), and the paired
bootstrap is now computed: the shipped lane's correct set is a strict superset
of Van's on 27 facings, so the 95 % lower bound is a hard +0.0 pp - a floor of
this pairing, not a margin, and it clears the -2 pp half. **G2 passes** (95 %
precision full, 97 % live). **G3's far-unknown half passes** (0.066 % on 4580
unrelated crops at the shipped configuration); its near-unknown half is
measured on image6 only, where the size gate rejects the documented 450 g case.
**G8 passes** (identity p95 well inside 60 ms). **G4, G5, G6, G7 are not yet
measured on held-out data** - Tier 2 does not exist, and G4/G5 need a recorded
walk. Nothing here is a device-path result.
Not complete until G1–G7 pass on the device path (Tier 3), not just offline.
@ -647,12 +650,12 @@ scores, always answered. Every "vs Van" claim in this record uses that row.
matcher 83% 79% 74% 61% 54% 33
local 83% 85% 60% 59% 93% 3
Against Van's real contract the local lane wins both axes at both resolutions:
75% at 91% against 64% at 38% (full), and 59% at 93% against 66% at 39%
(live). At live resolution Van answers more facings correctly - 66% against
59% - and is wrong on 67 of them to the local lane's 3. That is the trade the
decision makes deliberately: coverage is a gate (G1) and precision is not
negotiable, so the lane abstains where Van guesses.
Against Van's real contract the local lane wins both axes at full resolution:
75% at 91% against 64% at 38%. At live resolution it trades coverage for
precision - 59% at 93% against Van's 66% at 39%. Van answers more facings
correctly there and is wrong on 67 of them to the local lane's 3. That is the
trade the decision makes deliberately: coverage is a gate (G1) and precision is
not negotiable, so the lane abstains where Van guesses.
**One accept line serves both resolutions.** An earlier draft of this section
claimed full resolution needed a line near 6 to reach 95%, leaving 28/61
@ -674,8 +677,8 @@ audit measured, the 4.0 row is the shipped line:
6.0 59% 97% 1
The line is still provisional - it is tuned on Tier 1 and needs Tier-2
calibration - but it does not need to differ by input path, and the design
beats Van on both axes at both resolutions today.
calibration - but it does not need to differ by input path, and at full
resolution the design beats Van on both axes today.
The live count was pacing-dependent, and the cause was measurable: OCR ran on
every crop at every attempt, taking a crop from 5 ms to 25 ms and throughput
@ -738,6 +741,14 @@ Van names fewer facings than the local lane and is wrong on 7 of the 53 it
does name, plus 6 accepts of products the catalog does not hold. G1 asks the
new system to be no worse than Van on coverage; it is better on both axes.
The paired bootstrap (`code/score_stream.py --van ...`) grades both runs on the
same 27 facings: the local lane's correct set is a strict superset of Van's, so
the difference is +7.4 to +11.1 pp and the 95 % lower bound is a hard +0.0 pp.
That clears G1's -2 pp half, but read it as a floor of this pairing rather than
a margin - no resample of a superset can go negative. One of the five recorded
runs (`tracks_final.json`) is 17/27, level with Van, and does not clear the
bound; the other four do.
Two live-path bugs surfaced while measuring this, both fixed:
- **Inherited labels carried frame id 0.** `_inherit_labels` set `label_box`
@ -807,6 +818,11 @@ unanswerable before and after.
### Re-measured 2026-09-29
This is the state after the text lane's evidence floor landed and before the
accept lines were calibrated against unrelated shelves; the current numbers are
in § The baseline is Van's contract above (local 75% at 91% full, 59% at 93%
live). Kept as the dated snapshot that shows what the floor alone changed.
The text lane's evidence floor (shared bigrams between the read and the winning
row's name) and the local lane's signature pass-through landed, and the
decision ceiling was corrected: it divided a numerator that counted unenrolled
@ -823,16 +839,20 @@ facings by the enrolled count, so it read 117%.
local 83% 91% 64% 62% 93% 3
**E-3 answered.** `--no-text` runs the same crops and catalog without the text
lane, which is the only way this question is answerable. Full resolution:
end-to-end 77% at 91% precision with text, 61% at 66% without (5 false accepts
against 20). Live: 62% at 93% against 56% at 84% (3 against 7). The lane adds
16 pp and 6 pp of coverage while *raising* precision, against E-3's +5 pp bar.
It is not a veto-only lane.
lane, which is the only way this question is answerable. Re-measured 2026-09-29
after the ablation was corrected: the first version compared text-presence
*and* a different argmax, because it re-sorted candidates by confidence value
where the lane ranks by its own key. Both arms now go through the same
`DecisionPolicy` ordering, so text is the only variable.
Re-measured 2026-09-29 with the calibrated accept lines: 66% at 95% precision
with text, 55% at 81% without (2 false accepts against 8). The lane still adds
11 pp of coverage while raising precision by 14 pp, so the answer is unchanged
and the margin is wider than the audit's.
Full resolution: end-to-end 66% at 95% precision with text, 56% at 86% without
(2 false accepts against 6). Live: 56% at 97% against 48% at 100% (1 against
0). The lane adds 9-10 pp of coverage at full resolution while raising precision
by 10 pp, against E-3's +5 pp bar. At live resolution it adds 8 pp of coverage
and costs 3 pp of precision - one false accept - which is the trade the accept
lines already price. It is not a veto-only lane.
Reproduce: `code/eval_identity.py --lanes local --live [--no-text]`.
**Name-only signatures, and why.** The architecture sketch above originally
said the text signature was "name + OCR of its packshot". That was tried and
@ -907,7 +927,9 @@ Every threshold in this document was fitted on image6, where each crop is a
facing of a product the shelf actually holds. That is a closed-set tuning set,
and it hid the far-unknown rate completely: the shipped lane accepted 82 of
2650 crops from 60 Wikimedia Commons supermarket photos of milk, soy and snack
shelves - 3.09 %, against G3's 1 %.
shelves - 3.09 %, against G3's 1 %. (The crop count is the old full-resolution,
`detect_conf=0.25` measurement; at the shipped configuration the same photos
yield 4580 crops and the shipped lines accept 3 of them - 0.066 %.)
Those photos share nothing with the LOPIA catalog, so every accept is a false
accept by construction. Two mechanisms produced them:
@ -919,23 +941,33 @@ accept by construction. Two mechanisms produced them:
- **The local lane accepted pure geometry.** The residual accepts were matches
with no text overlap at all, scoring 2.26-4.81 against an accept line of 2.0.
Both lines moved: `TEXT_MIN_SHARED` 2 -> 4, `LOCAL_ACCEPT` 2.0 -> 4.0. Measured
on 2650 distractor crops and 109 image6 facings:
Both lines moved: `TEXT_MIN_SHARED` 2 -> 4, `LOCAL_ACCEPT` 2.0 -> 4.0. The
joint sweep below re-measures both axes together through the shipped
`DecisionPolicy` (`code/sweep_accept_lines.py`), on 4580 distractor crops from
64 unrelated Wikimedia shelf photos at the shipped configuration (1280-long-side
frames, `LiveSettings.conf=0.1`) and 109 image6 facings. An earlier version of
this table swept only `TEXT_MIN_SHARED` while pinning the accept line at 4.0, so
its "2.0" rows were not the old-line baseline and its "4.0 | 4" row duplicated
"2.0 | 4"; the numbers below replace it.
| `LOCAL_ACCEPT` | `TEXT_MIN_SHARED` | image6 shown / correct | precision | false accepts |
|---|---|---|---|---|
| 2.0 | 2 | 46 / 43 | 93.5 % | 82 (3.09 %) |
| 2.0 | 4 | 44 / 42 | 95.5 % | 8 (0.30 %) |
| **4.0** | **4** | **44 / 42** | **95.5 %** | **8 (0.30 %)** |
| 5.0 | 6 | 39 / 38 | 97.4 % | 3 (0.11 %) |
| `LOCAL_ACCEPT` | `TEXT_MIN_SHARED` | image6 full shown / correct | precision | coverage | image6 live precision | false accepts |
|---|---|---|---|---|---|---|
| 2.0 | 2 | 54 / 49 | 90.7 % | 76.6 % | 93.0 % | 60 (1.31 %) |
| 2.0 | 4 | 52 / 48 | 92.3 % | 75.0 % | 97.4 % | 8 (0.17 %) |
| 4.0 | 2 | 46 / 43 | 93.5 % | 67.2 % | 92.7 % | 55 (1.20 %) |
| **4.0** | **4** | **44 / 42** | **95.5 %** | **65.6 %** | **97.3 %** | **3 (0.07 %)** |
| 4.0 | 6 | 39 / 38 | 97.4 % | 59.4 % | 100 % | 0 (0.00 %) |
| 5.0 | 6 | 36 / 36 | 100 % | 56.2 % | 100 % | 0 (0.00 %) |
4.0/4 is the only region passing both G2 (>= 95 % precision) and G3 (<= 1 %
far-unknown) with coverage left. On the shipped evaluator that is 66 %
end-to-end at 95 % precision with 2 false accepts, against 75 % / 92 % / 4 at
the old 2.0 line - the coverage is what the precision costs, and G2 is the gate
that says to pay it. At live resolution the same lines give 56 % end-to-end at
97 % precision with 1 false accept, so the calibration holds on the path the
phone actually uses and not only at full resolution.
The shipped 4.0/4 row is the last one that passes G2 (>= 95 % precision) with
coverage left; 4.0/6 and 5.0/6 also pass both gates but give up 6-9 points of
coverage for precision the gate does not ask for. The old 2.0/2 line is 11
points of coverage richer and fails G3 outright at 1.31 %. On the shipped
evaluator 4.0/4 is 66 % end-to-end at 95 % precision with 2 false accepts,
against 75 % / 92 % / 4 at the old 2.0 line - the coverage is what the precision
costs, and G2 is the gate that says to pay it. At live resolution the same lines
give 56 % end-to-end at 97 % precision with 1 false accept, so the calibration
holds on the path the phone actually uses and not only at full resolution.
Reproduce: `.venv/bin/python code/eval_distractors.py --stock stock-lopia-clean
--write data/external-shelves/wikimedia-commons data/external-shelves/wikimedia-commons-2`.

View file

@ -3,73 +3,150 @@
Official manufacturer pack shots, one front view per SKU. This replaces the
views in `../stock-lopia/`, which were 71-119 px crops cut from the test shelf
`add/image6.jpg` itself: one 18 px sliver, duplicate crops, a wrong product on
`sku_l09`, and five SKUs with no view at all. Enrolling from the test shelf is
`sku_l09`, and four SKUs with no view at all. Enrolling from the test shelf is
why every dark package collapsed onto one SKU and why the identity head was
trained on its own exam.
Rebuild or extend with:
Eight of the twelve rows are enrolled; `sku_l04`, `sku_l07`, `sku_l08` and
`sku_l12` stay text-only (see *Known gaps*). Every enrolled view is at least
1000 px tall.
python code/rebuild_catalog.py --source stock-lopia-clean --output stock-lopia-clean \
--replace --min-views 1 --csv stock-lopia/stock.csv
Rebuild from the staged packshots with:
python code/rebuild_catalog.py --source data/packshots/source \
--output stock-lopia-clean --csv stock-lopia/stock.csv \
--replace --min-views 1
`--min-views 1` because official pages publish a single front view per JAN. The
default floor of 2 exists to catch accidental single-view enrollment from shelf
crops, where several views are available.
To re-run the audit without touching the catalog, point `--output` at a
scratch path inside the repo and keep the JSON:
python code/rebuild_catalog.py --source data/packshots/source \
--output data/packshots/audit-check --csv stock-lopia/stock.csv \
--min-views 1 --json data/packshots/audit-check.json
## Provenance
| SKU | Product | Source | Image |
|---|---|---|---|
| sku_l01 | キユーピー エッグケア 305g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577090301/) | `4901577090301L.jpg` |
| sku_l02 | キユーピー 血圧ケア アマニ油マヨネーズ 200g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577060052/) | `4901577060052L.jpg` |
| sku_l03 | キユーピー からしマヨネーズ 200g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577338779/) | `4901577338779L.jpg` |
| sku_l05 | キユーピー ゼロ ノンコレステロール 310g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577092381/) | `4901577092381L.jpg` |
| sku_l06 | キユーピー 内臓脂肪ケア フィッテ 210g | [kewpie.co.jp](https://www.kewpie.co.jp/products/close/) (discontinued 2026秋) | `4901577083600L.jpg` |
| sku_l09 | ピュアセレクト コクうま カロリーカット65% 360g | [ajinomoto.co.jp](https://www.ajinomoto.co.jp/products/detail/?ProductName=pureselect_2) | `A140600321` |
| sku_l11 | キユーピー ハーフ 400g | [kewpie.co.jp](https://www.kewpie.co.jp/products/detail/4901577031083/) | `4901577031083L.jpg` |
Product names are the catalog's, verbatim from `stock.csv`; sizes are measured.
Kewpie images follow `https://img.kewpie.co.jp/products_src/g3/imgs/<JAN13>L.jpg`;
Ajinomoto images follow
`https://www.ajinomoto.co.jp/products/Image.aspx?PicType=0&PIMNO=<PIMNO>`.
| SKU | Catalog name | View | Recorded source |
|---|---|---|---|
| sku_l01 | キューピー エッグケア 305g | 455x1000 | not recorded |
| sku_l02 | キューピー アマニ油マヨネーズ 200g | 468x1000 | not recorded |
| sku_l03 | キューピー からしマヨネーズ 200g | 450x1000 | not recorded |
| sku_l05 | キューピー ゼロ ノンコレステロール 310g | 461x1000 | not recorded |
| sku_l06 | キューピー フィッテ 210g | 654x1500 | [amazon.co.jp](https://www.amazon.co.jp/dp/B08V2676LK) |
| sku_l09 | ピュアセレクト コクうま カロリーカット65% 360g | 1000x1000 | not recorded |
| sku_l10 | ピュアセレクト マヨネーズ 400g | 1000x1000 | [rakuten.co.jp](https://item.rakuten.co.jp/matsukiyo/4901001114030/) (JAN 4901001114030) |
| sku_l11 | キューピー ハーフマヨネーズ 400g | 482x1000 | not recorded |
`data/packshots/source/ATTRIBUTION.json` records a URL for `sku_l06` and
`sku_l10` only. Its `_provenance_gap` note names the other six - l01, l02, l03,
l05, l09, l11 - as enrolled before any provenance was tracked, and the audit
reports them under `unattributed_skus` on every run. What *is* verified about
them, per that note: each is independent of the evaluation photo (template
match against `add/image6.jpg` peaks at 0.166, against 0.95 for a crop of it),
each is 1000 px or taller, and each one's own OCR matches its catalog row (see
*Name/OCR gate*). That is a traceability gap, not a correctness one.
`sku_l06` is the current package (`内臓脂肪を減らす`), which replaced the
superseded `内臓脂肪ケア` view on 2026-09-29. The catalog name was shortened to
match; the shelf tags were not part of that change.
## Name/OCR gate
A packshot's OCR has to explain its own catalog row's name, or it is refused.
Measured over the shipped packshots - every image read in all four rotations,
every read scored against all 12 catalog names:
| Class | n | Score |
|---|---|---|
| Correct packshot vs its own row | 8 | 0.9167 - 1.0000 |
| Wrong product vs any row | 1 image x 12 names | 0.0476 - 0.5833 |
`MIN_NAME_OCR = 0.75` is the midpoint of those two bands: 0.167 of headroom
above the worst correct packshot (l11, whose read omits `マヨネーズ`) and 0.167
below the wrong-product ceiling. The wrong-product image is
`references/rejected/kewpie-mayonnaise-450g.jpg`; its best score anywhere is
0.5833, tied against `キューピー からしマヨネーズ 200g` and
`キューピー ハーフマヨネーズ 400g`, and it bottoms out at 0.0476 against
`ピュアセレクト コクうま カロリーカット65% 360g`.
Reproduce the band with (run from the recognition root; unindented so it runs
verbatim):
```python
import csv, os, sys, cv2
sys.path.insert(0, 'code')
from ocr import build_ocr_engine
from rebuild_catalog import best_orientation_text, name_ocr_similarity
clean = 'stock-lopia-clean'
names = {r['SKU_ID']: r['product_name'] for r in csv.DictReader(open(clean + '/stock.csv'))}
engine = build_ocr_engine(None)
for n in sorted(os.listdir(clean)):
if n.endswith('.jpg'):
sku = n[:-4]
read = best_orientation_text(engine, cv2.imread(os.path.join(clean, n)), names[sku])
print(sku, round(name_ocr_similarity(names[sku], read), 4))
rej = 'references/rejected/kewpie-mayonnaise-450g.jpg'
read = best_orientation_text(engine, cv2.imread(os.path.join(clean, rej)), names['sku_l01'])
print('rejected max', max((round(name_ocr_similarity(v, read), 4), k) for k, v in names.items()))
```
The gate cannot separate siblings, and does not try: the `sku_l09` packshot
scores 0.8462 against the `sku_l10` name, because both are Ajinomoto
ピュアセレクト mayonnaise. That case is what `text_discriminated_siblings` flags
(l09 ~ l10, cosine 0.9817) - it is a cross-SKU geometry problem, not a gap a
threshold closes.
## Pyramid floor
Every enrolled view must be at least 800 px tall, or it is refused. This is a
hard gate, not a report. It exists because of `sku_l09`: the view enrolled at
305 px could not be stored as a 200/400/800 px pyramid, so its local lane
upscaled 305->400 px and scored 85-129 inliers against 10-40 for every other
reference, winning facings it should not. It was re-sourced at 1000x1000 on
2026-09-29. Reporting the defect instead of refusing it is what let it ship.
`data/packshots/source/sku_l07.jpg` is 600x600 and is refused on this rule
before OCR even runs.
## Known gaps
Five SKUs in `../stock-lopia/stock.csv` have no official pack shot here, so no
image lane can match them. They stay in `stock.csv` as text-only rows: the text
lane builds a signature from the printed name, which is how it can name or veto
a near-unknown the matcher cannot see. They are the current image-coverage
ceiling, not a recognition failure:
Four rows in `../stock-lopia/stock.csv` have no enrolled image, so no image
lane can match them. They stay in `stock.csv` as text-only rows: the text lane
builds a signature from the printed name, which is how it can name or veto a
near-unknown the matcher cannot see. They are the current image-coverage
ceiling, not a recognition failure.
| SKU | Catalog name | Why it is missing |
| SKU | Catalog name | Why there is no view |
|---|---|---|
| sku_l04 | やまや めんたいマヨネーズタイプ 200g | Yamaya's current lineup has no 200 g mentaiko mayo |
| sku_l07 | 丸和油脂 ラッキーマヨネーズ 200g | Maruwa's current catalog has no 200 g Lucky Mayonnaise |
| sku_l04 | やまや めんたいマヨネーズタイプ 200g | A packshot is staged and refused at name/OCR 0.00: the image reads only `明太手屋さんが つくつ た コ`, brand text that runs vertically. A correctly oriented or higher-resolution shot is what would clear it. |
| sku_l07 | 丸和油脂 ラッキーマヨネーズ 200g | A packshot is staged and refused twice: it is 600x600, under the pyramid floor, and scores name/OCR 0.5714. Every product bigram reads (`ラッキー マヨネーズ`); the six it misses are the maker prefix `丸和`/`和油`/`油脂` and the size `20`/`0G`/`00`, which the package does not print. |
| sku_l08 | スパイシーストーム ペリペリミックス 30g | No official 30 g Peri-Peri Mix found |
| sku_l10 | ピュアセレクト マヨネーズ 450g | Ajinomoto's current lineup is 400/250/600 g/1 kg; no 450 g |
| sku_l12 | 焼肉のたれ 中辛 | Brand not recorded in the catalog; ambiguous |
| sku_l12 | ユーラス 野菜をおいしく食べる旨たれ 380g | No pack shot sourced |
The catalog names for these five disagree with the manufacturers' current
lineups, so the shelf tags and the CSV need reconciling before enrollment.
`sku_l10` left this table on 2026-09-29. Its catalog name said 450 g, which
Ajinomoto does not make; the shelf tag in `add/image6.jpg` and the
manufacturer's lineup both say 400 g, so `stock.csv` was corrected and a
1000x1000 packshot (JAN 4901001114030) enrolled. See
`data/packshots/source/ATTRIBUTION.json`.
Keep the text-only rows when rebuilding: the CSV carry-over must preserve every
source row, not just the SKUs with a kept view. Dropping these five costs the
source row, not just the SKUs with a kept view. Dropping these four costs the
text lane the signatures that cover them.
`sku_l09.jpg` is enrolled but only 305 px tall, below the 800 px source floor
the enrollment protocol wants (§8 of the architecture decision). Its matcher
scores dwarf the other references (85-129 inliers against 10-40), so it wins
facings it should not; re-source it before trusting l09 precision. Measured on
the corrected image6 labels, l09 is the largest single-SKU share of accepts at
both resolutions: 18 of 48 accepts (37.5%) at full, 11 of 38 (28.9%) at live,
against a true share of 13 of 61 facings (21.3%). That is a hubness ratio of
1.76 and 1.36 against the completion gate's limit of 2.0, so G7 passes; the
stricter internal 30% share heuristic is what raises the alarm at full
resolution. Re-sourcing the reference is the fix, not a threshold change.
## Rejected
`references/rejected/kewpie-mayonnaise-450g.jpg` is Kewpie Mayonnaise 450g (JAN
4901577042072). It was briefly filed as `sku_l09` and then as `sku_l10` because
the catalog's names pointed there, but it is neither product: l09 is Ajinomoto
Pure Select コクうま カロリーカット 65% 360g and l10 is Pure Select マヨネーズ
450g. It is kept as provenance for that mix-up and is never enrolled.
4901577042072). It was filed as `sku_l09` and then as `sku_l10` because the
catalog's names pointed there, but it is neither: l09 is Ajinomoto Pure Select
コクうま カロリーカット 65% 360g and l10 is Pure Select マヨネーズ 400g. Both now
hold their own correct views. The rejected image is kept as provenance for that
mix-up and is never enrolled; the name/OCR gate scores it at most 0.5833
against any catalog name, below the 0.75 threshold.