The audit's own numbers were right; several claims built on them were not, and
the advisors found each one:
- 'wins both axes at both resolutions' - false at live, where the lane trades
59% coverage for 93% precision against Van's 66% at 39%. Stated as the trade
it is.
- G1's bootstrap now exists: the local lane's correct set is a strict superset
of Van's on the stream, so the 95% lower bound is a hard +0.0 pp - a floor of
that pairing, not a margin. tracks_final.json is level with Van and does not
clear it.
- The far-unknown rate is 0.066% on 4580 crops at the shipped configuration,
not 0.30% on 2650 at the old one.
- The E-3 ablation compared text-presence *and* a different argmax; both arms
now go through the same DecisionPolicy ordering. Text adds 9-10 pp of
coverage at full resolution while raising precision.
- The sweep table swept one axis while pinning the other; replaced with the
joint sweep.
- The two Re-measured tables are dated snapshots, labeled as such.
An earlier edit dropped the paragraph that introduces the section, leaving the
heading followed straight by a sentence starting 'and it hid...'. Restored, and
the document now has no heading without a blank line before it.
The pacing claim rested on single runs. Medians of three 12 s runs per pace,
through the shipped server: send 67% coverage / 88% precision, result 63% / 86%,
0 wrong in all six. The two paces are within 4 pp, so the sensitivity the audit
measured is gone and what is left is run-to-run noise.
The architecture sketch said the text signature was 'name + OCR of its
packshot'. The code scores the name alone, deliberately: adding the packshot's
own OCR is how l09 used to win l10's facings, because l09's packshot prints
l10's actual name. The sketch now says what the code does and why.
E-3 with the calibrated lines: 66% at 95% precision with the text lane, 55% at
81% without, 2 false accepts against 8. The lane adds 11 pp of coverage while
raising precision 14 pp, so the answer stands and the margin is wider than the
audit's.
The labels README pointed at evidence/image6_v2.json, which was a byte-identical
duplicate of eval/labels/image6.json and is now removed; the README says so
instead of describing a copy that no longer exists.
The header said 'Audited 2026-09-28' while the document now carries 2026-09-29
implementation results; it says both, and that audit-time figures are kept
beside the later ones rather than replaced.
The OCR cost row claimed 18.0 ms on 40 production crops with no committed
script behind it. code/bench_ocr.py is that measurement: 23.3 ms p50, 37.2 ms
p95, mean 23.1 - inside the recorded 18-26 ms band, at its upper half.
The G1 row compared against a van run whose server had loaded the legacy
stock-lopia catalog, so its ids were template-level (sku_005) and every answer
scored wrong. Re-run with both servers on stock-lopia-clean, 20 s pan 12:
van 17/27 (63%) coverage 53% precision 7 wrong 6 unenrolled accepts
local 19-20/27 (70-74%) 77-85% 0 wrong 0 unenrolled accepts
G1 asks the new system to be no worse than Van on coverage; it is better on
both axes, so G1 now reads as passing on the stream rather than unmeasured.
Both recordings are committed under stream-gated/.
The stream row was recorded before the accept lines moved. Three fresh 20 s pan
replays through the shipped lane: 70-74% coverage at 77-85% precision, 0 wrong
labels in all three (0-1 before), 8-14 unknown false accepts - the near-unknown
case, which the size gate and catalog completeness own rather than the accept
line.
Live resolution on the shipped evaluator: 56% end-to-end at 97% precision with
1 false accept, so the calibration is not a full-resolution artifact.
The recordings are committed under stream-gated/ so the row is reproducible.
The Wikimedia collector truncated the whole basename at 60 characters, so 13
files lost their extension and cv2 could not read them by name - the distractor
sets were quietly scoring 13 images fewer than they held. It now truncates the
stem and keeps the extension, and the files on disk are renamed to match.
The search terms also pull in .djvu volumes whose page 1 Commons serves as a
JPEG thumbnail. Twelve book scans were in the sets; they contributed 0 false
accepts, and they are not shelves, so the collector filters them and the
manifests no longer list them.
With all 2650 crops readable the sweep is re-measured end to end: 3.09% at the
old lines, 0.30% at the shipped 4.0/4, image6 44 shown / 42 correct / 95.5%.
The accept sweep table and the verdict's far-unknown sentence now carry those
numbers instead of the pre-calibration ones.
Also: the fetched images are gitignored (manifests and results are not), the
l10/l11 shelf-tag crops that back the catalog corrections are committed, and a
stale pre-gate eval.json and a duplicate of eval/labels/image6.json are gone.
The far-unknown claim in the verdict said 0.2-0.7% on 602 crops. That was the
audit's variant matcher on the add/ photos; the shipped lane at the shipped
lines was 3.61% on the Wikimedia sets. Both are now stated, with the sweep that
chose 4.0/4 and the live-endpoint result: 44 named, 42 correct, 95.5% precision
against 53/48/90.6% before.
Gate status is recorded per gate rather than as a blanket claim: G2 and G3's
far half pass offline, G8 passes, and G1/G4/G5/G6/G7 are not yet measured on
held-out data because Tier 2 does not exist.
The Wikimedia manifests and results are committed; the 59 MB of images are not,
because collect_shelf_images.py fetches them from the source URLs the manifests
record.
The F row's veto numbers were measured on v1 labels, which the audit itself
found to be 23% wrong. They are kept as the record of what was found, marked as
audit-time, and pointed at the current measurement rather than left to read as
present-day results. The OCR cost figure was re-measured: 18.0 ms/crop on 40
production-shaped crops, inside the recorded band.
The ceilings table, the stale-reference line and the metadata-integrity
paragraph all described the audit-time state as current. Each now says what was
found and what changed: the catalog ceiling is 53/64 = 83% (l10 and l06
enrolled, l12's facings labelled), the l06 package and l09 view are replaced,
and every stock.csv defect named there is corrected.
The OCR cost figure was re-measured rather than assumed: 18.0 ms/crop on 40
production-shaped crops, inside the recorded 18-26 ms band.
E-3 asks what the text lane is worth, and the record answered it with numbers
that no committed command could produce - the evaluator had no way to run the
local lane without the text lane. --no-text is that way, and it keeps the
comparison to one variable by building the lane either way, so the OCR cost is
paid in both arms.
Measured: full resolution 77% end-to-end at 91% precision with text against 61%
at 66% without (5 false accepts against 20); live 62% at 93% against 56% at 84%
(3 against 7). The lane adds 16 pp and 6 pp of coverage while raising precision,
against E-3's +5 pp bar.
Also refreshes the re-measured block above it, which still printed the
pre-correction 61/48 label counts and a stray 'HUB!' marker.
19 scored rather than 20: sku_l06_1.jpg is now the enrolled l06 view, so it is
reported as leaked instead of as a held-out correct accept. 12 correct,
7 abstain, 0 false accepts.
Both leak checks are named in the record and in the external README, since the
enrollment that closed F-12 is exactly the kind of change that silently
converts a held-out row into training data.
Twenty independent-source images across eight SKUs, leak-tested (max template
match against add/image6.jpg is 0.675) and scored through the shipped lane:
13 correct, 7 abstain, 0 false accepts. The seven abstentions are all
unenrolled or weakly-read SKUs.
The result worth keeping is sku_l07: no enrolled image, both images accepted on
text alone. That is the text lane's coverage claim holding on data it had no
part in building.
F-12 is marked fixed with its measured effect, and the packshot-OCR check that
would have caught it is recorded as implemented but not yet trustworthy - it is
a false negative on l04 and l07.
Five blocks still printed pre-correction figures. Each was re-run through the
evaluator and replaced with what it actually outputs now:
- The lane comparison: local 75% at 91% against Van 64% at 38% (full), and
59% at 93% against 66% at 39% (live). At live resolution Van now answers more
facings correctly and is wrong on 67 of them to the local lane's 3 - stated
plainly rather than left as a win on both axes.
- The accept sweep: 75/67/67/62% end-to-end at 91/91/93/95% precision, with the
evaluator's own column definitions spelled out, since my first re-derivation
used the wrong denominator.
- The G1 stream claim: four pan replays give 63-70% coverage at 76-84%
precision, not 75% at 100%.
- The upscale paragraph: the guard is intact, but sku_l09.jpg is now a
1000x1000 packshot, so the current run no longer exercises it.
- catalog_hash and the pyramid floor: the hash covers view bytes, not just
names, and every view is now above the 800 px floor.
Also refreshes the server smoke against the renamed catalog: 53 named in 2.58 s,
no view-suffixed ids, no missing product info. l12 is not among them - it has
no enrolled view, which is the documented gap.
The reproduction block still printed the pre-enrollment numbers (catalog 69%,
decision 86%); it now prints what the command actually outputs today, with the
catalog figure's fall from 87% to 83% explained rather than left to look like a
regression.
Adds the three row corrections with their evidence, the near-unknown fix and
its measured effect, and the state of the four unenrolled SKUs - including that
l08's row name and size are themselves unverified, since no retailer listing
for a 30g スパイシーストーム ペリペリミックス exists.
cat_full's *_shelf.jpg views are crops of add/image6.jpg itself - template
matching puts each one back at 0.998-0.999 confidence - so committing them
reintroduced the evaluation photo as a catalog, which is the leak the audit
removed from stock-lopia. cat_pack is the same views without the shelf copies.
Both are regenerated by lfo_setup.py from stock-lopia and the labels, so the
evidence stays reproducible without the repository carrying the leak. Verified
after removal: lfo_setup.py rebuilds all 16 views.
E-3: the text lane adds 31 pp coverage at full resolution and 23 pp at live
while raising precision, against a +5 pp bar. E-2: LocalLane is RootSIFT-only,
so ORB/BRISK have nothing left to drop from the shipped path.
`classify_boxes` returned only the identity, so every accepted still-image
answer reached the client as a bare SKU id with no name or price. Measured
through the real server on add/image6.jpg: 41 of 41 accepted detections had no
`product`. The live path attached the row; the still path did not, which is
the F-02 finding the decision document recorded as still live in production.
The row is attached where the decision is made, so both paths carry it, and
`recognize` passes it through to the wire. Verified through the running server:
41 of 41 accepted answers now carry product_name, price, and stock counts.
Also corrects the decision document's full-resolution claim, which was
measured before the reference-upscaling and text-signature fixes: it said 77%
precision with 11 false accepts and a hub alarm, where the current lane reads
88% with 5 and no alarm. The accept line does not need to differ by input path
- correct facings sit at p25 2.07 (full) and 2.87 (live), wrong ones at p75
1.00 and 1.66, so 2.0 is the boundary in both - and the results table and the
stream G1 numbers are refreshed to the current measurements.
`catalog_hash` hashed the CSV and the view *names*, so replacing
`sku_l09.jpg` with a different product's packshot left the hash unchanged and
a report could claim the same catalog while every l09 decision in it had
moved. It hashes each view's bytes now.
`identity_lanes` carried its own `normalize_text` and `bigrams`, near-copies
of the ones in `rebuild_catalog` that had already drifted (the lane's table
folded ユ to itself, the catalog's did not). The lane imports the catalog's
now, so one read scores the same either way.
`evidence/image6_v2.json` was byte-identical to `eval/labels/image6.json`; the
evidence scripts and the decision document point at the canonical file and the
copy is gone. `score_crop`'s docstring now states the contract its callers
already depend on: the dict's insertion order is the lane's ranking, and
re-sorting by value names a different SKU than the lane chose.
Tests: the lane is shared by the classify pool, so a concurrent-ranking test
pins that one BFMatcher does not cross crops under eight threads.
The pacing paragraph described a defect that no longer exists. Replaced with
the measured before/after, and added the two review findings (the missing
always-argmax baseline lane, the size-token fill-in) with their test counts.
Two review findings, both confirmed by measurement.
1. The evaluator had no always-argmax lane, so every "vs Van" claim compared
against the *gated* matcher, which is not Van's contract - the original demo
answered every crop with its best-scoring SKU and had no accept line. The new
'van' lane is that contract: same scores, no gate. Against it the local lane
wins both axes at both resolutions (full 61%/77% vs 52%/29%; live 59%/95% vs
57%/32%), but the live coverage margin is one facing, not seven, and the 95%
figure is live-resolution only - at full resolution the same lane reads 77%
with 11 false accepts and a hub alarm.
2. A crop reading only "400g" scored 0.667 on three SKUs, above TEXT_ACCEPT, so
the text lane named a product from a size line. The fill-in job now requires
two non-digit bigrams; the veto is unchanged, so "65%" still contradicts a
local answer. The raw OCR read travels with the ranking through
score_crop_lanes for that check.
Tests: 3 for the size gate (policy + live) and the van lane's own row. Removing
the gate fails all three. Suite 276 -> 280.
The live label count is a function of frame pacing, not a fixed property: a
track needs two attempts to settle, so pushing frames faster than the classifier
drains them reports fewer labels (12/8/2/6 at full speed on the same clip, 14
paced, 38 of 30 frames). Precision stays 100% and 0 unknowns are accepted at
every pacing. Also records the local-scale and DecisionPolicy tests.
The record said '2 false accepts of 48 unknowns', which conflated two fields:
the 2 are 0 unknown accepts plus 1 unenrolled in-catalog facing and 1 enrolled
misidentification. Also restores the hybrid-verify line the lane vocabulary
edit dropped, and reframes the l09 note: its hubness ratio is 1.76 full /
1.36 live against G7's limit of 2.0, so the gate passes and the 30% internal
share heuristic is what raises the alarm.
Replaces the DINO gallery as the recognition path with the measured design
from docs/research/2026-09-28-recognition-architecture/DECISION.md: RootSIFT
over a per-SKU reference pyramid, similarity-constrained RANSAC and a hue
check, fused with an OCR text lane that vetoes a match whose package text
names a different catalog row and answers on its own when geometry finds
nothing.
- identity_lanes.py: LocalLane, TextLane, DecisionPolicy. The text lane
builds each row's signature from the catalog name plus its packshot OCR, so
a row with no enrolled image is still answerable.
- engine.py: every backend groups scores by canonical SKU, so no
view-suffixed id reaches the wire and a name and price always resolve.
decide_crop fuses the lanes; score_crop_lanes feeds the live path. Both
lanes are built in warm_up (non-fatally) so the first frame is not the slow
one, and the text lane shares the engine's OCR lock.
- live.py: decides per crop on the lane's own rank order. The lane's order
comes from its rank key and its value is only the gate, so re-sorting by
value named a different SKU than the lane chose; averaging in an empty
observation also diluted a good confidence below the accept line.
- identity.py: a `local` score scale whose gate is one absolute line, because
the value is already a margin over the runner-up.
- app.py, run-demo-server.sh: `--identity local` is the default; the demo
server no longer starts on the leaked `stock-lopia` catalog.
- eval_live.py: the replay vocabulary accepts `local`, so a recording from
the shipped server can be scored.
- README.md: documents the three backends and what each is for.
Measured on the corrected image6 labels at live resolution: 36/61 in-catalog
facings named at 95% precision, 0 of 48 unknowns accepted, 11 labels sourced
from the text lane alone. Through the real server: /v1/recognize 90%
precision, /v1/stream 100% precision, no view-suffixed ids on either.
