Three findings from the deep review, all about the tag channel claiming more than it
can support.
1. The exact signal is a barcode. The reconciliation compared `tag.jan` against the
catalog's JAN, but that field held the regular price - a number compared against
a product code. It now reads a `barcode` field (populated only by a close-up
decode) and the printed number is compared as a price.
2. A price-mismatch violation required "an anchor", which was `tag.jan` - a price.
The anchor is now the tag's own arithmetic agreeing, or a name that actually
resembles the product, and the name signal carries its score as a field instead
of being parsed back out of a human-readable note string.
3. Tag identity could reach anywhere along a row: a facing whose own tag was not
detected inherited whichever readable tag was nearest, which can be a different
product several columns away. Assignment is now capped at the product's own run
(2.5 facing widths, at least 200 px), and a tag may only overrule the packaging
channel when it is corroborated or its name matches at 0.6 rather than 0.4.
162 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Stage 2 of the plan is "identity as retrieval", so the first thing measured was
whether retrieval can even work here. It cannot, on its own: an off-the-shelf
DINOv2 embedding does group identical products (two orange juices, two apple
juices, two almond drinks) but the fine-grained cases collapse - 32 cartons into
one cluster, and nine different bottled drinks grouped by *container shape*. That
matches the literature (RP2K, RPC all fine-tune; nobody ships off-the-shelf
features for SKU identity).
Meanwhile the shelf has been carrying the answer. Every product has a tag printed
under it, and the tag spells the product's name out in clean text, once per
product - the store's own label. This adds that as an identity channel:
- a tag with a readable name is matched against the catalog by name similarity;
- facings are assigned to the *nearest tag along their row* (the tags mark where
products sit), not by geometric grouping, which was measured to merge different
products into a 945 px "product" that then swallowed the wrong tag;
- rows are clustered first, because the same column on six shelves would
otherwise become one product.
Then the demo catalog itself had to be fixed: it listed another store's products
(senbei, Tropicana) while the shelf holds mayonnaise, so nothing could match. The
catalog for this shelf was read off the tags - 12 products with names and prices -
and the products were enrolled as packaging templates by cropping the facings the
tags had just named.
Measured through the real still path on add/image6.jpg (111 detections):
tag named the product 28 facings (all 28 verified correct by eye)
packaging named the product 74 facings
both spoke 23 facings, agreeing 16 (69 %)
facings with any identity 79
So packaging recognition makes a ~30 % error rate on the facings the tag can
check, and every disagreement carried a low packaging score. The tag therefore
wins a conflict, and the conflict is recorded (`identity_conflict`) instead of
hidden - a product whose tag says one thing and whose packaging says another is a
real finding, not noise to suppress.
144 tests pass; test_tag_identity.py adds 6.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Two false positives came out of running the rules against a real shelf, and both
had the same shape: the rules were arguing with our own detector instead of with
the shelf.
1. One tag per product, not per facing. Measured: a row carries 16-18 facings but
only 4-5 tags, because four facings of a mayonnaise share a single tag. Counting
facings invents missing tags on a shelf that is perfectly labelled. The rules
now group a row's facings into products - by identity when the facings were
recognised, by size and spacing otherwise - and count products.
2. A product's tag may sit anywhere along its run. Measured: a run of
ピュアセレクト facings was split by geometry because one facing was detected
narrower than its neighbours (54 px beside 150 px), and the fragment was then
reported as missing the tag that was sitting further along its own run. A
product now counts as missing only when no tag exists for its SKU anywhere in
the row, and a fragment far narrower than the row's typical product is not
judged at all.
3. A mismatch needs one trustworthy anchor. Measured noise: a wrong identity plus
a garbled tag read (name match 0.11, no JAN) produced a confident "tag says 90,
catalog says 188" that was really two bad reads disagreeing. Without the tag's
JAN or a name that resembles the product, that is now a hint, not a violation.
Measured on add/image6.jpg through the real still path: 0 false violations without
identities (was 1), and with identities the only finding left is a genuine
tag-versus-catalog price difference. Coverage hints now read "tags detected for 2
of 4 products in this row". 143 tests pass.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Web client: price-tag boxes in a distinct style (teal, chip with the parsed
price), an OCR section in the product card showing the tag's price, name,
discount and the reconciliation verdict with both values, and a compliance strip
under the stage. The photo stage now draws the image into the canvas and uses the
same containment as the media, so boxes cannot drift from the product (they did:
the canvas stretched to the stage while the photo was contained).
Server fixes the browser runs forced out:
- Photo mode reconciled before identities existed, so every tag read "tag only".
The tag bundle is cached with the still and classify re-reconciles against the
identities it just produced, returning them per detection.
- The still path only ever read the first four bands (plan hands out two per
call); it now drains every band, twice at different upscales so the vote has
independent reads.
- Voting is confidence-weighted and a dead heat stays unread; the plausible
shelf-price range is 10-9999, which stops a misread from becoming a ¥26,300
price. The compliance coverage bar moved to 0.7 after 0.5 produced a false
"group without price tag" on a shelf where every facing has one.
Measured on add/image6.jpg: 8 readable tags with names and prices, 942 ms for
the whole still, 7 real price mismatches, no false violations, and hints that
say plainly when a row could not be read.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The engine owns the OCR backend, a shared OCR lock and a dedicated tag worker, so
a tag pass can never occupy a classification worker or slow a frame. A live
session plans rail bands on the frame loop (cheap), runs them on that worker, and
attaches whatever the cache already knows; a still runs one synchronous pass.
Tracks and detections now carry price_tag and reconciliation, and the summary
carries price_tags plus compliance. Config can turn tags off per session.
Compliance now fails closed on evidence quality. The first real run produced 106
"missing tag" violations on a shelf where only 8 tags were readable - our tag
*detection* was incomplete, not the shelf. A row is only judged when at least
half its facings carry tag evidence; otherwise it reports "tags detected for 14
of 91 facings in this row" as a hint. Missing-tag findings are one per row with a
count, not one per facing, and price_mismatch still reports every disagreement.
On add/image6.jpg the result is now 7 real price mismatches and no false
violations.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
reconcile() compares one accepted identity with its tag and the catalog entry and
publishes a state plus the individual signals (JAN exact, name by longest common
substring, price against either the sale or the regular price). It never changes
the product identity: a disagreement is a second opinion shown next to the first,
and only a checksummed barcode outranks the matcher.
evaluate_compliance() is the first rule family: missing_price_tag, the owner's
missing_tag_group (>= 3 consecutive facings without a tag), orphan_tag,
price_mismatch, plus an unreadable_tags hint. Every rule fails closed on evidence
quality - a row whose tags could not be read produces "move closer", never a
violation. Findings must appear in 3 of the last 5 frames before they publish, so
the UI never flickers.
Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
