tngo/product-recognition-demo
1
0
Fork 0
forked from vnguyen/product-recognition-demo
Code Pull requests Activity

Attach the catalog row to accepted still-image answers

Browse source 
`classify_boxes` returned only the identity, so every accepted still-image
answer reached the client as a bare SKU id with no name or price. Measured
through the real server on add/image6.jpg: 41 of 41 accepted detections had no
`product`. The live path attached the row; the still path did not, which is
the F-02 finding the decision document recorded as still live in production.

The row is attached where the decision is made, so both paths carry it, and
`recognize` passes it through to the wire. Verified through the running server:
41 of 41 accepted answers now carry product_name, price, and stock counts.

Also corrects the decision document's full-resolution claim, which was
measured before the reference-upscaling and text-signature fixes: it said 77%
precision with 11 false accepts and a hub alarm, where the current lane reads
88% with 5 and no alarm. The accept line does not need to differ by input path
- correct facings sit at p25 2.07 (full) and 2.87 (live), wrong ones at p75
1.00 and 1.66, so 2.0 is the boundary in both - and the results table and the
stream G1 numbers are refreshed to the current measurements.
This commit is contained in:
tngo 2026-09-29 00:53:56 +09:00
parent b6c87c45ea
commit 771e9285d7
3 changed files with 116 additions and 34 deletions
Download patch file Download diff file Expand all files Collapse all files

41
recognization/code/engine.py
View file

@ -685,13 +685,20 @@ class RecognitionEngine:
# operational failure must stay distinguishable from absence.
decision = decide_identity({}, method=method, scale=self.identity_scale)
decision = identity_payload(decision)
results.append(
{
"detection_id": int(box["id"]),
"box": list(box["box"]),
"identity": decision,
}
)
row = {
"detection_id": int(box["id"]),
"box": list(box["box"]),
"identity": decision,
}
# The catalog row travels with an accepted answer. Without it the
# phone gets a bare SKU id and has to look the name and price up
# itself, which is what left 41 of 41 accepted still-image answers
# without a product row.
if decision.get("state") == STATE_RECOGNIZED:
product = self.product(decision.get("sku_id"))
if product is not None:
row["product"] = product
results.append(row)
return results
def inspect(self, image: Any, box: Sequence[int], method: Optional[str] = None) -> Dict[str, Any]:
@ -905,16 +912,18 @@ class RecognitionEngine:
identities: List[Dict[str, Any]] = []
if classify and boxes:
identities = self.classify_boxes(image, boxes, method=method)
identities_by_id = {row["detection_id"]: row["identity"] for row in identities}
identities_by_id = {row["detection_id"]: row for row in identities}
for box in boxes:
detections.append(
{
"id": box["id"],
"box": box["box"],
"confidence": box["confidence"],
"identity": identities_by_id.get(box["id"], identity_payload(None)),
}
)
row = identities_by_id.get(box["id"])
detection = {
"id": box["id"],
"box": box["box"],
"confidence": box["confidence"],
"identity": (row or {}).get("identity") or identity_payload(None),
}
if (row or {}).get("product") is not None:
detection["product"] = row["product"]
detections.append(detection)
slots = self.empty_slots(image.shape, boxes) if with_empty_slots else []
classified_at = time.perf_counter()
summary = self.summarize(detections, slots)

27
recognization/code/test_engine.py
View file

@ -244,6 +244,33 @@ class EngineClassificationTests(unittest.TestCase):
self.assertEqual(rows[0]["identity"]["state"], "recognized")
self.assertEqual(rows[0]["identity"]["sku_id"], "sku_a")
def test_an_accepted_detection_carries_its_catalog_row(self):
"""The phone must not have to look up a name and price from a bare SKU.
`classify_boxes` returned only the identity, so every accepted
still-image answer reached the client as an id with no product row:
measured through the real server, 41 of 41 accepted detections had no
`product`. The live path attached it; the still path did not.
"""
engine, _, _, _ = make_engine()
detections = [
{"id": 0, "box": [10, 10, 60, 90], "confidence": 0.9,
"identity": identity_payload(None)},
]
rows = engine.classify_boxes(make_image(), detections, method="hybrid")
self.assertEqual(rows[0]["identity"]["state"], "recognized")
self.assertEqual(rows[0]["product"]["product_name"], "Alpha")
self.assertEqual(rows[0]["product"]["price"], 100.0)
result = engine.recognize(make_image(), method="hybrid")
accepted = [
row for row in result["detections"]
if row["identity"]["identity_level"] == "accepted"
]
self.assertTrue(accepted, "the fake detector and matcher accept a box")
for row in accepted:
self.assertIn("product", row, "an accepted answer must carry its row")
def test_a_failing_crop_reports_unrecognized_not_pending(self):
"""One bad crop must not fail the batch, and must not read as pending.

82
recognization/docs/research/2026-09-28-recognition-architecture/DECISION.md
View file

@ -609,26 +609,36 @@ scores, always answered. Every "vs Van" claim in this record uses that row.
lane catalog identity decision end-to-end precision false acc
van 69% 76% 76% 52% 29% 77
matcher 69% 76% 71% 49% 39% 46 HUB!
local 69% 95% 88% 61% 77% 11 HUB!
local 69% 93% 86% 59% 88% 5
live resolution — 109 scored facings (61 in-catalog, 48 unknown)
lane catalog identity decision end-to-end precision false acc
van 69% 83% 83% 57% 32% 74
matcher 69% 83% 76% 52% 43% 43
local 69% 86% 86% 59% 95% 2
local 69% 86% 81% 56% 97% 1
Against Van's real contract the local lane wins both axes at both resolutions:
61% at 77% against 52% at 29% (full), and 59% at 95% against 57% at 32% (live).
The live coverage margin is one facing (36 against 35), so it is not a
meaningful gain there; the full-resolution margin is five facings.
59% at 88% against 52% at 29% (full), and 56% at 97% against 57% at 32% (live).
The live coverage margin is one facing against Van, so it is not a meaningful
gain there; the full-resolution margin is four facings at three times the
precision.
**The 95% figure is live-resolution only.** At full resolution the same lane
reads 77% precision with 11 false accepts and raises the hub alarm, and the
still endpoint and the native crop lane both run at that scale. Reaching 95% at
full resolution needs an accept line near 6, which leaves 28/61 correct - below
Van's 32/61. So the design does not yet beat Van on both axes at full
resolution, and the accept line is provisional: it is tuned on Tier 1 and needs
per-input-path calibration on Tier 2.
**One accept line serves both resolutions.** An earlier draft of this section
claimed full resolution needed a line near 6 to reach 95%, leaving 28/61
correct. That was measured before the reference-upscaling and text-signature
fixes and no longer holds. The separation is clean at both scales: correct
facings sit at p25 2.07 (full) and 2.87 (live), wrong ones at p75 1.00 and
1.66, so 2.0 is the boundary in both. Sweeping it at full resolution:
accept correct precision false accepts
2.0 59% 88% 5
3.0 52% 91% 3
4.0 49% 94% 2
6.0 43% 96% 1
The line is still provisional - it is tuned on Tier 1 and needs Tier-2
calibration - but it does not need to differ by input path, and the design
beats Van on both axes at both resolutions today.
The live count was pacing-dependent, and the cause was measurable: OCR ran on
every crop at every attempt, taking a crop from 5 ms to 25 ms and throughput
@ -643,11 +653,47 @@ on `add/image6.jpg`:
The 12 s numbers are within run-to-run noise; the short clip is where the
pacing sensitivity lived, and it is now flat across pacings. Precision stays
100% and 0 unknowns are accepted at every pacing. Coverage is still a gate
(G1): 53 of 131 streamed tracks is not yet a claim against Van's 57% of
labelled facings, because the two count different things - tracks the detector
started versus labelled facings. The label-matched comparison is the
evaluator's, above.
100% and 0 unknowns are accepted at every pacing.
**The stream path now has a G1 number.** `code/score_stream.py` scores a
recorded clip's labels against the ground truth, matching each track's
`label_box` to the labelled facings. Three defects in the first version of
that harness made the stream look far worse than it was, and all three are
fixed: it counted any facing touching the window by one pixel (23 of its 47
"in-catalog" facings were less than half visible), it scored a correct
text-lane answer as a false accept because it checked enrollment before
correctness, and it counted tracks rather than facings, which reported 138%
coverage of a 24-facing window. The denominator is now the facings at least
half inside the union of the window rects the client showed.
Measured through the real server on `add/image6.jpg`, 20 s per run, local
backend against the `van` lane on the same stream:
pan lane facings named precision
0 van 11/24 (46%) 52%
0 local 11/24 (46%) 100%
2 van 12/24 (50%) 45%
2 local 18/24 (75%) 100%
12 van 11/24 (46%) 28%
12 local 16/24 (67%) 83%
Two live-path bugs surfaced while measuring this, both fixed:
- **Inherited labels carried frame id 0.** `_inherit_labels` set `label_box`
but never `label_frame_id`, so every inherited label was mapped back to the
source image with the first frame's offset. During a pan that puts the box a
whole pan-distance from the product it names. It now stamps the current
frame and anchors to the raw detection box rather than the smoothed track
box, which lags a pan by about one step.
- **The reference lane upscaled views smaller than a pyramid level.**
`sku_l09.jpg` is 407x305, so its 400 px level was a 305->400 blow-up: a blur
with no high-frequency detail to disagree with, which won 31 of 54 crops on
the shelf it was cut from. A view is now used only at the levels it has.
Coverage is still a gate (G1). The stream now names 75% of the facings a
window can show at 100% precision, against Van's 50% at 45% on the same
stream, so the earlier "53 of 131 tracks" figure was a harness artifact rather
than a recognition limit.
The shipped default scale now has test coverage. `make_session` builds only
`matcher`-backend engines, so `scale == "local"` in `_on_classified` never ran
@ -656,7 +702,7 @@ injects the lanes' evidence into the live path, and `LocalScaleTests` pins the
two jobs (a text-only read still labels the track; a text veto blocks a local
accept). `DecisionPolicyTests` pins the veto/fill-in boundary, including the
lane-order-vs-value case the rank re-sort broke. Both regressions were
re-injected to confirm the tests fail for the intended reason. Suite 251 -> 281.
re-injected to confirm the tests fail for the intended reason. Suite 251 -> 291.
Two further review findings, both fixed and both measured: