tngo/product-recognition-demo
1
0
Fork 0
forked from vnguyen/product-recognition-demo
Code Pull requests Activity

Coverage-first live labelling, merged catalog, shelf-intelligence decision

Browse source 
The live path abstained too much: a detected product whose best candidate
missed the accept line was left blank, so a shelf where 59% of in-catalog
facings were shown read as broken. Show the best candidate immediately as
`provisional`, keep the track re-classifiable so a later attempt upgrades or
corrects it, and never blank a label already on screen.

- identity.py / identity_lanes.py: every decision now carries
  `provisional_sku` - the row the evidence supports well enough to show,
  separate from `best_sku`, which still means "cleared the gate". The accept
  line does not move: far-unknown false accepts stay at 0.066%.
- live.py: a below-gate candidate labels the track with
  `identity_level: provisional` and `confirmed = False`, so the planner and
  the crop lane keep offering it. Accepted labels stay frozen. Inherited
  provisional labels stay re-classifiable.
- Clients render provisional distinctly (amber chip, dashed box) via
  `LiveScanOverlayPolicy.isProvisional(identity)`, which now reads the wire
  level instead of only the state.
- build_stock_all.py merges stock, stock-lopia, stock-mediya and
  stock-milk150 into stock-all (111 images, 60 SKUs). The server could only
  read one folder, so anything outside it came back blank. stock-demo is
  byte-identical to mediya+milk150 and is not a source.

Measured on add/image6.jpg at live resolution: in-catalog facings shown
59.4% -> 86.0%, correct shown 38 -> 55, precision 97.4% -> 87.3%. On the
wire at 8fps 0.5x: 65 of 79 tracks labelled, 742 accepted + 309 provisional
observations, first label 1.27s.

docs/research/2026-09-29-shelf-intelligence/ carries the experiments and the
decision: group signals (combined AUC 0.991 sibling / 0.990 unrelated),
OCR scheduling (group representatives cut calls 4.4x and wall 3.3x), and the
finding that group propagation damages more labels than it rescues on this
shelf, so groups schedule compute rather than overwrite labels.

331 tests pass.
This commit is contained in:
tngo 2026-09-29 13:45:14 +09:00
parent fbf31d74fe
commit 0cf64a18bf
171 changed files with 554214 additions and 35 deletions
Download patch file Download diff file Expand all files Collapse all files

81
recognization/code/test_live.py
View file

@ -212,22 +212,33 @@ class LabelLifecycleTests(unittest.TestCase):
)), "the product should be labelled once classified")
engine.shutdown()
def test_label_is_never_shown_without_acceptance(self):
def test_a_weak_match_is_shown_provisional_not_accepted(self):
"""Coverage-first: a below-gate candidate is shown, but marked provisional.
The old contract left this box blank. Blank is now the worse outcome:
the operator sees a guess that the next attempt may upgrade, and the
wire says which it is, so a client can draw it differently.
"""
session, engine, _, _ = make_session(FakeMatcher(score=0.4)) # below the accept threshold
for frame_id in range(1, 8):
result = process(session, frame_id)
for track in result["tracks"]:
self.assertIsNone(track["identity"]["sku_id"],
"a weak match must never name a SKU")
process(session, frame_id)
result = process(session, 9)
track = result["tracks"][0]
self.assertEqual(track["identity"]["sku_id"], "sku_a",
"a weak match must still be shown")
self.assertEqual(track["identity"]["identity_level"], "provisional",
"a weak match must not claim to be accepted")
engine.shutdown()
def test_weak_match_ends_as_needs_confirmation(self):
def test_a_provisional_label_is_never_blanked(self):
"""Once shown, a guess stays: the budget running out must not take it back."""
session, engine, _, _ = make_session(FakeMatcher(score=0.4))
for frame_id in range(1, 8):
for frame_id in range(1, 12):
process(session, frame_id)
result = process(session, 8)
self.assertTrue(wait_for(lambda: process(session, 9)["tracks"][0]["identity"]["state"]
== "needs_confirmation", timeout=3.0))
result = process(session, 13)
track = result["tracks"][0]
self.assertEqual(track["identity"]["sku_id"], "sku_a",
"a spent budget must not blank a label already on screen")
engine.shutdown()
def test_restarted_id_is_re_verified_not_inherited(self):
@ -314,8 +325,10 @@ class CosineScaleTests(unittest.TestCase):
)), "a gallery-accepted head match must be labelled")
engine.shutdown()
def test_a_close_cosine_pair_stays_ambiguous(self):
"""The margin gate still applies: two near-equal scores must not name a SKU."""
def test_a_close_cosine_pair_is_shown_provisional(self):
"""The margin gate still applies to *acceptance*: two near-equal scores
must not be named as settled. Coverage-first shows the top one anyway,
marked provisional, because a blank box is the worse outcome."""
session, engine, _, _ = make_session(FakeMatcher(score=0.45), engine_cls=CosineEngine)
# The matcher contract is (scores, details); a bare dict unpacks into two
# strings and raises inside the future callback, where the failure is
@ -325,15 +338,17 @@ class CosineScaleTests(unittest.TestCase):
process(session, frame_id)
result = process(session, 9)
for track in result["tracks"]:
self.assertIsNone(track["identity"]["sku_id"],
"a cosine pair inside the margin must not be named")
self.assertEqual(track["identity"]["identity_level"], "provisional",
"a cosine pair inside the margin must not be accepted")
engine.shutdown()
def test_a_rejected_crop_cannot_dilute_an_accepted_one(self):
"""aggregate_votes divides by the whole window, so a miss must not lower the line.
A rejected crop first, then an accepted 0.45: the mean is 0.225. Gating the
aggregate at zero would label that, at a score the gallery calls unknown.
aggregate at zero would *accept* that, at a score the gallery calls unknown.
Coverage-first still shows it, but the level must stay provisional: the
dilution must not be promoted to a settled label.
"""
session, engine, _, _ = make_session(FakeMatcher(score=0.45), engine_cls=CosineEngine)
answers = iter([{}, {"sku_a": 0.45}])
@ -343,8 +358,8 @@ class CosineScaleTests(unittest.TestCase):
process(session, frame_id)
result = process(session, 13)
for track in result["tracks"]:
self.assertIsNone(track["identity"]["sku_id"],
"a diluted mean must not be labelled below the gallery's line")
self.assertEqual(track["identity"]["identity_level"], "provisional",
"a diluted mean must not be accepted below the gallery's line")
engine.shutdown()
@ -432,6 +447,38 @@ class LocalScaleTests(unittest.TestCase):
self.assertEqual(engine.ocr_calls, 1, "OCR must run once per track")
engine.shutdown()
def test_no_ocr_never_reads_a_crop(self):
"""The session's `ocr` switch turns the text lane off for every crop.
The text lane is the slowest thing the server does per crop, so the
phone can ask for a session that runs on geometry alone. Only the text
lane goes away: the local lane still names the track, because the whole
point of the setting is a faster session, not a quieter one.
"""
session, engine, _, _ = make_session(
engine_cls=LocalScaleEngine,
settings=LiveSettings(method="hybrid", ocr=False),
)
engine.lane_answers = [({"sku_a": 5.0}, [("sku_a", 0.9)], "コクうま 65%")] * 6
labelled = False
for frame_id in range(1, 14):
for track in process(session, frame_id)["tracks"]:
if track["identity"]["sku_id"] == "sku_a":
labelled = True
self.assertEqual(engine.ocr_calls, 0, "no crop may be read with the text lane off")
self.assertTrue(labelled, "the local lane must still name the track")
engine.shutdown()
def test_the_ocr_switch_is_reported_back_to_the_client(self):
"""The client reads the session's own value, not the one it asked for."""
session, engine, _, _ = make_session()
self.assertTrue(session.config()["ocr"], "the text lane is on by default")
session.configure(ocr=False)
self.assertFalse(session.config()["ocr"], "the switch must reach the config")
session.configure(ocr=True)
self.assertTrue(session.config()["ocr"], "and back on again")
engine.shutdown()
def test_a_size_only_read_is_not_cached(self):
"""An unreadable crop must not veto the product for the track's life.