tngo
0cf64a18bf
The live path abstained too much: a detected product whose best candidate missed the accept line was left blank, so a shelf where 59% of in-catalog facings were shown read as broken. Show the best candidate immediately as `provisional`, keep the track re-classifiable so a later attempt upgrades or corrects it, and never blank a label already on screen. - identity.py / identity_lanes.py: every decision now carries `provisional_sku` - the row the evidence supports well enough to show, separate from `best_sku`, which still means "cleared the gate". The accept line does not move: far-unknown false accepts stay at 0.066%. - live.py: a below-gate candidate labels the track with `identity_level: provisional` and `confirmed = False`, so the planner and the crop lane keep offering it. Accepted labels stay frozen. Inherited provisional labels stay re-classifiable. - Clients render provisional distinctly (amber chip, dashed box) via `LiveScanOverlayPolicy.isProvisional(identity)`, which now reads the wire level instead of only the state. - build_stock_all.py merges stock, stock-lopia, stock-mediya and stock-milk150 into stock-all (111 images, 60 SKUs). The server could only read one folder, so anything outside it came back blank. stock-demo is byte-identical to mediya+milk150 and is not a source. Measured on add/image6.jpg at live resolution: in-catalog facings shown 59.4% -> 86.0%, correct shown 38 -> 55, precision 97.4% -> 87.3%. On the wire at 8fps 0.5x: 65 of 79 tracks labelled, 742 accepted + 309 provisional observations, first label 1.27s. docs/research/2026-09-29-shelf-intelligence/ carries the experiments and the decision: group signals (combined AUC 0.991 sibling / 0.990 unrelated), OCR scheduling (group representatives cut calls 4.4x and wall 3.3x), and the finding that group propagation damages more labels than it rescues on this shelf, so groups schedule compute rather than overwrite labels. 331 tests pass.
157 lines
5 KiB
JSON
157 lines
5 KiB
JSON
{
|
|
"question": "False accepts per named analyze policy on unrelated shelf crops, analyze.decide on the sweep_accept_lines cache.",
|
|
"method": {
|
|
"decision": "analyze.decide(o, policy, signatures) - the frozen analysis's own call",
|
|
"policies": {
|
|
"strict": 4.0,
|
|
"balanced": 2.0,
|
|
"coverage-first": 0.5,
|
|
"near-argmax": 0.0
|
|
},
|
|
"gate_for_comparison": "identity_lanes.DecisionPolicy(local_accept=threshold, signatures)",
|
|
"text_scores": "TextLane.scores_from_text on the cached OCR read",
|
|
"fallback": "analyze.decide's near-argmax extra fallback only: keep rank[0] when the gate refused, unless the read contradicts that row's size or the text prefers another row"
|
|
},
|
|
"provenance": {
|
|
"cache": {
|
|
"path": "data/evaluations/2026-09-29-session/scratch/accept_line_cache.pkl",
|
|
"sha256": "1abf0312d354f9f00af6d5a339d0edde0d584c1beee1dab46d04966caf6e637a",
|
|
"bytes": 89680,
|
|
"version": 3,
|
|
"detect_conf": 0.1,
|
|
"records": [
|
|
"version",
|
|
"conf",
|
|
"photos",
|
|
"shelf_full",
|
|
"shelf_live",
|
|
"distractors"
|
|
],
|
|
"not_recorded": [
|
|
"stock_dir",
|
|
"catalog_hash",
|
|
"code_revision",
|
|
"source image paths or hashes",
|
|
"per-photo crop counts"
|
|
],
|
|
"note": "The cache attests no catalog, so the catalog match below is read off the current worktree, not out of the cache. The read is unpickled, not rebuilt."
|
|
},
|
|
"catalog": {
|
|
"dir": "stock-lopia-clean",
|
|
"catalog_hash": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
|
"matches_observations": true,
|
|
"observations_catalog_hash": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
|
"observations_code_revision": "fbf31d74feb12538dc07207cb80722dac38dd438"
|
|
},
|
|
"code_revision": "fbf31d74feb12538dc07207cb80722dac38dd438",
|
|
"signatures": {
|
|
"rows": 12,
|
|
"sha256": "3e2388f27ae5796155656c5aa10094fbf1012766b7cee088224573a84b2bb584"
|
|
}
|
|
},
|
|
"counts": {
|
|
"photos": 64,
|
|
"crops": 4580,
|
|
"crops_with_a_local_candidate": 155,
|
|
"crops_with_no_local_candidate": 4425,
|
|
"crops_with_a_raw_read": 1546,
|
|
"crops_scored_by_text": 1185
|
|
},
|
|
"policies": {
|
|
"strict": {
|
|
"local_accept": 4.0,
|
|
"fallback": false,
|
|
"accepts": 3,
|
|
"gate_accepts": 3,
|
|
"fallback_accepts": 0,
|
|
"by_source": {
|
|
"local": 0,
|
|
"text": 3
|
|
},
|
|
"rate": 0.0006550218340611354,
|
|
"rate_percent": 0.0655,
|
|
"crops_with_a_local_candidate": 155
|
|
},
|
|
"balanced": {
|
|
"local_accept": 2.0,
|
|
"fallback": false,
|
|
"accepts": 8,
|
|
"gate_accepts": 8,
|
|
"fallback_accepts": 0,
|
|
"by_source": {
|
|
"local": 5,
|
|
"text": 3
|
|
},
|
|
"rate": 0.0017467248908296944,
|
|
"rate_percent": 0.1747,
|
|
"crops_with_a_local_candidate": 155
|
|
},
|
|
"coverage-first": {
|
|
"local_accept": 0.5,
|
|
"fallback": false,
|
|
"accepts": 83,
|
|
"gate_accepts": 83,
|
|
"fallback_accepts": 0,
|
|
"by_source": {
|
|
"local": 80,
|
|
"text": 3
|
|
},
|
|
"rate": 0.018122270742358077,
|
|
"rate_percent": 1.8122,
|
|
"crops_with_a_local_candidate": 155
|
|
},
|
|
"near-argmax": {
|
|
"local_accept": 0.0,
|
|
"fallback": true,
|
|
"accepts": 119,
|
|
"gate_accepts": 115,
|
|
"fallback_accepts": 4,
|
|
"by_source": {
|
|
"local": 112,
|
|
"text": 3
|
|
},
|
|
"rate": 0.025982532751091702,
|
|
"rate_percent": 2.5983,
|
|
"crops_with_a_local_candidate": 155
|
|
}
|
|
},
|
|
"cross_check_far_unknown_lines": {
|
|
"source": "evidence/far-unknown-lines.json",
|
|
"note": "accept-threshold cells only, no fallback; producer script not in the tree",
|
|
"cells": {
|
|
"strict": {
|
|
"lines_accepts": 3,
|
|
"lines_crops": 4580,
|
|
"here_gate_accepts": 3,
|
|
"agrees": true
|
|
},
|
|
"balanced": {
|
|
"lines_accepts": 8,
|
|
"lines_crops": 4580,
|
|
"here_gate_accepts": 8,
|
|
"agrees": true
|
|
},
|
|
"coverage-first": {
|
|
"lines_accepts": 83,
|
|
"lines_crops": 4580,
|
|
"here_gate_accepts": 83,
|
|
"agrees": true
|
|
},
|
|
"near-argmax": {
|
|
"lines_accepts": 115,
|
|
"lines_crops": 4580,
|
|
"here_gate_accepts": 115,
|
|
"agrees": true
|
|
}
|
|
}
|
|
},
|
|
"limitations": [
|
|
"The cache records no catalog, code revision or image list, so 'same catalog as the frozen observations' is read off the current worktree, not attested by the cache.",
|
|
"Only 155 of 4580 crops produced a local candidate; the other 4425 are decided by the text lane alone or refused before any gate applies, so the accept floor is exercised on a small minority of the rows and the rates are dominated by empty evidence.",
|
|
"The catalog is 12 rows from one store. A false-accept rate is a function of catalog size, so these numbers do not transfer to a larger or differently-stocked catalog.",
|
|
"Crops come from one detection pass at detect_conf 0.1 on 1280-long-side frames, so the crop population is set by the detector, not only by the identity gate.",
|
|
"The cache has no per-photo breakdown, so no photo-level rate and no check that one image is not carrying the whole count.",
|
|
"The near-argmax fallback is a research construct in analyze.py, not shipped code; its row prices a policy that does not exist on the server yet.",
|
|
"far-unknown-lines.json's producer is not in the tree, so agreement with it is corroboration, not verification."
|
|
]
|
|
}
|