Make the catalog hash honest, share the text helpers, drop the duplicate labels
`catalog_hash` hashed the CSV and the view *names*, so replacing `sku_l09.jpg` with a different product's packshot left the hash unchanged and a report could claim the same catalog while every l09 decision in it had moved. It hashes each view's bytes now. `identity_lanes` carried its own `normalize_text` and `bigrams`, near-copies of the ones in `rebuild_catalog` that had already drifted (the lane's table folded ユ to itself, the catalog's did not). The lane imports the catalog's now, so one read scores the same either way. `evidence/image6_v2.json` was byte-identical to `eval/labels/image6.json`; the evidence scripts and the decision document point at the canonical file and the copy is gone. `score_crop`'s docstring now states the contract its callers already depend on: the dict's insertion order is the lane's ranking, and re-sorting by value names a different SKU than the lane chose. Tests: the lane is shared by the classify pool, so a concurrent-ranking test pins that one BFMatcher does not cross crops under eight threads.
This commit is contained in:
parent
fc49ba60e6
commit
b6c87c45ea
36 changed files with 89 additions and 1524 deletions
|
|
@ -22,7 +22,7 @@ detector). The hybrid lane also reports shortlist recall, because a true SKU
|
|||
outside the shortlist is a hard ceiling no verifier can lift.
|
||||
|
||||
python code/eval_identity.py \
|
||||
--labels docs/research/2026-09-28-recognition-architecture/evidence/image6_v2.json \
|
||||
--labels eval/labels/image6.json \
|
||||
--image add/image6.jpg --stock-dir stock-lopia-clean --lanes matcher --live
|
||||
|
||||
Labels are {"image", "facings": [{"box", "label"}]} with "unknown" for facings
|
||||
|
|
@ -70,7 +70,7 @@ LIVE_LONG_SIDE = 1280
|
|||
# Pixel equality is not a leak test: a template re-cut from the same photo with
|
||||
# a different box passes a same-shape comparison. A 64x64 grayscale signature
|
||||
# catches most of those, but it is a heuristic, not a proof — measured on
|
||||
# image6_v2 against stock-lopia, re-cuts in the realistic 0.94..1.06 box band
|
||||
# the corrected labels against stock-lopia, re-cuts in the realistic 0.94..1.06 box band
|
||||
# land 7..37 MAD from their facing's signature while the closest pair of
|
||||
# genuinely distinct same-SKU facings is 23.1, so the distributions overlap and
|
||||
# no threshold is clean. 20.0 catches 81% of that band with zero false
|
||||
|
|
@ -116,12 +116,13 @@ def file_hash(path: str) -> Optional[str]:
|
|||
|
||||
|
||||
def catalog_hash(stock_dir: str) -> Optional[str]:
|
||||
"""SHA-256 over the catalog's CSV bytes and its image names.
|
||||
"""SHA-256 over the catalog's CSV bytes and each view's bytes.
|
||||
|
||||
Image bytes are not hashed: a re-encoded view of the same product is the
|
||||
same catalog, and hashing every packshot would make the number move for
|
||||
reasons that do not change a single decision. A changed row or a different
|
||||
set of views does change it.
|
||||
The view bytes are hashed, not just their names. Names alone were not
|
||||
enough: replacing `sku_l09.jpg` with a different product's packshot left
|
||||
the hash unchanged, so a report could claim the same catalog while every
|
||||
l09 decision in it had moved. Re-encoding the same image does move the
|
||||
hash, which is the price of not being able to lie about what was scored.
|
||||
"""
|
||||
digest = hashlib.sha256()
|
||||
csv_path = os.path.join(stock_dir, "stock.csv")
|
||||
|
|
@ -137,8 +138,14 @@ def catalog_hash(stock_dir: str) -> Optional[str]:
|
|||
)
|
||||
except OSError:
|
||||
names = []
|
||||
digest.update(b"\n")
|
||||
digest.update("\n".join(names).encode("utf-8"))
|
||||
for name in names:
|
||||
digest.update(b"\n")
|
||||
digest.update(name.encode("utf-8"))
|
||||
try:
|
||||
with open(os.path.join(stock_dir, name), "rb") as handle:
|
||||
digest.update(handle.read())
|
||||
except OSError:
|
||||
digest.update(b"unreadable")
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
|
|
@ -557,7 +564,7 @@ def shortlist_recall(
|
|||
shortlist already dropped. The denominator is enrolled facings only — a
|
||||
facing whose SKU has no reference cannot be retrieved by any shortlist, so
|
||||
counting it here reports a retrieval failure that is really a catalog gap.
|
||||
Measured on image6_v2, that distinction is 67% vs 98% at top-5.
|
||||
Measured on the corrected labels, that distinction is 67% vs 98% at top-5.
|
||||
"""
|
||||
hits = total = 0
|
||||
misses: List[Dict[str, Any]] = []
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue