vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

Make the catalog hash honest, share the text helpers, drop the duplicate labels

Browse source 
`catalog_hash` hashed the CSV and the view *names*, so replacing
`sku_l09.jpg` with a different product's packshot left the hash unchanged and
a report could claim the same catalog while every l09 decision in it had
moved. It hashes each view's bytes now.

`identity_lanes` carried its own `normalize_text` and `bigrams`, near-copies
of the ones in `rebuild_catalog` that had already drifted (the lane's table
folded ユ to itself, the catalog's did not). The lane imports the catalog's
now, so one read scores the same either way.

`evidence/image6_v2.json` was byte-identical to `eval/labels/image6.json`; the
evidence scripts and the decision document point at the canonical file and the
copy is gone. `score_crop`'s docstring now states the contract its callers
already depend on: the dict's insertion order is the lane's ranking, and
re-sorting by value names a different SKU than the lane chose.

Tests: the lane is shared by the classify pool, so a concurrent-ranking test
pins that one BFMatcher does not cross crops under eight threads.
This commit is contained in:
tngo 2026-09-29 00:48:03 +09:00
parent fc49ba60e6
commit b6c87c45ea
36 changed files with 89 additions and 1524 deletions
Download patch file Download diff file Expand all files Collapse all files

24
recognization/code/identity_lanes.py
View file

@ -27,7 +27,6 @@ import math
import os
import re
import sys
import unicodedata
from dataclasses import dataclass
from typing import Any, Dict, List, Optional, Sequence, Tuple
@ -38,7 +37,7 @@ if current_dir not in sys.path:
import cv2 # noqa: E402
import numpy as np # noqa: E402
from rebuild_catalog import sku_of # noqa: E402
from rebuild_catalog import bigrams, normalize_text, sku_of # noqa: E402
# The local lane's accept line, in the units of `confidence()` below. Measured
# on the corrected image6 labels: 2.0 keeps 95% of shown labels correct at 59%
@ -258,23 +257,10 @@ class LocalLane:
return {sku: value for sku, value in self.rank(crop)}
# Katakana folding: OCR reads small kana and the long-vowel mark inconsistently,
# so キユーピー and キユピー have to land on the same bigrams.
_KANA_FOLD = str.maketrans(
"ッャュョァィゥェォヮヵヶユ", "ツヤユヨアイウエオワカケユ"
)
def normalize_text(text: str) -> str:
"""NFKC, uppercase, kana-folded, alphanumeric and `%` only."""
text = unicodedata.normalize("NFKC", text).upper()
text = text.translate(_KANA_FOLD)
text = text.replace("キユーピー", "キユピ").replace("キューピー", "キユピ")
return "".join(ch for ch in text if ch.isalnum() or ch == "%")
def bigrams(text: str) -> set:
return {text[i:i + 2] for i in range(len(text) - 1)}
# Text normalization and bigrams come from `rebuild_catalog`, which uses them
# to match a catalog name against a packshot's OCR. Two copies of the same
# folding rules drift: the lane's copy folded ユ to itself while the catalog's
# did not, so the same read scored differently depending on which one ran.
def identity_bigrams(read: str) -> set: