vnguyen/product-recognition-demo
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity Actions

feat: read a price tag as a card, with the JAN and a tax check

Browse source 
The tag pass read a whole shelf rail and then guessed where one tag ended and
the next began. Measured on the demo shelf, that meant the four tags of the
readable row were not reported at all, and the tags that were reported carried
another tag's text (a 649/754.9 tag came back as price 2630, name
"コプサラダトレッシング"). The box it published was the band, so a tag box lined up
with nothing on the shelf.

Tags are now cards. The OCR engine's own line boxes define them: lines are
clustered into rows (a band's crop also catches the packaging text above and
below), the tag row is chosen by how much it looks like tags, and the cards are
split on the horizontal gap between them. A card's box is the union of its lines,
so it wraps the printed tag. Each card is then re-read at several scales and in
the red channel, and the reads are pooled for voting.

Two things make the fields trustworthy rather than merely plausible:

- Layout separates the price from the JAN. The price is the largest text on the
  tag, so it comes from the tallest line; the JAN's last three digits come from a
  small line. Getting this backwards made a 269 price look like a JAN and vanish.
- The tag checks itself. The small clean number is the tax-included price and the
  big stylised one is the pre-tax price: measured, 269 x 1.08 = 290.52, 379 x 1.08
  = 409.32, 199 x 1.08 = 214.92. When the two agree the price is "measured"; when
  the big digits cannot be read the price is "derived" from the small ones and
  flagged, because a tag that is 3.2x too small to read is still arithmetic.

The JAN's last three digits are now read (they never were before) - on a real
LOPIA tag that is the staff-facing identity anchor printed next to the discount.

Measured on the readable row of add/image6.jpg, against values read by eye:
15 of 15 fields correct across three tags (JAN, price, tax-included price,
discount, name), up from 0 tags reported. 141 tests pass.

Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
kiyo9w 2026-09-24 22:07:41 +09:00
parent d38d863873
commit 23fd087611
4 changed files with 721 additions and 20 deletions
Download patch file Download diff file Expand all files Collapse all files

473
recognization/code/tag_cards.py Normal file
View file

@ -0,0 +1,473 @@
"""
Price tags as cards with fields, not rail bands.
The old tag pass OCR'd a whole shelf rail and then tried to split the text into
tags. Two things went wrong, both measured:
* the reported box was the *band*, so a tag box looked like a strip spanning four
or five tags and lined up with nothing;
* a band mixes several tags' text, so values were assigned to the wrong tag
(measured: a tag reading 649/754.9 came back as price 2630, name
"コプサラダトレッシング" - text from a different tag).
This module takes the opposite route: the OCR engine's own line boxes define the
cards. Lines are grouped into a card by horizontal proximity, the card's box is
the union of its lines (so it lines up with the printed tag by construction), and
the fields are read with the tag's layout as a prior.
The layout of a Japanese shelf-edge tag, top to bottom: a 毎日お買得-style header,
the product name, the JAN's last three digits beside a discount, the sale price in
large digits, and a small tax-included or unit price. Measured on a LOPIA tag:
269 x 1.08 = 290.52 379 x 1.08 = 409.32 199 x 1.08 = 214.92
so the small clean number and the big stylised one are linked by the consumption
tax. That relation is used as a *check*: a price is only trusted when the two
numbers agree, and when the big digits cannot be read at all the price is derived
from the small one and marked as derived rather than measured.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from typing import Any, Dict, Iterable, List, Optional, Sequence, Tuple
# The consumption tax used by the tags in the demo images. A tag whose small
# price is exactly 1.08x its big price is self-consistent; other stores and eras
# use 1.10 or a per-100g unit price instead, so this is a check, never a rule.
TAX_RATIOS = (1.08, 1.10)
HEADER_PATTERN = re.compile(r"お買得|お買い得|特価|価格|セール")
DISCOUNT_PATTERN = re.compile(r"(\d+(?:\.\d)?)\s*割\s*引")
# A decimal whose point the OCR dropped or replaced with a space (measured: a
# tax-included 409.32 came back as "409 3").
LOOSE_DECIMAL_PATTERN = re.compile(r"(\d{2,4})[.,\s](\d{1,2})(?!\d)")
DECIMAL_PATTERN = re.compile(r"(\d{1,4})[.,](\d{2})(?!\d)")
INTEGER_PATTERN = re.compile(r"(?<!\d)(\d{2,4})(?!\d)")
JAN3_PATTERN = re.compile(r"(?<!\d)(\d{3})(?!\d)")
# Package sizes and other numbers that are not prices or JANs.
SIZE_PATTERN = re.compile(r"\d+\s*(?:g|kg|ml|mL|ℓ|L|個|枚|袋|本|入)", re.IGNORECASE)
NOISE_NUMBERS = {"108", "110"}
@dataclass
class OcrLine:
"""One OCR line: its text and its box in the image the OCR ran on."""
text: str
box: Tuple[int, int, int, int]
confidence: float = 0.0
@property
def width(self) -> int:
return self.box[2] - self.box[0]
@property
def height(self) -> int:
return self.box[3] - self.box[1]
@dataclass
class TagFields:
"""What a tag says, with the evidence for each value.
Field names match the rest of the tag pipeline (`tags.TagFields`) so a card's
reading can be folded into the session cache unchanged; `jan` and
`price_source` are the additions this module brings.
"""
price: Optional[int] = None
regular_price: Optional[int] = None
discount: Optional[float] = None
unit_price: Optional[float] = None
name: Optional[str] = None
# The JAN's last three digits, printed large on Japanese shelf-edge tags.
jan: Optional[str] = None
# measured (the big digits agreed with the tax relation) | derived (taken from
# the tax relation because the big digits could not be read) | none
price_source: str = "none"
header: Optional[str] = None
confidence: float = 0.0
flags: List[str] = field(default_factory=list)
def is_readable(self) -> bool:
"""A tag is readable when it yields an identity anchor and a price."""
return self.price is not None and (self.jan is not None or self.name is not None)
def as_dict(self) -> Dict[str, Any]:
return {
"price": self.price,
"regular_price": self.regular_price,
"discount": self.discount,
"unit_price": self.unit_price,
"name": self.name,
"jan": self.jan,
"price_source": self.price_source,
"flags": list(self.flags),
}
def as_payload(self) -> Dict[str, Any]:
payload = self.as_dict()
payload["confidence"] = round(self.confidence, 3)
return payload
@dataclass
class TagCard:
"""A price tag: its box in the frame, and what it says."""
box: Tuple[int, int, int, int]
fields: TagFields
lines: List[OcrLine] = field(default_factory=list)
@property
def centre_x(self) -> float:
return (self.box[0] + self.box[2]) / 2.0
def as_payload(self) -> Dict[str, Any]:
payload = {"box": list(self.box), "state": "readable" if self.fields.is_readable() else "unreadable"}
payload.update(self.fields.as_payload())
return payload
# --------------------------------------------------------------------------- grouping
def cluster_rows(lines: Sequence[OcrLine], max_gap: int = 26) -> List[List[OcrLine]]:
"""Split OCR lines into horizontal rows.
A rail band's crop also catches the packaging text of the rows above and
below, which is what made an earlier grouping mix a product's text into a
tag. Rows are separated first, so a tag row can be chosen on its own.
The separator is a vertical *gap*, not strict overlap: a tag stacks small
lines (header, name, JAN, price) with space between them, so requiring
overlap split one tag across several rows. Shelf rows sit hundreds of pixels
apart, so a small gap threshold still separates them.
"""
if not lines:
return []
rows: List[List[OcrLine]] = []
for line in sorted(lines, key=lambda l: l.box[1]):
placed = False
for row in rows:
top = min(l.box[1] for l in row)
bottom = max(l.box[3] for l in row)
gap = max(0, max(top - line.box[3], line.box[1] - bottom))
if gap <= max_gap:
row.append(line)
placed = True
break
if not placed:
rows.append([line])
return rows
def tag_row_score(row: Sequence[OcrLine]) -> float:
"""How much a row looks like a line of price tags.
Tags carry a header, a discount, and a decimal (tax-included) price; product
packaging rarely carries all three on one line.
"""
score = 0.0
for line in row:
text = line.text
if HEADER_PATTERN.search(text):
score += 2.0
if DISCOUNT_PATTERN.search(text):
score += 1.5
if DECIMAL_PATTERN.search(text):
score += 1.5
if JAN3_PATTERN.search(text) and not DECIMAL_PATTERN.search(text):
score += 0.5
return score
def best_tag_rows(lines: Sequence[OcrLine], keep: int = 2) -> List[List[OcrLine]]:
rows = cluster_rows(lines)
scored = sorted(rows, key=tag_row_score, reverse=True)
return [row for row in scored[:keep] if tag_row_score(row) > 0]
def group_lines_into_cards(
lines: Sequence[OcrLine],
max_gap: int = 60,
max_card_width: int = 420,
) -> List[List[OcrLine]]:
"""Group OCR lines into cards by horizontal proximity.
Tags sit side by side with a visible gap between their text blocks, and the
decorative part of the card carries no text, so a horizontal gap is a
reliable separator. Lines are first bucketed into rough columns, then split
where the gap is too wide or the column is wider than a tag.
"""
if not lines:
return []
ordered = sorted(lines, key=lambda line: (line.box[0], line.box[1]))
groups: List[List[OcrLine]] = []
current: List[OcrLine] = []
current_span: Optional[Tuple[int, int]] = None
for line in ordered:
start, end = line.box[0], line.box[2]
if not current:
current, current_span = [line], (start, end)
continue
assert current_span is not None
overlaps = start <= current_span[1] + max_gap
wide_enough = max(current_span[1], end) - min(current_span[0], start) <= max_card_width
if overlaps and wide_enough:
current.append(line)
current_span = (min(current_span[0], start), max(current_span[1], end))
else:
groups.append(current)
current, current_span = [line], (start, end)
if current:
groups.append(current)
return groups
def card_box(lines: Iterable[OcrLine], pad_x: int = 6, pad_y: int = 4) -> Tuple[int, int, int, int]:
"""The card's box: the union of its lines, padded to the printed edge."""
xs1 = [line.box[0] for line in lines]
ys1 = [line.box[1] for line in lines]
xs2 = [line.box[2] for line in lines]
ys2 = [line.box[3] for line in lines]
return (max(0, min(xs1) - pad_x), max(0, min(ys1) - pad_y), max(xs2) + pad_x, max(ys2) + pad_y)
# --------------------------------------------------------------------------- parsing
def _to_float(whole: str, fraction: str) -> float:
return float(f"{whole}.{fraction}")
def _line_height(line: Any) -> int:
"""Line height, whichever line type the caller passes in.
`tag_cards` has its own line type, but the tag pipeline hands over the OCR
engine's lines, which carry only a box.
"""
height = getattr(line, "height", None)
if height is not None:
return int(height)
return int(line.box[3] - line.box[1])
def _vote(candidates: Sequence[Tuple[Any, float]]) -> Optional[Any]:
"""Pick a value by agreement across reads, not by one read's optimism.
Several passes over the same card produce several reads of the same field.
OCR confidence alone is a poor guide (measured: a wrong 12.4割引 came back
more confidently than the right 2.4割引), so agreement wins, and confidence
only breaks ties.
"""
if not candidates:
return None
scores: Dict[Any, List[float]] = {}
for value, confidence in candidates:
scores.setdefault(value, []).append(confidence)
return max(scores.items(), key=lambda item: (len(item[1]), sum(item[1]) / len(item[1])))[0]
def parse_tag_fields(lines: Sequence[OcrLine]) -> TagFields:
"""Read a tag's fields from its OCR lines, using the tag layout as a prior."""
fields = TagFields()
if not lines:
return fields
ordered = sorted(lines, key=lambda line: (line.box[1], line.box[0]))
texts = [(line.text.strip(), line) for line in ordered if line.text.strip()]
if not texts:
return fields
header = next((text for text, _ in texts if HEADER_PATTERN.search(text)), None)
if header:
fields.header = header
# Discounts: vote across reads, and keep only a plausible rate (a 12.4割引
# came from a spurious leading 1 in one pass).
discount_candidates: List[Tuple[float, float]] = []
for text, line in texts:
match = DISCOUNT_PATTERN.search(text)
if match:
value = float(match.group(1))
if 0.1 <= value <= 9.9:
discount_candidates.append((value, line.confidence))
fields.discount = _vote(discount_candidates)
decimal_candidates: List[Tuple[float, float]] = [] # (value, line confidence)
for text, line in texts:
strict = DECIMAL_PATTERN.findall(text)
for whole, fraction in strict:
value = _to_float(whole, fraction)
if 10.0 <= value <= 100000.0:
decimal_candidates.append((value, line.confidence))
if not strict and not SIZE_PATTERN.search(text):
for whole, fraction in LOOSE_DECIMAL_PATTERN.findall(text):
value = _to_float(whole, fraction if len(fraction) == 2 else fraction + "0")
if 10.0 <= value <= 100000.0:
decimal_candidates.append((value, line.confidence * 0.8))
fields.unit_price = _vote(decimal_candidates)
# The name: the longest line that is not a number, a header, or a price.
def looks_like_name(text: str) -> bool:
if HEADER_PATTERN.search(text) or DISCOUNT_PATTERN.search(text):
return False
if DECIMAL_PATTERN.search(text):
return False
letters = sum(1 for char in text if not char.isdigit() and char not in " .,()（）<>＜＞%¥円")
return letters >= 4
name_candidates = [(text, line) for text, line in texts if looks_like_name(text)]
if name_candidates:
# The longest read is the most complete one; ties go to the more confident.
name_candidates.sort(key=lambda item: (-len(item[0]), -item[1].confidence))
fields.name = name_candidates[0][0]
# Layout, not digit count, separates the price from the JAN: the price is the
# LARGEST text on the tag (its line is roughly twice as tall as the small
# print), and the JAN's last three digits sit on a small line of their own.
# Getting this order wrong made a 269 price look like a JAN and disappear.
integer_lines: List[Tuple[Any, List[str]]] = []
for text, line in texts:
if DECIMAL_PATTERN.search(text) or SIZE_PATTERN.search(text):
continue
numbers = [n for n in INTEGER_PATTERN.findall(text) if n not in NOISE_NUMBERS]
if numbers:
integer_lines.append((line, numbers))
price_line = None
measured = None
if integer_lines:
price_line, numbers = max(integer_lines, key=lambda item: _line_height(item[0]))
tall = max(_line_height(line) for line, _ in integer_lines)
# Only accept a price from a line that is actually the big print.
if _line_height(price_line) >= 0.75 * tall:
measured = float(max(numbers, key=int))
# The JAN's last three digits: a standalone 3-digit number on a small line
# (never the price's line), with package sizes excluded.
jan_candidates: List[Tuple[str, float]] = []
for line, numbers in integer_lines:
if line is price_line:
continue
for candidate in numbers:
if len(candidate) != 3 or candidate == "000":
continue
jan_candidates.append((candidate, line.confidence))
if jan_candidates:
jan_candidates.sort(key=lambda item: -item[1])
fields.jan = jan_candidates[0][0]
# The tax relation: the small number and the big one are linked. When the two
# agree, the price is measured; when the big digits cannot be read but the
# small ones can, the price is derived from them and marked as such.
derived: Optional[float] = None
if fields.unit_price:
for ratio in TAX_RATIOS:
candidate = fields.unit_price / ratio
rounded = round(candidate)
if abs(candidate - rounded) <= 0.02 and 10 <= rounded <= 100000:
derived = float(rounded)
break
if measured is not None and derived is not None:
if abs(measured - derived) <= max(2.0, 0.01 * derived):
fields.price, fields.price_source = int(round(derived)), "measured"
fields.flags.append("tax_consistent")
else:
# The big stylised digits and the small clean number disagree. The
# small one is the more reliable read (it is clean, small text that the
# OCR reads well), so publish its value and say where it came from.
fields.price, fields.price_source = int(round(derived)), "derived"
fields.flags.append("tax_mismatch")
elif derived is not None:
fields.price, fields.price_source = int(round(derived)), "derived"
fields.flags.append("price_from_unit")
elif measured is not None:
fields.price, fields.price_source = int(round(measured)), "measured"
if fields.price is None:
fields.flags.append("no_price")
if fields.unit_price is None:
fields.flags.append("no_unit_price")
if fields.jan is None:
fields.flags.append("no_jan")
# Confidence: the anchors we actually trust, not the OCR's own optimism.
score = 0.0
if fields.price is not None:
score += 0.5 if fields.price_source == "measured" else 0.35
if fields.jan is not None:
score += 0.25
if fields.name is not None:
score += 0.15
if fields.discount is not None:
score += 0.1
fields.confidence = min(1.0, score)
return fields
def card_from_lines(lines: Sequence[OcrLine]) -> TagCard:
return TagCard(box=card_box(lines), fields=parse_tag_fields(lines), lines=list(lines))
# --------------------------------------------------------------------------- reading
def read_card_lines(
engine: Any,
image: Any,
box: Tuple[int, int, int, int],
margin: int = 10,
scales: Sequence[float] = (1.0, 2.0, 3.0),
variants: Sequence[str] = ("gray", "red"),
) -> List[OcrLine]:
"""OCR one card several ways and pool the reads.
The fields on a tag are printed at very different sizes: the price is huge
and stylised, the JAN and the tax-included price are small and clean. A single
pass at one scale reads one of them well and the other badly, so the card is
read at three scales and in the red channel (where red-on-white text becomes
high contrast), and the results are pooled for voting.
"""
import cv2
height, width = image.shape[:2]
x1 = max(0, int(box[0]) - margin)
y1 = max(0, int(box[1]) - margin)
x2 = min(width, int(box[2]) + margin)
y2 = min(height, int(box[3]) + margin)
crop = image[y1:y2, x1:x2]
if crop.size == 0:
return []
pooled: List[OcrLine] = []
for variant in variants:
if variant == "red":
prepared = cv2.cvtColor(255 - crop[:, :, 2], cv2.COLOR_GRAY2BGR)
elif variant == "green":
prepared = cv2.cvtColor(crop[:, :, 1], cv2.COLOR_GRAY2BGR)
else:
prepared = crop
for scale in scales:
scaled = prepared if scale == 1.0 else cv2.resize(
prepared, None, fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC
)
result = engine.read(scaled)
for line in result.lines:
text = (line.text or "").strip()
if not text:
continue
pooled.append(
OcrLine(
text=text,
box=(
x1 + int(line.box[0] / scale),
y1 + int(line.box[1] / scale),
x1 + int(line.box[2] / scale),
y1 + int(line.box[3] / scale),
),
confidence=float(line.confidence),
)
)
return pooled

74
recognization/code/tags.py
View file

@ -33,6 +33,13 @@ if current_dir not in sys.path:
sys.path.insert(0, current_dir)
from ocr import OcrEngine, OcrLine, OcrResult, OcrUnavailable # noqa: E402
from tag_cards import ( # noqa: E402
TagFields as TagFieldsModel,
best_tag_rows,
card_from_lines,
group_lines_into_cards,
read_card_lines,
)
TAG_STATE_READABLE = "readable"
TAG_STATE_UNREADABLE = "unreadable"
@ -71,24 +78,10 @@ class RailBand:
product_ids: List[int] = field(default_factory=list)
@dataclass
class TagFields:
price: Optional[int] = None
regular_price: Optional[int] = None
discount: Optional[float] = None
unit_price: Optional[float] = None
name: Optional[str] = None
flags: List[str] = field(default_factory=list)
def as_dict(self) -> Dict[str, Any]:
return {
"price": self.price,
"regular_price": self.regular_price,
"discount": self.discount,
"unit_price": self.unit_price,
"name": self.name,
"flags": list(self.flags),
}
# The field model lives with the card reader; the pipeline consumes it unchanged.
# It adds the JAN's last three digits (the identity anchor printed on a Japanese
# tag) and records whether a price was measured or derived from the tax relation.
TagFields = TagFieldsModel
@dataclass
@ -476,7 +469,7 @@ class TagService:
)
for line in result.lines
]
candidates = cluster_tags(lines, band, image.shape)
candidates = self._card_candidates(image, lines, band, scale_bias)
with self._lock:
self._pass_id += 1
pass_id = self._pass_id
@ -489,6 +482,49 @@ class TagService:
with self._lock:
self._pending_bands.discard(key)
def _card_candidates(
self,
image: Any,
lines: Sequence[OcrLine],
band: RailBand,
scale_bias: float,
) -> List[Dict[str, Any]]:
"""Locate tag cards in a band, then read each card on its own.
The band read only *finds* the cards (its line boxes are enough to group
them); each card is then re-read at several scales and in the red channel,
and its fields come from the pooled reads. That is what makes a tag's box
line up with the printed tag and its values belong to the right card.
"""
rows = best_tag_rows(lines, keep=1)
if not rows:
return []
cards: List[Dict[str, Any]] = []
for group in group_lines_into_cards(rows[0]):
seed = card_from_lines(group)
box = seed.box
width = box[2] - box[0]
height = max(1, box[3] - box[1])
if width < TAG_MIN_WIDTH_RATIO * image.shape[1]:
continue
aspect = width / float(height)
if not (TAG_MIN_ASPECT <= aspect <= TAG_MAX_ASPECT):
continue
pooled = read_card_lines(self.engine, image, box, scales=(1.0, 2.0 * max(1.0, scale_bias)))
card = card_from_lines(pooled) if pooled else seed
card.box = box
if not card.fields.is_readable() and card.fields.price is None:
continue
cards.append(
{
"box": list(box),
"lines": card.lines,
"fields": card.fields,
"band_index": band.row_index,
}
)
return cards
def _band_settled(self, row_index: int) -> bool:
row_tags = [tag for tag in self._tags.values() if tag.band_index == row_index]
if not row_tags:

6
recognization/code/test_ocr_tags.py
View file

@ -210,7 +210,11 @@ class TagServiceTests(unittest.TestCase):
def reader(crop):
reads["count"] += 1
height, width = crop.shape[:2]
price = "269" if reads["count"] != 2 else "26"
# A card is read several times per pass (multi-scale + red channel), so
# one bad read among many is what the vote is for. (A misread that
# repeats on every read of the same channel would survive the vote -
# that is the documented consistent-misread limit.)
price = "26" if reads["count"] == 2 else "269"
return [
OcrLine("コーヒー", 0.6, [10, 10, 200, 30]),
OcrLine(price, 0.9, [20, 40, 190, 80]),

188
recognization/code/test_tag_cards.py Normal file
View file

@ -0,0 +1,188 @@
"""
Tests for price tags as cards with fields.
Two layers: the pure logic (grouping, layout priors, the tax relation) with
synthetic lines, and one real-model test against a shelf photograph whose tags
were read by eye, so the parser is measured against the printed truth.
"""
from __future__ import annotations
import os
import sys
import unittest
current_dir = os.path.dirname(os.path.abspath(__file__))
project_root = os.path.dirname(current_dir)
if os.path.exists(os.path.join(project_root, "stock")):
os.chdir(project_root)
if current_dir not in sys.path:
sys.path.insert(0, current_dir)
from tag_cards import ( # noqa: E402
OcrLine,
best_tag_rows,
card_from_lines,
cluster_rows,
group_lines_into_cards,
parse_tag_fields,
)
def line(text, x1, y1, x2, y2, confidence=0.9):
return OcrLine(text=text, box=(x1, y1, x2, y2), confidence=confidence)
def a_tag(name="キューピー ゼロ ノンコレステロール＜310g＞", jan="357", price="269",
unit="290.52", discount="2.4割引", origin=(0, 0)):
"""A tag's printed lines, laid out the way a real one is."""
ox, oy = origin
return [
line("毎日お買得", ox + 20, oy + 5, ox + 150, oy + 22, 0.5),
line(name, ox + 10, oy + 26, ox + 240, oy + 44, 0.3),
line(jan, ox + 12, oy + 48, ox + 46, oy + 62, 0.3),
line(discount, ox + 60, oy + 50, ox + 140, oy + 64, 0.3),
line(price, ox + 30, oy + 60, ox + 130, oy + 96, 0.6), # the big print
line(unit, ox + 140, oy + 66, ox + 210, oy + 80, 0.5),
]
class GroupingTests(unittest.TestCase):
def test_a_row_of_tags_becomes_separate_cards(self):
lines = a_tag(origin=(0, 0)) + a_tag(origin=(300, 0), jan="400", price="379")
rows = best_tag_rows(lines)
self.assertEqual(len(rows), 1, "one rail line of tags")
cards = group_lines_into_cards(rows[0])
self.assertEqual(len(cards), 2, "the gap between tags separates them")
def test_packaging_text_above_the_rail_is_its_own_row(self):
# Product text sits above the rail; a tag row must be chosen on its own.
product = [line("キユーピー マヨネーズ 450g", 100, 0, 320, 18, 0.3)]
lines = product + a_tag(origin=(0, 60))
rows = cluster_rows(lines)
self.assertEqual(len(rows), 2)
chosen = best_tag_rows(lines, keep=1)
self.assertEqual(len(chosen), 1)
self.assertTrue(all(item.box[1] >= 60 for item in chosen[0]),
"the tag row wins, not the packaging row")
class ParsingTests(unittest.TestCase):
def test_the_tax_relation_recovers_a_price_the_big_digits_got_wrong(self):
# Measured failure: the stylised big digits read as 262/252/258 while the
# small clean number read 290.52. 290.52 / 1.08 = 269 exactly.
fields = parse_tag_fields(a_tag(price="262"))
self.assertEqual(fields.price, 269)
self.assertEqual(fields.price_source, "derived")
self.assertIn("tax_mismatch", fields.flags)
def test_an_agreeing_pair_is_reported_as_measured(self):
fields = parse_tag_fields(a_tag(price="269"))
self.assertEqual(fields.price, 269)
self.assertEqual(fields.price_source, "measured")
self.assertIn("tax_consistent", fields.flags)
def test_a_three_digit_price_is_not_mistaken_for_the_jan(self):
# Regression: with the price and the JAN both three digits, the JAN check
# used to claim the price and the tag lost it entirely.
fields = parse_tag_fields(a_tag(price="269", jan="357"))
self.assertEqual(fields.jan, "357")
self.assertEqual(fields.price, 269)
def test_a_package_size_is_not_a_price(self):
fields = parse_tag_fields([
line("キューピー ゼロ ノンコレステロール", 10, 10, 240, 30, 0.3),
line("<310g>", 200, 12, 250, 30, 0.3),
line("357", 12, 48, 46, 62, 0.3),
line("269", 30, 60, 130, 96, 0.6),
])
self.assertEqual(fields.price, 269)
self.assertNotEqual(fields.price, 310)
def test_the_discount_is_read_and_sanity_checked(self):
self.assertEqual(parse_tag_fields(a_tag()).discount, 2.4)
# 12.4割引 is not a real rate; the plausible read wins.
fields = parse_tag_fields(a_tag(discount="12.4割引"))
self.assertIsNone(fields.discount)
def test_the_name_is_the_longest_text_line(self):
fields = parse_tag_fields(a_tag())
self.assertIn("ノンコレステロール", fields.name or "")
def test_a_tag_without_a_price_is_not_readable(self):
fields = parse_tag_fields([line("毎日お買得", 0, 0, 100, 20, 0.5)])
self.assertFalse(fields.is_readable())
self.assertIn("no_price", fields.flags)
def test_the_card_box_is_the_union_of_its_lines(self):
# The box wraps the printed lines (with a little padding), so it lines up
# with the tag instead of with the rail band it was found in.
card = card_from_lines(a_tag(origin=(50, 100)))
self.assertLessEqual(card.box[0], 60) # the leftmost line starts at ox+10
self.assertLessEqual(card.box[1], 105)
self.assertGreaterEqual(card.box[2], 260)
self.assertGreaterEqual(card.box[3], 190)
class RealShelfTagTests(unittest.TestCase):
"""One measured row of a real LOPIA shelf (add/image6.jpg), read by eye."""
# (x centre, jan, price, tax-included price, discount, name fragment)
EXPECTED = [
(175, "357", 269, 290.52, 2.4, "ノンコレステロール"),
(555, "400", 379, 409.32, 0.5, None),
(975, "248", 199, 214.92, 1.9, "ラッキーマヨネーズ"),
]
def test_a_real_rail_reads_its_tags(self):
import cv2
from ocr import VisionOcr
from tag_cards import read_card_lines
image_path = "add/image6.jpg"
if not os.path.exists(image_path):
self.skipTest("demo image not present")
engine = VisionOcr()
if not engine.available():
self.skipTest("no OCR backend on this host")
image = cv2.imread(image_path)
band = image[630:800]
seed_lines = [
OcrLine(text=item.text, box=(item.box[0], item.box[1] + 630, item.box[2], item.box[3] + 630),
confidence=item.confidence)
for item in engine.read(band).lines if item.text.strip()
]
rows = best_tag_rows(seed_lines, keep=1)
self.assertTrue(rows, "the rail must yield a tag row")
cards = []
for group in group_lines_into_cards(rows[0]):
seed = card_from_lines(group)
pooled = read_card_lines(engine, image, seed.box)
card = card_from_lines(pooled) if pooled else seed
card.box = seed.box
cards.append(card)
cards = [card for card in cards if card.fields.price or card.fields.jan]
self.assertGreaterEqual(len(cards), 3, "the readable row carries at least three tags")
correct = 0
for centre_x, jan, price, unit, discount, name in self.EXPECTED:
card = min(cards, key=lambda c: abs(c.centre_x - centre_x))
fields = card.fields
if fields.jan == jan:
correct += 1
if fields.price == price:
correct += 1
if fields.unit_price == unit:
correct += 1
if fields.discount == discount:
correct += 1
if name is None or (fields.name and name in fields.name):
correct += 1
self.assertEqual(correct, 15, f"expected every field of every tag, got {correct} of 15")
if __name__ == "__main__":
unittest.main()