e25fcda67d
Enrolling a packshot from an external set is normal - it is how F-12 was closed - and it silently turns one of those rows into training data. The external sets were scored by an ad-hoc script that never checked, so sku_l06_1.jpg was being reported as a held-out correct accept while being byte-identical to the enrolled view. code/eval_external.py now scores those sets with the same leaked() check the shelf evaluator runs, and refuses a row that has since been enrolled. That check needed fixing first. Its 64x64 signature fires on two different photographs of the same product: the l09 rear label from a retailer page sits 19.5 from the enrolled front packshot, under the 20.0 line, while being a different photograph (resized MAD 26.3). A signature hit is now confirmed by correlation - a genuine re-cut, re-encode or rescale correlates at 0.99+, that pair at 0.28 - so the two signals together are decisive where either alone is not. Verified against both directions: identical, q55 re-encode and 700px rescale are caught; the different-photo pair is not. Also adds the van backend (always-argmax, no gate) so the stream comparison in DECISION.md is reproducible rather than asserted, with two tests pinning that it answers where the shipped gate abstains and still refuses an empty score set. External result on the corrected basis: 19 scored, 12 correct, 7 abstain, 0 false accepts, 1 leak excluded. Suite 307 -> 309.
84 lines
No EOL
1.6 KiB
JSON
84 lines
No EOL
1.6 KiB
JSON
{
|
|
"dataset": "other-mayo",
|
|
"catalog": "stock-lopia-clean",
|
|
"note": "External proxy images, not Tier 2. Scored through the shipped LocalLaneAdapter + DecisionPolicy.",
|
|
"rows": [
|
|
{
|
|
"file": "sku_l04_1.jpg",
|
|
"truth": "sku_l04",
|
|
"best": null,
|
|
"confidence": 0.0,
|
|
"status": "rejected",
|
|
"source": null,
|
|
"verdict": "abstain"
|
|
},
|
|
{
|
|
"file": "sku_l04_2.jpg",
|
|
"truth": "sku_l04",
|
|
"best": null,
|
|
"confidence": 0.0,
|
|
"status": "unknown",
|
|
"source": null,
|
|
"verdict": "abstain"
|
|
},
|
|
{
|
|
"file": "sku_l04_3.jpg",
|
|
"truth": "sku_l04",
|
|
"best": null,
|
|
"confidence": 0.3294,
|
|
"status": "ambiguous",
|
|
"source": null,
|
|
"verdict": "abstain"
|
|
},
|
|
{
|
|
"file": "sku_l04_4.jpg",
|
|
"truth": "sku_l04",
|
|
"best": null,
|
|
"confidence": -1.6712,
|
|
"status": "unknown",
|
|
"source": null,
|
|
"verdict": "abstain"
|
|
},
|
|
{
|
|
"file": "sku_l07_1.jpg",
|
|
"truth": "sku_l07",
|
|
"best": "sku_l07",
|
|
"confidence": 0.0,
|
|
"status": "success",
|
|
"source": "text",
|
|
"verdict": "correct"
|
|
},
|
|
{
|
|
"file": "sku_l07_2.jpg",
|
|
"truth": "sku_l07",
|
|
"best": "sku_l07",
|
|
"confidence": -0.5503,
|
|
"status": "success",
|
|
"source": "text",
|
|
"verdict": "correct"
|
|
},
|
|
{
|
|
"file": "sku_l09_1.jpg",
|
|
"truth": "sku_l09",
|
|
"best": "sku_l09",
|
|
"confidence": 27.9546,
|
|
"status": "success",
|
|
"source": "local",
|
|
"verdict": "correct"
|
|
},
|
|
{
|
|
"file": "sku_l09_2.jpg",
|
|
"truth": "sku_l09",
|
|
"best": "sku_l09",
|
|
"confidence": 0.975,
|
|
"status": "success",
|
|
"source": "text",
|
|
"verdict": "correct"
|
|
}
|
|
],
|
|
"scored": 8,
|
|
"leaked": 0,
|
|
"correct": 4,
|
|
"abstain": 4,
|
|
"wrong": 0
|
|
} |