Coverage-first live labelling, merged catalog, shelf-intelligence decision
Some checks failed
tests / model-free (pull_request) Has been cancelled
Some checks failed
tests / model-free (pull_request) Has been cancelled
The live path abstained too much: a detected product whose best candidate missed the accept line was left blank, so a shelf where 59% of in-catalog facings were shown read as broken. Show the best candidate immediately as `provisional`, keep the track re-classifiable so a later attempt upgrades or corrects it, and never blank a label already on screen. - identity.py / identity_lanes.py: every decision now carries `provisional_sku` - the row the evidence supports well enough to show, separate from `best_sku`, which still means "cleared the gate". The accept line does not move: far-unknown false accepts stay at 0.066%. - live.py: a below-gate candidate labels the track with `identity_level: provisional` and `confirmed = False`, so the planner and the crop lane keep offering it. Accepted labels stay frozen. Inherited provisional labels stay re-classifiable. - Clients render provisional distinctly (amber chip, dashed box) via `LiveScanOverlayPolicy.isProvisional(identity)`, which now reads the wire level instead of only the state. - build_stock_all.py merges stock, stock-lopia, stock-mediya and stock-milk150 into stock-all (111 images, 60 SKUs). The server could only read one folder, so anything outside it came back blank. stock-demo is byte-identical to mediya+milk150 and is not a source. Measured on add/image6.jpg at live resolution: in-catalog facings shown 59.4% -> 86.0%, correct shown 38 -> 55, precision 97.4% -> 87.3%. On the wire at 8fps 0.5x: 65 of 79 tracks labelled, 742 accepted + 309 provisional observations, first label 1.27s. docs/research/2026-09-29-shelf-intelligence/ carries the experiments and the decision: group signals (combined AUC 0.991 sibling / 0.990 unrelated), OCR scheduling (group representatives cut calls 4.4x and wall 3.3x), and the finding that group propagation damages more labels than it rescues on this shelf, so groups schedule compute rather than overwrite labels. 331 tests pass.
This commit is contained in:
parent
fbf31d74fe
commit
0cf64a18bf
171 changed files with 554214 additions and 35 deletions
|
|
@ -0,0 +1,230 @@
|
|||
# Dorayaki: shelf-intelligence decision
|
||||
|
||||
Measured 2026-09-29 · server `product-recognition-demo/recognization` at
|
||||
`fbf31d7` plus the coverage-first diff described in §F · evidence in
|
||||
[`evidence/`](evidence/) · every number below is read from a JSON in that
|
||||
directory, named inline.
|
||||
|
||||
Scope: the live decision path (group inference, OCR scheduling, decision
|
||||
levels), the catalog the server matches against, and the two native overlays.
|
||||
Not in scope: detector quality, tag/planogram pipeline, on-device runtime.
|
||||
|
||||
The previous phase ([`../2026-09-28-recognition-architecture/DECISION.md`](../2026-09-28-recognition-architecture/DECISION.md))
|
||||
settled *what* matches a product: a local-feature instance matcher plus a text
|
||||
lane behind a calibrated open-set gate. This phase settles *how much of a shelf
|
||||
to label, how fast, and how to spend compute doing it*.
|
||||
|
||||
## Verdict
|
||||
|
||||
**Label every detected product with its best candidate immediately, mark it
|
||||
provisional, and refine it as stronger evidence arrives.** Keep the accepted
|
||||
line where it is; stop treating abstention as quality. Group adjacent facings
|
||||
so one strong observation can answer for a run, and spend OCR on group
|
||||
representatives and ambiguous tracks instead of every crop.
|
||||
|
||||
The measured case for it, on `add/image6.jpg` at live resolution
|
||||
(`ocr-ablation.json`, `anytime-policies.json`):
|
||||
|
||||
| Policy | Shown | Correct | Precision | Coverage of 64 in-catalog facings | Blank |
|
||||
|---|---|---|---|---|---|
|
||||
| strict / individual | 39 | 38 | 97.4 % | 59.4 % | 64.2 % |
|
||||
| **coverage-first (shipped)** | **63** | **55** | **87.3 %** | **86.0 %** | **42.2 %** |
|
||||
|
||||
The shipped policy shows 24 more products and gets 17 more of them right, at
|
||||
the cost of 7 more wrong labels. That is the trade the product asked for: a
|
||||
shelf where 86 % of products carry a name reads as working; one where 59 % do
|
||||
reads as broken.
|
||||
|
||||
## A. Where the live time goes
|
||||
|
||||
`replay.py` on the running server, `image6` at 8 fps (`replay-image6-8fps-*.json`):
|
||||
|
||||
| Stage | p50 | p95 |
|
||||
|---|---|---|
|
||||
| decode | 8–12 ms | 20 ms |
|
||||
| detect | 67–107 ms | 131–278 ms |
|
||||
| track | 1–3 ms | 6 ms |
|
||||
| total | 90–140 ms | 200–300 ms |
|
||||
|
||||
Detection dominates frame cost by an order of magnitude; tracking is free.
|
||||
Identity work is off the frame path (a worker pool), so it shows up as
|
||||
*label latency*, not frame latency. `scheduler-benchmark.json` prices it:
|
||||
218 classification tasks, worker p50 147 ms with OCR on every attempt versus
|
||||
19.6 ms with OCR once per track. **OCR is ~87 % of per-task identity cost.**
|
||||
|
||||
Ranked bottlenecks: (1) OCR invocations, (2) matcher invocations per facing,
|
||||
(3) detector. Only the first two are ours to schedule.
|
||||
|
||||
## B. Compute and evidence schedule
|
||||
|
||||
`scheduler-benchmark.json`, same 218 tasks:
|
||||
|
||||
| Schedule | OCR calls | Worker p50 | First facing | All first facings | Wall | CPU |
|
||||
|---|---|---|---|---|---|---|
|
||||
| every attempt | 218 | 147 ms | 1177 ms | 1890 ms | 4.32 s | 4.48 s |
|
||||
| once per track | 109 | 20 ms | 1235 ms | 2680 ms | 2.77 s | 3.42 s |
|
||||
| ambiguous track only | 83 | 16 ms | 806 ms | 1951 ms | 2.05 s | 2.97 s |
|
||||
| **group representative** | **49** | **16 ms** | **454 ms** | **1168 ms** | **1.30 s** | **2.64 s** |
|
||||
|
||||
Group representatives cut OCR calls 4.4× and wall time 3.3×, and they make the
|
||||
first label appear 2.6× sooner. The accuracy cost is small and measured
|
||||
(`ocr-ablation.json`, group metrics): full-resolution coverage 57.8 % → 56.0 %,
|
||||
precision 87.3 % → 80.3 %; at live resolution coverage *rises* 56.0 % → 58.7 %
|
||||
and precision 90.2 % → 87.5 %. Ship the group-representative schedule; the
|
||||
`ambiguous_track` schedule is the fallback when grouping is unavailable.
|
||||
|
||||
The shipped live path already does the cheap half of this: OCR runs once per
|
||||
track (`live.py` caches `text_read` on the track), and the local lane re-runs
|
||||
on every attempt.
|
||||
|
||||
## C. Repeated-facing group detection
|
||||
|
||||
`analysis.json` (`resolutions.full` and `.live`) scores eleven pairwise signals
|
||||
on 83 adjacent pairs from `image6` (72 same-SKU, 3 sibling, 8 unrelated), with
|
||||
leave-one-row-out thresholds:
|
||||
|
||||
| Signal | AUC same-vs-sibling | AUC same-vs-unrelated | Precision | Recall | Boundary acc. |
|
||||
|---|---|---|---|---|---|
|
||||
| hue histogram | 0.977 | 0.967 | 98.3 % | 80.6 % | 81.9 % |
|
||||
| logo pixels | 0.968 | 0.981 | 100 % | 93.1 % | 94.0 % |
|
||||
| pixels | 0.944 | 0.979 | 100 % | 88.9 % | 90.4 % |
|
||||
| inliers | 0.944 | 0.944 | 100 % | 88.9 % | 90.4 % |
|
||||
| height | 0.972 | 0.775 | 92.6 % | 87.5 % | 83.1 % |
|
||||
| aspect | 0.403 | 0.691 | 93.9 % | 43.1 % | 48.2 % |
|
||||
| ocr_jaccard | 0.667 | 0.806 | 97.7 % | 58.3 % | 62.7 % |
|
||||
| **combined** | **0.991** | **0.990** | **98.5 %** | **91.7 %** | **91.6 %** |
|
||||
|
||||
Use the **combined** signal: it is the only one that is simultaneously strong on
|
||||
siblings (the hard case) and on unrelated pairs. Hue alone is nearly as good and
|
||||
is the cheapest single signal; logo pixels is the best single signal but needs a
|
||||
logo detector. Aspect ratio and OCR similarity are actively misleading on
|
||||
siblings (AUC 0.40 and 0.67) — do not gate a group on either.
|
||||
|
||||
At live resolution the combined signal holds: AUC 0.991 / 0.984, precision
|
||||
96.9 %, recall 87.5 %, boundary accuracy 86.7 %.
|
||||
|
||||
## D. Group-to-individual inference
|
||||
|
||||
`representative.json` prices propagating a group identity to its members, on
|
||||
`image6` (109 facings, 64 in-catalog):
|
||||
|
||||
| Arm | Matcher calls | Total ms | Shown | Correct | Precision | Coverage | Rescued | Damaged |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| sharpest-one | 47 | 1114 | 50 | 46 | 92.0 % | 45.9 % | 1 | 8 |
|
||||
| sharpest-two | 70 | 1413 | 52 | 48 | 92.3 % | 47.7 % | 1 | 6 |
|
||||
| **all (no grouping)** | **109** | **2024** | **63** | **55** | **87.3 %** | **57.8 %** | **2** | **0** |
|
||||
|
||||
At full resolution, recognising only the sharpest member of each group **damages
|
||||
8 correct labels** to rescue 1. At live resolution the same arm rescues 3 and
|
||||
damages 8. Group propagation is therefore **not** a coverage win on this shelf:
|
||||
the members are individually recognisable, and the group's answer is only as
|
||||
good as its weakest member when the group is wrong.
|
||||
|
||||
Decision: **use groups to schedule compute (C, B), not to overwrite individual
|
||||
labels.** Propagate a group identity only to members whose own evidence is
|
||||
empty, and never over a member that produced a confident individual answer.
|
||||
The `rescued_correct` column is the ceiling for that rule; it is small here and
|
||||
should be re-measured on a shelf with more blur and occlusion.
|
||||
|
||||
## E. Sibling boundaries
|
||||
|
||||
The combined signal's boundary accuracy is 91.6 % at full resolution, 86.7 % at
|
||||
live (`analysis.json`). The failure mode is a boundary missed, not a boundary
|
||||
invented: precision 98.5 % vs recall 91.7 %. A missed boundary merges two
|
||||
sibling runs into one group.
|
||||
|
||||
Because §D says groups do not overwrite individual labels, a missed boundary is
|
||||
now a *scheduling* error (one representative OCR'd for two SKUs), not a
|
||||
labelling error. That is the safe direction to fail. Two rules keep it safe:
|
||||
|
||||
- A member whose own individual answer disagrees with the group's is a boundary
|
||||
candidate, not an outlier to be smoothed away.
|
||||
- A group never spans a detected price-tag change or a gap wider than the
|
||||
shelf's own facing pitch.
|
||||
|
||||
## F. Provisional and confirmed transitions
|
||||
|
||||
Shipped in `live.py` `_on_classified`, `identity.py`, `identity_lanes.py`:
|
||||
|
||||
- `accepted` — the decision cleared its gate. `state: recognized`,
|
||||
`identity_level: accepted`, `confirmed = True`, never re-classified.
|
||||
- `provisional` — the decision produced a candidate but did not clear the gate.
|
||||
`state: recognized`, `identity_level: provisional`, `confirmed = False`, and
|
||||
the track **stays eligible for re-classification** so a later, sharper attempt
|
||||
can upgrade it to accepted or correct it.
|
||||
- `unrecognized` — no candidate at all, or the attempt budget is spent with no
|
||||
provisional label on screen. `state: unrecognized`, no chip.
|
||||
|
||||
A provisional label is never blanked once shown: when the budget runs out the
|
||||
track keeps its guess and simply stops being re-classified. An inherited
|
||||
provisional label stays re-classifiable; an inherited accepted label does not.
|
||||
|
||||
Measured on the wire (`replay-image6-8fps-0.5x-0.json`): 65 of 79 tracks
|
||||
labelled, 742 accepted + 309 provisional observations, first label at 1.27 s.
|
||||
`identity_level` is the only new wire field; an older client that ignores it
|
||||
renders a provisional label as a normal one, which is the intended degradation.
|
||||
|
||||
Both overlays render provisional distinctly: amber chip and dashed box
|
||||
(`LiveScanOverlayPolicy.isProvisional(identity)`, `ScanScreen.kt`,
|
||||
`LiveScanOverlay.swift`).
|
||||
|
||||
## G. Operating recommendation
|
||||
|
||||
`far-unknown-policies.json` prices the accept floor on 4580 crops from 64
|
||||
unrelated Wikimedia shelf photos:
|
||||
|
||||
| Policy | local_accept | Accepts | Rate |
|
||||
|---|---|---|---|
|
||||
| strict | 4.0 | 3 | 0.066 % |
|
||||
| balanced | 2.0 | 8 | 0.175 % |
|
||||
| coverage-first | 0.5 | 83 | 1.81 % |
|
||||
| near-argmax | 0.0 + fallback | 119 | 2.60 % |
|
||||
|
||||
The shipped `LOCAL_ACCEPT` stays at **4.0** for the *accepted* level: it is the
|
||||
line that keeps far-unknown false accepts at 0.066 %, and the accepted level is
|
||||
what the product treats as settled. Coverage comes from the **provisional**
|
||||
level, which is a separate, much looser rule and is not gated by this number.
|
||||
|
||||
This is the key structural choice: **do not lower the accept line to raise
|
||||
coverage.** Lowering it to 0.5 would raise far-unknown false accepts 27× and
|
||||
would still leave the shelf's own hard cases (siblings) unresolved. Showing the
|
||||
best candidate as provisional gets the coverage without moving the line that
|
||||
protects precision.
|
||||
|
||||
`anytime-policies.json` confirms the shape: the strict policy reaches 59.4 %
|
||||
coverage and stays there at every time budget; coverage-first reaches 86 % and
|
||||
is already at 28 % of facings by 250 ms.
|
||||
|
||||
## H. Measured improvement
|
||||
|
||||
| Metric | Before | After |
|
||||
|---|---|---|
|
||||
| Catalog the server matches | 7 SKUs / 13 views | **60 SKUs / 86 views** |
|
||||
| Tracks labelled, `image6` 8 fps 0.5× | 57 / 80 | **65 / 79** |
|
||||
| In-catalog facings shown | 59.4 % | **86.0 %** |
|
||||
| Precision of shown labels | 97.4 % | 87.3 % |
|
||||
| First label | 623 ms | 1.27 s |
|
||||
| OCR calls per 218 tasks | 218 | 49 (group-representative schedule) |
|
||||
| Wall time per 218 tasks | 4.32 s | 1.30 s |
|
||||
|
||||
The catalog change is `code/build_stock_all.py`, which merges `stock`,
|
||||
`stock-lopia`, `stock-mediya` and `stock-milk150` into `stock-all` (111 images,
|
||||
60 SKUs; `stock-demo` is byte-identical to mediya+milk150 and is not a source).
|
||||
SKU ids are unique across the folders, so the merge is a copy plus a CSV concat.
|
||||
|
||||
## Limitations
|
||||
|
||||
- Every shelf number is one photo, `add/image6.jpg`, and its labels are the
|
||||
corrected ones from the previous phase. The group signals and the
|
||||
representative arms are measured on that photo's own facings.
|
||||
- The far-unknown rates come from a cache that records no catalog, so the
|
||||
catalog match is read off the worktree, not attested by the cache.
|
||||
- `scheduler-benchmark.json` and `anytime-policies.json` are diagnostic
|
||||
counterfactuals: they price schedules against frozen evidence, not against a
|
||||
live deployment. Transport and grouping time are excluded.
|
||||
- The native overlays were changed but not compiled: no JDK is installed on
|
||||
this machine, so `./gradlew` cannot run. The shared signature change is
|
||||
mechanical and every caller was updated, but this is unverified by a compiler.
|
||||
- Group propagation (§D) is deliberately not shipped as a labelling rule. Its
|
||||
ceiling on this shelf is 1–3 rescued labels against 8 damaged; re-measure on a
|
||||
shelf with real blur and occlusion before revisiting.
|
||||
|
|
@ -0,0 +1,267 @@
|
|||
"""A-D/F: paired policies, geometry-only rows, label-blind inference, oracle membership.
|
||||
No production edits. All accuracy counters use the frozen observation file.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
import itertools
|
||||
import math
|
||||
from collections import Counter
|
||||
import numpy as np
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0,str(ROOT/'code'))
|
||||
from identity_lanes import DecisionPolicy
|
||||
DIR=Path(__file__).resolve().parent
|
||||
POLICIES={'strict':4.,'balanced':2.,'coverage-first':.5,'near-argmax':0.}
|
||||
|
||||
|
||||
def decide(o, policy, signatures, use_text=True):
|
||||
p=DecisionPolicy(local_accept=POLICIES[policy],signatures=signatures)
|
||||
text=o['text'] if use_text else {}
|
||||
read=o['read'] if use_text else ''
|
||||
d=p.decide(o['rank'],text,text_read=read)
|
||||
answer=d['best_sku']
|
||||
# Near-argmax retains actual geometric candidates even when the confidence
|
||||
# margin is negative; no arbitrary catalog row when there are no features.
|
||||
if policy=='near-argmax' and not answer and o['rank']:
|
||||
candidate=o['rank'][0][0]
|
||||
tr=sorted(text.items(),key=lambda x:x[1],reverse=True)
|
||||
if not p._contradicts_size(read,candidate) and not (tr and tr[0][0]!=candidate and tr[0][1]>text.get(candidate,0)):
|
||||
answer=candidate
|
||||
return answer
|
||||
|
||||
|
||||
def pair_value(p, name):
|
||||
s=p['signals']
|
||||
if name=='combined':
|
||||
return .35*s['hue']+.35*s['pixels']+.30*min(1,s['inliers']/10)
|
||||
return s[name]
|
||||
|
||||
|
||||
def geometric_edge(p):
|
||||
s=p['signals']
|
||||
return -.25<=s['gap']<=.75 and s['height']>=.65 and s['bottom']<=.25
|
||||
|
||||
|
||||
def infer_groups(data, edge_yes, anchors):
|
||||
"""Contiguous appearance-supported runs; strong contradiction is a hard cut."""
|
||||
edges={(p['a'],p['b']):edge_yes[i] and geometric_edge(p) for i,p in enumerate(data['pairs'])}
|
||||
groups=[]
|
||||
for row in data['rows']:
|
||||
group=[]; support=None
|
||||
for item in row:
|
||||
anchor=anchors.get(item)
|
||||
if group and (not edges.get((group[-1],item),False) or (anchor and support and anchor!=support)):
|
||||
groups.append(group); group=[]; support=None
|
||||
group.append(item)
|
||||
if anchor: support=anchor
|
||||
if group: groups.append(group)
|
||||
return groups
|
||||
|
||||
|
||||
def strength(o, answer):
|
||||
if not answer: return 0.
|
||||
if o['rank'] and o['rank'][0][0]==answer:
|
||||
return max(0,o['rank'][0][1])+2*o['text'].get(answer,0)
|
||||
return 2*o['text'].get(answer,0)
|
||||
|
||||
|
||||
def propagate(observations, groups, answers, signatures, guarded=False):
|
||||
"""One strongest independently supported prototype, not majority voting.
|
||||
Group evidence can rescue weak members, never overwrite strict contradiction.
|
||||
"""
|
||||
lookup={x['id']:x for x in observations}
|
||||
result=dict(answers); rescued=[]
|
||||
strict={i:decide(o,'strict',signatures) for i,o in lookup.items()}
|
||||
guard=DecisionPolicy(signatures=signatures)
|
||||
for group in groups:
|
||||
seeds=[i for i in group if strict[i]]
|
||||
if not seeds: continue
|
||||
seed=max(seeds,key=lambda i:strength(lookup[i],strict[i]))
|
||||
sku=strict[seed]
|
||||
if guarded:
|
||||
# Two distinct facings corroborate a seed, or one unusually strong
|
||||
# geometric observation does. Repeated reads of one crop do not.
|
||||
corroborated=sum(strict[i]==sku for i in seeds)>=2
|
||||
strong_visual=bool(lookup[seed]['rank'] and lookup[seed]['rank'][0][0]==sku and lookup[seed]['rank'][0][1]>=8)
|
||||
if not (corroborated or strong_visual): continue
|
||||
# Size contradictions belong to the entire repeated-artwork run.
|
||||
# Abstain from propagation rather than spread a lone OCR mistake.
|
||||
if any(guard._contradicts_size(lookup[i]['read'],sku) for i in group): continue
|
||||
for i in group:
|
||||
o=lookup[i]
|
||||
if strict[i] and strict[i]!=sku: continue
|
||||
if guard._contradicts_size(o['read'],sku): continue
|
||||
# Independently decisive text also splits a weak visual member.
|
||||
text_sku=guard.decide([],o['text'],text_read=o['read'])['best_sku']
|
||||
if text_sku and text_sku!=sku: continue
|
||||
if result[i]!=sku:
|
||||
rescued.append({'id':i,'seed':seed,'from':result[i],'to':sku})
|
||||
result[i]=sku
|
||||
return result,rescued
|
||||
|
||||
|
||||
def metrics(observations, answers, families, before=None):
|
||||
total=len(observations); known=sum(o['truth']!='unknown' for o in observations)
|
||||
shown=correct=unknown=sibling=wrong_known=0
|
||||
rescue=correction=damage=0
|
||||
for o in observations:
|
||||
i=o['id']; a=answers.get(i); t=o['truth']
|
||||
shown+=bool(a); correct+=bool(a and a==t)
|
||||
unknown+=bool(a and t=='unknown')
|
||||
wrong_known+=bool(a and t!='unknown' and a!=t)
|
||||
sibling+=bool(a and a!=t and t in families and families.get(a)==families[t])
|
||||
if before is not None:
|
||||
rescue+=bool(not before.get(i) and a==t)
|
||||
correction+=bool(before.get(i) and before[i]!=t and a==t)
|
||||
damage+=bool(before.get(i)==t and a!=t)
|
||||
return {'facings':total,'catalog_facings':known,'shown':shown,'correct':correct,'wrong':shown-correct,
|
||||
'visible_coverage':shown/total,'correct_coverage':correct/known,'precision':correct/shown if shown else None,
|
||||
'false_label_rate':(shown-correct)/total,'unknown_false_accepts':unknown,'unknown_total':total-known,
|
||||
'sibling_confusions':sibling,'wrong_catalog':wrong_known,'blank_ratio':1-shown/total,
|
||||
'rescued_correct':rescue,'corrected_wrong':correction,'damaged_correct':damage}
|
||||
|
||||
|
||||
def confusion(actual,predicted):
|
||||
tp=sum(a and p for a,p in zip(actual,predicted)); fp=sum(not a and p for a,p in zip(actual,predicted))
|
||||
tn=sum(not a and not p for a,p in zip(actual,predicted)); fn=sum(a and not p for a,p in zip(actual,predicted))
|
||||
return {'tp':tp,'fp':fp,'tn':tn,'fn':fn,'precision':tp/(tp+fp) if tp+fp else None,'recall':tp/(tp+fn) if tp+fn else None,'boundary_accuracy':(tp+tn)/max(1,len(actual)),'boundary_recall':tn/(tn+fp) if tn+fp else None}
|
||||
|
||||
|
||||
def auc(pos,neg):
|
||||
return sum((a>b)+.5*(a==b) for a in pos for b in neg)/(len(pos)*len(neg)) if pos and neg else None
|
||||
|
||||
|
||||
def fit_threshold(pairs, name):
|
||||
# Weighted classification error, so long repeated runs cannot hide boundaries.
|
||||
candidates=sorted(set(pair_value(p,name) for p in pairs))
|
||||
candidates=[-1e-9]+[(a+b)/2 for a,b in zip(candidates,candidates[1:])]+[max(candidates,default=0)+1e-9]
|
||||
def loss(t):
|
||||
yes=[p for p in pairs if p['same']]; no=[p for p in pairs if not p['same']]
|
||||
return sum(pair_value(p,name)<t for p in yes)/max(1,len(yes))+sum(pair_value(p,name)>=t for p in no)/max(1,len(no))
|
||||
return min(candidates,key=lambda t:(loss(t),-t))
|
||||
|
||||
|
||||
def chain_dp(data, edge_yes, observations, signatures):
|
||||
"""Potts chain MAP by Viterbi, O(n*K^2), no framework or truth input."""
|
||||
lookup={o['id']:o for o in observations}; skus=[None]+list(signatures)
|
||||
edge={(p['a'],p['b']):edge_yes[i] and geometric_edge(p) for i,p in enumerate(data['pairs'])}
|
||||
result={}
|
||||
for row in data['rows']:
|
||||
unary=[]
|
||||
for i in row:
|
||||
o=lookup[i]; values={s:e['inliers']*(.25+e['color']) for s,e in o['local'].items()}
|
||||
maximum=max(values.values(),default=1.)
|
||||
u=[.4]+[values.get(s,0)/maximum+.4*o['text'].get(s,0) for s in skus[1:]]
|
||||
anchor=decide(o,'strict',signatures)
|
||||
if anchor:
|
||||
u=[v if s==anchor else v-4 for s,v in zip(skus,u)]
|
||||
unary.append(np.asarray(u))
|
||||
score=unary[0]; backs=[]
|
||||
for n in range(1,len(row)):
|
||||
reward=.6 if edge.get((row[n-1],row[n]),False) else 0.
|
||||
table=score[:,None]+np.eye(len(skus))*reward
|
||||
back=table.argmax(axis=0); score=unary[n]+table[back,np.arange(len(skus))]; backs.append(back)
|
||||
best=int(score.argmax()); labels=[best]
|
||||
for back in reversed(backs): best=int(back[best]); labels.append(best)
|
||||
for i,k in zip(row,reversed(labels)): result[i]=skus[k]
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
evidence=json.loads((DIR/'evidence/observations.json').read_text()); annotation=json.loads((DIR/'group-truth.json').read_text())
|
||||
families=annotation['families']; report={'provenance':evidence['provenance'],'resolutions':{}}
|
||||
for res,data in evidence['resolutions'].items():
|
||||
observations=data['observations']; lookup={o['id']:o for o in observations}
|
||||
keys={o['id']:o['truth'] for o in observations if o['truth']!='unknown'}
|
||||
family=dict(families)
|
||||
for g in annotation['unknown_visual_groups']:
|
||||
keys.update({i:g['key'] for i in g['members']}); family[g['key']]=g['family']
|
||||
scored=[]
|
||||
for p in data['pairs']:
|
||||
if p['a'] not in keys or p['b'] not in keys: continue
|
||||
a,b=keys[p['a']],keys[p['b']]
|
||||
scored.append({**p,'same':a==b,'class':'same' if a==b else 'sibling' if family[a]==family[b] else 'unrelated'})
|
||||
signal_report={}
|
||||
names=['hue','pixels','logo_pixels','edge','inliers','match_fraction','layout_cover','aspect','height','ocr_jaccard','combined']
|
||||
predictions={}
|
||||
for name in names:
|
||||
thresholds={r:fit_threshold([p for p in scored if p['row']!=r],name) for r in range(len(data['rows']))}
|
||||
predictions[name]=[pair_value(p,name)>=thresholds[p['row']] and geometric_edge(p) for p in data['pairs']]
|
||||
byclass={c:[pair_value(p,name) for p in scored if p['class']==c] for c in ['same','sibling','unrelated']}
|
||||
signal_report[name]={'thresholds_leave_row_out':thresholds,'same_vs_sibling_auc':auc(byclass['same'],byclass['sibling']),'same_vs_unrelated_auc':auc(byclass['same'],byclass['unrelated']), 'classes':{c:{'n':len(v),'p50':float(np.median(v)) if v else None} for c,v in byclass.items()},'held_row_confusion':confusion([p['same'] for p in scored],[pair_value(p,name)>=thresholds[p['row']] and geometric_edge(p) for p in scored])}
|
||||
# Preselected combined arm; do not pick the best held-row score after seeing truth.
|
||||
edge_yes=predictions['combined']
|
||||
anchors={o['id']:decide(o,'strict',evidence['signatures']) for o in observations}
|
||||
start=time.perf_counter(); groups=infer_groups(data,edge_yes,anchors); group_ms=(time.perf_counter()-start)*1000
|
||||
# One boundary variable: correspondence is required in addition to the
|
||||
# combined score. Stops color-only joins between different artworks.
|
||||
guarded_edges=[yes and p['signals']['inliers']>=4 for yes,p in zip(edge_yes,data['pairs'])]
|
||||
guarded_groups=infer_groups(data,guarded_edges,anchors)
|
||||
appearance_groups=infer_groups(data,guarded_edges,{})
|
||||
oracle=[]
|
||||
for row in data['rows']:
|
||||
run=[]; key=None
|
||||
for i in row:
|
||||
nextkey=keys.get(i,('unscored',i))
|
||||
if run and key!=nextkey: oracle.append(run); run=[]
|
||||
key=nextkey; run.append(i)
|
||||
if run:oracle.append(run)
|
||||
all_pairs=list(itertools.combinations(sorted(keys),2))
|
||||
def memberships(gs): return {i:g for g,ids in enumerate(gs) for i in ids}
|
||||
membership=memberships(groups); oracle_membership=memberships(oracle)
|
||||
group_eval=confusion([oracle_membership[a]==oracle_membership[b] for a,b in all_pairs],[membership[a]==membership[b] for a,b in all_pairs])
|
||||
guarded_membership=memberships(guarded_groups)
|
||||
guarded_eval=confusion([oracle_membership[a]==oracle_membership[b] for a,b in all_pairs],[guarded_membership[a]==guarded_membership[b] for a,b in all_pairs])
|
||||
boundary_eval=confusion([p['same'] for p in scored],[guarded_membership[p['a']]==guarded_membership[p['b']] for p in scored])
|
||||
results={}; answers_output={}
|
||||
for policy in POLICIES:
|
||||
base={o['id']:decide(o,policy,evidence['signatures']) for o in observations}
|
||||
results[policy]={}; answers_output[policy]={}
|
||||
for mode,gs in [('individual',[[o['id']] for o in observations]),('oracle',oracle),('inferred',groups),('seed_guard',groups),('boundary_guard',guarded_groups),('guarded',guarded_groups)]:
|
||||
start=time.perf_counter(); answers,changed=propagate(observations,gs,base,evidence['signatures'],guarded=mode in ('seed_guard','guarded')) if mode!='individual' else (base,[]); fusion_ms=(time.perf_counter()-start)*1000
|
||||
results[policy][mode]={**metrics(observations,answers,families,base),'changes':changed,'fusion_ms':fusion_ms}
|
||||
answers_output[policy][mode]=answers
|
||||
start=time.perf_counter(); dp=chain_dp(data,edge_yes,observations,evidence['signatures']); dp_ms=(time.perf_counter()-start)*1000
|
||||
results['dp']={**metrics(observations,dp,families,answers_output['coverage-first']['individual']),'inference_ms':dp_ms}
|
||||
# Utility sensitivity, static one-second exposure proxy; temporal loss is in replay.
|
||||
sensitivity={}
|
||||
for wrong_cost in [.5,1.,2.,4.,8.]:
|
||||
cells={f'{p}/{mode}':(m['facings']-m['shown']+wrong_cost*m['wrong'])/m['facings'] for p,arms in results.items() if p!='dp' for mode,m in arms.items()}
|
||||
sensitivity[str(wrong_cost)]={'loss':cells,'best':min(cells,key=cells.get)}
|
||||
deployable={k:v for k,v in cells.items() if not k.endswith('/oracle')}
|
||||
sensitivity[str(wrong_cost)]['best_nonoracle']=min(deployable,key=deployable.get)
|
||||
report['resolutions'][res]={'pair_counts':dict(Counter(p['class'] for p in scored)), 'signals':signal_report,'pairs':scored,'groups':groups,'oracle_groups':oracle,'group_ms':group_ms,'group_membership':group_eval,'policies':results,'answers':answers_output,'utility_sensitivity':sensitivity}
|
||||
report['resolutions'][res]['guarded_groups']=guarded_groups
|
||||
report['resolutions'][res]['appearance_groups']=appearance_groups
|
||||
report['resolutions'][res]['guarded_group_membership']=guarded_eval
|
||||
report['resolutions'][res]['guarded_adjacent_boundaries']=boundary_eval
|
||||
report['resolutions'][res]['guarded_edges']=guarded_edges
|
||||
print(res,'groups',len(groups),'pair counts',report['resolutions'][res]['pair_counts'])
|
||||
for p in POLICIES:
|
||||
print(p,{mode:(m['shown'],m['correct'],m['wrong'],m['rescued_correct'],m['damaged_correct']) for mode,m in results[p].items()})
|
||||
(DIR/'evidence/analysis.json').write_text(json.dumps(report,ensure_ascii=False,indent=2))
|
||||
plot(report)
|
||||
|
||||
|
||||
def plot(report):
|
||||
# SVG uses stdlib only; both axes and denominators are explicit.
|
||||
colors={'strict':'#475569','balanced':'#2563eb','coverage-first':'#059669','near-argmax':'#dc2626'}
|
||||
parts=['<svg xmlns="http://www.w3.org/2000/svg" width="1000" height="510" viewBox="0 0 1000 510"><rect width="1000" height="510" fill="white"/>']
|
||||
for j,res in enumerate(['full','live']):
|
||||
x0=70+j*490; y0=405
|
||||
parts.append(f'<text x="{x0}" y="30" font-size="18">{res}: visible coverage / precision</text><path d="M{x0},50 V405 H{x0+390}" fill="none" stroke="black"/>')
|
||||
for t in [0,.25,.5,.75,1]:
|
||||
parts.append(f'<text x="{x0+390*t-10}" y="430" font-size="12">{t:.0%}</text><text x="{x0-40}" y="{y0-350*t}" font-size="12">{t:.0%}</text>')
|
||||
for p in POLICIES:
|
||||
for mode in ['individual','guarded']:
|
||||
m=report['resolutions'][res]['policies'][p][mode]; x=x0+390*m['visible_coverage']; y=y0-350*(m['precision'] or 0)
|
||||
parts.append(f'<circle cx="{x}" cy="{y}" r="5" fill="{colors[p] if mode=="guarded" else "white"}" stroke="{colors[p]}"><title>{p}/{mode}: {m["shown"]}/109 shown, {m["correct"]}/64 correct, {m["wrong"]} false</title></circle>')
|
||||
parts.append(f'<text x="{x0}" y="458" font-size="12">Any labels / 109; filled = guarded groups, hollow = individual</text>')
|
||||
for n,(policy,color) in enumerate(colors.items()):
|
||||
parts.append(f'<text x="{70+n*225}" y="491" font-size="14" fill="{color}">{policy}</text>')
|
||||
parts.append('</svg>');(DIR/'evidence/frontier.svg').write_text(''.join(parts))
|
||||
|
||||
if __name__=='__main__': main()
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
"""Policy counterfactuals on measured worker evidence arrival times.
|
||||
All crops enter together; these are not camera/UI end-to-end timings.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import json
|
||||
from statistics import median
|
||||
from analyze import POLICIES,decide,infer_groups,propagate,metrics
|
||||
|
||||
DIR=Path(__file__).resolve().parent;OUT=DIR/'evidence'
|
||||
|
||||
|
||||
def main():
|
||||
frozen=json.loads((OUT/'observations.json').read_text())
|
||||
data=frozen['resolutions']['live'];sig=frozen['signatures']
|
||||
edges=json.loads((OUT/'analysis.json').read_text())['resolutions']['live']['guarded_edges']
|
||||
events=json.loads((OUT/'anytime-events.json').read_text())
|
||||
families=json.loads((DIR/'group-truth.json').read_text())['families']
|
||||
report={'clock':'Measured local/OCR completion; frozen appearance edges are precomputed, grouping and transport time excluded. Diagnostic counterfactual, not live deployment.','policies':{}}
|
||||
horizon=events['complete_ms']+1000
|
||||
for policy in POLICIES:
|
||||
for grouped in (False,True):
|
||||
rows={o['id']:{**o,'rank':[],'local':{},'text':{},'read':''} for o in data['observations']}
|
||||
history={i:[(0.,None)] for i in rows};snapshots={};current={i:None for i in rows}
|
||||
for event in sorted(events['events'],key=lambda e:e['ms']):
|
||||
for delay in (250,500,1000):
|
||||
if delay<event['ms'] and str(delay) not in snapshots:snapshots[str(delay)]=metrics(list(rows.values()),current,families)
|
||||
o=rows[event['id']]
|
||||
if event['kind']=='local':o['rank']=event['rank']
|
||||
else:o['read']=event['read'];o['text']=event['text']
|
||||
obs=list(rows.values());base={i:decide(o,policy,sig) for i,o in rows.items()}
|
||||
groups=infer_groups(data,edges,{i:decide(o,'strict',sig) for i,o in rows.items()})
|
||||
current=propagate(obs,groups,base,sig,guarded=True)[0] if grouped else base
|
||||
for i,sku in current.items():
|
||||
if sku!=history[i][-1][1]:history[i].append((event['ms'],sku))
|
||||
for delay in (250,500,1000):snapshots.setdefault(str(delay),metrics(list(rows.values()),current,families))
|
||||
first=[];stable=[];flips=0;corrections=0;blank_ms=wrong_ms=sibling_ms=0.
|
||||
for i,h in history.items():
|
||||
truth=rows[i]['truth'];named=[at for at,sku in h if sku]
|
||||
if named:first.append(min(named))
|
||||
if h[-1][1]==truth and truth!='unknown':stable.append(h[-1][0])
|
||||
previous=None
|
||||
for (_,sku),(end,_) in zip(h,h[1:]+[(horizon,None)]):
|
||||
if sku and previous and sku!=previous:
|
||||
flips+=1;corrections+=sku==truth
|
||||
if sku:previous=sku
|
||||
for (at,sku),(end,_) in zip(h,h[1:]+[(horizon,None)]):
|
||||
duration=end-at
|
||||
if not sku:blank_ms+=duration
|
||||
elif sku!=truth:
|
||||
wrong_ms+=duration
|
||||
if truth in families and families.get(sku)==families[truth]:sibling_ms+=duration
|
||||
final=metrics(list(rows.values()),current,families)
|
||||
assert final['correct']+final['wrong']==final['shown']<=109
|
||||
report['policies'][policy+('/grouped' if grouped else '/individual')]={'final':final,'timed':snapshots,'first_any_ms':min(first) if first else None,'first_per_facing_p50_ms':median(first) if first else None,'time_to_final_correct_p50_ms':median(stable) if stable else None,'identity_flips':flips,'wrong_to_correct_events':corrections,'blank_facing_ms':blank_ms,'wrong_facing_ms':wrong_ms,'wrong_sibling_facing_ms':sibling_ms,'loss_proxy':(blank_ms+1.5*sibling_ms+4*(wrong_ms-sibling_ms)+200*flips)/(109*horizon),'horizon_ms':horizon}
|
||||
(OUT/'anytime-policies.json').write_text(json.dumps(report,ensure_ascii=False,indent=2))
|
||||
for name,r in report['policies'].items():print(name,r['final']['shown'],r['final']['correct'],round(r['first_per_facing_p50_ms'],1),r['identity_flips'],round(r['loss_proxy'],3))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
"""Actual eight-worker local/OCR calls; fixed crops, one variable per arm."""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
import statistics
|
||||
import threading
|
||||
import concurrent.futures
|
||||
import os
|
||||
import psutil
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0,str(ROOT/'code'))
|
||||
import cv2
|
||||
from identity_lanes import LocalLane,TextLane
|
||||
from ocr import build_ocr_engine
|
||||
from eval_identity import _catalog_rows
|
||||
DIR=Path(__file__).resolve().parent; OUT=DIR/'evidence'
|
||||
|
||||
|
||||
def percentile(v,n):
|
||||
return round(sorted(v)[min(len(v)-1,int((len(v)-1)*n))],2)
|
||||
|
||||
|
||||
def main():
|
||||
cv2.setNumThreads(1)
|
||||
image=cv2.imread(str(ROOT/'add/image6.jpg'))
|
||||
scale=1280/max(image.shape[:2]);image=cv2.resize(image,(round(image.shape[1]*scale),round(image.shape[0]*scale)),interpolation=cv2.INTER_AREA)
|
||||
evidence=json.loads((OUT/'observations.json').read_text());rows=evidence['resolutions']['live']['observations']
|
||||
schedule=json.loads((OUT/'ocr-ablation.json').read_text())['arms']['live']
|
||||
crops={o['id']:image[o['box'][1]:o['box'][3],o['box'][0]:o['box'][2]] for o in rows}
|
||||
local=LocalLane(str(ROOT/'stock-lopia-clean'))
|
||||
text=TextLane.from_catalog(_catalog_rows(str(ROOT/'stock-lopia-clean')),str(ROOT/'stock-lopia-clean'),build_ocr_engine('vision'),lock=threading.Lock())
|
||||
report={}
|
||||
local.rank(next(iter(crops.values())))
|
||||
text.read(next(iter(crops.values())))
|
||||
# These benchmarks do not decode, detect, track or transmit; they isolate
|
||||
# classifier worker contention with the same 109 observed shelf crops.
|
||||
modes=[('every_attempt',2,lambda i,a:True),('once_per_track',2,lambda i,a:a==0),('ambiguous_track',2,lambda i,a:a==0 and i in schedule['ambiguous_track']['ocr_ids']),('group_representative',2,lambda i,a:a==0 and i in schedule['group_representative']['ocr_ids'])]
|
||||
proc=psutil.Process()
|
||||
for name,attempts,ocr_on in modes:
|
||||
tasks=[(i,a,ocr_on(i,a)) for a in range(attempts) for i in crops]
|
||||
durations=[];first={};done={};ocr_count=0
|
||||
cpu0=proc.cpu_times();started=time.perf_counter()
|
||||
def work(item):
|
||||
i,a,use_ocr=item;start=time.perf_counter();rank=local.rank(crops[i]);read=text.read(crops[i]) if use_ocr else ''
|
||||
return i,a,round((time.perf_counter()-start)*1000,2),bool(read)
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
|
||||
for f in concurrent.futures.as_completed([pool.submit(work,item) for item in tasks]):
|
||||
i,a,elapsed,read=f.result();now=(time.perf_counter()-started)*1000
|
||||
durations.append(elapsed);first.setdefault(i,now);done[i]=now;ocr_count+=bool(read)
|
||||
wall=time.perf_counter()-started;cpu1=proc.cpu_times();cpu=cpu1.user+cpu1.system-cpu0.user-cpu0.system
|
||||
report[name]={'tasks':len(tasks),'ocr_invocations':sum(int(k) for _,_,k in tasks),'ocr_nonempty_reads':ocr_count,'worker_ms_p50':percentile(durations,.5),'worker_ms_p95':percentile(durations,.95),'first_facing_ms_p50':percentile(list(first.values()),.5),'all_first_facings_ms':round(max(first.values()),1),'wall_s':round(wall,3),'throughput_tasks_s':round(len(tasks)/wall,1),'cpu_s':round(cpu,2),'first_results':{str(i):round(v,1) for i,v in first.items()}}
|
||||
print(name,report[name]['wall_s'],report[name]['all_first_facings_ms'],report[name]['ocr_invocations'],flush=True)
|
||||
(OUT/'scheduler-benchmark.json').write_text(json.dumps(report,indent=2))
|
||||
# Separate visual completion from serialized text: no OCR lock occupies a
|
||||
# local worker. Measure evidence arrival, not phone rendering or inference.
|
||||
started=time.perf_counter();events=[]
|
||||
with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
|
||||
pending={pool.submit(local.rank,crop):i for i,crop in crops.items()}
|
||||
for future in concurrent.futures.as_completed(pending):
|
||||
events.append({'id':pending[future],'kind':'local','ms':(time.perf_counter()-started)*1000,'rank':future.result()})
|
||||
visual_done=(time.perf_counter()-started)*1000
|
||||
for i in schedule['ambiguous_track']['ocr_ids']:
|
||||
read=text.read(crops[i])
|
||||
events.append({'id':i,'kind':'text','ms':(time.perf_counter()-started)*1000,'read':read,'text':text.scores_from_text(read)})
|
||||
(OUT/'anytime-events.json').write_text(json.dumps({'provenance':evidence['provenance'],'visual_done_ms':visual_done,'complete_ms':(time.perf_counter()-started)*1000,'events':events},ensure_ascii=False,indent=2))
|
||||
print('visual_first',round(visual_done,1),'ms to all visual results',flush=True)
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,118 @@
|
|||
"""Frozen A-F observations; imports shipped lanes, never edits production.
|
||||
Run from recognization/: .venv/bin/python docs/research/2026-09-29-shelf-intelligence/collect.py
|
||||
"""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import json
|
||||
import time
|
||||
from dataclasses import asdict
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0, str(ROOT / 'code'))
|
||||
import cv2
|
||||
import numpy as np
|
||||
from identity_lanes import LocalLane, TextLane, Evidence, _root_sift, _hue_histogram
|
||||
from ocr import build_ocr_engine
|
||||
from eval_identity import catalog_hash, file_hash, code_revision, _catalog_rows
|
||||
|
||||
OUT = Path(__file__).resolve().parent / 'evidence'
|
||||
|
||||
|
||||
def rows_from_boxes(items):
|
||||
"""Bottom-aligned shelf rows, using geometry only, no labels or catalog IDs."""
|
||||
rows = []
|
||||
for item in sorted(items, key=lambda x: x['box'][3]):
|
||||
b = item['box']
|
||||
candidates = [(abs(b[3] - np.median([x['box'][3] for x in row])), row) for row in rows]
|
||||
distance, row = min(candidates, key=lambda x: x[0]) if candidates else (float('inf'), [])
|
||||
if distance <= .25 * max(1, b[3] - b[1]):
|
||||
row.append(item)
|
||||
else:
|
||||
rows.append([item])
|
||||
return [sorted(row, key=lambda x: (x['box'][0] + x['box'][2]) / 2) for row in rows]
|
||||
|
||||
|
||||
def features(crop, sift):
|
||||
gray = cv2.cvtColor(crop, cv2.COLOR_BGR2GRAY)
|
||||
kp, des = sift.detectAndCompute(gray, None)
|
||||
small = cv2.resize(crop, (32, 64), interpolation=cv2.INTER_AREA)
|
||||
lab = cv2.cvtColor(small, cv2.COLOR_BGR2LAB).astype(float) / 255
|
||||
edge = cv2.resize(cv2.Canny(gray, 70, 140), (16, 32)).astype(float).ravel() / 255
|
||||
return {'kp': kp, 'des': _root_sift(des), 'hist': _hue_histogram(crop), 'lab': lab,
|
||||
'edge': edge, 'shape': gray.shape, 'sharpness': float(cv2.Laplacian(gray, cv2.CV_64F).var())}
|
||||
|
||||
|
||||
def pair_signals(a, b):
|
||||
result = {'hue': float(np.minimum(a['hist'], b['hist']).sum()),
|
||||
'pixels': float(max(0, 1 - np.abs(a['lab'] - b['lab']).mean() * 3)),
|
||||
'logo_pixels': float(max(0, 1 - np.abs(a['lab'][:32] - b['lab'][:32]).mean() * 3)),
|
||||
'edge': float(np.dot(a['edge'], b['edge']) / max(1e-6, np.linalg.norm(a['edge']) * np.linalg.norm(b['edge']))),
|
||||
'inliers': 0, 'match_fraction': 0., 'layout_cover': 0.}
|
||||
if a['des'] is not None and b['des'] is not None and len(b['des']) >= 2:
|
||||
pairs = cv2.BFMatcher(cv2.NORM_L2).knnMatch(a['des'], b['des'], k=2)
|
||||
good = [x[0] for x in pairs if len(x) == 2 and x[0].distance < .75 * x[1].distance]
|
||||
if len(good) >= 4:
|
||||
src = np.float32([a['kp'][m.queryIdx].pt for m in good])
|
||||
dst = np.float32([b['kp'][m.trainIdx].pt for m in good])
|
||||
_, mask = cv2.estimateAffinePartial2D(src, dst, method=cv2.RANSAC, ransacReprojThreshold=.02 * max(b['shape']))
|
||||
if mask is not None:
|
||||
yes = mask.ravel().astype(bool)
|
||||
result['inliers'] = int(yes.sum())
|
||||
result['match_fraction'] = result['inliers'] / max(1, min(len(a['kp']), len(b['kp'])))
|
||||
if yes.sum() >= 3:
|
||||
result['layout_cover'] = min(float(cv2.contourArea(cv2.convexHull(src[yes])) / np.prod(a['shape'])), float(cv2.contourArea(cv2.convexHull(dst[yes])) / np.prod(b['shape'])))
|
||||
return result
|
||||
|
||||
|
||||
def main():
|
||||
OUT.mkdir(parents=True, exist_ok=True)
|
||||
cv2.setNumThreads(1)
|
||||
cv2.setRNGSeed(20260929)
|
||||
label_path = ROOT / 'eval/labels/image6.json'
|
||||
labels = json.loads(label_path.read_text())
|
||||
source = cv2.imread(str(ROOT / labels['image']))
|
||||
stock = str(ROOT / 'stock-lopia-clean')
|
||||
local = LocalLane(stock)
|
||||
text = TextLane.from_catalog(_catalog_rows(stock), stock, build_ocr_engine('vision'))
|
||||
sift = cv2.SIFT_create(nfeatures=2000)
|
||||
records = [f for f in labels['facings'] if f['label'] != 'exclude']
|
||||
report = {'provenance': {'code': code_revision(), 'catalog': catalog_hash(stock), 'labels': file_hash(str(label_path)), 'image': file_hash(str(ROOT / labels['image'])), 'opencv_threads': 1, 'seed': 20260929}, 'signatures': text.signatures, 'resolutions': {}}
|
||||
# Contact sheet is visual annotation evidence, not an inference input.
|
||||
sheet = np.full((int(np.ceil(len(records)/12))*200, 12*110, 3), 245, np.uint8)
|
||||
for n, f in enumerate(records):
|
||||
x1,y1,x2,y2 = f['box']; crop = source[y1:y2,x1:x2]
|
||||
h,w = crop.shape[:2]; ratio = min(100/w, 170/h)
|
||||
tile = cv2.resize(crop, (max(1,int(w*ratio)), max(1,int(h*ratio))))
|
||||
x,y = n%12*110,n//12*200
|
||||
sheet[y+25:y+25+tile.shape[0],x:x+tile.shape[1]] = tile
|
||||
cv2.putText(sheet, str(f['index']), (x+4,y+18), cv2.FONT_HERSHEY_SIMPLEX,.5,(0,0,0),1)
|
||||
cv2.imwrite(str(OUT/'facings.jpg'), sheet)
|
||||
for resolution, scale in [('full',1.),('live',1280/max(source.shape[:2]))]:
|
||||
image = source if scale == 1 else cv2.resize(source, (round(source.shape[1]*scale),round(source.shape[0]*scale)), interpolation=cv2.INTER_AREA)
|
||||
obs, appearances = [], {}
|
||||
for f in records:
|
||||
box = [round(v*scale) for v in f['box']]; x1,y1,x2,y2=box
|
||||
crop = image[y1:y2,x1:x2]
|
||||
start=time.perf_counter(); ev=local.evidence(crop); local_ms=(time.perf_counter()-start)*1000
|
||||
ranked=sorted(ev.items(),key=lambda kv:local.rank_key(kv[1]),reverse=True)
|
||||
rank=[(sku,local.confidence(e,ranked[i+1][1] if i+1<len(ranked) else Evidence())) for i,(sku,e) in enumerate(ranked)]
|
||||
start=time.perf_counter(); read=text.read(crop); ocr_ms=(time.perf_counter()-start)*1000
|
||||
start=time.perf_counter(); appearance=features(crop,sift); features_ms=(time.perf_counter()-start)*1000
|
||||
appearances[f['index']]=appearance
|
||||
obs.append({'id':f['index'],'box':box,'truth':f['label'],'local':{sku:asdict(e) for sku,e in ev.items()},'rank':rank,'text':text.scores_from_text(read),'read':read,'local_ms':local_ms,'ocr_ms':ocr_ms,'pair_features_ms':features_ms,'sharpness':appearance['sharpness']})
|
||||
rows=rows_from_boxes(obs); pairs=[]
|
||||
for r,row in enumerate(rows):
|
||||
for a,b in zip(row,row[1:]):
|
||||
start=time.perf_counter(); signals=pair_signals(appearances[a['id']],appearances[b['id']]); elapsed=(time.perf_counter()-start)*1000
|
||||
ab,bb=a['box'],b['box']; aw,ah=ab[2]-ab[0],ab[3]-ab[1]; bw,bh=bb[2]-bb[0],bb[3]-bb[1]
|
||||
signals.update({'aspect':min(aw/ah,bw/bh)/max(aw/ah,bw/bh),'height':min(ah,bh)/max(ah,bh),'gap':(bb[0]-ab[2])/max(aw,bw),'bottom':abs(ab[3]-bb[3])/max(ah,bh)})
|
||||
from rebuild_catalog import bigrams
|
||||
at,bt=bigrams(a['read']),bigrams(b['read'])
|
||||
signals['ocr_jaccard']=len(at&bt)/max(1,len(at|bt))
|
||||
pairs.append({'a':a['id'],'b':b['id'],'row':r,'signals':signals,'pair_ms':elapsed})
|
||||
report['resolutions'][resolution]={'observations':obs,'rows':[[f['id'] for f in row] for row in rows],'pairs':pairs}
|
||||
print(resolution, len(obs),'facings', len(rows),'rows', 'local/ocr ms', round(sum(x['local_ms'] for x in obs)),round(sum(x['ocr_ms'] for x in obs)),flush=True)
|
||||
(OUT/'observations.json').write_text(json.dumps(report,ensure_ascii=False,indent=2))
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,725 @@
|
|||
{
|
||||
"clock": "Measured local/OCR completion; frozen appearance edges are precomputed, grouping and transport time excluded. Diagnostic counterfactual, not live deployment.",
|
||||
"policies": {
|
||||
"strict/individual": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 39,
|
||||
"correct": 38,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.3577981651376147,
|
||||
"correct_coverage": 0.59375,
|
||||
"precision": 0.9743589743589743,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6422018348623852,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 31,
|
||||
"correct": 31,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.28440366972477066,
|
||||
"correct_coverage": 0.484375,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.7155963302752293,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 32,
|
||||
"correct": 32,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.29357798165137616,
|
||||
"correct_coverage": 0.5,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.7064220183486238,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 37,
|
||||
"correct": 37,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.3394495412844037,
|
||||
"correct_coverage": 0.578125,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6605504587155964,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 14.775583054870367,
|
||||
"first_per_facing_p50_ms": 88.42595806345344,
|
||||
"time_to_final_correct_p50_ms": 85.3563750279136,
|
||||
"identity_flips": 0,
|
||||
"wrong_to_correct_events": 0,
|
||||
"blank_facing_ms": 243097.88916585967,
|
||||
"wrong_facing_ms": 1237.2076249448583,
|
||||
"wrong_sibling_facing_ms": 0.0,
|
||||
"loss_proxy": 0.6847073949299143,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"strict/grouped": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 51,
|
||||
"correct": 50,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.46788990825688076,
|
||||
"correct_coverage": 0.78125,
|
||||
"precision": 0.9803921568627451,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5321100917431192,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 40,
|
||||
"correct": 40,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.3669724770642202,
|
||||
"correct_coverage": 0.625,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6330275229357798,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 41,
|
||||
"correct": 41,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.3761467889908257,
|
||||
"correct_coverage": 0.640625,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6238532110091743,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 50,
|
||||
"correct": 50,
|
||||
"wrong": 0,
|
||||
"visible_coverage": 0.45871559633027525,
|
||||
"correct_coverage": 0.78125,
|
||||
"precision": 1.0,
|
||||
"false_label_rate": 0.0,
|
||||
"unknown_false_accepts": 0,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5412844036697247,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 14.775583054870367,
|
||||
"first_per_facing_p50_ms": 114.48554205708206,
|
||||
"time_to_final_correct_p50_ms": 114.48554205708206,
|
||||
"identity_flips": 0,
|
||||
"wrong_to_correct_events": 0,
|
||||
"blank_facing_ms": 205600.77199933585,
|
||||
"wrong_facing_ms": 1237.2076249448583,
|
||||
"wrong_sibling_facing_ms": 0.0,
|
||||
"loss_proxy": 0.581200469109323,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"balanced/individual": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 41,
|
||||
"correct": 40,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.3761467889908257,
|
||||
"correct_coverage": 0.625,
|
||||
"precision": 0.975609756097561,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6238532110091743,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 38,
|
||||
"correct": 36,
|
||||
"wrong": 2,
|
||||
"visible_coverage": 0.3486238532110092,
|
||||
"correct_coverage": 0.5625,
|
||||
"precision": 0.9473684210526315,
|
||||
"false_label_rate": 0.01834862385321101,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.6513761467889908,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 38,
|
||||
"correct": 36,
|
||||
"wrong": 2,
|
||||
"visible_coverage": 0.3486238532110092,
|
||||
"correct_coverage": 0.5625,
|
||||
"precision": 0.9473684210526315,
|
||||
"false_label_rate": 0.01834862385321101,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.6513761467889908,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 42,
|
||||
"correct": 41,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.3853211009174312,
|
||||
"correct_coverage": 0.640625,
|
||||
"precision": 0.9761904761904762,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6146788990825688,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 14.775583054870367,
|
||||
"first_per_facing_p50_ms": 82.28679199237376,
|
||||
"time_to_final_correct_p50_ms": 80.85418748669326,
|
||||
"identity_flips": 1,
|
||||
"wrong_to_correct_events": 0,
|
||||
"blank_facing_ms": 230677.1714988863,
|
||||
"wrong_facing_ms": 3862.650291994214,
|
||||
"wrong_sibling_facing_ms": 0.0,
|
||||
"loss_proxy": 0.6799624189707845,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"balanced/grouped": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 52,
|
||||
"correct": 51,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.47706422018348627,
|
||||
"correct_coverage": 0.796875,
|
||||
"precision": 0.9807692307692307,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5229357798165137,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 45,
|
||||
"correct": 43,
|
||||
"wrong": 2,
|
||||
"visible_coverage": 0.41284403669724773,
|
||||
"correct_coverage": 0.671875,
|
||||
"precision": 0.9555555555555556,
|
||||
"false_label_rate": 0.01834862385321101,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5871559633027523,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 45,
|
||||
"correct": 43,
|
||||
"wrong": 2,
|
||||
"visible_coverage": 0.41284403669724773,
|
||||
"correct_coverage": 0.671875,
|
||||
"precision": 0.9555555555555556,
|
||||
"false_label_rate": 0.01834862385321101,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5871559633027523,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 52,
|
||||
"correct": 51,
|
||||
"wrong": 1,
|
||||
"visible_coverage": 0.47706422018348627,
|
||||
"correct_coverage": 0.796875,
|
||||
"precision": 0.9807692307692307,
|
||||
"false_label_rate": 0.009174311926605505,
|
||||
"unknown_false_accepts": 1,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5229357798165137,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 14.775583054870367,
|
||||
"first_per_facing_p50_ms": 101.45575006026775,
|
||||
"time_to_final_correct_p50_ms": 114.48554205708206,
|
||||
"identity_flips": 2,
|
||||
"wrong_to_correct_events": 1,
|
||||
"blank_facing_ms": 198906.68479132,
|
||||
"wrong_facing_ms": 3797.012208029628,
|
||||
"wrong_sibling_facing_ms": 0.0,
|
||||
"loss_proxy": 0.5920905968176576,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"coverage-first/individual": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 56,
|
||||
"correct": 49,
|
||||
"wrong": 7,
|
||||
"visible_coverage": 0.5137614678899083,
|
||||
"correct_coverage": 0.765625,
|
||||
"precision": 0.875,
|
||||
"false_label_rate": 0.06422018348623854,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.4862385321100917,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 60,
|
||||
"correct": 47,
|
||||
"wrong": 13,
|
||||
"visible_coverage": 0.5504587155963303,
|
||||
"correct_coverage": 0.734375,
|
||||
"precision": 0.7833333333333333,
|
||||
"false_label_rate": 0.11926605504587157,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 4,
|
||||
"blank_ratio": 0.4495412844036697,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 60,
|
||||
"correct": 47,
|
||||
"wrong": 13,
|
||||
"visible_coverage": 0.5504587155963303,
|
||||
"correct_coverage": 0.734375,
|
||||
"precision": 0.7833333333333333,
|
||||
"false_label_rate": 0.11926605504587157,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 4,
|
||||
"blank_ratio": 0.4495412844036697,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 62,
|
||||
"correct": 51,
|
||||
"wrong": 11,
|
||||
"visible_coverage": 0.5688073394495413,
|
||||
"correct_coverage": 0.796875,
|
||||
"precision": 0.8225806451612904,
|
||||
"false_label_rate": 0.10091743119266056,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.4311926605504587,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 13.799707987345755,
|
||||
"first_per_facing_p50_ms": 82.28679199237376,
|
||||
"time_to_final_correct_p50_ms": 73.84883298072964,
|
||||
"identity_flips": 2,
|
||||
"wrong_to_correct_events": 1,
|
||||
"blank_facing_ms": 172579.93904012255,
|
||||
"wrong_facing_ms": 30759.05395997688,
|
||||
"wrong_sibling_facing_ms": 3306.7995419260114,
|
||||
"loss_proxy": 0.7943018770048237,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"coverage-first/grouped": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 61,
|
||||
"correct": 55,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.5596330275229358,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.9016393442622951,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.44036697247706424,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 62,
|
||||
"correct": 49,
|
||||
"wrong": 13,
|
||||
"visible_coverage": 0.5688073394495413,
|
||||
"correct_coverage": 0.765625,
|
||||
"precision": 0.7903225806451613,
|
||||
"false_label_rate": 0.11926605504587157,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 4,
|
||||
"blank_ratio": 0.4311926605504587,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 62,
|
||||
"correct": 49,
|
||||
"wrong": 13,
|
||||
"visible_coverage": 0.5688073394495413,
|
||||
"correct_coverage": 0.765625,
|
||||
"precision": 0.7903225806451613,
|
||||
"false_label_rate": 0.11926605504587157,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 4,
|
||||
"blank_ratio": 0.4311926605504587,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 65,
|
||||
"correct": 55,
|
||||
"wrong": 10,
|
||||
"visible_coverage": 0.5963302752293578,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.8461538461538461,
|
||||
"false_label_rate": 0.09174311926605505,
|
||||
"unknown_false_accepts": 9,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.4036697247706422,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 13.799707987345755,
|
||||
"first_per_facing_p50_ms": 88.42595806345344,
|
||||
"time_to_final_correct_p50_ms": 82.28679199237376,
|
||||
"identity_flips": 4,
|
||||
"wrong_to_correct_events": 3,
|
||||
"blank_facing_ms": 160348.0866663158,
|
||||
"wrong_facing_ms": 28053.20287507493,
|
||||
"wrong_sibling_facing_ms": 3306.7995419260114,
|
||||
"loss_proxy": 0.7317643767039906,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"near-argmax/individual": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 60,
|
||||
"correct": 50,
|
||||
"wrong": 10,
|
||||
"visible_coverage": 0.5504587155963303,
|
||||
"correct_coverage": 0.78125,
|
||||
"precision": 0.8333333333333334,
|
||||
"false_label_rate": 0.09174311926605505,
|
||||
"unknown_false_accepts": 8,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.4495412844036697,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 67,
|
||||
"correct": 48,
|
||||
"wrong": 19,
|
||||
"visible_coverage": 0.6146788990825688,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.7164179104477612,
|
||||
"false_label_rate": 0.1743119266055046,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 7,
|
||||
"blank_ratio": 0.3853211009174312,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 67,
|
||||
"correct": 48,
|
||||
"wrong": 19,
|
||||
"visible_coverage": 0.6146788990825688,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.7164179104477612,
|
||||
"false_label_rate": 0.1743119266055046,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 7,
|
||||
"blank_ratio": 0.3853211009174312,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 66,
|
||||
"correct": 52,
|
||||
"wrong": 14,
|
||||
"visible_coverage": 0.6055045871559633,
|
||||
"correct_coverage": 0.8125,
|
||||
"precision": 0.7878787878787878,
|
||||
"false_label_rate": 0.12844036697247707,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.39449541284403666,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 13.799707987345755,
|
||||
"first_per_facing_p50_ms": 79.42158298101276,
|
||||
"time_to_final_correct_p50_ms": 75.28845797060058,
|
||||
"identity_flips": 5,
|
||||
"wrong_to_correct_events": 4,
|
||||
"blank_facing_ms": 158024.130997248,
|
||||
"wrong_facing_ms": 42189.443835872225,
|
||||
"wrong_sibling_facing_ms": 3306.7995419260114,
|
||||
"loss_proxy": 0.8819879587235034,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
},
|
||||
"near-argmax/grouped": {
|
||||
"final": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 64,
|
||||
"correct": 55,
|
||||
"wrong": 9,
|
||||
"visible_coverage": 0.5871559633027523,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.859375,
|
||||
"false_label_rate": 0.08256880733944955,
|
||||
"unknown_false_accepts": 8,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.4128440366972477,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"timed": {
|
||||
"250": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 68,
|
||||
"correct": 49,
|
||||
"wrong": 19,
|
||||
"visible_coverage": 0.6238532110091743,
|
||||
"correct_coverage": 0.765625,
|
||||
"precision": 0.7205882352941176,
|
||||
"false_label_rate": 0.1743119266055046,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 7,
|
||||
"blank_ratio": 0.37614678899082565,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"500": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 68,
|
||||
"correct": 49,
|
||||
"wrong": 19,
|
||||
"visible_coverage": 0.6238532110091743,
|
||||
"correct_coverage": 0.765625,
|
||||
"precision": 0.7205882352941176,
|
||||
"false_label_rate": 0.1743119266055046,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 7,
|
||||
"blank_ratio": 0.37614678899082565,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"1000": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 68,
|
||||
"correct": 55,
|
||||
"wrong": 13,
|
||||
"visible_coverage": 0.6238532110091743,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.8088235294117647,
|
||||
"false_label_rate": 0.11926605504587157,
|
||||
"unknown_false_accepts": 12,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.37614678899082565,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
}
|
||||
},
|
||||
"first_any_ms": 13.799707987345755,
|
||||
"first_per_facing_p50_ms": 77.35222950577736,
|
||||
"time_to_final_correct_p50_ms": 82.28679199237376,
|
||||
"identity_flips": 7,
|
||||
"wrong_to_correct_events": 6,
|
||||
"blank_facing_ms": 148948.3510403661,
|
||||
"wrong_facing_ms": 39452.93850102462,
|
||||
"wrong_sibling_facing_ms": 3306.7995419260114,
|
||||
"loss_proxy": 0.8278239993391047,
|
||||
"horizon_ms": 3323.5481249867007
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,399 @@
|
|||
{
|
||||
"image": "add/image6.jpg",
|
||||
"stock_dir": "stock-lopia-clean",
|
||||
"labels": "eval/labels/image6.json",
|
||||
"provenance": {
|
||||
"catalog_hash": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
||||
"labels_hash": "6fc1a220a97c75666882b4bcd9dc102ed08c8924677b97d615850a0983286c1a",
|
||||
"image_hash": "a08b89ba37d27c9d5a81bcd56bcd042e6baf54d2f182bbaaf47e97f14c9cd525",
|
||||
"code_revision": "fbf31d74feb12538dc07207cb80722dac38dd438"
|
||||
},
|
||||
"facings": 111,
|
||||
"leakage": {
|
||||
"excluded": 0,
|
||||
"labels": []
|
||||
},
|
||||
"hub_share_limit": 0.3,
|
||||
"enrollment": {
|
||||
"catalog_skus": [
|
||||
"sku_l01",
|
||||
"sku_l02",
|
||||
"sku_l03",
|
||||
"sku_l04",
|
||||
"sku_l05",
|
||||
"sku_l06",
|
||||
"sku_l07",
|
||||
"sku_l08",
|
||||
"sku_l09",
|
||||
"sku_l10",
|
||||
"sku_l11",
|
||||
"sku_l12"
|
||||
],
|
||||
"enrolled_skus": [
|
||||
"sku_l01",
|
||||
"sku_l02",
|
||||
"sku_l03",
|
||||
"sku_l05",
|
||||
"sku_l06",
|
||||
"sku_l09",
|
||||
"sku_l10",
|
||||
"sku_l11"
|
||||
],
|
||||
"unenrolled_skus": {
|
||||
"sku_l07": 6,
|
||||
"sku_l12": 3,
|
||||
"sku_l04": 1,
|
||||
"sku_l08": 1
|
||||
},
|
||||
"facings_without_enrollment": 11,
|
||||
"scored_facings": 109
|
||||
},
|
||||
"results": {
|
||||
"full": [
|
||||
{
|
||||
"lane": "van",
|
||||
"accept": 1.0,
|
||||
"margin": 0.15,
|
||||
"in_catalog": 64,
|
||||
"out_catalog": 45,
|
||||
"enrolled": 53,
|
||||
"correct": 44,
|
||||
"wrong": 65,
|
||||
"rejected": 0,
|
||||
"catalog_ceiling": 0.8281,
|
||||
"identity_ceiling": 0.8302,
|
||||
"decision_ceiling": 0.8302,
|
||||
"end_to_end_coverage": 0.6875,
|
||||
"false_accepts": 65,
|
||||
"unknown_false_accepts": 45,
|
||||
"unenrolled_false_accepts": 11,
|
||||
"false_accept_rate": 1.0,
|
||||
"near_unknown_false_accept_rate": 1.0,
|
||||
"precision": 0.4037,
|
||||
"max_sku_share": 0.1927,
|
||||
"hub_sku": "sku_l05",
|
||||
"hub_alarm": false,
|
||||
"histogram": {
|
||||
"sku_l05": 21,
|
||||
"sku_l09": 20,
|
||||
"sku_l03": 17,
|
||||
"sku_l02": 13,
|
||||
"sku_l11": 12,
|
||||
"sku_l10": 12,
|
||||
"sku_l06": 8,
|
||||
"sku_l01": 6
|
||||
},
|
||||
"per_sku": {
|
||||
"sku_l01": {
|
||||
"correct": 5,
|
||||
"total": 5
|
||||
},
|
||||
"sku_l02": {
|
||||
"correct": 4,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l03": {
|
||||
"correct": 4,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l04": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l05": {
|
||||
"correct": 3,
|
||||
"total": 3
|
||||
},
|
||||
"sku_l06": {
|
||||
"correct": 3,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l07": {
|
||||
"correct": 0,
|
||||
"total": 6
|
||||
},
|
||||
"sku_l08": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l09": {
|
||||
"correct": 13,
|
||||
"total": 13
|
||||
},
|
||||
"sku_l10": {
|
||||
"correct": 9,
|
||||
"total": 11
|
||||
},
|
||||
"sku_l11": {
|
||||
"correct": 3,
|
||||
"total": 9
|
||||
},
|
||||
"sku_l12": {
|
||||
"correct": 0,
|
||||
"total": 3
|
||||
}
|
||||
},
|
||||
"ms_per_crop": 19.0
|
||||
},
|
||||
{
|
||||
"lane": "local",
|
||||
"accept": 4.0,
|
||||
"margin": 0.15,
|
||||
"in_catalog": 64,
|
||||
"out_catalog": 45,
|
||||
"enrolled": 53,
|
||||
"correct": 42,
|
||||
"wrong": 2,
|
||||
"rejected": 65,
|
||||
"catalog_ceiling": 0.8281,
|
||||
"identity_ceiling": 0.9245,
|
||||
"decision_ceiling": 0.717,
|
||||
"end_to_end_coverage": 0.6562,
|
||||
"false_accepts": 2,
|
||||
"unknown_false_accepts": 2,
|
||||
"unenrolled_false_accepts": 0,
|
||||
"false_accept_rate": 0.0444,
|
||||
"near_unknown_false_accept_rate": 0.0,
|
||||
"precision": 0.9545,
|
||||
"max_sku_share": 0.2273,
|
||||
"hub_sku": "sku_l09",
|
||||
"hub_alarm": false,
|
||||
"histogram": {
|
||||
"sku_l09": 10,
|
||||
"sku_l10": 9,
|
||||
"sku_l01": 5,
|
||||
"sku_l03": 5,
|
||||
"sku_l07": 5,
|
||||
"sku_l06": 4,
|
||||
"sku_l05": 3,
|
||||
"sku_l11": 2,
|
||||
"sku_l02": 1
|
||||
},
|
||||
"per_sku": {
|
||||
"sku_l01": {
|
||||
"correct": 5,
|
||||
"total": 5
|
||||
},
|
||||
"sku_l02": {
|
||||
"correct": 1,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l03": {
|
||||
"correct": 4,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l04": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l05": {
|
||||
"correct": 3,
|
||||
"total": 3
|
||||
},
|
||||
"sku_l06": {
|
||||
"correct": 4,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l07": {
|
||||
"correct": 4,
|
||||
"total": 6
|
||||
},
|
||||
"sku_l08": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l09": {
|
||||
"correct": 10,
|
||||
"total": 13
|
||||
},
|
||||
"sku_l10": {
|
||||
"correct": 9,
|
||||
"total": 11
|
||||
},
|
||||
"sku_l11": {
|
||||
"correct": 2,
|
||||
"total": 9
|
||||
},
|
||||
"sku_l12": {
|
||||
"correct": 0,
|
||||
"total": 3
|
||||
}
|
||||
},
|
||||
"ms_per_crop": 29.1
|
||||
}
|
||||
],
|
||||
"live": [
|
||||
{
|
||||
"lane": "van",
|
||||
"accept": 1.0,
|
||||
"margin": 0.15,
|
||||
"in_catalog": 64,
|
||||
"out_catalog": 45,
|
||||
"enrolled": 53,
|
||||
"correct": 44,
|
||||
"wrong": 65,
|
||||
"rejected": 0,
|
||||
"catalog_ceiling": 0.8281,
|
||||
"identity_ceiling": 0.8302,
|
||||
"decision_ceiling": 0.8302,
|
||||
"end_to_end_coverage": 0.6875,
|
||||
"false_accepts": 65,
|
||||
"unknown_false_accepts": 45,
|
||||
"unenrolled_false_accepts": 11,
|
||||
"false_accept_rate": 1.0,
|
||||
"near_unknown_false_accept_rate": 1.0,
|
||||
"precision": 0.4037,
|
||||
"max_sku_share": 0.2018,
|
||||
"hub_sku": "sku_l11",
|
||||
"hub_alarm": false,
|
||||
"histogram": {
|
||||
"sku_l11": 22,
|
||||
"sku_l03": 17,
|
||||
"sku_l05": 15,
|
||||
"sku_l06": 14,
|
||||
"sku_l09": 14,
|
||||
"sku_l10": 11,
|
||||
"sku_l02": 10,
|
||||
"sku_l01": 6
|
||||
},
|
||||
"per_sku": {
|
||||
"sku_l01": {
|
||||
"correct": 5,
|
||||
"total": 5
|
||||
},
|
||||
"sku_l02": {
|
||||
"correct": 3,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l03": {
|
||||
"correct": 4,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l04": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l05": {
|
||||
"correct": 3,
|
||||
"total": 3
|
||||
},
|
||||
"sku_l06": {
|
||||
"correct": 2,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l07": {
|
||||
"correct": 0,
|
||||
"total": 6
|
||||
},
|
||||
"sku_l08": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l09": {
|
||||
"correct": 10,
|
||||
"total": 13
|
||||
},
|
||||
"sku_l10": {
|
||||
"correct": 9,
|
||||
"total": 11
|
||||
},
|
||||
"sku_l11": {
|
||||
"correct": 8,
|
||||
"total": 9
|
||||
},
|
||||
"sku_l12": {
|
||||
"correct": 0,
|
||||
"total": 3
|
||||
}
|
||||
},
|
||||
"ms_per_crop": 6.5
|
||||
},
|
||||
{
|
||||
"lane": "local",
|
||||
"accept": 4.0,
|
||||
"margin": 0.15,
|
||||
"in_catalog": 64,
|
||||
"out_catalog": 45,
|
||||
"enrolled": 53,
|
||||
"correct": 36,
|
||||
"wrong": 1,
|
||||
"rejected": 72,
|
||||
"catalog_ceiling": 0.8281,
|
||||
"identity_ceiling": 0.9057,
|
||||
"decision_ceiling": 0.6038,
|
||||
"end_to_end_coverage": 0.5625,
|
||||
"false_accepts": 1,
|
||||
"unknown_false_accepts": 1,
|
||||
"unenrolled_false_accepts": 0,
|
||||
"false_accept_rate": 0.0222,
|
||||
"near_unknown_false_accept_rate": 0.0,
|
||||
"precision": 0.973,
|
||||
"max_sku_share": 0.2432,
|
||||
"hub_sku": "sku_l09",
|
||||
"hub_alarm": false,
|
||||
"histogram": {
|
||||
"sku_l09": 9,
|
||||
"sku_l10": 8,
|
||||
"sku_l07": 5,
|
||||
"sku_l01": 4,
|
||||
"sku_l03": 3,
|
||||
"sku_l11": 3,
|
||||
"sku_l05": 2,
|
||||
"sku_l06": 2,
|
||||
"sku_l02": 1
|
||||
},
|
||||
"per_sku": {
|
||||
"sku_l01": {
|
||||
"correct": 4,
|
||||
"total": 5
|
||||
},
|
||||
"sku_l02": {
|
||||
"correct": 1,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l03": {
|
||||
"correct": 3,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l04": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l05": {
|
||||
"correct": 2,
|
||||
"total": 3
|
||||
},
|
||||
"sku_l06": {
|
||||
"correct": 2,
|
||||
"total": 4
|
||||
},
|
||||
"sku_l07": {
|
||||
"correct": 4,
|
||||
"total": 6
|
||||
},
|
||||
"sku_l08": {
|
||||
"correct": 0,
|
||||
"total": 1
|
||||
},
|
||||
"sku_l09": {
|
||||
"correct": 9,
|
||||
"total": 13
|
||||
},
|
||||
"sku_l10": {
|
||||
"correct": 8,
|
||||
"total": 11
|
||||
},
|
||||
"sku_l11": {
|
||||
"correct": 3,
|
||||
"total": 9
|
||||
},
|
||||
"sku_l12": {
|
||||
"correct": 0,
|
||||
"total": 3
|
||||
}
|
||||
},
|
||||
"ms_per_crop": 19.8
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 930 KiB |
|
|
@ -0,0 +1,27 @@
|
|||
{
|
||||
"4.0": {
|
||||
"accepts": 3,
|
||||
"crops": 4580,
|
||||
"rate": 0.0006550218340611354
|
||||
},
|
||||
"2.0": {
|
||||
"accepts": 8,
|
||||
"crops": 4580,
|
||||
"rate": 0.0017467248908296944
|
||||
},
|
||||
"1.0": {
|
||||
"accepts": 45,
|
||||
"crops": 4580,
|
||||
"rate": 0.009825327510917031
|
||||
},
|
||||
"0.5": {
|
||||
"accepts": 83,
|
||||
"crops": 4580,
|
||||
"rate": 0.018122270742358077
|
||||
},
|
||||
"0.0": {
|
||||
"accepts": 115,
|
||||
"crops": 4580,
|
||||
"rate": 0.025109170305676855
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,157 @@
|
|||
{
|
||||
"question": "False accepts per named analyze policy on unrelated shelf crops, analyze.decide on the sweep_accept_lines cache.",
|
||||
"method": {
|
||||
"decision": "analyze.decide(o, policy, signatures) - the frozen analysis's own call",
|
||||
"policies": {
|
||||
"strict": 4.0,
|
||||
"balanced": 2.0,
|
||||
"coverage-first": 0.5,
|
||||
"near-argmax": 0.0
|
||||
},
|
||||
"gate_for_comparison": "identity_lanes.DecisionPolicy(local_accept=threshold, signatures)",
|
||||
"text_scores": "TextLane.scores_from_text on the cached OCR read",
|
||||
"fallback": "analyze.decide's near-argmax extra fallback only: keep rank[0] when the gate refused, unless the read contradicts that row's size or the text prefers another row"
|
||||
},
|
||||
"provenance": {
|
||||
"cache": {
|
||||
"path": "data/evaluations/2026-09-29-session/scratch/accept_line_cache.pkl",
|
||||
"sha256": "1abf0312d354f9f00af6d5a339d0edde0d584c1beee1dab46d04966caf6e637a",
|
||||
"bytes": 89680,
|
||||
"version": 3,
|
||||
"detect_conf": 0.1,
|
||||
"records": [
|
||||
"version",
|
||||
"conf",
|
||||
"photos",
|
||||
"shelf_full",
|
||||
"shelf_live",
|
||||
"distractors"
|
||||
],
|
||||
"not_recorded": [
|
||||
"stock_dir",
|
||||
"catalog_hash",
|
||||
"code_revision",
|
||||
"source image paths or hashes",
|
||||
"per-photo crop counts"
|
||||
],
|
||||
"note": "The cache attests no catalog, so the catalog match below is read off the current worktree, not out of the cache. The read is unpickled, not rebuilt."
|
||||
},
|
||||
"catalog": {
|
||||
"dir": "stock-lopia-clean",
|
||||
"catalog_hash": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
||||
"matches_observations": true,
|
||||
"observations_catalog_hash": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
||||
"observations_code_revision": "fbf31d74feb12538dc07207cb80722dac38dd438"
|
||||
},
|
||||
"code_revision": "fbf31d74feb12538dc07207cb80722dac38dd438",
|
||||
"signatures": {
|
||||
"rows": 12,
|
||||
"sha256": "3e2388f27ae5796155656c5aa10094fbf1012766b7cee088224573a84b2bb584"
|
||||
}
|
||||
},
|
||||
"counts": {
|
||||
"photos": 64,
|
||||
"crops": 4580,
|
||||
"crops_with_a_local_candidate": 155,
|
||||
"crops_with_no_local_candidate": 4425,
|
||||
"crops_with_a_raw_read": 1546,
|
||||
"crops_scored_by_text": 1185
|
||||
},
|
||||
"policies": {
|
||||
"strict": {
|
||||
"local_accept": 4.0,
|
||||
"fallback": false,
|
||||
"accepts": 3,
|
||||
"gate_accepts": 3,
|
||||
"fallback_accepts": 0,
|
||||
"by_source": {
|
||||
"local": 0,
|
||||
"text": 3
|
||||
},
|
||||
"rate": 0.0006550218340611354,
|
||||
"rate_percent": 0.0655,
|
||||
"crops_with_a_local_candidate": 155
|
||||
},
|
||||
"balanced": {
|
||||
"local_accept": 2.0,
|
||||
"fallback": false,
|
||||
"accepts": 8,
|
||||
"gate_accepts": 8,
|
||||
"fallback_accepts": 0,
|
||||
"by_source": {
|
||||
"local": 5,
|
||||
"text": 3
|
||||
},
|
||||
"rate": 0.0017467248908296944,
|
||||
"rate_percent": 0.1747,
|
||||
"crops_with_a_local_candidate": 155
|
||||
},
|
||||
"coverage-first": {
|
||||
"local_accept": 0.5,
|
||||
"fallback": false,
|
||||
"accepts": 83,
|
||||
"gate_accepts": 83,
|
||||
"fallback_accepts": 0,
|
||||
"by_source": {
|
||||
"local": 80,
|
||||
"text": 3
|
||||
},
|
||||
"rate": 0.018122270742358077,
|
||||
"rate_percent": 1.8122,
|
||||
"crops_with_a_local_candidate": 155
|
||||
},
|
||||
"near-argmax": {
|
||||
"local_accept": 0.0,
|
||||
"fallback": true,
|
||||
"accepts": 119,
|
||||
"gate_accepts": 115,
|
||||
"fallback_accepts": 4,
|
||||
"by_source": {
|
||||
"local": 112,
|
||||
"text": 3
|
||||
},
|
||||
"rate": 0.025982532751091702,
|
||||
"rate_percent": 2.5983,
|
||||
"crops_with_a_local_candidate": 155
|
||||
}
|
||||
},
|
||||
"cross_check_far_unknown_lines": {
|
||||
"source": "evidence/far-unknown-lines.json",
|
||||
"note": "accept-threshold cells only, no fallback; producer script not in the tree",
|
||||
"cells": {
|
||||
"strict": {
|
||||
"lines_accepts": 3,
|
||||
"lines_crops": 4580,
|
||||
"here_gate_accepts": 3,
|
||||
"agrees": true
|
||||
},
|
||||
"balanced": {
|
||||
"lines_accepts": 8,
|
||||
"lines_crops": 4580,
|
||||
"here_gate_accepts": 8,
|
||||
"agrees": true
|
||||
},
|
||||
"coverage-first": {
|
||||
"lines_accepts": 83,
|
||||
"lines_crops": 4580,
|
||||
"here_gate_accepts": 83,
|
||||
"agrees": true
|
||||
},
|
||||
"near-argmax": {
|
||||
"lines_accepts": 115,
|
||||
"lines_crops": 4580,
|
||||
"here_gate_accepts": 115,
|
||||
"agrees": true
|
||||
}
|
||||
}
|
||||
},
|
||||
"limitations": [
|
||||
"The cache records no catalog, code revision or image list, so 'same catalog as the frozen observations' is read off the current worktree, not attested by the cache.",
|
||||
"Only 155 of 4580 crops produced a local candidate; the other 4425 are decided by the text lane alone or refused before any gate applies, so the accept floor is exercised on a small minority of the rows and the rates are dominated by empty evidence.",
|
||||
"The catalog is 12 rows from one store. A false-accept rate is a function of catalog size, so these numbers do not transfer to a larger or differently-stocked catalog.",
|
||||
"Crops come from one detection pass at detect_conf 0.1 on 1280-long-side frames, so the crop population is set by the detector, not only by the identity gate.",
|
||||
"The cache has no per-photo breakdown, so no photo-level rate and no check that one image is not carrying the whole count.",
|
||||
"The near-argmax fallback is a research construct in analyze.py, not shipped code; its row prices a policy that does not exist on the server yet.",
|
||||
"far-unknown-lines.json's producer is not in the tree, so agreement with it is corroboration, not verification."
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
<svg xmlns="http://www.w3.org/2000/svg" width="1000" height="480" viewBox="0 0 1000 480"><rect width="1000" height="480" fill="white"/><text x="70" y="30" font-size="18">full: visible coverage / precision</text><path d="M70,50 V405 H460" fill="none" stroke="black"/><text x="60" y="430" font-size="12">0%</text><text x="30" y="405" font-size="12">0%</text><text x="157.5" y="430" font-size="12">25%</text><text x="30" y="317.5" font-size="12">25%</text><text x="255.0" y="430" font-size="12">50%</text><text x="30" y="230.0" font-size="12">50%</text><text x="352.5" y="430" font-size="12">75%</text><text x="30" y="142.5" font-size="12">75%</text><text x="450" y="430" font-size="12">100%</text><text x="30" y="55" font-size="12">100%</text><circle cx="227.43119266055047" cy="70.90909090909088" r="5" fill="white" stroke="#475569"><title>strict/individual: 44/109 shown, 42/64 correct, 2 false</title></circle><circle cx="266.7889908256881" cy="67.72727272727275" r="5" fill="#475569" stroke="#475569"><title>strict/guarded: 55/109 shown, 53/64 correct, 2 false</title></circle><circle cx="256.0550458715596" cy="81.9230769230769" r="5" fill="white" stroke="#2563eb"><title>balanced/individual: 52/109 shown, 48/64 correct, 4 false</title></circle><circle cx="281.10091743119267" cy="78.72881355932202" r="5" fill="#2563eb" stroke="#2563eb"><title>balanced/guarded: 59/109 shown, 55/64 correct, 4 false</title></circle><circle cx="288.25688073394497" cy="106.63934426229508" r="5" fill="white" stroke="#059669"><title>coverage-first/individual: 61/109 shown, 52/64 correct, 9 false</title></circle><circle cx="295.41284403669727" cy="99.44444444444446" r="5" fill="#059669" stroke="#059669"><title>coverage-first/guarded: 63/109 shown, 55/64 correct, 8 false</title></circle><circle cx="306.1467889908257" cy="129.24242424242425" r="5" fill="white" stroke="#dc2626"><title>near-argmax/individual: 66/109 shown, 52/64 correct, 14 false</title></circle><circle cx="313.302752293578" cy="121.91176470588238" r="5" fill="#dc2626" stroke="#dc2626"><title>near-argmax/guarded: 68/109 shown, 55/64 correct, 13 false</title></circle><text x="70" y="458" font-size="12">Any labels / 109; filled = guarded groups, hollow = individual</text><text x="560" y="30" font-size="18">live: visible coverage / precision</text><path d="M560,50 V405 H950" fill="none" stroke="black"/><text x="550" y="430" font-size="12">0%</text><text x="520" y="405" font-size="12">0%</text><text x="647.5" y="430" font-size="12">25%</text><text x="520" y="317.5" font-size="12">25%</text><text x="745.0" y="430" font-size="12">50%</text><text x="520" y="230.0" font-size="12">50%</text><text x="842.5" y="430" font-size="12">75%</text><text x="520" y="142.5" font-size="12">75%</text><text x="940" y="430" font-size="12">100%</text><text x="520" y="55" font-size="12">100%</text><circle cx="692.3853211009175" cy="64.45945945945942" r="5" fill="white" stroke="#475569"><title>strict/individual: 37/109 shown, 36/64 correct, 1 false</title></circle><circle cx="742.4770642201835" cy="61.86274509803923" r="5" fill="#475569" stroke="#475569"><title>strict/guarded: 51/109 shown, 50/64 correct, 1 false</title></circle><circle cx="699.5412844036697" cy="63.974358974359006" r="5" fill="white" stroke="#2563eb"><title>balanced/individual: 39/109 shown, 38/64 correct, 1 false</title></circle><circle cx="746.0550458715596" cy="61.730769230769226" r="5" fill="#2563eb" stroke="#2563eb"><title>balanced/guarded: 52/109 shown, 51/64 correct, 1 false</title></circle><circle cx="753.2110091743119" cy="100.37037037037038" r="5" fill="white" stroke="#059669"><title>coverage-first/individual: 54/109 shown, 47/64 correct, 7 false</title></circle><circle cx="778.256880733945" cy="89.42622950819674" r="5" fill="#059669" stroke="#059669"><title>coverage-first/guarded: 61/109 shown, 55/64 correct, 6 false</title></circle><circle cx="767.5229357798165" cy="115.34482758620692" r="5" fill="white" stroke="#dc2626"><title>near-argmax/individual: 58/109 shown, 48/64 correct, 10 false</title></circle><circle cx="788.9908256880734" cy="104.21875" r="5" fill="#dc2626" stroke="#dc2626"><title>near-argmax/guarded: 64/109 shown, 55/64 correct, 9 false</title></circle><text x="560" y="458" font-size="12">Any labels / 109; filled = guarded groups, hollow = individual</text></svg>
|
||||
|
After Width: | Height: | Size: 4.2 KiB |
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,788 @@
|
|||
{
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.78,
|
||||
"server_cpu_s": 63.94,
|
||||
"server_cpu_pct_one_core": 545.9,
|
||||
"rtt_ms": {
|
||||
"p50": 65.3,
|
||||
"p95": 102.3
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.28,
|
||||
"p95": 2.62
|
||||
},
|
||||
"detect": {
|
||||
"p50": 57.9,
|
||||
"p95": 85.93
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.64,
|
||||
"p95": 14.19
|
||||
},
|
||||
"total": {
|
||||
"p50": 61.09,
|
||||
"p95": 97.65
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.02,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-0.5x-1": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.79,
|
||||
"server_cpu_s": 64.79,
|
||||
"server_cpu_pct_one_core": 552.6,
|
||||
"rtt_ms": {
|
||||
"p50": 63.1,
|
||||
"p95": 90.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.28,
|
||||
"p95": 2.73
|
||||
},
|
||||
"detect": {
|
||||
"p50": 56.28,
|
||||
"p95": 82.78
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.43,
|
||||
"p95": 2.22
|
||||
},
|
||||
"total": {
|
||||
"p50": 60.12,
|
||||
"p95": 86.55
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.02,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 6.94,
|
||||
"server_cpu_s": 34.88,
|
||||
"server_cpu_pct_one_core": 507.4,
|
||||
"rtt_ms": {
|
||||
"p50": 55.6,
|
||||
"p95": 102.3
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.21,
|
||||
"p95": 2.62
|
||||
},
|
||||
"detect": {
|
||||
"p50": 49.16,
|
||||
"p95": 92.26
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.4,
|
||||
"p95": 4.58
|
||||
},
|
||||
"total": {
|
||||
"p50": 52.97,
|
||||
"p95": 95.81
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 8.06,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-1x-1": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 6.94,
|
||||
"server_cpu_s": 36.14,
|
||||
"server_cpu_pct_one_core": 525.6,
|
||||
"rtt_ms": {
|
||||
"p50": 59.6,
|
||||
"p95": 91.6
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.29,
|
||||
"p95": 2.7
|
||||
},
|
||||
"detect": {
|
||||
"p50": 54.53,
|
||||
"p95": 85.74
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.54,
|
||||
"p95": 4.01
|
||||
},
|
||||
"total": {
|
||||
"p50": 57.78,
|
||||
"p95": 89.62
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 7.98,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-image6-15fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 15,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 75,
|
||||
"sent": 75,
|
||||
"processed": 72,
|
||||
"not_returned": 3,
|
||||
"elapsed_s": 12.0,
|
||||
"server_cpu_s": 77.48,
|
||||
"server_cpu_pct_one_core": 647.9,
|
||||
"rtt_ms": {
|
||||
"p50": 111.4,
|
||||
"p95": 175.8
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.82,
|
||||
"p95": 10.94
|
||||
},
|
||||
"detect": {
|
||||
"p50": 62.72,
|
||||
"p95": 125.26
|
||||
},
|
||||
"track": {
|
||||
"p50": 6.79,
|
||||
"p95": 15.05
|
||||
},
|
||||
"total": {
|
||||
"p50": 80.12,
|
||||
"p95": 139.71
|
||||
}
|
||||
},
|
||||
"max_tracks": 82,
|
||||
"ever_labelled_track_ids": 58,
|
||||
"first_visible_ms": 372.83754197414964,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 75,
|
||||
"frames_processed": 72,
|
||||
"frames_dropped": 3,
|
||||
"server_fps": 7.6,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "810e38cff9ca71b1f072de0ffbb47de3834221a6a3e4ba04989c22977b39a28f",
|
||||
"visible": {
|
||||
"visible_facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"ever_named": 58,
|
||||
"stable_correct_catalog": 51,
|
||||
"stable_correct_coverage": 0.797,
|
||||
"blank_ratio": 0.468,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 109,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 108,
|
||||
"any_label": 3,
|
||||
"correct_label_ever": 3,
|
||||
"vrc": 0.028,
|
||||
"correct_coverage": 0.028
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 103,
|
||||
"any_label": 13,
|
||||
"correct_label_ever": 13,
|
||||
"vrc": 0.126,
|
||||
"correct_coverage": 0.126
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 2048.8,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 42468.7,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
},
|
||||
"replay-image6-15fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 15,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 75,
|
||||
"sent": 75,
|
||||
"processed": 38,
|
||||
"not_returned": 37,
|
||||
"elapsed_s": 7.11,
|
||||
"server_cpu_s": 37.87,
|
||||
"server_cpu_pct_one_core": 538.7,
|
||||
"rtt_ms": {
|
||||
"p50": 150.7,
|
||||
"p95": 272.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.82,
|
||||
"p95": 10.29
|
||||
},
|
||||
"detect": {
|
||||
"p50": 56.53,
|
||||
"p95": 172.17
|
||||
},
|
||||
"track": {
|
||||
"p50": 7.95,
|
||||
"p95": 82.75
|
||||
},
|
||||
"total": {
|
||||
"p50": 74.43,
|
||||
"p95": 220.09
|
||||
}
|
||||
},
|
||||
"max_tracks": 82,
|
||||
"ever_labelled_track_ids": 40,
|
||||
"first_visible_ms": 645.1856669737026,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 75,
|
||||
"frames_processed": 38,
|
||||
"frames_dropped": 37,
|
||||
"server_fps": 7.81,
|
||||
"classify_pending": 16,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "810e38cff9ca71b1f072de0ffbb47de3834221a6a3e4ba04989c22977b39a28f",
|
||||
"visible": {
|
||||
"visible_facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"ever_named": 38,
|
||||
"stable_correct_catalog": 35,
|
||||
"stable_correct_coverage": 0.547,
|
||||
"blank_ratio": 0.651,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 108,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 106,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 100,
|
||||
"any_label": 13,
|
||||
"correct_label_ever": 13,
|
||||
"vrc": 0.13,
|
||||
"correct_coverage": 0.13
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1401.0,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 5073.2,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.91,
|
||||
"server_cpu_s": 69.74,
|
||||
"server_cpu_pct_one_core": 588.2,
|
||||
"rtt_ms": {
|
||||
"p50": 100.4,
|
||||
"p95": 158.0
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.62,
|
||||
"p95": 10.66
|
||||
},
|
||||
"detect": {
|
||||
"p50": 65.52,
|
||||
"p95": 97.8
|
||||
},
|
||||
"track": {
|
||||
"p50": 4.6,
|
||||
"p95": 10.48
|
||||
},
|
||||
"total": {
|
||||
"p50": 78.76,
|
||||
"p95": 122.88
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 57,
|
||||
"first_visible_ms": 623.0825000675395,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 3.98,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 57,
|
||||
"stable_correct_catalog": 49,
|
||||
"stable_correct_coverage": 0.817,
|
||||
"blank_ratio": 0.394,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 93,
|
||||
"any_label": 11,
|
||||
"correct_label_ever": 11,
|
||||
"vrc": 0.118,
|
||||
"correct_coverage": 0.118
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1789.5,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 56251.4,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-0.5x-1": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.9,
|
||||
"server_cpu_s": 69.78,
|
||||
"server_cpu_pct_one_core": 588.5,
|
||||
"rtt_ms": {
|
||||
"p50": 104.2,
|
||||
"p95": 197.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.89,
|
||||
"p95": 10.12
|
||||
},
|
||||
"detect": {
|
||||
"p50": 65.43,
|
||||
"p95": 91.23
|
||||
},
|
||||
"track": {
|
||||
"p50": 5.53,
|
||||
"p95": 24.17
|
||||
},
|
||||
"total": {
|
||||
"p50": 79.25,
|
||||
"p95": 158.8
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 53,
|
||||
"first_visible_ms": 723.3483329182491,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.0,
|
||||
"classify_pending": 5,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 53,
|
||||
"stable_correct_catalog": 48,
|
||||
"stable_correct_coverage": 0.8,
|
||||
"blank_ratio": 0.436,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 93,
|
||||
"any_label": 3,
|
||||
"correct_label_ever": 3,
|
||||
"vrc": 0.032,
|
||||
"correct_coverage": 0.032
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 2809.8,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 30646.1,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 39,
|
||||
"not_returned": 1,
|
||||
"elapsed_s": 6.99,
|
||||
"server_cpu_s": 41.58,
|
||||
"server_cpu_pct_one_core": 598.9,
|
||||
"rtt_ms": {
|
||||
"p50": 176.5,
|
||||
"p95": 234.0
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.88,
|
||||
"p95": 10.85
|
||||
},
|
||||
"detect": {
|
||||
"p50": 60.34,
|
||||
"p95": 162.65
|
||||
},
|
||||
"track": {
|
||||
"p50": 7.29,
|
||||
"p95": 21.33
|
||||
},
|
||||
"total": {
|
||||
"p50": 81.93,
|
||||
"p95": 178.03
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 50,
|
||||
"first_visible_ms": 433.02949995268136,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 39,
|
||||
"frames_dropped": 1,
|
||||
"server_fps": 8.19,
|
||||
"classify_pending": 16,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 50,
|
||||
"stable_correct_catalog": 46,
|
||||
"stable_correct_coverage": 0.767,
|
||||
"blank_ratio": 0.468,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 93,
|
||||
"any_label": 1,
|
||||
"correct_label_ever": 1,
|
||||
"vrc": 0.011,
|
||||
"correct_coverage": 0.011
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 88,
|
||||
"any_label": 10,
|
||||
"correct_label_ever": 10,
|
||||
"vrc": 0.114,
|
||||
"correct_coverage": 0.114
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1990.1,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 4131.2,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-1x-1": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 7.0,
|
||||
"server_cpu_s": 42.49,
|
||||
"server_cpu_pct_one_core": 612.2,
|
||||
"rtt_ms": {
|
||||
"p50": 101.8,
|
||||
"p95": 150.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.84,
|
||||
"p95": 10.59
|
||||
},
|
||||
"detect": {
|
||||
"p50": 59.15,
|
||||
"p95": 90.7
|
||||
},
|
||||
"track": {
|
||||
"p50": 6.55,
|
||||
"p95": 14.88
|
||||
},
|
||||
"total": {
|
||||
"p50": 75.24,
|
||||
"p95": 107.44
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 52,
|
||||
"first_visible_ms": 350.2598749473691,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 7.98,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 52,
|
||||
"stable_correct_catalog": 47,
|
||||
"stable_correct_coverage": 0.783,
|
||||
"blank_ratio": 0.447,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"any_label": 0,
|
||||
"correct_label_ever": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 93,
|
||||
"any_label": 7,
|
||||
"correct_label_ever": 7,
|
||||
"vrc": 0.075,
|
||||
"correct_coverage": 0.075
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 88,
|
||||
"any_label": 15,
|
||||
"correct_label_ever": 15,
|
||||
"vrc": 0.17,
|
||||
"correct_coverage": 0.17
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1620.5,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"observed_wrong_label_ms_upper": 12247.0,
|
||||
"note": "Server receive timestamp as proxy for UI display; per-facing entry from 50% visible source window, not physical entrance. Unmatched track labels excluded. Wrong-label ms is sampled persistence only."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,830 @@
|
|||
{
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.78,
|
||||
"server_cpu_s": 63.94,
|
||||
"server_cpu_pct_one_core": 545.9,
|
||||
"rtt_ms": {
|
||||
"p50": 65.3,
|
||||
"p95": 102.3
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.28,
|
||||
"p95": 2.62
|
||||
},
|
||||
"detect": {
|
||||
"p50": 57.9,
|
||||
"p95": 85.93
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.64,
|
||||
"p95": 14.19
|
||||
},
|
||||
"total": {
|
||||
"p50": 61.09,
|
||||
"p95": 97.65
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.02,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-0.5x-1": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.79,
|
||||
"server_cpu_s": 64.79,
|
||||
"server_cpu_pct_one_core": 552.6,
|
||||
"rtt_ms": {
|
||||
"p50": 63.1,
|
||||
"p95": 90.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.28,
|
||||
"p95": 2.73
|
||||
},
|
||||
"detect": {
|
||||
"p50": 56.28,
|
||||
"p95": 82.78
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.43,
|
||||
"p95": 2.22
|
||||
},
|
||||
"total": {
|
||||
"p50": 60.12,
|
||||
"p95": 86.55
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.02,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 6.94,
|
||||
"server_cpu_s": 34.88,
|
||||
"server_cpu_pct_one_core": 507.4,
|
||||
"rtt_ms": {
|
||||
"p50": 55.6,
|
||||
"p95": 102.3
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.21,
|
||||
"p95": 2.62
|
||||
},
|
||||
"detect": {
|
||||
"p50": 49.16,
|
||||
"p95": 92.26
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.4,
|
||||
"p95": 4.58
|
||||
},
|
||||
"total": {
|
||||
"p50": 52.97,
|
||||
"p95": 95.81
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 8.06,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-hanzomon-supermarket-pan-132-152-8fps-1x-1": {
|
||||
"summary": {
|
||||
"source": "data/external-eval/tier3-sources/media/hanzomon-supermarket-pan-132-152.mp4",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 6.94,
|
||||
"server_cpu_s": 36.14,
|
||||
"server_cpu_pct_one_core": 525.6,
|
||||
"rtt_ms": {
|
||||
"p50": 59.6,
|
||||
"p95": 91.6
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 2.29,
|
||||
"p95": 2.7
|
||||
},
|
||||
"detect": {
|
||||
"p50": 54.53,
|
||||
"p95": 85.74
|
||||
},
|
||||
"track": {
|
||||
"p50": 1.54,
|
||||
"p95": 4.01
|
||||
},
|
||||
"total": {
|
||||
"p50": 57.78,
|
||||
"p95": 89.62
|
||||
}
|
||||
},
|
||||
"max_tracks": 17,
|
||||
"ever_labelled_track_ids": 0,
|
||||
"first_visible_ms": null,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 7.98,
|
||||
"classify_pending": 0,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "d5f77b68cb9e7e776f473c8d869e03c4fa23db2c0f6cb8b34c3b218ea12147b8"
|
||||
},
|
||||
"replay-image6-15fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 15,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 75,
|
||||
"sent": 75,
|
||||
"processed": 72,
|
||||
"not_returned": 3,
|
||||
"elapsed_s": 12.0,
|
||||
"server_cpu_s": 77.48,
|
||||
"server_cpu_pct_one_core": 647.9,
|
||||
"rtt_ms": {
|
||||
"p50": 111.4,
|
||||
"p95": 175.8
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.82,
|
||||
"p95": 10.94
|
||||
},
|
||||
"detect": {
|
||||
"p50": 62.72,
|
||||
"p95": 125.26
|
||||
},
|
||||
"track": {
|
||||
"p50": 6.79,
|
||||
"p95": 15.05
|
||||
},
|
||||
"total": {
|
||||
"p50": 80.12,
|
||||
"p95": 139.71
|
||||
}
|
||||
},
|
||||
"max_tracks": 82,
|
||||
"ever_labelled_track_ids": 58,
|
||||
"first_visible_ms": 372.83754197414964,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 75,
|
||||
"frames_processed": 72,
|
||||
"frames_dropped": 3,
|
||||
"server_fps": 7.6,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "810e38cff9ca71b1f072de0ffbb47de3834221a6a3e4ba04989c22977b39a28f",
|
||||
"visible": {
|
||||
"visible_facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"ever_named": 58,
|
||||
"never_named_ratio": 0.4678899082568807,
|
||||
"last_observed_correct_catalog": 45,
|
||||
"last_observed_correct_coverage": 0.703125,
|
||||
"stable_correct_catalog_500ms": 44,
|
||||
"stable_correct_coverage_500ms": 0.6875,
|
||||
"time_to_trailing_stable_correct_ms_p50": 3453.6,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 109,
|
||||
"catalog_denominator": 64,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 108,
|
||||
"catalog_denominator": 64,
|
||||
"any_label": 3,
|
||||
"correct_label": 3,
|
||||
"vrc": 0.027777777777777776,
|
||||
"correct_catalog_coverage": 0.046875
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 103,
|
||||
"catalog_denominator": 64,
|
||||
"any_label": 13,
|
||||
"correct_label": 13,
|
||||
"vrc": 0.1262135922330097,
|
||||
"correct_catalog_coverage": 0.203125
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 2048.85,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 34939.099999999984,
|
||||
"unmatched_label_samples": 11,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
},
|
||||
"replay-image6-15fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 15,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 75,
|
||||
"sent": 75,
|
||||
"processed": 38,
|
||||
"not_returned": 37,
|
||||
"elapsed_s": 7.11,
|
||||
"server_cpu_s": 37.87,
|
||||
"server_cpu_pct_one_core": 538.7,
|
||||
"rtt_ms": {
|
||||
"p50": 150.7,
|
||||
"p95": 272.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.82,
|
||||
"p95": 10.29
|
||||
},
|
||||
"detect": {
|
||||
"p50": 56.53,
|
||||
"p95": 172.17
|
||||
},
|
||||
"track": {
|
||||
"p50": 7.95,
|
||||
"p95": 82.75
|
||||
},
|
||||
"total": {
|
||||
"p50": 74.43,
|
||||
"p95": 220.09
|
||||
}
|
||||
},
|
||||
"max_tracks": 82,
|
||||
"ever_labelled_track_ids": 40,
|
||||
"first_visible_ms": 645.1856669737026,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 75,
|
||||
"frames_processed": 38,
|
||||
"frames_dropped": 37,
|
||||
"server_fps": 7.81,
|
||||
"classify_pending": 16,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "810e38cff9ca71b1f072de0ffbb47de3834221a6a3e4ba04989c22977b39a28f",
|
||||
"visible": {
|
||||
"visible_facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"ever_named": 38,
|
||||
"never_named_ratio": 0.6513761467889908,
|
||||
"last_observed_correct_catalog": 17,
|
||||
"last_observed_correct_coverage": 0.265625,
|
||||
"stable_correct_catalog_500ms": 8,
|
||||
"stable_correct_coverage_500ms": 0.125,
|
||||
"time_to_trailing_stable_correct_ms_p50": 1066.75,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 108,
|
||||
"catalog_denominator": 64,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 106,
|
||||
"catalog_denominator": 64,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 100,
|
||||
"catalog_denominator": 63,
|
||||
"any_label": 8,
|
||||
"correct_label": 8,
|
||||
"vrc": 0.08,
|
||||
"correct_catalog_coverage": 0.12698412698412698
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1445.8,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 970.7999999999995,
|
||||
"unmatched_label_samples": 19,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-0.5x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.91,
|
||||
"server_cpu_s": 69.74,
|
||||
"server_cpu_pct_one_core": 588.2,
|
||||
"rtt_ms": {
|
||||
"p50": 100.4,
|
||||
"p95": 158.0
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.62,
|
||||
"p95": 10.66
|
||||
},
|
||||
"detect": {
|
||||
"p50": 65.52,
|
||||
"p95": 97.8
|
||||
},
|
||||
"track": {
|
||||
"p50": 4.6,
|
||||
"p95": 10.48
|
||||
},
|
||||
"total": {
|
||||
"p50": 78.76,
|
||||
"p95": 122.88
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 57,
|
||||
"first_visible_ms": 623.0825000675395,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 3.98,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 57,
|
||||
"never_named_ratio": 0.3936170212765957,
|
||||
"last_observed_correct_catalog": 48,
|
||||
"last_observed_correct_coverage": 0.8,
|
||||
"stable_correct_catalog_500ms": 48,
|
||||
"stable_correct_coverage_500ms": 0.8,
|
||||
"time_to_trailing_stable_correct_ms_p50": 1789.5,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 93,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 11,
|
||||
"correct_label": 11,
|
||||
"vrc": 0.11827956989247312,
|
||||
"correct_catalog_coverage": 0.1864406779661017
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1789.5,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 56251.400000000016,
|
||||
"unmatched_label_samples": 4,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-0.5x-1": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 0.5,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 11.9,
|
||||
"server_cpu_s": 69.78,
|
||||
"server_cpu_pct_one_core": 588.5,
|
||||
"rtt_ms": {
|
||||
"p50": 104.2,
|
||||
"p95": 197.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.89,
|
||||
"p95": 10.12
|
||||
},
|
||||
"detect": {
|
||||
"p50": 65.43,
|
||||
"p95": 91.23
|
||||
},
|
||||
"track": {
|
||||
"p50": 5.53,
|
||||
"p95": 24.17
|
||||
},
|
||||
"total": {
|
||||
"p50": 79.25,
|
||||
"p95": 158.8
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 53,
|
||||
"first_visible_ms": 723.3483329182491,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 4.0,
|
||||
"classify_pending": 5,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 53,
|
||||
"never_named_ratio": 0.43617021276595747,
|
||||
"last_observed_correct_catalog": 47,
|
||||
"last_observed_correct_coverage": 0.7833333333333333,
|
||||
"stable_correct_catalog_500ms": 46,
|
||||
"stable_correct_coverage_500ms": 0.7666666666666667,
|
||||
"time_to_trailing_stable_correct_ms_p50": 2571.0,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 93,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 3,
|
||||
"correct_label": 3,
|
||||
"vrc": 0.03225806451612903,
|
||||
"correct_catalog_coverage": 0.05084745762711865
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 2809.8,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 30646.09999999999,
|
||||
"unmatched_label_samples": 4,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-1x-0": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 0,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 39,
|
||||
"not_returned": 1,
|
||||
"elapsed_s": 6.99,
|
||||
"server_cpu_s": 41.58,
|
||||
"server_cpu_pct_one_core": 598.9,
|
||||
"rtt_ms": {
|
||||
"p50": 176.5,
|
||||
"p95": 234.0
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.88,
|
||||
"p95": 10.85
|
||||
},
|
||||
"detect": {
|
||||
"p50": 60.34,
|
||||
"p95": 162.65
|
||||
},
|
||||
"track": {
|
||||
"p50": 7.29,
|
||||
"p95": 21.33
|
||||
},
|
||||
"total": {
|
||||
"p50": 81.93,
|
||||
"p95": 178.03
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 50,
|
||||
"first_visible_ms": 433.02949995268136,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 39,
|
||||
"frames_dropped": 1,
|
||||
"server_fps": 8.19,
|
||||
"classify_pending": 16,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 50,
|
||||
"never_named_ratio": 0.46808510638297873,
|
||||
"last_observed_correct_catalog": 41,
|
||||
"last_observed_correct_coverage": 0.6833333333333333,
|
||||
"stable_correct_catalog_500ms": 31,
|
||||
"stable_correct_coverage_500ms": 0.5166666666666667,
|
||||
"time_to_trailing_stable_correct_ms_p50": 2221.0,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 93,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 1,
|
||||
"correct_label": 1,
|
||||
"vrc": 0.010752688172043012,
|
||||
"correct_catalog_coverage": 0.01694915254237288
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 88,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 10,
|
||||
"correct_label": 10,
|
||||
"vrc": 0.11363636363636363,
|
||||
"correct_catalog_coverage": 0.1694915254237288
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1990.0999999999997,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 3505.4999999999986,
|
||||
"unmatched_label_samples": 5,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
},
|
||||
"replay-image6-8fps-1x-1": {
|
||||
"summary": {
|
||||
"source": "image6",
|
||||
"fps": 8,
|
||||
"rate": 1,
|
||||
"trial": 1,
|
||||
"input_frames": 40,
|
||||
"sent": 40,
|
||||
"processed": 40,
|
||||
"not_returned": 0,
|
||||
"elapsed_s": 7.0,
|
||||
"server_cpu_s": 42.49,
|
||||
"server_cpu_pct_one_core": 612.2,
|
||||
"rtt_ms": {
|
||||
"p50": 101.8,
|
||||
"p95": 150.2
|
||||
},
|
||||
"stages_ms": {
|
||||
"decode": {
|
||||
"p50": 8.84,
|
||||
"p95": 10.59
|
||||
},
|
||||
"detect": {
|
||||
"p50": 59.15,
|
||||
"p95": 90.7
|
||||
},
|
||||
"track": {
|
||||
"p50": 6.55,
|
||||
"p95": 14.88
|
||||
},
|
||||
"total": {
|
||||
"p50": 75.24,
|
||||
"p95": 107.44
|
||||
}
|
||||
},
|
||||
"max_tracks": 80,
|
||||
"ever_labelled_track_ids": 52,
|
||||
"first_visible_ms": 350.2598749473691,
|
||||
"label_flips": 0,
|
||||
"errors": [],
|
||||
"latest_stats": {
|
||||
"crops": {
|
||||
"requested": 0,
|
||||
"accepted": 0,
|
||||
"rejected": 0
|
||||
},
|
||||
"frames_received": 40,
|
||||
"frames_processed": 40,
|
||||
"frames_dropped": 0,
|
||||
"server_fps": 7.98,
|
||||
"classify_pending": 3,
|
||||
"classify_failures": 0,
|
||||
"classify_ms_p50": null
|
||||
}
|
||||
},
|
||||
"input_hash": "bd59b38fb89c59b96c0a53157e27512be7c11754b5188890efa7496b43e5e44f",
|
||||
"visible": {
|
||||
"visible_facings": 94,
|
||||
"catalog_facings": 60,
|
||||
"ever_named": 52,
|
||||
"never_named_ratio": 0.44680851063829785,
|
||||
"last_observed_correct_catalog": 46,
|
||||
"last_observed_correct_coverage": 0.7666666666666667,
|
||||
"stable_correct_catalog_500ms": 45,
|
||||
"stable_correct_coverage_500ms": 0.75,
|
||||
"time_to_trailing_stable_correct_ms_p50": 1856.7999999999997,
|
||||
"timed": {
|
||||
"250": {
|
||||
"visible_denominator": 94,
|
||||
"catalog_denominator": 60,
|
||||
"any_label": 0,
|
||||
"correct_label": 0,
|
||||
"vrc": 0.0,
|
||||
"correct_catalog_coverage": 0.0
|
||||
},
|
||||
"500": {
|
||||
"visible_denominator": 93,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 7,
|
||||
"correct_label": 7,
|
||||
"vrc": 0.07526881720430108,
|
||||
"correct_catalog_coverage": 0.11864406779661017
|
||||
},
|
||||
"1000": {
|
||||
"visible_denominator": 88,
|
||||
"catalog_denominator": 59,
|
||||
"any_label": 15,
|
||||
"correct_label": 15,
|
||||
"vrc": 0.17045454545454544,
|
||||
"correct_catalog_coverage": 0.2542372881355932
|
||||
}
|
||||
},
|
||||
"first_label_from_entry_ms_p50": 1620.4500000000003,
|
||||
"corrections": [],
|
||||
"identity_flips": 0,
|
||||
"sampled_wrong_label_ms": 11646.300000000003,
|
||||
"unmatched_label_samples": 4,
|
||||
"note": "Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification."
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,616 @@
|
|||
{
|
||||
"provenance": {
|
||||
"code": "fbf31d74feb12538dc07207cb80722dac38dd438",
|
||||
"catalog": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
||||
"labels": "6fc1a220a97c75666882b4bcd9dc102ed08c8924677b97d615850a0983286c1a",
|
||||
"image": "a08b89ba37d27c9d5a81bcd56bcd042e6baf54d2f182bbaaf47e97f14c9cd525",
|
||||
"opencv_threads": 1,
|
||||
"seed": 20260929
|
||||
},
|
||||
"arms": {
|
||||
"full": {
|
||||
"sharpest-one": {
|
||||
"matcher_calls": 48,
|
||||
"matcher_ms_serial_projection": 571.3,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 1129.5,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 39,
|
||||
"correct": 33,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.3577981651376147,
|
||||
"correct_coverage": 0.515625,
|
||||
"precision": 0.8461538461538461,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.6422018348623852,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 19
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 51,
|
||||
"correct": 46,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.46788990825688076,
|
||||
"correct_coverage": 0.71875,
|
||||
"precision": 0.9019607843137255,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5321100917431192,
|
||||
"rescued_correct": 1,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 8
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"seed": 10,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"seed": 10,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 36,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 42,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 58,
|
||||
"seed": 59,
|
||||
"from": null,
|
||||
"to": "sku_l10"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sharpest-two": {
|
||||
"matcher_calls": 71,
|
||||
"matcher_ms_serial_projection": 870.3,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 1428.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 45,
|
||||
"correct": 39,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.41284403669724773,
|
||||
"correct_coverage": 0.609375,
|
||||
"precision": 0.8666666666666667,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5871559633027523,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 13
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 53,
|
||||
"correct": 48,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.48623853211009177,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.9056603773584906,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5137614678899083,
|
||||
"rescued_correct": 1,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 6
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"seed": 11,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"seed": 11,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 36,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
}
|
||||
]
|
||||
},
|
||||
"all": {
|
||||
"matcher_calls": 109,
|
||||
"matcher_ms_serial_projection": 1466.3,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 2024.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 61,
|
||||
"correct": 52,
|
||||
"wrong": 9,
|
||||
"visible_coverage": 0.5596330275229358,
|
||||
"correct_coverage": 0.8125,
|
||||
"precision": 0.8524590163934426,
|
||||
"false_label_rate": 0.08256880733944955,
|
||||
"unknown_false_accepts": 7,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.44036697247706424,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 63,
|
||||
"correct": 55,
|
||||
"wrong": 8,
|
||||
"visible_coverage": 0.5779816513761468,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.873015873015873,
|
||||
"false_label_rate": 0.07339449541284404,
|
||||
"unknown_false_accepts": 7,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.42201834862385323,
|
||||
"rescued_correct": 2,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 73,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"live": {
|
||||
"sharpest-one": {
|
||||
"matcher_calls": 55,
|
||||
"matcher_ms_serial_projection": 276.1,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 485.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 37,
|
||||
"correct": 33,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.3394495412844037,
|
||||
"correct_coverage": 0.515625,
|
||||
"precision": 0.8918918918918919,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6605504587155964,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 14
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 47,
|
||||
"correct": 43,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.43119266055045874,
|
||||
"correct_coverage": 0.671875,
|
||||
"precision": 0.9148936170212766,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5688073394495412,
|
||||
"rescued_correct": 3,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 8
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 45,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 54,
|
||||
"seed": 56,
|
||||
"from": null,
|
||||
"to": "sku_l10"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sharpest-two": {
|
||||
"matcher_calls": 77,
|
||||
"matcher_ms_serial_projection": 408.0,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 617.3,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 44,
|
||||
"correct": 38,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.4036697247706422,
|
||||
"correct_coverage": 0.59375,
|
||||
"precision": 0.8636363636363636,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5963302752293578,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 9
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 53,
|
||||
"correct": 48,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.48623853211009177,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.9056603773584906,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5137614678899083,
|
||||
"rescued_correct": 4,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 4
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"seed": 12,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 34,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 40,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 44,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
}
|
||||
]
|
||||
},
|
||||
"all": {
|
||||
"matcher_calls": 109,
|
||||
"matcher_ms_serial_projection": 618.2,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 827.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 54,
|
||||
"correct": 47,
|
||||
"wrong": 7,
|
||||
"visible_coverage": 0.4954128440366973,
|
||||
"correct_coverage": 0.734375,
|
||||
"precision": 0.8703703703703703,
|
||||
"false_label_rate": 0.06422018348623854,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.5045871559633027,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 61,
|
||||
"correct": 55,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.5596330275229358,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.9016393442622951,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.44036697247706424,
|
||||
"rescued_correct": 7,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 1,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"seed": 13,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 42,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 69,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
},
|
||||
{
|
||||
"id": 73,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
},
|
||||
{
|
||||
"id": 75,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,616 @@
|
|||
{
|
||||
"provenance": {
|
||||
"code": "fbf31d74feb12538dc07207cb80722dac38dd438",
|
||||
"catalog": "1bc3c1d10087f92192b8081a007fa72b4f091446c273a0ab3825cbed9bdf602c",
|
||||
"labels": "6fc1a220a97c75666882b4bcd9dc102ed08c8924677b97d615850a0983286c1a",
|
||||
"image": "a08b89ba37d27c9d5a81bcd56bcd042e6baf54d2f182bbaaf47e97f14c9cd525",
|
||||
"opencv_threads": 1,
|
||||
"seed": 20260929
|
||||
},
|
||||
"arms": {
|
||||
"full": {
|
||||
"sharpest-one": {
|
||||
"matcher_calls": 47,
|
||||
"matcher_ms_serial_projection": 556.1,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 1114.2,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 38,
|
||||
"correct": 33,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.3486238532110092,
|
||||
"correct_coverage": 0.515625,
|
||||
"precision": 0.868421052631579,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.6513761467889908,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 19
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 50,
|
||||
"correct": 46,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.45871559633027525,
|
||||
"correct_coverage": 0.71875,
|
||||
"precision": 0.92,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5412844036697247,
|
||||
"rescued_correct": 1,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 8
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"seed": 10,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"seed": 10,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 18,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 36,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 42,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 58,
|
||||
"seed": 59,
|
||||
"from": null,
|
||||
"to": "sku_l10"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sharpest-two": {
|
||||
"matcher_calls": 70,
|
||||
"matcher_ms_serial_projection": 855.0,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 1413.2,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 44,
|
||||
"correct": 39,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.4036697247706422,
|
||||
"correct_coverage": 0.609375,
|
||||
"precision": 0.8863636363636364,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5963302752293578,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 13
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 52,
|
||||
"correct": 48,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.47706422018348627,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.9230769230769231,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5229357798165137,
|
||||
"rescued_correct": 1,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 6
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 12,
|
||||
"seed": 11,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 13,
|
||||
"seed": 11,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 17,
|
||||
"seed": 19,
|
||||
"from": null,
|
||||
"to": "sku_l05"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 36,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
}
|
||||
]
|
||||
},
|
||||
"all": {
|
||||
"matcher_calls": 109,
|
||||
"matcher_ms_serial_projection": 1466.3,
|
||||
"group_descriptor_ms": 512.5,
|
||||
"pair_match_ms": 45.6,
|
||||
"total_identity_and_group_ms_serial_projection": 2024.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 61,
|
||||
"correct": 52,
|
||||
"wrong": 9,
|
||||
"visible_coverage": 0.5596330275229358,
|
||||
"correct_coverage": 0.8125,
|
||||
"precision": 0.8524590163934426,
|
||||
"false_label_rate": 0.08256880733944955,
|
||||
"unknown_false_accepts": 7,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.44036697247706424,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 63,
|
||||
"correct": 55,
|
||||
"wrong": 8,
|
||||
"visible_coverage": 0.5779816513761468,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.873015873015873,
|
||||
"false_label_rate": 0.07339449541284404,
|
||||
"unknown_false_accepts": 7,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.42201834862385323,
|
||||
"rescued_correct": 2,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 73,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
"live": {
|
||||
"sharpest-one": {
|
||||
"matcher_calls": 55,
|
||||
"matcher_ms_serial_projection": 276.1,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 485.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 37,
|
||||
"correct": 33,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.3394495412844037,
|
||||
"correct_coverage": 0.515625,
|
||||
"precision": 0.8918918918918919,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.6605504587155964,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 14
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 47,
|
||||
"correct": 43,
|
||||
"wrong": 4,
|
||||
"visible_coverage": 0.43119266055045874,
|
||||
"correct_coverage": 0.671875,
|
||||
"precision": 0.9148936170212766,
|
||||
"false_label_rate": 0.03669724770642202,
|
||||
"unknown_false_accepts": 4,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5688073394495412,
|
||||
"rescued_correct": 3,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 8
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 35,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 39,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 45,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 54,
|
||||
"seed": 56,
|
||||
"from": null,
|
||||
"to": "sku_l10"
|
||||
}
|
||||
]
|
||||
},
|
||||
"sharpest-two": {
|
||||
"matcher_calls": 77,
|
||||
"matcher_ms_serial_projection": 408.0,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 617.3,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 44,
|
||||
"correct": 38,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.4036697247706422,
|
||||
"correct_coverage": 0.59375,
|
||||
"precision": 0.8636363636363636,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.5963302752293578,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 9
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 53,
|
||||
"correct": 48,
|
||||
"wrong": 5,
|
||||
"visible_coverage": 0.48623853211009177,
|
||||
"correct_coverage": 0.75,
|
||||
"precision": 0.9056603773584906,
|
||||
"false_label_rate": 0.045871559633027525,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 0,
|
||||
"wrong_catalog": 0,
|
||||
"blank_ratio": 0.5137614678899083,
|
||||
"rescued_correct": 4,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 4
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"seed": 4,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"seed": 12,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 20,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 21,
|
||||
"seed": 23,
|
||||
"from": null,
|
||||
"to": "sku_l06"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 33,
|
||||
"seed": 34,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 38,
|
||||
"seed": 40,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 44,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
}
|
||||
]
|
||||
},
|
||||
"all": {
|
||||
"matcher_calls": 109,
|
||||
"matcher_ms_serial_projection": 618.2,
|
||||
"group_descriptor_ms": 197.9,
|
||||
"pair_match_ms": 11.4,
|
||||
"total_identity_and_group_ms_serial_projection": 827.4,
|
||||
"individual": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 54,
|
||||
"correct": 47,
|
||||
"wrong": 7,
|
||||
"visible_coverage": 0.4954128440366973,
|
||||
"correct_coverage": 0.734375,
|
||||
"precision": 0.8703703703703703,
|
||||
"false_label_rate": 0.06422018348623854,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 2,
|
||||
"blank_ratio": 0.5045871559633027,
|
||||
"rescued_correct": 0,
|
||||
"corrected_wrong": 0,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"inferred": {
|
||||
"facings": 109,
|
||||
"catalog_facings": 64,
|
||||
"shown": 61,
|
||||
"correct": 55,
|
||||
"wrong": 6,
|
||||
"visible_coverage": 0.5596330275229358,
|
||||
"correct_coverage": 0.859375,
|
||||
"precision": 0.9016393442622951,
|
||||
"false_label_rate": 0.05504587155963303,
|
||||
"unknown_false_accepts": 5,
|
||||
"unknown_total": 45,
|
||||
"sibling_confusions": 1,
|
||||
"wrong_catalog": 1,
|
||||
"blank_ratio": 0.44036697247706424,
|
||||
"rescued_correct": 7,
|
||||
"corrected_wrong": 1,
|
||||
"damaged_correct": 0
|
||||
},
|
||||
"changes": [
|
||||
{
|
||||
"id": 0,
|
||||
"seed": 1,
|
||||
"from": null,
|
||||
"to": "sku_l01"
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"seed": 13,
|
||||
"from": null,
|
||||
"to": "sku_l03"
|
||||
},
|
||||
{
|
||||
"id": 24,
|
||||
"seed": 25,
|
||||
"from": "sku_l03",
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 28,
|
||||
"seed": 25,
|
||||
"from": null,
|
||||
"to": "sku_l07"
|
||||
},
|
||||
{
|
||||
"id": 43,
|
||||
"seed": 42,
|
||||
"from": null,
|
||||
"to": "sku_l09"
|
||||
},
|
||||
{
|
||||
"id": 69,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
},
|
||||
{
|
||||
"id": 73,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
},
|
||||
{
|
||||
"id": 75,
|
||||
"seed": 67,
|
||||
"from": null,
|
||||
"to": "sku_l11"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,494 @@
|
|||
{
|
||||
"every_attempt": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 218,
|
||||
"ocr_nonempty_reads": 168,
|
||||
"worker_ms_p50": 141.37,
|
||||
"worker_ms_p95": 215.5,
|
||||
"first_facing_ms_p50": 1308.53,
|
||||
"all_first_facings_ms": 2043.2,
|
||||
"wall_s": 3.914,
|
||||
"throughput_tasks_s": 55.7,
|
||||
"cpu_s": 3.99,
|
||||
"first_results": {
|
||||
"7": 194.6,
|
||||
"8": 220.4,
|
||||
"6": 237.5,
|
||||
"5": 256.2,
|
||||
"4": 279.1,
|
||||
"3": 391.6,
|
||||
"1": 415.2,
|
||||
"0": 436.6,
|
||||
"9": 457.6,
|
||||
"10": 476.8,
|
||||
"11": 495.3,
|
||||
"12": 511.0,
|
||||
"13": 535.2,
|
||||
"14": 554.5,
|
||||
"16": 569.0,
|
||||
"17": 582.5,
|
||||
"18": 600.6,
|
||||
"19": 617.7,
|
||||
"20": 636.9,
|
||||
"21": 660.5,
|
||||
"22": 684.8,
|
||||
"23": 707.4,
|
||||
"24": 724.2,
|
||||
"25": 737.7,
|
||||
"26": 751.6,
|
||||
"27": 767.3,
|
||||
"28": 780.4,
|
||||
"29": 794.6,
|
||||
"30": 807.7,
|
||||
"31": 820.3,
|
||||
"32": 832.4,
|
||||
"33": 843.9,
|
||||
"34": 874.0,
|
||||
"35": 899.0,
|
||||
"36": 914.2,
|
||||
"37": 937.5,
|
||||
"38": 951.8,
|
||||
"39": 978.6,
|
||||
"40": 1003.4,
|
||||
"41": 1015.1,
|
||||
"42": 1039.1,
|
||||
"43": 1051.8,
|
||||
"44": 1077.6,
|
||||
"45": 1092.6,
|
||||
"46": 1105.3,
|
||||
"47": 1122.7,
|
||||
"48": 1135.4,
|
||||
"49": 1159.6,
|
||||
"50": 1185.3,
|
||||
"51": 1209.7,
|
||||
"52": 1230.3,
|
||||
"53": 1238.0,
|
||||
"54": 1261.3,
|
||||
"55": 1287.3,
|
||||
"56": 1308.5,
|
||||
"57": 1315.7,
|
||||
"58": 1332.5,
|
||||
"59": 1357.7,
|
||||
"60": 1368.5,
|
||||
"61": 1380.6,
|
||||
"62": 1393.2,
|
||||
"63": 1399.6,
|
||||
"64": 1412.3,
|
||||
"65": 1427.3,
|
||||
"66": 1446.3,
|
||||
"67": 1462.6,
|
||||
"68": 1477.2,
|
||||
"69": 1493.9,
|
||||
"70": 1512.2,
|
||||
"71": 1527.5,
|
||||
"72": 1544.5,
|
||||
"73": 1560.8,
|
||||
"74": 1576.5,
|
||||
"75": 1589.9,
|
||||
"76": 1596.6,
|
||||
"77": 1609.5,
|
||||
"78": 1616.1,
|
||||
"79": 1622.7,
|
||||
"80": 1629.0,
|
||||
"81": 1644.2,
|
||||
"82": 1666.8,
|
||||
"83": 1681.6,
|
||||
"84": 1689.9,
|
||||
"85": 1704.2,
|
||||
"86": 1711.2,
|
||||
"87": 1729.0,
|
||||
"88": 1747.7,
|
||||
"89": 1762.4,
|
||||
"90": 1783.6,
|
||||
"91": 1796.0,
|
||||
"92": 1812.0,
|
||||
"93": 1834.9,
|
||||
"94": 1849.3,
|
||||
"95": 1868.6,
|
||||
"96": 1881.7,
|
||||
"97": 1900.9,
|
||||
"98": 1908.4,
|
||||
"99": 1927.5,
|
||||
"100": 1939.5,
|
||||
"101": 1958.5,
|
||||
"102": 1965.1,
|
||||
"103": 1971.9,
|
||||
"104": 1981.4,
|
||||
"105": 1990.0,
|
||||
"106": 1996.7,
|
||||
"107": 2013.7,
|
||||
"108": 2021.4,
|
||||
"109": 2035.5,
|
||||
"110": 2043.2
|
||||
}
|
||||
},
|
||||
"once_per_track": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 109,
|
||||
"ocr_nonempty_reads": 84,
|
||||
"worker_ms_p50": 14.63,
|
||||
"worker_ms_p95": 172.44,
|
||||
"first_facing_ms_p50": 1125.16,
|
||||
"all_first_facings_ms": 1882.7,
|
||||
"wall_s": 1.944,
|
||||
"throughput_tasks_s": 112.1,
|
||||
"cpu_s": 2.77,
|
||||
"first_results": {
|
||||
"6": 31.5,
|
||||
"7": 46.2,
|
||||
"8": 70.2,
|
||||
"4": 90.0,
|
||||
"5": 105.8,
|
||||
"3": 130.5,
|
||||
"0": 156.1,
|
||||
"1": 183.3,
|
||||
"9": 202.8,
|
||||
"10": 224.6,
|
||||
"11": 242.6,
|
||||
"12": 258.2,
|
||||
"13": 289.2,
|
||||
"14": 308.5,
|
||||
"16": 326.8,
|
||||
"17": 347.5,
|
||||
"18": 366.9,
|
||||
"19": 390.3,
|
||||
"20": 408.3,
|
||||
"21": 432.0,
|
||||
"22": 453.7,
|
||||
"23": 475.7,
|
||||
"24": 493.1,
|
||||
"25": 511.8,
|
||||
"26": 526.8,
|
||||
"27": 545.8,
|
||||
"28": 559.3,
|
||||
"29": 579.8,
|
||||
"30": 593.9,
|
||||
"31": 607.2,
|
||||
"32": 622.6,
|
||||
"33": 634.3,
|
||||
"34": 668.3,
|
||||
"35": 694.4,
|
||||
"36": 709.4,
|
||||
"37": 733.6,
|
||||
"38": 757.7,
|
||||
"39": 787.3,
|
||||
"40": 812.3,
|
||||
"41": 819.8,
|
||||
"42": 842.8,
|
||||
"43": 856.8,
|
||||
"44": 882.3,
|
||||
"45": 896.4,
|
||||
"46": 911.6,
|
||||
"47": 929.1,
|
||||
"48": 942.0,
|
||||
"49": 968.0,
|
||||
"50": 996.9,
|
||||
"51": 1021.3,
|
||||
"52": 1042.7,
|
||||
"53": 1051.3,
|
||||
"54": 1075.2,
|
||||
"55": 1100.9,
|
||||
"56": 1125.2,
|
||||
"57": 1132.2,
|
||||
"58": 1151.2,
|
||||
"59": 1171.9,
|
||||
"60": 1184.2,
|
||||
"61": 1197.0,
|
||||
"62": 1209.1,
|
||||
"63": 1215.8,
|
||||
"64": 1228.9,
|
||||
"65": 1241.0,
|
||||
"66": 1259.9,
|
||||
"67": 1275.2,
|
||||
"68": 1289.6,
|
||||
"69": 1303.3,
|
||||
"70": 1318.0,
|
||||
"71": 1331.0,
|
||||
"72": 1345.7,
|
||||
"73": 1361.6,
|
||||
"74": 1374.4,
|
||||
"75": 1390.8,
|
||||
"76": 1397.8,
|
||||
"77": 1412.8,
|
||||
"78": 1419.5,
|
||||
"79": 1426.8,
|
||||
"80": 1432.9,
|
||||
"81": 1445.5,
|
||||
"82": 1464.7,
|
||||
"83": 1481.1,
|
||||
"84": 1491.0,
|
||||
"85": 1505.2,
|
||||
"86": 1518.0,
|
||||
"87": 1532.5,
|
||||
"88": 1557.2,
|
||||
"89": 1570.9,
|
||||
"90": 1593.1,
|
||||
"91": 1608.1,
|
||||
"92": 1625.4,
|
||||
"93": 1644.2,
|
||||
"94": 1659.2,
|
||||
"95": 1681.4,
|
||||
"96": 1696.0,
|
||||
"97": 1723.5,
|
||||
"98": 1731.5,
|
||||
"99": 1748.9,
|
||||
"100": 1756.2,
|
||||
"101": 1775.1,
|
||||
"102": 1784.5,
|
||||
"103": 1791.0,
|
||||
"104": 1798.5,
|
||||
"105": 1809.3,
|
||||
"106": 1817.8,
|
||||
"107": 1836.5,
|
||||
"108": 1850.6,
|
||||
"109": 1868.9,
|
||||
"110": 1882.7
|
||||
}
|
||||
},
|
||||
"ambiguous_track": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 83,
|
||||
"ocr_nonempty_reads": 58,
|
||||
"worker_ms_p50": 10.79,
|
||||
"worker_ms_p95": 137.87,
|
||||
"first_facing_ms_p50": 561.78,
|
||||
"all_first_facings_ms": 1304.6,
|
||||
"wall_s": 1.374,
|
||||
"throughput_tasks_s": 158.7,
|
||||
"cpu_s": 2.33,
|
||||
"first_results": {
|
||||
"1": 13.6,
|
||||
"5": 14.0,
|
||||
"4": 14.4,
|
||||
"3": 16.3,
|
||||
"0": 17.5,
|
||||
"10": 21.8,
|
||||
"11": 23.0,
|
||||
"12": 23.9,
|
||||
"7": 24.9,
|
||||
"13": 29.9,
|
||||
"18": 33.6,
|
||||
"19": 38.3,
|
||||
"8": 49.9,
|
||||
"6": 66.4,
|
||||
"9": 87.5,
|
||||
"16": 111.3,
|
||||
"14": 148.9,
|
||||
"17": 177.4,
|
||||
"20": 214.3,
|
||||
"21": 242.7,
|
||||
"22": 264.0,
|
||||
"23": 286.2,
|
||||
"24": 301.9,
|
||||
"25": 317.2,
|
||||
"26": 335.5,
|
||||
"34": 340.3,
|
||||
"35": 345.9,
|
||||
"27": 352.2,
|
||||
"37": 356.3,
|
||||
"28": 366.6,
|
||||
"39": 370.8,
|
||||
"40": 376.7,
|
||||
"29": 380.8,
|
||||
"42": 385.8,
|
||||
"30": 401.9,
|
||||
"44": 405.8,
|
||||
"45": 408.3,
|
||||
"31": 416.5,
|
||||
"32": 429.2,
|
||||
"33": 444.8,
|
||||
"49": 450.8,
|
||||
"50": 458.1,
|
||||
"36": 458.3,
|
||||
"52": 464.3,
|
||||
"51": 465.5,
|
||||
"54": 471.3,
|
||||
"38": 474.1,
|
||||
"55": 476.9,
|
||||
"41": 480.9,
|
||||
"43": 496.0,
|
||||
"46": 511.4,
|
||||
"47": 528.0,
|
||||
"48": 546.4,
|
||||
"53": 553.0,
|
||||
"57": 561.8,
|
||||
"56": 588.4,
|
||||
"58": 604.8,
|
||||
"59": 626.8,
|
||||
"67": 633.5,
|
||||
"60": 639.6,
|
||||
"61": 651.8,
|
||||
"62": 665.8,
|
||||
"63": 672.7,
|
||||
"64": 685.9,
|
||||
"65": 700.1,
|
||||
"66": 718.8,
|
||||
"68": 740.8,
|
||||
"69": 754.5,
|
||||
"70": 769.0,
|
||||
"71": 784.5,
|
||||
"72": 800.8,
|
||||
"73": 817.3,
|
||||
"74": 830.6,
|
||||
"75": 847.5,
|
||||
"76": 854.4,
|
||||
"77": 866.1,
|
||||
"78": 873.6,
|
||||
"79": 880.1,
|
||||
"80": 886.3,
|
||||
"81": 901.2,
|
||||
"82": 923.4,
|
||||
"83": 944.6,
|
||||
"84": 951.7,
|
||||
"85": 967.1,
|
||||
"86": 974.1,
|
||||
"87": 988.4,
|
||||
"88": 1004.6,
|
||||
"89": 1018.3,
|
||||
"90": 1039.4,
|
||||
"91": 1051.9,
|
||||
"92": 1066.0,
|
||||
"93": 1083.9,
|
||||
"94": 1097.8,
|
||||
"95": 1120.2,
|
||||
"96": 1133.7,
|
||||
"97": 1153.7,
|
||||
"98": 1161.3,
|
||||
"99": 1181.4,
|
||||
"100": 1188.4,
|
||||
"101": 1207.5,
|
||||
"102": 1213.8,
|
||||
"103": 1220.2,
|
||||
"104": 1227.9,
|
||||
"105": 1239.6,
|
||||
"106": 1246.4,
|
||||
"107": 1269.9,
|
||||
"108": 1277.9,
|
||||
"109": 1294.4,
|
||||
"110": 1304.6
|
||||
}
|
||||
},
|
||||
"group_representative": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 49,
|
||||
"ocr_nonempty_reads": 29,
|
||||
"worker_ms_p50": 8.78,
|
||||
"worker_ms_p95": 121.51,
|
||||
"first_facing_ms_p50": 298.94,
|
||||
"all_first_facings_ms": 725.0,
|
||||
"wall_s": 0.796,
|
||||
"throughput_tasks_s": 274.0,
|
||||
"cpu_s": 1.97,
|
||||
"first_results": {
|
||||
"7": 9.8,
|
||||
"6": 11.1,
|
||||
"8": 11.5,
|
||||
"4": 13.6,
|
||||
"5": 14.0,
|
||||
"1": 15.6,
|
||||
"0": 15.7,
|
||||
"3": 19.1,
|
||||
"10": 19.8,
|
||||
"12": 22.9,
|
||||
"13": 25.8,
|
||||
"11": 27.2,
|
||||
"18": 27.7,
|
||||
"19": 32.8,
|
||||
"20": 35.9,
|
||||
"22": 36.7,
|
||||
"21": 38.9,
|
||||
"16": 41.7,
|
||||
"24": 41.9,
|
||||
"26": 45.3,
|
||||
"28": 46.6,
|
||||
"27": 47.4,
|
||||
"29": 52.1,
|
||||
"9": 62.9,
|
||||
"33": 65.2,
|
||||
"34": 70.1,
|
||||
"14": 82.6,
|
||||
"17": 101.3,
|
||||
"37": 105.7,
|
||||
"38": 110.3,
|
||||
"23": 123.4,
|
||||
"40": 129.6,
|
||||
"25": 137.9,
|
||||
"42": 142.9,
|
||||
"43": 145.1,
|
||||
"44": 149.0,
|
||||
"30": 150.7,
|
||||
"46": 154.0,
|
||||
"31": 165.4,
|
||||
"48": 168.0,
|
||||
"49": 173.9,
|
||||
"32": 177.5,
|
||||
"50": 181.2,
|
||||
"51": 184.5,
|
||||
"52": 187.0,
|
||||
"54": 192.8,
|
||||
"55": 198.6,
|
||||
"35": 203.8,
|
||||
"36": 219.0,
|
||||
"58": 223.0,
|
||||
"39": 249.5,
|
||||
"41": 257.3,
|
||||
"45": 272.1,
|
||||
"47": 288.4,
|
||||
"53": 298.9,
|
||||
"56": 320.5,
|
||||
"57": 329.8,
|
||||
"66": 333.4,
|
||||
"67": 340.1,
|
||||
"68": 347.3,
|
||||
"59": 351.1,
|
||||
"69": 354.3,
|
||||
"71": 361.8,
|
||||
"60": 363.9,
|
||||
"72": 369.3,
|
||||
"73": 369.8,
|
||||
"74": 375.5,
|
||||
"61": 376.7,
|
||||
"75": 377.2,
|
||||
"76": 378.6,
|
||||
"62": 390.5,
|
||||
"63": 398.9,
|
||||
"64": 412.4,
|
||||
"82": 418.5,
|
||||
"83": 423.8,
|
||||
"65": 425.6,
|
||||
"70": 442.3,
|
||||
"86": 448.7,
|
||||
"78": 449.0,
|
||||
"87": 456.9,
|
||||
"89": 460.8,
|
||||
"77": 464.7,
|
||||
"79": 471.6,
|
||||
"92": 477.9,
|
||||
"80": 478.0,
|
||||
"94": 486.2,
|
||||
"95": 492.7,
|
||||
"81": 498.7,
|
||||
"84": 505.9,
|
||||
"97": 506.7,
|
||||
"85": 529.1,
|
||||
"100": 532.3,
|
||||
"101": 535.6,
|
||||
"88": 545.4,
|
||||
"90": 566.3,
|
||||
"91": 578.7,
|
||||
"93": 595.6,
|
||||
"96": 609.6,
|
||||
"99": 628.2,
|
||||
"98": 636.2,
|
||||
"102": 642.4,
|
||||
"103": 648.8,
|
||||
"104": 656.1,
|
||||
"105": 664.6,
|
||||
"106": 671.3,
|
||||
"107": 688.8,
|
||||
"108": 698.2,
|
||||
"109": 713.8,
|
||||
"110": 725.0
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,494 @@
|
|||
{
|
||||
"every_attempt": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 218,
|
||||
"ocr_nonempty_reads": 160,
|
||||
"worker_ms_p50": 147.43,
|
||||
"worker_ms_p95": 282.7,
|
||||
"first_facing_ms_p50": 1177.31,
|
||||
"all_first_facings_ms": 1889.7,
|
||||
"wall_s": 4.324,
|
||||
"throughput_tasks_s": 50.4,
|
||||
"cpu_s": 4.48,
|
||||
"first_results": {
|
||||
"6": 44.7,
|
||||
"7": 65.6,
|
||||
"4": 89.7,
|
||||
"8": 119.6,
|
||||
"5": 145.0,
|
||||
"3": 174.0,
|
||||
"1": 197.3,
|
||||
"0": 219.2,
|
||||
"9": 236.5,
|
||||
"10": 259.0,
|
||||
"11": 280.2,
|
||||
"12": 297.7,
|
||||
"13": 327.1,
|
||||
"14": 351.6,
|
||||
"16": 367.7,
|
||||
"17": 384.4,
|
||||
"18": 405.4,
|
||||
"19": 424.3,
|
||||
"20": 448.8,
|
||||
"21": 472.6,
|
||||
"22": 497.3,
|
||||
"23": 521.5,
|
||||
"24": 536.7,
|
||||
"25": 551.8,
|
||||
"26": 569.8,
|
||||
"27": 586.7,
|
||||
"28": 602.2,
|
||||
"29": 618.7,
|
||||
"30": 634.1,
|
||||
"31": 652.5,
|
||||
"32": 666.7,
|
||||
"33": 678.2,
|
||||
"34": 708.7,
|
||||
"35": 738.1,
|
||||
"36": 753.3,
|
||||
"37": 778.6,
|
||||
"38": 794.8,
|
||||
"39": 823.3,
|
||||
"40": 852.2,
|
||||
"41": 860.2,
|
||||
"42": 884.5,
|
||||
"43": 893.5,
|
||||
"44": 919.3,
|
||||
"45": 935.8,
|
||||
"46": 949.9,
|
||||
"47": 968.2,
|
||||
"48": 983.1,
|
||||
"49": 1012.0,
|
||||
"50": 1043.9,
|
||||
"51": 1074.2,
|
||||
"52": 1096.8,
|
||||
"53": 1104.5,
|
||||
"54": 1129.3,
|
||||
"55": 1155.6,
|
||||
"56": 1177.3,
|
||||
"57": 1184.9,
|
||||
"58": 1205.2,
|
||||
"59": 1226.4,
|
||||
"60": 1238.1,
|
||||
"61": 1249.9,
|
||||
"62": 1273.9,
|
||||
"63": 1280.6,
|
||||
"64": 1298.0,
|
||||
"65": 1310.9,
|
||||
"66": 1328.2,
|
||||
"67": 1343.1,
|
||||
"68": 1356.6,
|
||||
"69": 1371.1,
|
||||
"70": 1385.1,
|
||||
"71": 1400.9,
|
||||
"72": 1417.1,
|
||||
"73": 1433.7,
|
||||
"74": 1449.1,
|
||||
"75": 1463.3,
|
||||
"76": 1470.1,
|
||||
"77": 1482.5,
|
||||
"78": 1489.1,
|
||||
"79": 1501.8,
|
||||
"80": 1509.1,
|
||||
"81": 1521.6,
|
||||
"82": 1538.1,
|
||||
"83": 1552.3,
|
||||
"84": 1562.3,
|
||||
"85": 1576.4,
|
||||
"86": 1583.4,
|
||||
"87": 1596.4,
|
||||
"88": 1615.8,
|
||||
"89": 1628.9,
|
||||
"90": 1654.3,
|
||||
"91": 1670.0,
|
||||
"92": 1684.9,
|
||||
"93": 1702.5,
|
||||
"94": 1718.3,
|
||||
"95": 1738.7,
|
||||
"96": 1751.8,
|
||||
"97": 1768.2,
|
||||
"98": 1777.9,
|
||||
"99": 1793.7,
|
||||
"100": 1801.3,
|
||||
"101": 1809.8,
|
||||
"102": 1816.0,
|
||||
"103": 1822.3,
|
||||
"104": 1829.8,
|
||||
"105": 1836.2,
|
||||
"106": 1845.0,
|
||||
"107": 1863.7,
|
||||
"108": 1870.8,
|
||||
"109": 1882.5,
|
||||
"110": 1889.7
|
||||
}
|
||||
},
|
||||
"once_per_track": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 109,
|
||||
"ocr_nonempty_reads": 80,
|
||||
"worker_ms_p50": 19.6,
|
||||
"worker_ms_p95": 301.62,
|
||||
"first_facing_ms_p50": 1234.75,
|
||||
"all_first_facings_ms": 2679.9,
|
||||
"wall_s": 2.773,
|
||||
"throughput_tasks_s": 78.6,
|
||||
"cpu_s": 3.42,
|
||||
"first_results": {
|
||||
"6": 63.4,
|
||||
"4": 91.5,
|
||||
"8": 117.0,
|
||||
"7": 139.9,
|
||||
"0": 174.8,
|
||||
"5": 202.5,
|
||||
"3": 234.2,
|
||||
"1": 260.5,
|
||||
"9": 277.4,
|
||||
"10": 299.3,
|
||||
"11": 321.6,
|
||||
"12": 340.8,
|
||||
"13": 368.7,
|
||||
"14": 393.0,
|
||||
"16": 409.4,
|
||||
"17": 426.1,
|
||||
"18": 446.5,
|
||||
"19": 468.5,
|
||||
"20": 489.1,
|
||||
"21": 512.9,
|
||||
"22": 542.5,
|
||||
"23": 567.6,
|
||||
"24": 582.5,
|
||||
"25": 595.9,
|
||||
"26": 614.7,
|
||||
"27": 631.2,
|
||||
"28": 650.5,
|
||||
"29": 665.8,
|
||||
"30": 684.6,
|
||||
"31": 699.9,
|
||||
"32": 717.4,
|
||||
"33": 724.7,
|
||||
"34": 759.0,
|
||||
"35": 784.1,
|
||||
"36": 798.5,
|
||||
"37": 825.5,
|
||||
"38": 842.1,
|
||||
"39": 870.7,
|
||||
"40": 896.2,
|
||||
"41": 906.8,
|
||||
"42": 930.2,
|
||||
"43": 940.8,
|
||||
"44": 968.3,
|
||||
"45": 985.0,
|
||||
"46": 999.3,
|
||||
"47": 1018.4,
|
||||
"48": 1030.8,
|
||||
"49": 1058.0,
|
||||
"50": 1088.6,
|
||||
"51": 1114.0,
|
||||
"52": 1136.3,
|
||||
"53": 1143.4,
|
||||
"54": 1170.1,
|
||||
"55": 1199.9,
|
||||
"56": 1234.7,
|
||||
"57": 1255.6,
|
||||
"58": 1502.1,
|
||||
"59": 1617.1,
|
||||
"60": 1668.1,
|
||||
"61": 1712.2,
|
||||
"62": 1745.8,
|
||||
"63": 1756.5,
|
||||
"64": 1801.4,
|
||||
"65": 1855.2,
|
||||
"66": 1892.8,
|
||||
"67": 1934.4,
|
||||
"68": 1970.9,
|
||||
"69": 2010.9,
|
||||
"70": 2044.4,
|
||||
"71": 2074.5,
|
||||
"72": 2097.3,
|
||||
"73": 2113.8,
|
||||
"74": 2130.2,
|
||||
"75": 2148.2,
|
||||
"76": 2158.0,
|
||||
"77": 2179.3,
|
||||
"78": 2188.0,
|
||||
"79": 2196.1,
|
||||
"80": 2204.6,
|
||||
"81": 2221.9,
|
||||
"82": 2242.2,
|
||||
"83": 2264.6,
|
||||
"84": 2273.1,
|
||||
"85": 2290.5,
|
||||
"86": 2303.0,
|
||||
"87": 2322.9,
|
||||
"88": 2351.0,
|
||||
"89": 2372.1,
|
||||
"90": 2397.2,
|
||||
"91": 2413.8,
|
||||
"92": 2429.7,
|
||||
"93": 2448.5,
|
||||
"94": 2463.6,
|
||||
"95": 2484.2,
|
||||
"96": 2504.6,
|
||||
"97": 2522.3,
|
||||
"98": 2535.9,
|
||||
"99": 2554.4,
|
||||
"100": 2561.2,
|
||||
"101": 2571.2,
|
||||
"102": 2578.0,
|
||||
"103": 2585.3,
|
||||
"104": 2592.6,
|
||||
"105": 2600.6,
|
||||
"106": 2612.3,
|
||||
"107": 2631.5,
|
||||
"108": 2646.0,
|
||||
"109": 2665.4,
|
||||
"110": 2679.9
|
||||
}
|
||||
},
|
||||
"ambiguous_track": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 83,
|
||||
"ocr_nonempty_reads": 54,
|
||||
"worker_ms_p50": 15.74,
|
||||
"worker_ms_p95": 221.05,
|
||||
"first_facing_ms_p50": 806.24,
|
||||
"all_first_facings_ms": 1950.5,
|
||||
"wall_s": 2.049,
|
||||
"throughput_tasks_s": 106.4,
|
||||
"cpu_s": 2.97,
|
||||
"first_results": {
|
||||
"4": 16.7,
|
||||
"3": 20.0,
|
||||
"5": 21.8,
|
||||
"0": 21.9,
|
||||
"1": 22.7,
|
||||
"10": 29.5,
|
||||
"11": 31.8,
|
||||
"12": 35.4,
|
||||
"8": 37.5,
|
||||
"13": 38.9,
|
||||
"19": 46.9,
|
||||
"18": 47.8,
|
||||
"7": 55.9,
|
||||
"6": 77.3,
|
||||
"9": 98.5,
|
||||
"14": 123.9,
|
||||
"16": 141.2,
|
||||
"17": 166.9,
|
||||
"20": 192.6,
|
||||
"21": 260.9,
|
||||
"22": 304.0,
|
||||
"23": 364.4,
|
||||
"24": 402.3,
|
||||
"25": 425.3,
|
||||
"26": 452.3,
|
||||
"34": 458.9,
|
||||
"35": 466.8,
|
||||
"27": 480.2,
|
||||
"37": 485.2,
|
||||
"28": 502.8,
|
||||
"39": 508.9,
|
||||
"40": 517.7,
|
||||
"29": 524.9,
|
||||
"42": 535.8,
|
||||
"30": 567.2,
|
||||
"44": 572.2,
|
||||
"45": 576.0,
|
||||
"31": 589.8,
|
||||
"32": 606.1,
|
||||
"33": 615.6,
|
||||
"49": 623.2,
|
||||
"50": 633.6,
|
||||
"36": 640.8,
|
||||
"51": 641.8,
|
||||
"52": 650.0,
|
||||
"54": 659.0,
|
||||
"55": 668.0,
|
||||
"38": 668.5,
|
||||
"41": 682.9,
|
||||
"43": 696.6,
|
||||
"46": 714.5,
|
||||
"47": 747.1,
|
||||
"48": 783.4,
|
||||
"53": 793.3,
|
||||
"57": 806.2,
|
||||
"56": 833.2,
|
||||
"58": 862.9,
|
||||
"59": 910.5,
|
||||
"67": 919.9,
|
||||
"60": 934.5,
|
||||
"61": 955.9,
|
||||
"62": 973.7,
|
||||
"63": 989.2,
|
||||
"64": 1008.9,
|
||||
"65": 1041.5,
|
||||
"66": 1068.5,
|
||||
"68": 1084.0,
|
||||
"69": 1107.3,
|
||||
"70": 1122.4,
|
||||
"71": 1156.8,
|
||||
"72": 1186.4,
|
||||
"73": 1208.1,
|
||||
"74": 1234.2,
|
||||
"75": 1258.4,
|
||||
"76": 1272.2,
|
||||
"77": 1288.4,
|
||||
"78": 1299.1,
|
||||
"79": 1308.8,
|
||||
"80": 1321.1,
|
||||
"81": 1339.1,
|
||||
"82": 1361.2,
|
||||
"83": 1385.7,
|
||||
"84": 1404.2,
|
||||
"85": 1424.5,
|
||||
"86": 1437.3,
|
||||
"87": 1460.1,
|
||||
"88": 1488.4,
|
||||
"89": 1529.9,
|
||||
"90": 1560.0,
|
||||
"91": 1582.8,
|
||||
"92": 1608.1,
|
||||
"93": 1637.4,
|
||||
"94": 1658.4,
|
||||
"95": 1687.7,
|
||||
"96": 1707.7,
|
||||
"97": 1734.3,
|
||||
"98": 1749.8,
|
||||
"99": 1776.6,
|
||||
"100": 1784.3,
|
||||
"101": 1797.0,
|
||||
"102": 1805.8,
|
||||
"103": 1814.2,
|
||||
"104": 1829.9,
|
||||
"105": 1839.1,
|
||||
"106": 1851.1,
|
||||
"107": 1894.2,
|
||||
"108": 1907.3,
|
||||
"109": 1936.3,
|
||||
"110": 1950.5
|
||||
}
|
||||
},
|
||||
"group_representative": {
|
||||
"tasks": 218,
|
||||
"ocr_invocations": 49,
|
||||
"ocr_nonempty_reads": 30,
|
||||
"worker_ms_p50": 16.33,
|
||||
"worker_ms_p95": 191.82,
|
||||
"first_facing_ms_p50": 453.78,
|
||||
"all_first_facings_ms": 1167.8,
|
||||
"wall_s": 1.304,
|
||||
"throughput_tasks_s": 167.2,
|
||||
"cpu_s": 2.64,
|
||||
"first_results": {
|
||||
"6": 17.0,
|
||||
"7": 23.2,
|
||||
"5": 30.7,
|
||||
"0": 30.7,
|
||||
"4": 30.7,
|
||||
"1": 32.1,
|
||||
"8": 34.5,
|
||||
"3": 36.6,
|
||||
"10": 49.2,
|
||||
"11": 49.3,
|
||||
"12": 51.7,
|
||||
"13": 55.4,
|
||||
"19": 64.3,
|
||||
"18": 65.1,
|
||||
"20": 67.3,
|
||||
"21": 72.8,
|
||||
"22": 78.6,
|
||||
"24": 78.6,
|
||||
"26": 86.9,
|
||||
"27": 87.8,
|
||||
"9": 88.3,
|
||||
"28": 97.0,
|
||||
"29": 97.5,
|
||||
"16": 115.9,
|
||||
"33": 119.3,
|
||||
"34": 126.4,
|
||||
"14": 144.9,
|
||||
"17": 165.6,
|
||||
"37": 170.4,
|
||||
"38": 175.4,
|
||||
"25": 189.7,
|
||||
"40": 197.0,
|
||||
"23": 226.5,
|
||||
"42": 233.3,
|
||||
"43": 236.3,
|
||||
"44": 241.0,
|
||||
"30": 249.1,
|
||||
"46": 253.4,
|
||||
"32": 268.9,
|
||||
"48": 272.6,
|
||||
"49": 280.5,
|
||||
"31": 288.4,
|
||||
"50": 291.4,
|
||||
"51": 298.3,
|
||||
"52": 299.4,
|
||||
"54": 307.4,
|
||||
"55": 315.4,
|
||||
"35": 331.9,
|
||||
"36": 356.8,
|
||||
"58": 361.8,
|
||||
"39": 392.1,
|
||||
"41": 400.7,
|
||||
"45": 425.5,
|
||||
"47": 445.3,
|
||||
"53": 453.8,
|
||||
"56": 479.1,
|
||||
"57": 488.3,
|
||||
"66": 492.4,
|
||||
"67": 503.4,
|
||||
"68": 513.0,
|
||||
"59": 516.8,
|
||||
"69": 520.3,
|
||||
"71": 529.4,
|
||||
"60": 536.3,
|
||||
"72": 540.8,
|
||||
"73": 548.2,
|
||||
"74": 551.4,
|
||||
"61": 556.1,
|
||||
"76": 556.3,
|
||||
"75": 558.2,
|
||||
"62": 584.5,
|
||||
"63": 603.7,
|
||||
"64": 623.1,
|
||||
"82": 631.7,
|
||||
"83": 638.2,
|
||||
"65": 655.7,
|
||||
"70": 680.7,
|
||||
"86": 688.8,
|
||||
"78": 696.6,
|
||||
"87": 698.6,
|
||||
"89": 703.5,
|
||||
"77": 713.3,
|
||||
"79": 728.8,
|
||||
"92": 736.0,
|
||||
"80": 738.0,
|
||||
"94": 747.9,
|
||||
"95": 756.1,
|
||||
"81": 806.6,
|
||||
"97": 820.1,
|
||||
"84": 821.3,
|
||||
"85": 853.6,
|
||||
"100": 857.2,
|
||||
"101": 862.0,
|
||||
"88": 875.3,
|
||||
"90": 904.9,
|
||||
"91": 921.7,
|
||||
"93": 948.7,
|
||||
"96": 971.3,
|
||||
"99": 990.8,
|
||||
"98": 1003.9,
|
||||
"102": 1012.4,
|
||||
"103": 1021.1,
|
||||
"104": 1039.1,
|
||||
"105": 1049.6,
|
||||
"106": 1063.8,
|
||||
"107": 1098.9,
|
||||
"108": 1115.3,
|
||||
"109": 1142.0,
|
||||
"110": 1167.8
|
||||
}
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -0,0 +1,250 @@
|
|||
"""The four named policies on the far-unknown crops, priced exactly.
|
||||
|
||||
`far-unknown-lines.json` sweeps the two open-set gates as a grid - LOCAL_ACCEPT x
|
||||
TEXT_MIN_SHARED, one shipped `DecisionPolicy` per cell, no fallback. That table
|
||||
says where the gates cross; it cannot say what a *named* policy costs on a shelf
|
||||
holding nothing in the catalog. Near-argmax is not a threshold at all - it keeps a
|
||||
geometric candidate the gate refused, behind two refusals of its own - and no cell
|
||||
in that grid expresses it.
|
||||
|
||||
So the four policies go through `analyze.decide`, the same call the frozen
|
||||
observation table uses, over the same cached crops `sweep_accept_lines.py` built.
|
||||
`gate_accepts` is the shipped `DecisionPolicy` on identical evidence, so the gap
|
||||
between the two counts per policy is exactly what the fallback adds. Every accept
|
||||
on an unrelated Wikimedia shelf is a false accept by construction.
|
||||
|
||||
Run from the recognition root:
|
||||
|
||||
.venv/bin/python docs/research/2026-09-29-shelf-intelligence/far_unknown.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import json
|
||||
import pickle
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
DIR = Path(__file__).resolve().parent
|
||||
ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
sys.path.insert(0, str(ROOT / "code"))
|
||||
sys.path.insert(0, str(DIR))
|
||||
|
||||
import analyze # noqa: E402
|
||||
from eval_identity import _catalog_rows, catalog_hash, code_revision # noqa: E402
|
||||
from identity_lanes import DecisionPolicy, TextLane, _ranked # noqa: E402
|
||||
from sweep_accept_lines import CACHE_VERSION # noqa: E402
|
||||
|
||||
CACHE = "data/evaluations/2026-09-29-session/scratch/accept_line_cache.pkl"
|
||||
LINES = "evidence/far-unknown-lines.json"
|
||||
OBSERVATIONS = "evidence/observations.json"
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--stock", default="stock-lopia-clean")
|
||||
parser.add_argument("--cache", default=CACHE)
|
||||
parser.add_argument("--out", default=str(DIR / "evidence/far-unknown-policies.json"))
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
# Hash the exact bytes that get unpickled, so the recorded digest and the
|
||||
# measured crops cannot come from two different reads of a rewritten file.
|
||||
blob = Path(args.cache).read_bytes()
|
||||
cache = pickle.loads(blob)
|
||||
if cache.get("version") != CACHE_VERSION:
|
||||
raise SystemExit(
|
||||
f"cache version {cache.get('version')!r} != sweep's {CACHE_VERSION}; "
|
||||
"rebuild it with code/sweep_accept_lines.py --rebuild"
|
||||
)
|
||||
|
||||
# No OCR engine: the reads are already in the cache, so `read()` is never
|
||||
# called and building a recognizer would only add a dependency to fail.
|
||||
lane = TextLane.from_catalog(_catalog_rows(args.stock), args.stock)
|
||||
signatures = lane.signatures
|
||||
|
||||
distractors = cache["distractors"]
|
||||
observations = []
|
||||
for ranked, read in distractors:
|
||||
text = lane.scores_from_text(read)
|
||||
# The cached rank is passed through untouched, and the dict scores are
|
||||
# ordered exactly as the grid sweep ordered them, so the four policies
|
||||
# here and the four columns there read identical evidence.
|
||||
assert _ranked(text) == sorted(text.items(), key=lambda kv: kv[1], reverse=True)
|
||||
observations.append({"rank": ranked, "text": text, "read": read})
|
||||
assert len(observations) == len(distractors)
|
||||
|
||||
results = {}
|
||||
for policy, threshold in analyze.POLICIES.items():
|
||||
gate = DecisionPolicy(local_accept=threshold, signatures=signatures)
|
||||
accepts = fallback_only = with_rank = 0
|
||||
by_source = {"local": 0, "text": 0}
|
||||
for o in observations:
|
||||
if o["rank"]:
|
||||
with_rank += 1
|
||||
decision = gate.decide(o["rank"], o["text"], text_read=o["read"])
|
||||
answer = analyze.decide(o, policy, signatures)
|
||||
if not answer:
|
||||
continue
|
||||
accepts += 1
|
||||
if decision["best_sku"]:
|
||||
# The fallback only runs when the gate refused, so a gated
|
||||
# answer must survive `analyze.decide` unchanged.
|
||||
assert answer == decision["best_sku"]
|
||||
by_source[decision["source"]] += 1
|
||||
else:
|
||||
fallback_only += 1
|
||||
# Accounting: every crop decided once, accepts split exhaustively, and
|
||||
# the fallback belongs to near-argmax alone. A weaker floor accepts
|
||||
# weakly more, so the counts rise as the threshold falls.
|
||||
assert accepts - fallback_only >= 0
|
||||
assert by_source["local"] + by_source["text"] + fallback_only == accepts
|
||||
assert fallback_only == 0 or policy == "near-argmax"
|
||||
results[policy] = {
|
||||
"local_accept": threshold,
|
||||
"fallback": policy == "near-argmax",
|
||||
"accepts": accepts,
|
||||
"gate_accepts": accepts - fallback_only,
|
||||
"fallback_accepts": fallback_only,
|
||||
"by_source": by_source,
|
||||
"rate": accepts / len(observations),
|
||||
"rate_percent": round(accepts / len(observations) * 100, 4),
|
||||
"crops_with_a_local_candidate": with_rank,
|
||||
}
|
||||
|
||||
ordered = [results[name]["accepts"] for name in analyze.POLICIES]
|
||||
assert ordered == sorted(ordered), f"accepts not monotone in the accept floor: {ordered}"
|
||||
|
||||
crops = len(observations)
|
||||
report = {
|
||||
"question": (
|
||||
"False accepts per named analyze policy on unrelated shelf crops, "
|
||||
"analyze.decide on the sweep_accept_lines cache."
|
||||
),
|
||||
"method": {
|
||||
"decision": "analyze.decide(o, policy, signatures) - the frozen analysis's own call",
|
||||
"policies": {name: analyze.POLICIES[name] for name in analyze.POLICIES},
|
||||
"gate_for_comparison": "identity_lanes.DecisionPolicy(local_accept=threshold, signatures)",
|
||||
"text_scores": "TextLane.scores_from_text on the cached OCR read",
|
||||
"fallback": (
|
||||
"analyze.decide's near-argmax extra fallback only: keep rank[0] when the gate "
|
||||
"refused, unless the read contradicts that row's size or the text prefers another row"
|
||||
),
|
||||
},
|
||||
"provenance": {
|
||||
"cache": {
|
||||
"path": args.cache,
|
||||
"sha256": hashlib.sha256(blob).hexdigest(),
|
||||
"bytes": len(blob),
|
||||
"version": cache["version"],
|
||||
"detect_conf": cache["conf"],
|
||||
"records": ["version", "conf", "photos", "shelf_full", "shelf_live", "distractors"],
|
||||
"not_recorded": [
|
||||
"stock_dir",
|
||||
"catalog_hash",
|
||||
"code_revision",
|
||||
"source image paths or hashes",
|
||||
"per-photo crop counts",
|
||||
],
|
||||
"note": (
|
||||
"The cache attests no catalog, so the catalog match below is read off the "
|
||||
"current worktree, not out of the cache. The read is unpickled, not rebuilt."
|
||||
),
|
||||
},
|
||||
"catalog": {
|
||||
"dir": args.stock,
|
||||
"catalog_hash": catalog_hash(args.stock),
|
||||
},
|
||||
"code_revision": code_revision(),
|
||||
"signatures": {
|
||||
"rows": len(signatures),
|
||||
"sha256": hashlib.sha256(
|
||||
json.dumps(signatures, sort_keys=True, ensure_ascii=False).encode()
|
||||
).hexdigest(),
|
||||
},
|
||||
},
|
||||
"counts": {
|
||||
"photos": cache["photos"],
|
||||
"crops": crops,
|
||||
"crops_with_a_local_candidate": sum(1 for o in observations if o["rank"]),
|
||||
"crops_with_no_local_candidate": sum(1 for o in observations if not o["rank"]),
|
||||
"crops_with_a_raw_read": sum(1 for o in observations if o["read"]),
|
||||
# A read can be non-empty and still score nothing: `scores_from_text`
|
||||
# returns {} under four normalized characters.
|
||||
"crops_scored_by_text": sum(1 for o in observations if o["text"]),
|
||||
},
|
||||
"policies": results,
|
||||
}
|
||||
|
||||
# Context, not an accounting input: the older grid table's same-threshold
|
||||
# cells, whose producer is not in the tree, so its settings stay unverified.
|
||||
lines_path = DIR / LINES
|
||||
lines = json.loads(lines_path.read_text()) if lines_path.is_file() else {}
|
||||
report["cross_check_far_unknown_lines"] = {
|
||||
"source": LINES,
|
||||
"note": "accept-threshold cells only, no fallback; producer script not in the tree",
|
||||
"cells": {
|
||||
name: {
|
||||
"lines_accepts": line["accepts"],
|
||||
"lines_crops": line["crops"],
|
||||
"here_gate_accepts": results[name]["gate_accepts"],
|
||||
"agrees": line["accepts"] == results[name]["gate_accepts"]
|
||||
and line["crops"] == crops,
|
||||
}
|
||||
for name, row in results.items()
|
||||
if (line := lines.get(str(row["local_accept"])))
|
||||
},
|
||||
}
|
||||
|
||||
# What the cache cannot attest, read off the frozen table instead: the lane
|
||||
# built here carries the same 12 names the observation table was scored on.
|
||||
frozen = json.loads((DIR / OBSERVATIONS).read_text())
|
||||
report["provenance"]["catalog"].update({
|
||||
"matches_observations": frozen["signatures"] == signatures,
|
||||
"observations_catalog_hash": frozen["provenance"]["catalog"],
|
||||
"observations_code_revision": frozen["provenance"]["code"],
|
||||
})
|
||||
report["limitations"] = [
|
||||
"The cache records no catalog, code revision or image list, so 'same catalog as the "
|
||||
"frozen observations' is read off the current worktree, not attested by the cache.",
|
||||
f"Only {report['counts']['crops_with_a_local_candidate']} of {crops} crops produced a local "
|
||||
f"candidate; the other {report['counts']['crops_with_no_local_candidate']} are decided by the "
|
||||
"text lane alone or refused before any gate applies, so the accept floor is exercised on a "
|
||||
"small minority of the rows and the rates are dominated by empty evidence.",
|
||||
"The catalog is 12 rows from one store. A false-accept rate is a function of catalog size, so "
|
||||
"these numbers do not transfer to a larger or differently-stocked catalog.",
|
||||
"Crops come from one detection pass at detect_conf "
|
||||
f"{cache['conf']} on 1280-long-side frames, so the crop population is set by the detector, "
|
||||
"not only by the identity gate.",
|
||||
"The cache has no per-photo breakdown, so no photo-level rate and no check that one image is "
|
||||
"not carrying the whole count.",
|
||||
"The near-argmax fallback is a research construct in analyze.py, not shipped code; its row "
|
||||
"prices a policy that does not exist on the server yet.",
|
||||
"far-unknown-lines.json's producer is not in the tree, so agreement with it is corroboration, "
|
||||
"not verification.",
|
||||
]
|
||||
|
||||
Path(args.out).write_text(json.dumps(report, indent=1, ensure_ascii=False) + "\n")
|
||||
|
||||
print(f"cache {args.cache} sha256 {report['provenance']['cache']['sha256'][:16]}… "
|
||||
f"v{cache['version']} conf {cache['conf']}")
|
||||
print(f"catalog {args.stock}: {len(signatures)} rows, hash "
|
||||
f"{report['provenance']['catalog']['catalog_hash'][:16]}…, matches observations "
|
||||
f"{report['provenance']['catalog']['matches_observations']}")
|
||||
print(f"{cache['photos']} unrelated shelf photos, {crops} crops, "
|
||||
f"{report['counts']['crops_with_a_local_candidate']} with a local candidate")
|
||||
print()
|
||||
print(f"{'policy':>15} {'accept':>7} {'fallb':>6} {'accepts':>8} {'gate':>6} "
|
||||
f"{'local':>6} {'text':>6} {'rate':>8}")
|
||||
for name, row in results.items():
|
||||
print(f"{name:>15} {row['local_accept']:>7.1f} {str(row['fallback']):>6} "
|
||||
f"{row['accepts']:>8} {row['gate_accepts']:>6} {row['by_source']['local']:>6} "
|
||||
f"{row['by_source']['text']:>6} {row['rate_percent']:>7.4f}%")
|
||||
print()
|
||||
print(f"wrote {args.out}")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
{
|
||||
"source": "Main visual inspection of add/image6.jpg and evidence/facings.jpg, 2026-09-29; existing catalog labels unchanged",
|
||||
"status": "Single annotator diagnostic repeated-artwork labels; not independently verified JAN ground truth",
|
||||
"rule": "Unknowns receive a distinct visual product key only when repeat artwork is visible. Null omitted IDs are not scored for grouping. No unknown equality shortcut.",
|
||||
"families": {"sku_l01":"kewpie-mayo","sku_l02":"kewpie-mayo","sku_l03":"kewpie-mayo","sku_l04":"yamaya-mayo","sku_l05":"kewpie-mayo","sku_l06":"kewpie-mayo","sku_l07":"lucky-mayo","sku_l08":"peri-mix","sku_l09":"pure-select","sku_l10":"pure-select","sku_l11":"kewpie-mayo","sku_l12":"yuras-sauce"},
|
||||
"unknown_visual_groups": [
|
||||
{"key":"demiglace-pouch","family":"demiglace","members":[31,32]},
|
||||
{"key":"heinz-ketchup","family":"heinz","members":[65,66]},
|
||||
{"key":"black-label-dressing","family":"dressing","members":[76,77,78,79]},
|
||||
{"key":"kewpie-450g","family":"kewpie-mayo","members":[94,95,96,97,81,82,83,84,85,86,87,88]},
|
||||
{"key":"sesame-dressing","family":"dressing","members":[89,90,91]},
|
||||
{"key":"bottom-dark-dressing","family":"dressing","members":[99,100,101]}
|
||||
],
|
||||
"ambiguous_unknown_ids":[16,60,61,62,63,64,80,92,93,98,102,103,104,105,106,107,108,109,110],
|
||||
"notes":["104 is an overlapping aggregate crop in legacy labels; do not use it for pair calibration", "Bottom partial Kewpie facings may differ by size or overlap; no repeated-SKU truth assigned", "Family definitions are evaluation-only and never supplied to grouping inference"]
|
||||
}
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
"""E: same cached crop evidence, alter only which real OCR reads are available."""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import copy
|
||||
import statistics
|
||||
from analyze import decide,propagate,metrics,infer_groups
|
||||
|
||||
DIR=Path(__file__).resolve().parent
|
||||
OUT=DIR/'evidence'
|
||||
|
||||
|
||||
def main():
|
||||
data=json.loads((OUT/'observations.json').read_text()); analysis=json.loads((OUT/'analysis.json').read_text())
|
||||
families=json.loads((DIR/'group-truth.json').read_text())['families']
|
||||
report={'provenance':data['provenance'],'arms':{}}
|
||||
for res,d in data['resolutions'].items():
|
||||
obs=d['observations']; sig=data['signatures']; groups=analysis['resolutions'][res]['appearance_groups']
|
||||
lookup={o['id']:o for o in obs}
|
||||
visual={o['id']:decide(o,'strict',sig,use_text=False) for o in obs}
|
||||
# Only uncertainty in the visual evidence schedules expensive text.
|
||||
ambiguous={o['id'] for o in obs if not visual[o['id']] or len(o['rank'])>1 and o['rank'][0][1]<5}
|
||||
arms={
|
||||
'every_attempt':{'ocr_ids':{o['id'] for o in obs},'attempts':2},
|
||||
'once_per_track':{'ocr_ids':{o['id'] for o in obs},'attempts':2},
|
||||
'ambiguous_track':{'ocr_ids':ambiguous,'attempts':2},
|
||||
'group_representative':{'ocr_ids':{max(group,key=lambda i:lookup[i]['sharpness']) for group in groups if any(i in ambiguous for i in group)},'attempts':2},
|
||||
}
|
||||
result={}
|
||||
for name,arm in arms.items():
|
||||
if name=='group_representative':
|
||||
source={i:max(g,key=lambda k:lookup[k]['sharpness']) for g in groups for i in g}
|
||||
else:source={i:i for i in lookup}
|
||||
masked=[]
|
||||
for o in obs:
|
||||
clone=copy.deepcopy(o); origin=source[o['id']]
|
||||
# Representative's read is reusable only when its group
|
||||
# correspondence passed the boundary guard. Read is not a
|
||||
# perfect label; it goes through the unchanged policy.
|
||||
if origin not in arm['ocr_ids']:
|
||||
clone['text']={};clone['read']=''
|
||||
else:
|
||||
clone['text']=lookup[origin]['text'];clone['read']=lookup[origin]['read']
|
||||
masked.append(clone)
|
||||
bases={o['id']:decide(o,'coverage-first',sig) for o in masked}
|
||||
available_groups=infer_groups(d,analysis['resolutions'][res]['guarded_edges'],{o['id']:decide(o,'strict',sig) for o in masked})
|
||||
fused,changes=propagate(masked,available_groups,bases,sig,guarded=True)
|
||||
number=len(arm['ocr_ids'])*(arm['attempts'] if name=='every_attempt' else 1)
|
||||
ocr_ms=sum(lookup[i]['ocr_ms'] for i in arm['ocr_ids'])*(arm['attempts'] if name=='every_attempt' else 1)
|
||||
local_ms=sum(o['local_ms'] for o in obs)*arm['attempts']
|
||||
result[name]={'individual':metrics(masked,bases,families),'group':metrics(masked,fused,families,bases),'ocr_calls':number,'ocr_calls_per_facing':number/len(obs),'ocr_ms_serial_projection':round(ocr_ms,1),'matcher_calls':len(obs)*arm['attempts'],'matcher_ms_serial_projection':round(local_ms,1),'read_coverage':sum(bool(lookup[i]['read']) for i in arm['ocr_ids']),'ocr_ids':sorted(arm['ocr_ids']),'rescues':changes}
|
||||
report['arms'][res]=result
|
||||
print(res)
|
||||
for name,r in result.items():print(name,r['ocr_calls'],r['group']['shown'],r['group']['correct'],r['group']['wrong'],r['ocr_ms_serial_projection'])
|
||||
(OUT/'ocr-ablation.json').write_text(json.dumps(report,ensure_ascii=False,indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
{
|
||||
"scope": "Research prototypes only; production files unchanged",
|
||||
"baseline_revision": "fbf31d74feb12538dc07207cb80722dac38dd438",
|
||||
"primary_data": "eval/labels/image6.json",
|
||||
"catalog": "stock-lopia-clean",
|
||||
"data_status": "Single development photo, previously tuned. No cross-visit generalization claim.",
|
||||
"controls": ["Freeze catalog and labels; hash inputs", "Extract evidence once, replay identical evidence across policies", "Inference receives boxes and observations, never identity ground truth", "Oracle arm alone receives true group membership", "Unknown is not a SKU: never merge unknowns merely because labels equal", "Report labels over all 109 facings and correct labels over all 64 catalog facings separately", "Keep geometry-derived rows fixed across methods", "Compare deterministic runs to chain dynamic programming", "Report group rescue, correction and damage separately", "Separate measured wall time from cached-evidence simulations", "Never call a synthetic pan a recorded walk", "Use paced video timestamps, not result-synchronous sends, for full-rate replay"],
|
||||
"hypotheses": {
|
||||
"A": "Color plus normalized appearance and local correspondences separate repeated facings better than either alone; brand siblings remain difficult.",
|
||||
"B": "Geometry-only rows plus appearance-supported run edges identify repeats without truth labels; leave-one-row-out calibration limits same-row tuning.",
|
||||
"C": "Perfect membership sharing observed strong member evidence recovers at least 5 percentage points of catalog-facing correct coverage. No oracle SKU assignment.",
|
||||
"D": "Inferred group recovery retains useful oracle gain without more than one damaged originally correct facing; measure rather than require a positive result.",
|
||||
"E": "Visual-first scheduling and selective group representative OCR reduce actual OCR calls and make first evidence available sooner; never infer speed from call counts alone.",
|
||||
"F": "Coverage-first has greater product utility than strict even with lower shown-label precision. Near-argmax is a measured stress point, not the presumed winner.",
|
||||
"G": "Independent frame pacing exposes queue churn that the existing send/result client cannot test because both wait for every result. Compare identical source frame sequences at native sample pacing and 0.5x."
|
||||
},
|
||||
"policies": {"strict":4.0,"balanced":2.0,"coverage-first":0.5,"near-argmax":0.0},
|
||||
"confirmation": "Strict production decision is strong evidence, not calibrated probability. All extra identities provisional.",
|
||||
"utility_costs_per_facing_second": {"blank":1.0,"wrong_sibling_provisional":1.5,"wrong_other_provisional":4.0,"wrong_confirmed":8.0,"correct":0.0,"flip_event":0.2},
|
||||
"utility_sensitivity": "Sweep wrong-label cost; no empirical user-cost calibration claimed",
|
||||
"required_outputs": ["raw per-facing and per-pair evidence", "boundary/group confusion counts", "policy frontier plot", "OCR arm costs", "per-frame live timelines", "latency p50/p95 and CPU", "decision A-H with limitations and implementation acceptance"]
|
||||
}
|
||||
|
|
@ -0,0 +1,112 @@
|
|||
"""G: fixed video frames, independently paced sender/receiver, real WebSocket server.
|
||||
Run against the already-running local server; no source frame/video is copied.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import os
|
||||
import json
|
||||
import time
|
||||
import asyncio
|
||||
import statistics
|
||||
import subprocess
|
||||
import hashlib
|
||||
import argparse
|
||||
import resource
|
||||
from collections import defaultdict
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0,str(ROOT/'code'))
|
||||
import cv2
|
||||
from websockets.asyncio.client import connect
|
||||
from stream_client import FrameSource,encode
|
||||
OUT=Path(__file__).resolve().parent/'evidence'
|
||||
|
||||
|
||||
def q(values,p):
|
||||
if not values:return None
|
||||
return round(sorted(values)[min(len(values)-1,int((len(values)-1)*p))],2)
|
||||
|
||||
|
||||
def input_frames(source, fps, duration=5):
|
||||
if source=='image6':
|
||||
view=FrameSource(str(ROOT/'add/image6.jpg'),width=1280,pan_step=5.6)
|
||||
out=[]
|
||||
for n in range(int(duration*fps)):
|
||||
out.append((encode(view.frame(n),1280,75),view.offset,view.scale))
|
||||
view.release()
|
||||
return out
|
||||
cap=cv2.VideoCapture(str(ROOT/source))
|
||||
sample=[]; frame_count=int(cap.get(cv2.CAP_PROP_FRAME_COUNT)); native_fps=cap.get(cv2.CAP_PROP_FPS)
|
||||
start=max(0,(frame_count-int(duration*native_fps))//2)
|
||||
for n in range(int(duration*fps)):
|
||||
cap.set(cv2.CAP_PROP_POS_FRAMES,start+int(n*native_fps/fps))
|
||||
ok,frame=cap.read()
|
||||
if not ok:break
|
||||
sample.append((encode(frame,1280,75),(0,0),1.))
|
||||
cap.release()
|
||||
return sample
|
||||
|
||||
|
||||
async def trial(source,frames,fps,rate,trial_id):
|
||||
# Real-time source sampling is frozen across the two pace arms. At .5x each
|
||||
# source frame is shown longer, rather than processing a different sequence.
|
||||
secret=(ROOT/'.demo-token').read_text().strip()
|
||||
started=time.perf_counter(); results=[]; sent={}; first={}; transitions=defaultdict(list); errors=[]
|
||||
max_tracks=0; pid=int(subprocess.check_output(['lsof','-tiTCP:8099','-sTCP:LISTEN'],text=True).splitlines()[0])
|
||||
from psutil import Process
|
||||
proc=Process(pid); cpu0=proc.cpu_times(); cpu_before=time.perf_counter()
|
||||
async with connect('ws://127.0.0.1:8099/v1/stream',additional_headers={'Authorization':f'Bearer {secret}'},max_size=16*1024*1024) as socket:
|
||||
hello=json.loads(await socket.recv())
|
||||
if hello.get('type')!='hello':raise RuntimeError('server did not greet authenticated session')
|
||||
await socket.send(json.dumps({'type':'config','tags':False,'empty_slots':False}))
|
||||
await socket.recv()
|
||||
async def receive():
|
||||
nonlocal max_tracks
|
||||
while True:
|
||||
try: raw=await asyncio.wait_for(socket.recv(), timeout=2)
|
||||
except asyncio.TimeoutError: break
|
||||
msg=json.loads(raw)
|
||||
if msg.get('type')!='result':
|
||||
errors.append(msg);continue
|
||||
now=time.perf_counter(); tracks=msg['tracks']; max_tracks=max(max_tracks,len(tracks))
|
||||
visible=[]
|
||||
for tr in tracks:
|
||||
state=tr['identity']; sku=state.get('sku_id'); tid=tr['track_id']
|
||||
if sku:
|
||||
if tid not in first:first[tid]=now
|
||||
previous=transitions[tid]
|
||||
if not previous or previous[-1]['sku']!=sku:previous.append({'ms':round((now-started)*1000,1),'sku':sku,'state':state.get('state')})
|
||||
box=tr.get('box') or tr.get('raw_box')
|
||||
origin,scale=frames[min(len(frames)-1,max(0,msg['frame_id']-1))][1:]
|
||||
visible.append({'id':tid,'sku':sku,'state':state.get('state'),'identity_level':state.get('identity_level'),'box':[round(box[j]/scale+origin[j%2],1) for j in range(4)] if box else [],'object_generation':tr.get('object_generation'),'restarts':tr.get('restarts'), 'source':state.get('source')})
|
||||
results.append({'frame_id':msg['frame_id'],'received_ms':round((now-started)*1000,1),'sent_ms':round((sent.get(msg['frame_id'],started)-started)*1000,1),'timing_ms':msg.get('timing_ms'),'stats':msg.get('stats'), 'tracks':visible})
|
||||
receiver=asyncio.create_task(receive())
|
||||
for n,(frame,*_) in enumerate(frames):
|
||||
deadline=started+n/(fps*rate)
|
||||
await asyncio.sleep(max(0,deadline-time.perf_counter()))
|
||||
await socket.send((n+1).to_bytes(4,'big')+frame)
|
||||
sent[n+1]=time.perf_counter()
|
||||
await asyncio.wait_for(receiver, timeout=max(15,len(frames)/fps/rate+6))
|
||||
elapsed=time.perf_counter()-started; cpu1=proc.cpu_times();cpu=(cpu1.user+cpu1.system-cpu0.user-cpu0.system)
|
||||
received={r['frame_id'] for r in results}
|
||||
stages={k:[r['timing_ms'][k] for r in results if r.get('timing_ms') and r['timing_ms'].get(k) is not None] for k in ['decode','detect','track','total']}
|
||||
rtt=[r['received_ms']-r['sent_ms'] for r in results]
|
||||
levels=defaultdict(int)
|
||||
for r in results:
|
||||
for t in r['tracks']:
|
||||
if t['sku']: levels[t.get('identity_level') or 'unset']+=1
|
||||
summary={'source':source,'fps':fps,'rate':rate,'trial':trial_id,'input_frames':len(frames),'sent':len(sent),'processed':len(results),'not_returned':len(sent)-len(received),'elapsed_s':round(elapsed,2),'server_cpu_s':round(cpu,2),'server_cpu_pct_one_core':round(cpu/max(.001,time.perf_counter()-cpu_before)*100,1),'rtt_ms':{'p50':q(rtt,.5),'p95':q(rtt,.95)},'stages_ms':{k:{'p50':q(v,.5),'p95':q(v,.95)} for k,v in stages.items()},'max_tracks':max_tracks,'ever_labelled_track_ids':len(first),'first_visible_ms':min((v-started)*1000 for v in first.values()) if first else None,'label_flips':sum(max(0,len(x)-1) for x in transitions.values()),'identity_levels':dict(levels),'errors':errors,'latest_stats':results[-1]['stats'] if results else None}
|
||||
output={'schema_version':2,'coordinate_basis':'displayed box mapped with result frame origin','summary':summary,'timelines':results,'transitions':dict(transitions),'input_hash':hashlib.sha256(b''.join(x[0] for x in frames)).hexdigest()}
|
||||
path=OUT/f'replay-{Path(source).stem}-{fps}fps-{rate}x-{trial_id}.json';path.write_text(json.dumps(output,ensure_ascii=False,indent=1))
|
||||
print(path.name,summary['processed'],'/',summary['sent'],'first',summary['first_visible_ms'],'detect',summary['stages_ms']['detect'],'cpu',summary['server_cpu_s'],flush=True)
|
||||
|
||||
|
||||
async def main():
|
||||
parser=argparse.ArgumentParser();parser.add_argument('--source',default='image6');parser.add_argument('--fps',type=int,default=8);parser.add_argument('--seconds',type=int,default=5);parser.add_argument('--trials',type=int,default=2)
|
||||
args=parser.parse_args();OUT.mkdir(exist_ok=True)
|
||||
frames=input_frames(args.source,args.fps,args.seconds)
|
||||
if not frames:raise SystemExit('source yielded no frames')
|
||||
for i in range(args.trials):
|
||||
for rate in (1,.5):await trial(args.source,frames,args.fps,rate,i)
|
||||
|
||||
if __name__=='__main__':asyncio.run(main())
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
"""Cached-evidence representative ablation; serial cost projections, not wall time.
|
||||
Selection uses appearance-only groups; unavailable matcher evidence is masked.
|
||||
"""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import sys
|
||||
import statistics
|
||||
import time
|
||||
from collections import Counter
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[3];sys.path.insert(0,str(ROOT/'code'))
|
||||
from identity_lanes import DecisionPolicy
|
||||
from analyze import decide,metrics,propagate,infer_groups
|
||||
DIR=Path(__file__).resolve().parent;OUT=DIR/'evidence'
|
||||
|
||||
|
||||
def main():
|
||||
observations=json.loads((OUT/'observations.json').read_text());analysis=json.loads((OUT/'analysis.json').read_text());family=json.loads((DIR/'group-truth.json').read_text())['families'];summary={}
|
||||
for res,d in observations['resolutions'].items():
|
||||
rows=d['observations'];lookup={x['id']:x for x in rows}; groups=analysis['resolutions'][res]['appearance_groups']; sig=observations['signatures']
|
||||
baseline={o['id']:decide(o,'coverage-first',sig) for o in rows}
|
||||
arms={}
|
||||
for arm in ('sharpest-one','sharpest-two','all'):
|
||||
selected=set()
|
||||
for group in groups:
|
||||
ordered=sorted(group,key=lambda i:lookup[i]['sharpness'],reverse=True)
|
||||
selected.update(ordered[:len(group) if arm=='all' else 2 if arm=='sharpest-two' else 1])
|
||||
# Unlike the OCR-representative arm, unselected members have no
|
||||
# matcher observation. Their individually read OCR may still speak.
|
||||
masked=[]
|
||||
for o in rows:
|
||||
clone=dict(o)
|
||||
if o['id'] not in selected:clone['rank']=[];clone['local']={}
|
||||
masked.append(clone)
|
||||
individual={o['id']:decide(o,'coverage-first',sig) for o in masked}
|
||||
available_groups=infer_groups(d,analysis['resolutions'][res]['guarded_edges'],{o['id']:decide(o,'strict',sig) for o in masked})
|
||||
inferred,changes=propagate(masked,available_groups,individual,sig,guarded=True)
|
||||
matcher_ms=sum(lookup[i]['local_ms'] for i in selected)
|
||||
descriptor_ms=sum(lookup[i]['pair_features_ms'] for i in lookup) # group prep retains descriptor extraction
|
||||
pair_ms=sum(p['pair_ms'] for p in d['pairs'])
|
||||
arms[arm]={'matcher_calls':len(selected),'matcher_ms_serial_projection':round(matcher_ms,1),'group_descriptor_ms':round(descriptor_ms,1),'pair_match_ms':round(pair_ms,1),'total_identity_and_group_ms_serial_projection':round(matcher_ms+descriptor_ms+pair_ms,1),'individual':metrics(masked,individual,family,baseline),'inferred':metrics(masked,inferred,family,baseline),'changes':changes}
|
||||
summary[res]=arms
|
||||
for arm,v in arms.items():print(res,arm,v['matcher_calls'],v['inferred']['correct'],v['inferred']['wrong'],v['total_identity_and_group_ms_serial_projection'])
|
||||
(OUT/'representative.json').write_text(json.dumps({'provenance':observations['provenance'],'arms':summary},ensure_ascii=False,indent=2))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
"""Sampled WebSocket display proxy; not instrumented phone rendering."""
|
||||
from pathlib import Path
|
||||
import json
|
||||
import sys
|
||||
from collections import defaultdict
|
||||
from statistics import median
|
||||
|
||||
ROOT=Path(__file__).resolve().parents[3]
|
||||
sys.path.insert(0,str(ROOT/'code'))
|
||||
from stream_client import FrameSource
|
||||
from score_stream import iou,visible_fraction
|
||||
DIR=Path(__file__).resolve().parent; OUT=DIR/'evidence'
|
||||
LABELS=json.loads((ROOT/'eval/labels/image6.json').read_text())['facings']
|
||||
|
||||
|
||||
def windows(source,frames):
|
||||
if source!='image6': return {}
|
||||
view=FrameSource(str(ROOT/'add/image6.jpg'),width=1280,pan_step=5.6)
|
||||
result={}
|
||||
for n in range(frames):
|
||||
view.frame(n);result[n+1]=view.windows[-1]
|
||||
view.release();return result
|
||||
|
||||
|
||||
def score(run):
|
||||
s=run['summary'];w=windows(s['source'],s['input_frames'])
|
||||
histories=defaultdict(list);entry={};first={};unmatched=0
|
||||
truth={f['index']:f['label'] for f in LABELS if f['label']!='exclude'}
|
||||
for sequence,r in enumerate(run['timelines']):
|
||||
facings=[f for f in LABELS if f['label']!='exclude' and visible_fraction(f['box'],w[r['frame_id']])>=.5]
|
||||
for f in facings:entry.setdefault(f['index'],r['sent_ms'])
|
||||
# Greedy descending IoU, one track per facing and one facing per track.
|
||||
# Match before considering SKU so a correct label never wins a tie by truth.
|
||||
pairs=sorted(((iou(t['box'],f['box']),j,f['index']) for j,t in enumerate(r['tracks']) for f in facings),reverse=True)
|
||||
assigned={};used=set()
|
||||
for overlap,j,key in pairs:
|
||||
if overlap<.2:break
|
||||
if key not in assigned and j not in used:
|
||||
assigned[key]=r['tracks'][j]['sku'];used.add(j)
|
||||
unmatched+=sum(bool(t['sku']) and j not in used for j,t in enumerate(r['tracks']))
|
||||
for f in facings:
|
||||
key=f['index'];sku=assigned.get(key)
|
||||
histories[key].append((r['received_ms'],sku,sequence))
|
||||
if sku:first.setdefault(key,r['received_ms'])
|
||||
timed={}
|
||||
for delay in (250,500,1000):
|
||||
eligible=[k for k,h in histories.items() if h[-1][0]>=entry[k]+delay]
|
||||
values={k:next((sku for at,sku,_ in reversed(histories[k]) if at<=entry[k]+delay),None) for k in eligible}
|
||||
known=[k for k in eligible if truth[k]!='unknown']
|
||||
named=sum(bool(v) for v in values.values());correct=sum(bool(v) and v==truth[k] for k,v in values.items())
|
||||
timed[str(delay)]={'visible_denominator':len(eligible),'catalog_denominator':len(known),'any_label':named,'correct_label':correct,'vrc':named/len(eligible) if eligible else None,'correct_catalog_coverage':correct/len(known) if known else None}
|
||||
corrections=[];flips=0;wrong_ms=0.;stable=[];stable_at=[];last_correct=0
|
||||
for key,h in histories.items():
|
||||
previous=None
|
||||
for (at,sku,sequence),(end,_,next_sequence) in zip(h,h[1:]):
|
||||
if sku and sku!=truth[key] and next_sequence==sequence+1:wrong_ms+=end-at
|
||||
for at,sku,_ in h:
|
||||
if sku and previous and sku!=previous:
|
||||
flips+=1
|
||||
if sku==truth[key]:corrections.append(key)
|
||||
if sku:previous=sku
|
||||
if truth[key]=='unknown' or h[-1][1]!=truth[key]:continue
|
||||
last_correct+=1;start=len(h)-1
|
||||
while start>0 and h[start-1][1]==truth[key] and h[start-1][2]+1==h[start][2]:start-=1
|
||||
if h[-1][0]-h[start][0]>=500:
|
||||
stable.append(key);stable_at.append(h[start][0]-entry[key])
|
||||
known=sum(truth[k]!='unknown' for k in histories)
|
||||
return {'visible_facings':len(histories),'catalog_facings':known,'ever_named':len(first),'never_named_ratio':1-len(first)/len(histories),'last_observed_correct_catalog':last_correct,'last_observed_correct_coverage':last_correct/known if known else None,'stable_correct_catalog_500ms':len(stable),'stable_correct_coverage_500ms':len(stable)/known if known else None,'time_to_trailing_stable_correct_ms_p50':median(stable_at) if stable_at else None,'timed':timed,'first_label_from_entry_ms_p50':median(first[k]-entry[k] for k in first) if first else None,'corrections':sorted(set(corrections)),'identity_flips':flips,'sampled_wrong_label_ms':wrong_ms,'unmatched_label_samples':unmatched,'note':'Receive timestamps proxy display; entry is first processed source window with >=50% visibility. Stable means a trailing correct run spanning >=500 ms of consecutive received results, not future stability. Wrong exposure holds each received state until next result while facing stays visible; final intervals censored. Crop-position matching is diagnostic, not native overlay verification.'}
|
||||
|
||||
|
||||
def main():
|
||||
reports={}
|
||||
for path in sorted(OUT.glob('replay-*.json')):
|
||||
if path.name in ('replay-scores.json','replay-scores-initial.json'):continue
|
||||
run=json.loads(path.read_text());name=path.stem
|
||||
reports[name]={'summary':run['summary'],'input_hash':run['input_hash']}
|
||||
if run['summary']['source']=='image6':reports[name]['visible']=score(run)
|
||||
(OUT/'replay-scores.json').write_text(json.dumps(reports,ensure_ascii=False,indent=2))
|
||||
for name,r in reports.items():
|
||||
v=r.get('visible',{});print(name,v.get('last_observed_correct_catalog'),v.get('stable_correct_catalog_500ms'),v.get('timed'))
|
||||
|
||||
if __name__=='__main__':main()
|
||||
|
|
@ -0,0 +1,327 @@
|
|||
"""I: bounded temporal pair-cache diagnostic over saved replay timelines.
|
||||
|
||||
Counters only: hits/misses, unique pairs, updates, absence invalidations, and
|
||||
how often a named identity changed between consecutive named observations.
|
||||
No latency claim, no SKU propagation, no accuracy claim, no production edits.
|
||||
|
||||
Reuses the frozen geometry conventions: `collect.rows_from_boxes` (bottom-aligned
|
||||
shelf rows) and `analyze.geometric_edge` (the gap/height/bottom edge rule).
|
||||
"""
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import json
|
||||
import hashlib
|
||||
import statistics
|
||||
from collections import Counter, defaultdict
|
||||
|
||||
DIR = Path(__file__).resolve().parent
|
||||
sys.path.insert(0, str(DIR))
|
||||
from analyze import geometric_edge
|
||||
from collect import rows_from_boxes
|
||||
from eval_identity import code_revision
|
||||
|
||||
# Declared before looking at any hit count: a member box is "unchanged" while it
|
||||
# keeps IoU >= 0.9 with the box the edge was cached against, i.e. under ~10% of
|
||||
# the smaller box's area moved between the frame the edge was measured on and
|
||||
# the frame it is consulted.
|
||||
DECLARED_KEEP_IOU = 0.9
|
||||
SENSITIVITY_IOU = (0.8, 0.9, 0.95, 1.0)
|
||||
PRIMARY = 'replay-image6-8fps-1x-1.json'
|
||||
GROUPS = {'arms': 'evidence', 'legacy_label_anchor_arms': 'evidence/initial-replays'}
|
||||
|
||||
|
||||
def iou(a, b):
|
||||
inter = max(0.0, min(a[2], b[2]) - max(a[0], b[0])) * max(0.0, min(a[3], b[3]) - max(a[1], b[1]))
|
||||
union = (a[2] - a[0]) * (a[3] - a[1]) + (b[2] - b[0]) * (b[3] - b[1]) - inter
|
||||
return inter / union if union > 0 else 0.0
|
||||
|
||||
|
||||
def edge_signals(a, b):
|
||||
"""collect.pair_signals' geometry half: a is left of b in a rows_from_boxes row."""
|
||||
aw, ah = a[2] - a[0], a[3] - a[1]
|
||||
bw, bh = b[2] - b[0], b[3] - b[1]
|
||||
return {'gap': (b[0] - a[2]) / max(aw, bw), 'height': min(ah, bh) / max(ah, bh), 'bottom': abs(a[3] - b[3]) / max(ah, bh)}
|
||||
|
||||
|
||||
class PairCache:
|
||||
"""Edge memo keyed by track identities AND object generations.
|
||||
|
||||
A cached edge is reused only when the pair is still adjacent, both members
|
||||
still carry the generation it was cached under, and both boxes are still
|
||||
inside the declared IoU of the boxes it was cached against. Anything else
|
||||
is a miss, and a miss rewrites the entry.
|
||||
"""
|
||||
def __init__(self, keep_iou=DECLARED_KEEP_IOU):
|
||||
self.keep_iou = keep_iou
|
||||
self.entries = {}
|
||||
self.lookups = self.hits = self.misses = self.updates = 0
|
||||
self.drift = []
|
||||
self.reasons = Counter()
|
||||
|
||||
@staticmethod
|
||||
def pair(ka, kb):
|
||||
return (ka[0], kb[0]) if ka[0] <= kb[0] else (kb[0], ka[0])
|
||||
|
||||
def get(self, ka, kb, ba, bb, adjacent=True):
|
||||
self.lookups += 1
|
||||
if not adjacent:
|
||||
return self._miss('adjacency')
|
||||
entry = self.entries.get(self.pair(ka, kb))
|
||||
if entry is None:
|
||||
return self._miss('new')
|
||||
if entry['keys'] != (ka, kb):
|
||||
return self._miss('generation')
|
||||
if iou(ba, entry['boxes'][ka]) < self.keep_iou or iou(bb, entry['boxes'][kb]) < self.keep_iou:
|
||||
return self._miss('geometry')
|
||||
self.hits += 1
|
||||
self.drift += [iou(ba, entry['boxes'][ka]), iou(bb, entry['boxes'][kb])]
|
||||
return entry
|
||||
|
||||
def put(self, ka, kb, ba, bb, edge, row):
|
||||
pair = self.pair(ka, kb)
|
||||
if pair in self.entries:
|
||||
self.updates += 1
|
||||
self.entries[pair] = {'keys': (ka, kb), 'boxes': {ka: ba, kb: bb}, 'edge': edge, 'row': row}
|
||||
|
||||
def _miss(self, reason):
|
||||
self.misses += 1
|
||||
self.reasons[reason] += 1
|
||||
return None
|
||||
|
||||
|
||||
def self_check():
|
||||
"""A cached edge must not survive a generation change or an adjacency change."""
|
||||
a, b = [10, 100, 90, 300], [95, 102, 175, 300]
|
||||
ka, kb = (7, 0), (9, 0)
|
||||
assert geometric_edge({'signals': edge_signals(a, b)}) is True
|
||||
c = PairCache()
|
||||
assert c.get(ka, kb, a, b) is None and c.reasons['new'] == 1
|
||||
c.put(ka, kb, a, b, True, 0)
|
||||
assert c.get(ka, kb, a, b)['edge'] is True
|
||||
assert c.get((7, 1), kb, a, b) is None and c.reasons['generation'] == 1
|
||||
assert c.get((9, 1), ka, a, b) is None and c.reasons['generation'] == 2, 'unordered key must still catch a generation change'
|
||||
assert c.get(ka, kb, a, b, adjacent=False) is None and c.reasons['adjacency'] == 1
|
||||
assert c.get(ka, kb, [10, 100, 90, 420], b) is None and c.reasons['geometry'] == 1
|
||||
assert c.get(ka, kb, a, b)['edge'] is True, 'a declined lookup must not evict a live entry'
|
||||
return {'first_lookup': 'miss/new', 'repeat_lookup': 'hit', 'generation_invalidations': 2,
|
||||
'adjacency_invalidations': 1, 'geometry_invalidations': 1, 'entry_survived_declined_lookups': True}
|
||||
|
||||
|
||||
def generations(timelines):
|
||||
"""Per-result track_id -> generation, plus where the value came from.
|
||||
|
||||
live.py increments TrackState.object_generation and _restart_count in the
|
||||
same two branches -- a recycled id, and an id absent from the previous
|
||||
frame -- off the same counters, so restarts == object_generation for every
|
||||
track. object_generation is not in _track_payload, so on the wire restarts
|
||||
is the only generation evidence. With neither recorded, the generation falls
|
||||
back to a gap proxy, and the proxy is always reported next to the wire value
|
||||
so the two can be compared instead of assumed equal.
|
||||
"""
|
||||
per_result, proxy, recorded, previous = [], {}, set(), {}
|
||||
for index, frame in enumerate(timelines):
|
||||
current = {}
|
||||
for track in frame['tracks']:
|
||||
if track.get('object_generation') is not None:
|
||||
recorded.add('object_generation')
|
||||
if track.get('restarts') is not None:
|
||||
recorded.add('restarts')
|
||||
for track in frame['tracks']:
|
||||
tid = track['id']
|
||||
if tid in previous and index - previous[tid] > 1:
|
||||
proxy[tid] = proxy.get(tid, 0) + 1
|
||||
value = track.get('object_generation')
|
||||
if value is None:
|
||||
value = track.get('restarts')
|
||||
current[tid] = int(value) if value is not None else proxy.get(tid, 0)
|
||||
per_result.append(current)
|
||||
previous = {track['id']: index for track in frame['tracks']}
|
||||
source = next((name for name in ('object_generation', 'restarts') if name in recorded), 'absence_proxy')
|
||||
return per_result, source, proxy
|
||||
|
||||
|
||||
def run(path, keep_iou):
|
||||
data = json.loads(path.read_text())
|
||||
timelines = data['timelines']
|
||||
per_result, source, proxy = generations(timelines)
|
||||
cache = PairCache(keep_iou)
|
||||
named = defaultdict(list)
|
||||
unique_ids, unique_pairs, previous_pairs = set(), set(), set()
|
||||
per_result_pairs, rows_per_result, adjacency_rebuilds = [], [], 0
|
||||
named_obs = unnamed_obs = degenerate = edge_yes = 0
|
||||
final_generation, generation_history = {}, {}
|
||||
for index, frame in enumerate(timelines):
|
||||
tracks = []
|
||||
for track in frame['tracks']:
|
||||
box = track.get('box')
|
||||
if not box or box[2] <= box[0] or box[3] <= box[1]:
|
||||
degenerate += 1
|
||||
continue
|
||||
tracks.append({'id': track['id'], 'box': box})
|
||||
unique_ids.add(track['id'])
|
||||
final_generation[track['id']] = per_result[index].get(track['id'], 0)
|
||||
generation_history.setdefault(track['id'], set()).add(final_generation[track['id']])
|
||||
if track.get('sku'):
|
||||
named[track['id']].append(track['sku'])
|
||||
named_obs += 1
|
||||
else:
|
||||
unnamed_obs += 1
|
||||
rows = rows_from_boxes(tracks)
|
||||
current = set()
|
||||
for row_index, row in enumerate(rows):
|
||||
for left, right in zip(row, row[1:]):
|
||||
ka = (left['id'], per_result[index].get(left['id'], 0))
|
||||
kb = (right['id'], per_result[index].get(right['id'], 0))
|
||||
entry = cache.get(ka, kb, left['box'], right['box'])
|
||||
if entry is None:
|
||||
edge = geometric_edge({'signals': edge_signals(left['box'], right['box'])})
|
||||
edge_yes += int(edge)
|
||||
cache.put(ka, kb, left['box'], right['box'], edge, row_index)
|
||||
else:
|
||||
edge_yes += int(entry['edge'])
|
||||
current.add(cache.pair(ka, kb))
|
||||
rows_per_result.append(len(rows))
|
||||
per_result_pairs.append(len(current))
|
||||
unique_pairs |= current
|
||||
adjacency_rebuilds += len(current - previous_pairs)
|
||||
previous_pairs = current
|
||||
unchanged = changed = 0
|
||||
for sequence in named.values():
|
||||
for a, b in zip(sequence, sequence[1:]):
|
||||
unchanged += int(a == b)
|
||||
changed += int(a != b)
|
||||
disagreeing = {}
|
||||
for tid, values in generation_history.items():
|
||||
wire_value = final_generation.get(tid, 0)
|
||||
if len(values) > 1 or wire_value != proxy.get(tid, 0):
|
||||
disagreeing[str(tid)] = {'wire': wire_value, 'gap_proxy': proxy.get(tid, 0),
|
||||
'changed_during_run': len(values) > 1}
|
||||
total = cache.lookups
|
||||
return {
|
||||
'results': len(timelines), 'schema_version': data.get('schema_version'),
|
||||
'coordinate_basis': data.get('coordinate_basis'),
|
||||
'source': data['summary']['source'], 'fps': data['summary']['fps'],
|
||||
'rate': data['summary']['rate'], 'trial': data['summary']['trial'],
|
||||
'frames': {'sent': data['summary']['sent'], 'returned': data['summary']['processed']},
|
||||
'declared_keep_iou': keep_iou,
|
||||
'rows': {'total': sum(rows_per_result), 'p50_per_result': statistics.median(rows_per_result),
|
||||
'max_per_result': max(rows_per_result, default=0)},
|
||||
'adjacent_pairs': {'per_result_recompute_workload': sum(per_result_pairs),
|
||||
'p50_per_result': statistics.median(per_result_pairs),
|
||||
'unique_pairs': len(unique_pairs),
|
||||
'adjacency_rebuilds': adjacency_rebuilds},
|
||||
'tracks': {'unique_ids': len(unique_ids), 'observations': named_obs + unnamed_obs,
|
||||
'named_observations': named_obs, 'unnamed_observations': unnamed_obs,
|
||||
'degenerate_boxes_skipped': degenerate,
|
||||
'named_tracks': len(named),
|
||||
'distinct_skus': len({sku for sequence in named.values() for sku in sequence})},
|
||||
'cache': {'lookups': total, 'hits': cache.hits, 'misses': cache.misses,
|
||||
'hit_rate': cache.hits / total if total else None,
|
||||
'pair_computations_with_cache': cache.misses,
|
||||
'pair_computations_avoided': cache.hits,
|
||||
'miss_reasons': dict(cache.reasons), 'updates': cache.updates,
|
||||
'entries': len(cache.entries), 'edge_yes_computed_or_reused': edge_yes,
|
||||
'hit_drift_iou_vs_cached_box': {'min': min(cache.drift, default=None),
|
||||
'p50': statistics.median(cache.drift) if cache.drift else None,
|
||||
'max': max(cache.drift, default=None),
|
||||
'samples': len(cache.drift)}},
|
||||
'generations': {
|
||||
'source': source,
|
||||
'wire_generation_events': sum(value for value in final_generation.values()),
|
||||
'tracks_with_generation_change': sum(1 for values in generation_history.values() if len(values) > 1),
|
||||
'gap_proxy_events': sum(proxy.values()),
|
||||
'gap_proxy_disagreements': len(disagreeing),
|
||||
'gap_proxy_disagreeing_tracks': disagreeing,
|
||||
'generation_invalidations': cache.reasons['generation'],
|
||||
},
|
||||
'identity': {'named_transitions': unchanged + changed, 'unchanged': unchanged, 'changed': changed,
|
||||
'note': 'consecutive named observations of one track id; a change means the label flipped, '
|
||||
'repeated observations of an unchanged label count neither way'},
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
report = {
|
||||
'purpose': 'bounded diagnostic: how much of the geometry-only adjacent-pair work on a replay timeline '
|
||||
'is redundant between consecutive results',
|
||||
'declared_criterion': {
|
||||
'cache_key': '(track_id, generation) for both members, unordered',
|
||||
'keep_iou': DECLARED_KEEP_IOU,
|
||||
'reuse_requires': ['pair still adjacent in this result row ordering',
|
||||
'both member generations unchanged since the entry was written',
|
||||
f'both member boxes IoU >= {DECLARED_KEEP_IOU} with the boxes the entry was written against'],
|
||||
'cached_value': 'the geometry-only edge verdict (analyze.geometric_edge over collect pair_signals geometry)',
|
||||
},
|
||||
'self_check': self_check(),
|
||||
'provenance': {'code': code_revision()},
|
||||
'arms': {},
|
||||
'legacy_label_anchor_arms': {},
|
||||
'sensitivity_hit_rate': {},
|
||||
'limitations': [
|
||||
'appearance: only the geometry-only adjacent edge is cacheable from a replay timeline. The appearance '
|
||||
'half of analyze.pair_value (hue, pixels, logo_pixels, inliers, ocr_jaccard) needs per-track appearance '
|
||||
'features a timeline does not carry, so this is the geometry/adjacency reuse ceiling, not a full '
|
||||
'pair-score cache. The counters bound a cache of one boolean, not of descriptor cost.',
|
||||
'object_generation: replay.py records it, but live.py _track_payload never populates it, so it is null on '
|
||||
'every track in every arm. live.py increments object_generation and restarts in the same two branches '
|
||||
'off the same counters, so restarts carries the generation and is what these arms key on. A cache key '
|
||||
'naming object_generation specifically is therefore not yet constructible from the wire.',
|
||||
'session generation: a method change bumps LiveSession._generation and resets every track in place, '
|
||||
'leaving per-track object_generation untouched. A per-track generation key does not cover that; a real '
|
||||
'cache needs the session generation in the key too, and no arm here exercises a method change.',
|
||||
'absence: a gap between consecutive recorded results is not server absence. Arms return 9 of 40 sent '
|
||||
'frames, so a track can be missing from a recorded result while the server never lost it. gap_proxy_events '
|
||||
'is reported next to the wire generation and disagrees with it in exactly the arms listed under '
|
||||
'gap_proxy_disagreeing_tracks; the wire value is the one used.',
|
||||
'box basis differs between the two groups. `arms` is schema 2, coordinate_basis "displayed box mapped '
|
||||
'with result frame origin". `legacy_label_anchor_arms` is the archived schema-1 run, where replay.py '
|
||||
'took label_box (always non-empty, so raw_box/box never applied) and mapped it with the label frame '
|
||||
'pan offset; an unlabelled track has label_frame_id == 0, so its box was mapped with frame 0 origin, '
|
||||
'a pure x error of up to ~197 source px on image6 against a median box width of 89 px. The two groups '
|
||||
'are not comparable to each other and are reported separately.',
|
||||
'scope: arms differ only in pacing and source, and the still-image arms replay one development photo '
|
||||
'(add/image6.jpg, 111 labelled facings). No independent generalization, and no native UI or device path '
|
||||
'was exercised. Arms are also truncated under server load, so the named primary arm is the smallest of '
|
||||
'the set and its counters are the weakest.',
|
||||
'counters only: no latency saving is claimed, because the timeline carries no cost for the work a hit '
|
||||
'skips, and the declared IoU is a geometric restatement of "the box did not move", not a measurement.',
|
||||
'chained drift: the criterion compares each member against the box the entry was last written '
|
||||
'against, not against the previous result, so a box that creeps a little every frame can stay under '
|
||||
'the IoU forever while ending up far from where the verdict was computed. hit_drift_iou_vs_cached_box '
|
||||
'is the observed bound on that; a production cache that needs a hard staleness bound should compare '
|
||||
'against the previous result and cap the age of an entry.',
|
||||
],
|
||||
}
|
||||
for group, folder in GROUPS.items():
|
||||
for path in sorted((DIR / folder).glob('replay-*.json')):
|
||||
if 'timelines' not in json.loads(path.read_text()):
|
||||
continue
|
||||
arm = run(path, DECLARED_KEEP_IOU)
|
||||
report[group][path.name] = arm
|
||||
report['provenance'].setdefault('sources', []).append(
|
||||
{'group': group, 'file': path.name, 'sha256': hashlib.sha256(path.read_bytes()).hexdigest(),
|
||||
'primary': path.name == PRIMARY and group == 'arms'})
|
||||
report['sensitivity_hit_rate'].setdefault(group, {})[path.name] = {
|
||||
str(value): run(path, value)['cache']['hit_rate'] for value in SENSITIVITY_IOU}
|
||||
out = DIR / 'evidence/temporal-cache.json'
|
||||
out.write_text(json.dumps(report, ensure_ascii=False, indent=2))
|
||||
print('self_check', report['self_check'])
|
||||
for group in GROUPS:
|
||||
for name, arm in report[group].items():
|
||||
print(group, name, 'results', arm['results'], 'sent/returned', arm['frames'],
|
||||
'unique_pairs', arm['adjacent_pairs']['unique_pairs'],
|
||||
'lookups', arm['cache']['lookups'], 'hits', arm['cache']['hits'],
|
||||
'misses', arm['cache']['misses'], 'hit_rate', arm['cache']['hit_rate'],
|
||||
'reasons', arm['cache']['miss_reasons'],
|
||||
'gen_source', arm['generations']['source'],
|
||||
'wire_gen_events', arm['generations']['wire_generation_events'],
|
||||
'proxy_disagree', arm['generations']['gap_proxy_disagreements'],
|
||||
'identity_unchanged', arm['identity']['unchanged'],
|
||||
'identity_changed', arm['identity']['changed'])
|
||||
print('wrote', out)
|
||||
if PRIMARY in report['arms']:
|
||||
print('primary', PRIMARY, json.dumps(report['arms'][PRIMARY], indent=1)[:1400])
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue