docs(decision): measure G1 against Van on the same stream and catalog

Browse source

The G1 row compared against a van run whose server had loaded the legacy
stock-lopia catalog, so its ids were template-level (sku_005) and every answer
scored wrong. Re-run with both servers on stock-lopia-clean, 20 s pan 12:

  van    17/27 (63%) coverage   53% precision   7 wrong   6 unenrolled accepts
  local  19-20/27 (70-74%)      77-85%          0 wrong   0 unenrolled accepts

G1 asks the new system to be no worse than Van on coverage; it is better on
both axes, so G1 now reads as passing on the stream rather than unmeasured.
Both recordings are committed under stream-gated/.

...

This commit is contained in:

kiyo9w 2026-09-29 04:28:20 +09:00

parent 01d9a5e614

commit 81a2ab10e7

3 changed files with 2913 additions and 6 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

1169

recognization/data/evaluations/2026-09-29-session/stream-gated/van_score.json Normal file

View file

File diff suppressed because it is too large Load diff

1725

recognization/data/evaluations/2026-09-29-session/stream-gated/van_tracks.json Normal file

View file

File diff suppressed because it is too large Load diff