First time? Walk top-down from station 01 — the arc only makes sense in order.
Just want the numbers? Jump to the summit, then the research page.
Methodology-first? Start at the junction, where we audit the benchmarks themselves.
station 06 · the arena2026-07-05flagship · external
Our first third-party benchmark — their harness, their judge, their protocol,
unmodified. 48.7% category mean against EverMemOS’s published 44.6
on the leaderboard protocol (gpt-4.1-mini answers), and 51.5% with
deepseek-v4-flash at a fifth of the answer cost. Plus the honest part: the five
engine bugs the arena surfaced — multi-word tsquery crash, idle-connection corpses,
event-time-blind imports, a silent supersedes no-op, the REM cold-start flood —
every one fixed and shipped in 1.0.0-rc7 before we wrote a word about
winning.
48.7 vs 44.6 mean
53.5% pooled · flash
5 bugs fixed & shipped
enter station 06 →
station 05 · the summit2026-05-18flagship
The full bench-driven engineering loop: iter00 anchor R@5 = 0.6851 to iter100 champion
R@5 = 0.8426, across four architectural eras (retrieval → formation →
rerank-feedback → top-K sharpness). The empirical proof that
the bottleneck migrated upward in the cognition stack — and it is
as honest about the failed levers as the winning ones: naive canonicalization regression,
salience 3.0 over-boost, the KU rebake we rolled back.
iter00 → iter100
R@5 0.6851 → 0.8426
R@10 = 0.9000
R@1 = 0.6255
4 eras
enter station 05 →
station 04 · the junction2026-05-18flagship · methodology
Why we built our own memory benchmark instead of trusting the published ones. The
Hindsight int() truncation bug — 9 of 10 ability evaluators silently
dropped 0.5 partial credits. The 4/20 rubric-defect rate on BEAM-10M conv-1,
cross-confirmed in a blind audit. The 16pp judge spread on identical answers. The 10
models published as “0/N · not viable” that we ran at 90%.
If your benchmark cannot survive its own audit, your #1 is decoration, not
signal.
9/10 evaluators bugged
4/20 rubric defects
16pp judge spread
0/N → 18/20
enter station 04 →
station 03 · the climb2026-05-18benchmarks · formation
100 iterations on the hard LongMemEval-oracle 500q across four architectural eras. R@5
climbed 0.6851 → 0.8426 (+15.75pp absolute, crossing the 0.84 stretch target), R@10
broke the 0.90 barrier, ssu R@10 hit a perfect 1.0000. The retrieval-side knob surface
saturated at 0.7404; memory formation broke through it; the rerank-feedback discovery
pushed further still.
R@5 = 0.8426
R@10 = 0.9000
MRR = 0.7124
ssu R@10 = 1.0000
enter station 03 →
station 02 · the corridor2026-05-13infrastructure
One pg_restore, one sha256, four BEAM scales plus the full LongMemEval
triplet. Plus a 282× bulk-write refactor and the two pg_attribute-namespace bugs
that were silently leaking embedding dimensions across schemas.
957 MB snapshot
~3 s restore
88 min → 75 s
207 provenance rows
enter station 02 →
station 01 · the entrance2026-05-10audit
The note that started the maze. Why we ran the same 10 models Hindsight published as
“not viable”, and what 270 million parameters can do when you stop gating on
JSON. Eight rounds of GPT-5.5 review, v2 NO → v8 unconditional yes, no residual
caveat.
R@5 = 0.9787
MRR = 0.9114
188/200 = 94.0%
0 errors
enter station 01 →