diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index e220077..64acd01 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -294,6 +294,16 @@ Notes: 3. The budget record's c3 degradation class retires; KNOWN-LIMITATIONS updated **Notes**: graph-layout v2 with landmark distances precomputed at build time; upload script unchanged. +### Phase 08.1: Contraction Hierarchies (Routing Backend v3) — the lever that closes SC-1: build-time node ordering + contraction + shortcut edges, simple uphill bidirectional query, optimality by construction (INSERTED) + +**Goal:** [Urgent work - to be planned] +**Requirements**: TBD +**Depends on:** Phase 8 +**Plans:** 5/5 plans complete + +Plans: +- [ ] TBD (run /gsd-plan-phase 08.1 to break down) + ### Phase 9: Routing Extras (Toll / Speed / Share / Voice) **Goal**: The REQUIREMENTS extras go live on the engine-A stack. **Depends on**: Phase 8. diff --git a/.planning/STATE.md b/.planning/STATE.md index ca69770..8a7157f 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,16 +2,16 @@ gsd_state_version: 1.0 milestone: v1.x milestone_name: routing-engine-upgrades-and-extras -status: executing -stopped_at: "08-05 executed (D-03 optimality gate GREEN after the gate caught the four-family ALT steering returning suboptimal routes — fromTable families removed from the potential, Rule 1; the CPU campaign measured HONEST FAILs: 25-route p95 152.788 ms / extended 185.692 ms vs the 8 ms budget; the pre-committed L=32 ladder rung measured worse and stopped; c3 NOT retired — every record carries the quoted post-ALT truth; NEXT: maintainer decisions — R2 upload (deploy-time) + the SC-1 escalation track: direction-split potentials / CH / Workers Paid reversal / keep c3)" -last_updated: "2026-10-25T12:09:32.000Z" +status: ready_to_plan +stopped_at: "F1 remediation applied (debug alt-potential-4x-slowdown RC-1): clean-columns-only ALT containment in routing/src/astar.ts — per-node sentinel skip removed; suite re-spec'd RED-first (A11) + new A13 mixed-column case; new committed oracle sweep measure-astar-oracle-sweep.mjs 0/200 suboptimal; CPU back in the pre-ALT band (p95 136.6 ms this host vs 1037.6 pre-fix — honest c3 FAIL vs 8 ms stands); D-03 gate green. Commits c071e20 / 2a5a0b5 / 97d6e35. NEXT: maintainer R2 upload + the SC-1 escalation decision (unchanged)" +last_updated: "2026-10-25T14:15:06.953Z" last_activity: 2026-10-25 progress: total_phases: 4 - completed_phases: 1 + completed_phases: 2 total_plans: 5 completed_plans: 5 - percent: 100 + percent: 50 --- # Project State @@ -25,9 +25,9 @@ See: .planning/PROJECT.md (updated 2026-10-22) ## Current Position -Phase: 8 -Plan: 08-05 complete (all five plans executed; commits `149636c` / `79a302c` / `720ad43` + the measurement session) -Status: Phase 8 executed — maintainer decisions open (R2 upload; the SC-1 escalation track) +Phase: 08.1 +Plan: Not started +Status: Ready to plan Last activity: 2026-10-25 Progress: [██████████] 100% @@ -36,7 +36,7 @@ Progress: [██████████] 100% **Velocity:** -- Total plans completed: 34 +- Total plans completed: 39 - Average duration: — - Total execution time: — @@ -52,6 +52,7 @@ Progress: [██████████] 100% | 5 | 5 | - | - | | 6 | 8 | - | - | | 7 | 6 | - | - | +| 8 | 5 | - | - | ## Accumulated Context @@ -69,7 +70,7 @@ None yet. ### Blockers/Concerns -- **PHASE 8 ESCALATION (open, 2026-10-25):** the CPU verdict measured FAIL post-ALT (25-route p95 152.788 ms / extended 185.692 ms vs the 8 ms budget; the L=32 probe worse) — the c3 class remains. The theoretical wall: no two-sided-Lipschitz landmark extension exists on a graph with sentinel fringes; sound+fast needs direction-split potentials with a re-proven meet rule (search-skeleton work). Options: direction-split potentials (new phase) / CH (Phase-6 deferral) / Workers Paid reversal ($0 law) / keep c3 (current shipped posture). Records: docs/BUDGETS.md §c3, docs/KNOWN-LIMITATIONS.md, 08-05-SUMMARY.md. +- **PHASE 8 ESCALATION (open, 2026-10-25):** the CPU verdict measured FAIL post-ALT (25-route p95 152.788 ms / extended 185.692 ms vs the 8 ms budget; the L=32 probe worse) — the c3 class remains. The theoretical wall: no two-sided-Lipschitz landmark extension exists on a graph with sentinel fringes; sound+fast needs direction-split potentials with a re-proven meet rule (search-skeleton work). Options: direction-split potentials (new phase) / CH (Phase-6 deferral) / Workers Paid reversal ($0 law) / keep c3 (current shipped posture). Records: docs/BUDGETS.md §c3, docs/KNOWN-LIMITATIONS.md, 08-05-SUMMARY.md. **F1 addendum (2026-10-25, post-plan):** the debug session also exposed RC-1 — a LIVE correctness bug in the ALT potential (per-node sentinel skip broke cross-edge consistency; 1/194 fresh pairs suboptimal vs Dijkstra) — fixed the same day (clean-columns-only containment; commits c071e20/2a5a0b5/97d6e35; new committed oracle-sweep gate 0/200). On the current 0/24-clean binary ALT is inert (exact haversine), so the CPU band is pre-ALT again (p95 136.6 ms this host) and this escalation is UNCHANGED — a future clean-column binary re-arms ALT with no code change; F2 (transposed tables) is the then-relevant performance follow-up. - ~~PHASE 4 device-gate legs UNVERIFIED~~ RESOLVED 2026-10-24: legs 1–4+6 PASS on real hardware; leg 5 user-accepted (BotFather Main Mini App configured; unknown-id surface covered by automated gates). Known-open cosmetic: /cari absent from the client command menu (Telegram non-default setMyCommands defect; command works when typed; register script re-arms all scopes for retry). - Design-source conflict (root `DESIGN.md` typography vs `telemetry_rams/DESIGN.md` + mockups) — Phase 1 resolves; mockup markup is layout reference only, design doc tokens are binding (scrub rounded/blur utilities) - ~~VPS provisioning starts in Phase 4 (Photon bake-off)~~ SUPERSEDED by Phase 4 D-03: search is an in-Worker curated gazetteer ($0); VPS provisioning (Valhalla + Photon) defers to Phase 5; Photon is a documented SearchSource backend #3 (free-first mandate: one VPS, Oracle Cloud Always Free candidate) @@ -91,6 +92,6 @@ None yet. ## Session Continuity -Last session: 2026-10-25T12:09:32.000Z -Stopped at: "08-05 executed (D-03 gate green after the fromTable fix; CPU verdict honest FAIL 152.788/185.692 ms; ladder stop at L=32; c3 remains — escalation open); NEXT: maintainer R2 upload + the SC-1 escalation decision" -Resume file: .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-05-SUMMARY.md +Last session: 2026-10-25T14:15:06.939Z +Stopped at: "F1 remediation applied (debug alt-potential-4x-slowdown RC-1): clean-columns-only ALT containment; oracle sweep 0/200; CPU in the pre-ALT band (c3 stands); D-03 green — gates re-run 2026-10-25" +Resume file: None diff --git a/.planning/debug/alt-potential-4x-slowdown.md b/.planning/debug/alt-potential-4x-slowdown.md new file mode 100644 index 0000000..dc16502 --- /dev/null +++ b/.planning/debug/alt-potential-4x-slowdown.md @@ -0,0 +1,70 @@ +--- +status: resolved_f1_applied +trigger: "ALT triangle-inequality potentials made bidirectional A* ~4x SLOWER on real KL graph (p95 38ms -> 152-185ms); L=32 worse than L=24 — implementation-cost smell, not theoretical wall. Diagnose + propose fix inside potential()/per-search setup only." +created: 2026-10-25T00:00:00Z +updated: 2026-10-25T14:12:00Z +--- + +## Current Focus + +resolution: F1 APPLIED (post-plan remediation session) — cleanColumnsFor per-graph memo + search-scoped activeL (clean columns only); per-node sentinel skip REMOVED; RC-1 closed. Verification: routing 81/81 (A11 re-spec'd RED-first, A13 new mixed-column case), bot 126/126, typechecks clean, new committed oracle sweep 0/200 suboptimal (seed 0x08f1), CPU back in the pre-ALT band (p95 136.6 ms this host vs 1037.6 pre-fix; honest c3 FAIL vs 8 ms stands), D-03 gate green (0 violations). Commits c071e20 / 2a5a0b5 / 97d6e35. F2 (transposed table) remains FUTURE — only relevant when a clean-column binary is rebuilt; ALT is inert (exact haversine) on the current 0/24-clean binary by design. +test: complete — re-specified suite is the standing seal (A9 all-dirty, A10 search-scope skip, A11 F1 structural pins, A12 all-clean ALT-active, A13 subset active list) + measure-astar-oracle-sweep.mjs is the committed fresh-pair optimality pin D-03's fixed corpus structurally cannot be +expecting: nothing — RC-1 resolved; RC-2 escalation (c3) unchanged and owned by the maintainer decision recorded in 08-05-SUMMARY +next_action: none (maintainer may reopen for F2 when a clean-column binary exists) + +## Symptoms + +expected: ALT bounds dominate haversine bound → fewer expansions → faster or equal CPU (gate p95 budget 8 ms) +actual: p95 152.788 ms (25-route gate), 185.692 ms (100-pair); pre-ALT baseline p95 ≈ 38 ms; L=32 probe WORSE (182–204 ms) +errors: none (correctness green — A1 625-pair equivalence + D-03 optimality gate pass) +reproduction: routing/scripts/measure-route-cpu.mjs over real kl-roadgraph.v1.bin (311k nodes, L=24, coverage 0.980) +started: after Phase 8-03 integration of ALT steering into astar.ts + +## Eliminated + +- hypothesis: "ALT reduces expansions; the slowdown is purely per-node cost" — evidence: bench2 counters show expansions INCREASE +15.7% with ALT (66,286 -> 76,671/route; every routed pair; +51% on mid-valley->bukit-bintang); unidirectional control shows the SAME tables PRUNE (0.59x) — the increase is specific to the shared-average BH +- hypothesis: "per-search setup fills cause the regression" — evidence: git ea9584d shows hfCache/hbCache/pf/big arrays existed pre-ALT; measured fills 1.7 ms/search; goalToU/origToU are Float64Array(24) — trivial +- hypothesis: "NaN-sentinel float compare path is float-heavy" — evidence: violation dump shows sentinel handling is a CORRECTNESS problem (family-set variance), not a cost problem; branches are predictable +- hypothesis: "L=32 worse because of table size alone" — evidence: per-node read count scales with L (2×32 vs 2×24 scattered DRAM reads) — consistent with +20% time at L=32; secondary to RC-1/RC-2 + +## Evidence + +- timestamp: 2026-10-25T21:05Z + checked: git ea9584d (pre-ALT astar.ts) vs current + found: pre-ALT potential() = same hfCache/hbCache/pf structure, bound body = pure metersTo; ONLY delta is the ALT L-scan in bound() + implication: the entire regression lives inside bound() + its per-search constant tables +- timestamp: 2026-10-25T21:10Z + checked: real gate measure-route-cpu.mjs run today on this host + found: p95 1037.618 ms (campaign record: 152.788) — host ~7x slower than campaign conditions; bench 'shipped' variant 1124 ms = within 8% of real gate today + implication: bench faithful; use ratios, quote environment caveat +- timestamp: 2026-10-25T21:20Z + checked: bench2 (clean per-variant functions, 25-route seeded set, best-of-3) + found: hav p50 37.6/p95 363.2/mean 121.2, exp 66,286 | shipped 123.2/899.4/332.7, exp 76,671 (+15.7%) | transposed+merged+pfCache 72.3/546.5/197.3 | no parity failures (bench seconds == shipped seconds all pairs) + implication: costlier nodes (~2.5 µs/new-node scan) + more nodes (+15.7%) both real; transposed recovers 41% but still +63% over hav +- timestamp: 2026-10-25T21:25Z + checked: unidirectional A* probes (hav vs max(hav,ALT)) + found: uni expansions alt/hav = 0.59 (54,447 -> 31,927 mean); ratio < 1 on 21/23 pairs + implication: tables + bounds genuinely prune UNIDIRECTIONALLY; the BH (h_f-h_b)/2 average is what flips the sign +- timestamp: 2026-10-25T21:35Z + checked: bench3 alpha sweep (max(hav, a*ALT), transposed, real seconds vs shipped) + found: a=0.25/0.5 returned +3.48 s suboptimal on sunway-pyramid-to-pavilion-kl (1120.89 vs 1117.41); a=0 and a=1 clean on these 25 pairs + implication: intermediate scaling TRIGGERS the latent inconsistency stochastically — a soundness mechanism exists beyond the fromTable issue +- timestamp: 2026-10-25T21:45Z + checked: bench5/bench6 unit-correct per-edge consistency check + violation dump + found: a=0: 0/0 violations | a=0.25: 56 bwd (worst 66.3 s) | a=1 SHIPPED: 334 fwd (worst 1026 s) + 11,369 bwd (worst 774 s); all dumped violations have to[v,l*]=0xffff with to[u,l*] finite; table triangle check: 0 violations on all sampled landmarks + implication: per-node sentinel skip = per-node family-set change = the inconsistency; tables exact; code comment's own "no per-node l-mask" law is violated by the sentinel skip +- timestamp: 2026-10-25T21:50Z + checked: bench7 shipped-vs-Dijkstra-oracle on 200 fresh seeded cross-quadrant pairs + found: shipped 1/194 SUBOPTIMAL (+0.278%, pair 141683->292552: 1859.3 vs 1854.1); pure-hav control 0/194 + implication: RC-1 is a LIVE correctness bug, not just latent; D-03's fixed corpus is insufficient coverage +- timestamp: 2026-10-25T21:55Z + checked: bench8 landmark column census + found: 0/24 clean columns; each column ~6,285-6,292 sentinels (2.02%); 6,285 nodes reach NO landmark (one disconnected island fragment) + implication: search-scoped clean-column dropping degrades to exactly haversine on this binary (sound); ALT can only be live on a rebuilt binary whose anchors have sentinel-free columns — impossible while the island exists (08-05's no-filler proof generalizes) + +## Resolution + +root_cause: TWO defects share one surface (potential()/bound() in astar.ts). RC-1 CORRECTNESS: the per-node sentinel skip (`if (tv === 0xffff) continue`) varies the active landmark family set PER NODE — exactly the "per-node ℓ-mask" the code's own comment forbids — so the toTable bound is NOT cross-edge consistent: when node u has finite to[u,ℓ] and neighbor v has to[v,ℓ]=0xffff (v is in the 6,285-node island, 2.02%), h_f(u) stays at the ALT bound (up to 1,905 s) while h_f(v) degrades to haversine → measured 334 forward + 11,369 backward consistency violations on real edges (worst 1,026 s / 774 s); shipped routeAstar returns SUBOPTIMAL seconds on 1/194 fresh random pairs (+0.278% vs Dijkstra oracle, pair 141683->292552); pure-haversine control 0/194. Tables are triangle-exact (0 violations all-landmark sweep) — the skip logic is the bug. D-03 green = sampling luck (recorded pairs avoid the island). RC-2 PERFORMANCE (the 4×): per-new-node bound() = 2 passes x 24 scattered u16 reads (row-major [l*n+v], 622 KB stride) into the 14.9 MB table > 12 MB L3 → DRAM-bound ~2.5 µs/new-node x 77k new nodes ≈ +190-250 ms; PLUS expansions INCREASE +15.7% (all 23 routed pairs; +51% worst) — the shared-Ikeda average (h_f-h_b)/2 flattens the max-composite bound's directional guidance (unidirectional control: SAME bounds PRUNE 0.59x — tables good, BH-average config is the problem). Setup fills 1.7 ms negligible. +fix: F1 (soundness, in-constraint, reverseFor-precedent): per-graph WeakMap memo `reachableTo[l] = (to-column has zero 0xffff)` (O(L*n) once per isolate); per-search active-landmark list = clean columns only; bound() loops the active list with NO per-node sentinel branch (constant-table NaN goal/origin checks retained). On the current binary 0/24 columns are clean -> potentials degrade to exactly haversine -> sound (0/194 control), restores pre-ALT CPU band, A1 fixture (strongly connected) keeps ALT live so the logic stays tested. F2 (performance, for future clean-column binaries): transposed toTable [v*L+l] per-graph memo (14.9 MB one-time, 104-192 ms build) + merged single-pass (read tv once, update both bestG/bestO) + single pfCache Float64Array replacing hfCache/hbCache/pf — measured mean 332.7 -> 197.3 ms (-41%), bound cost 3.5 -> 0.31 µs/node, bit-identical potential values (verified: identical expansions + seconds on all 25 pairs). REQUIRED TEST WORK: A9's island-fixture assertions must be re-specified (island columns are dirty -> ALT drops search-scoped everywhere in that fixture); add a strongly-connected fixture for ALT-active coverage; add the bench7-style seeded random-pair Dijkstra-oracle sweep (200 pairs) as a real-graph optimality pin (this is the gate that would have caught RC-1; D-03's fixed corpus cannot). +verification: bench1/2/3/5/6/7/8 throwaway harnesses in /tmp/opencode/alt-bench/ (never committed); bench replica parity: shipped seconds matched on all 25 gate pairs, no PARITY FAIL lines; alpha-sweep consistency check (unit-correct) pins the violation mechanism to per-node sentinel skips; alpha=0.25/0.5 reproduced REAL suboptimality (+3.48 s on sunway-pyramid-to-pavilion-kl) proving the mechanism end-to-end; bench8 census: 0/24 clean columns, 6,285 no-reach nodes, 2.02% sentinel rate per column. Environment caveat: this host measured the real gate ~7x slower than the 08-05 campaign record (p95 1037.6 ms today vs 152.788 recorded) — all ratios self-consistent within-session; bench-shipped matched real-gate-today within 8%. +files_changed: [] diff --git a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-04-SUMMARY.md b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-04-SUMMARY.md index 7c62b27..94b8ac5 100644 --- a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-04-SUMMARY.md +++ b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-04-SUMMARY.md @@ -133,6 +133,14 @@ None - no external service configuration required. (The pending rebuild needs on *Phase: 08-alt-landmark-preprocessing-routing-backend-v2* *Completed: 2026-10-25 (task 1; task 2 pending Overpass availability)* +### Orchestrator addendum — task 2 COMPLETED (2026-10-25, same day) + +The maintainer-time rebuild succeeded once Overpass recovered (HTTP 502 was transient): +- `node routing/scripts/build-road-graph.mjs` → wrote `routing/data/kl-roadgraph.v1.bin` ARG2: 43.2 MB (45,341,620 B), 311,361 nodes / 677,126 edges / 21,469 names, maxSpeed 110 +- Landmark facts: L = 24 (D-02 rule, round(311361/13000) = 24), quantum 0.1 s, coverage min/mean 0.980/0.980 (gate ≥ 0.9), table bytes 28.5 MB, sentinels 150,986 to / 1,365,034 from +- `--check routing/data/kl-roadgraph.v1.bin` → exit 0, facts decoded from the binary +- The 08-05 BLOCKED note above is superseded: the ARG2 binary exists on the main tree — the campaign can run directly. R2 re-upload (`node routing/scripts/upload-road-graph.mjs`) remains a deploy-time user action, not an execution blocker. + ## Self-Check: PASSED - `routing/scripts/build-road-graph.mjs` exists with the stage (grep-verified counts above) — FOUND diff --git a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-05-SUMMARY.md b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-05-SUMMARY.md index 5e1b2a8..cc614dc 100644 --- a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-05-SUMMARY.md +++ b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-05-SUMMARY.md @@ -130,6 +130,41 @@ No new network endpoints, auth paths, file access patterns, or schema changes - `grep -c "P95_BUDGET_MS = 8" routing/scripts/measure-route-cpu.mjs` = 1; `grep -c "next-milestone" scripts/lib/budget-gates.mjs docs/BUDGETS.md` = 0+0; `nodes[0]|nodes.length` in the gate = 0 - The record file: untouched (verified by the gate's own mirror asserts + git) +## F1 Remediation Addendum (post-plan, 2026-10-25) + +**Debug session:** `.planning/debug/alt-potential-4x-slowdown.md` — RC-1 (LIVE correctness bug) + RC-2 (the 4× CPU slowdown). D-03's green above was sampling luck: the fixed corpus never crosses a violating edge harmfully, while fresh random pairs caught the shipped `routeAstar` suboptimal on **1/194** (+0.278%, pair 141683→292552) vs a Dijkstra oracle. Root cause: `bound()`'s per-node sentinel skip varied the active landmark family set PER NODE across edges leaving the 6,285-node disconnected island (2.02% of cells). + +**Fix (F1, exactly as the debug report specified):** `routing/src/astar.ts` +- Per-graph WeakMap memo `cleanColumnsFor` — `clean[l]` = "to-column ℓ has zero 0xFFFF cells" (O(L·n) sequential rows once per graph/isolate; `reverseFor` precedent). +- Per-search active list `activeL` = clean columns only; `bound()` loops the active list with NO per-node sentinel branch (constant-table goal/origin NaN checks retained as belt). +- On the current binary (0/24 clean columns — island census confirmed) potentials degrade to EXACTLY haversine: the designed sound fallback; the pre-ALT CPU band returns. +- ALT stays LIVE and tested: the committed mini-graph is 0/2 dirty columns and A12's ring is all-clean (both verified by table scan), so the ALT path keeps running in the suite. +- BH stop rule, reductions, balanced alternation, meet probes, real-seconds reconstruction: untouched. F2 (transposed table) explicitly out of scope — future clean-column binaries only. + +**Test work (re-specified per the debug report):** +- A9: island fixture re-pinned as ALL-DIRTY (every to-column holds a B-node sentinel → the whole fixture runs haversine); the old "real ALT bounds live in A" claim (per-node-skip semantics) removed. +- A10: mechanism comment updated to the containment law. +- A11: structural pin rewritten to the F1 shape — `cleanColumnsFor`/`activeL` must exist in code, inline `=== 0xffff` sentinel compares BANNED (RC-1's exact shape), named-constant law pinned. Committed RED-first: failed on shipped code exactly as designed. +- A12: re-sealed as the ALT-ACTIVE counterpart (all to-columns clean, pinned by table scan). +- A13 (new): mixed clean/dirty columns via a one-way-bottleneck twin-ring fixture — the subset-active-list branch; 64-pair Dijkstra equivalence + 16 honest nulls. + +**New committed gate:** `routing/scripts/measure-astar-oracle-sweep.mjs` — 200 COMPARED fresh seeded cross-metro pairs (seed `0x08f1`, NOT the D-03 corpus; `generateCrossMetroPairs` core reused; 3× oversample with snap-miss rejection printed; exported `sweepOracle` core + CLI guard; honest SKIP when the gitignored binary is absent). Gate: 0 suboptimal vs a brute-force label-setting Dijkstra oracle. + +**Verification (all re-run post-fix, this host):** + +| Gate | Result | +| ---- | ------ | +| `npm run routing:test` | 81 passed, 0 failed (A11 RED pins green, A13 new) | +| `npm test -w bot` | 126 passed, 0 failed | +| `routing:typecheck` + `bot:typecheck` | clean | +| Oracle sweep (new) | GATE PASS: 0/200 suboptimal (182 routed, 18 unreachable, worst Δ +0.000%) | +| `measure-route-cpu.mjs` | p95 136.562 ms / mean 54.081 ms — the pre-ALT band on this host (pre-fix today measured 1037.618 ms; ~7.6× back). Honest c3 record stands: still a FAIL vs the 8 ms budget | +| `measure-optimality-gate.mjs` (D-03) | GATE PASS: 0 violations, max \|Δ\| 0.005%, verdict classes preserved | + +**Commits:** `c071e20` (test RED), `2a5a0b5` (fix F1), `97d6e35` (oracle sweep). Bench/instrumentation harnesses remain in `/tmp/opencode/alt-bench/` (never committed). + +**Impact on the plan's verdicts:** the D-03 gate green above is now backed by a fresh-pair pin that structurally can't pass on sampling luck; the CPU verdict is unchanged in kind (c3 escalation stands — ALT is inert on this binary until a clean-column rebuild exists, and a rebuilt clean-column binary re-arms ALT with no code change). + ## User Setup Required None new. The maintainer-time R2 upload (`node routing/scripts/upload-road-graph.mjs` + the `--check` roundtrip) remains the deploy-time act — manual by design (account-level validation). **Decision needed:** pick the escalation track (direction-split potentials / CH / Workers Paid reversal / keep c3). diff --git a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VERIFICATION.md b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VERIFICATION.md new file mode 100644 index 0000000..511439c --- /dev/null +++ b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VERIFICATION.md @@ -0,0 +1,172 @@ +--- +phase: 08-alt-landmark-preprocessing-routing-backend-v2 +verified: 2026-10-25T14:43:22Z +status: gaps_found +score: 1/3 roadmap success criteria (SC-2 ✓; SC-1 ✗; SC-3 ✗ as written — correctly not taken). All 5 plans' machinery must-haves verified. +overrides_applied: 0 +re_verification: + previous_status: none + previous_score: n/a + gaps_closed: [] + gaps_remaining: + - "SC-1: longest KL-metro routes route within the free-plan CPU budget" + - "SC-3: the c3 degradation class retires" + regressions: [] +gaps: + - truth: "Longest KL-metro routes route within the free-plan CPU budget (p95 under the 8 ms ceiling, measured by the existing CPU gate) — SC-1" + status: failed + reason: "The measured verdict is a FAIL and stayed a FAIL through the pre-committed ladder: p95 152.788 ms (25-route CI gate) / 185.692 ms (D-04 extended 100-pair verdict) vs the 8 ms budget on the campaign host; this verifier's own re-run measured p95 107.493 ms — same band, same honest FAIL. The L=32 ladder rung measured worse (182.725 / 204.499 ms). Post-F1, ALT is INERT on the current binary by design: 0/24 landmark toTable columns are sentinel-clean (the 6,285-node island, 2.02% of cells), so potentials degrade to exactly haversine and the pre-ALT CPU band returned. The theoretically sound fast configuration (direction-split potentials π_f/π_r with a re-proven bidirectional meet rule) is search-skeleton work that Phase 8's own untouchable lines (stop rule, reductions, alternation, meet probes, reconstruction) forbid. Every number is quoted verbatim into docs/BUDGETS.md, docs/KNOWN-LIMITATIONS.md, docs/ROAD-GRAPH.md, and scripts/lib/budget-gates.mjs (drift guard 400 ms; the 8 ms budget unchanged, its FAIL still printed)." + artifacts: + - path: "routing/scripts/measure-route-cpu.mjs" + issue: "None — the gate is honest and healthy (P95_BUDGET_MS = 8 unchanged; prints GATE FAIL). The gap is the measured number, not the tool." + - path: "routing/src/astar.ts" + issue: "ALT potentials are sound-but-inert on this binary (clean-columns-only F1 law; 0/24 clean columns). Making them fast requires direction-split potentials — code outside this phase's sanctioned surface." + missing: + - "USER STRATEGIC DECISION (not a fix plan): pick the escalation track recorded in docs/BUDGETS.md §c3 and 08-05-SUMMARY — (1) direction-split potentials with a re-proven bidirectional meet rule (search-skeleton work; needs its own planned phase with the BH meet rule re-proven), (2) Contraction Hierarchies (the Phase-6 deferral, the Malaysia-scale tier), (3) Workers Paid (rejected by the $0 law — reversal needs explicit sign-off), or (4) keep c3 as the designed degradation (the current shipped posture: router-pending on long cross-metro reroutes). Option 4 is the only one requiring zero further code." + - truth: "The budget record's c3 degradation class retires; KNOWN-LIMITATIONS updated — SC-3" + status: failed + reason: "The retirement half of the criterion did NOT happen — correctly: the pre-committed law (retirement never fires on failing numbers) forbids it, and SC-1 failed. The record-update half IS complete and honest: KNOWN-LIMITATIONS' cross-metro section is the /cari-shaped post-ALT record (WHAT CHANGED / what REMAINS, quoted gate lines), its ALT roadmap-pointer row reads 'Built (Phase 8, 2026-10-25): optimality verified; the CPU verdict measured over budget', BUDGETS and ROAD-GRAPH carry the same quoted numbers, and 'next-milestone' wording is purged everywhere (grep = 0 across all four records)." + artifacts: + - path: "docs/KNOWN-LIMITATIONS.md" + issue: "None — the honest record is exactly what D-05's fail branch prescribes. Retirement is contingent on SC-1 and cannot honestly happen while it fails." + missing: + - "Contingent on the SC-1 decision above: only after a passing CPU verdict does a closure plan rewrite these records as a retirement." +deferred: [] +human_verification: + - test: "Decide the c3 escalation track (the gap above): direction-split potentials / CH / Workers Paid reversal / keep c3." + expected: "A maintainer decision recorded against docs/BUDGETS.md §c3; if 'keep c3', Phase 9+ proceed with the current honest-degradation posture; if a code track, it becomes a planned phase of its own." + why_human: "Strategic trade-off between $0 law, scope, and schedule — not verifiable from the codebase." + - test: "Verify the R2 migration state: the shipped decode path is ARG2-ONLY (D-01), so if the R2 object kl-roadgraph.v1.bin still holds ARG1 bytes, the deployed Worker returns router-pending for EVERY route until the rebuilt binary is uploaded once. Run `node routing/scripts/upload-road-graph.mjs` then the `--check` roundtrip against R2 (maintainer-time, account-gated)." + expected: "The R2 object decodes as ARG2 (version 2, L=24, coverage 0.980) and production /api/route serves real routes again." + why_human: "R2 contents and deploy state are outside the repo; the upload is deliberately a manual, account-validated act." +--- + +# Phase 8: ALT Landmark Preprocessing (Routing Backend v2) — Verification Report + +**Phase Goal:** Cross-metro routes route in-budget on the free plan: ALT (A\*, Landmarks, Triangle-inequality) preprocessing over the CSR graph upgrades engine A's worst case while staying $0 — the c2 rung of the measured ladder, executed. +**Verified:** 2026-10-25T14:43:22Z +**Status:** gaps_found +**Re-verification:** No — initial verification + +## Goal Achievement — the honest verdict + +The phase's **machinery landed completely and at high quality**: ARG2 codec, landmark preprocessing, ALT steering, build-stage integration, the D-03 optimality gate, and every record rewritten honestly. The phase's **core measurable deliverable did not land**: the CPU verdict is a FAIL (p95 107–186 ms across hosts/modes vs the 8 ms budget), the pre-committed ladder ran to its documented stop, and the debug session proved the wall is structural (ALT inert on this binary; the fast configuration is provably unsound under the shared-average BH; the fix is out-of-constraint skeleton work). SC-2 (optimality) is the one success criterion achieved — and it was hard-won: the phase's own gates caught and fixed two real suboptimality bugs (Rule-1 fromTable, RC-1 per-node sentinel skip → F1). + +This is the honest-FAIL branch the plan pre-committed to, executed faithfully. The gap that remains is a **maintainer strategic decision**, not a defect a closure plan can silently fix. + +### Observable Truths (roadmap contract) + +| # | Truth | Status | Evidence | +| --- | --- | --- | --- | +| 1 | SC-1: Longest KL-metro routes route within the free-plan CPU budget (p95 under the ceiling, measured by the existing CPU gate) | ✗ FAILED | Campaign: `GATE FAIL: p95 152.788 ms exceeds the 8 ms budget (A1)` + extended `GATE FAIL: p95 185.692 ms` (n=53); L=32 probe worse (182.725 / 204.499). **This verifier re-ran the gate:** `GATE FAIL: p95 107.493 ms exceeds the 8 ms budget` — same honest band. Post-F1 ALT is inert on the current binary (0/24 clean toTable columns — the 6,285-node island), so the c2 rung bought no measured CPU improvement. | +| 2 | SC-2: Route optimality unaffected (ALT admissible — corpus re-run green with identical tolerance verdicts) | ✓ VERIFIED | **Re-ran both gates this session:** D-03 gate `GATE PASS: 53/53 routed pairs equal within ±0.1%, verdict classes preserved (100/100 record pairs; 6 unreachable + 41 unsnappable preserved, 0 re-snapped)`, 0 violations, max \|Δ\| 0.005%, epsilon mirror intact; oracle sweep `GATE PASS: 0/200 \|Δ\| violations` (182 routed, 18 unreachable, worst \|Δ\| +0.000%) on fresh seed-0x08f1 pairs the fixed corpus structurally cannot cover. routing:test 82/82 (incl. A1 625-pair Dijkstra-equivalence, A11/A12/A13 pins). | +| 3 | SC-3: The budget record's c3 degradation class retires; KNOWN-LIMITATIONS updated | ✗ FAILED (as written — correctly not taken) | Retirement never fires on failing numbers (the pre-committed law). The update half IS done: KNOWN-LIMITATIONS carries the quoted post-ALT record, the ALT row reads "Built (Phase 8, 2026-10-25): optimality verified; the CPU verdict measured over budget — the degradation stands"; "next-milestone" purged from all four records (grep = 0). | + +**Score:** 1/3 roadmap success criteria. (All five plans' machinery must-haves — ARG2 codec laws, landmark module, ALT containment, build-stage integration, gates + records — verified; see below.) + +**Score note:** SC-3's failure is *compliant* behavior, not neglect — the criterion is arithmetically unreachable while SC-1 fails, and faking it would have violated the phase's own honest-engineering law. It is nonetheless reported as not-achieved because the class did not retire. + +### Plan-Level Machinery (must_haves from the 5 PLANs — all verified) + +| Plan | Claimed machinery | Verified evidence | +| --- | --- | --- | +| 08-01 | Pre-ALT baseline frozen BEFORE the codec; ARG1 guard; --extended D-04 flag; default mode byte-compatible | `measure-optimality-baseline.mjs` refuses non-ARG1 binaries (version check at L153–155); `alt-baseline.json` is coordinate-pinned (origin/destination lat-lng, never node indices) with the epsilon block `{name: OPTIMALITY_EPSILON_PCT, pct: 0.1}`; `--extended` imports `generateCrossMetroPairs` from the baseline module (ONE pair-set definition, two consumers); `P95_BUDGET_MS = 8` grep = 1 | +| 08-02 | ARG2 codec, exact-size-before-allocation, ARG1→null, fixture regenerated through the ONE encode path, landmark.ts the single table authority | `graph.ts`: magic `0x41524732`, version 2, `landmarkCount/landmarkQuantumS/landmarkTo/landmarkFrom` fields, `buffer.byteLength !== lay.total → null` at L228 (before view allocation); `graph.test.mjs:139` consumes `mini-graph-arg1.bin` as the rejection vector; `mini-graph.bin` regenerated (1668→1884 B); `landmark.ts` exports `farthestPointAnchors/landmarkTables/landmarkCoverage` + `LANDMARK_QUANTUM_S/MAX_QUANTA/I32_WEIGHT_CEILING` (WR-04 fix present) | +| 08-03 | ALT inside the proven BH skeleton; clean-columns containment (F1); rollout locks | `astar.ts`: `cleanColumnsFor` (L233) + search-scoped `activeL` (L358–362), `LANDMARK_SENTINEL_U16` named constant, haversine max-fallback; **`landmarkFrom` appears ZERO times in astar.ts** (the A11 law — stronger than required); `rollout.test.mjs` RO1/RO2 structural pins ran green in the suite; worker-router.ts and upload-road-graph.mjs contain no version/format branches (grep = 0) | +| 08-04 | Build preprocessing stage via the ONE landmark module; D-02 scaling rule; honest gates; --check | `build-road-graph.mjs`: imports the landmark module (never re-implements Dijkstra), `NODES_PER_LANDMARK = 13000` → clamp(round(N/13000),16,32), `LANDMARK_MIN_COVERAGE_FRACTION = 0.9` override-proof, `--landmarks` knob (cap 128); **live `--check` on the real binary:** nodeCount 311,361 / edgeCount 677,126 / landmarkCount 24 / quantum 0.100 / coverage min/mean 0.980/0.980 — the canonical L=24 build, upload-ready | +| 08-05 | D-03 gate with constant-mirror; honest campaign; records tell the post-ALT truth | Gate asserts `record.epsilon.name === "OPTIMALITY_EPSILON_PCT" && pct === 0.1` BEFORE measuring (L258); reads the frozen record, never regenerates pairs; budget-gates drift guard `ROUTE_CPU_DRIFT_GUARD_MS = 400` with the quoted post-ALT band; BUDGETS/KNOWN-LIMITATIONS/ROAD-GRAPH all quote the same gate lines (53 routed / 152.788 / 185.692 / 182.725 / 204.499) — mutually consistent, verified against the REVIEW.md cross-check | + +### Required Artifacts + +| Artifact | Expected | Status | Details | +| -------- | -------- | ------ | ------- | +| `routing/src/graph.ts` | ARG2 single-authority codec | ✓ VERIFIED | Substantive + wired (decoded the real binary live via `--check`; consumed by bot tests 126/126) | +| `routing/src/landmark.ts` | ONE landmark compute core | ✓ VERIFIED | All three exports + WR-04 i32 ceiling guard; consumed by build script + fixture builder + tests | +| `routing/src/astar.ts` | ALT potentials, F1 containment | ✓ VERIFIED | cleanColumnsFor/activeL present; landmarkFrom absent; toTable-only (sound subset) | +| `routing/scripts/measure-optimality-baseline.mjs` | Baseline recorder + shared pair core | ✓ VERIFIED | ARG1-version guard; exports consumed by measure-route-cpu | +| `routing/test/fixtures/alt-baseline.json` | Frozen pre-ALT record | ✓ VERIFIED | 29.6 KB, coordinate-pinned, epsilon block; untouched (gate mirror proves per run) | +| `routing/test/fixtures/mini-graph-arg1.bin` | Preserved ARG1 rejection vector | ✓ VERIFIED | 1668 B, byte-preserved; consumed by graph.test.mjs:139 | +| `routing/test/fixtures/mini-graph.bin` | Regenerated ARG2 fixture | ✓ VERIFIED | 1884 B; suite-wide consumption green | +| `routing/scripts/measure-route-cpu.mjs` | CPU gate + --extended | ✓ VERIFIED | Budget constant untouched; extended mode shares the pair core | +| `routing/scripts/measure-optimality-gate.mjs` | D-03 gate | ✓ VERIFIED | Mirror assert; GATE PASS re-run this session | +| `routing/scripts/measure-astar-oracle-sweep.mjs` | 200-pair fresh oracle sweep | ✓ VERIFIED | WR-01 symmetric \|Δ\| fix present (L244); GATE PASS re-run this session | +| `routing/scripts/build-road-graph.mjs` | Preprocessing stage + knobs | ✓ VERIFIED | Scaling rule, coverage gate, --landmarks, --check landmark facts | +| `routing/test/rollout.test.mjs` | D-01 rollout locks | ✓ VERIFIED | RO1/RO2 pins green in suite; targets verified format-unaware | +| `scripts/lib/budget-gates.mjs` | Re-anchored route-cpu law | ✓ VERIFIED | Drift guard 400; quoted c3 record; no next-milestone | +| `docs/BUDGETS.md` / `docs/KNOWN-LIMITATIONS.md` / `docs/ROAD-GRAPH.md` | Post-ALT honest records | ✓ VERIFIED | Quoted numbers mutually consistent; ALT row flipped; ARG2 layout section present | +| `routing/data/kl-roadgraph.v1.bin` | Canonical L=24 ARG2 binary (gitignored) | ✓ VERIFIED | On disk, 45.3 MB; decodes ARG2 with L=24/quantum 0.1/coverage 0.980 | + +### Key Link Verification + +| From | To | Via | Status | Details | +| ---- | -- | --- | ------ | ------- | +| astar.ts potential() | ARG2 landmark tables | `graph.landmarkTo` reads via cleanColumnsFor/activeL | WIRED | L233–242, L358–402 | +| measure-route-cpu --extended | measure-optimality-baseline | `import { generateCrossMetroPairs }` | WIRED | L60, L110 — one pair set, two consumers | +| measure-optimality-gate | alt-baseline.json | reads frozen record + epsilon mirror | WIRED | RECORD_PATH L53; mirror assert L258 fires before measurement | +| build-road-graph preprocessing | landmark.ts | dynamic import of the ONE module | WIRED | L202; no second Dijkstra exists (grep across scripts = 0) | +| graph.test.mjs | mini-graph-arg1.bin | ARG1 decode-rejection vector | WIRED | L139 | +| rollout.test.mjs | upload script + worker-router | structural grep pins | WIRED | RO1 pins the R2 key; RO2 pins no version branch — both targets verified clean | +| mini-graph.bin / bot test builders | ARG2 encode path | same encodeGraph authority | WIRED | bot 126/126 green after fixture regen | + +### Data-Flow Trace (Level 4) + +Not applicable to dynamic-UI artifacts (this phase ships no UI). The data-flow analog — real graph bytes → decode → search → gates — was exercised end-to-end this session: the 45.3 MB binary decodes (311,361 nodes / 677,126 edges / L=24), routes (23/25 routed in the CPU sample), and feeds both optimality gates with real measured seconds (D-03: 53 pairs; oracle: 182 routed pairs). No static/hollow data paths found. + +### Behavioral Spot-Checks (all run by this verifier, this host, 2026-10-25) + +| Behavior | Command | Result | Status | +| -------- | ------- | ------ | ------ | +| Routing suite (A1–A13, G1–G11, L, S, RO suites) | `npm run routing:test` | 82 passed, 0 failed | ✓ PASS | +| Bot suite (ARG2 fixture consumers) | `npm test -w bot` | 126 passed, 0 failed | ✓ PASS | +| D-03 optimality gate (SC-2) | `node routing/scripts/measure-optimality-gate.mjs` | `GATE PASS`, exit 0 — 0 violations, max \|Δ\| 0.005% | ✓ PASS | +| 200-pair fresh oracle sweep (SC-2 pin) | `node routing/scripts/measure-astar-oracle-sweep.mjs` | `GATE PASS: 0/200 \|Δ\| violations` (182 routed, worst +0.000%) | ✓ PASS | +| CPU gate (SC-1) | `node routing/scripts/measure-route-cpu.mjs` | `GATE FAIL: p95 107.493 ms exceeds the 8 ms budget` | ✗ FAIL (honest — the SC-1 gap) | +| Binary identity + landmark facts | `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` | ARG2; L=24; quantum 0.100; coverage 0.980; sentinels 150,986 to / 1,365,034 from | ✓ PASS | +| Budget-audit vectors | `node scripts/test/budget-audit.test.mjs` | `19/19 vectors ok` | ✓ PASS | + +(Overpass build NOT re-run per instructions; the on-disk binary is the canonical L=24 build.) + +### Requirements Coverage + +| Requirement | Source Plan | Description | Status | Evidence | +| ----------- | ---------- | ----------- | ------ | -------- | +| SC3 (carry-over) | 08-01..05 frontmatter | Phase-5 criterion: fixture backend ladder + designed router-pending state | ✓ SATISFIED (no regression) | routing 82/82, bot 126/126; rollout pins hold; router-pending law intact | +| SC4 (carry-over) | 08-01..05 frontmatter | Phase-5 routing-class carry-over | ✓ SATISFIED (no regression) | Same suite evidence | +| Phase-8 requirement ("the budget record's numbers improve") | ROADMAP §Phase 8 | No new REQ ids; the c3 numbers improve | ✗ BLOCKED | The measured numbers did NOT improve (p95 107–186 ms vs the pre-ALT ~38 ms campaign band); the record honestly documents the non-improvement. Covered by the SC-1 gap above. | +| Orphaned requirements | — | none | — | REQUIREMENTS.md maps no additional ids to Phase 8 (grep: SC3/SC4 appear only as ROADMAP-level carry-overs) | + +### Anti-Patterns Found + +| File | Line | Pattern | Severity | Impact | +| ---- | ---- | ------- | -------- | ------ | +| — | — | No TODO/FIXME/placeholder/stub patterns in any phase file (grep clean across all 7 scripts + 3 src modules) | — | — | +| routing/src/astar.ts | 315–316, 515, 598 | `return null` / `return []` | ℹ️ Info | Fail-honest designed degradation (input validation, unreachable miss, empty path) — not stubs | +| measure-*.mjs | various | `= []` accumulators | ℹ️ Info | Local accumulator init, not hardcoded empty data | + +Open code-review findings carried for the record (documented in 08-REVIEW.md, not re-litigated here): **WR-03** (baseline recorder lacks oversampling — the frozen evidence base is 53 measured pairs, not ~100; maintainer-deferred, record immutable) and **IN-01..IN-07** (doc-statement drift, defensive gaps in future-record paths). WR-01/02/04 verified applied: symmetric |Δ| compare at oracle L244, CPU3 honest-SKIP law, `LANDMARK_I32_WEIGHT_CEILING` guard at landmark.ts L131 — each with its fix commit (11e55c1, 95399ae, e77bbe9) present in history. + +### Human Verification Required + +1. **The c3 escalation decision (the gap itself — belongs to the USER).** + **Test:** Choose the track: (1) direction-split potentials with a re-proven bidirectional meet rule (needs its own phase — skeleton work Phase 8's laws forbade), (2) Contraction Hierarchies (Phase-6 deferral), (3) Workers Paid reversal (needs explicit sign-off against the $0 law), (4) keep c3 as the designed degradation (current shipped posture; zero code). + **Expected:** A recorded decision against docs/BUDGETS.md §c3 that unblocks either Phase 9 (option 4) or a new planned phase (options 1–2). + **Why human:** Strategic trade-off across cost law, scope, and schedule — not decidable from code. +2. **R2 migration state (D-01's one-time upload).** + **Test:** `node routing/scripts/upload-road-graph.mjs` + `--check` roundtrip against R2 (maintainer-time, account-gated). + **Expected:** The R2 object `kl-roadgraph.v1.bin` holds the ARG2 build (v2, L=24, coverage 0.980); production `/api/route` serves real routes. Until then, any deploy of the current Worker code is router-pending on every route (ARG1 bytes decode to null by design). + **Why human:** R2 contents and deploy timing are outside the repo; the upload is deliberately manual. + +### Gaps Summary + +One root cause, two failed criteria. **The CPU win — the phase's core deliverable — did not land** (SC-1), and therefore **the c3 retirement correctly did not fire** (SC-3). What DID land, verified end-to-end this session: the ARG2 layout (single decode path, fail-honest, exact-size law), the landmark machinery (module, build-stage integration, D-02 scaling rule, honest coverage/quantum gates, the canonical L=24 binary), optimality hardening beyond the plan's bar (D-03 gate + 200-pair fresh oracle sweep, both green — after the phase's own gates caught two real suboptimality bugs), and four records rewritten to the quoted post-ALT truth with zero retirement theater. + +The remaining work is not a bug fix: the debug session proved a perfectly implemented two-sided ALT cannot beat haversine in this bidirectional configuration on a graph with sentinel fringes (0/24 clean columns), and the sound fast path (direction-split potentials) requires re-proving the BH meet rule — work this phase's untouchable lines forbade. **This is the escalation decision recorded in docs/BUDGETS.md §c3 and 08-05-SUMMARY, and it belongs to the maintainer**, not to a closure plan. No later milestone phase covers it: Phase 9 builds on the current posture, Phase 10 is the battery record, and Phase 11 (Engine B) moves routing client-side without promising the Worker path in-budget — so nothing here is deferrable. + +--- + +_Verified: 2026-10-25T14:43:22Z_ +_Verifier: OpenCode (gsd-verifier)_ + +### Decision addendum (2026-10-25, maintainer) + +The SC-1 escalation decision was made: **Contraction Hierarchies, as Phase 08.1** (inserted, "Routing Backend v3") — the only lever the measured evidence supports reaching the 8 ms budget. Direction-split potentials are NOT closed: the measured prizes (0.59× expansions unidirectional control; −41% transposed table) and the full proof-obligation surface are handed off in `docs/ALT-DIRECTION-SPLIT.md` as a standalone experiment (explicitly open to being worked with a different LLM/harness). SC-1 and SC-3 transfer to Phase 08.1; this phase's debt is the honest record above plus that handoff. SC-2's hardening stands as the phase's optimality proof. diff --git a/docs/ALT-DIRECTION-SPLIT.md b/docs/ALT-DIRECTION-SPLIT.md new file mode 100644 index 0000000..34b7cfd --- /dev/null +++ b/docs/ALT-DIRECTION-SPLIT.md @@ -0,0 +1,72 @@ +# ALT Direction-Split Potentials — Research Handoff + +**Status:** OPEN EXPERIMENT — deliberately deferred from Phase 8 (2026-10-25) +**Purpose:** a self-contained document so the direction-split experiment can be picked up — by us, a fork, or a different LLM/harness — and solved *straight away* without re-deriving any of this. +**Author context:** distilled from Phase 8's campaign, debug session `.planning/debug/alt-potential-4x-slowdown.md`, and the verification record. All numbers below are MEASURED on this repo's gates/benches, not estimated. + +--- + +## 1. Problem statement + +The routing engine (`routing/src/`) runs **bidirectional A\* (BH, Ikeda shared-average construction)** over a CSR road graph. ALT landmark potentials were integrated (Phase 8) and made *sound* via the F1 clean-columns law, but on the real KL binary **0/24 landmark columns are sentinel-clean** (a 6,285-node disconnected island makes every to-column carry a `0xffff` cell), so sound ALT degrades to haversine in production. + +The **direction-split question**: can the potential use BOTH table families soundly — `h_f` from to-tables (forward search), `h_b` from from-tables (backward search) — with a re-proven meet rule, and does that reach the free-plan CPU budget? + +## 2. What is already proven / shipped (do not re-derive) + +| Fact | Evidence | +|---|---| +| toTable triangle bounds are one-directional Lipschitz: the u-side term needs the reverse edge; a oneway doesn't provide one. Two-sided mixing inside the shared Ikeda average `p = (h_f − h_b)/2` is INCONSISTENT on graphs with oneways. | D-03 gate's Rule-1 catch: suboptimal routes x038 (1234.6 s vs 1214.8 s oracle), x045 (1475.5 vs 1460.6) | +| F1 containment (SHIPPED, commit `2a5a0b5`): per-graph WeakMap `cleanColumnsFor`; per-search `activeL` = clean columns only; `bound()` has NO per-node sentinel branch (inline `=== 0xffff` compares are BANNED — A11 pins this). Consistency violations went 334 fwd / 11,369 bwd → 0. | routing 82/82; oracle sweep 0/200 | +| Tables are triangle-exact (0 violations in an all-edges unit-correct sweep) — the bug was always the per-node family switching, never the data. | debug session bench5 | +| The tables PRUNE WELL: unidirectional control with these tables = **0.59× expansions** of haversine-only. The data is good; the BH configuration eats it. | debug bench (uni control) | +| The BH shared-Ikeda average FLATTENS the composite bound: expansions INCREASED +15.7% with ALT (66,286 → 76,671/route; +51% on mid-valley→bukit-bintang) despite the bound being pointwise-tighter on 99.7% of nodes. | debug bench7-class runs | +| Costlier nodes, not fewer nodes, dominated the regression: each `bound()` = 2 passes × 24 scattered u16 reads, row-major `[ℓ·n+v]` (622 KB stride) into a 14.9 MB table > 12 MB L3 → ~2.5 µs/node (vs 0.31 µs transposed, 0.26 µs 2× haversine). ~85% of the delta. L=32 worse than L=24 because reads scale with L. | debug microbenches | +| F2 (transposed toTable `[v·L+l]` + merged single-pass): mean 332.7 → **197.3 ms (−41%)**, bit-identical potentials, NOT shipped (out of scope — only useful for future clean-column binaries). | debug bench (F2) | +| Pre-ALT band: bidirectional A\* + haversine, p95 ≈ 38 ms on the campaign host (~107–152 ms on slower hosts — host-relative; gate = p95 8 ms budget, free-plan Worker ceiling 10 ms). | `routing/scripts/measure-route-cpu.mjs`, D-04 extended verdict | + +**Shipped posture today:** toTable-only steering (sound), clean-columns containment, haversine max-fallback, ALT live in the suite (A12/A13) and inert in production. Optimality: D-03 gate + 200-pair oracle sweep, both green. + +## 3. The theory gap — what must be proven + +ALT's correctness inside BH requires the potential pair to satisfy the Ikeda condition (the average `(h_f − h_b)/2` is consistent iff both families are *two-sided* consistent). With sentinel fringes, to-tables and from-tables each carry holes in DIFFERENT places, so any two-sided combination needs either: + +- **P-1 (per-direction consistency):** `h_f` built ONLY from to-tables, `h_b` ONLY from from-tables, each individually consistent across every relaxable edge. Sentinels inside ONE direction must degrade that direction's bound at that node (to haversine max-fallback) WITHOUT changing the active family set mid-search (the F1 lesson — per-node ℓ-masking is banned; the skip must be search-scoped per direction). +- **P-2 (the meet rule, THE hard part):** BH's stop rule (`μ + top_f + top_b ≥ bound`) is proven for symmetric-consistent potentials. Direction-split potentials break the symmetry — re-prove (or re-derive) a sound stopping criterion for the asymmetric case, OR prove the standard condition still holds under the split. Get this wrong and routes are silently suboptimal (the exact bug class D-03 was built to catch). +- **P-3 (optimality, empirical):** the 200-pair seeded Dijkstra-oracle sweep (`routing/scripts/measure-astar-oracle-sweep.mjs`, seed `0x08f1`, |Δ| both directions) must show 0 violations; A1's 625-pair fixture equivalence stays green. +- **P-4 (steering, empirical):** expansions with direction-split ALT < expansions haversine-only, on the extended 100-pair seeded verdict — this is where the 0.59× uni prize must survive the bidirectional setting. + +## 4. Constraints (non-negotiable) + +- Optimality non-negotiable: admissibility/consistency proofs precede any speed claim +- $0 runtime (free-plan Worker); JS typed arrays; pure modules (no I/O in routing/src) +- The ARG2 layout may gain fields (ARG3) if needed — but one decode path, fail-honest, exact-size law (D-01 family) +- Named constants; print-the-measurement gates; honest FAIL records (no retirement theater) +- The phase-8 "untouchable lines" (stop rule/reductions/alternation) are untouchable WITHIN phase 8 — a direction-split experiment's whole point is re-proving them; that work belongs in its own phase with its own gates + +## 5. Candidate approaches to experiment with + +- **A. Direction-split + meet-rule re-proof** — the mainline (Section 3). Highest prize, highest proof burden. +- **B. Unidirectional ALT** — drop BH for the ALT path: uni control already measured 0.59× expansions; simpler correctness (no meet rule); cost: lose BH's radius-halving. Measure both bands. +- **C. BH with per-direction haversine + ALT only as a unidirectional pruning filter** — hybrid: prune the unidirectional side's frontier with table bounds while BH runs haversine. Correctness via dominance, not potentials. +- **D. Min-combination variants** — the Ikeda average is one of several consistent combinations; `min(h_f-bound, h_b-bound)` asymmetric forms have literature precedent (need the consistency proof for the chosen form). +- **E. Different L / quantum / column-hygiene binary** — rebuild with anchors chosen so more columns are sentinel-clean (e.g., anchors constrained to the giant SCC's "well-connected side") — changes the DATA to fit the algorithm instead of the reverse. + +## 6. Verification assets (all exist, all re-runnable) + +- `routing/scripts/measure-astar-oracle-sweep.mjs` — 200-pair fresh oracle (the gate that catches suboptimality fixtures can't) +- `routing/scripts/measure-optimality-gate.mjs` + frozen `routing/test/fixtures/alt-baseline.json` — D-03 +- `routing/scripts/measure-route-cpu.mjs` (+ `--extended`) — the CPU verdicts +- `routing/test/astar.test.mjs` — A1 (625-pair equivalence), A9–A13 (containment/island/structural) +- The debug session's bench harnesses were throwaway (/tmp) — re-derive per Section 2's calibration method (within-session ratios only; hosts differ ~7×) + +## 7. Success criteria for the experiment + +1. A written proof (or literature citation + adaptation argument) for P-1 and P-2 BEFORE implementation +2. Oracle sweep 0/200 + A1 green + full battery +3. Expanded-measurement: expansions ratio AND p95 vs haversine-only, same seeded pairs +4. Honest verdict: p95 under the 8 ms budget on the free-plan host class → a real phase proposal; documented shortfall → the wall is proven, CH remains the lever, this document gets its closure record + +--- + +*Research handoff written: 2026-10-25 (Phase 8 close; lever decision: CH as Phase 08.1)*