Files

51 KiB
Raw Permalink Blame History

Phase 8: ALT Landmark Preprocessing (Routing Backend v2) - Research

Researched: 2026-10-25 Domain: ALT (A*, Landmarks, Triangle inequality) preprocessing over a hand-rolled typed-array CSR graph; binary layout versioning (ARG1 → ARG2); Cloudflare Workers free-plan CPU budget Confidence: HIGH (core algorithm + integration verified against code + literature; build-runtime figures derived from measured numbers and flagged)

<user_constraints>

User Constraints (from CONTEXT.md — LOCKED, do not re-litigate)

D-01 — ARG2-only, fail-honest

Graph layout version bumps to a new version carrying landmark tables. Old ARG1 binaries decode to null → the Phase 6 designed router-pending state covers them. Exactly one decode path and one search path — no v1 fallback branch, no silent quality regression. Upload script stays unchanged (roadmap lock); re-uploading the rebuilt binary is the whole migration (same R2 key, new bytes).

D-02 — Landmark count is a scaling rule, not a fixed constant

Selection approach is locked: build-time Dijkstra passes over well-spread anchor nodes (farthest-point / hill-climb family over the road graph); anchors are road-network nodes, NOT POIs. The count is measurement-justified and node-count-proportional (expected lean ~16–32 at KL's 311k nodes). A future Malaysia-wide build re-derives the count from the same rule; nothing over-provisioned for a graph that doesn't exist.

D-03 — Equal seconds, ties allowed (the optimality gate)

Corpus re-run asserts: total route seconds match the pre-ALT baseline exactly or within a named tiny epsilon (~±0.1%) and every pair keeps the same verdict class (reachable/unreachable). ALT changes steering, never the cost model. Bidirectional search + different potentials may legitimately change tie-breaking (a different equal-cost path) — tie-path differences are documented behavior, NOT failures. Bit-identical node paths is explicitly NOT the gate.

D-04 — Extended seeded maintainer-time sample for the ALT verdict

The existing 25-route seeded gate (measure-route-cpu.mjs, seed 0x06d4) stays as the CI-size snapshot; the ALT verdict additionally runs a maintainer-time extended seeded sample (~100 cross-metro worst-case pairs). Same print-the-measurement law. Maintainer-time measurement is free.

D-05 — Retire the c3 class with evidence

docs/KNOWN-LIMITATIONS.md's cross-metro CPU degradation section is replaced by a short retirement record: what changed (ALT preprocessing), the measured numbers that retired it (p95 under the ceiling, from D-04's run), and the date. The ALT row in the limitations' roadmap-pointer table flips from "Next-milestone item, not built" to built (Phase 8). Because of D-01 there is NO "fallback for landmark-less graphs" to document — an old graph is simply router-pending, which is already a designed, documented degradation.

OpenCode's Discretion (research must recommend inside these)

  • Landmark-selection algorithm specifics (farthest-point vs hill-climb vs hybrid), anchor count formula constants, quantization of stored distances within the scaling rule
  • Preprocessing build-time cost budget (each landmark = one full graph pass; build script runtime stays sanity-bounded)
  • The exact epsilon value and its name (named constant, replay-tested, per phase-6 constant law)
  • ARG2 layout byte design (header extensions aligned to the typed-array law; corrupt-binary fail-honest validation carried forward)
  • How the committed mini-graph fixture and test suite version-bump (new-ARG2 fixtures; old-v1 fixture kept only as a decode-rejection case)
  • KNOWN-LIMITATIONS retirement wording within the honest-engineering tone law

Deferred Ideas (OUT OF SCOPE — do not touch)

  • Contraction Hierarchies (Phase 6 deferral stands; the noted future tier for Malaysia-wide scale)
  • Engine B client-side router consuming the ARG2 asset (Phase 11)
  • Turn restrictions in the graph (Phase 6 deferral stands)
  • Any traffic/time-dependent weighting (v2 per REQUIREMENTS) </user_constraints>

Summary

This phase adds ALT preprocessing to the existing bidirectional A* (routeAstar in routing/src/astar.ts) without a new search skeleton, a new binary layout (ARG2) that appends quantized landmark distance tables to the pinned CSR layout, and a measurement campaign (build → check → upload → gate → corpus → docs) that retires the c3 degradation class. The decisive verified finding: the repo's BH potential structure is already the canonical bidirectional-ALT combination scheme. The route-planning survey (Bast, Delling, Goldberg, Müller-Hannemann, Pajor, Sanders, Wagner 2015) states that two arbitrary feasible potentials πf, πr are combined into consistent potentials by using (πf−πr)/2 forward and (πr−πf)/2 backward — which is literally the code's existing p_f(v) = (h_f(v) − h_b(v))/2, p_b = −p_f (Ikeda) form with its proven v-independent meet-sum constant. Consequently ALT slots in by replacing the two haversine/max-speed heuristics h_f, h_b with ALT triangle-inequality bounds — the only function that changes in routeAstar is potential(). The heap logic, balanced alternation, meet probes, stop rule, and real-seconds reconstruction all stay byte-for-byte identical, and A1's exhaustive 625-pair Dijkstra-equivalence sweep remains the standing optimality proof.

The second decisive finding is the quantization trick that makes u16 tables exactly correct with no epsilon fudge: instead of quantizing stored landmark distances and patching admissibility with a subtractive constant, the build quantizes the edge weights of a surrogate metric (ĉ(u,v) = floor(seconds/q) in fixed 0.1 s units) and runs the landmark Dijkstras over that integer metric. Distances within the ĉ metric are exactly 1-Lipschitz in ĉ, and ĉ ≤ c edge-wise, so ALT bounds computed inside the metric are (a) exactly admissible lower bounds on real seconds (d̂(v,ℓ) − d̂(t,ℓ) ≤ d̂(v,t) ≤ d(v,t) — the directed triangle via t) and (b) exactly 1-Lipschitz w.r.t. the real f32 edges (since |d̂(u,ℓ)−d̂(v,ℓ)| ≤ ĉ(u,v) ≤ c(u,v)). Both properties A* needs — admissibility and the feasibility/consistency that the balanced-potential construction and the BH stop rule require — hold by construction with integer arithmetic, no rounding correction, no epsilon. The U16 range check (KL's worst distance ≈ 5,400 s ≈ 54,000 units of 0.1 s < 65,535) closes the design.

Budget arithmetic (from the measured 311,325 nodes / 677,068 edges / 14.7 MB base): dual-family u16 tables cost L × 311,325 × 2 B × 2 ≈ 1.19 MB per landmark — L=16 → ~33.7 MB total, L=24 → ~43.2 MB, L=32 → ~52.7 MB. All fit the enforced 1–80 MB envelope with the advisory 30–80 MB band satisfied (the current below-band WARNING disappears at L≥16). Build-time preprocessing is ~2L Dijkstra passes over 677k edges ≈ 2 × 24 × ~40 ms ≈ 2 s of measured-order work (the 38 ms unidirectional A* p95 over the same graph bounds one full pass) — comfortably inside any sane build budget. Expected query behavior: the 20–30 km cross-metro class that currently measures p95 38–93 ms is exactly the regime where ALT's sharpened potentials cut settled nodes by roughly an order of magnitude; the pass verdict (p95 ≤ 8 ms) is expected but not guaranteed — the gate decides, and the D-02 scaling rule (raise L toward 32) is the pre-planned remediation knob before anything else.

Primary recommendation: Implement ALT as (1) an ARG2 layout appending two u16 landmark-table families (to-landmark + from-landmark) produced by integer ĉ-metric Dijkstras; (2) a potential()-only change in astar.ts that max-combines the four triangle-bound families per landmark with sentinel-skip and a haversine fallback; (3) a committed coordinate-identified baseline record measured before the code change, then the D-03 seconds-equality gate + D-04 extended 100-pair verdict + the existing 25-route p95 gate; (4) D-05's docs retirement. Do not touch upload-road-graph.mjs, worker-router.ts's API contract, the snap module, or the reverse-CSR/WeakMap patterns.

Architectural Responsibility Map

Capability Primary Tier Secondary Tier Rationale
Landmark selection + table Dijkstras Build scripts (routing/scripts/, Node, maintainer-time) — Preprocessing is build-time by design (D-02, context); no runtime preprocessing is even conceivable in the Worker (V8 isolate CPU budget)
ARG2 binary encode/decode routing/src/graph.ts (typed arrays, pure) — Single authority law: encode is inverse of decode, one module
ALT potentials at query time routing/src/astar.ts routeAstar — The search is the only consumer; probe lookups are typed-array reads inside the existing lazy-potential mechanism
Binary hosting + rollout Cloudflare R2 (same bucket/key) bot/src/routing/worker-router.ts R2 ROAD_GRAPH binding already deployed; Worker reads bytes → decodeGraph; zero API change
Rollout degradation (old binary present) Worker loadGraph → decode null → 200-null Mini App router-pending UI D-01 fail-honest posture already shipped and tested (RT1–RT11)
Measurement + optimality gates routing/scripts/measure-route-cpu.mjs + new baseline/gate tooling corpus scripts Maintainer-time repo tools, never runtime imports
Docs retirement record docs/KNOWN-LIMITATIONS.md, docs/ROAD-GRAPH.md docs/CONFIGURATION.md (verify touch) D-05 + fork contract surface

Standard Stack

Core

Component Version Purpose Why Standard
Node.js v24.x (nvm; 24.17/24.18 installed) [VERIFIED: shell] Build scripts + hand-rolled test runners Repo standard
routing/src/graph.ts codec current (ARG1 v1) The layout single authority this phase versions to ARG2 Encode/decode laws + zero-copy typed-array alignment laws already
routing/src/astar.ts routeAstar current (BH/Ikeda bidirectional) The search ALT plugs into via potential() Exhaustively proven optimal (A1, 625-pair sweep); meet-sum law
Zero new dependencies — Landmarks are arithmetic over existing typed arrays; Dijkstra runs inside build scripts Repo zero-dep law (Phase 6 added zero runtime deps)

Supporting (existing, no changes)

routing/scripts/build-road-graph.mjs (gains a preprocessing stage), routing/scripts/measure-route-cpu.mjs (gains the extended-sample flag + ladder-history rung), routing/test/fixtures/build-mini-graph.mjs (regenerates fixture as ARG2), routing/test/harness.mjs (registry unchanged).

Alternatives Considered (from Phase 6's A1 ratification — all waived, listed for completeness only)

Instead of ALT (c2) Option Why rejected / out of scope
— (c1) Workers Paid $5/mo Breaks the $0 mandate (A1 ratification)
— (c3) Accept degradation retired BY this phase (D-05)
— Contraction Hierarchies Deferred future tier (Phase 6 D-02; Malaysia-scale job per D-02 rationale)

Installation: none — zero dependencies. (Node via nvm present; verify node --version resumes on PATH inside the executing session's shell, v24.x installed.)

Architecture Patterns

System Architecture Diagram

                     ┌──────────────────────── MAINTAINER TIME ───────────────────────────┐
 Overpass extract ──▶│ build-road-graph.mjs                                               │
 (same query)        │  fetch → dedup → CSR edges  ──▶  NEW preprocessing stage:           │
                     │   1. quantize edges to integer ĉ (0.1 s units, floor)              │
                     │   2. select L anchors (farthest-point over ĉ, seeded/ID-tiebreak)  │
                     │   3. per landmark: reversed-graph ĉ-Dijkstra → TO table            │
                     │   4. per landmark: forward-graph  ĉ-Dijkstra → FROM table          │
                     │   5. SCC-coverage assertion per anchor (fail honest otherwise)     │
                     │   6. encodeGraph (ARG2) — header v2 + u16 table sections           │
                     │   output: routing/data/kl-roadgraph.v1.bin  (ARG2 bytes, ~43 MB)    │──┐
                     │   --check prints node/edge/landmark stats                          │  │
                     └────────────────────────────────────────────────────────────────────┘  │
                                                                                              ▼
                     ┌────────────────────── RUNTIME (Cloudflare Workers isolate) ─────────┐   upload (UNCHANGED
 bot /api/route ────▶│ worker-router.loadGraph: 1 R2 GET (same key) → decodeGraph         │◀── script, same key)
                     │   ARG2 header? → typed-array views (zero-copy), tables included    │
                     │   ARG1/corrupt/absent? → null → 200 null (router-pending, D-01)    │
                     │ routeAstar(graph, o, g):                                           │
                     │   per-search landmark constants (L lookups)                        │
                     │   lazy potential(v) = (h_f(v) − h_b(v))/2                          │
                     │     h_f(v) = max over valid ℓ of ALT toward-goal bounds            │
                     │        (fall back to haversine bound at sentinel nodes)            │
                     │     h_b(v) = same toward ORIGIN                                    │
                     │   …everything else in routeAstar UNTOUCHED…                        │
                     └────────────────────────────────────────────────────────────────────┘
                                                   output: route → deriveSteps → RouteResult (unchanged)

Primary use case trace: maintainer builds ARG2 → uploads same key → Worker cold isolate R2 GET → decodeGraph validates v2 + exact size → route request → snap → routeAstar with ALT potentials → real-seconds recompute → deriveSteps. Rollout path: stale ARG1 in R2 → decode null → router-pending (designed, tested) until one upload.

Component-file mapping (implementation side)

Piece Lives in
ARG2 codec (layoutFor, header, views, exact-size law) routing/src/graph.ts (extended in place — remains THE single authority)
ALT potential + table probes routing/src/astar.ts (routeAstar.potential() only)
Preprocessing stage (quantize, select, Dijkstras, coverage assert) routing/scripts/build-road-graph.mjs (new stage between edge assembly and encode)
Extended sample + verdict routing/scripts/measure-route-cpu.mjs (parameter/flag — "a parameter, not a rewrite" per 08-CONTEXT)
D-03 optimality gate + baseline record new maintainer-time script beside route-corpus.mjs (naming discretion), fixtures/record under routing/test/fixtures/ or routing/data/ as fits the repo's artifact policy
Fixture version bump routing/test/fixtures/build-mini-graph.mjs + committed mini-graph.bin (ARG2) + old bytes kept as the v1 rejection vector
D-05 records docs/KNOWN-LIMITATIONS.md, docs/ROAD-GRAPH.md, (check) docs/CONFIGURATION.md

Extension discipline: append, never mutate. All ARG1 sections keep their absolute offsets and sizes; the landmark section is a suffix; the only header region touched is the 4 reserved pad bytes plus 8 fresh bytes past 64.

ARG2 layout (little-endian):
  header (72 bytes, was 64):
    magic    u32 @0    = 0x41524732 ("ARG2")                    [VERIFIED pattern: graph.ts]
    version  u32 @4    = 2
    nodeCount u32 @8 | edgeCount u32 @12 | nameCount u32 @16 | namePoolLen u32 @20
    bbox 4×f64 @24..47 | maxSpeedKmh f32 @52..55          (unchanged)
    landmarkCount u32 @56 (was pad; ≥ 1 REQUIRED — see law below)
    landmarkQuantumS f32 @60 (recommended default 0.1 — the ĉ unit, seconds)
    reserved u32 @64..67 (= 0)
    reserved u32 @68..71 (= 0)
  …all ARG1 node/edge/name sections byte-identical to ARG1, HEADER_SIZE = 72
   (72 % 8 == 0 keeps the f64 lat/lng views 8-byte aligned — the alignment law holds)…
  landmark tables (suffix; TOTAL must extend the ARG1-implied size by exactly
   2 × landmarkCount × nodeCount × 2 bytes so the exact-size law extends):
    toTable   u16[landmarkCount × nodeCount]   rounded 4-byte-aligned start
             row ℓ = quantized ĉ-distance v→ℓ for every node v
             sentinel 0xFFFF = v cannot reach ℓ
    fromTable u16[landmarkCount × nodeCount]   (immediately after)
             row ℓ = quantized ĉ-distance ℓ→v
    landmarkNodes u32[landmarkCount]           rows' anchor node ids (audit + --check prints)
  total = 72 + ARG1 sections + 2·L·N·2 + 4·L (+ alignment pad to 4)

Laws carried forward into decodeGraph:

  • magic/version mismatch → null (D-01: version 1 → null; version ≥ 3 → null). The existing GRAPH_VERSION constant becomes 2; G8's pin updates.
  • Exact byte-size check BEFORE any view allocation (T-06-03b) — now including the landmark suffix; a hostile huge-landmarkCount header rejects on arithmetic.
  • landmarkCount === 0 → null: D-01's no-silent-degradation law extended to the content level. A landmark-less ARG2 would silently route with weak steering; the build always ships ≥ 1 anchor, so decode refusing 0 keeps "no fallback branch" true at every layer. [ASSUMED — this is my recommended interpretation of D-01's spirit; flag in plan review.]
  • Sentinel sanity: decode asserts every u16 table byte read is structurally possible (nothing to validate beyond exact-size + type law; 0xFFFF is data).

Byte-budget table (measured base 14,749,343 B = 14.7 MB):

L (anchors) u16 dual-family table bytes Total binary Envelope? Advisory band?
16 18.7 MB ~33.4 MB ✓ (≤80) ✓ (≥30 — CURRENT below-band WARNING disappears)
24 28.1 MB ~42.8 MB ✓ ✓
32 37.4 MB ~52.1 MB ✓ ✓
32 with f32 tables — ~90.7 MB ✗ over 80 MB —

f32 tables die at L=32; u16 is what makes the L=16–32 scaling rule byte-safe. Engine-B/Phase 11's "phone payload ~30–50 MB class" stays satisfied for L ≤ ~28.

u16/quantum range law: with quantum 0.1 s, tables hold distances ≤ 65,535 → max expressible ≈ 6,553 s. KL's worst-route seconds ≈ 5,400 s measured-class → ~54,000 units, comfortable. Build asserts no stored value and no intermediate Dijkstra label exceeds 65,000 units (fail honest, printed). For forks at larger scale: the header's landmarkQuantumS is the knob (raise to 0.2 s × 65,535 ≈ 3.6 h), documented in ROAD-GRAPH.md. [VERIFIED numbers from phase-6 build stats; quantum ceiling arithmetic VERIFIED trivially.]

Pattern 1: the ALT potential inside the existing BH (THE core integration)

What: routeAstar's potentials are currently h_f(v) = metersTo(v, goal)/maxSpeedMps, h_b(v) = metersTo(v, origin)/maxSpeedMps, combined as p(v) = (h_f(v) − h_b(v))/2 (astar.ts lines 232–251), with the doc-comment proof that the meet keys satisfy d_f(v) + d_b(v) = real(o→g through v) − p_f(o) + p_f(g). The survey verifies this class of construction is exactly how bidirectional A* takes arbitrary feasible potentials: "[one] can combine two arbitrary feasible functions πf and πr into consistent potentials by using (πf−πr)/2 for the forward search and (πr−πf)/2 for the backward search". So ALT shipping means: replace the bodies of h_f/h_b with ALT triangle bounds; the combination, reductions, meet-probes, stop rule, and reconstruction are untouched.

The four bound families per landmark ℓ (Bast et al. survey, §2.2 Goal-Directed Techniques — verbatim inequality set):

  • toward-goal family A: dist(v,t) ≥ dist(v,ℓ) − dist(t,ℓ) — uses toTable for both terms: per-node toTable[ℓ][v], per-search constant toTable[ℓ][t].
  • toward-goal family B: dist(v,t) ≥ dist(ℓ,t) − dist(ℓ,v) — uses fromTable both: fromTable[ℓ][t] const, fromTable[ℓ][v] per-node.
  • toward-origin family A: dist(v,o) ≥ dist(v,ℓ) − dist(o,ℓ) — toTable for both (per-node + const).
  • toward-origin family B: dist(v,o) ≥ dist(ℓ,o) − dist(ℓ,v) — fromTable for both.

Rich lower bound = h_f(v) = QUANTUM × max over ℓ of max(famA, famB) in units, converted to seconds; h_b(v) likewise toward origin. Both h_f and h_b are then (i) admissible lower bounds on real seconds and (ii) exactly consistent (1-Lipschitz) — see Pattern 2 for why (i) and (ii) hold exactly, and Pattern 3 for the sentinel-island caveat plus its containment.

Runtime shape (allocation-free, A8-law compatible):

// search start: L-length constant tables (Float64Array sec) + skip-sentinel masks
const gTo = new Float64Array(L), gFrom = new Float64Array(L); // per-search goal consts, NaN = sentinel
const oTo = new Float64Array(L), oFrom = new Float64Array(L); // origin consts
// per-node lazy evaluation, max-of-bounds loop over L; on "no valid ℓ" fall back to
// the (already-consistent) haversine bound computed as today.

This is the same lazy/cache shape the module already has (hfCache/hbCache Float64Arrays, NaN-init). Cost per node touch: L ≈ 16–32 iterations of 2 typed-array reads + arithmetic — comparable to today's two haversine trig chains; the win is that vastly FEWER nodes are touched. No Map, no per-node objects (A8 pin stays green).

Why this works with p_b = −p_f: the survey-scheme potentials πf = h_f, πr = h_b are each feasible (lower bounds toward their own target); (πf−πr)/2 and (πr−πf)/2 are consistent in exactly the code's sign convention (candidate = du + c + pfu − pv forward; du + c − pfu + pvv backward telescopes to d_b(v) = d(v,g) − p_f(v) + p_f(g)). The v-independent meet-sum follows; the stop topF + topB ≥ mu retains its optimality proof verbatim. [VERIFIED: survey formulation; code sign-analysis verified line-by-line against astar.ts during research.]

ALT beats the haversine bound exactly where it's weak — survey: "geographical distance divided by the maximum travel speed… corresponding bounds are poor, and the performance gain is small or non-existent." That is the measured Phase 6 failure mode (38 ms p95 despite BH). Recommendation (discretion): also max the ALT bound with the existing haversine bound per direction — max of two consistent admissible heuristics stays consistent and admissible, and it guarantees a sane bound at sentinel/edge nodes. [ASSUMED max-rule; standard result.]

Pattern 2: the ĉ-metric surrogate — quantization WITHOUT an admissibility epsilon

The classic u16 question ("does rounding break admissibility?") is resolved by never quantizing stored distances at all — quantize the metric, once:

  1. Build computes integer weights ĉ(u,v) = Math.floor(seconds(u,v) / q) with q = 0.1 s (0 ≤ ĉ·q ≤ c — floor keeps the surrogate BELOW live edge cost; ≤ q loss per edge, ≈ 0.1 s on 1–5 s edges ⇒ < ~2% weight error localized to short residential edges; NIL on arterials).
  2. Landmark Dijkstras run over ĉ in exact integer arithmetic (distances in q-units; JS numbers hold these exactly ≤ 65,535). No float round-up rule, no per-entry fudge.
  3. The two triangle families are evaluated within the ĉ metric: h_ℓ(v) = d̂(v,ℓ) − d̂(t,ℓ) ≤ d̂(v,t) ≤ d(v,t) — the first ≤ is the directed triangle (v→t ≥ (v→ℓ) − (t→ℓ), i.e. d(v,ℓ) ≤ d(v,t) + d(t,ℓ)), the second is ĉ ≤ c. EXACT lower bound with no constant subtraction. Every intermediate step is an exact statement about one integer metric; "difference of rounded distances" — the usual quantization leak — never occurs because a true metric is defined by the quantized weights.
  4. Consistency (the property the balanced-potential construction + stop rule need): |d̂(u,ℓ) − d̂(v,ℓ)| ≤ ĉ(u,v) ≤ c(u,v) — Dijkstra distances are Lipschitz-1 in their own graph's edges, and ĉ ≤ c makes the bound hold against REAL f32 edgeSeconds. So the potentials are feasible on every real edge, both directions, exactly.
  5. max over landmarks: |max_ℓ f_ℓ(u) − max_ℓ f_ℓ(v)| ≤ max_ℓ |f_ℓ(u) − f_ℓ(v)| ≤ c(u,v) — the max-combination stays exactly consistent (max of a fixed set of 1-Lipschitz functions is 1-Lipschitz).

Storage impact: distances same small integer domain as edge counts → u16 suffices; total bytes = 2 families × L × N × 2 B (see budget table). Build arithmetic (u16 stores + Int32/Float64 accumulation) is standard practice last verified in CH-era implementations that integerize weights for exactly this reason [ASSUMED as common practice; the exactness proof is mine and self-contained].

Build must assert the metric doesn't overflow: no final table value and no intermediate Dijkstra label > 65,000 units. If it ever fails (future fork at nation scale): raise q via the header knob before re-encoding (documented). Fail-honest exit otherwise.

Pattern 3: sentinel / small-island handling (the classic ALT edge case, contained)

Sentinel (0xFFFF) appears only when v cannot reach / be reached from landmark ℓ — i.e. v lies outside ℓ's SCC fragment. Known graph facts: the real KL build is ONE giant SCC covering 305k of 311k nodes (Phase 6 build fact); landmarks are chosen inside it (build asserts each anchor's Dijkstra reached ≥ N·coverageFact nodes — abuse the same fragmentation machinery the build already has).

Containment layers (recommendation):

  1. Build-time: anchors must live in the giant SCC; per-anchor coverage assertion; fail-honest otherwise (named constant LANDMARK_MIN_COVERAGE_FRACTION, e.g. 0.90, measured justification).
  2. Search-scoped constant skip: for a given (o, g) search, drop landmark ℓ entirely from the max if its goal- or origin-side constant is sentinel (g or o can't reach ℓ) — a per-SEARCH decision, never per-node.
  3. Per-node fallback: if a node's table entries are sentinel for all remaining ℓ (a fringe island node), its h_f/h_b fall back to the haversine bound (existing code, admissible + consistent everywhere).
  4. Correctness containment even on the boundary: every node u that lies on ANY o→g path necessarily reaches g; if g reaches some valid ℓ then u reaches ℓ too (u → g → ℓ), so u's table entries are non-sentinel for every useful ℓ — the fully-connected consistency argument applies on precisely the productive search region. Dead-end island nodes only ever see locally-misordered keys, cannot create a premature valid mu, and their real-seconds are recomputed exact. ∎ [ASSUMED: my proof sketch — planner should schedule the A-case that empirically seals it: the A1 sweep MUST include an islanded synthetic fixture where ALT is ACTIVE and Dijkstra-equivalence holds on every pair.]
  5. Empirical seal: A1's 625-pair exhaustive Dijkstra-equivalence re-run on the ARG2 fixture with ALT potentials live, plus a new dedicated island case.

Anti-Patterns to Avoid

  • Quantizing stored distances then patching with an epsilon bound: breaks exact consistency (max-of-bounds + per-node masking interacts badly); the ĉ-metric removes the whole class.
  • A per-node skip rule that varies the landmark set mid-max: kills the max-of-Lipschitz consistency argument. Skip at the SEARCH scope (constants) and fall back per NODE, never mask individual ℓ per node while others stay active.
  • Reusing pool/DBG-style offsets: the landmark suffix must extend layoutFor's total arithmetic so decode's exact-size law stays arithmetic-only (T-06-03b).
  • Rebuilding the node order silently: the baseline record must identify pairs by COORDINATES (snap at run time), never graph node indices — a re-fetch of Overpass (or any dedup-order drift) shifts indices and would fake a tie-difference. (The existing 25-route gate samples quadrant node indices — fine for CI-size CPU snapshotting, NOT for cross-version optimality equality.)
  • Losing sentinels into Float64Array: 0xFFFF must be treated as sentinel BEFORE conversion (65535 × quantum is a large-but-finite bogus distance that would quietly dominate the max).
  • Renaming the R2 key or local artifact path: upload script and key are locked; the "v1" suffix is a storage NAME, the ARG2 header is the format truth. Document loudly; never fix the name in this phase.
  • Second encodeGraph: GraphInput gains landmark input; all call sites (graph.test synthetics, fixture builder, bot webhook corridor graphs) update in the same change. No v2-only encode function.

Don't Hand-Roll

Problem Don't Build Use Instead Why
Shortest-path preprocessing to cut A* worst case custom steering hacks, speed-class masks, radius clamps ALT with well-spread anchors (the literature's measured lever; beaten only by CH at scale, which is deferred) Preserves exact optimality (proven formulation), $0, one-function integration
Lower-bound math on quantized values admissibility epsilon patches, round-up tables + mixed corrections ĉ-metric integer Dijkstra (Pattern 2) Exact admissibility + consistency by construction
Bidirectional potential new symmetric/dual search variant, meet-constant search the existing πf/πr → (πf−πr)/2 scheme already in astar.ts Verified canonical; the stop rule/meet-probe machinery carries over untouched
Landmark selection quality metric hand-tuned POI picks ("anchors are NOT POIs" — D-02 context) farthest-point over the graph metric, seeded/ID-deterministic Determinism, reproducibility, spread; measured verdict decides count

Key insight: the repo's remaining risk is measurement honesty, not algorithm novelty — every needed primitive (exact potentials, integer tables, fail-honest decode, seeded gates) already has a tested home in the codebase.

Common Pitfalls

Pitfall 1: baseline cannot be re-measured after ARG2 lands

What goes wrong: D-03 compares post-ALT seconds to pre-ALT seconds. Once graph.ts versions to 2, decodeGraph rejects the v1 binary — the "old code + old binary" baseline is no longer reproducible from the working tree. Why it happens: D-01's ARG2-only law (correctly) removed the v1 path. How to avoid: measure and COMMIT the baseline record FIRST (a Wave-0 task): current pre-phase code + the existing v1 build, pairs identified by coordinates, seconds + verdict class + pair fingerprints written to a committed JSON. All comparison happens against the record. Warning signs: a plan that generates the baseline AFTER the codec change. JSON float caveat: record seconds as JSON numbers only if produced by the same f32→f64 accumulation the runtime uses; routeAstar returns exact f64 accumulations of f32 edgeSeconds, JSON round-trips IEEE doubles losslessly — safe [VERIFIED: JS number semantics; the same equality law the A1 sweep uses].

Pitfall 2: node-index instability across rebuilds

What goes wrong: the 25-route gate pricks quadrant arrays by node INDEX; a re-build of the binary (even byte-different only in name bytes) reorders nodes, and the same seed selects different pairs — the pre/post "same pair" claim collapses. How to avoid: the extended sample + baseline record pin pairs by lat/lng (derived from the graph's order-independent bbox + seeded PRNG, snapped at run time); document that index-keyed sampling is a CI-size convenience only. Warning signs: any "verify same route as baseline" logic that prints node ids instead of endpoint coordinates.

What goes wrong: ALT with too few anchors leaves the worst pairs slow; the wrong next move (heuristic scale-down, speed tweaks) breaks the A1 law. How to avoid: pre-commit the remediation ladder in the plan: (a) raise L within the 16–32 rule and rebuild (lines up with D-02's scaling law); (b) only then report faithfully (print-the-measurement law gates even a FAIL verdict; a measured-fail c2 is an escalation, never a hack). Warning signs: any edit to routeAstar's stop rule or reductions during the campaign.

Pitfall 4: header-extension breaks f64 alignment or exact-size law

What goes wrong: moving HEADER_SIZE from 64 to 72 must keep node f64s 8-byte aligned (72 % 8 = 0 ✓) and decode's total-size arithmetic must cover the variable-size suffix — a miscounted suffix manifests as "corrupt binary" on the WORKER without a single test failing (if tests skip the big fixture). How to avoid: the G-suite gains count-driven table arithmetic cases; --check prints landmarkCount + coverage; the round-trip verify command in the fork contract re-runs after upload. Warning signs: RangeError on any typed-array construction (alignment) — instant signal.

Pitfall 5: the u16 quantum ceiling surprises a fork

What goes wrong: at 0.1 s units, distances ≥ 6,553.5 s are inexpressible; a larger-city fork overflows silently into wrong sentinel-collisions. How to avoid: build-time assertion (≤ 65,000 units) + ROAD-GRAPH.md documents the landmarkQuantumS knob and its formula. Warning signs: table bytes saturated at exactly 0xFFFF in --check stats.

Pitfall 6: docs divorce (v1/ARG2 naming)

What goes wrong: keeping the R2 key kl-roadgraph.v2.bin… — NOT ALLOWED anyway (upload locked); but if ROAD-GRAPH.md / CONFIGURATION.md silently still say "v1 layout", the next fork's rebuild breaks. How to avoid: D-05's docs tasks include ROAD-GRAPH.md's layout table (ARG2 version field, table sections, quantum knob) and the CPU-gate decision table gaining the (c2) rung. Warning signs: any mention of "ARG1" as still-decodeable.

Runtime State Inventory

Refactor/version-bump phase — included. No rename of user-facing strings.

Category Items Found Action Required
Stored data R2 object araharah-graphs/kl-roadgraph.v1.bin currently ARG1 bytes Re-upload once post-build (whole migration per D-01). No data migration — the object is a build artifact; decode-null rollout covers the window
Live service config bot/wrangler.jsonc ROAD_GRAPH binding (bucket name), GRAPH_KEY constant in worker-router.ts ("kl-roadgraph.v1.bin") NONE — both unchanged (D-01 lock). The name-vs-format rule documented
OS-registered state None — no cron/scheduler registrations for the graph None (verified: cron arm reads the same loadGraph, no separate key)
Secrets/env vars bot/.dev.vars four keys (BOT_TOKEN/WEBHOOK_SECRET/WEBHOOK_URL/MINI_APP_URL) — unchanged None
Build artifacts routing/data/kl-roadgraph.v1.bin (gitignored, 14.7 MB, ARG1 on disk) — must be REBUILT by the phase; committed mini-graph fixture mini-graph.bin currently ARG1 → regenerated as ARG2; old bytes preserved as the v1 rejection vector Rebuild + regenerate fixture + keep rejection fixture (fixture build script gains the landmark stage)

Nothing found in category: OS-registered state — verified (repo has no launchd/systemd/pm2 surfaces; Workers cron is declared in wrangler.jsonc and untouched).

State of the Art

Old Approach Current Approach When Changed Impact
Geographic max-speed A* bound (2026-10-24 unidirectional) ALT landmark bounds This phase Survey-verified: geo/max-speed is "poor… small or nonexistent gain"; ALT is the proven sharpening
Unidirectional → BH 2026-10-24 BH + ALT (planes exactly on top) This phase Steering restored on both sides; no new search skeleton
ARG1 format (2026-10-24, 14.7 MB) ARG2 append-only extension, u16 suffix This phase Exact-size law extends mechanically; v1 decode retired

Deprecated/outdated: none of the repo's patterns become dead; the reverse-CSR WeakMap, snap grid, corpus harness, RT/H gate suites all carry unchanged.

Validation Architecture

Test Framework

Property Value
Framework hand-rolled registry (routing/test/harness.mjs) + node:test-style runner, zero deps [VERIFIED: harness.mjs]
Config file none — routing/test/runner.mjs + npm workspace scripts (npm run routing:test, npm test -w bot)
Quick run command npm run routing:test (seconds)
Full suite command npm run routing:test && npm test -w bot && npm run routing:typecheck && npm run bot:typecheck
Big-graph gates node routing/scripts/build-road-graph.mjs (preprocessing stage) · … --check · node routing/scripts/measure-route-cpu.mjs (p95 ≤ 8) · extended verdict flag · D-03 gate script

Phase Requirements → Test Map

No new REQ ids (SC3/SC4 carry-over). The phase requirements below mirror the ROADMAP §Phase 8 success criteria + the locked decisions:

Req ID Behavior Test Type Automated Command File Exists?
SC-1 P95 within free-plan budget p95 ≤ 8 ms on the real ARG2 build measurement gate node routing/scripts/measure-route-cpu.mjs ✅ exists (extends: history + campaign)
SC-1′ extended verdict (D-04) ~100 cross-metro seeded pairs, p95 ≤ 8 maintainer-time (script flag) node routing/scripts/measure-route-cpu.mjs --extended (flag name discretion) ❌ Wave 0
SC-2 optimality unchanged (D-03) corpus re-run: seconds equal (± named epsilon ~±0.1%) + same verdict class per pair vs committed baseline offline gate script + in-process mirror new node routing/scripts/<optimality-gate>.mjs (name discret.) ❌ Wave 0 (baseline record ✅ prerequisite task)
A1 ALT-equivalence (outstanding optimality proof) 625-pair sweep on ARG2 fixture WITH ALT potentials live unit (suite) npm run routing:test (A1 case) ✅ exists — must stay green; new island case ❌
G-suite ARG2 codec ARG2 roundtrip incl. tables; ARG1 fixture → decode null; corrupt/version/landmarkCount=0/oversized-count → null; zero-copy views; sentinel integrity unit (suite) npm run routing:test (G cases) partially — new cases ❌, G3/G8 re-pin ❌
Rollout null path (RT trunks) corridor/worker behavior on ARG2 corridor graph; decode-null → 200 null unchanged bot suite npm test -w bot (RT1–RT11 + H1–H8) ✅ — corridor graphs flow through updated encodeGraph signature
Build pipeline preprocessing stage prints L, per-anchor coverage, quantum check, byte budget; exit 1 honest script gate node routing/scripts/build-road-graph.mjs (+ --check) ✅ extends (stage ❌ new)
Upload unchanged upload-road-graph.mjs byte-for-byte unchanged structural grep pin (RP-style) recorded in bot/routing suite (add 1-line pin case) ❌ small new case
SC-3 docs retirement (D-05) KNOWN-LIMITATIONS c3 section → retirement record w/ measured numbers + date; ALT roadmap row flipped; ROAD-GRAPH.md CPU-gate table gains (c2) rung doc acceptance (human-verifiable, supported by printed gates) review leg of phase gate edit ❌ new

Sampling Rate

  • Per task commit: npm run routing:test (fast, includes A1 sweep + new G/A cases).
  • Per wave merge: full suite (routing + bot + typechecks) + build --check on a committed-adjacent stub if built.
  • Phase gate: the REAL build → --check → 25-route gate (p95 ≤ 8) → extended verdict → D-03 optimality gate green → upload once → bot deploy heartbeat (existing RT smoke path) → D-05 docs record carries the printed numbers.

Wave 0 Gaps (new files/cases before implementation tasks)

  • Baseline record + recorder tool (D-03 prerequisite — MUST be Wave 0 / first task; Pitfall 1)
  • Extended-sample flag implementation (D-04) — deterministic coordinate-pair generator, MIN_PAIR_KM-style named constant, p95 print + exit-gate
  • fixtures/mini-graph-arg1.bin — bytes preserved from the current committed ARG1 fixture (decode-rejection vector)
  • fixtures/build-mini-graph.mjs gains a landmark stage (tiny L; ARG2 output)
  • New G cases (ARG2 codec + rejection matrix) and an island A case (Pattern 3 seal)
  • Upload-unchanged structural pin
  • Test framework install: none needed (zero-dep harness present)

Constraint to re-verify in planning: budget-audit (Phase 7) re-runs measure-route-cpu.mjs — keep its CLI contract (silent default mode = the 25-route gate, exit codes) so the audit keeps passing, or account for the flag addition there.

Security Domain

security_enforcement enabled by default; included.

Applicable ASVS Categories

ASVS Category Applies Standard Control
V2 Authentication no (untouched) — Bearer-claim gate on /api/route unchanged existing getClaimByToken posture
V3 Session Management no KV TTL classes untouched
V4 Access Control no (maintainer scripts stay repo-market internal) scripts never imported by runtime (existing law)
V5 Input Validation yes — the binary is hostile input decodeGraph exact-size + version + landmarkCount laws (T-06-03/03b extended to the suffix); hostile header on tiny buffer rejects arithmetically BEFORE view allocation
V6 Cryptography no HMAC/uniqueness paths untouched
V7/V11 … no unchanged posture

Known Threat Patterns for this stack

Pattern STRIDE Standard Mitigation
Hostile/corrupt binary with monster landmarkCount (allocation bomb) Tampering/DoS exact-size check before allocation (T-06-03b law extends — verified pattern in graph.ts today)
Sentinel-flood graphs (all u16 = 0xFFFF) degrading search CPU into 1102 DoS search-scoped constant skip drops such ℓ wholesale; the CPU gate prints per-route ms; category-only logs unchanged
Rollout confusion: Service receives ARG1 Tampering/integrity decode null → 200 null (router-pending); --check + roundtrip-verify groom
Coordinate/payload logging leakage Info disclosure category-only log law carries unchanged; gate prints per-route ms + ids-of-nothing (RT1–RT3 pins remain)

Threat surface unchanged beyond the binary content — same one /api/route boundary, same Bearer claim, no new endpoints (RP1 law holds; nothing new fetches).

Assumptions Log

# Claim Section Risk if Wrong
A1 The (πf−πr)/2 + (πr−πf)/2 survey formulation is exactly what astar.ts's p_f/p_b already implement (signs verified against candidate equations) Pattern 1 Would force a potential-pair redesign — but verified line-by-line; low
A2 ALT bound families computed in the ĉ metric are exactly admissible AND exactly consistent vs real f32 edges (directed-triangle + Lipschitz + ĉ≤c arguments) Pattern 2 Admissibility epsilon would be needed (a named-constant variant exists as fallback: subtract one quantum K); medium — planner should treat the epsilon-variant as a plan-level contingency note only; A1 sweep is the empirical seal
A3 Sentinel/island handling: nodes on any o→g path have non-sentinel entries for validated anchors; boundary edges can't corrupt mu Pattern 3 Contained by A1 + island case + real-seconds recompute; would degrade to haversine-only behavior (perf regression, not correctness)
A4 Build preprocessing cost ≈ 2L Dijkstra passes ≈ 2 s measured-order at L=24 (derived from measured 38 ms/full-graph pass) Budget Larger build times acceptable anyway (maintainer-time); low
A5 landmarkCount === 0 → decode null is the correct reading of D-01's no-silent-degradation spirit ARG2 layout If the maintainer wants a landmark-less v2 capability, decode should accept 0 and fall back to haversine silently (contradicts D-01 wording "no v1 fallback branch"); flagged for plan review
A6 p95 ≤ 8 achievable with L in 16–32 for the 20–30 km class (order-of-magnitude settled-node reduction expectation vs measured 38–93 ms) Summary/gate Verdict may land FAIL at first L — the D-02 scaling rule is the measured knob; escalate honestly if still failing at L=32 (the same print-the-measurement law)
A7 u16 sentinel 0xFFFF never collides with real data (build asserts ≤ 65,000) u16 design Overflow → silent table corruption; the assertion makes it loud
A8 f64 JSON round trip of route seconds in the baseline record is lossless Pitfall 1 Equality gate needs a fudge — trivially fixed by storing unit strings; low

Open Questions

  1. Final L count and its exact rule constant.

    • What we know: expected 16–32; D-02 makes it a measured product of the gate.
    • What's unclear: should the plan hard-pin a formula (e.g. clamp(round(N/13000), 16, 32)) or build once per candidate and pick the smallest passing L?
    • Recommendation: parameterize L in the build script (CLI arg + named default); land default 24; the campaign prints p95 at 24 and (only if FAIL) at 32; the verdict documents the chosen count.
  2. landmarkQuantumS value.

    • What we know: 0.1 s fits KL (54,000 max units) and gives sub-decisecond resolution; forks can raise it.
    • Recommendation: 0.1, header field, doc'd knob.
  3. Does docs/CONFIGURATION.md / budget-audit need a touch?

    • Verified likely: budget-audit re-runs measure-route-cpu (per KNOWN-LIMITATIONS "run the audit yourself"); flag addition must stay default-compatible. Plan task: read bot/scripts budget-audit runner + keep contract.

Environment Availability

Dependency Required By Available Version Fallback
Node.js build/test/measurement ✓ v24.15/24.17/24.18 (nvm) [VERIFIED] —
npx/wrangler upload (unchanged), R2 roundtrip verify ✓ (repo standard; R2 binding live) — —
Overpass public API build re-fetch (preprocessing rides the SAME extract) ✓ (used 2026-10-24) — retry logic already in build script
git commits ✓ — —
No new deps — — — —

Missing dependencies with no fallback: none. Missing dependencies with fallback: none. (PATH caveat: bare node resolves only with nvm sourced; executing sessions run through npm scripts which inherit the shell's node.)

Code Examples

(Reference-grade) the reduced-cost search with ALT potentials — the EXISTING loop, unchanged

// astar.ts (current, stays) — only `potential()` internals change:
const expandForward = fHeap.size <= bHeap.size;
const u = (expandForward ? fHeap : bHeap).pop()!;
// … forward relaxation:
const v = graph.edgeTarget[e]!;
const pv = potential(v);
const candidate = du + graph.edgeSeconds[e]! + pfu - pv;
// … the stop rule / meet probes / real-seconds reconstruction unchanged.

Potential() rewrite sketch (typed arrays, allocation-free, sentinel-safe)

// L = graph.landmarkCount; T_TO = graph.landmarkTo (u16 rows of N);
// T_FROM = graph.landmarkFrom; Q = graph.landmarkQuantumS;
per search init:
  for ℓ: goalTo[ℓ]   = toTable[ℓ*N+goal]   (NaN if 0xFFFF — search-scoped skip)
         goalFrom[ℓ] = fromTable[ℓ*N+goal]
  … likewise for origin
per-node lazy potential(v):
  bestTo = -Infinity, bestFrom = -Infinity
  for ℓ in 0..L-1:
    const tv = toTable[ℓ*N+v];   if (tv === 0xFFFF) continue node-entry   // per-node fallback candidate
    if (goalTo[ℓ]   not NaN) bestTo   = max(bestTo,   tv - goalTo[ℓ]);
    if (goalFrom[ℓ] not NaN) bestFrom = max(bestFrom,  goalFrom[ℓ] - fromTable[ℓ*N+v]);
  // h_f = bestTo > -inf ? max(bestTo, havF(v)) * Q : havF(v)      ← fallback keeps isle behavior sane
  // h_b likewise with origin consts; return (h_f - h_b)/2            ← the SURVEY combination, exactly today's

(Sketch only — final shape/naming is the executor's; laws to preserve: no Map, lazy caches, no logging.)

Build preprocessing stage skeleton

// inside build-road-graph.mjs, after `edges` assembly, before encodeGraph:
const QUANTUM_S = 0.1;                       // named constant; header field value
const quanta = Math.floor(seconds / QUANTUM_S);   // integer ĉ, ≥ per-edge (0.1s unit)
// 1) integer Dijkstra over REVERSED edges from each anchor → toTable row
// 2) integer Dijkstra over forward edges from each anchor   → fromTable row
// 3) farthest-point selection over the ĉ metric (seeded, degree-≥2 filtered, ID tie-break)
// 4) coverage assertion per anchor; quantum-overflow assert; encode through encodeGraph

Sources

Primary (HIGH confidence)

  • Bast, Delling, Goldberg, Müller-Hannemann, Pajor, Sanders, Wagner — Route Planning in Transportation Networks (arXiv:1504.05140, fetched via ar5iv): bidirectional potential combination (πf−πr)/2 scheme; ALT triangle-inequality bounds verbatim + feasibility claim; geographic-bound weakness; landmark selection guidance ("well-spaced close to the boundary leads to the best results"; minor-speedup caveat).
  • Campbell — Computing Shortest Paths using A*, Landmarks, and Polygon Inequalities (arXiv:1603.01607): ALT storage θ(|L|·|V|); heuristic = max over landmark inequalities; consistency is theorem-grade (their ALP variant NOT consistent — the class of pitfall Pattern 3 guards).
  • Goldberg & Harrelson — Computing the Shortest Path: A* Search Meets Graph Theory (SODA 2005; Princeton/MSR TR-2004-24 PDFs surfaced via search): far/sector landmark selection ("For each sector, pick a vertex farthest away from the center…").
  • Goldberg & Werneck — Better Landmarks within Reach (via wea PDF snippet): the "avoid" greedy strategy ("picks landmarks one by one, always in regions… not already covered").
  • Repo code + docs read in full this session: routing/src/graph.ts, routing/src/astar.ts, routing/scripts/{build-road-graph,measure-route-cpu,route-corpus,upload-road-graph}.mjs, routing/test/{harness.mjs,graph.test.mjs,astar.test.mjs}, bot/src/routing/worker-router.ts, docs/ROAD-GRAPH.md, docs/KNOWN-LIMITATIONS.md, fixture builder, .planning/{STATE,REQUIREMENTS}.md, .planning/phases/{06…,07…,08…}/….

Secondary (MEDIUM confidence)

  • Build-runtime cost estimate A4 (derived, not measured).
  • u16 integer-weight practice precedent (common in the CH-era ecosystem, not fetched specifically).

Tertiary (LOW confidence)

  • Expected p95 magnitude post-ALT (A6) — no published KL-metro analogue; the gate is the truth.

Metadata

Confidence breakdown:

  • Standard stack / integration shape: HIGH — verified against code + the canonical survey; the potential-integration finding is the load-bearing fact and it is pinned twice (survey + line-by-line code check).
  • ARG2 layout: HIGH — append-only discipline keeps every pinned law mechanical; byte math checkerboarded against measured counts.
  • Quantization exactness: HIGH on the math (self-contained proofs documented in Pattern 2); its empirical seal relies on the A1 sweep + new island case (planner should keep both mandatory).
  • Perf verdict: MEDIUM — order-of-magnitude argument from measured baselines; the gate decides.
  • Pitfalls: HIGH (all six are chase-able in this repo's own artifacts).

Research date: 2026-10-25 Valid until: 2026-11-24 (binary layout insights stable; only Node minor versions drift)