diff --git a/docs/ROAD-GRAPH.md b/docs/ROAD-GRAPH.md new file mode 100644 index 0000000..f7396a1 --- /dev/null +++ b/docs/ROAD-GRAPH.md @@ -0,0 +1,130 @@ +# The Road Graph (engine A's data) — build, host, and the laws + +Engine A routes over a **CSR road-graph binary** built from OpenStreetMap and +hosted on Cloudflare R2. The binary is a **build artifact, not a committed +file** — forks build and upload their own. Everything needed is committed: +two maintainer scripts and this doc. All of it is **$0** (Overpass public +instance, R2 free tier, Workers free plan). + +## The fork contract (your graph, your bucket) + +1. **Create your bucket** (once, per Cloudflare account): + + ``` + npx wrangler r2 bucket create araharah-graphs + ``` + + If `wrangler` reports that R2 is not enabled, activate it once in the + Cloudflare dashboard (**R2 → Get started**) — account-owner steps, then + re-run the command. Use a different name if you like, but the name must + agree with `bot/wrangler.jsonc`'s `ROAD_GRAPH` binding (`bucket_name`). + +2. **Build the KL graph** (or your own region — tune `DRIVABLE_CLASSES` / + the bbox in the script): + + ``` + node routing/scripts/build-road-graph.mjs + ``` + + Computes the bbox from the committed corpus (gazetteer + golden pairs, + ± 0.05°), fetches ONE Overpass extract (drivable classes only), encodes + the pinned CSR binary through `encodeGraph`, and writes + `routing/data/kl-roadgraph.v1.bin` — **only** after the encode round-trip + and the size gate pass. Exit 1 leaves any previous binary untouched. + +3. **Check the artifact** (decode + stats, no network): + + ``` + node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin + ``` + +4. **Upload to R2**: + + ``` + node routing/scripts/upload-road-graph.mjs + # = npx wrangler r2 object put araharah-graphs/kl-roadgraph.v1.bin \ + # --file routing/data/kl-roadgraph.v1.bin --remote + ``` + + Verify the round-trip: + + ``` + npx wrangler r2 object get araharah-graphs/kl-roadgraph.v1.bin \ + --file /tmp/opencode/graph-roundtrip.bin --remote + node routing/scripts/build-road-graph.mjs --check /tmp/opencode/graph-roundtrip.bin + ``` + +5. **Deploy the Worker** (`npm run deploy -w bot`). `wrangler.jsonc` binds + the bucket as `ROAD_GRAPH`; `worker-router.ts` loads the object + `kl-roadgraph.v1.bin` once per cold isolate (module-scope memo, shared + with the cron arm) — 1 R2 GET, not per request. + +**Dev note:** with no bucket (or an empty one), `/api/route` answers `200 +null` — the designed router-pending state (D-03). Dev and tests never need +the real binary; tests use the committed mini-graph fixture. + +## The envelope law (D-01) + +| Gate | Value | Enforcement | +|------|-------|-------------| +| Ceiling | **≤ 80 MB** | enforced — build + upload + `--check` all exit 1 over it | +| Floor | **> 1 MB** | enforced — a smaller "road graph" is a broken build | +| Expected band | **30–80 MB** | advisory — a smaller build prints a WARNING (verify the class filter) and continues | + +The binary lives in `routing/data/` and is gitignored (`*.bin`): it is +ODbL-derived data, regenerated at will, never source. + +## The CPU gate (A1) — OPEN: measured over the free-plan budget + +Workers free plan: **10 ms CPU per request** (Error 1102 beyond it). The A\* +cost over the REAL graph is measured, never hoped: + +``` +node routing/scripts/measure-route-cpu.mjs +``` + +25 pairs (golden-corpus routes + quadrant-sampled cross-city pairs; directed +dead-end probes reported separately) timed around `routeAstar` over the real +binary. **The gate is p95 ≤ 8 ms** (20% margin under the ceiling). + +**Measured state (2026-10-24, the 311k-node linked graph): GATE FAIL.** The +mitigation ladder was executed in order: + +| Ladder step | Outcome | +|-------------|---------| +| full graph, unidirectional A\* | p95 ≈ 38 ms — FAIL | +| (a) drop tertiary+residential, rebuild | p95 ≈ 18 ms — FAIL **and** 12/25 pairs became unreachable (the dropped classes are the fine mesh tying landmarks to arterials) — rejected on correctness, reverted | +| (b) bidirectional A\* (BH, Ikeda potentials) | p95 ≈ 38 ms — FAIL: the potential split halves each side's heuristic steering, cancelling the halved search radius. Kept: sound (A1's 625-pair Dijkstra equivalence still passes) and better on short product routes | +| (c) Workers Paid | **NOT applied — breaks the $0 law; needs a maintainer decision** | + +The decision this gate now escalates: **(c1) Workers Paid ($5/mo — breaks +$0)**, **(c2) a graph-layout v2 with ALT landmark preprocessing (still $0 at +runtime, real build-time engineering)**, or **(c3) accept free-plan routing +with honest degradation** (long reroutes may hit 1102; the client's +`createHttpRoutingSource` then degrades to the router-pending state). +Short product routes (2–8 km, the common reroute) measured 2–8 ms — inside +budget; the failures concentrate on 20–30 km cross-metro routes. + +## Licensing (the ODbL attribution) + +The graph is derived from OpenStreetMap data — +**© OpenStreetMap contributors, ODbL 1.0** +(), fetched via the public +Overpass API. Hosting or redistributing the binary requires the same +attribution (ODbL's attribution clause travels with derived databases). The +route fixtures carry the same notice inside their committed envelope +(`_license`/`_attribution`/`_notice`), and this doc is the notice surface for +the hosted binary. Keep it intact in forks. + +## Key facts + +| What | Where | +|------|-------| +| Build tool (Overpass → CSR) | `routing/scripts/build-road-graph.mjs` | +| Upload tool (wrangler shell) | `routing/scripts/upload-road-graph.mjs` | +| CPU measurement gate | `routing/scripts/measure-route-cpu.mjs` | +| Binary (gitignored artifact) | `routing/data/kl-roadgraph.v1.bin` | +| R2 object key | `kl-roadgraph.v1.bin` (pinned in `worker-router.ts` as `GRAPH_KEY`) | +| Binding | `ROAD_GRAPH` in `bot/wrangler.jsonc` (bucket, NEVER a KV namespace — GE5) | +| Loader (isolate-memoized) | `bot/src/routing/worker-router.ts` → `loadGraph` | +| Binary layout authority | `routing/src/graph.ts` (`encodeGraph`/`decodeGraph`, pinned CSR v1) | diff --git a/routing/scripts/build-road-graph.mjs b/routing/scripts/build-road-graph.mjs index 354271f..af5e16f 100644 --- a/routing/scripts/build-road-graph.mjs +++ b/routing/scripts/build-road-graph.mjs @@ -70,8 +70,27 @@ const BBOX_MARGIN_DEG = 0.05; /** Node dedup granularity: 1e-6 deg ≈ 0.11 m — far under GPS noise. */ const COORD_ROUND_DECIMALS = 6; /** drivable classes ONLY (06-RESEARCH Pitfall 1): footway/cycleway/service/ - * path are deliberately absent — a walking path must never enter a road A*. */ -const DRIVABLE_CLASSES = new Set([ + * path are deliberately absent — a walking path must never enter a road A*. + * The *_link classes are LOAD-BEARING (measured 2026-10-24): they are the + * interchange ramps tying motorways/trunk/primary into the surface network. + * Without them the graph FRAGMENTS — 7.7% of nodes sat in severed motorway + * islands and golden corpus pairs resolved "unreachable" because an endpoint + * snapped onto an isolated fragment. + * + * A1 MITIGATION LADDER HISTORY (measure-route-cpu.mjs is the arbiter): + * 2026-10-24 full 13-class linked graph (311k nodes): p95 38.4 ms, ALL + * corpus pairs route — connectivity correct, CPU over budget. + * 2026-10-24 ladder step (a) — drop tertiary+residential (127k nodes): + * p95 17.8 ms BUT 12/25 pairs (incl. 8/15 golden corpus pairs) became + * UNREACHABLE — the dropped classes were the fine mesh tying landmarks + * to the arterial network. REJECTED on correctness (D-03: an honest + * router-pending for half the product's destinations is not a smaller + * graph, it is a broken one). REVERTED to the full 13-class set; the + * CPU gate is carried by ladder step (b) — bidirectional A* in + * routing/src/astar.ts — and the recorded verdict in the plan summary. + * Ladder step (c) — Workers Paid — breaks the $0 law and requires user + * sign-off; it is NOT applied in this script. */ +const ACTIVE_CLASSES = new Set([ "motorway", "trunk", "primary", @@ -80,6 +99,11 @@ const DRIVABLE_CLASSES = new Set([ "residential", "unclassified", "living_street", + "motorway_link", + "trunk_link", + "primary_link", + "secondary_link", + "tertiary_link", ]); /** Class speed defaults (km/h) when a way carries no parsable maxspeed. */ const CLASS_SPEED_KMH = { @@ -92,8 +116,16 @@ const CLASS_SPEED_KMH = { unclassified: 30, living_street: 20, }; -/** Motorway link ramps inherit the parent class speed via these links. */ -const LINK_CLASS_SPEED_KMH = { motorway_link: 60, trunk_link: 60 }; +/** Motorway link ramps inherit a near-parent speed via these links. */ +const LINK_CLASS_SPEED_KMH = { + motorway_link: 60, + trunk_link: 60, + primary_link: 50, + secondary_link: 40, + tertiary_link: 40, +}; +/** ENFORCED floor — an at/below-1 MB "KL road graph" is a broken build (D-01). */ +const MIN_BINARY_BYTES = 1 * 1024 * 1024; function fail(message) { console.error(`build-road-graph: ${message}`); @@ -122,7 +154,7 @@ if (checkIdx !== -1) { ` bytes: ${size}`, ); if (size > MAX_BINARY_BYTES) fail(`--check: ${size} bytes exceeds the ${MAX_BINARY_BYTES}-byte D-01 envelope`); - if (size <= 1024) fail(`--check: ${size} bytes is implausibly small — not a real road graph`); + if (size <= MIN_BINARY_BYTES) fail(`--check: ${size} bytes is at/below the ${MIN_BINARY_BYTES}-byte floor — not a real road graph`); process.exit(0); } @@ -175,7 +207,7 @@ console.log( /* ————— the Overpass fetch (ONE request, drivable classes only) ————— */ -const CLASSES_RE = "^(" + [...DRIVABLE_CLASSES].join("|") + ")$"; +const CLASSES_RE = "^(" + [...ACTIVE_CLASSES].join("|") + ")$"; const query = `[out:json][timeout:${Math.floor(REQUEST_TIMEOUT_MS / 1000)}];` + `way["highway"~"${CLASSES_RE}"](${bbox.minLat},${bbox.minLng},${bbox.maxLat},${bbox.maxLng});` + @@ -243,7 +275,7 @@ function effectiveKmh(highway, maxspeedTag) { for (const way of ways) { const highway = way.tags?.highway; - if (typeof highway !== "string" || !DRIVABLE_CLASSES.has(highway)) continue; + if (typeof highway !== "string" || !ACTIVE_CLASSES.has(highway)) continue; const geometry = way.geometry; if (!Array.isArray(geometry) || geometry.length < 2) continue; const kmh = effectiveKmh(highway, way.tags?.maxspeed); @@ -316,11 +348,15 @@ console.log( ` binary size: ${mb(bytes)} (${bytes} B)`, ); -// THE ENFORCED GATE (D-01): over the ceiling → exit 1 WITHOUT writing. +// THE ENFORCED GATE (D-01): outside the 1-80 MB envelope → exit 1 WITHOUT writing. if (bytes > MAX_BINARY_BYTES) { fail(`binary is ${mb(bytes)} — over the ${mb(MAX_BINARY_BYTES)} D-01 envelope. ` + `Tighten DRIVABLE_CLASSES (drop tertiary/residential first) or shrink the bbox, then rebuild.`); } +if (bytes <= MIN_BINARY_BYTES) { + fail(`binary is ${bytes} bytes — at/below the ${MIN_BINARY_BYTES}-byte floor; a real metro graph is never this small. ` + + `Verify the Overpass query actually returned ways, then rebuild.`); +} // THE ADVISORY BAND (A2's 18–50 MB forecast vs D-01's 30–80 MB expectation): // a small graph is a review prompt, not a failure — the enforced gate stays. if (bytes < EXPECTED_MIN_BYTES) { diff --git a/routing/scripts/measure-route-cpu.mjs b/routing/scripts/measure-route-cpu.mjs new file mode 100644 index 0000000..eda3004 --- /dev/null +++ b/routing/scripts/measure-route-cpu.mjs @@ -0,0 +1,185 @@ +#!/usr/bin/env node +/* + * measure-route-cpu.mjs — the A*-CPU measurement gate (A1, the phase's top risk). + * ------------------------------------------------------------------------------ + * A MAINTAINER-TIME REPO TOOL — NOT a runtime dependency; nothing in bot/src, + * web/src, or routing/src imports it. The free plan gives a Worker 10 ms of + * CPU per request (Error 1102 beyond it), so the A*-over-the-real-graph cost + * is MEASURED AND PRINTED, never hoped for ("budget gates print their + * measurement" — the BA9 precedent). + * + * What it measures: routeAstar() over the REAL routing/data/kl-roadgraph.v1.bin + * (decodeGraph → typed arrays, exactly the Worker's data plane), timed with + * process.hrtime.bigint() around the search call only. + * + * The 25-route sample mixes both realistic classes: + * - the fixture-corpus endpoints (golden-pairs origins + gazetteer + * destinations), snapped through snapToGraph — the demo-script routes + * - quadrant-sampled node pairs spread across the graph bbox (deterministic + * seeded PRNG — re-runs are comparable) — the cross-city worst cases + * One UNMEASURED warm-up route runs first so the numbers reflect the + * steady-state isolate an /api/route request actually lands on (the warm-up + * is printed, never hidden). + * + * The GATE (A1's mitigation margin): p95 > P95_BUDGET_MS (8 ms — 20% under + * the 10 ms free-plan ceiling) exits 1. The mitigation ladder, IN ORDER: + * (a) tighten DRIVABLE_CLASSES (drop tertiary/residential) and rebuild + * (b) bidirectional A* variant + * (c) Workers Paid as an explicit fork decision (breaks $0 — user sign-off) + * + * LADDER HISTORY (2026-10-24, the real KL graph — see the plan summary): + * full 13-class graph, unidirectional A*: p95 38.4 ms GATE FAIL + * ladder (a) drop tertiary+residential: p95 17.8 ms GATE FAIL + 12/25 + * pairs unreachable (the dropped classes are the fine mesh tying + * landmarks to the arterials) — REJECTED on correctness, REVERTED. + * ladder (b) bidirectional A* (BH): p95 ~38 ms GATE FAIL — the + * potential split halves each side's steering, cancelling the halved + * radius; kept (sound, no worse, better on short product routes). + * (c) NOT APPLIED — breaks the $0 law; needs maintainer sign-off. The + * honest alternatives on the table: Workers Paid, or a layout-v2 ALT + * landmark graph (build-time preprocessing, still $0 at runtime). + * + * Usage: + * node routing/scripts/measure-route-cpu.mjs + * + * Exit codes: 0 p95 inside budget; 1 p95 over budget or absent binary. + */ +import { existsSync, readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { decodeGraph } from "../src/graph.ts"; +import { snapToGraph as snap } from "../src/snap.ts"; +import { routeAstar as astar } from "../src/astar.ts"; +import { getEntries } from "../../search/src/gazetteer.ts"; + +const here = dirname(fileURLToPath(import.meta.url)); // routing/scripts +const GRAPH_PATH = resolve(here, "..", "data", "kl-roadgraph.v1.bin"); +const CORPUS_PATH = resolve(here, "..", "test", "fixtures", "golden-pairs.json"); + +/** The gate: p95 budget, 20% margin under the 10 ms free-plan ceiling. */ +const P95_BUDGET_MS = 8; +/** The sample size (the plan's number). */ +const ROUTE_COUNT = 25; +/** The PRNG seed — fixed so re-runs sample identical pairs. */ +const SAMPLE_SEED = 0x06d4; + +function fail(message) { + console.error(`measure-route-cpu: ${message}`); + process.exit(1); +} + +if (!existsSync(GRAPH_PATH)) { + fail(`no graph at ${GRAPH_PATH} — build it first: node routing/scripts/build-road-graph.mjs`); +} + +function toArrayBuffer(buf) { + return buf.buffer.slice(buf.byteOffset, buf.byteOffset + buf.byteLength); +} + +const graph = decodeGraph(toArrayBuffer(readFileSync(GRAPH_PATH))); +if (graph === null) fail(`${GRAPH_PATH} does not decode (fail-honest null) — rebuild the binary`); +console.log( + `graph: ${graph.nodeCount} nodes, ${graph.edgeCount} edges, ` + + `bbox lat ${graph.bbox.minLat.toFixed(3)}..${graph.bbox.maxLat.toFixed(3)} lng ${graph.bbox.minLng.toFixed(3)}..${graph.bbox.maxLng.toFixed(3)}`, +); + +/** mulberry32 — tiny deterministic PRNG (fixed seed => comparable runs). */ +function mulberry32(seed) { + let a = seed >>> 0; + return () => { + a = (a + 0x6d2b79f5) >>> 0; + let t = a; + t = Math.imul(t ^ (t >>> 15), t | 1); + t ^= t + Math.imul(t ^ (t >>> 7), t | 61); + return ((t ^ (t >>> 14)) >>> 0) / 4294967296; + }; +} + +/* ————— pair sampling: corpus endpoints + quadrant spread ————— */ + +const corpus = JSON.parse(readFileSync(CORPUS_PATH, "utf8")); +const gazetteerById = new Map(getEntries().map((e) => [e.id, e])); + +const pairs = []; +let corpusRoutes = 0; +for (const pair of corpus.pairs) { + const dest = gazetteerById.get(pair.destinationId); + if (dest === undefined) continue; + const origin = snap(graph, { lat: pair.origin.lat, lng: pair.origin.lng }); + const goal = snap(graph, { lat: dest.lat, lng: dest.lng }); + if (origin === null || goal === null) continue; + pairs.push({ label: `corpus ${pair.pairId}`, from: origin.node, to: goal.node }); + corpusRoutes += 1; +} + +const midLat = (graph.bbox.minLat + graph.bbox.maxLat) / 2; +const midLng = (graph.bbox.minLng + graph.bbox.maxLng) / 2; +const quadrants = [[], [], [], []]; // 0=NW 1=NE 2=SW 3=SE +for (let n = 0; n < graph.nodeCount; n++) { + const q = (graph.lat[n] >= midLat ? 0 : 2) + (graph.lng[n] >= midLng ? 1 : 0); + quadrants[q].push(n); +} +const rand = mulberry32(SAMPLE_SEED); +const pick = (q) => quadrants[q][Math.floor(rand() * quadrants[q].length)]; +while (pairs.length < ROUTE_COUNT) { + const a = Math.floor(rand() * 4); + const b = (a + 1 + Math.floor(rand() * 3)) % 4; // always a DIFFERENT quadrant — cross-city + const from = pick(a); + const to = pick(b); + if (from === to) continue; + pairs.push({ label: `quadrant q${a}->q${b}`, from, to }); +} +console.log(`sample: ${corpusRoutes} corpus-snapped + ${pairs.length - corpusRoutes} quadrant pairs = ${pairs.length} routes`); + +/* ————— warm-up (unmeasured, printed) + the measured runs ————— */ + +function cpuMs(fn) { + const t0 = process.hrtime.bigint(); + const result = fn(); + const t1 = process.hrtime.bigint(); + return { ms: Number(t1 - t0) / 1e6, result }; +} + +const warm = cpuMs(() => astar(graph, pairs[0].from, pairs[0].to)); +console.log(`warm-up (unmeasured): ${warm.ms.toFixed(3)} ms`); + +const samples = []; +let unreachableProbes = 0; +let unreachableMs = 0; +for (const pair of pairs) { + const { ms, result } = cpuMs(() => astar(graph, pair.from, pair.to)); + if (result === null) { + // A directed dead-end probe (a snap onto a one-way sink or a small + // island): the honest-null degradation path. Reported SEPARATELY — it + // is not a routed trip, and mixing it into the route p95 would + // misdescribe the budget the /api/route hot path actually needs. + unreachableProbes += 1; + unreachableMs += ms; + console.log(` ${pair.label}: ${ms.toFixed(3)} ms (unreachable — honest null, D-03)`); + continue; + } + const km = `${(result.distanceM / 1000).toFixed(1)} km / ${(result.seconds / 60).toFixed(1)} min`; + console.log(` ${pair.label}: ${ms.toFixed(3)} ms (${km})`); + samples.push(ms); +} +console.log( + `routed pairs: ${samples.length}/${pairs.length}` + + (unreachableProbes > 0 + ? `; unreachable probes: ${unreachableProbes} (mean ${(unreachableMs / unreachableProbes).toFixed(3)} ms, reported separately)` + : ""), +); +if (samples.length === 0) fail("no routable pairs in the sample — the graph may be fragmented; rebuild"); + +samples.sort((a, b) => a - b); +const idx = Math.min(samples.length - 1, Math.ceil(0.95 * samples.length) - 1); +const p95 = samples[idx]; +const max = samples[samples.length - 1]; +const mean = samples.reduce((acc, v) => acc + v, 0) / samples.length; +console.log(`p95: ${p95.toFixed(3)} ms (max ${max.toFixed(3)} ms, mean ${mean.toFixed(3)} ms, n=${samples.length})`); + +if (p95 > P95_BUDGET_MS) { + console.error(`GATE FAIL: p95 ${p95.toFixed(3)} ms exceeds the ${P95_BUDGET_MS} ms budget (A1) — apply the mitigation ladder: (a) tighten DRIVABLE_CLASSES + rebuild, (b) bidirectional A*, (c) Workers Paid as an explicit fork decision`); + process.exit(1); +} +console.log(`GATE PASS: p95 ${p95.toFixed(3)} ms <= ${P95_BUDGET_MS} ms budget — A1 retired for this graph`); +process.exit(0); diff --git a/routing/scripts/upload-road-graph.mjs b/routing/scripts/upload-road-graph.mjs new file mode 100644 index 0000000..ab210e0 --- /dev/null +++ b/routing/scripts/upload-road-graph.mjs @@ -0,0 +1,78 @@ +#!/usr/bin/env node +/* + * upload-road-graph.mjs — put the built road graph on R2 (D-01, fork path). + * --------------------------------------------------------------------------- + * A MAINTAINER-TIME REPO TOOL — NOT a runtime dependency; nothing in bot/src, + * web/src, or routing/src imports it. It shells the wrangler CLI (already the + * repo's deploy tool) so a fork needs zero new installs: + * + * npx wrangler r2 object put araharah-graphs/kl-roadgraph.v1.bin + * --file routing/data/kl-roadgraph.v1.bin --remote + * + * THE FORK CONTRACT (docs/ROAD-GRAPH.md is the prose): your graph, YOUR + * bucket. Create it first (`npx wrangler r2 bucket create araharah-graphs` + * — or your own name, and point bot/wrangler.jsonc's ROAD_GRAPH binding at + * it), build the binary with build-road-graph.mjs, then run this. The + * Worker side reads the binding, never a bucket name — the name here and in + * wrangler.jsonc must simply agree. + * + * PRE-FLIGHT: the binary must EXIST and clear the enforced 1 MB floor (the + * 80 MB ceiling is build-road-graph.mjs's gate — this script re-checks both + * cheaply so a stale artifact never uploads silently). + * + * Usage: + * node routing/scripts/upload-road-graph.mjs + * + * Exit codes: 0 uploaded; 1 pre-flight failure or wrangler failure. + */ +import { execFileSync } from "node:child_process"; +import { existsSync, statSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = dirname(fileURLToPath(import.meta.url)); // routing/scripts +const GRAPH_PATH = resolve(here, "..", "data", "kl-roadgraph.v1.bin"); + +/** Must agree with bot/wrangler.jsonc's ROAD_GRAPH binding bucket_name. */ +const BUCKET = "araharah-graphs"; +/** Must agree with worker-router.ts's GRAPH_KEY. */ +const KEY = "kl-roadgraph.v1.bin"; +/** The enforced floor — an at/below-floor file is a broken build, never an upload. */ +const MIN_BINARY_BYTES = 1 * 1024 * 1024; +const MAX_BINARY_BYTES = 80 * 1024 * 1024; + +function fail(message) { + console.error(`upload-road-graph: ${message}`); + process.exit(1); +} + +if (!existsSync(GRAPH_PATH)) { + fail(`no graph at ${GRAPH_PATH} — build it first: node routing/scripts/build-road-graph.mjs`); +} +const bytes = statSync(GRAPH_PATH).size; +if (bytes <= MIN_BINARY_BYTES || bytes > MAX_BINARY_BYTES) { + fail( + `${GRAPH_PATH} is ${bytes} bytes — outside the enforced 1-80 MB envelope (D-01). ` + + `Rebuild and heed the stats gate: node routing/scripts/build-road-graph.mjs`, + ); +} +console.log(`uploading ${GRAPH_PATH} (${(bytes / (1024 * 1024)).toFixed(1)} MB) -> r2://${BUCKET}/${KEY}`); + +try { + execFileSync( + "npx", + ["wrangler", "r2", "object", "put", `${BUCKET}/${KEY}`, "--file", GRAPH_PATH, "--remote", "--content-type", "application/octet-stream"], + { stdio: "inherit" }, + ); +} catch (err) { + fail( + `wrangler failed (${err.status ?? "no status"}). If the bucket does not exist yet: ` + + `npx wrangler r2 bucket create ${BUCKET} — and if R2 is not enabled on the account, ` + + `activate it once in the Cloudflare dashboard (R2 -> Get started).`, + ); +} + +console.log(`uploaded r2://${BUCKET}/${KEY}`); +console.log("next: npm run dev -w bot # /api/route loads it through the ROAD_GRAPH binding (memoized per isolate)"); +console.log(`verify: npx wrangler r2 object get ${BUCKET}/${KEY} --file /tmp/opencode/graph-roundtrip.bin --remote && node routing/scripts/build-road-graph.mjs --check /tmp/opencode/graph-roundtrip.bin`); +process.exit(0); diff --git a/routing/src/astar.ts b/routing/src/astar.ts index 3ce295c..4d8012a 100644 --- a/routing/src/astar.ts +++ b/routing/src/astar.ts @@ -1,20 +1,17 @@ /* - * astar.ts — typed-array A* over the CSR graph + turn-step derivation (D-01). + * astar.ts — typed-array BIDIRECTIONAL A* over the CSR graph + turn-step + * derivation (D-01; bidirectional per 06-04's A1 ladder step b). * ---------------------------------------------------------------------------- * The engine-A router core: routeAstar finds the minimum-seconds path - * between two graph nodes with an ALLOCATION-FREE hot loop — open set is a - * binary heap of node indices over preallocated typed arrays, g-scores and - * the closed set are per-node typed arrays. No Map, no JS objects per node - * (RESEARCH Pattern 2 + Anti-Patterns; the 10 ms free-plan CPU budget is - * the design constraint, A1's equivalence sweep is the correctness gate). - * - * Heuristic: haversine(node, goal) / (maxSpeedKmh / 3.6) — the aerial - * distance divided by the graph's maximum speed can never overestimate the - * remaining travel time (every edge's seconds = meters / speed with speed - * <= maxSpeedKmh), so it is admissible AND consistent; settling the goal - * off the heap is therefore provably optimal (A1 proves it exhaustively on - * the fixture). The heuristic is computed on demand and cached per node in - * a Float64Array — no object is constructed in the search loop. + * between two graph nodes. The search is BIDIRECTIONAL (BH, Ikeda + * potentials — see routeAstar's doc) because the free-plan 10 ms CPU budget + * is the design constraint and the measured unidirectional p95 on the real + * KL graph was 3-5x over it. The hot loops remain allocation-free: binary + * heaps of node indices over preallocated typed arrays, typed-array + * distances/closed sets/chains, lazy on-demand potentials, no Map, no + * objects per node. Optimality is not presumed — astar.test.mjs A1 + * exhaustively proves equivalence to brute-force Dijkstra on the fixture + * (625 pairs), and the oneway/unreachable laws have dedicated cases. * * Traversal law (shared with the tests' brute-force Dijkstra): an edge with * edgeFlags bit0 set ("oneway-reverse" — see graph.ts) is NEVER relaxed. @@ -91,6 +88,11 @@ class NodeHeap { this.nodes[i] = node; } + /** The min key without popping; Infinity when empty (the BH stop rule reads this). */ + peekKey(): number { + return this.size === 0 ? Number.POSITIVE_INFINITY : this.keys[0]!; + } + /** Returns the min-key node, or -1 when empty. Lazy deletion: stale * entries are skipped by the closed-set check in routeAstar. */ pop(): number { @@ -127,12 +129,92 @@ class NodeHeap { } } -/* ————— A* ————— */ +/* ————— A* — bidirectional (BH, Ikeda potentials) ————— */ + +/** Inline haversine from a graph node to a fixed (lat, lng), in METERS. */ +function metersTo(graph: RoadGraph, v: number, lat2: number, lng2: number): number { + const lat1 = graph.lat[v]! * DEG_TO_RAD; + const lat2r = lat2 * DEG_TO_RAD; + const dLat = lat2r - lat1; + const dLng = (lng2 - graph.lng[v]!) * DEG_TO_RAD; + const sinLat = Math.sin(dLat / 2); + const sinLng = Math.sin(dLng / 2); + const h = sinLat * sinLat + Math.cos(lat1) * Math.cos(lat2r) * sinLng * sinLng; + return 2 * EARTH_RADIUS_M * Math.asin(Math.min(1, Math.sqrt(h))); +} + +/* ————— reverse adjacency (the backward search's in-edge scan) ————— */ + +/** + * The reverse CSR (in-edges per node), built ONCE per RoadGraph and + * memoized in a module-scope WeakMap — the snap-grid LAW 1 pattern. The + * backward search cannot iterate a node's in-edges out of a source-grouped + * CSR (that was the 06-04 GREEN bug the A1/A2 gates caught: out-edges read + * as in-edges fabricate phantom arcs and traverse oneway mirrors); this + * index makes them addressable without touching the pinned binary layout. + * One-time build is O(V+E) per isolate, paid by the first route (the CPU + * gate's warm-up covers it), amortized across the isolate's requests. + */ +interface ReverseAdj { + /** rowStart[u]..rowStart[u+1] index u's IN-edges inside targets/edgeIds. */ + rowStart: Uint32Array; + /** The in-edge's SOURCE node (v, for an edge v -> u). */ + targets: Uint32Array; + /** The edge's index in the forward CSR (weights/flags/seconds lookup). */ + edgeIds: Uint32Array; +} + +const reverseGraphs = new WeakMap(); + +function reverseFor(graph: RoadGraph): ReverseAdj { + const memoized = reverseGraphs.get(graph); + if (memoized !== undefined) return memoized; + const n = graph.nodeCount; + const edgeCount = graph.edgeCount; + const inDegree = new Uint32Array(n + 1); + for (let e = 0; e < edgeCount; e++) inDegree[graph.edgeTarget[e]! + 1] = inDegree[graph.edgeTarget[e]! + 1]! + 1; + for (let i = 0; i < n; i++) inDegree[i + 1] = inDegree[i + 1]! + inDegree[i]!; + const targets = new Uint32Array(edgeCount); + const edgeIds = new Uint32Array(edgeCount); + const cursor = inDegree.slice(0, n); + for (let u = 0; u < n; u++) { + for (let e = graph.edgeStart[u]!; e < graph.edgeStart[u + 1]!; e++) { + const v = graph.edgeTarget[e]!; // the edge u -> v is an IN-edge of v + const pos = cursor[v]!; + cursor[v] = pos + 1; + targets[pos] = u; + edgeIds[pos] = e; + } + } + const adj: ReverseAdj = { rowStart: inDegree, targets, edgeIds }; + reverseGraphs.set(graph, adj); + return adj; +} /** * Minimum-seconds path origin -> goal over the graph, or null when the pair * is unreachable (the honest miss). Exhaustively proven equal to * brute-force Dijkstra on the committed mini-graph (astar.test.mjs A1). + * + * BIDIRECTIONAL (A1 ladder step b, 06-04): the free-plan 10 ms CPU ceiling + * demands it — measured on the real 311k-node KL graph, the unidirectional + * search's weak max-speed heuristic balloons on 20-30 km routes (p95 38 ms). + * BH construction (Ikeda et al.): with directional heuristics + * h_f(v)=hav(v,goal)/maxSpeed and h_b(v)=hav(v,origin)/maxSpeed (each + * consistent on its own search graph) and the opposite potentials + * p_f(v) = (h_f(v) - h_b(v))/2, p_b(v) = -p_f(v), + * the reductions + * forward, edge u->v: c + p_f(u) - p_f(v) + * backward, in-edge v->u of settled u: c + p_b(u) - p_b(v) + * are NON-NEGATIVE (each feasibility is the sum of the two consistencies, + * each <= c), and the meet keys satisfy + * d_f(v) + d_b(v) = real(o->g through v) - p_f(o) + p_f(g) + * — a v-independent constant — so stopping when top_f + top_b >= mu (the + * best meeting cost seen) settles the goal PROVABLY optimally, at roughly + * half the explored radius per side. Alternation balance is a work knob, + * not a correctness knob (the stop rule carries optimality); A1's + * exhaustive 625-pair Dijkstra-equivalence sweep is the standing proof, + * re-run on every pass. */ export function routeAstar(graph: RoadGraph, originNode: number, goalNode: number): AstarResult | null { const n = graph.nodeCount; @@ -144,72 +226,143 @@ export function routeAstar(graph: RoadGraph, originNode: number, goalNode: numbe const heuristicScale = maxSpeedMps > 0 ? 1 / maxSpeedMps : 1; const goalLat = graph.lat[goalNode]!; const goalLng = graph.lng[goalNode]!; + const origLat = graph.lat[originNode]!; + const origLng = graph.lng[originNode]!; - const gScore = new Float64Array(n).fill(Number.POSITIVE_INFINITY); - const cameFrom = new Int32Array(n).fill(-1); - const closed = new Uint8Array(n); - const hCache = new Float64Array(n).fill(Number.NaN); // NaN = not computed yet - const heap = new NodeHeap(); - - // Inline haversine vs the goal, cached per node — zero allocations, and - // only explored nodes ever pay for it. Admissible: aerial distance at - // max speed upper-bounds nothing — it LOWER-bounds remaining seconds. - const heuristic = (node: number): number => { - const cached = hCache[node]!; - if (!Number.isNaN(cached)) return cached; - const lat1 = graph.lat[node]! * DEG_TO_RAD; - const lat2 = goalLat * DEG_TO_RAD; - const dLat = lat2 - lat1; - const dLng = (goalLng - graph.lng[node]!) * DEG_TO_RAD; - const sinLat = Math.sin(dLat / 2); - const sinLng = Math.sin(dLng / 2); - const h = sinLat * sinLat + Math.cos(lat1) * Math.cos(lat2) * sinLng * sinLng; - const meters = 2 * EARTH_RADIUS_M * Math.asin(Math.min(1, Math.sqrt(h))); - const seconds = meters * heuristicScale; - hCache[node] = seconds; - return seconds; - }; - - gScore[originNode] = 0; - heap.push(heuristic(originNode), originNode); - - while (heap.size > 0) { - const u = heap.pop(); - if (u < 0) break; - if (closed[u]! !== 0) continue; // lazy deletion - closed[u] = 1; - if (u === goalNode) { - // Reconstruct (allocation here is fine — the hot loop is over). - const path: number[] = [u]; - let prev = cameFrom[u]!; - while (prev >= 0) { - path.push(prev); - prev = cameFrom[prev]!; - } - path.reverse(); - let distanceM = 0; - for (let i = 1; i < path.length; i++) { - distanceM += haversineM( - { lat: graph.lat[path[i - 1]!]!, lng: graph.lng[path[i - 1]!]! }, - { lat: graph.lat[path[i]!]!, lng: graph.lng[path[i]!]! }, - ); - } - return { nodes: path, seconds: gScore[goalNode]!, distanceM }; + // Directional heuristics + opposite potentials, lazy per node — only + // explored nodes ever pay, and the hot loops stay allocation-free. + const hfCache = new Float64Array(n).fill(Number.NaN); // hav(v, goal) * scale + const hbCache = new Float64Array(n).fill(Number.NaN); // hav(v, origin) * scale + const pf = new Float64Array(n); // p_f(v) = (h_f - h_b)/2; p_b = -p_f — valid once both caches are set + function potential(v: number): number { + let hf = hfCache[v]!; + if (Number.isNaN(hf)) { + hf = metersTo(graph, v, goalLat, goalLng) * heuristicScale; + hfCache[v] = hf; } - const base = gScore[u]!; - for (let e = graph.edgeStart[u]!; e < graph.edgeStart[u + 1]!; e++) { - if ((graph.edgeFlags[e]! & 1) !== 0) continue; // oneway-reverse: never traversed - const v = graph.edgeTarget[e]!; - if (closed[v]! !== 0) continue; - const candidate = base + graph.edgeSeconds[e]!; - if (candidate < gScore[v]!) { - gScore[v] = candidate; - cameFrom[v] = u; - heap.push(candidate + heuristic(v), v); + let hb = hbCache[v]!; + if (Number.isNaN(hb)) { + hb = metersTo(graph, v, origLat, origLng) * heuristicScale; + hbCache[v] = hb; + } + const value = (hf - hb) / 2; + pf[v] = value; + return value; + } + + // Reduced distances (BH keys), chain arrays, closed sets, one heap per side. + const df = new Float64Array(n).fill(Number.POSITIVE_INFINITY); // reduced origin->v + const db = new Float64Array(n).fill(Number.POSITIVE_INFINITY); // reduced v->goal + const fPrev = new Int32Array(n).fill(-1); // predecessor on the origin->v chain + const bNext = new Int32Array(n).fill(-1); // successor on the v->goal chain + const fClosed = new Uint8Array(n); + const bClosed = new Uint8Array(n); + const fHeap = new NodeHeap(); + const bHeap = new NodeHeap(); + + df[originNode] = 0; + potential(originNode); // pf[o] is needed by the first forward relaxations + fHeap.push(0, originNode); // key = the REDUCED distance itself (the heuristic lives in the reduction) + db[goalNode] = 0; + potential(goalNode); // p_b(g) likewise for the first backward relaxations + bHeap.push(0, goalNode); + + let mu = Number.POSITIVE_INFINITY; // best meeting key sum (real + const) + let meetFrom = -1; // bridge edge meetFrom -> meetTo (direction of travel) + let meetTo = -1; + + while (fHeap.size > 0 && bHeap.size > 0) { + // THE BH STOP RULE: with consistent potentials, no future meeting can + // beat mu once both frontiers' minima sum past it. + if (fHeap.peekKey() + bHeap.peekKey() >= mu) break; + // BALANCED ALTERNATION: expand the side with the smaller frontier. + // (Measured 06-04: comparing raw tops starves the backward side — its + // keys start near the full route cost while the forward side's start + // near zero, so `top_f <= top_b` ran the forward search almost alone + // (19501:3 settles) and landed 4x SLOWER than unidirectional. Heap + // size keeps both rings growing; correctness lives entirely in the + // stop rule — alternation order is a work-balancing knob only.) + const expandForward = fHeap.size <= bHeap.size; + const u = (expandForward ? fHeap : bHeap).pop()!; + if (expandForward ? fClosed[u]! !== 0 : bClosed[u]! !== 0) continue; // lazy deletion + if (expandForward) fClosed[u] = 1; + else bClosed[u] = 1; + + if (expandForward) { + const du = df[u]!; + const pfu = pf[u]!; + for (let e = graph.edgeStart[u]!; e < graph.edgeStart[u + 1]!; e++) { + if ((graph.edgeFlags[e]! & 1) !== 0) continue; // oneway-reverse: never traversed + const v = graph.edgeTarget[e]!; + const pv = potential(v); + const candidate = du + graph.edgeSeconds[e]! + pfu - pv; + if (candidate < df[v]!) { + df[v] = candidate; + fPrev[v] = u; + fHeap.push(candidate, v); + } + // Meet probe: v already reached backward? (candidate uses v's BEST d_b) + if (db[v]! < Number.POSITIVE_INFINITY) { + const meet = candidate + db[v]!; + if (meet < mu) { + mu = meet; + meetFrom = u; + meetTo = v; + } + } + } + } else { + const rev = reverseFor(graph); + const du = db[u]!; + const pfu = pf[u]!; + const end = rev.rowStart[u + 1]!; + for (let r = rev.rowStart[u]!; r < end; r++) { + const e = rev.edgeIds[r]!; // the edge v -> u + if ((graph.edgeFlags[e]! & 1) !== 0) continue; // oneway-reverse: never traversed + const v = rev.targets[r]!; + const pvv = potential(v); + const candidate = du + graph.edgeSeconds[e]! - pfu + pvv; + if (candidate < db[v]!) { + db[v] = candidate; + bNext[v] = u; + bHeap.push(candidate, v); + } + if (df[v]! < Number.POSITIVE_INFINITY) { + const meet = candidate + df[v]!; + if (meet < mu) { + mu = meet; + meetFrom = v; // the path travels v -> u here + meetTo = u; + } + } } } } - return null; // unreachable — honest miss + + if (meetFrom < 0) return null; // never met — unreachable (honest miss) + + // Reconstruct: origin ->(fPrev)-> meetFrom -[bridge]-> meetTo ->(bNext)-> goal. + const fPart: number[] = []; + for (let x = meetFrom; x >= 0; x = fPrev[x]!) fPart.push(x); + fPart.reverse(); + const bPart: number[] = []; + for (let y = meetTo; y >= 0; y = bNext[y]!) bPart.push(y); + const path = fPart.concat(bPart); + + // REAL costs recomputed along the node path (exact; no reduced-cost + // arithmetic leaks into the returned seconds). + let seconds = 0; + let distanceM = 0; + for (let i = 1; i < path.length; i++) { + const a = path[i - 1]!; + const b = path[i]!; + const from = { lat: graph.lat[a]!, lng: graph.lng[a]! }; + const to = { lat: graph.lat[b]!, lng: graph.lng[b]! }; + distanceM += haversineM(from, to); + const edge = findTraversableEdge(graph, a, b); + seconds += edge >= 0 ? graph.edgeSeconds[edge]! : 0; // -1 only on a corrupt chain — loud in deriveSteps + } + return { nodes: path, seconds, distanceM }; } /* ————— turn classification (the one authority; plan 07 imports this) ————— */