araharah-executor
720ad43765
Task 4, re-scoped per the plan's honest-FAIL branch: the c3 class is NOT retired — every record now states the measured post-ALT truth. - KNOWN-LIMITATIONS: the cross-metro section is the /cari-shaped record — WHAT CHANGED (ALT built, D-03 optimality green quoted) / what REMAINS (the CPU verdict FAIL lines quoted with date; the one-directional- Lipschitz wall named; the escalation open); the roadmap-pointer ALT row flips to Built (Phase 8) with the measured verdict — never 'not built' again - ROAD-GRAPH: the ladder table gains the (c2) rung (EXECUTED, measured FAIL, the L=32 probe quoted); the decision paragraph states c2 executed-verdict-fail and names the real remaining options; new 'The ARG2 binary layout' section (72-byte header, landmarkCount/ landmarkQuantumS, u16 suffix + sentinels, the quantum knob's ≈6,553 s ceiling, the storage-name-vs-format-truth law); key facts carry the optimality gate and the ARG2 wording; the fork contract's build step teaches the landmark stage (no new commands) - docs/CONFIGURATION.md: verify-only — all path/script claims still true, no edit needed - staged as selective hunks; the tree's pre-existing doc-sync edits (battery paragraph, 'six maintainer scripts' intro) remain uncommitted greps: 'Next-milestone item, not built' = 0; the measured p95 strings quoted; (c2)/landmarkQuantumS/ARG2 all present; ARG1 appears only in layout/router-pending contexts.
169 lines
8.8 KiB
Markdown
169 lines
8.8 KiB
Markdown
# The Road Graph (engine A's data) — build, host, and the laws
|
||
|
||
Engine A routes over a **CSR road-graph binary** built from OpenStreetMap and
|
||
hosted on Cloudflare R2. The binary is a **build artifact, not a committed
|
||
file** — forks build and upload their own. Everything needed is committed:
|
||
two maintainer scripts and this doc. All of it is **$0** (Overpass public
|
||
instance, R2 free tier, Workers free plan).
|
||
|
||
## The fork contract (your graph, your bucket)
|
||
|
||
1. **Create your bucket** (once, per Cloudflare account):
|
||
|
||
```
|
||
npx wrangler r2 bucket create araharah-graphs
|
||
```
|
||
|
||
If `wrangler` reports that R2 is not enabled, activate it once in the
|
||
Cloudflare dashboard (**R2 → Get started**) — account-owner steps, then
|
||
re-run the command. Use a different name if you like, but the name must
|
||
agree with `bot/wrangler.jsonc`'s `ROAD_GRAPH` binding (`bucket_name`).
|
||
|
||
2. **Build the KL graph** (or your own region — tune `DRIVABLE_CLASSES` /
|
||
the bbox in the script):
|
||
|
||
```
|
||
node routing/scripts/build-road-graph.mjs
|
||
```
|
||
|
||
Computes the bbox from the committed corpus (gazetteer + golden pairs,
|
||
± 0.05°), fetches ONE Overpass extract (drivable classes only), runs the
|
||
ALT preprocessing stage through the ONE landmark module (the build prints
|
||
L, quantum, per-anchor coverage, table bytes — L=24 at KL's ~311k nodes
|
||
by the D-02 rule; `--landmarks <n>` overrides for measured retries),
|
||
encodes the pinned CSR binary through `encodeGraph`, and writes
|
||
`routing/data/kl-roadgraph.v1.bin` — **only** after the encode round-trip
|
||
and the size gate pass. Exit 1 leaves any previous binary untouched.
|
||
|
||
3. **Check the artifact** (decode + stats, no network):
|
||
|
||
```
|
||
node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin
|
||
```
|
||
|
||
4. **Upload to R2**:
|
||
|
||
```
|
||
node routing/scripts/upload-road-graph.mjs
|
||
# = npx wrangler r2 object put araharah-graphs/kl-roadgraph.v1.bin \
|
||
# --file routing/data/kl-roadgraph.v1.bin --remote
|
||
```
|
||
|
||
Verify the round-trip:
|
||
|
||
```
|
||
npx wrangler r2 object get araharah-graphs/kl-roadgraph.v1.bin \
|
||
--file /tmp/opencode/graph-roundtrip.bin --remote
|
||
node routing/scripts/build-road-graph.mjs --check /tmp/opencode/graph-roundtrip.bin
|
||
```
|
||
|
||
5. **Deploy the Worker** (`npm run deploy -w bot`). `wrangler.jsonc` binds
|
||
the bucket as `ROAD_GRAPH`; `worker-router.ts` loads the object
|
||
`kl-roadgraph.v1.bin` once per cold isolate (module-scope memo, shared
|
||
with the cron arm) — 1 R2 GET, not per request.
|
||
|
||
**Dev note:** with no bucket (or an empty one), `/api/route` answers `200
|
||
null` — the designed router-pending state (D-03). Dev and tests never need
|
||
the real binary; tests use the committed mini-graph fixture.
|
||
|
||
## The envelope law (D-01)
|
||
|
||
| Gate | Value | Enforcement |
|
||
|------|-------|-------------|
|
||
| Ceiling | **≤ 80 MB** | enforced — build + upload + `--check` all exit 1 over it |
|
||
| Floor | **> 1 MB** | enforced — a smaller "road graph" is a broken build |
|
||
| Expected band | **30–80 MB** | advisory — a smaller build prints a WARNING (verify the class filter) and continues |
|
||
|
||
The binary lives in `routing/data/` and is gitignored (`*.bin`): it is
|
||
ODbL-derived data, regenerated at will, never source.
|
||
|
||
## The CPU gate (A1) — OPEN: measured over the free-plan budget, post-ALT
|
||
|
||
Workers free plan: **10 ms CPU per request** (Error 1102 beyond it). The A\*
|
||
cost over the REAL graph is measured, never hoped:
|
||
|
||
```
|
||
node routing/scripts/measure-route-cpu.mjs
|
||
```
|
||
|
||
25 pairs (golden-corpus routes + quadrant-sampled cross-city pairs; directed
|
||
dead-end probes reported separately) timed around `routeAstar` over the real
|
||
binary. **The gate is p95 ≤ 8 ms** (20% margin under the ceiling).
|
||
|
||
**Measured state (2026-10-25, the 311k-node ARG2 graph): GATE FAIL.** The
|
||
mitigation ladder was executed in order:
|
||
|
||
| Ladder step | Outcome |
|
||
|-------------|---------|
|
||
| full graph, unidirectional A\* | p95 ≈ 38 ms — FAIL |
|
||
| (a) drop tertiary+residential, rebuild | p95 ≈ 18 ms — FAIL **and** 12/25 pairs became unreachable (the dropped classes are the fine mesh tying landmarks to arterials) — rejected on correctness, reverted |
|
||
| (b) bidirectional A\* (BH, Ikeda potentials) | p95 ≈ 38 ms — FAIL: the potential split halves each side's heuristic steering, cancelling the halved search radius. Kept: sound (A1's 625-pair Dijkstra equivalence still passes) and better on short product routes |
|
||
| (c2) ALT landmark preprocessing (Phase 8) | **EXECUTED 2026-10-25 — FAIL: p95 152.788 ms** (25-route; n=23) and **185.692 ms** (the D-04 extended 100-pair cross-metro verdict; n=53), both on the shipped L=24 build. The `--landmarks 32` probe measured **worse** (182.725 / 204.499 ms) — under the sound steering, more anchors buy no pruning and cost per-node work. Optimality verified separately: the D-03 gate reproduces the frozen pre-ALT record within ±0.1% (53/53 routed pairs). Kept: the steering is sound (A1 still passes; the A11/A12 pins forbid the unsound four-family variant) |
|
||
| (c) Workers Paid | **NOT applied — breaks the $0 law; needs a maintainer decision** |
|
||
|
||
The decision this gate now escalates: **(c1) Workers Paid ($5/mo — breaks
|
||
$0)**, **direction-split potentials with a re-proven bidirectional meet rule
|
||
(the fromTable triangle bounds are one-directional Lipschitz — their u-side
|
||
term needs the reverse edge, which a oneway does not provide — so the shared
|
||
Ikeda average p = (h_f − h_b)/2 cannot carry them; that search-skeleton work
|
||
is a new phase)**, **Contraction Hierarchies (the Phase-6 deferral)**, or
|
||
**(c3) keep accepting free-plan routing with honest degradation** (long
|
||
reroutes may hit 1102; the client's `createHttpRoutingSource` then degrades
|
||
to the router-pending state). Short product routes (2–8 km, the common
|
||
reroute) resolve inside budget; the failures concentrate on 20–30 km
|
||
cross-metro routes. Measured numbers live in `docs/BUDGETS.md`'s c3 section.
|
||
|
||
## The ARG2 binary layout (Phase 8)
|
||
|
||
The binary's header says **version 2** ("ARG2"); all ARG1 node/edge/name
|
||
sections keep their absolute offsets, and the landmark tables ride as a
|
||
suffix:
|
||
|
||
- **Header, 72 bytes** (was 64): the extra fields are `landmarkCount` u32
|
||
(@60 — ≥ 1 REQUIRED; a landmark-less ARG2 refuses to decode, D-01's
|
||
no-silent-degradation law), `landmarkQuantumS` f32 (@64 — the ĉ unit in
|
||
seconds), and reserved padding to the 8-byte-aligned size.
|
||
- **Suffix**: two u16 tables of `landmarkCount × nodeCount` cells —
|
||
`landmarkTo` (d̂(v, ℓ): v → anchor) then `landmarkFrom` (d̂(ℓ, v)) — plus
|
||
`landmarkNodes` u32[L] (the anchors, for audits). A cell value 0xFFFF is
|
||
DATA (the unreachable sentinel), never validated away; the search reads
|
||
it raw as u16 before any float conversion.
|
||
- **The quantum knob**: distances are stored in 0.1 s quanta, so a u16 cell
|
||
expresses up to ≈ 6,553 s. The build fails loud past 65,000 quanta — a
|
||
larger-region fork raises `landmarkQuantumS` (e.g. 0.2 s ⇒ ≈ 3.6 h) and
|
||
rebuilds; never clamp.
|
||
- **Storage name vs format truth**: the R2 key stays
|
||
`kl-roadgraph.v1.bin` (the upload path is locked); the "v1" is a storage
|
||
NAME — the header's version field is the only format truth. Old ARG1
|
||
bytes decode to `null` → the designed router-pending state until the
|
||
rebuilt binary is uploaded once.
|
||
- The search consumes only the `landmarkTo` family (two-sided-Lipschitz —
|
||
see the CPU-gate section for why the `landmarkFrom` family is excluded
|
||
from steering); `landmarkFrom` remains in the layout for audits and
|
||
future search designs.
|
||
|
||
## Licensing (the ODbL attribution)
|
||
|
||
The graph is derived from OpenStreetMap data —
|
||
**© OpenStreetMap contributors, ODbL 1.0**
|
||
(<https://opendatacommons.org/licenses/odbl/1-0/>), fetched via the public
|
||
Overpass API. Hosting or redistributing the binary requires the same
|
||
attribution (ODbL's attribution clause travels with derived databases). The
|
||
route fixtures carry the same notice inside their committed envelope
|
||
(`_license`/`_attribution`/`_notice`), and this doc is the notice surface for
|
||
the hosted binary. Keep it intact in forks.
|
||
|
||
## Key facts
|
||
|
||
| What | Where |
|
||
|------|-------|
|
||
| Build tool (Overpass → CSR + ALT preprocessing) | `routing/scripts/build-road-graph.mjs` |
|
||
| Upload tool (wrangler shell) | `routing/scripts/upload-road-graph.mjs` |
|
||
| CPU measurement gate | `routing/scripts/measure-route-cpu.mjs` |
|
||
| Optimality gate (D-03) | `routing/scripts/measure-optimality-gate.mjs` |
|
||
| Binary (gitignored artifact) | `routing/data/kl-roadgraph.v1.bin` |
|
||
| R2 object key | `kl-roadgraph.v1.bin` (pinned in `worker-router.ts` as `GRAPH_KEY`) |
|
||
| Binding | `ROAD_GRAPH` in `bot/wrangler.jsonc` (bucket, NEVER a KV namespace — GE5) |
|
||
| Loader (isolate-memoized) | `bot/src/routing/worker-router.ts` → `loadGraph` |
|
||
| Binary layout authority | `routing/src/graph.ts` (`encodeGraph`/`decodeGraph`, pinned CSR — header version 2, the ARG2 layout above) |
|