201 lines
18 KiB
Markdown
201 lines
18 KiB
Markdown
---
|
||
phase: 08-alt-landmark-preprocessing-routing-backend-v2
|
||
plan: 04
|
||
type: execute
|
||
wave: 4
|
||
depends_on: ["08-03"]
|
||
files_modified:
|
||
- routing/scripts/build-road-graph.mjs
|
||
autonomous: true
|
||
requirements: [SC3, SC4]
|
||
must_haves:
|
||
truths:
|
||
- "The build computes landmark tables at build time through the ONE landmark module (farthestPointAnchors + landmarkTables + landmarkCoverage from ../src/landmark.ts) — no second Dijkstra implementation ever exists in the script (single-authority law)"
|
||
- "L follows the D-02 scaling rule clamp(round(N/13000), 16, 32) — node-count-proportional, ~24 at KL's 311k nodes — with a maintainer --landmarks override as the measured campaign knob; anchors are road-network nodes (NOT POIs), deterministic"
|
||
- "Build fails honestly (fail() → exit 1, previous binary untouched — collect-then-write) when any anchor's coverage < LANDMARK_MIN_COVERAGE_FRACTION or landmarkTables throws the quantum-ceiling error (raise the quantum, never clamp)"
|
||
- "--check prints the landmark facts (count, quantum, coverage) decoded from the binary; the stats gate prints L/quantum/coverage/table-bytes BEFORE the write"
|
||
artifacts:
|
||
- path: "routing/scripts/build-road-graph.mjs"
|
||
provides: "the preprocessing stage between edge compaction and encode; the landmarks block into the one encodeGraph; extended stats + --check prints; the --landmarks override"
|
||
key_links:
|
||
- from: "routing/scripts/build-road-graph.mjs preprocessing stage"
|
||
to: "routing/src/landmark.ts"
|
||
via: "dynamic import of LANDMARK_QUANTUM_S + farthestPointAnchors + landmarkTables + landmarkCoverage — the only table implementation"
|
||
pattern: "farthestPointAnchors"
|
||
- from: "routing/scripts/build-road-graph.mjs encodeGraph call"
|
||
to: "routing/src/graph.ts (Plan 08-02's required landmarks block)"
|
||
via: "landmarks: { quantumS, nodes: anchors, to, from } — the one authority, no v2-only encode"
|
||
pattern: "landmarks:\\s*\\{"
|
||
- from: "routing/scripts/build-road-graph.mjs --check"
|
||
to: "graph.landmarkCount / landmarkQuantumS (decoded fields)"
|
||
via: "landmark stats in the --check print block"
|
||
pattern: "landmarkCount"
|
||
---
|
||
|
||
<objective>
|
||
The build-side half of ALT: a preprocessing stage in `build-road-graph.mjs` that quantizes nothing itself, implements nothing itself — it feeds the graph's edges to the one landmark module, gates the result honestly, and encodes ARG2 through the one authority — then rebuilds the REAL KL binary with it.
|
||
|
||
Purpose: the byte payload this plan produces is what the measurement campaign (08-05) gates on; the D-02 scaling rule and the coverage/quantum gates live here so no gate can be skipped at campaign time.
|
||
Output: staged `build-road-graph.mjs` + a real, checked-green ARG2 `routing/data/kl-roadgraph.v1.bin` on disk.
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@$HOME/.config/opencode/get-shit-done/workflows/execute-plan.md
|
||
@$HOME/.config/opencode/get-shit-done/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/PROJECT.md
|
||
@.planning/STATE.md
|
||
@.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-CONTEXT.md
|
||
@.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-RESEARCH.md
|
||
@.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-PATTERNS.md
|
||
@.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VALIDATION.md
|
||
|
||
<interfaces>
|
||
<!-- The build tool being staged. Extracted from routing/scripts/build-road-graph.mjs 2026-10-25 (375 lines — read it in full first). -->
|
||
|
||
Structure to fit (everything is named-constant + fail-honest + collect-then-write already):
|
||
- Constants block with rationale doc-comments: lines 56–128 (MAX_BINARY_BYTES 80 MB, EXPECTED_MIN_BYTES 30 MB advisory, MIN_BINARY_BYTES 1 MB, the A1 LADDER HISTORY comment at 80–92 — ALT constants join this block and the ladder comment gains rung (c2)).
|
||
- `function fail(message)` → console.error + exit 1 (lines 130–133).
|
||
- `--check <file>` mode: lines 137–159 — decodeGraph + stats print + envelope checks, NO network. It currently prints nodeCount/edgeCount/nameCount/maxSpeedKmh/bbox/bytes.
|
||
- Edge assembly ends at line 331 (`edges` final after degree-0 compaction; shape `{ from, to, seconds, flags, nameId }`).
|
||
- Encode: line 336 `const buffer = encodeGraph({ lat: cLat, lng: cLng, edges, names, maxSpeedKmh: maxSeenKmh })`.
|
||
- Stats gate: lines 339–367 (print-every-stat, then the enforced 1–80 MB gates, then the advisory band WARNING).
|
||
- Collect-then-write: lines 369–375 — writeFileSync happens ONLY after every gate passed.
|
||
|
||
landmark.ts (Plan 08-02 task 1) exports consumed here:
|
||
```ts
|
||
LANDMARK_QUANTUM_S = 0.1;
|
||
farthestPointAnchors(input: LandmarkInput, count: number, opts?): Uint32Array; // deterministic, degree-≥2-filtered
|
||
landmarkTables(input: LandmarkInput, anchors: Uint32Array): { to: Uint16Array; from: Uint16Array };
|
||
// LandmarkInput = { nodeCount, edges: [{from,to,seconds,flags}], quantumS }
|
||
// THROWS when any quantized distance > 65,000 (build must catch → fail() with the raise-quantum hint)
|
||
landmarkCoverage(to: Uint16Array, anchors: Uint32Array, nodeCount: number): number[]; // per-anchor reachable fraction
|
||
```
|
||
encodeGraph (Plan 08-02) REQUIRES `landmarks: { quantumS: number; nodes: ArrayLike<number>; to: Uint16Array; from: Uint16Array }` — a v1-shaped call now fails loudly.
|
||
|
||
Measured real-graph facts (Phase 6 build, 2026-10-24): 311,325 nodes / 677,068 directed edges / 14.7 MB base; ONE giant SCC covering ~305k of 311k nodes (coverage headroom: 0.98 ≫ the 0.90 gate); worst distance ≈ 5,400 s ≈ 54,000 quanta < 65,000 ceiling. Scaling rule → round(311,325 / 13,000) ≈ 24 landmarks. Byte budget: ~1.19 MB/landmark → ~42.8 MB total at L=24 — inside the enforced envelope AND the 30–80 MB advisory band (the below-band WARNING should disappear).
|
||
</interfaces>
|
||
</context>
|
||
|
||
<tasks>
|
||
|
||
<task type="auto">
|
||
<name>task 1: The preprocessing stage + --check landmark facts (code)</name>
|
||
<files>routing/scripts/build-road-graph.mjs</files>
|
||
<read_first>
|
||
- routing/scripts/build-road-graph.mjs IN FULL (375 lines — you are inserting one stage and extending two print blocks; the fetch/assembly/collect-then-write laws are pinned)
|
||
- routing/src/landmark.ts IN FULL (Plan 08-02's module — the exact LandmarkInput shape and the quantum-ceiling throw message you catch)
|
||
- .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-RESEARCH.md (D-02 scaling rule; Pattern 2 — the script never quantizes, the module does; Pitfall 5 — the quantum knob doctrine; Open Question 1 — the parameterize-L recommendation this task implements)
|
||
- .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-PATTERNS.md (build-road-graph section — named-constant law, fail() shape, gate-then-write block)
|
||
- routing/test/fixtures/build-mini-graph.mjs (Plan 08-02 task 3 — the tiny-L precedent of the same stage shape)
|
||
</read_first>
|
||
<action>
|
||
One stage, two print extensions, one override — nothing else moves.
|
||
|
||
1. Named constants join the constants block (rationale doc-comments, measured justification, ladder-history style):
|
||
```js
|
||
/** D-02 scaling rule: node-count-proportional landmark count (NOT a fixed
|
||
* generous constant — per-landmark bytes scale with N, so over-provisioning
|
||
* explodes at Malaysia scale where ALT's edge itself narrows). */
|
||
const LANDMARK_COUNT_MIN = 16;
|
||
const LANDMARK_COUNT_MAX = 32;
|
||
const NODES_PER_LANDMARK = 13000; // 311,325 / 13,000 ≈ 24 at KL scale
|
||
/** Build-time coverage gate: every anchor's ĉ-Dijkstra must reach at least
|
||
* this fraction of nodes (the giant SCC covers ~0.98 — headroom is honest). */
|
||
const LANDMARK_MIN_COVERAGE_FRACTION = 0.9;
|
||
```
|
||
2. CLI override parsed BEFORE the `--check` dispatch: `--landmarks <n>` — a positive integer, hard-capped at 128 (fail-loud beyond; the knob exists for measured retries within/at the top of the rule, not for unbounded table bombs). Usage doc-comment gains the flag.
|
||
3. THE STAGE — after the degree-0 compaction (line 331) and BEFORE the encode (line 336):
|
||
- `const { LANDMARK_QUANTUM_S, farthestPointAnchors, landmarkTables, landmarkCoverage } = await import("../src/landmark.ts");`
|
||
- `const landmarkCount = landmarksOverride ?? Math.min(LANDMARK_COUNT_MAX, Math.max(LANDMARK_COUNT_MIN, Math.round(cLat.length / NODES_PER_LANDMARK)));` (print the rule's inputs and the chosen L)
|
||
- Project the edges ONCE for the module: `const lmEdges = edges.map((e) => ({ from: e.from, to: e.to, seconds: e.seconds, flags: e.flags }));` (the module's input shape; no mutation of the encode-side array).
|
||
- `const anchors = farthestPointAnchors({ nodeCount: cLat.length, edges: lmEdges, quantumS: LANDMARK_QUANTUM_S }, landmarkCount);`
|
||
- `const { to, from } = landmarkTables({ nodeCount: cLat.length, edges: lmEdges, quantumS: LANDMARK_QUANTUM_S }, anchors);` wrapped in try/catch → `fail("landmark tables exceeded the u16 quantum ceiling (<message>) — raise LANDMARK_QUANTUM_S in routing/src/landmark.ts and rebuild (Pitfall 5)")` on the module's throw.
|
||
- Coverage gate: `const coverage = landmarkCoverage(to, anchors, cLat.length);` → `min < LANDMARK_MIN_COVERAGE_FRACTION` ⇒ `fail(...)` naming L, the offending anchor index/id, its fraction, and the giant-SCC expectation.
|
||
- Sentinel audit print: total 0xFFFF entries per family (cheap honesty; --check mirrors it).
|
||
4. Encode passes the block: `encodeGraph({ lat: cLat, lng: cLng, edges, names, maxSpeedKmh: maxSeenKmh, landmarks: { quantumS: LANDMARK_QUANTUM_S, nodes: anchors, to, from } })` — the ONE authority; no local fallback encode, no try-to-encode-without-landmarks branch.
|
||
5. Stats print extension (the gate-then-write block): landmark lines — count, quantum, min/mean coverage, table bytes (`2·L·N·2 + 4·L`), sentinel totals — BEFORE the size gates. The existing envelope gates are UNCHANGED (80 MB ceiling / 1 MB floor / 30 MB advisory).
|
||
6. `--check` extension: the decoded graph now carries `landmarkCount` / `landmarkQuantumS` / `landmarkNodes` — print them plus the per-anchor coverage (import landmarkCoverage; the u16 tables are on the decoded graph) and the sentinel totals. No network.
|
||
7. Header doc-comment: the A1 LADDER HISTORY block gains rung (c2) — "EXECUTED in this script's preprocessing stage (Phase 8): build-time landmark tables + ALT steering; the verdict lives in measure-route-cpu.mjs's history and docs/ROAD-GRAPH.md". Usage block documents `--landmarks`.
|
||
</action>
|
||
<verify>
|
||
<automated>export PATH="$HOME/.nvm/versions/node/v24.18.0/bin:$PATH" && node --check routing/scripts/build-road-graph.mjs && node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin 2>&1 | tail -2; test $? -eq 1 && echo "ARG1-HONEST-REFUSE-OK"; grep -c "farthestPointAnchors" routing/scripts/build-road-graph.mjs</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- grep gates: `farthestPointAnchors` ≥ 1; `landmarkTables` ≥ 1; `landmarkCoverage` ≥ 2 (stage + --check); `LANDMARK_MIN_COVERAGE_FRACTION` ≥ 2 (const + gate); `NODES_PER_LANDMARK` ≥ 1; `--landmarks` ≥ 2 (parse + usage doc).
|
||
- The encodeGraph call carries `landmarks: {` (grep ≥ 1); the script contains NO local Dijkstra/priority-queue implementation (`grep -ci "dijkstra\|priorityqueue" routing/scripts/build-road-graph.mjs` counts only doc-comment mentions of the module's Dijkstras — zero code hits).
|
||
- `--check routing/data/kl-roadgraph.v1.bin` exits 1 at this commit with the decode-null honest message — the on-disk binary is STILL ARG1 (the rebuild is task 2); the fail-honest path is proven, not assumed.
|
||
- Collect-then-write preserved: `writeFileSync` appears exactly once, after the gates (grep -c = 1).
|
||
- `node --check` (syntax) passes.
|
||
</acceptance_criteria>
|
||
<done>The build tool now produces ARG2 bytes with build-time tables through the one landmark module and the one encode authority, gated honestly on coverage and quantum ceiling.</done>
|
||
</task>
|
||
|
||
<task type="auto">
|
||
<name>task 2: The real rebuild (network, maintainer-time) — an ARG2 KL binary on disk</name>
|
||
<files></files>
|
||
<read_first>
|
||
- The staged script (task 1 of this plan) — the exact stdout shape you will quote
|
||
- .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-RESEARCH.md (Environment Availability — Overpass public API, REQUEST_TIMEOUT_MS 300 s, one request; A4 build-cost ≈ 2L Dijkstra passes ≈ seconds-class)
|
||
- .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-CONTEXT.md (D-02 — the count is measured-justified; D-01 — same output path/key)
|
||
</read_first>
|
||
<action>
|
||
Run the REAL build and check the artifact. These are maintainer-time commands with a network fetch (minutes); quote the full landmark stats block in the summary (print-the-measurement law).
|
||
|
||
1. `node routing/scripts/build-road-graph.mjs` — expected stdout: the bbox line (identical bbox to the 2026-10-24 build — it is computed from the COMMITTED corpus ± margin), the fetch, the graph-stats block, the NEW landmark block (L = 24 by the rule at ~311k nodes; quantum 0.1; coverage min/mean; table bytes; sentinel totals), the envelope gates green, the write line. Expected size ≈ 43 MB (14.7 MB base + ~28 MB tables at L=24) — inside the envelope AND the advisory band; the old below-band WARNING should NOT appear.
|
||
2. `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` — expected exit 0 printing nodeCount/edgeCount + landmarkCount/quantum/coverage/sentinels.
|
||
3. Record in the summary: L used, per-anchor coverage min/mean, table bytes, total MB, node/edge counts. These are the campaign's build facts (08-05 compares record-vs-current counts from them).
|
||
4. HONEST FAILURE PATHS — do not paper over any of these:
|
||
- Overpass unreachable/timeout → the script exits 1, the previous binary untouched. STOP: record the failure; the campaign plans need the real binary. Do NOT fabricate, shrink the query, or retry beyond the script's own law.
|
||
- Coverage gate FAIL → the build refuses (exit 1). Do NOT lower LANDMARK_MIN_COVERAGE_FRACTION to force a pass — investigate (fragmented extract?) and report.
|
||
- Quantum ceiling throw → raise nothing silently; report (KL is measured-safe at 54,000/65,000 units — a throw means something is wrong, not tight).
|
||
5. Do NOT run the CPU gates here — every verdict belongs to the campaign (08-05). This plan ends at a checked-green artifact.
|
||
</action>
|
||
<verify>
|
||
<automated>export PATH="$HOME/.nvm/versions/node/v24.18.0/bin:$PATH" && node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin 2>&1 | tail -8 && echo "REBUILD-CHECK-GREEN"</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- `--check` exits 0 on `routing/data/kl-roadgraph.v1.bin` and prints `landmarkCount` ≥ 16 with quantum 0.1 and per-anchor coverage ≥ 0.9.
|
||
- The binary size is inside the enforced 1–80 MB envelope (expected ≈ 43 MB; the printed `binary size` line is quoted in the summary).
|
||
- The build's stdout (stats + landmark block + write line) is quoted in the summary — the measured facts exist outside the terminal.
|
||
- `git status`: routing/src/** and routing/test/** UNCHANGED by this task (a build run touches only the gitignored artifact — any source drift here is a plan violation).
|
||
</acceptance_criteria>
|
||
<done>An ARG2 KL binary exists locally, built through the staged pipeline with the D-02 rule's L, checked green, with every gate number printed and quoted.</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<threat_model>
|
||
## Trust Boundaries
|
||
|
||
| Boundary | Description |
|
||
|----------|-------------|
|
||
| Overpass extract → build | the fetch is remote data; a drifted/corrupt extract must not silently become a hosted binary |
|
||
| --landmarks override → table size | a fat-fingered count must not produce an envelope-busting or table-bomb binary |
|
||
|
||
## STRIDE Threat Register
|
||
|
||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||
|-----------|----------|-----------|-------------|-----------------|
|
||
| T-08-09 | Tampering | drifted/corrupt extract → subtly wrong binary | mitigate | the existing envelope gates + the NEW coverage/quantum gates run before any write; the D-03 optimality gate (08-05) then compares seconds against the frozen record — count drift prints a WARNING there; nothing hosts until the campaign passes |
|
||
| T-08-10 | DoS | build runtime / table size | mitigate | stage cost ≈ 2L Dijkstra passes (seconds-class, A4); `--landmarks` hard-capped at 128; the 80 MB envelope gate fires before any write regardless of L |
|
||
| T-08-11 | Repudiation | forcing a pass by lowering L or the coverage bar | mitigate | the coverage gate uses the same fraction for ANY L; the override cannot lower it; every build prints L + coverage — the summary quotes them, so a gamed build is visible in the record |
|
||
| T-08-12 | Information disclosure | build stdout | accept | prints counts/coverage/km-class stats of PUBLIC OSM data — no user coordinates exist in the build path |
|
||
</threat_model>
|
||
|
||
<verification>
|
||
- `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` exit 0 with landmark facts.
|
||
- `npm run routing:test` still green (the suite never imports the build script; this proves no accidental coupling).
|
||
- The summary quotes the full build stats + landmark block.
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
build-road-graph.mjs gains the preprocessing stage with the D-02 scaling rule, the honest coverage/quantum gates, and the --landmarks measured knob — implementing nothing the landmark module doesn't already — and a real ARG2 KL binary sits on disk, checked green, with every gate number printed and quoted in the summary.
|
||
</success_criteria>
|
||
|
||
<output>
|
||
After completion, create `.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-04-SUMMARY.md`
|
||
</output>
|