150 lines
14 KiB
Markdown
150 lines
14 KiB
Markdown
---
|
||
phase: 08-alt-landmark-preprocessing-routing-backend-v2
|
||
plan: 04
|
||
subsystem: routing
|
||
tags: [alt, landmarks, preprocessing, build-tool, overpass, fail-honest, coverage-gate, quantum-ceiling, scaling-rule]
|
||
|
||
requires:
|
||
- phase: 08-alt-landmark-preprocessing-routing-backend-v2 (plan 08-02)
|
||
provides: the ONE landmark module (LANDMARK_QUANTUM_S + farthestPointAnchors + landmarkTables + landmarkCoverage) and the ARG2 encodeGraph that REQUIRES a landmarks block
|
||
- phase: 08-alt-landmark-preprocessing-routing-backend-v2 (plan 08-03)
|
||
provides: the live ALT consumer (routeAstar potentials) these build-time tables feed
|
||
- phase: 06-turn-by-turn-guidance-live-driving
|
||
provides: the measured real-graph facts the gates lean on (311,325 nodes / 677,068 edges / 14.7 MB base / one giant SCC ≈ 0.98 coverage)
|
||
|
||
provides:
|
||
- the staged build tool: a preprocessing stage between edge compaction and encode that feeds the graph's edges to the ONE landmark module — the script quantizes and implements nothing itself (single-authority law)
|
||
- the D-02 scaling rule in code: L = clamp(round(N/13000), 16, 32) (~24 at KL's 311k nodes) + the --landmarks <n> measured knob (positive int, hard cap 128, parsed before --check)
|
||
- honest pre-write gates: quantum-ceiling throw → fail with the raise-quantum hint (Pitfall 5); per-anchor coverage ≥ 0.9 with the SAME fraction for any L (T-08-11); anchor shortfall fails loud
|
||
- extended stats gate (L/quantum/coverage min-mean/table bytes/sentinel totals BEFORE the size gates) and --check landmark facts decoded from the binary (count/quantum/anchors/coverage/sentinels, no network)
|
||
- A1 LADDER HISTORY rung (c2) recorded in the script header: EXECUTED in this preprocessing stage
|
||
affects: [08-05 (the campaign gates on the ARG2 binary this plan's rebuild leg must produce), future Malaysia-wide fork builds (re-derive L from the same rule), docs/ROAD-GRAPH.md quantum-knob documentation]
|
||
|
||
tech-stack:
|
||
added: []
|
||
patterns:
|
||
- "collect-then-write extends to the landmark stage: coverage/quantum gates run BEFORE encode, the write stays the last statement after every gate"
|
||
- "gate constants are override-proof: --landmarks changes the count, never LANDMARK_MIN_COVERAGE_FRACTION (T-08-11 enforced structurally)"
|
||
|
||
key-files:
|
||
created: []
|
||
modified:
|
||
- routing/scripts/build-road-graph.mjs (the preprocessing stage + --check landmark facts + --landmarks knob + ladder rung c2)
|
||
|
||
key-decisions:
|
||
- "Task 2's real rebuild STOPPED at Overpass HTTP 502 per the plan's own honest-failure path (no retry beyond the script's one-request law, no query change, no fabrication); the previous ARG1 binary is byte-untouched — the rebuild is a maintainer-time re-run of the same command"
|
||
- "Anchor shortfall (farthestPointAnchors returning fewer than requested) fails loud instead of encoding a weaker ALT than the rule asked for — printed L must equal carried L (input validation, Rule 2)"
|
||
|
||
patterns-established:
|
||
- "--check mirrors the build's printed facts from the written bytes (landmarkCoverage re-computed off the decoded u16 tables) — the print-the-measurement law extends to the artifact audit"
|
||
|
||
requirements-completed: [] # SC3/SC4 are phase success-criteria carry-overs, not REQUIREMENTS.md ids; their VERDICTS are 08-05's campaign gates (this plan ships the machinery only, and its rebuild leg is pending — see Issues Encountered)
|
||
|
||
duration: ~6min
|
||
completed: 2026-10-25
|
||
---
|
||
|
||
# Phase 8 Plan 08-04: Build-Side ALT Preprocessing Summary
|
||
|
||
**The build tool now produces ARG2 through the ONE landmark module — D-02 scaling rule (L ≈ 24 at KL scale), hard coverage/quantum gates before any write, a --landmarks measured knob, and --check landmark facts — but the real KL rebuild is PENDING: Overpass returned HTTP 502 and the plan's honest-failure path stopped there with the previous ARG1 binary untouched.**
|
||
|
||
## Performance
|
||
|
||
- **Duration:** ~6 min (11:08–11:14 UTC)
|
||
- **Started:** 2026-10-25T11:08:00Z
|
||
- **Completed:** 2026-10-25T11:13:25Z
|
||
- **Tasks:** 1/2 complete (task 1 committed; task 2 stopped at the plan's own honest-failure path)
|
||
- **Files modified:** 1
|
||
|
||
## Accomplishments
|
||
|
||
- **The preprocessing stage landed with zero second implementations**: after degree-0 compaction and before the encode, the script projects the edge array once (`{from, to, seconds, flags}` — the module's input shape, no mutation of the encode side) and feeds it to `farthestPointAnchors` → `landmarkTables` → `landmarkCoverage` from `../src/landmark.ts`. No local Dijkstra, no local priority queue, no local quantization — `grep -ci "dijkstra\|priorityqueue"` hits only 3 doc-comment lines (the module's Dijkstras, by name).
|
||
- **D-02 is code, not custom**: `L = clamp(round(N/13000), 16, 32)` with named, rationale-commented constants; `--landmarks <n>` replaces the rule wholesale for measured retries (validated positive integer, hard cap 128 — fail-loud beyond; parsed BEFORE `--check` dispatch so the knob composes with both modes).
|
||
- **The gates are honest and override-proof**: a `landmarkTables` throw → `fail(... raise LANDMARK_QUANTUM_S in routing/src/landmark.ts ... (Pitfall 5: raise the quantum, never clamp))`; coverage min < 0.9 → fail naming L, the offending anchor index/id, its fraction, and the giant-SCC expectation; anchor shortfall → fail rather than encode fewer anchors than the rule asked. The SAME coverage fraction gates any `--landmarks` value (T-08-11).
|
||
- **The measurement prints exist at both ends**: the stats gate prints L (rule vs override), quantum, coverage min/mean, table bytes (`2·L·N·2 + 4·L`), and sentinel totals BEFORE the size gates; `--check` now prints `landmarkCount` / `landmarkQuantumS` / `landmarkNodes` / coverage min-mean / sentinel totals decoded from the binary (verified live on the committed ARG2 mini-graph: L=2, quantum 0.100, coverage 1.000/1.000, sentinels 0).
|
||
- **Collect-then-write preserved**: `writeFileSync(OUT_PATH, ...)` remains the single call site, after every gate — the failed rebuild proved it (exit 1, binary byte-identical).
|
||
- **The fail-honest ARG1 path is proven, not assumed**: `--check routing/data/kl-roadgraph.v1.bin` exits 1 at this commit with `is NOT a valid GRAPH_VERSION CSR binary (decodeGraph null)` — the on-disk binary is still ARG1 exactly as the plan's acceptance criteria require at this stage.
|
||
- **No coupling leaked**: `npm run routing:test` stays 79/79 green (the suite never imports the build script).
|
||
|
||
## Task Commits
|
||
|
||
Each completed task was committed atomically:
|
||
|
||
1. **task 1: The preprocessing stage + --check landmark facts (code)** — `698fdf4` (feat)
|
||
2. **task 2: The real rebuild (network, maintainer-time)** — **NO COMMIT — stopped at Overpass HTTP 502 per the plan's honest-failure path; no file changed, nothing to commit**
|
||
|
||
## Files Created/Modified
|
||
|
||
- `routing/scripts/build-road-graph.mjs` (+173/−16) — the ALT preprocessing stage, the landmark constants block (D-02 rule + coverage gate + override cap), the `--landmarks` argv parse, the encodeGraph call carrying `landmarks: { quantumS, nodes, to, from }`, the extended stats/`--check` prints, the usage/exit-code docs, and A1 LADDER HISTORY rung (c2)
|
||
|
||
## Decisions Made
|
||
|
||
- **Stopped at the first 502 instead of retrying**: the plan pre-commits "Overpass unreachable → the script exits 1, the previous binary untouched. STOP: record the failure. Do NOT fabricate, shrink the query, or retry beyond the script's own law," and the orchestrator confirmed "record it honestly in SUMMARY.md as a deviation and stop at that point." A 502 IS a fetch failure; an executor quietly hammering a rate-limited public API is exactly what the STOP rule exists to prevent. Wave 5's maintainer-time campaign can rebuild with one command.
|
||
- **Anchor-shortfall fail-loud (Rule 2, input validation)**: `farthestPointAnchors` may legitimately return fewer than requested (candidates exhausted / fragmentation); silently encoding fewer landmarks than the printed L would break the print-the-measurement law, so the shortfall exits 1 with a fragmentation hint. Degenerate zero-anchor case covered separately.
|
||
|
||
## Deviations from Plan
|
||
|
||
### Honest-Stop (pre-planned path, not an auto-fix)
|
||
|
||
**1. Task 2's real rebuild blocked by Overpass HTTP 502 — stopped per the plan's own failure path**
|
||
- **Found during:** task 2 (the real rebuild)
|
||
- **What happened:** `node routing/scripts/build-road-graph.mjs` fetched the corpus bbox, hit Overpass, and got `HTTP 502` (server-side transient); the script exited 1 at its own `!res.ok` gate
|
||
- **Action taken:** NO retry, NO query change, NO fabrication — recorded and stopped, per task 2's HONEST FAILURE PATHS and the plan's collect-then-write law
|
||
- **State left behind:** `routing/data/kl-roadgraph.v1.bin` byte-untouched (ARG1, 15,449,343 B, mtime 2026-10-24 13:45); `git status` clean for routing/src/**, routing/test/**, routing/scripts/** (task-2 acceptance: zero source drift — holds)
|
||
- **Full build stdout (the measurement that exists outside the terminal):**
|
||
|
||
```
|
||
bbox (corpus extremes ± 0.05 deg): [2.9397, 101.5569, 3.2268, 101.7876]
|
||
fetching ways from https://overpass-api.de/api/interpreter (this can take minutes)…
|
||
build-road-graph: Overpass returned HTTP 502
|
||
```
|
||
(exit 1 — the bbox line matches the 2026-10-24 build: computed from the COMMITTED corpus ± margin, mechanical pinning intact)
|
||
|
||
---
|
||
|
||
**Total deviations:** 1 (the pre-planned honest stop; 0 auto-fixed bugs)
|
||
**Impact on plan:** Task 1 shipped complete and verified. Task 2's deliverable (a checked-green ARG2 binary on disk) is PENDING — produce it by re-running `node routing/scripts/build-road-graph.mjs` (minutes, Overpass willing), then `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` must exit 0 printing landmarkCount ≥ 16 / quantum 0.1 / coverage ≥ 0.9. Expected ≈ 43 MB (14.7 MB base + ~28 MB tables at L=24), inside the envelope and the 30–80 MB advisory band.
|
||
|
||
## Issues Encountered
|
||
|
||
- **Overpass HTTP 502** (task 2) — the public endpoint's well-known transient failure mode. Handled by the plan's pre-committed honest stop (see Deviations). The binary rebuild is the ONLY outstanding deliverable of this plan; everything needed for it is staged and committed.
|
||
- Nothing else — task 1's full verification battery passed on the first run (syntax, ARG1 honest-refuse exit 1, all grep gates, single writeFileSync call site, zero Dijkstra code hits, routing:test 79/79).
|
||
|
||
## Verification Record (as-run)
|
||
|
||
- `node --check routing/scripts/build-road-graph.mjs` → SYNTAX-OK
|
||
- `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` → exit 1, `--check: routing/data/kl-roadgraph.v1.bin is NOT a valid GRAPH_VERSION CSR binary (decodeGraph null)` — ARG1-HONEST-REFUSE proven (the rebuild is the pending task 2)
|
||
- `node routing/scripts/build-road-graph.mjs --check routing/test/fixtures/mini-graph.bin` → prints `landmarkCount: 2 / landmarkQuantumS: 0.100 / landmarkNodes: 0, 18 / landmark coverage min/mean: 1.000 / 1.000 / landmark sentinel cells: 0 to / 0 from` (exit 1 solely at the pre-existing 1 MB floor gate — a fixture, not a road graph)
|
||
- `--landmarks 0` → exit 1 (positive-integer law); `--landmarks 999` → exit 1 at the 128 cap
|
||
- grep gates: `farthestPointAnchors` 6, `landmarkTables` 5, `landmarkCoverage` 6, `LANDMARK_MIN_COVERAGE_FRACTION` 6, `NODES_PER_LANDMARK` 3, `--landmarks` 10, `landmarks: {` 1, `writeFileSync(OUT_PATH` 1 (after the gates), Dijkstra/priority-queue CODE hits 0 (3 doc-comment lines)
|
||
- `npm run routing:test` → 79 passed, 0 failed
|
||
|
||
## User Setup Required
|
||
|
||
None - no external service configuration required. (The pending rebuild needs only network access to the public Overpass API — no keys, no accounts.)
|
||
|
||
## Next Phase Readiness
|
||
|
||
- **08-05 (campaign):** BLOCKED on the rebuild leg — the campaign's first act should be the maintainer-time re-run: `node routing/scripts/build-road-graph.mjs` (expect the landmark block: L = 24 by the rule at ~311k nodes, quantum 0.1, coverage min ≈ 0.98 per the giant-SCC fact, table bytes ≈ 28 MB, total ≈ 43 MB, no below-band WARNING), then `--check` exit 0. If Overpass fails again, that is the campaign's first honest data point, not a blocker to work around.
|
||
- **The staged tool is campaign-ready:** every gate number prints before the write; the `--landmarks` knob is the D-02 remediation lever (raise toward 32) without touching any search code; RO1/RO2's rollout pins are untouched by this plan.
|
||
- **Known state:** the real KL binary on disk is still ARG1 → decodes null → router-pending (designed, D-01) until the rebuild + re-upload; all committed fixtures carry tables, so every suite surface runs ALT live.
|
||
|
||
---
|
||
*Phase: 08-alt-landmark-preprocessing-routing-backend-v2*
|
||
*Completed: 2026-10-25 (task 1; task 2 pending Overpass availability)*
|
||
|
||
### Orchestrator addendum — task 2 COMPLETED (2026-10-25, same day)
|
||
|
||
The maintainer-time rebuild succeeded once Overpass recovered (HTTP 502 was transient):
|
||
- `node routing/scripts/build-road-graph.mjs` → wrote `routing/data/kl-roadgraph.v1.bin` ARG2: 43.2 MB (45,341,620 B), 311,361 nodes / 677,126 edges / 21,469 names, maxSpeed 110
|
||
- Landmark facts: L = 24 (D-02 rule, round(311361/13000) = 24), quantum 0.1 s, coverage min/mean 0.980/0.980 (gate ≥ 0.9), table bytes 28.5 MB, sentinels 150,986 to / 1,365,034 from
|
||
- `--check routing/data/kl-roadgraph.v1.bin` → exit 0, facts decoded from the binary
|
||
- The 08-05 BLOCKED note above is superseded: the ARG2 binary exists on the main tree — the campaign can run directly. R2 re-upload (`node routing/scripts/upload-road-graph.mjs`) remains a deploy-time user action, not an execution blocker.
|
||
|
||
## Self-Check: PASSED
|
||
|
||
- `routing/scripts/build-road-graph.mjs` exists with the stage (grep-verified counts above) — FOUND
|
||
- Task commit `698fdf4` exists in git history — FOUND
|
||
- routing:test 79/79 at HEAD — FOUND (run post-commit)
|
||
- Task 2 has intentionally NO commit (honest stop, zero file changes) — VERIFIED (git status clean on all routing paths; binary md5 6a94156aaa1d58542cef98c4ffc52b7e, byte-untouched)
|