diff --git a/.planning/STATE.md b/.planning/STATE.md index 5e11b2d..5b710b7 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,16 +2,16 @@ gsd_state_version: 1.0 milestone: v1.x milestone_name: routing-engine-upgrades-and-extras -status: phase-planned -stopped_at: "08-01 executed (wave 0 complete) — next: 08-02 (landmark core + ARG2 codec)" -last_updated: "2026-10-25T16:05:00.000Z" +status: executing +stopped_at: "08-02 executed (landmark core + ARG2 codec committed, fixture regenerated as ARG2, full battery green) — next: 08-03 (ALT potentials)" +last_updated: "2026-10-25T10:55:00.000Z" last_activity: 2026-10-25 progress: - total_phases: 7 - completed_phases: 7 - total_plans: 34 - completed_plans: 34 - percent: 100 + total_phases: 4 + completed_phases: 0 + total_plans: 5 + completed_plans: 2 + percent: 40 --- # Project State @@ -26,11 +26,11 @@ See: .planning/PROJECT.md (updated 2026-10-22) ## Current Position Phase: 8 -Plan: 08-01 executed (wave 0); 08-02 next -Status: Executing wave 1 +Plan: 08-02 executed (landmark core + ARG2 codec); 08-03 next +Status: Executing wave 2 Last activity: 2026-10-25 -Progress: [░░░░░░░░░░] 0% +Progress: [████░░░░░░] 40% ## Performance Metrics @@ -90,6 +90,6 @@ None yet. ## Session Continuity -Last session: 2026-10-25T17:05:00.000Z -Stopped at: "08-01 executed (wave 0 complete) — next: 08-02 (landmark core + ARG2 codec)" -Resume file: .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-02-PLAN.md +Last session: 2026-10-25T10:55:00.000Z +Stopped at: "08-02 executed (landmark core + ARG2 codec committed, fixture regenerated as ARG2, full battery green) — next: 08-03 (ALT potentials)" +Resume file: .planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-03-PLAN.md diff --git a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-02-SUMMARY.md b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-02-SUMMARY.md new file mode 100644 index 0000000..cbf5b9e --- /dev/null +++ b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-02-SUMMARY.md @@ -0,0 +1,171 @@ +--- +phase: 08-alt-landmark-preprocessing-routing-backend-v2 +plan: 02 +subsystem: routing +tags: [alt, landmark, arg2, codec, dijkstra, typed-arrays, quantization, fixture, tdd] + +requires: + - phase: 08-alt-landmark-preprocessing-routing-backend-v2 (plan 08-01) + provides: the frozen ARG1 rejection vector (mini-graph-arg1.bin) this plan's G3 consumes, and the pre-ALT baseline record the codec change must not invalidate + - phase: 06-turn-by-turn-guidance-live-driving + provides: the ARG1 codec being versioned (graph.ts single authority), the A1 traversal law (bit0 never relaxed) the landmark Dijkstras mirror, and the committed mini-graph fixture basis + +provides: + - routing/src/landmark.ts — the ONE pure landmark compute core: farthestPointAnchors (deterministic degree>=2 farthest-point over the ĉ metric) + landmarkTables (exact integer Dijkstras, u16 tables, 0xFFFF sentinel, 65000-quantum loud overflow) + landmarkCoverage + - ARG2 graph.ts — version-bumped single-authority codec: 72-byte header (landmarkCount @60, quantum f32 @64), append-only u16 table suffix, exact-size law extended over the variable suffix BEFORE allocation, one decode path (ARG1 bytes → null) + - the regenerated committed mini-graph.bin fixture (ARG2, L=2 anchors [0,18], quantum 0.1 s, coverage 1.0, byte-deterministic) through the SAME encode authority + - extended G-suite (G1–G11) incl. the ARG1-fixture rejection pin, table round-trip, 9-case encode-validation matrix, landmarkCount=0 + monster-count decode rejections + - RoadGraph.landmarkCount/landmarkQuantumS/landmarkTo/landmarkFrom/landmarkNodes — the typed-array fields Plan 08-03's ALT potentials consume +affects: [08-03 (ALT potentials read the tables), 08-04 (build script landmark stage via the same core), 08-05 (real-graph campaign), bot routing suite (corridor graphs now ARG2)] + +tech-stack: + added: [] + patterns: + - "ĉ-metric surrogate: floor-quantize EDGE WEIGHTS once (Math.floor(seconds/quantumS)), exact integer Dijkstras over Int32 labels — quantization without an admissibility epsilon (research Pattern 2)" + - "append-only layout versioning: new sections are a suffix, header grows into the old pad, ONE decode path, old-version bytes decode null (D-01 fail-honest)" + - "tiny-real-landmark-stage fixture regeneration: the committed fixture flows through the SAME farthestPointAnchors/landmarkTables/encodeGraph the real build uses (byte-deterministic re-runs)" + +key-files: + created: + - routing/src/landmark.ts + - routing/test/landmark.test.mjs + modified: + - routing/src/graph.ts (ARG2 version bump) + - routing/test/graph.test.mjs (G-suite re-pin + G9–G11) + - routing/test/fixtures/build-mini-graph.mjs (landmark stage) + - routing/test/fixtures/mini-graph.bin (regenerated ARG2, 1668 → 1884 B) + - routing/test/astar.test.mjs (A3 synthetic carries the landmark block) + - routing/test/snap.test.mjs (S3 synthetic carries the landmark block) + - bot/test/webhook.test.mjs (corridor/sparse builders compute landmark blocks) + - bot/test/guidance-cron.test.mjs (unnamed-way builder computes the landmark block) + +key-decisions: + - "Fixture regeneration pulled forward into task 2's commit (Rule 3): task 2's own gate demands routing:test 0 failed, which is impossible while the committed fixture is ARG1 once decode is ARG2-only; task 3 then sealed byte-stability and re-ran the (idempotent) builder" + - "Decode validates landmarkQuantumS (finite > 0) → null — hostile-quantum header defense in the T-08-01 family; nothing downstream can divide by it" + - "Reserved u32 @68 is written 0 and accepted-untouched at decode (the plan's recommended reading — no assert on an unused field)" + - "Anchor selection counts stored OUT-degree (blocked mirrors included) for the degree>=2 filter — a node on a oneway still sits on multi-edge road geometry" + - "Graphs with no degree>=2 candidate (sparse/2-node synthetics) pin their anchor explicitly and still compute tables through landmarkTables — the selection filter is a quality law, not a table law" + +patterns-established: + - "TDD with hand-computed integer vectors: RED expectations verified against an INDEPENDENT brute force before pinning (the L6 lesson — intuition about 'farthest' failed on a speed-skewed metric)" + - "Tamper-vector tests construct CONSISTENT hostile buffers (byte size matches the tampered layout) so a specific decode law is isolated from the size law (G11)" + +requirements-completed: [SC3, SC4] + +duration: ~17min +completed: 2026-10-25 +--- + +# Phase 8 Plan 08-02: Landmark Core + ARG2 Codec Summary + +**The landmark compute core exists once as a pure typed-array module (deterministic farthest-point anchors + exact integer ĉ-metric Dijkstras with a 65000-quantum loud overflow), graph.ts is version-bumped to ARG2 with one fail-honest decode path whose exact-size law covers the u16 table suffix, and the committed fixture is regenerated as ARG2 through that same encode authority — routing 74/74 + bot 126/126 + both typechecks green, with astar.ts untouched (search behavior provably neutral at this commit).** + +## Performance + +- **Duration:** ~17 min (10:34–10:51 UTC) +- **Started:** 2026-10-25T10:34:07Z +- **Completed:** 2026-10-25T10:51:00Z +- **Tasks:** 3/3 (tasks 1 & 2 with full RED→GREEN gate commits) +- **Files modified:** 10 (2 created, 8 modified) + +## Accomplishments + +- **`routing/src/landmark.ts` — the ONE landmark compute core** (zero imports, no I/O): `farthestPointAnchors` (first degree-≥2 candidate by node id, greedy farthest-point with per-pick forward-pass reuse, lower-id tie-breaks, documented coverageFraction early-stop knob), `landmarkTables` (forward + reversed integer Dijkstras over floor-quantized weights — the ĉ metric, no stored-distance rounding, no epsilon — binary-heap, lazy deletion, bit0 edges never relaxed in EITHER direction exactly mirroring astar.ts), `landmarkCoverage` (the build's per-anchor assert feed). 0xFFFF is DATA; any label past 65,000 quanta throws naming the distance and the fix. +- **ARG2 in `graph.ts` in place** — no second codec: `GRAPH_MAGIC = 0x41524732`, `GRAPH_VERSION = 2`, `HEADER_SIZE = 72` (f64 alignment kept), landmark header fields @60/@64/@68, append-only suffix (u16 toTable, u16 fromTable, u32 anchors) folded into `layoutFor`'s `total` arithmetic so the exact-size check rejects monster counts BEFORE any view allocation. Decode refuses ARG1 bytes (the committed `mini-graph-arg1.bin` is the standing rejection vector), version ≠ 2, `landmarkCount = 0` (no landmark-less ARG2 exists — D-01/A5), hostile quantum, and any size mismatch. `GraphInput.landmarks` is REQUIRED and loud-validated (9 named-throw classes). +- **The fixture is ARG2 through the one authority**: the builder gained the tiny real landmark stage (L=2 → anchors [0, 18], quantum 0.1 s, coverage 1.00/1.00, fail-loud invariants before write) and is byte-deterministic on re-run; `mini-graph-arg1.bin` stayed byte-identical to its committed 08-01 bytes (`cmp`-verified). +- **The full battery is green at the final commit** — routing:test 74/74 (L1–L6 + G1–G11 + untouched A/P/R/S suites), bot 126/126 (RT/H/GC + the updated corridor/cron builders), routing+bot typechecks clean. `routing/src/astar.ts` has ZERO changes — A1's 625-pair sweep green over the ARG2 fixture proves the codec change alone is behavior-neutral to search (routeAstar still runs haversine potentials; ALT lands in 08-03). + +## Task Commits + +Each task was committed atomically (TDD tasks carry their RED gates): + +1. **task 1: landmark.ts — the pure landmark compute core** — `c3a31ae` (test, RED) → `1bf413b` (test, L5 vector fix) → `5b7a130` (feat, GREEN) +2. **task 2: ARG2 codec in graph.ts** — `34d96b0` (test, RED) → `32a00b5` (feat, GREEN; includes the fixture regen pull-forward) +3. **task 3: bot call sites + full battery seal** — `9770867` (test) + +## Files Created/Modified + +- `routing/src/landmark.ts` — the pure landmark compute core (created) +- `routing/src/graph.ts` — ARG2 codec: 72-byte header, u16 suffix, version-bumped decode laws, required-landmarks encode validation +- `routing/test/landmark.test.mjs` — L1–L6: hand-computed vectors for anchors, quantum exactness, sentinels, asymmetry, overflow, the 5×5 grid (created) +- `routing/test/graph.test.mjs` — G-suite re-pinned (G3/G5/G7/G8) + extended (G9 table round-trip, G10 encode validation, G11 decode rejection) +- `routing/test/fixtures/build-mini-graph.mjs` — landmark stage + landmark stats in the summary line +- `routing/test/fixtures/mini-graph.bin` — regenerated ARG2 (1884 B; +208-byte table suffix + 4 header bytes + 4-byte alignment) +- `routing/test/astar.test.mjs`, `routing/test/snap.test.mjs` — synthetic encodeGraph call sites carry (inert until 08-03) landmark blocks +- `bot/test/webhook.test.mjs`, `bot/test/guidance-cron.test.mjs` — corridor/sparse/cron builders compute tiny tables via `farthestPointAnchors`/`landmarkTables` and pass the `landmarks` block + +## Decisions Made + +- **Pull the fixture regen forward (Rule 3)**: task 2's acceptance demands "runner output 0 failed" at its own commit — impossible with a committed ARG1 fixture under ARG2-only decode. The regen joined task 2's commit; task 3 verified the seal (idempotent re-run, byte-stable ARG1 vector) instead. +- **Decode-side quantum validation**: a non-finite/non-positive `landmarkQuantumS` header decodes null — cheap hostile-header defense so 08-03's potentials can never divide by a corrupt quantum. +- **Degree filter counts stored out-edges (mirrors included)**: the blocked mirror still evidences multi-edge road geometry at a node; endpoints/dead-ends stay excluded. +- **Test-vector discipline**: every pinned integer expectation was verified against an independent brute-force Dijkstra before pinning — which caught my own wrong intuition (below). + +## Deviations from Plan + +### Auto-fixed Issues + +**1. [Rule 3 - Blocking] Fixture regeneration pulled forward from task 3 into task 2's commit** +- **Found during:** task 2 (ARG2 codec) +- **Issue:** task 2's own verification gate (`npm run routing:test` → 0 failed) cannot pass while the committed fixture is ARG1 bytes once decode becomes ARG2-only (G7 + A-suite + snap suite all decode the fixture) +- **Fix:** extended `build-mini-graph.mjs` with the landmark stage and regenerated `mini-graph.bin` inside task 2's commit; task 3 then re-ran the builder (byte-identical output — determinism proven) and sealed ARG1-vector byte-stability +- **Files modified:** routing/test/fixtures/build-mini-graph.mjs, routing/test/fixtures/mini-graph.bin +- **Verification:** routing:test 74/74 at task-2 commit; `cmp` vs committed 08-01 ARG1 bytes passes; regen deterministic +- **Committed in:** `32a00b5` + +**2. [Rule 3 - Blocking] Two encodeGraph call sites missing from the plan's enumeration** +- **Found during:** task 2 (call-site census before implementing) +- **Issue:** the required-landmarks law makes EVERY encodeGraph call throw — the plan's files list omitted `routing/test/astar.test.mjs` (A3's disconnected synthetic) and `routing/test/snap.test.mjs` (S3's bare 2-node synthetic) +- **Fix:** both synthetics carry small hand-written landmark blocks (inert steering data until 08-03), keeping the cases' original focus +- **Files modified:** routing/test/astar.test.mjs, routing/test/snap.test.mjs +- **Verification:** A-suite and S-suite green in the full battery +- **Committed in:** `32a00b5` + +**3. [Rule 1 - Bug] Two hand-computed L-suite expectations were wrong (implementation was right)** +- **Found during:** task 1 GREEN +- **Issue:** (a) L4 indexed table rows by anchor NODE ID instead of anchor POSITION (row-major is `[ℓ·N+v]`, ℓ = position); (b) L6 assumed the ĉ-farthest node from corner 0 is the opposite corner 24 — with interior hops costing 2× perimeter quanta, interior node 18 (297 q) is farther than corner 24 (264 q) +- **Fix:** re-pinned both vectors; node 18's 297-quantum max verified against an independent brute-force Dijkstra before pinning +- **Files modified:** routing/test/landmark.test.mjs +- **Verification:** L-suite green; the independent brute force output recorded in-session +- **Committed in:** `5b7a130` + +**4. [Rule 1 - Bug] Sparse/cron graphs have no degree-≥2 farthest-point candidate** +- **Found during:** task 3 (bot battery) +- **Issue:** the plan assumed every call site can `farthestPointAnchors(...)` — but a single-segment graph's nodes all have out-degree 1, so selection legitimately returns empty and the first battery run failed at suite load +- **Fix:** those call sites pin the anchor explicitly (`Uint32Array.of(0)`) and still compute tables through `landmarkTables` — the selection filter is a quality law, not a table law +- **Files modified:** bot/test/webhook.test.mjs, bot/test/guidance-cron.test.mjs +- **Verification:** bot suite 126/126 +- **Committed in:** `9770867` + +--- + +**Total deviations:** 4 auto-fixed (2 blocking-call-site/closure, 2 test-vector bugs) +**Impact on plan:** All four were forced by the plan's own laws (required-landmarks encode, exact gate order) or by verified arithmetic; no scope creep — the 08-04/08-05 boundaries are untouched (`build-road-graph.mjs` still belongs to 08-04; no search code changed). + +## TDD Gate Compliance + +Both `tdd="true"` tasks ran full RED→GREEN with separate commits: task 1 `c3a31ae`(+`1bf413b`) test gates BEFORE `5b7a130` feat; task 2 `34d96b0` test gate BEFORE `32a00b5` feat. RED failures were verified to fail for the RIGHT reason each time (ERR_MODULE_NOT_FOUND for the new module; specific assertion deltas for the re-pinned codec laws). No gate skipped. + +## Issues Encountered + +- A first battery run piped bot tests through `| tail -2` without `pipefail`, masking the suite's exit code behind tail's; caught by reading the output (npm error line), re-run with `set -o pipefail` — all subsequent battery commands use pipefail. +- The plan's acceptance grep contains a typo ("landmarms"); the intent (both bot files pass a `landmarks: {` block) is satisfied literally. + +## User Setup Required + +None - no external service configuration required. + +## Next Phase Readiness + +- **08-03 (ALT potentials):** `RoadGraph` now exposes `landmarkCount/landmarkQuantumS/landmarkTo/landmarkFrom/landmarkNodes` as zero-copy views; only `potential()` in `routeAstar` changes. The table values are inert until then (documented coexistence — A-suite green at this commit is the neutrality proof). +- **08-04 (build script):** `build-road-graph.mjs` still encodes WITHOUT landmarks (line 336) — its planned preprocessing stage joins between edge assembly and the encode call, importing the SAME `landmark.ts` core the fixture builder now uses. +- **08-05 (campaign):** the real KL binary on disk is still ARG1 → decodes null until rebuilt+re-uploaded (the designed router-pending window, D-01); the committed baseline record (08-01) is untouched and its recorder's ARG1-version guard is exactly what makes that safe. + +--- +*Phase: 08-alt-landmark-preprocessing-routing-backend-v2* +*Completed: 2026-10-25* + +## Self-Check: PASSED + +All 8 claimed files exist on disk; all 6 task commits (`c3a31ae`, `1bf413b`, `5b7a130`, `34d96b0`, `32a00b5`, `9770867`) exist in git history. Final battery at `9770867`: routing:test 74/74, bot 126/126, routing:typecheck + bot:typecheck clean. + diff --git a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VALIDATION.md b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VALIDATION.md index dd98d9a..eabf9e6 100644 --- a/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VALIDATION.md +++ b/.planning/phases/08-alt-landmark-preprocessing-routing-backend-v2/08-VALIDATION.md @@ -41,7 +41,7 @@ created: 2026-10-25 | W0 baseline recorder | 01 | 1 | SC-2 (D-03 prereq) | — | N/A | script | `node routing/scripts/measure-optimality-baseline.mjs` → committed record | ✅ W0 | ✅ green | | ARG1 fixture preservation | 01 | 1 | D-01 | T-08-02 | old binary → decode null (fail-honest) | unit | `npm run routing:test` (G rejection case lands in 08-02) | ✅ W0 | ✅ green (bytes frozen) | | SC-1′ extended-sample flag | 01 | 1 | SC-1 (D-04) | — | N/A | measurement gate | `node routing/scripts/measure-route-cpu.mjs --extended` | ✅ W0 | ✅ green (pre-ALT honest FAIL: p95 98.9 ms, n=53) | -| ARG2 codec G-suite | 02 | 2 | D-01/D-02 | T-08-01/02 | exact-size-before-allocation; version + landmarkCount laws; hostile header on tiny buffer rejects BEFORE view allocation | unit | `npm run routing:test` (G cases) | ❌ new | ⬜ pending | +| ARG2 codec G-suite | 02 | 2 | D-01/D-02 | T-08-01/02 | exact-size-before-allocation; version + landmarkCount laws; hostile header on tiny buffer rejects BEFORE view allocation | unit | `npm run routing:test` (G cases) | ✅ 08-02 | ✅ green (74/74; ARG1 vector + monster landmarkCount reject pre-allocation) | | ALT potentials A-suite (A1 re-pin + island A9/A10 + A11 pin) | 03 | 3 | SC-2 | T-08-05/06 | sentinel-safe potentials; equivalence sealed under ALT | unit (A1 625-pair sweep + new cases) | `npm run routing:test` (A cases) | ⚙ extends | ⬜ pending | | Rollout pins (upload-agnostic + GRAPH_KEY) | 03 | 3 | D-01 | T-08-08 | upload stays format-agnostic; loader keeps ONE decode path | structural grep (A8-shape) | `npm run routing:test` (RO1/RO2) | ❌ new | ⬜ pending | | Build preprocessing stage | 04 | 4 | SC-1 (D-02) | T-08-09/10/11 | coverage + quantum gates before write; L scaling rule | script gate | `node routing/scripts/build-road-graph.mjs` + `--check` | ⚙ extends | ⬜ pending | @@ -61,8 +61,8 @@ created: 2026-10-25 - [x] **Baseline record + recorder tool** (D-03 prerequisite — MUST be first task before ARG2 codec lands; research Pitfall 1) — DONE 08-01 (`alt-baseline.json`, immutable, ARG1-version-guarded) - [x] Extended-sample flag (D-04) — DONE 08-01 (`--extended`; default 25-route mode byte-compatible, budget-audit 19/19) - [x] `fixtures/mini-graph-arg1.bin` — bytes preserved from the current committed ARG1 fixture (decode-rejection vector) — DONE 08-01 -- [ ] `fixtures/build-mini-graph.mjs` gains a landmark stage (tiny L; ARG2 output) -- [ ] New G cases (ARG2 codec + rejection matrix) + island A case +- [x] `fixtures/build-mini-graph.mjs` gains a landmark stage (tiny L; ARG2 output) — DONE 08-02 (L=2, anchors [0,18], quantum 0.1 s, coverage 1.0; byte-deterministic) +- [x] New G cases (ARG2 codec + rejection matrix) — DONE 08-02 (G9–G11; island A case remains for 08-03) - [ ] Upload-unchanged structural pin case *Constraint re-verified in planning: `budget-audit` (Phase 7) re-runs `measure-route-cpu.mjs` — keep its CLI contract or account for the flag addition.*