araharah-executor
67026b9483
- 08.1-01 task 2: re-pin graph.test.mjs in-memory G-cases (G1/G3/G8/G9/G10/G11)
and astar.test.mjs in-memory/ALT cases; suspend only committed-fixture G7 and
A1/A2/A4/A5/A6 (the runner auto-discovers all suites and parses no argv, so
task 2's verify could never exit 0 otherwise)
- 08.1-01 task 3: own un-suspension of G7 and A1/A2/A4/A5/A6 after the ARG3
fixture regeneration
- 08.1-01 task 1: drop the cosmetic --test-name-pattern flag (ignored by the
hand-rolled runner; full suite runs instead)
- 08.1-02 task 2: use grep -cF for the rollout decodeGraph call-site gate
(BRE form exited 2 with 'Unmatched ( or \\(')
- 08.1-02 task 1: replace the no-op '|| test $? -eq 1' landmark gate with a
'test "$(...)" -eq 0' count assertion that can actually fail
37 KiB
37 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 08.1-contraction-hierarchies-routing-backend-v3-the-lever-that-cl | 01 | execute | 1 |
|
true |
|
|
Purpose: This is the mechanism that closes SC-1. The hierarchy prunes the request-time search structurally, while D-04/D-05 remove the ALT engine and its 28.5 MB landmark suffix rather than leaving a dormant fallback. Output: Pure CH construction/query modules, ARG3 codec, CH-backed routeAstar, ARG3 fixtures, and updated routing/bot test suites.
<execution_context> @$HOME/.config/opencode/get-shit-done/workflows/execute-plan.md @$HOME/.config/opencode/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/08.1-contraction-hierarchies-routing-backend-v3-the-lever-that-cl/08.1-CONTEXT.md @.planning/phases/08.1-contraction-hierarchies-routing-backend-v3-the-lever-that-cl/08.1-RESEARCH.md @.planning/phases/08.1-contraction-hierarchies-routing-backend-v3-the-lever-that-cl/08.1-PATTERNS.md @.planning/phases/08.1-contraction-hierarchies-routing-backend-v3-the-lever-that-cl/08.1-VALIDATION.md @routing/src/graph.ts @routing/src/astar.ts @routing/src/landmark.ts @routing/src/index.ts @routing/test/graph.test.mjs @routing/test/astar.test.mjs @routing/test/fixtures/build-mini-graph.mjs @routing/test/rollout.test.mjs Existing contracts to preserve: ```typescript // routing/src/astar.ts export interface AstarResult { nodes: number[]; seconds: number; distanceM: number; } export function routeAstar(graph: RoadGraph, originNode: number, goalNode: number): AstarResult | null; export function deriveSteps(graph: RoadGraph, nodes: number[]): RouteStep[]; export function classifyTurn(prevBearingDeg: number, nextBearingDeg: number): GuidanceTurnKind; ```New internal contracts to implement:
// routing/src/contraction.ts
export interface ContractionInput {
nodeCount: number;
edges: Array<{ from: number; to: number; seconds: number; flags: number }>;
}
export interface ContractionResult {
rank: Uint32Array;
shortcutStart: Uint32Array;
shortcutTarget: Uint32Array;
shortcutSeconds: Float32Array;
shortcutVia: Uint32Array;
stats: {
shortcutCount: number;
ratioPerNode: number;
witnessSearches: number;
witnessBudgetHits: number;
};
}
export const CH_LAMBDA_DELETED_NEIGHBOURS = 1;
export const CH_WPS_MAX_SETTLED = 60;
export const CH_WPS_SETTLED_PER_DEGREE = 4;
export const CH_SHORTCUT_EXPLOSION_RATIO = 4;
export function contractGraph(input: ContractionInput): ContractionResult;
// routing/src/graph.ts
export interface ChTablesInput {
rank: Uint32Array;
shortcutStart: Uint32Array;
shortcutTarget: Uint32Array;
shortcutSeconds: Float32Array;
shortcutVia: Uint32Array;
}
export interface RoadGraph {
chShortcutCount: number;
chRank: Uint32Array;
shortcutStart: Uint32Array;
shortcutTarget: Uint32Array;
shortcutSeconds: Float32Array;
shortcutVia: Uint32Array;
}
ARG3 fork contract to implement exactly:
header 72 bytes, little-endian:
0 magic u32 = 0x41524733 ("ARG3")
4 version u32 = 3
8 nodeCount u32
12 edgeCount u32
16 nameCount u32
20 namePoolLen u32
24 bbox 4 x f64
56 maxSpeedKmh f32
60 chShortcutCount u32 (0 -> decode null)
64 reserved0 u32
68 reserved1 u32
sections:
lat[N] ++ lng[N] ++ edgeStart[N+1] ++ edgeTarget[E] ++ edgeFlags[E]
++ edgeSeconds[E] ++ edgeNameId[E] ++ nameOffset[M+1] ++ namePool
++ chRank[N] ++ shortcutStart[N+1] ++ shortcutTarget[S]
++ shortcutSeconds[S] ++ shortcutVia[S]
Implement these exact constants and exports:
- `CH_LAMBDA_DELETED_NEIGHBOURS = 1`
- `CH_WPS_MAX_SETTLED = 60`
- `CH_WPS_SETTLED_PER_DEGREE = 4`
- `CH_SHORTCUT_EXPLOSION_RATIO = 4`
- `ContractionInput { nodeCount, edges: Array<{from,to,seconds,flags}> }`
- `ContractionResult { rank, shortcutStart, shortcutTarget, shortcutSeconds, shortcutVia, stats }`
- `contractGraph(input): ContractionResult`
Implement the bottom-up CH algorithm exactly:
1. Validate node ids and finite non-negative edge seconds before constructing CSR views. An edge with `flags & 1` is blocked in every direction and must never be relaxed.
2. Compute priority as `edgeDifference(u) + CH_LAMBDA_DELETED_NEIGHBOURS * deletedNeighbours(u)`. Use a lazy binary priority queue. When a popped priority is stale, recompute it and requeue if it improved. Break priority ties by the lower node id.
3. Assign `rank[u]` as the strictly increasing contraction position, never as the priority value. Thus rank is a total order over all nodes.
4. For each node `u` being contracted, enumerate overlay in-neighbours `a` and overlay out-neighbours `b`. The candidate shortcut is directed `a -> b`, with seconds `w(a,u) + w(u,b)` and `via = u`.
5. Run each witness search on the overlay graph with `u` removed and all previously contracted nodes absent. Bound it by both the candidate distance and `max(CH_WPS_MAX_SETTLED, CH_WPS_SETTLED_PER_DEGREE * localDegree)`. If the witness search is bounded out or finds no path within the candidate distance, emit the shortcut; bounded witness searches may add superfluous shortcuts but must not remove a required one.
6. Remove `u` and its incident overlay edges before the next witness search. Emit shortcuts in deterministic order: ascending `from`, then ascending `to`.
7. Build shortcut CSR arrays where `shortcutStart` has length `nodeCount + 1` and every source's shortcuts are contiguous.
8. Fail loud if `shortcutCount > CH_SHORTCUT_EXPLOSION_RATIO * nodeCount`, with a message naming the ratio and the measured count.
9. Populate `stats` with `shortcutCount`, `ratioPerNode = shortcutCount / nodeCount`, `witnessSearches`, and `witnessBudgetHits`.
Determinism is mandatory: no PRNG, no `Date`, no Map iteration order, no reliance on object key order. Identical input arrays must yield identical output arrays. Do not use any external CH library and do not add a dependency.
In `layoutFor`, calculate the CH suffix in this order and use 4-byte alignment before every u32/f32 section: `chRank = (namePool + namePoolLen + 3) & ~3`, then `shortcutStart = chRank + 4*nodeCount`, `shortcutTarget = shortcutStart + 4*(nodeCount+1)`, `shortcutSeconds = shortcutTarget + 4*shortcutCount`, `shortcutVia = shortcutSeconds + 4*shortcutCount`, `total = shortcutVia + 4*shortcutCount`. Run the exact-size arithmetic check before allocating any view. `chShortcutCount === 0` must return null even when the byte size matches. `decodeGraph` must never throw for malformed bytes and must never return a half-built graph.
Create `routing/src/ch-query.ts` with the pure uphill bidirectional CH query. Its public result shape must be `{ nodes: number[]; seconds: number; distanceM: number }`. The forward search relaxes only edges whose target rank is strictly greater than the source rank. The backward search relaxes only incoming edges whose source rank is strictly greater than the settled node's rank. Use separate forward and backward edge access; never reuse the forward shortcut list in the backward search. Apply the shared traversal law in every scan: `if ((edgeFlags[e] & 1) !== 0) continue;`.
The stop rule is `if (forwardHeap.peekKey() + backwardHeap.peekKey() >= mu) break;`. Alternating the smaller frontier first is only a work-balance knob and must not affect optimality. Use the edge-based meet probe already present in astar.ts. After reconstructing the parent chains, recursively unpack every shortcut through its `via` field until only original edges remain. THEN compute `seconds` by summing `graph.edgeSeconds[findTraversableEdge(...)]` over consecutive original-edge pairs, and compute `distanceM` by summing haversine over that same unpacked path. Shortcut weights steer the search only; they never enter reported seconds.
Rewrite `routing/src/astar.ts` so `routeAstar` validates origin/goal, handles origin === goal, and delegates to the CH query for every other request. Preserve `AstarResult`, `deriveSteps`, `classifyTurn`, and all exported turn constants exactly. Delete the ALT machinery: `LANDMARK_SENTINEL_U16`, `cleanColumnsFor`, the `cleanColumnGraphs` memo, `metersTo`, `bound`, `potential`, the ALT/BH search body, and every landmark-table reference. Per D-04 there is exactly one search path.
Update `routing/src/index.ts` to export the existing public API and additively export `contractGraph`, `ContractionInput`, `ContractionResult`, and the four `CH_*` constants if the query implementation needs them. Export no landmark symbols.
Delete `routing/src/landmark.ts` and `routing/test/landmark.test.mjs` per D-05. Add the ARG3 codec cases G1-G3 to `routing/test/ch-query.test.mjs` using in-memory encode/decode vectors only; do not read or assert the committed mini-graph in this task, because that fixture is still ARG2 until task 3 regenerates it.
RE-PIN `routing/test/graph.test.mjs` IN THIS TASK. The codec flips to ARG3 here, so every in-memory G-case must be re-pinned here or the suite cannot go green. The routing runner auto-discovers ALL suites and parses NO argv, so `npm run routing:test` runs the whole `routing/test/` directory — there is no way to exclude `graph.test.mjs` from this task's verify.
`routing/test/graph.test.mjs` re-pinning, case by case:
- `syntheticInput()` (currently around line 74) — replace the `landmarks: {...}` block with a `ch:` block built from `contractGraph` over the same 3 edges, or with a hand-built CH block (rank of length 3, `shortcutStart` of length 4, matching target/seconds/via arrays). The fixture must produce a graph with at least one shortcut so the round-trip is meaningful.
- G1 (around line 100) — assert the CH fields round-trip: `chRank` length equals nodeCount, `shortcutStart` length equals nodeCount+1, `shortcutTarget`/`shortcutSeconds`/`shortcutVia` lengths equal `chShortcutCount`, and the values match the input exactly. Keep the existing coord/target/flag/weight/name/bbox assertions unchanged.
- G2 (around line 126) — unchanged: tampered magic still decodes to null.
- G3 (around line 134) — keep the ARG1 fixture rejection. The version tamper must change: version 1 still decodes to null, but the "version 3 tamper" case must become a version 4 tamper (the new `GRAPH_VERSION` is 3, so a buffer stamped 4 must be refused). A version-2 tamper may be added as well.
- G4 (around line 150) — keep the truncated/empty/sub-header/hostile-count assertions. The hostile header must stamp `GRAPH_MAGIC` and the NEW `GRAPH_VERSION` (3).
- G5 (around line 171) — unchanged: zero-copy views over the input buffer. Add `chRank.buffer === buf` as a CH zero-copy assertion.
- G6 (around line 185) — unchanged: name pool behavior.
- G7 (around line 195) — SUSPEND in this task. This is the ONLY case that reads the committed `routing/test/fixtures/mini-graph.bin`, which is still ARG2 until task 3 regenerates it. Replace its body with a `test.skip`-style no-op that documents the deferral (for example a case named `G7 committed ARG3 fixture — SUSPENDED until task 3 regenerates mini-graph.bin` whose body asserts `true`). Do NOT delete the case and do NOT change the committed fixture in this task. Task 3 re-pins G7 in place.
- G8 (around line 237) — re-pin the constants: `GRAPH_MAGIC === 0x41524733` ('ARG3') and `GRAPH_VERSION === 3`. `UNNAMED_NAME_ID` stays `0xffffffff`.
- G9 (around line 244) — re-pin the table round-trip to the CH tables: assert `chRank` restores exactly, `shortcutStart` restores exactly, `shortcutTarget`/`shortcutSeconds`/`shortcutVia` restore exactly, `chShortcutCount` matches, and the CH arrays are zero-copy views over the input buffer.
- G10 (around line 260) — re-pin the encode-validation cases to CH inputs: a missing `ch` block, a `chRank` whose length differs from nodeCount, a `shortcutStart` whose length differs from nodeCount+1, a `shortcutTarget` whose length differs from `chShortcutCount`, a `shortcutVia` entry outside `[0, nodeCount)`, a non-finite `shortcutSeconds` value, and a `chShortcutCount` that disagrees with the arrays. Each must throw an `Error` whose message starts with `encodeGraph: `.
- G11 (around line 280) — re-pin the decode rejection: `chShortcutCount = 0` in the header decodes to null even when the byte size matches (byte 60 is now `chShortcutCount`, not `landmarkCount`), and a monster `chShortcutCount` on a tiny buffer decodes to null by arithmetic before any allocation. Recompute the "exactly-sized" buffer arithmetic for the ARG3 layout (the L=0 landmark layout no longer applies — the equivalent is a buffer whose byte length matches an ARG3 layout with `chShortcutCount = 0`).
- The file's header comment block (around lines 1-51) must be rewritten to describe the ARG3 layout (72-byte header, `chShortcutCount` u32 @60, reserved u32 @64/@68, the CH suffix of `chRank` ++ `shortcutStart` ++ `shortcutTarget` ++ `shortcutSeconds` ++ `shortcutVia`) and the G1-G11 case list. Remove every reference to the landmark tables and to `0x41524732`.
RE-PIN `routing/test/astar.test.mjs` IN THIS TASK. Its cases split the same way:
- `loadMiniGraph()` (around line 92) reads the committed `routing/test/fixtures/mini-graph.bin`, which is still ARG2 in this task. The cases that use it are A1 (around line 187), A2 (around line 216), A4 (around line 274), A5 (around line 309), and A6 (around line 328). SUSPEND those five cases in this task with a documented `test.skip`-style no-op (the fixture is still ARG2 until task 3 regenerates it). Do NOT delete them and do NOT change the committed fixture in this task — task 3 re-pins them in place.
- A3 (around line 239) is in-memory: replace its `landmarks: {...}` block with a `ch:` block and keep the unreachable-pair law.
- A7 (around line 356) and A8 (around line 378) are structural: keep them, but extend A8 to assert the CH path (no `new Map(` in `astar.ts`/`ch-query.ts`).
- A9 (around line 478), A10 (around line 535), A12 (around line 401), and A13 (around line 612) are ALT-specific: per D-04/D-05 they are folded into the CH query suite (`routing/test/ch-query.test.mjs`) rather than retained. Their island fixture (around line 134) imports `../src/landmark.ts` and builds a `landmarks:` block — rewrite it to build a `ch:` block through `contractGraph`, and re-point the surviving assertions at the CH query. A11 (around line 561) asserts the ALT clean-column law in `astar.ts` source: replace it with the CH-law equivalent (the uphill rank filter exists in `ch-query.ts`, and `astar.ts` contains no landmark token).
The committed-fixture ARG3 decode pin (the G4 case added in task 3) and the un-suspension of G7 / A1 / A2 / A4 / A5 / A6 all belong to task 3, after the fixture is regenerated through the ARG3 encode path and the ARG2 vector is preserved.
Replace the landmark stage in `routing/test/fixtures/build-mini-graph.mjs` with a CH stage. Import `contractGraph` from `../../src/contraction.ts`, call it with the 25-node grid's directed edges, and pass its result as `ch:` to `encodeGraph`. Before writing, fail loud unless: `new Set(rank).size === 25`; every rank is in [0,25); the shortcut count is within the expected grid band; every `via` is a node id in [0,25); every shortcut's seconds equal its two constituent edge weights; and the graph remains strongly connected.
Update `bot/test/webhook.test.mjs`: replace the `landmark.ts` import and `landmarkBlock()` with a `chBlock(nodeCount, edges)` helper that computes `contractGraph` and returns the CH block. Replace every `landmarks: { ... }` argument to `encodeGraph` with `ch: { ... }`. There are three graph builders in this file; all must use the ARG3 path.
Update `bot/test/guidance-cron.test.mjs`: replace the pinned-anchor 2-node graph builder with a `contractGraph` call and encode with `ch:`. The 2-node graph must contract successfully; no anchor pin is allowed.
Run `node routing/test/fixtures/build-mini-graph.mjs` to regenerate `mini-graph.bin` as ARG3 and confirm the printed node/edge/shortcut/byte counts. After regeneration succeeds and the ARG2 rejection vector is preserved, re-pin `routing/test/graph.test.mjs` for ARG3 and add the committed-fixture G4 case: it must read `routing/test/fixtures/mini-graph.bin`, assert ARG3 magic/version, 25 nodes, 80 edges, rank length 25, and at least one shortcut, and assert that `routing/test/fixtures/mini-graph-arg2.bin` decodes to null. Keep this graph.test.mjs G4 pin in this task so fixture regeneration, ARG2 preservation, and the committed-fixture assertion are owned atomically.
UN-SUSPEND the committed-fixture cases that task 2 parked, now that `mini-graph.bin` is ARG3:
- `routing/test/graph.test.mjs` G7 — replace the suspended no-op with the real case: read `routing/test/fixtures/mini-graph.bin`, assert it decodes non-null, assert 25 nodes, 80 edges, 3 names, maxSpeed 60, exactly ONE oneway flag, the hand-built grid bbox, both speed classes present, `chRank` length 25, and at least one shortcut. Delete the ARG2 landmark assertions (landmarkCount, landmarkQuantumS, landmarkTo/landmarkFrom lengths, the anchor-cell loop). Also assert that `routing/test/fixtures/mini-graph-arg2.bin` decodes to null.
- `routing/test/astar.test.mjs` A1, A2, A4, A5, A6 — replace each suspended no-op with its real body. `loadMiniGraph()` now decodes an ARG3 graph; A1's 625-pair exhaustive Dijkstra equivalence, A2's oneway law, A4's `deriveSteps` L-path, A5's depart/arrange law, and A6's `classifyTurn` pins must all be restored and green. If any assertion depended on an ALT-only field, replace it with the CH equivalent (for example `chRank`/`chShortcutCount` instead of `landmarkCount`).
<threat_model>
Trust Boundaries
| Boundary | Description |
|---|---|
| R2 object -> Worker decoder | The R2-held road-graph bytes are untrusted to the decoder: corruption, truncation, hostile counts, or misalignment must produce null, never a throw or half-built graph. |
| Request -> Worker -> CH query | Untrusted route parameters reach the search only after the existing Bearer claim and bbox validation; the CH query must not broaden that surface. |
| Build pipeline -> binary | The Overpass-derived graph is transformed by build-time contraction into the persisted hierarchy; a stale hierarchy against changed topology is a build-artifact discipline problem. |
STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|---|---|---|---|---|
| T-08.1-01 | Tampering / DoS | decodeGraph ARG3 header and section arithmetic |
mitigate | layoutFor computes every offset in one place; exact-size validation runs before any typed-array allocation; chShortcutCount === 0 returns null; malformed bytes return null rather than throwing. Pinned by graph.test.mjs G3/G4/G11 successors. |
| T-08.1-02 | DoS | Misaligned Float32Array/Uint32Array CH views |
mitigate | Keep HEADER_SIZE = 72 for 8-byte f64 alignment and align the CH suffix with (namePool + namePoolLen + 3) & ~3; G1 round-trip and hostile-buffer tests decode the committed ARG3 fixture without RangeError. |
| T-08.1-03 | Tampering | Directed shortcut confusion across oneway edges | mitigate | Shortcuts are directed only; forward and backward searches use separate edge access; bit0 edges are never relaxed in forward, backward, or witness scans; the directed ring/chord/spur sweep compares both directions with the Dijkstra oracle. |
| T-08.1-04 | Tampering | Stale CH hierarchy after topology or weight changes | mitigate | ARG3 is rebuilt and re-uploaded as one whole artifact under the unchanged key; docs/ROAD-GRAPH.md records that the hierarchy is valid only for the topology and weights it was built against. |
| T-08.1-05 | DoS | Unbounded build-time witness search | mitigate | Witness search is bounded by candidate distance and max(CH_WPS_MAX_SETTLED, CH_WPS_SETTLED_PER_DEGREE * localDegree); build prints witness-search and budget-hit counts. |
| T-08.1-06 | Tampering / DoS | Shortcut explosion inflating binary and query cost | mitigate | CH_SHORTCUT_EXPLOSION_RATIO = 4 fails the build loud; the existing 1-80 MB envelope remains enforced; no fixed payload ceiling is invented. |
| T-08.1-07 | Information Disclosure | Coordinates, route payloads, or user identity in logs | accept | The phase adds no logging and changes no /api/route response shape; existing category-only log law and Bearer-claim identity remain unchanged. No new log statement may be introduced in routing modules. |
| T-08.1-08 | Spoofing / Tampering | Dormant ALT path or second decoder producing a different answer | mitigate | D-04/D-05 deletion is pinned structurally: no ALT tokens in astar.ts, landmark.ts absent, no landmark exports, one decode path in the Worker; routeAstar delegates every request to the CH query. |
| T-08.1-09 | DoS | CPU exhaustion on the request path | mitigate | The uphill query uses the peekKey() stop rule and returns null for unreachable pairs; the local CPU gate and tightened drift guard in Plan 02 are the enforcement point. |
| </threat_model> |
npm run routing:test && npm test -w bot && npm run routing:typecheck && npm run bot:typecheck && npm run web:typecheck
Also verify structural deletion and the one-path law:
test ! -e routing/src/landmark.ts && test ! -e routing/test/landmark.test.mjs
grep -v '^[[:space:]]*\*' routing/src/astar.ts | grep -v '^[[:space:]]*//' | grep -c 'landmarkTo\|landmarkFrom\|cleanColumnsFor\|LANDMARK_SENTINEL'
grep -c 'decodeGraph(' bot/src/routing/worker-router.ts
The committed-fixture cases suspended in task 2 (G7; A1/A2/A4/A5/A6) must be un-suspended by task 3 — no case in the plan may be left skipped at the end:
grep -c 'SUSPENDED\|skip' routing/test/graph.test.mjs
grep -c 'SUSPENDED\|skip' routing/test/astar.test.mjs
Expected: routing and bot suites green; all typechecks green; structural ALT count 0; Worker decodeGraph count 1; both suspension counts 0.
<success_criteria>
- CH construction and query behavior is proven by the routing unit suite, including a hand-computed fixture, overlay witness law, oneway direction, and original-edge seconds.
- ARG3 is the only encoded/decoded format; ARG1 and ARG2 decode to null through the existing fail-honest path.
routeAstaris the sole exported search and is CH-backed for every request; no ALT runtime code or landmark payload remains.- The committed mini-graph and both bot graph builders are ARG3-shaped and produced through the same encode path.
npm run routing:test,npm test -w bot, and all three typechecks pass. </success_criteria>