93 lines
6.4 KiB
Markdown
93 lines
6.4 KiB
Markdown
---
|
||
phase: 8
|
||
slug: alt-landmark-preprocessing-routing-backend-v2
|
||
status: planned
|
||
nyquist_compliant: true
|
||
wave_0_complete: true
|
||
created: 2026-10-25
|
||
---
|
||
|
||
# Phase 8 — Validation Strategy
|
||
|
||
> Per-phase validation contract for feedback sampling during execution.
|
||
|
||
---
|
||
|
||
## Test Infrastructure
|
||
|
||
| Property | Value |
|
||
|----------|-------|
|
||
| **Framework** | hand-rolled registry (`routing/test/harness.mjs`) + `node:test`-style runner, **zero deps** |
|
||
| **Config file** | none — `routing/test/runner.mjs` + npm workspace scripts |
|
||
| **Quick run command** | `npm run routing:test` |
|
||
| **Full suite command** | `npm run routing:test && npm test -w bot && npm run routing:typecheck && npm run bot:typecheck` |
|
||
| **Estimated runtime** | ~10–30 s (suite); big-graph gates are maintainer-time (build/measure) |
|
||
|
||
---
|
||
|
||
## Sampling Rate
|
||
|
||
- **After every task commit:** Run `npm run routing:test`
|
||
- **After every plan wave:** Run full suite (routing + bot + both typechecks)
|
||
- **Before `/gsd-verify-work`:** Full suite must be green
|
||
- **Max feedback latency:** 30 seconds (suite); the real-graph gates are phase-gate-only
|
||
|
||
---
|
||
|
||
## Per-task Verification Map
|
||
|
||
| task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||
| W0 baseline recorder | 01 | 1 | SC-2 (D-03 prereq) | — | N/A | script | `node routing/scripts/measure-optimality-baseline.mjs` → committed record | ✅ W0 | ✅ green |
|
||
| ARG1 fixture preservation | 01 | 1 | D-01 | T-08-02 | old binary → decode null (fail-honest) | unit | `npm run routing:test` (G rejection case lands in 08-02) | ✅ W0 | ✅ green (bytes frozen) |
|
||
| SC-1′ extended-sample flag | 01 | 1 | SC-1 (D-04) | — | N/A | measurement gate | `node routing/scripts/measure-route-cpu.mjs --extended` | ✅ W0 | ✅ green (pre-ALT honest FAIL: p95 98.9 ms, n=53) |
|
||
| ARG2 codec G-suite | 02 | 2 | D-01/D-02 | T-08-01/02 | exact-size-before-allocation; version + landmarkCount laws; hostile header on tiny buffer rejects BEFORE view allocation | unit | `npm run routing:test` (G cases) | ✅ 08-02 | ✅ green (74/74; ARG1 vector + monster landmarkCount reject pre-allocation) |
|
||
| ALT potentials A-suite (A1 re-pin + island A9/A10 + A11 pin) | 03 | 3 | SC-2 | T-08-05/06 | sentinel-safe potentials; equivalence sealed under ALT | unit (A1 625-pair sweep + new cases) | `npm run routing:test` (A cases) | ✅ 08-03 | ✅ green (RED `92b97fc` → GREEN `ddd2f3c`; A1 625 pairs + A9 169-pair island + A10 skip + A11 structural, ALT live) |
|
||
| Rollout pins (upload-agnostic + GRAPH_KEY) | 03 | 3 | D-01 | T-08-08 | upload stays format-agnostic; loader keeps ONE decode path | structural grep (A8-shape) | `npm run routing:test` (RO1/RO2) | ✅ 08-03 | ✅ green (`8fac68b`; RO1/RO2 + full battery: routing 79/79, bot 126/126, both typechecks) |
|
||
| Build preprocessing stage | 04 | 4 | SC-1 (D-02) | T-08-09/10/11 | coverage + quantum gates before write; L scaling rule | script gate | `node routing/scripts/build-road-graph.mjs` + `--check` | ⚙ extends | ⬜ pending |
|
||
| Real ARG2 rebuild + check | 04 | 4 | SC-1 prereq | T-08-09 | envelope + coverage green on the real binary | script gate (network, maintainer-time) | `node routing/scripts/build-road-graph.mjs --check routing/data/kl-roadgraph.v1.bin` | ⚙ extends | ⬜ pending |
|
||
| CPU gate + extended verdict | 05 | 5 | SC-1 | T-08-16 | print-the-measurement; pre-committed L ladder | measurement | `node routing/scripts/measure-route-cpu.mjs` (+ `--extended`) | ✅ extends | ⬜ pending |
|
||
| Optimality gate (D-03) | 05 | 5 | SC-2 | T-08-13/15 | epsilon mirror; seconds+verdict only; record immutable | offline gate | `node routing/scripts/measure-optimality-gate.mjs` | ❌ new | ⬜ pending |
|
||
| Budget-record c3 truth-out | 05 | 5 | SC-3 | T-08-14 | gate law + audit vectors re-anchored; no accepted-degradation wording | suite + grep | `node scripts/test/budget-audit.test.mjs` | ⚙ extends | ⬜ pending |
|
||
| Docs retirement (D-05) | 05 | 5 | SC-3 | T-08-14 | quoted measured numbers; honest remainders | doc acceptance | review leg (supported by printed gate output) + grep gates | ❌ new | ⬜ pending |
|
||
|
||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||
|
||
---
|
||
|
||
## Wave 0 Requirements
|
||
|
||
- [x] Test framework: none needed (zero-dep harness present)
|
||
- [x] **Baseline record + recorder tool** (D-03 prerequisite — MUST be first task before ARG2 codec lands; research Pitfall 1) — DONE 08-01 (`alt-baseline.json`, immutable, ARG1-version-guarded)
|
||
- [x] Extended-sample flag (D-04) — DONE 08-01 (`--extended`; default 25-route mode byte-compatible, budget-audit 19/19)
|
||
- [x] `fixtures/mini-graph-arg1.bin` — bytes preserved from the current committed ARG1 fixture (decode-rejection vector) — DONE 08-01
|
||
- [x] `fixtures/build-mini-graph.mjs` gains a landmark stage (tiny L; ARG2 output) — DONE 08-02 (L=2, anchors [0,18], quantum 0.1 s, coverage 1.0; byte-deterministic)
|
||
- [x] New G cases (ARG2 codec + rejection matrix) — DONE 08-02 (G9–G11; island A case remains for 08-03)
|
||
- [x] Upload-unchanged structural pin case — DONE 08-03 (RO1/RO2 in `routing/test/rollout.test.mjs`)
|
||
|
||
*Constraint re-verified in planning: `budget-audit` (Phase 7) re-runs `measure-route-cpu.mjs` — keep its CLI contract or account for the flag addition.*
|
||
|
||
---
|
||
|
||
## Manual-Only Verifications
|
||
|
||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||
|----------|-------------|------------|-------------------|
|
||
| D-05 docs retirement wording | SC-3 | Honest tone is a human judgment; the evidence itself is automated | Read `docs/KNOWN-LIMITATIONS.md` retirement record against the printed gate numbers |
|
||
| R2 upload of the v2 binary (rollout act) | D-01 | Account-level wrangler/R2 act, maintainer-time | `node routing/scripts/upload-road-graph.mjs` after cargo edits are done → roundtrip `--check` |
|
||
|
||
*All other phase behaviors have automated verification.*
|
||
|
||
---
|
||
|
||
## Validation Sign-Off
|
||
|
||
- [x] All tasks have `<automated>` verify or Wave 0 dependencies
|
||
- [x] Sampling continuity: no 3 consecutive tasks without automated verify
|
||
- [x] Wave 0 covers all MISSING references
|
||
- [x] No watch-mode flags
|
||
- [x] Feedback latency < 30 s (suite tasks; the real-graph build/gates are maintainer-time phase-gate-only, per the sampling note above)
|
||
- [x] `nyquist_compliant: true` set in frontmatter
|
||
|
||
**Approval:** planning complete 2026-10-25 (plans 08-01…08-05); execution pending user approval
|