diff --git a/.planning/phases/04-destination-search-kl-gazetteer/04-02-SUMMARY.md b/.planning/phases/04-destination-search-kl-gazetteer/04-02-SUMMARY.md new file mode 100644 index 0000000..8e01284 --- /dev/null +++ b/.planning/phases/04-destination-search-kl-gazetteer/04-02-SUMMARY.md @@ -0,0 +1,200 @@ +--- +phase: 04-destination-search-kl-gazetteer +plan: 02 +subsystem: search +tags: [search, golden-corpus, d14-gate, overpass, sqlite, nominatim, tdd] +requires: + - "plan 04-01 search workspace (contracts, normalizer, scorer, gazetteer, loader)" + - "Node 24 type-stripping (.mjs script imports ../src/index.ts)" + - "node:sqlite (built into Node 24.18.0 — converter output path)" +provides: + - "D-13 golden corpus v1 — 75 pinned hand-written cases in 6 categories with real negatives" + - "D-14 gate — `npm run search:corpus` (exit 0 only on top-3 >= 0.9 + zero leaks) and the in-process CO1-CO8 gate inside `npm run search:test`" + - "D-06 tool — search/scripts/overpass-topup.mjs (one UA'd request, PROPOSAL output, soft-exit 0 on busy, --source=overpass|nominatim|fixture)" + - "D-05 tool — search/scripts/json-to-sqlite.mjs (same id set, real .sqlite via node:sqlite, ODbL notice embedded)" + - "docs/SEARCH-QUALITY.md — the bake-off close: measured rates, lever None, coordinate verification" +affects: + - "plan 04-03 (bot /cari handler ships against a closed, green corpus gate)" + - "plan 04-04 (web deep-link reader — same module, corpus unchanged)" + - "future top-ups (the Overpass proposer + SEARCH-QUALITY record define the maintenance loop)" +tech-stack: + added: + - "node:sqlite DatabaseSync (converter only — repo tool, zero added deps)" + patterns: + - "measure-once/report-twice: the runner exports measureCorpus(); the suite asserts the same numbers in-process" + - "main-guarded .mjs scripts (CLI when invoked, importable library otherwise)" + - "source-introspection law gates (TP1/TP6/TP7, CV2/CV3) following bot/test/privacy.test.mjs" +key-files: + created: + - search/test/fixtures/golden-corpus.json + - search/scripts/gazetteer-corpus.mjs + - search/test/corpus.test.mjs + - search/scripts/overpass-topup.mjs + - search/test/fixtures/overpass-response.json + - search/scripts/json-to-sqlite.mjs + - search/test/topup.test.mjs + - search/test/converter.test.mjs + - docs/SEARCH-QUALITY.md + modified: + - search/package.json (corpus delegate) + - package.json (search:corpus delegate) + - search/src/data/kl-gazetteer.json (pavilion-kl coordinate fix) +decisions: + - "Corpus authored to the shipped 30-entry gazetteer per the plan's ILLUSTRATIVE clause: dropped non-existent targets (Intermark, Cheras South, MV — 2-char queries sit below the scorer's min-length guard by design) and pinned equivalents that exist (IOI, Pudu, Tmn Perdana, Kl Towr, ...)" + - "Near-miss negatives chosen with measured margins: Penang 0.625 and Kampung Cina ~0.58, both in [0.55, 0.72) — lowering the admission line leaks them and fails CO2+CO8" + - "measureCorpus() exported from the runner and reused by corpus.test.mjs — one measurement, two reporters, no drift" + - "--fixture= implies --source=fixture (a fixture flag silently hitting the live network would be a trap)" + - "Converter emits a REAL .sqlite via node:sqlite (probed available in Node 24.18.0) — no dependency added; ODbL notice embedded in a metadata table + NOTICE sidecar" +metrics: + duration: "~14 min executor time" + completed: 2026-10-23 + tasks: 3 + filesCreated: 9 + filesModified: 3 +--- + +# Phase 4 Plan 2: Golden Corpus + Bake-off Close Summary + +**One-liner:** the D-13/D-14 experiment closed mechanically — a 75-case pinned corpus measured +top-3 68/68 (100%) with 7/7 negatives refused at margin 0.643 < 0.72 on the FIRST run +(lever: None), enforced twice (`npm run search:corpus` exit-code gate + CO1-CO8 inside +`npm run search:test`), alongside the two forkability tools: an Overpass top-up proposer +(one UA'd request, PROPOSAL-only output, exit-0-on-busy) and a JSON→SQLite converter that +preserves the id set exactly. + +## What Was Built + +### Task 1 — pinned golden corpus + D-14 runner + Nyquist gate (RED `dea7971` → GREEN `b2689f7`) +- `golden-corpus.json`: 75 hand-written cases — landmark 17, abbreviation 15, glued 7, + typo 14, alias 15, negative 7 — every positive pinning a real gazetteer id + (start_param-safe charset), `threshold.top3_hit_rate: 0.9`. 16 non-negative cases are not + string-equal to any alias/name (tautology floor is 15). Negatives include two MEASURED + near-misses (Penang 0.625, Kampung Cina ~0.58) so a lowered admission line fails CO2+CO8. +- `gazetteer-corpus.mjs`: imports the SAME `src/index.ts` the bot uses (D-03), measures + top-1/top-3 + per-category rates + avg ms/query + the refusal margin, prints MISSED/LEAKED + blocks, the threshold line, and the verbatim escalation ladder on failure; exits 0 only on + threshold + zero leaks. Exports `measureCorpus()`; main-guarded so the suite can import it. +- `corpus.test.mjs`: CO1-CO8 — threshold, zero leaks, tautology floor, dangling-id charset, + size+category floors, per-category reporting, and the pinned refusal margin (replicating + the scorer's two-level whole-string+token fuzzy shape over `damerau`). +- Delegates: `search:corpus` (root), `corpus` (search workspace). +- Measured: **top-3 68/68 (100.0%), negatives refused 7/7, margin 0.643 < 0.72, 1.07 ms/query — + threshold met on the first run.** The two top-1 misses ("KCLC" 3-way fuzzy tie broken + id-ascending; "Sunway Vellocity" legitimately riding PREFIX via the "Sunway" alias) are + documented expected behaviour. + +### Task 2 — Overpass top-up (D-06) + JSON→SQLite converter (D-05) (RED `ec98c8d` → GREEN `c6a5f2b`) +- `overpass-topup.mjs`: maintainer-time repo tool under the six hard laws — never a runtime + dependency (TP1), ONE request per run via a single shared `requestJson` helper for both the + Overpass and Nominatim paths (TP7), mandatory identifying User-Agent (TP2 — live-verified + 406-without), bbox-bounded single-family query pinned verbatim (TP3), busy/502/429/timeout/ + unparseable → `Overpass instance busy — retry later` + exit 0 (TP4), PROPOSAL-only output + whose source never names the curated file (TP6). Flags: `--source=overpass|nominatim|fixture`, + `--endpoint=`/`OVERPASS_ENDPOINT`, `--bbox=`, `--q=`, `--tag-family=`, `--fixture=`, `--out=`. + `--fixture` implies the fixture source (offline by default when a fixture is named). +- `overpass-response.json`: committed real-shape Overpass response (6 KL elements — nodes with + lat/lon, ways/relation with center) for deterministic offline tests. +- `json-to-sqlite.mjs`: reads the same committed `kl-gazetteer.json`, emits + `kl-gazetteer.sqlite` (places + metadata tables) via `node:sqlite` — zero added + dependencies — with the SAME id set (CV1 reads it back through DatabaseSync) and the ODbL + notice embedded in the metadata table + a NOTICE sidecar (ODbL §4.4, T-04-20). Deterministic: + dataset order, no timestamps. +- Suites: TP1-TP7 + CV1-CV3 all green; grep gates confirm zero `overpass`/`sqlite` mentions + under `bot/src/` and `web/src/`. + +### Task 3 — bake-off close + coordinate verification (`99bc9c9`) +- `docs/SEARCH-QUALITY.md`: the close record — metric table, per-category table, escalation + lever section naming exactly one lever (**None — met on the first run; no + abbreviations.ts row, no alias, no corpus edit**), corpus corrections (none, threshold + still 0.9), reproduction commands, known limitations (hand-written corpus scope, + refusal-not-routing in v1, the recorded fuzzy-tier scaling decision). +- Coordinate verification (live Nominatim, identifying UA, 1 req/s, 10 seed entries): + **8/10 within 0.01°.** `bukit-bintang` outlier = area-vs-point discrepancy (no change). + `pavilion-kl` = real ~2 km coordinate error in the 04-01 seed → corrected to + (3.149, 101.7132), post-fix delta 0.0002° (Rule 1 fix below). +- Full sweep green: `search:test` 57/57, `search:typecheck`, `bot:test` 50/50, `web:test` 33/33. + +## Commits + +| task | commit | message | +| ---- | ------ | ------- | +| 1 RED | `dea7971` | test(04-02): RED — golden-corpus D-14 gate suite (CO1-CO8) | +| 1 GREEN | `b2689f7` | feat(04-02): pinned golden corpus + D-14 corpus runner + Nyquist gate | +| 2 RED | `ec98c8d` | test(04-02): RED — D-06 topup law suite (TP1-TP7) + D-05 converter suite (CV1-CV3) | +| 2 GREEN | `c6a5f2b` | feat(04-02): Overpass top-up proposer (D-06) + JSON-to-SQLite converter (D-05) | +| 3 | `99bc9c9` | docs(04-02): close the D-14 bake-off — threshold met on the first run, lever None | + +## Verification Results + +- `npm run search:corpus` — exit 0: `top-3 68/68 (100.0%)`, `negatives refused 7/7`, + `refusal margin 0.643 < 0.72`, `threshold top3 >= 90.0% — PASS` +- `npm run search:test` — **57 passed, 0 failed** (CO1-CO8 + TP1-TP7 + CV1-CV3 + the 39 from 04-01) +- `npm run search:typecheck` — green +- `npm test -w bot` (50) / `npm test -w web` (33) — green, untouched +- `grep -rl 'overpass' bot/src/ web/src/` → 0; `grep -rl 'sqlite' bot/src/ web/src/` → 0 +- Overpass script proven offline: committed fixture → 6-candidate PROPOSAL, exit 0; + busy-shaped fixture → exactly the soft-exit line, exit 0 +- Converter acceptance probe: id set preserved exactly (sqlite round-trip) + +## Deviations from Plan + +**1. [Rule 1 - Bug] `--fixture=` silently hit the live network** +- **Found during:** task 2 (GREEN probes) +- **Issue:** the first manual probe passed `--fixture=...` without `--source=fixture`; the + script fell through to a live Overpass request, so the busy-fixture acceptance check would + have printed a live PROPOSAL instead of the soft-exit line. +- **Fix:** presence of `--fixture` now implies the fixture source. (The accidental live run + made exactly one UA'd request — LAW 2 was never violated — and usefully proved the live + Overpass path works from this environment with the mandated User-Agent.) +- **Files:** search/scripts/overpass-topup.mjs + +**2. [Rule 1 - Bug] TP3 failed — no literal bbox tuple in source** +- **Found during:** task 2 (GREEN) +- **Issue:** the query interpolated `(${bbox})`, so the source contained no literal + parenthesized bbox tuple for TP3's regex. +- **Fix:** the canonical default query is now pinned verbatim — + `nwr["tourism"="attraction"](3.05,101.55,3.25,101.80)` — with flag substitutions building + the same shape for overrides. +- **Files:** search/scripts/overpass-topup.mjs + +**3. [Rule 1 - Bug] pavilion-kl committed coordinate ~2 km west of the mall** +- **Found during:** task 3 (the plan's prescribed Nominatim spot-verification) +- **Issue:** seed entry carried lng 101.695; Nominatim (and the mall's true position) place + Pavilion Kuala Lumpur at ~(3.1492, 101.7130). +- **Fix:** entry corrected to (3.149, 101.7132); recorded in SEARCH-QUALITY.md under + "Coordinate verification". Corpus is coordinate-blind; all suites stayed green. +- **Files:** search/src/data/kl-gazetteer.json, docs/SEARCH-QUALITY.md + +**4. [Planner-note] corpus illustrative strings resolved against the real gazetteer** +- The plan marks its query strings ILLUSTRATIVE and requires pinning to existing ids. Targets + with no gazetteer entry (The Intermark, Cheras South) and queries the scorer must refuse by + design ("MV" — 2 chars, below the min-length guard) were replaced with equivalent pinned + cases (IOI, Pudu, Tmn Perdana, Kl Towr, Merdeaka 118, ...). Floors all exceeded. + +## Auth Gates + +None — no secrets, no auth flows. The only network surfaces are maintainer-time repo tools +(Overpass/Nominatim/Nominatim-verify), all of which worked from this environment with +identifying User-Agents. + +## Known Stubs + +None in the shipped-search sense — every artifact is wired to real behavior. The plan's +"stub" language for the converter means backend #2 itself is deliberately unbuilt (D-05, +post-v1); the converter that keeps its format reachable is a full, working implementation. + +## Threat Flags + +None — the built surface is exactly the plan's threat model (T-04-12…T-04-20): the corpus +tampering gates (CO3/CO6 + the 0.9 threshold grep), the Overpass runtime-dependency and +never-writes laws (TP1/TP6/TP7), the busy-soft-exit (TP4), and the ODbL notice traveling +with converter output (T-04-20, embedded in the metadata table). + +## TDD Gate Compliance + +PASSED — task 1: RED `dea7971` (8 CO cases failing on missing corpus) → GREEN `b2689f7`; +task 2: RED `ec98c8d` (8 TP/CV cases failing on missing scripts) → GREEN `c6a5f2b`. Task 3 +is a non-TDD measurement/record task (no RED/GREEN applies). RED failures were verified +before each GREEN commit. + +## Self-Check: PASSED