---
phase: 04-destination-search-kl-gazetteer
plan: 01
subsystem: search
tags: [search, gazetteer, normalizer, damerau, scoring, licensing, workers, tdd]
requires:
  - npm workspaces root (theme/bot/web)
  - Node 24 type-stripping (corpus-runner pattern)
provides:
  - "@araharah/search workspace — the shared SearchSource seam (D-03)"
  - "GazetteerEntry / SearchQuery / SearchHit / SearchSource contracts (imported by plans 04-03/04-04)"
  - "normalize() — symmetric Malay place-name normalizer (rule table + folding + glued split)"
  - "damerau() — hand-rolled OSA distance, zero deps"
  - "TIER + FUZZY_ADMIT_SIMILARITY — disjoint-tier scorer + the 0.72 admission floor"
  - "createGazetteerSource() / search() / byId() — the in-Worker gazetteer backend"
  - "30-entry curated kl-gazetteer.json with locals' aliases (D-08)"
  - "GPL-3.0 LICENSE + LICENSES.md (ODbL-1.0 data notice) + corrected README"
affects:
  - "plan 04-02 (corpus runner imports the module; docs/SEARCH-QUALITY.md)"
  - "plan 04-03 (bot /cari handler imports createGazetteerSource; Worker bundle marker grep)"
  - "plan 04-04 (web deep-link reader imports the same module; L2 web-bundle marker half arms)"
tech-stack:
  added:
    - "search/ — fifth npm workspace, buildless (pure .ts + .json, zero runtime deps)"
    - "esbuild import attributes: import ... with { type: 'json' } (Node 24 + esbuild 0.28.2 + Vite)"
  patterns:
    - "hand-rolled zero-dependency test runner (bot/test precedent)"
    - "lazy module-scope memoization (bot/test/initdata.test.mjs precedent)"
    - "header-contract module comments (bot/src/security/initdata.ts precedent)"
key-files:
  created:
    - search/package.json
    - search/tsconfig.json
    - search/src/types.ts
    - search/src/rules/abbreviations.ts
    - search/src/normalize.ts
    - search/src/distance.ts
    - search/src/score.ts
    - search/src/data/kl-gazetteer.json
    - search/src/gazetteer.ts
    - search/src/source.ts
    - search/src/index.ts
    - search/test/runner.mjs
    - search/test/harness.mjs
    - search/test/normalizer.test.mjs
    - search/test/distance.test.mjs
    - search/test/score.test.mjs
    - search/test/gazetteer.test.mjs
    - search/test/loader-env.test.mjs
    - LICENSE
    - LICENSES.md
  modified:
    - package.json (workspaces + search:test/search:typecheck delegates)
    - package-lock.json (workspace linkage)
    - README.md (licence section, four-package table, layout, commands)
decisions:
  - "Abbreviation table is DATA: 49 vendored rows (JUPEM via Nominatim PR #4056) + explicit Presint/Prsn vowel-drop + MS 2039 A3 Kpg forms; extends by file edit (D-09)"
  - "Canonical normalizer form is the LONG form — 'Jln Ampang' and 'Jalan Ampang' normalize identically (D-09 symmetry)"
  - "TIER bands are disjoint literals: EXACT 1.00 / ALIAS_EXACT 0.90 / PREFIX 0.75 / TOKEN_SET 0.65 / FUZZY = 0.60 CEILING (a fuzzy score is fuzzySim × 0.60, never a 0.00 multiplier)"
  - "Admission is tiered: cheap tiers always pass, FUZZY only at similarity >= 0.72 — 'KCLC' admits (0.45), 'Penang' refuses ([])"
  - "TOKEN_SET evaluates before PREFIX (the plan's S5 vector pins the token-boundary case)"
  - "Dataset is static-imported with the 'with { type: json }' attribute — proven to inline under esbuild; loader index built lazily (never at module scope)"
metrics:
  duration: "~18 min executor time (three server restarts extended wall clock; state recovered from filesystem each time)"
  completed: 2026-10-23
  tasks: 3
  filesCreated: 20
  filesModified: 3
---

# Phase 4 Plan 1: Shared Search Module Summary

**One-liner:** `@araharah/search` — a buildless, zero-runtime-dependency search workspace (JUPEM
abbreviation table as data, symmetric Malay normalizer, hand-rolled OSA Damerau, disjoint-tier
scorer with a 0.72 fuzzy-admission floor, 30-entry curated KL gazetteer with locals' aliases, lazy
three-environment loader) plus the repo's GPL-3.0/ODbL licensing — `search("KLCC")` resolves
`suria-klcc` at 0.90 and `search("Kuching")` honestly refuses `[]`.

## What Was Built

### Task 1 — workspace scaffold + contracts + normalizer (RED `ebcd542` → GREEN `a9f9ab3`)
- Fifth npm workspace `search/` (`@araharah/search`, private, type: module, **zero runtime deps**)
  with root delegates `search:test` / `search:typecheck`; buildless — consumers import
  `@araharah/search/src/index.ts` subpaths with extensions (the `@araharah/theme` precedent).
- `types.ts`: the verbatim `GazetteerEntry` / `SearchQuery` / `SearchHit` / `SearchSource` contracts
  from RESEARCH §Pattern 1 — `search()` async (empty array = honest refusal), `byId()` separate and
  synchronous. Downstream plans import these names.
- `rules/abbreviations.ts`: the JUPEM-derived table as pure DATA — 49 rows (48 from the Nominatim
  transcription + the explicit `Presint → Prsn` vowel-drop row), inverted once by a single generic
  `buildAbbreviationMap()` (abbreviation → full forms in table order; full form → itself).
- `normalize.ts`: NFKD fold → lowercase → punctuation collapse (keeps `/` for "Jln 3/1") →
  whole-token expansion (first full form wins; canonical = long form) → glued-word split
  (longest-starter-first, remainder ≥ 3 chars) → collapse. Idempotent (T7), symmetric (T2).
- T1–T9: rule vectors, symmetry, whole-token-only law ("Pdang" never expands), slash preservation,
  glued split, vowel-drop, idempotence, folding, and the table-is-data gate (one function, ≥ 47 rows).

### Task 2 — Damerau distance + disjoint-tier scorer (RED `25ee1e9` → GREEN `6368bad`)
- `distance.ts`: ~40-line OSA Damerau-Levenshtein, three-row band with the adjacent-transposition
  term, **zero imports** (D-07). Pinned by 9 hand-computed vectors (D1–D9) incl. `klcc/kclc = 1`
  and `sunway/sunwya = 1`.
- `score.ts`: `TIER = { EXACT: 1.00, ALIAS_EXACT: 0.90, PREFIX: 0.75, TOKEN_SET: 0.65, FUZZY: 0.60 }`
  — strict literals, strictly disjoint bands; FUZZY is the band **ceiling** (score = fuzzySim × 0.60).
  `FUZZY_ADMIT_SIMILARITY = 0.72` exported for source.ts and the corpus runner. `scoreOne()` order:
  min-length guard (sub-3-char sides → 0 even on exact) → EXACT → TOKEN_SET (token-boundary superset)
  → PREFIX (mid-word) → fuzzy residual (max of whole-string and token-level similarity × 0.60).
  `scoreCandidate()` scores [name, ...aliases], demotes an exact-ALIAS hit to ALIAS_EXACT so a
  name-exact outranks it, returns the RAW matched string (corpus provenance), and enforces the
  tiered admission (sub-floor fuzzy → null). S1–S7 green.

### Task 3 — gazetteer + lazy loader + licensing (RED `f1b31ca` → GREEN `8579da9`)
- `data/kl-gazetteer.json`: 30 hand-curated KL-metro landmarks (10 required + 20 suggested),
  canonical OSM names with locals' aliases as first-class data (`"KLCC"` → suria-klcc, `"DUKE"` →
  the-duke, `"Sg Wang"` → sungei-wang), ODbL-1.0 + OSM attribution header traveling inside the
  file, pretty-printed 2-space.
- `gazetteer.ts`: the three laws — lazy memoized index never at module scope (Workers startup,
  T-04-04); static `import ... with { type: "json" }` (D-07); no logging/storage (D-15). Names and
  aliases normalized ONCE at index-build (D-09 symmetry).
- `source.ts`: `createGazetteerSource()` — name "gazetteer", async `search()` + synchronous
  slug-validated `byId()` (honest null); deterministic ranking (score DESC, id ASC). The tiered
  admission makes "Kuching"/"Penang" refuse `[]` — never a fabricated pin (T-04-07).
- `index.ts`: the named-only public API.
- Licensing: **LICENSE** = the verbatim GPL-3.0 text fetched from gnu.org (35,149 bytes);
  **LICENSES.md** = the dual-licence note (code GPL-3.0, data ODbL-1.0 + © OpenStreetMap
  contributors, §4.4 share-alike on derived indexes); **README** licence section corrected (no more
  "all rights reserved"), search/ documented in the layout, package table, and commands.
- G1–G11 + L1–L3 green. L1 proves bare-Node loading + `search("KLCC")` → suria-klcc + honest
  negatives; L2 proves the static-import law (fetch/readFileSync banned) with the web-bundle marker
  half skipped VISIBLY until plan 04-04 lands; L3 proves esbuild INLINES the dataset (the bundle
  carries the `suria-klcc` marker) and the wrangler dry-run exits 0.

## Commits

| task | commit | message |
| ---- | ------ | ------- |
| 1 RED | `ebcd542` | test(04-01): RED — search workspace scaffold + normalizer vector suite (T1-T9) |
| 1 GREEN | `a9f9ab3` | feat(04-01): search contracts + JUPEM abbreviation table as DATA + symmetric normalizer |
| 2 RED | `25ee1e9` | test(04-01): RED — OSA distance vectors (D1-D9) + disjoint-tier scorer suite (S1-S7) |
| 2 GREEN | `6368bad` | feat(04-01): OSA Damerau distance + disjoint-tier scorer (the residual net) |
| 3 RED | `f1b31ca` | test(04-01): RED — gazetteer schema/license gates (G1-G11) + three-environment loader proof (L1-L3) |
| 3 GREEN | `8579da9` | feat(04-01): curated KL gazetteer + lazy three-environment loader + GPL/ODbL licensing |

## Verification Results

- `npm run search:test` — **39 passed, 0 failed** (T1–T9, D1–D9, S1–S7, G1–G11, L1–L3, all named)
- `npm run search:typecheck` — green (`tsc --noEmit`, strict + noUncheckedIndexedAccess)
- esbuild bundle of the search module contains the `suria-klcc` marker (build-time inline proof);
  `npx wrangler deploy --dry-run` in bot/ exits 0 (build guard; its marker grep is 04-03's gate)
- `npm test -w bot` (50) and `npm test -w web` (33) — unaffected
- Acceptance probes: `search("KLCC")[0]` = suria-klcc @ 0.90 via raw alias "KLCC";
  `search("Jln Ampang")` and `search("KCLC")` include suria-klcc; `search("Kuching")` and
  `search("Penang")` = `[]`
- `search/` is buildless — no dist output, working tree clean

## Deviations from Plan

**1. [Rule 3 - Blocking] Added "Kpg" to the Kampung row beyond the 48 vendored rows**
- **Found during:** task 1 (GREEN)
- **Issue:** the plan's own T1 vector requires `"Kpg Ampang" → "kampung ampang"`, but "Kpg" is not
  among the 48 minimum rows the contract lists.
- **Fix:** extended the Kampung row to `Kg,Kpg` per MS 2039:2006 Annex A3 (same source family as
  the transcription), documented in the table header. T9's ≥ 47 row count still holds (49 rows).
- **Files:** search/src/rules/abbreviations.ts

**2. [Rule 1 - Bug] S7 assertion refined during GREEN (task 2)**
- **Found during:** task 2
- **Issue:** the RED suite asserted `scoreCandidate("kuching", [...]) === null`, but the plan's
  contract makes admission the SOURCE's job — scoreCandidate legitimately returns sub-admission
  fuzzy scores ("Kuching" vs suria-klcc ≈ 0.17).
- **Fix:** S7 now asserts the sub-admission similarity (`score / TIER.FUZZY < FUZZY_ADMIT_SIMILARITY`)
  instead of null; the honest-refusal `[]` behavior is asserted at the source level (L1).
- **Files:** search/test/score.test.mjs

**3. [Planner-note] S5's literal `=== 0` was unsatisfiable as written**
- **Issue:** the plan's S5 text says `scoreOne("taman bangsar", "taman bukit") === 0`, but the
  plan's own S6 semantics (and its `node -e` acceptance check, which only forbids TOKEN_SET) give
  every ≥3-char pair a nonzero fuzzy residual (~0.39 here).
- **Fix:** asserted `!== TIER.TOKEN_SET && < TIER.FUZZY` — matching the acceptance criterion and
  the parenthetical intent ("must NOT be TOKEN_SET").

**4. [Planner-note] TOKEN_SET evaluates before PREFIX**
- **Issue:** the scorer_contract lists PREFIX before TOKEN_SET, but its own S5 vector pins
  `scoreOne("taman bangsar", "taman bangsar kib") === TIER.TOKEN_SET` — which is also a string
  prefix. S4 (mid-word PREFIX) still holds under TOKEN_SET-first ordering.
- **Fix:** token-set subset is checked before the string-prefix test.

**5. [Recovery] Three server restarts mid-run**
- Lost tool results caused duplicate/variant file writes. Recovered from the filesystem each time:
  re-verified content, re-ran gates, amended the task-2 GREEN commit (`6368bad`) to hold the final
  variant of distance.ts/score.ts/distance.test.mjs. All commits verified green post-recovery.

## Auth Gates

None — the module reads no environment, no secrets, no network (D-04/D-11 trivially satisfied).

## Known Stubs

None — every export is wired to real data/behavior; no TODO/FIXME/placeholder anywhere in search/.

## Threat Flags

None — the built surface is exactly the plan's threat model (T-04-01…T-04-11): no new endpoints,
no auth paths, no storage, no env reads. T-04-01/02 (no logging/storage) are gated by G10;
T-04-04 (lazy loader) by G11; T-04-07 (honest refusal) by L1 + the SRCH probes.

## TDD Gate Compliance

PASSED — each task has a `test(...)` RED commit followed by a `feat(...)` GREEN commit:
`ebcd542→a9f9ab3`, `25ee1e9→6368bad`, `f1b31ca→8579da9`. RED phases were verified failing
(9, 16, and 13 failing cases respectively) before each GREEN.

## Self-Check: PASSED

- All 20 created files exist on disk (verified via `find search/ -type f` = 16 source/test files
  + LICENSE + LICENSES.md + the 2 root manifests).
- All 6 commit hashes present in `git log --grep="04-01"` (verified above).
- Suite green at 39/39, typecheck green, bot/web suites green.