arah2/.planning/phases/04-destination-search-kl-gazetteer/04-CONTEXT.md

Phase 4: Destination Search + KL Gazetteer - Context

Gathered: 2026-10-23 Status: Ready for planning

## Phase Boundary

Users type /cari <query> in chat (bilingual aliases already reserved), the bot answers from an in-Worker curated KL gazetteer with the best result as a native venue card (sendVenue) plus inline alternates, and tapping OPEN IN ARAHARAH launches the Mini App with the destination preloaded via a startapp deep link. The search-quality experiment CLOSES this phase: a pinned golden corpus with thresholds decides gazetteer sufficiency vs escalation (normalizer edits → WASM geocoder backend #2 → Photon-on-VPS, in that order). The Malaysia-only WASM geocoder is a documented future backend, not this phase's deliverable. VPS provisioning defers to Phase 5 (Valhalla). License: GPL-3.0 — forkability is a design law.

## Implementation Decisions

Search interaction surface

• D-01: Search entry is the /cari command in chat (with /pergi_ke reserved for Phase 5 semantics; bilingual alias scheme per Phase 2 D-05). Free-text-to-bot stays reserved; no inline mode; no in-webview search UI this phase (a shared search module keeps that door open).
• D-02: Shortlist presentation: the best result ships immediately as the venue card with its OPEN button; 2–4 alternates ride as inline buttons on the same message. One tap to proceed, one tap to switch — no second message round.
• D-03: Search backend is a pluggable SearchSource interface implemented in-Worker: curated KL gazetteer queried in-Worker, $0 infrastructure. VPS/Photon provisioning defers to Phase 5 (Valhalla's road graph needs the VPS anyway). Photon remains a documented escalation path behind the interface.

Forkability + license (user-surfaced design law)

• D-04: GPL-3.0 is the project license. Forkability laws are binding from this phase: no hardcoded secrets (env only — already true), data as plain committed files, one-command deploy, one free-tier Cloudflare account per fork.
• D-05: The user's Malaysia-only WASM geocoder vision (tantivy/sqlite-trigram index, statically hosted, client-side webview search — a genuine privacy win) is captured as SearchSource backend #2, an OSS-track artifact after v1. The JSON↔SQLite converter script keeps its dataset format reachable from day one. Backend #3 = Photon-on-VPS. Neither ships in this phase.

Gazetteer + Malay normalizer

• D-06: Entries are hand-curated seed + a repo-script Overpass one-shot that proposes top-up candidates for maintainer approval into the curated file. The script is a repo tool, not a runtime dependency.
• D-07: Canonical format is typed JSON in the repo (V8-native parse, no WASM in the Worker bundle, diff-friendly PRs). Rationale: at ~200–500 rows, SQLite/sql.js-in-Worker is ~1.5MB + cold-start overhead of pure cost.
• D-08: Aliases live natively inside each entry ("KLCC" → Suria KLCC, KLCC Park; IJN, PWTC, Stesen/Station…) — locals' shorthand is the main corpus quality lever.
• D-09: The Malay normalizer is a rule table + folding + fuzzy-residual module: bidirectional abbreviation table (Jln↔Jalan, Tmn↔Taman, Kg↔Kampung, vowel-drop-aware Presint/Prsn forms), case/whitespace folding, glued-word split — applied to BOTH the query and entry aliases at index time. Fuzzy scoring (Damerau) is only the residual net for genuine typos on longer words. Per-rule test vectors; extends by table edit, not code change.

Venue card + deep-link hand-off

• D-10: The venue card is Bot API sendVenue (native pin/name/address semantics SRCH-03 names) with inline buttons: OPEN IN ARAHARAH (web_app button carrying startapp) + alternates.
• D-11: The deep link carries a plain gazetteer entry id (?startapp=dest_<id>). The webview maps id → destination from its own copy of the same gazetteer module (shared module, build-time). No crypto: the payload's only threat is a user pranking themselves; stale-webview datasets re-map via the shared module version.
• D-12: ONBD-03 scope: minimal PWA hooks — addToHomeScreen prompt handling + a startapp-less boot landing in the search-ready map (existing Phase 3 staged boot covers it). Full PWA manifest/offline story is documented-deferred.

Acceptance corpus + thresholds (the bake-off close)

• D-13: A pinned golden corpus in the repo (~60–100 queries): golden landmarks (KLCC, Sunway Pyramid, DUKE), abbreviation forms, typo set (transpositions, missing keys), glued words, aliases, kampung-style names, plus deliberate negatives (out-of-KL: "Kuching", "Penang") that must get an honest "not in v1 area" answer, never a fabricated pin. Each positive pins its expected entry id. Reviewers extend by editing one file.
• D-14: The bake-off closes on: top-3 contains the expected id for ≥90% of positives AND negatives are refused honestly. Below threshold, escalation order is fixed: (1) alias/normalizer table edits → re-run; (2) SearchSource backend #2 (WASM geocoder); (3) Photon-on-VPS. The phase records which lever got pulled and re-runs the corpus — the experiment ends with a documented decision, not an open question.

Privacy (carried + made specific)

• D-15: Search queries are user payload: category-only logging (Phase 2 law) — no query text persisted. KV caching of search results, if any, is TTL-only with short TTL and keyed by normalized query (content is public place data, never user identity). Venue choices are not stored.

OpenCode's Discretion

• Scoring function shape (weighted trigram/Jaro vs Damerau) within the rule-table architecture — corpus decides empirically
• Alternates ranking + tie-breaking details; "not in v1 area" copy (en/ms, clinical tone)
• Overpass query shape for the top-up script; gazetteer entry schema fields beyond (id, name, aliases, lat, lng, category)
• D1/KV TTL result-cache mechanics or none (perf is likely fine without at this scale)

## Specific Ideas

• The venue card message should read like the bot's clinical voice: one line confirming the destination, instruments-grade terseness.
• The search module is shared by design: bot handler AND webview import the same gazetteer + normalizer (web copy of the module is build-time; id mapping stays in sync by construction).
• Corpus file doubles as documentation of what "locals' shorthand" means — keep it curated, not generated.

<canonical_refs>

Canonical References

Downstream agents MUST read these before planning or implementing.

Phase scope + decisions

• .planning/ROADMAP.md §Phase 4 — goal, 5 success criteria, bake-off-closes mandate
• .planning/REQUIREMENTS.md §SRCH-01/02/03, ONBD-02/03
• .planning/phases/03-mini-app-shell-geolocation/03-CONTEXT.md — boot choreography (startapp lands in the staged flow), session contract, strings/i18n precedent
• .planning/phases/02-bot-skeleton-initdata-security/02-CONTEXT.md — command registration, aliases, category-only logging, KV TTL-only law

Codebase

• bot/src/bot/bot.ts, bot/src/bot/handlers/ — handler patterns, command wiring
• bot/src/i18n/strings.ts — bilingual strings module (search copy follows)
• web/src/session/, web/src/main.ts — boot flow the deep link lands in; web/src/mock/ harness (destination-preload profiles for captures)
• bot/wrangler.jsonc, bot/src/config.ts — the Worker env the SearchSource lives in
• AGENTS.md — repo conventions; GPL-3.0 LICENSE file to be added this phase

</canonical_refs>

<code_context>

Existing Code Insights

Reusable Assets

• web/src/i18n/strings.ts pattern → bot/src/i18n/strings.ts (search copy en/ms)
• Hand-rolled test runners (bot/web/theme) — corpus runner is a new suite in the same shape
• Mock-Telegram harness (web) — add a ?dest=<id> boot profile for ONBD-02 capture evidence
• bot/src/webhook.ts route table — /cari handler slots beside location-fallback

Established Patterns

-npm workspaces + delegate scripts; TDD RED→GREEN; category-only logs; TTL-only KV; strings-module copy gates

Integration Points

• startapp param arrives inside Telegram.WebApp.initData (start_param field) — session flow untouched; webview reads it post-validation
• sendVenue needs no new scopes; web_app inline button URL = deployed origin (already armed in Phase 3)
• The SearchSource interface is the seam Phases 5 (origin/destination), 6, and backend #2/#3 plug into

</code_context>

## Deferred Ideas

• Malaysia-only WASM geocoder (tantivy/sqlite-trigram), statically hosted, client-side webview search — SearchSource backend #2, OSS track after v1
• Photon-on-VPS — SearchSource backend #3; VPS provisioning happens in Phase 5 for Valhalla regardless
• In-webview search UI — needs a UI-SPEC round; shared module already keeps it cheap
• Full PWA (manifest, offline shell) — documented-deferred beyond minimal ONBD-03 hooks
• Free-text bot search + inline mode — reserved interaction surfaces
• Log-derived corpus (real queries) — becomes possible only post-launch (Phase 7+)

---

Phase: 04-destination-search-kl-gazetteer Context gathered: 2026-10-23