forked from c-fu/arah2
ambek .planning
This commit is contained in:
araharah-executor
@@ -0,0 +1,114 @@
|
||||
# Phase 4: Destination Search + KL Gazetteer - Context
|
||||
|
||||
**Gathered:** 2026-10-23
|
||||
**Status:** Ready for planning
|
||||
|
||||
<domain>
|
||||
## Phase Boundary
|
||||
|
||||
Users type `/cari <query>` in chat (bilingual aliases already reserved), the bot answers from an in-Worker curated KL gazetteer with the best result as a native venue card (sendVenue) plus inline alternates, and tapping OPEN IN ARAHARAH launches the Mini App with the destination preloaded via a `startapp` deep link. The search-quality experiment CLOSES this phase: a pinned golden corpus with thresholds decides gazetteer sufficiency vs escalation (normalizer edits → WASM geocoder backend #2 → Photon-on-VPS, in that order). The Malaysia-only WASM geocoder is a documented future backend, not this phase's deliverable. VPS provisioning defers to Phase 5 (Valhalla). License: GPL-3.0 — forkability is a design law.
|
||||
|
||||
</domain>
|
||||
|
||||
<decisions>
|
||||
## Implementation Decisions
|
||||
|
||||
### Search interaction surface
|
||||
- **D-01:** Search entry is the **`/cari` command** in chat (with `/pergi_ke` reserved for Phase 5 semantics; bilingual alias scheme per Phase 2 D-05). Free-text-to-bot stays reserved; no inline mode; no in-webview search UI this phase (a shared search module keeps that door open).
|
||||
- **D-02:** Shortlist presentation: the **best result ships immediately** as the venue card with its OPEN button; **2–4 alternates ride as inline buttons on the same message**. One tap to proceed, one tap to switch — no second message round.
|
||||
- **D-03:** Search backend is a **pluggable `SearchSource` interface implemented in-Worker**: curated KL gazetteer queried in-Worker, $0 infrastructure. **VPS/Photon provisioning defers to Phase 5** (Valhalla's road graph needs the VPS anyway). Photon remains a documented escalation path *behind the interface*.
|
||||
|
||||
### Forkability + license (user-surfaced design law)
|
||||
- **D-04:** **GPL-3.0** is the project license. Forkability laws are binding from this phase: no hardcoded secrets (env only — already true), data as plain committed files, one-command deploy, one free-tier Cloudflare account per fork.
|
||||
- **D-05:** The user's **Malaysia-only WASM geocoder** vision (tantivy/sqlite-trigram index, statically hosted, client-side webview search — a genuine privacy win) is captured as **SearchSource backend #2**, an OSS-track artifact *after* v1. The JSON↔SQLite converter script keeps its dataset format reachable from day one. **Backend #3 = Photon-on-VPS.** Neither ships in this phase.
|
||||
|
||||
### Gazetteer + Malay normalizer
|
||||
- **D-06:** Entries are **hand-curated seed + a repo-script Overpass one-shot** that proposes top-up candidates for maintainer approval into the curated file. The script is a repo tool, not a runtime dependency.
|
||||
- **D-07:** Canonical format is **typed JSON in the repo** (V8-native parse, no WASM in the Worker bundle, diff-friendly PRs). Rationale: at ~200–500 rows, SQLite/sql.js-in-Worker is ~1.5MB + cold-start overhead of pure cost.
|
||||
- **D-08:** **Aliases live natively inside each entry** (`"KLCC"` → Suria KLCC, KLCC Park; IJN, PWTC, Stesen/Station…) — locals' shorthand is the main corpus quality lever.
|
||||
- **D-09:** The Malay normalizer is a **rule table + folding + fuzzy-residual** module: bidirectional abbreviation table (Jln↔Jalan, Tmn↔Taman, Kg↔Kampung, vowel-drop-aware Presint/Prsn forms), case/whitespace folding, glued-word split — applied to BOTH the query and entry aliases at index time. Fuzzy scoring (Damerau) is only the residual net for genuine typos on longer words. Per-rule test vectors; extends by table edit, not code change.
|
||||
|
||||
### Venue card + deep-link hand-off
|
||||
- **D-10:** The venue card is **Bot API `sendVenue`** (native pin/name/address semantics SRCH-03 names) with inline buttons: OPEN IN ARAHARAH (`web_app` button carrying `startapp`) + alternates.
|
||||
- **D-11:** The deep link carries a **plain gazetteer entry id** (`?startapp=dest_<id>`). The webview maps id → destination from its own copy of the same gazetteer module (shared module, build-time). No crypto: the payload's only threat is a user pranking themselves; stale-webview datasets re-map via the shared module version.
|
||||
- **D-12:** ONBD-03 scope: **minimal PWA hooks** — `addToHomeScreen` prompt handling + a startapp-less boot landing in the search-ready map (existing Phase 3 staged boot covers it). Full PWA manifest/offline story is documented-deferred.
|
||||
|
||||
### Acceptance corpus + thresholds (the bake-off close)
|
||||
- **D-13:** A **pinned golden corpus** in the repo (~60–100 queries): golden landmarks (KLCC, Sunway Pyramid, DUKE), abbreviation forms, typo set (transpositions, missing keys), glued words, aliases, kampung-style names, plus **deliberate negatives** (out-of-KL: "Kuching", "Penang") that must get an honest "not in v1 area" answer, never a fabricated pin. Each positive pins its expected entry id. Reviewers extend by editing one file.
|
||||
- **D-14:** The bake-off closes on: **top-3 contains the expected id for ≥90% of positives AND negatives are refused honestly**. Below threshold, escalation order is fixed: (1) alias/normalizer table edits → re-run; (2) SearchSource backend #2 (WASM geocoder); (3) Photon-on-VPS. The phase records which lever got pulled and re-runs the corpus — the experiment ends with a documented decision, not an open question.
|
||||
|
||||
### Privacy (carried + made specific)
|
||||
- **D-15:** Search queries are user payload: **category-only logging** (Phase 2 law) — no query text persisted. KV caching of search results, if any, is TTL-only with short TTL and keyed by normalized query (content is public place data, never user identity). Venue choices are not stored.
|
||||
|
||||
### OpenCode's Discretion
|
||||
- Scoring function shape (weighted trigram/Jaro vs Damerau) within the rule-table architecture — corpus decides empirically
|
||||
- Alternates ranking + tie-breaking details; "not in v1 area" copy (en/ms, clinical tone)
|
||||
- Overpass query shape for the top-up script; gazetteer entry schema fields beyond (id, name, aliases, lat, lng, category)
|
||||
- D1/KV TTL result-cache mechanics or none (perf is likely fine without at this scale)
|
||||
|
||||
</decisions>
|
||||
|
||||
<specifics>
|
||||
## Specific Ideas
|
||||
|
||||
- The venue card message should read like the bot's clinical voice: one line confirming the destination, instruments-grade terseness.
|
||||
- The search module is shared by design: bot handler AND webview import the same gazetteer + normalizer (web copy of the module is build-time; id mapping stays in sync by construction).
|
||||
- Corpus file doubles as documentation of what "locals' shorthand" means — keep it curated, not generated.
|
||||
|
||||
</specifics>
|
||||
|
||||
<canonical_refs>
|
||||
## Canonical References
|
||||
|
||||
**Downstream agents MUST read these before planning or implementing.**
|
||||
|
||||
### Phase scope + decisions
|
||||
- `.planning/ROADMAP.md` §Phase 4 — goal, 5 success criteria, bake-off-closes mandate
|
||||
- `.planning/REQUIREMENTS.md` §SRCH-01/02/03, ONBD-02/03
|
||||
- `.planning/phases/03-mini-app-shell-geolocation/03-CONTEXT.md` — boot choreography (startapp lands in the staged flow), session contract, strings/i18n precedent
|
||||
- `.planning/phases/02-bot-skeleton-initdata-security/02-CONTEXT.md` — command registration, aliases, category-only logging, KV TTL-only law
|
||||
|
||||
### Codebase
|
||||
- `bot/src/bot/bot.ts`, `bot/src/bot/handlers/` — handler patterns, command wiring
|
||||
- `bot/src/i18n/strings.ts` — bilingual strings module (search copy follows)
|
||||
- `web/src/session/`, `web/src/main.ts` — boot flow the deep link lands in; `web/src/mock/` harness (destination-preload profiles for captures)
|
||||
- `bot/wrangler.jsonc`, `bot/src/config.ts` — the Worker env the SearchSource lives in
|
||||
- `AGENTS.md` — repo conventions; GPL-3.0 LICENSE file to be added this phase
|
||||
|
||||
</canonical_refs>
|
||||
|
||||
<code_context>
|
||||
## Existing Code Insights
|
||||
|
||||
### Reusable Assets
|
||||
- `web/src/i18n/strings.ts` pattern → `bot/src/i18n/strings.ts` (search copy en/ms)
|
||||
- Hand-rolled test runners (bot/web/theme) — corpus runner is a new suite in the same shape
|
||||
- Mock-Telegram harness (web) — add a `?dest=<id>` boot profile for ONBD-02 capture evidence
|
||||
- `bot/src/webhook.ts` route table — `/cari` handler slots beside location-fallback
|
||||
|
||||
### Established Patterns
|
||||
-npm workspaces + delegate scripts; TDD RED→GREEN; category-only logs; TTL-only KV; strings-module copy gates
|
||||
|
||||
### Integration Points
|
||||
- `startapp` param arrives inside `Telegram.WebApp.initData` (`start_param` field) — session flow untouched; webview reads it post-validation
|
||||
- `sendVenue` needs no new scopes; web_app inline button URL = deployed origin (already armed in Phase 3)
|
||||
- The SearchSource interface is the seam Phases 5 (origin/destination), 6, and backend #2/#3 plug into
|
||||
|
||||
</code_context>
|
||||
|
||||
<deferred>
|
||||
## Deferred Ideas
|
||||
|
||||
- Malaysia-only WASM geocoder (tantivy/sqlite-trigram), statically hosted, client-side webview search — SearchSource backend #2, OSS track after v1
|
||||
- Photon-on-VPS — SearchSource backend #3; VPS provisioning happens in Phase 5 for Valhalla regardless
|
||||
- In-webview search UI — needs a UI-SPEC round; shared module already keeps it cheap
|
||||
- Full PWA (manifest, offline shell) — documented-deferred beyond minimal ONBD-03 hooks
|
||||
- Free-text bot search + inline mode — reserved interaction surfaces
|
||||
- Log-derived corpus (real queries) — becomes possible only post-launch (Phase 7+)
|
||||
|
||||
</deferred>
|
||||
|
||||
---
|
||||
|
||||
*Phase: 04-destination-search-kl-gazetteer*
|
||||
*Context gathered: 2026-10-23*
|
||||
@@ -0,0 +1,58 @@
|
||||
# Phase 4: Destination Search + KL Gazetteer - Discussion Log
|
||||
|
||||
> **Audit trail only.** Do not use as input to planning, research, or execution agents.
|
||||
> Decisions captured in CONTEXT.md — this log preserves the analysis.
|
||||
|
||||
**Date:** 2026-10-23
|
||||
**Phase:** 04-destination-search-kl-gazetteer
|
||||
**Mode:** default (interactive)
|
||||
|
||||
## Areas Discussed
|
||||
|
||||
### Search interaction surface
|
||||
| Question | Options presented | Selection |
|
||||
|---|---|---|
|
||||
| Where does the user type? | /cari command / free-text / in-webview / command+shared module | /cari command |
|
||||
| Shortlist presentation | best+inline alternates / list-then-choose / single result | Best + inline alternates |
|
||||
| Backend architecture | gazetteer-first pluggable / KL-scale WASM now / full pivot | Gazetteer-first pluggable |
|
||||
|
||||
### Photon/VPS detour (user education round)
|
||||
User asked what Photon is, why static hosts (Cloudflare Pages/GitHub Pages) can't run it, and proposed SQLite/sql.js and Rust-WASM (tantivy) alternatives — including a full "Malaysia-only Photon-lite" sketch (Geofabrik pbf → SQLite/FTS5 → sql.js in browser on GitHub Pages, forkable). Explained: static hosts execute nothing; the Worker is mandatory for the webhook regardless; full-MY index exceeds Worker memory caps; client-side engine means huge mobile downloads; chat-side /cari needs server-side answers. The forkable endgame is preserved as SearchSource backend #2. **Notable: user's pushback killed the default VPS-now plan** — search is $0 in-Worker; VPS defers to Phase 5 (Valhalla).
|
||||
|
||||
### License (user-surfaced area)
|
||||
| Question | Options presented | Selection |
|
||||
|---|---|---|
|
||||
| License for forkability | MIT / Apache-2.0 / GPL-3.0 / later | **GPL-3.0** |
|
||||
|
||||
### Gazetteer + Malay normalizer
|
||||
| Question | Options presented | Selection |
|
||||
|---|---|---|
|
||||
| Sourcing | hand-curated / OSM pipeline / curated + Overpass top-up | Curated + Overpass top-up |
|
||||
| Canonical format | JSON in repo / SQLite now (user asked "how heavy?": ~1.5MB WASM + cold-start of pure cost at this scale) | JSON in repo |
|
||||
| Normalizer | rules+folding+fuzzy residual / fuzzy only / research decides | Rules + folding + fuzzy residual (after a worked-example explanation of why short-word fuzzy fails: jln→jalan is high-distance, tmn≈Tun collisions, vowel-drop conventions, glued words, Stesen/Station synonym class) |
|
||||
| Aliases | in entries / separate table | Aliases in entries |
|
||||
|
||||
### Venue card + deep-link hand-off
|
||||
| Question | Options presented | Selection |
|
||||
|---|---|---|
|
||||
| startapp payload | plain id / full payload / signed | Plain gazetteer id |
|
||||
| Venue card | sendVenue + buttons / styled message | sendVenue + buttons |
|
||||
| ONBD-03 scope | minimal PWA hooks / full PWA | Minimal PWA hooks |
|
||||
|
||||
### Acceptance corpus + thresholds
|
||||
| Question | Options presented | Selection |
|
||||
|---|---|---|
|
||||
| Corpus shape | pinned golden set / log-derived | Pinned golden set (~60–100, incl. negatives) |
|
||||
| Bake-off close | 90% top-3 + honest negatives / 80% top-1 / judgment | 90% top-3 + honest negatives; escalation order fixed (table edits → backend #2 → Photon) |
|
||||
|
||||
## Corrections Made
|
||||
|
||||
None reversed; two user questions reshaped defaults materially (VPS deferral; license capture). Third-round education (fuzzy-vs-rules) ended in the recommended option being chosen with informed consent.
|
||||
|
||||
## Auto-Resolved
|
||||
|
||||
Not applicable.
|
||||
|
||||
## External Research
|
||||
|
||||
Deferred to planning's research step per roadmap (gazetteer sourcing, Overpass shapes, corpus composition details).
|
||||
Reference in New Issue
Block a user