docs(04): plan revision 1 — fix wave-1 bundle proof + acceptance-criteria corrections
This commit is contained in:
araharah-executor
@@ -43,7 +43,7 @@ must_haves:
|
||||
- "'Jln Ampang' and 'Jalan Ampang' normalize to the IDENTICAL string; 'Tmn' expands to 'taman'; 'Kg' to 'kampung'; 'Prsn' to 'presint'"
|
||||
- "A transposition typo ('KCLC') still resolves to the expected entry id, via the Damerau-Levenshtein residual tier"
|
||||
- "An exact normalized match ALWAYS outranks any fuzzy match — the tier score bands are disjoint and asserted by a test"
|
||||
- "The gazetteer JSON loads identically in three environments: the wrangler bundle, the Vite bundle, and bare Node 24 (the corpus runner)"
|
||||
- "The gazetteer JSON loads identically in three environments: the Worker bundle (esbuild/wrangler — inline proven on the search module's own esbuild bundle, dry-run as the build guard), the Vite bundle, and bare Node 24 (the corpus runner)"
|
||||
- "The Worker never builds an index at module scope — the index is built lazily on first search and memoized"
|
||||
- "A search over the full gazetteer costs well under the Workers Free 10 ms CPU/request budget"
|
||||
- "The repo carries a GPL-3.0 LICENSE and the gazetteer carries an ODbL-1.0 + OpenStreetMap attribution notice"
|
||||
@@ -54,7 +54,7 @@ must_haves:
|
||||
contains: "export interface SearchSource"
|
||||
- path: "search/src/rules/abbreviations.ts"
|
||||
provides: "the vendored JUPEM-derived bidirectional abbreviation table as DATA (D-09)"
|
||||
contains: "ABBREVIATIONS"
|
||||
contains: "ABBREVIATION_ROWS"
|
||||
- path: "search/src/normalize.ts"
|
||||
provides: "pure normalize(): NFKD-fold → whitespace/punctuation collapse → whole-token abbreviation expansion → glued-word split (D-09)"
|
||||
- path: "search/src/distance.ts"
|
||||
@@ -82,12 +82,12 @@ must_haves:
|
||||
- path: "search/test/gazetteer.test.mjs"
|
||||
provides: "entry schema, unique slug-safe ids, alias presence, ODbL header, LICENSE presence (D-07, D-08)"
|
||||
- path: "search/test/loader-env.test.mjs"
|
||||
provides: "the three-environment loader proof: bare Node import, wrangler dry-run bundle, Vite bundle (the phase's top mechanical risk)"
|
||||
provides: "the three-environment loader proof: bare Node import, the esbuild inline-bundle marker + wrangler dry-run build guard, the Vite static-import assertion (the phase's top mechanical risk)"
|
||||
key_links:
|
||||
- from: "search/src/normalize.ts"
|
||||
to: "search/src/rules/abbreviations.ts"
|
||||
via: "the whole-token expansion reads the imported table; a table edit extends behavior without a code change (D-09)"
|
||||
pattern: "ABBREVIATIONS"
|
||||
pattern: "ABBREVIATION_ROWS"
|
||||
- from: "search/src/gazetteer.ts"
|
||||
to: "search/src/data/kl-gazetteer.json"
|
||||
via: "a static JSON import (build-time inlined), never a runtime fetch — Workers have no fs"
|
||||
@@ -429,7 +429,7 @@ The index maps each entry's NORMALIZED name AND each NORMALIZED alias → the en
|
||||
```typescript
|
||||
export type { GazetteerEntry, SearchQuery, SearchHit, SearchSource } from "./types.ts";
|
||||
export { normalize } from "./normalize.ts";
|
||||
export { TIER } from "./score.ts";
|
||||
export { TIER, FUZZY_ADMIT_SIMILARITY } from "./score.ts";
|
||||
export { damerau } from "./distance.ts";
|
||||
export { getEntries, getEntryById, getIndex } from "./gazetteer.ts";
|
||||
export { createGazetteerSource, search } from "./source.ts";
|
||||
@@ -449,7 +449,7 @@ Ranking: sort by tier band DESC, then by score DESC, then by entry id ASC (a det
|
||||
|
||||
<task type="auto" tdd="true">
|
||||
<name>task 1: shared workspace scaffold + types + the JUPEM abbreviation table as DATA</name>
|
||||
<files>package.json, search/package.json, search/tsconfig.json, search/src/types.ts, search/src/rules/abbreviations.ts, search/test/normalizer.test.mjs</files>
|
||||
<files>package.json, search/package.json, search/tsconfig.json, search/src/types.ts, search/src/rules/abbreviations.ts, search/src/normalize.ts, search/test/normalizer.test.mjs</files>
|
||||
<read_first>
|
||||
- `package.json` (root — the `workspaces` array is `["theme","bot","web"]`; add `"search"`; keep every existing script)
|
||||
- `theme/package.json` (the manifest shape to copy — `name`, `type: "module"`, no `exports`/`main`, so consumers import sub-paths directly)
|
||||
@@ -586,7 +586,7 @@ Cases (T-numbered, one per law):
|
||||
- `T6 vowel-drop` — `normalize("Prsn 15") === "presint 15"` and `normalize("Presint 15") === "presint 15"`.
|
||||
- `T7 idempotence` — for every vector `v`: `normalize(normalize(v)) === normalize(v)`.
|
||||
- `T8 case/whitespace folding` — `normalize(" SUNWAY PYRAMID ") === "sunway pyramid"`.
|
||||
- `T9 the table is data` — read `search/src/rules/abbreviations.ts` as TEXT (comments stripped) and assert it contains no `function` keyword (the module is data + one builder function, not algorithm code) and that the row count is >= 47.
|
||||
- `T9 the table is data` — read `search/src/rules/abbreviations.ts` as TEXT (comments stripped) and assert that the ONLY `function` in the file is `buildAbbreviationMap` — no other function, and no control-flow keywords (`if`, `for`, `=>`) outside it — so the module is data plus one generic builder, never algorithm code; and assert the row count is >= 47.
|
||||
|
||||
**`search/src/normalize.ts`** — implement EXACTLY per the `<normalizer_contract>` pipeline. The exported signature is `export function normalize(input: string): string`. Import the table via `import { buildAbbreviationMap } from "./rules/abbreviations.ts";` and build the map ONCE at module scope (the map is ~60 entries of plain data — cheap; the D-09 "lazy" law applies to the 500-entry gazetteer INDEX, not to this table).
|
||||
|
||||
@@ -645,7 +645,8 @@ Write `search/src/normalize.ts` NOW as part of this task so the suite in this ta
|
||||
```
|
||||
|
||||
**`search/src/score.ts`** — the disjoint-tier scorer, per the `<scorer_contract>` block. Exports:
|
||||
- `export const TIER = { EXACT: 1.00, ALIAS_EXACT: 0.90, PREFIX: 0.75, TOKEN_SET: 0.65, FUZZY: 0.00 } as const;`
|
||||
- `export const TIER = { EXACT: 1.00, ALIAS_EXACT: 0.90, PREFIX: 0.75, TOKEN_SET: 0.65, FUZZY: 0.60 } as const;` — `FUZZY` is the band's CEILING (0.60), NOT a zero multiplier: a fuzzy score is `fuzzySim * TIER.FUZZY`, so the whole tier lands strictly inside `(0, 0.60)` (a similarity of 1.0 would have been caught by the EXACT tier first). Multiplying the similarity by `0.00` would zero the entire residual tier — see the `<scorer_contract>` block's NOTE.
|
||||
- `export const FUZZY_ADMIT_SIMILARITY = 0.72;` — the fuzzy admission floor, defined HERE (the `<scorer_contract>` block places it in score.ts) and imported by `source.ts` (task 3) and re-exported through `index.ts` (the `<index_contract>` block) so the corpus runner (plan 04-02's CO8) reads the floor from the module instead of hardcoding it.
|
||||
- `export function scoreOne(queryNorm: string, aliasNorm: string): number`
|
||||
- `export function scoreCandidate(queryNorm: string, aliasNorms: readonly string[]): { score: number; matchedAlias: string } | null` — returns the BEST tier hit and WHICH alias matched (the corpus report needs that provenance), or `null` when nothing scores.
|
||||
|
||||
@@ -660,7 +661,7 @@ Implementation notes:
|
||||
**`search/test/distance.test.mjs`** — RED-first. One `test(...)` per vector from the `<distance_contract>` block, with the hand-computed expected value in the message. Load the module lazily (task 1's pattern).
|
||||
|
||||
**`search/test/score.test.mjs`** — RED-first. Cases:
|
||||
- `S1 tier disjointness` — for each pair of adjacent tiers, `TIER[a] > TIER[b]`; and `TIER.FUZZY === 0` with every FUZZY score `<= 0.60` and `> 0` for a near miss.
|
||||
- `S1 tier disjointness` — for each pair of adjacent tiers, `TIER[a] > TIER[b]` (including `TIER.TOKEN_SET` 0.65 > `TIER.FUZZY` 0.60); and `TIER.FUZZY === 0.60` (the band's CEILING) with every admitted FUZZY score strictly inside `(0, 0.60)` — `> 0` for a near miss, `< 0.60` because a similarity of 1.0 is caught by the EXACT tier first.
|
||||
- `S2 min-length guard` — `scoreOne("ab", "abcd") === 0` and `scoreOne("kl", "kl") === 0` (both shorter than 3 → 0, EVEN for an exact match — the guard fires first).
|
||||
- `S3 exact outranks fuzzy` — `scoreOne("klcc", "klcc") > scoreOne("klcc", "klca")` and `scoreOne("klcc", "klcc") > TIER.FUZZY` (an exact match always beats any fuzzy hit).
|
||||
- `S4 prefix tier` — `scoreOne("jalan amp", "jalan ampang") === TIER.PREFIX`.
|
||||
@@ -676,7 +677,7 @@ Implementation notes:
|
||||
<acceptance_criteria>
|
||||
- `npm run search:test` exits 0 and the output contains `PASS S1` through `PASS S7`
|
||||
- `npm run search:typecheck` exits 0
|
||||
- `grep -q 'export const TIER' search/src/score.ts` exits 0 AND `grep -q 'EXACT: *1.00' search/src/score.ts` exits 0 AND `grep -q 'ALIAS_EXACT: *0.90' search/src/score.ts` exits 0 AND `grep -q 'PREFIX: *0.75' search/src/score.ts` exits 0 AND `grep -q 'TOKEN_SET: *0.65' search/src/score.ts` exits 0 AND `grep -q 'FUZZY: *0.00' search/src/score.ts` exits 0 (the disjoint bands are exact literals)
|
||||
- `grep -q 'export const TIER' search/src/score.ts` exits 0 AND `grep -q 'EXACT: *1.00' search/src/score.ts` exits 0 AND `grep -q 'ALIAS_EXACT: *0.90' search/src/score.ts` exits 0 AND `grep -q 'PREFIX: *0.75' search/src/score.ts` exits 0 AND `grep -q 'TOKEN_SET: *0.65' search/src/score.ts` exits 0 AND `grep -q 'FUZZY: *0.60' search/src/score.ts` exits 0 (the disjoint bands are exact literals — FUZZY is the 0.60 ceiling, never a 0.00 multiplier) AND `grep -q 'FUZZY_ADMIT_SIMILARITY = 0.72' search/src/score.ts` exits 0 (the admission floor is an exported constant, not an inlined literal)
|
||||
- `grep -q 'export function damerau' search/src/distance.ts` exits 0
|
||||
- `node -e "import('./search/src/distance.ts').then(m=>{const d=m.damerau; const cases=[['ab','ba',1],['a cat','an act',2],['','',0],['','abc',3],['abc','',3],['same','same',0],['klcc','kclc',1],['sunway','sunwya',1],['kitten','sitting',3]]; for(const [a,b,e] of cases){const g=d(a,b); if(g!==e) throw new Error(a+'/'+b+' => '+g+' expected '+e);} console.log('distance OK')})"` exits 0
|
||||
- `node -e "import('./search/src/score.ts').then(m=>{if(m.scoreOne('ab','abcd')!==0) throw new Error('min-length guard'); if(!(m.scoreOne('klcc','klcc')>m.scoreOne('klcc','klca'))) throw new Error('exact must outrank fuzzy'); if(m.scoreOne('taman bangsar','taman bukit')===m.TIER.TOKEN_SET) throw new Error('token-set false positive'); console.log('score OK')})"` exits 0
|
||||
@@ -715,7 +716,7 @@ If `npm run search:typecheck` or the loader-env suite rejects the import attribu
|
||||
|
||||
The index build: for each entry, normalize the entry's `name` and each of its `aliases`, and map each normalized string → the entry. The map is built ONCE, lazily, memoized in module scope (`let index: Map<string, GazetteerEntry[]> | null = null;`).
|
||||
|
||||
**`search/src/source.ts`** — the D-03 `SearchSource` implementation. Exports `createGazetteerSource(): SearchSource` and a synchronous `search(text: string, limit?: number): SearchHit[]` convenience wrapper. Behaviour:
|
||||
**`search/src/source.ts`** — the D-03 `SearchSource` implementation. Exports `createGazetteerSource(): SearchSource` and a synchronous `search(text: string, limit?: number): SearchHit[]` convenience wrapper. The module's HEADER COMMENT states the tiered admission rule with the numbers, verbatim-in-spirit from the `<index_contract>` block: non-fuzzy tiers always admitted; a FUZZY hit only at similarity >= `FUZZY_ADMIT_SIMILARITY` (0.72) — "KCLC" vs "KLCC" is sim 0.75 → admitted (score 0.45), "Penang" vs "Padang" is sim 0.667 → refused (an empty array, never a fabricated pin). Behaviour:
|
||||
- Normalize the query text.
|
||||
- For every candidate (each entry's normalized name + each alias), score with `scoreCandidate`.
|
||||
- Keep only ADMITTED hits: EXACT / ALIAS_EXACT / PREFIX / TOKEN_SET always; a FUZZY hit only when its similarity >= `FUZZY_ADMIT_SIMILARITY` (0.72). Below admission, the query is "not in the v1 area" and the result is an EMPTY array, never a nearest-match pin (D-13/D-14).
|
||||
@@ -741,7 +742,7 @@ The index build: for each entry, normalize the entry's `name` and each of its `a
|
||||
**`search/test/loader-env.test.mjs`** — THE PHASE'S TOP MECHANICAL RISK, proven mechanically. Three environments, three assertions:
|
||||
- `L1 bare Node (type-stripping)` — `await import("../src/index.ts")` works and `getEntries().length >= 30`; the runner file itself IS this environment (Node 24 strips types by default).
|
||||
- `L2 the Vite bundle inlines the gazetteer` — the web bundle is built by `npm run build -w theme && npm run build -w web` and the emitted `web/dist/assets/*.js` contains a gazetteer id marker (e.g. `"suria-klcc"`) — proving the JSON was inlined at BUILD time, not fetched at runtime. (The web workspace imports `@araharah/search` starting in plan 04-04; until then this case asserts the SEARCH workspace's own module can be bundled — assert it against the search workspace's own esbuild-free check: `node -e "import('./search/src/index.ts')"` PLUS a static-text assertion that `search/src/gazetteer.ts` contains a STATIC import of the JSON, not a `fetch(` or `readFileSync`. If plan 04-04 has landed by the time this suite runs, ALSO assert the web bundle marker; otherwise skip that half with an explicit `test.skip`-style guard that still asserts the static-import half. Make the skip visible in the runner output.)
|
||||
- `L3 the Worker bundle inlines the gazetteer` — `cd bot && npx wrangler deploy --dry-run` (wrangler 4.151.3 supports `--dry-run`; `wrangler dev` does NOT) must succeed, and the dry-run output must show the gazetteer id marker in the bundled Worker output. Capture the dry-run output to a temp file and grep it for `suria-klcc`. `wrangler deploy --dry-run` prints the bundled worker size and (with `--outdir`) writes the bundle — use `--outdir` to a temp directory and grep the emitted JS for the marker, which is the deterministic proof.
|
||||
- `L3 the esbuild/Worker bundler inlines the gazetteer` — TWO halves. **The inline half (provable at wave 1):** bundle the search module directly with esbuild — the same bundler family wrangler uses (esbuild 0.28.2 is hoisted at `node_modules/.bin/esbuild`): `node_modules/.bin/esbuild search/src/index.ts --bundle --outfile=/tmp/opencode/araharah-search-bundle/index.js`, then grep the emitted JS for the `suria-klcc` marker — a static JSON import MUST be inlined at build time, never fetched at runtime. **The Worker dry-run half (build regression guard ONLY):** `cd bot && npx wrangler deploy --dry-run --outdir=<tmp>` (wrangler 4.151.3 supports `--dry-run`; `wrangler dev` does NOT) must EXIT 0. Do NOT grep the dry-run bundle for the marker in this plan: at wave 1 no bot source imports `@araharah/search` (`bot.ts` gains `createGazetteerSource()` only in plan 04-03 task 2), so the Worker bundle cannot contain the marker yet and a marker grep here would red-light the whole suite. The Worker-bundle marker grep is plan 04-03 task 2's acceptance criterion. If plan 04-03 has already landed when this suite runs, ALSO assert the Worker-bundle marker — guarded by the same explicit skip-guard style L2 uses for the web bundle, with the skip made visible in the runner output.
|
||||
|
||||
**`LICENSE`** — the VERBATIM GPL-3.0 text from https://www.gnu.org/licenses/gpl-3.0.txt. Do NOT paraphrase, do NOT summarize, do NOT truncate the "how to apply" section. The file must contain the strings "GNU GENERAL PUBLIC LICENSE", "Version 3, 29 June 2007", and "TERMS AND CONDITIONS".
|
||||
|
||||
@@ -782,16 +783,16 @@ to include the `search/` workspace line:
|
||||
search/ @araharah/search — src/types.ts, src/normalize.ts, src/score.ts, src/gazetteer.ts, src/data/kl-gazetteer.json
|
||||
```
|
||||
|
||||
**Acceptance:** `npm run search:test` exits 0 (including `PASS G1`…`PASS G11` and `PASS L1`…`PASS L3`); `npm run search:typecheck` exits 0; `npx wrangler deploy --dry-run` in `bot/` succeeds with the gazetteer marker inlined.
|
||||
**Acceptance:** `npm run search:test` exits 0 (including `PASS G1`…`PASS G11` and `PASS L1`…`PASS L3`); `npm run search:typecheck` exits 0; the esbuild bundle of the search module contains the `suria-klcc` marker; `npx wrangler deploy --dry-run` in `bot/` exits 0 (the Worker bundle builds — its bundle-marker grep is plan 04-03 task 2's criterion, after `bot.ts` imports the source).
|
||||
</action>
|
||||
<verify>
|
||||
<automated>cd /home/C-Fu/araharah && npm run search:test 2>&1 | tail -40 ; npm run search:typecheck 2>&1 | tail -5 ; cd bot && npx wrangler deploy --dry-run --outdir /tmp/opencode/araharah-worker-dryrun 2>&1 | tail -5 ; grep -c 'suria-klcc' /tmp/opencode/araharah-worker-dryrun/index.js 2>/dev/null || grep -rc 'suria-klcc' /tmp/opencode/araharah-worker-dryrun/ 2>/dev/null | head -1</automated>
|
||||
<automated>cd /home/C-Fu/araharah && npm run search:test 2>&1 | tail -40 ; npm run search:typecheck 2>&1 | tail -5 ; node_modules/.bin/esbuild search/src/index.ts --bundle --outfile=/tmp/opencode/araharah-search-bundle/index.js 2>&1 | tail -3 ; grep -c 'suria-klcc' /tmp/opencode/araharah-search-bundle/index.js ; cd bot && npx wrangler deploy --dry-run --outdir /tmp/opencode/araharah-worker-dryrun 2>&1 | tail -3</automated>
|
||||
</verify>
|
||||
<acceptance_criteria>
|
||||
- `npm run search:test` exits 0 and the output contains `PASS G1` through `PASS G11` and `PASS L1` through `PASS L3`
|
||||
- `npm run search:typecheck` exits 0
|
||||
- `cd bot && npx wrangler deploy --dry-run --outdir /tmp/opencode/araharah-worker-dryrun` exits 0 (the Worker bundle builds — the L3 half of the three-environment proof)
|
||||
- `grep -rl 'suria-klcc' /tmp/opencode/araharah-worker-dryrun/ | wc -l` returns >= 1 (the gazetteer JSON is INLINED into the Worker bundle — a runtime fetch would be a hard failure in a Worker)
|
||||
- `cd bot && npx wrangler deploy --dry-run --outdir /tmp/opencode/araharah-worker-dryrun` exits 0 (the Worker bundle builds — a build regression guard; the bundle-marker grep is plan 04-03 task 2's criterion, where `bot.ts` actually imports the source)
|
||||
- `node_modules/.bin/esbuild search/src/index.ts --bundle --outfile=/tmp/opencode/araharah-search-bundle/index.js` exits 0 AND `grep -c 'suria-klcc' /tmp/opencode/araharah-search-bundle/index.js` returns >= 1 (esbuild — the bundler family wrangler uses — INLINES the static JSON import at build time; a runtime fetch would be a hard failure in a Worker. This is the wave-1-provable inline proof, since no bot source imports `@araharah/search` until plan 04-03 task 2)
|
||||
- `node -e "const g=require('./search/src/data/kl-gazetteer.json'); if(g.entries.length<30) throw new Error('seed too small: '+g.entries.length); if(!g._license||!g._license.includes('ODbL')) throw new Error('no ODbL header'); if(!g._attribution||!g._attribution.includes('OpenStreetMap')) throw new Error('no OSM attribution'); console.log('gazetteer OK '+g.entries.length+' entries')"` exits 0
|
||||
- `node -e "const g=require('./search/src/data/kl-gazetteer.json'); const ids=g.entries.map(e=>e.id); if(new Set(ids).size!==ids.length) throw new Error('duplicate ids'); for(const e of g.entries){ if(!/^[a-z0-9-]{1,64}$/.test(e.id)) throw new Error('bad id '+e.id); if(!e.aliases||e.aliases.length===0) throw new Error('no aliases '+e.id); if(!Number.isFinite(e.lat)||!Number.isFinite(e.lng)) throw new Error('bad coords '+e.id); if(e.lat<2.9||e.lat>3.4||e.lng<101.4||e.lng>102.0) throw new Error('out of KL bbox '+e.id);} console.log('entries OK')"` exits 0
|
||||
- `test -f LICENSE && grep -c 'GNU GENERAL PUBLIC LICENSE' LICENSE | head -1` returns >= 1 AND `grep -q 'Version 3, 29 June 2007' LICENSE` exits 0 (the VERBATIM GPL-3.0 text — D-04)
|
||||
@@ -803,7 +804,7 @@ search/ @araharah/search — src/types.ts, src/normalize.ts, src/score.ts, src
|
||||
- `node -e "import('./search/src/index.ts').then(m=>{const hits=m.search('KLCC',5); if(!hits.length) throw new Error('KLCC found nothing'); if(hits[0].entry.id!=='suria-klcc') throw new Error('KLCC => '+hits[0].entry.id); if(m.search('Kuching',3).length!==0) throw new Error('negative leaked'); if(m.search('Penang',3).length!==0) throw new Error('negative leaked'); console.log('search OK: '+hits[0].entry.id)})"` exits 0 (SRCH-01 lands; the D-13 negatives are refused honestly)
|
||||
- `node -e "import('./search/src/index.ts').then(m=>{const h=m.search('Jln Ampang',5); if(!h.some(x=>x.entry.id==='suria-klcc')) throw new Error('Jln Ampang missed KLCC'); const t=m.search('KCLC',5); if(!t.some(x=>x.entry.id==='suria-klcc')) throw new Error('typo missed'); console.log('SRCH-02 OK')})"` exits 0
|
||||
</acceptance_criteria>
|
||||
<done>The curated KL gazetteer ships as committed, ODbL-attributed JSON with locals' aliases; the loader parses it lazily and memoizes a normalized index that survives all three bundling environments with the JSON INLINED (proven by the Worker dry-run bundle containing the id marker); the repo carries the verbatim GPL-3.0 LICENSE and the ODbL data notice; and the README no longer says "all rights reserved".</done>
|
||||
<done>The curated KL gazetteer ships as committed, ODbL-attributed JSON with locals' aliases; the loader parses it lazily and memoizes a normalized index that survives all three bundling environments with the JSON INLINED (proven at wave 1 by the esbuild bundle of the search module containing the id marker; the wrangler dry-run is asserted as an exit-0 build guard, with its bundle-marker grep landing in plan 04-03 task 2 once `bot.ts` imports the source); the repo carries the verbatim GPL-3.0 LICENSE and the ODbL data notice; and the README no longer says "all rights reserved".</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
@@ -826,7 +827,7 @@ search/ @araharah/search — src/types.ts, src/normalize.ts, src/score.ts, src
|
||||
| T-04-02 | Information Disclosure | Query text reaching a storage surface | mitigate | The module imports no storage API and calls none (G10 bans `localStorage`/`sessionStorage`/`indexedDB`/`document.cookie` in `search/src/*.ts`). The gazetteer is a build-time constant, not user data. |
|
||||
| T-04-03 | Denial of Service | Hostile query text — long string, pathological repetition — forcing an unbounded scan or allocation | mitigate | The query is bounded BEFORE this module is reached (plan 04-03 caps the raw command argument at 128 chars and rejects over-cap input). Inside the module the normalized query is tokenized once and compared against a finite, pre-built alias index; the fuzzy tier runs only on survivors of the token/prefix filters and only for tokens >= 3 characters, so the per-query work is bounded by the index size, not by the query length. The scorer refuses to score any side shorter than 3 characters. |
|
||||
| T-04-04 | Denial of Service | Index construction at module scope blowing the Workers 1 s startup budget (RESEARCH Pitfall 8) | mitigate | LAW 1 in the loader: the index is built lazily inside `loadIndex()` and memoized in module scope — never at module top level (G11 asserts the top level contains no `new Map(` and no JSON traversal). The same pattern as `bot/test/initdata.test.mjs`'s lazy validator load. |
|
||||
| T-04-05 | Denial of Service | Naive full-scan Damerau over ~2500 alias pairs costing ~8 ms/query against a 10 ms Workers Free CPU limit (RESEARCH §Verification A1 — REPRODUCED at 8.243 ms) | mitigate | The scorer is TIERED: the EXACT / ALIAS_EXACT / PREFIX / TOKEN_SET tiers are string comparisons against a pre-built hash index (O(1) per candidate), and the Damerau FUZZY tier runs only on candidates that survive the cheap tiers AND have both sides >= 3 characters. The index maps each normalized alias directly to its entry, so an exact alias hit never touches the fuzzy tier at all. Plan 04-04's timing assertion measures the whole query path. |
|
||||
| T-04-05 | Denial of Service | Naive full-scan Damerau over ~2500 alias pairs costing ~8 ms/query against a 10 ms Workers Free CPU limit (RESEARCH §Verification A1 — REPRODUCED at 8.243 ms) | mitigate | The scorer is TIERED and the per-query cost is bounded by CANDIDATE COUNT, not query length: the EXACT / ALIAS_EXACT / PREFIX / TOKEN_SET tiers are comparisons against a pre-built hash index (O(1) per candidate), so a query with any cheap-tier hit never reaches the fuzzy scan. A pure-typo query (zero cheap-tier survivors) DOES pay a full fuzzy scan — that is the honest worst case, REPRODUCED at ~8.2 ms for ~2000 alias pairs — and every fuzzy comparison stays bounded: both sides >= 3 chars, OSA is O(len^2) on short strings, and the scan runs over a FIXED committed dataset. SCALING DECISION (RESEARCH directive #1, stated explicitly rather than hand-waved): the ~30-entry seed and D-07's 500-entry ceiling keep the full-scan worst case inside the 10 ms limit, so the trigram/blocking-key index is documented as the NEXT step, not built now; `docs/SEARCH-QUALITY.md`'s "Known limitations" (plan 04-02) records the trigger (top-up scale approaching the CPU limit) and the fix (a first-letter + length-band blocking key in front of the OSA tier), so the escalation is a recorded decision, not a surprise Error-1102. |
|
||||
| T-04-06 | Tampering | A malformed committed entry (duplicate id, non-finite coordinate, empty alias list) reaching a venue card as a fabricated or broken pin | mitigate | G1/G2/G5 assert the dataset's invariants as a gate: unique slug-safe ids, non-empty aliases, finite coordinates inside the KL bbox. A non-finite coordinate would surface as `NaN` on the venue card — the gate makes that impossible to ship. |
|
||||
| T-04-07 | Tampering | An out-of-area query answered with the nearest KL pin (a fabricated location) — an honesty failure (RESEARCH §Security V8, D-13/D-14) | mitigate | The tiered admission rule: EXACT/ALIAS_EXACT/PREFIX/TOKEN_SET always admitted; a FUZZY hit only at similarity >= `FUZZY_ADMIT_SIMILARITY` (0.72). "Kuching" and "Penang" must yield `[]` — asserted in this plan's acceptance criteria and re-asserted by the corpus runner in plan 04-02. An empty shortlist is a legitimate, load-bearing return value of the `SearchSource` contract, and the bot renders the honest refusal copy (plan 04-03). |
|
||||
| T-04-08 | Tampering | The rule table growing a code-shaped entry point — a rule that changes behavior via code rather than a table edit (D-09's "extends by table edit, not code change" law) | mitigate | T9 asserts the rules file contains no algorithm and the row count is >= 47; the table is a `ReadonlyArray<readonly [string, string]>` of pairs, inverted once by a single generic builder. Any new abbreviation is a row, not a branch. |
|
||||
@@ -838,7 +839,7 @@ search/ @araharah/search — src/types.ts, src/normalize.ts, src/score.ts, src
|
||||
<verification>
|
||||
- `npm run search:test` green — 9 normalizer vectors + 9 distance vectors + 7 score vectors + 11 gazetteer/license vectors + 3 loader-environment vectors, all named
|
||||
- `npm run search:typecheck` green (`tsc --noEmit`, `strict` + `noUncheckedIndexedAccess`)
|
||||
- `cd bot && npx wrangler deploy --dry-run` exits 0 AND the emitted bundle contains `suria-klcc` — the Worker bundle INLINES the dataset (never a runtime fetch)
|
||||
- the esbuild bundle of the search module (`node_modules/.bin/esbuild search/src/index.ts --bundle`) contains `suria-klcc` — esbuild (the bundler family wrangler uses) INLINES the dataset at build time, never a runtime fetch; `cd bot && npx wrangler deploy --dry-run` exits 0 as a build regression guard (its bundle-marker grep is asserted in plan 04-03 task 2, after `bot.ts` imports `createGazetteerSource()`)
|
||||
- `npm test -w bot` and `npm test -w web` still green (this plan adds no bot/web source, so the existing suites must be unaffected)
|
||||
- `node -e "import('./search/src/index.ts')..."` proves `search('KLCC')` → `suria-klcc` at rank 1 and `search('Kuching')` → `[]`
|
||||
- `git status` shows no build output staged (`search/` is buildless — nothing under `search/dist/` may exist)
|
||||
@@ -852,7 +853,7 @@ search/ @araharah/search — src/types.ts, src/normalize.ts, src/score.ts, src
|
||||
- D-03: the `SearchSource` seam exists with an async `search()` and a synchronous `byId()`, so backend #2 (WASM geocoder) and #3 (Photon) are format changes rather than rewrites
|
||||
- D-15: the shared module contains no logging and no storage surface at all
|
||||
- D-04: the repo carries the verbatim GPL-3.0 LICENSE, `LICENSES.md` records the ODbL-1.0 + OpenStreetMap notice for the gazetteer, and the README no longer claims "all rights reserved"
|
||||
- The three-environment loader risk is closed mechanically: bare Node, the Vite bundle, and the Worker dry-run bundle all resolve the dataset with the JSON INLINED at build time
|
||||
- The three-environment loader risk is closed mechanically: bare Node resolves the dataset, the esbuild/Worker bundler family provably INLINES it (the search module's own esbuild bundle carries the id marker), and the Vite assertion follows L2's explicit skip-guard; the Worker dry-run exits 0 as a build guard, and its bundle-marker grep completes in plan 04-03 task 2 once `bot.ts` imports the source
|
||||
</success_criteria>
|
||||
|
||||
<output>
|
||||
|
||||
Reference in New Issue
Block a user