arah2/docs/TESTING.md

<!-- generated-by: gsd-doc-writer -->

# Testing

arah² has no third-party test framework. Every suite in this repo is a hand-rolled, zero-dependency Node script that registers cases into a small array and drains it. The runners were written first in `theme/`, then copied verbatim into `bot/` and `web/` — the three share one shape, deliberately.

Beyond the suites, two additional gates stand in for CI (this repo has none — see [CI integration](#ci-integration)): `tsc --noEmit` typechecks, and the theme's off-token audit is itself a tested CLI.

---

## Contents

- [Test framework and setup](#test-framework-and-setup)
- [Running tests](#running-tests)
- [Writing new tests](#writing-new-tests)
- [Test patterns](#test-patterns)
- [Coverage requirements](#coverage-requirements)
- [Visual capture workflow](#visual-capture-workflow)
- [CI integration](#ci-integration)

---

## Test framework and setup

### What "framework" means here

| Workspace | Runner | Harness | Registered via | Command |
|---|---|---|---|---|
| `bot/` | `bot/test/runner.mjs` | `bot/test/harness.mjs` | `import { test } from "./harness.mjs"` | `npm run bot:test` |
| `web/` | `web/test/runner.mjs` | `web/test/harness.mjs` | `import { test } from "./harness.mjs"` | `npm run web:test` |
| `theme/` | inline in `theme/scripts/audit.test.mjs` | local `check()` helper | `check("name", cond, detail)` | `npm test -w theme` |

There is **no jest, vitest, mocha, playwright-test, or any other runner** in any `package.json`. The only test-relevant dev dependency anywhere is `typescript` (`tsc --noEmit`, used as a compile-time gate) and `playwright` (used by the capture scripts, not by any suite).

### The harness (10 lines, duplicated per workspace)

`bot/test/harness.mjs` and `web/test/harness.mjs` are byte-identical:

```js
const tests = [];
export function test(name, fn) {
  tests.push({ name, fn });
}
export const registry = tests;
```

`test(name, fn)` pushes a case. `fn` may be sync or `async` — the runner `await`s every one.

### The runner (47 lines, duplicated per workspace)

`bot/test/runner.mjs` and `web/test/runner.mjs` are also byte-identical in behaviour:

1. `readdirSync(import.meta.dirname)`, filter `*.test.mjs`, sort.
2. Dynamic-`import()` each suite inside a `try/catch`. A suite that throws at load time is **not** fatal — it is pushed into the registry as a single failing case named `suite load failed: <file>`. The run always completes.
3. Drain `registry` in registration order: `PASS <name>` on stdout, or `FAIL <name>` plus the error stack on stderr.
4. Print `[runner] <n> passed, <m> failed` and set `process.exitCode = 1` if anything failed.

Registration order is therefore: **suite files in alphabetical order, cases in file order**. There is no shuffling, no parallelism, no filtering, no watch mode.

### TypeScript is loaded directly

Both `bot/test/` and `web/test/` `await import("../src/....ts")` directly against the shipped source — no build step, no transpile step, no fixtures directory. This works because Node strips types natively (the code comments call this "Node 24 type stripping"). <!-- VERIFY: minimum Node.js version for native `.ts` import support — no `engines` field or `.nvmrc` is committed; the test suites themselves reference "Node 24 type stripping". -->

### Assertion style

Every suite in `bot/` and `web/` uses **`node:assert`'s strict mode**, imported as:

```js
import { strict as assert } from "node:assert";
```

In practice: `assert.ok(...)` for existence checks, `assert.equal` / `assert.strictEqual` for scalar equality, `assert.deepEqual` / `assert.deepStrictEqual` for structural equality, `assert.match` / `assert.doesNotMatch` for regex, and `assert.throws` / `assert.doesNotThrow` when a throw is the expected result. Every assertion carries a message string explaining the invariant being pinned.

`theme/scripts/audit.test.mjs` does not use `node:assert` at all — it defines a local `check(name, cond, detail = '')` that accumulates into `passes[]` / `failures[]`, reports them all at the end, and exits 1 if any failed.

---

## Running tests

### Everything, per workspace

```bash
# from the repo root
npm run bot:test        # → node test/runner.mjs in bot/
npm run web:test        # → node test/runner.mjs in web/
npm test -w theme       # → node ./scripts/audit.test.mjs in theme/
```

There is no root-level `npm test` aggregate — run all three, or add one to `package.json` when you need it.

### Running one suite

The runners auto-discover by filename and offer no filter flag. To run a single file, invoke it directly from inside the workspace:

```bash
cd bot && node test/initdata.test.mjs
```

This works because a suite file's top-level `test(...)` calls register into `harness.mjs`, and `runner.mjs` is what drains the registry — so importing a suite alone reports nothing. **A single suite file run this way executes the module side effects (imports, fixture writes) but produces no pass/fail output.** Always go through the runner:

```bash
cd bot && node test/runner.mjs        # all bot suites
cd web && node test/runner.mjs        # all web suites
```

`theme` is the exception — its suite is a standalone script and *does* report standalone:

```bash
cd theme && node scripts/audit.test.mjs
```

### Typecheck gates

The `bot/` and `web/` suites do not compile. Run these alongside:

```bash
npm run bot:typecheck    # tsc --noEmit in bot/
npm run web:typecheck    # tsc --noEmit in web/
```

### The theme audit as a standalone tool

The audit is both the thing under test and a useful CLI in its own right. Run it on the real build:

```bash
npm run audit            # → node ./scripts/audit.mjs ./dist/rams.css
```

Its exit codes are a contract: `0` clean, `1` violations (each printed as `file:line: declaration`), `2` token-sheet drift or error. See [theme/scripts/audit.mjs](theme/scripts/audit.mjs) for the full usage block and its scope adjudications.

```mermaid
graph TD
    ROOT["npm run bot:test / web:test / audit"] --> BR["bot/test/runner.mjs"]
    ROOT --> WR["web/test/runner.mjs"]
    ROOT --> TR["theme/scripts/audit.test.mjs"]

    BR --> BDISC["readdir *.test.mjs → dynamic import"]
    BDISC --> BREG["bot/test/harness.mjs registry"]
    BREG --> BCASES["commands · initdata · location-pin · privacy · sessions · webhook"]

    WR --> WDISC["readdir *.test.mjs → dynamic import"]
    WDISC --> WREG["web/test/harness.mjs registry"]
    WREG --> WCASES["build-artifact · copy-gate · location-chain · no-storage · session · theme"]

    TR --> TFIX["fixtures under /tmp/opencode/audit-test/"]
    TFIX --> TSPAWN["spawnSync scripts/audit.mjs"]
    TSPAWN --> TCHECK["check() → passes[] / failures[]"]
    TCHECK --> TREAL["also audits real dist/rams.css"]

    classDef entry fill:#EB8008,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
    classDef runner fill:#1A1A1A,stroke:#EB8008,stroke-width:2px,color:#F7F0E3
    classDef harness fill:#D9D2C6,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
    classDef theme fill:#F5EDE0,stroke:#736B1E,stroke-width:2px,color:#1E1B14

    class ROOT entry
    class BR,WR,TR runner
    class BDISC,BREG,WDISC,WREG harness
    class TFIX,TSPAWN,TCHECK,TREAL theme
```

---

## Writing new tests

### Naming and location

| Rule | Value |
|---|---|
| File name | `<subject>.test.mjs` — the runner only discovers this suffix |
| Directory | `bot/test/` or `web/test/`, flat, no subdirectories |
| Import the harness | `import { test } from "./harness.mjs";` |
| Case name | Prefix with a stable case code: `"WK1 POST valid initData (under cap) → 200 {token: uuid}"`, `"GEO3 no banned storage token…"`, `"CS4 D-02: the command tree source has no reply-menu…"`. The code is the identity; the prose is the explanation. |

A new file dropped into `bot/test/` or `web/test/` is picked up automatically on the next run — no manifest, no import list, no registration step anywhere.

### The minimum shape

```js
#!/usr/bin/env node
/*
 * <subject>.test.mjs — what this suite pins, and under which requirement id.
 * ------------------------------------------------------------------------
 *   <CODE>  one line per case, so a reader can see the contract without
 *           reading the code.
 */
import { strict as assert } from "node:assert";
import { test } from "./harness.mjs";

const { someExport } = await import("../src/some/module.ts");

test("<CODE> <what is asserted> <and why>", () => {
  assert.equal(someExport("input"), "expected", "why this matters");
});

test("<CODE> async path", async () => {
  const res = await someAsync();
  assert.ok(res.ok, `expected ok:true, got ${JSON.stringify(res)}`);
});
```

For `theme/`, add cases to `theme/scripts/audit.test.mjs` using the local helper instead:

```js
{
  const r = runAudit([dirtyFixture]);
  check('T2 dirty stylesheet → exit 1', r.status === 1, `status=${r.status} error=${r.error}`);
}
```

Note the theme suite uses a **bare block** (`{ … }`) rather than a named function, and relies on `passes[]`/`failures[]` instead of exceptions.

### Where the source of truth lives

Every suite header comment lists its cases up front. That header is the documentation — keep it in sync when adding a case. Several suites reference the decision ids (`D-02`, `PRIV-02`, `GEO-03`, `CS1`, `COPY5`, `WK3`) that the case is pinning; reuse that convention.

---

## Test patterns

The repo has converged on four recognisable patterns. Follow the closest precedent rather than inventing a new one.

### 1. Injected fakes over mocks and over the real platform

`bot/test/sessions.test.mjs` and `bot/test/webhook.test.mjs` both hand-roll a ~20-line `KVNamespace` fake rather than depend on `miniflare` or `wrangler`:

```js
function makeKV() {
  const map = new Map();
  const ttlLog = [];
  return {
    async get(key) { return map.has(key) ? map.get(key) : null; },
    async put(key, value, opts) {
      ttlLog.push({ key, expirationTtl: opts?.expirationTtl ?? null });
      map.set(key, value);
    },
    async delete(key) { map.delete(key); },
    map,
    ttlLog,
  };
}
```

Two variants exist and the difference matters:

- **Asserting fake** (`sessions.test.mjs`) — `put()` itself throws if `expirationTtl` is missing or non-finite, making the TTL law unbreakable from the inside.
- **Recording fake** (`webhook.test.mjs`, `location-pin.test.mjs`) — `put()` records everything into `ttlLog` and the case asserts on the log afterwards.

### 2. Global stubs: install, then restore in `finally`

Node has no DOM, no `navigator`, no `document`. Suites stub them per case and always restore:

```js
function stubNavigator(geolocationOrNull) {
  const original = Object.getOwnPropertyDescriptor(globalThis, "navigator");
  Object.defineProperty(globalThis, "navigator", { value: /* … */, configurable: true, writable: true });
  return () => {
    if (original) Object.defineProperty(globalThis, "navigator", original);
    else delete globalThis.navigator;
  };
}

// in the case
const restoreNav = stubNavigator({ geolocation: true });
try {
  // …
} finally {
  restoreNav();
}
```

`web/test/theme.test.mjs` does the same for `globalThis.document` (`installDocumentStub()` returns the `dataset` object, which *is* the assertion target). The headers are explicit about why: "a leaked stub would silently make a later pass/fail for the wrong reason."

`web/test/location-chain.test.mjs` also stubs `globalThis.fetch`. `bot/test/location-pin.test.mjs` patches a different layer entirely — grammY's node shim reads node-fetch's cached `default` export, so `createRequire` reaches `node-fetch` and swaps `.default` for the duration of the case.

### 3. Source-introspection gates ("grep the shipped source")

Structural claims — "no menu surface exists", "copy lives only in `strings.ts`", "nothing calls `setStyle`" — are pinned by *reading the file*, not by mocking behaviour. This is the most distinctive pattern in the repo and it appears in both workspaces:

- `bot/test/commands.test.mjs` CS4 — asserts `/keyboard/i` does not appear in any command source.
- `bot/test/privacy.test.mjs` PG1 — every line containing `.put(` must also contain `expirationTtl` referencing a `SCREAMING_CASE` constant.
- `bot/test/privacy.test.mjs` PG2 — no persistence-shaped export names exist.
- `web/test/copy-gate.test.mjs` COPY5 — no shell module inlines user-facing copy.
- `web/test/no-storage.test.mjs` GEO3 — no banned storage token appears in `web/src` code.
- `web/test/session.test.mjs` SESS4, `web/test/theme.test.mjs` PX02 case 3 — same shape.

**The `web/` gates strip comments first; the three `bot/` gates do not.** `bot/test/commands.test.mjs` CS4 and `bot/test/privacy.test.mjs` PG1/PG2 read raw `readFileSync(file, "utf8")` and match lines directly, with no `replace(` anywhere in `bot/test`. In `web/`, the convention is two replaces, repeated verbatim across files:

```js
const code = src
  .replace(/\/\*[\s\S]*?\*\//g, " ")          // block comments
  .replace(/(^|[^:\\])\/\/[^\n]*/g, "$1");    // line comments (skips URLs)
```

The reason the `web/` gates strip is documented in `no-storage.test.mjs`: `web/src/mock/telegram-mock.ts` documents the storage ban *in prose*, and prose is not a call site.

### 4. Build-artifact gates (assert on the shipped bundle)

`web/test/build-artifact.test.mjs` reads `web/dist` — the **production** build only, never `web/dist-harness/` — and proves what actually ships:

| Case | Assertion |
|---|---|
| BA1 | No emitted `.js`/`.css` references a third-party CDN host |
| BA2 | At least one self-hosted `.woff2` was copied through |
| BA3 | A hashed MapLibre worker chunk exists under `dist/assets` |
| BA4 | The built JS carries the inlined Rams style JSON |
| BA5 | `installMockWebApp` / `__ARAHARAH_MOCK__` appear in **no** emitted `.js` **or** `.html` |
| BA6 | No emitted HTML carries the runner page's fetch stubs |

These cases require `web/dist` to exist. Each one starts with `assert.ok(existsSync(DIST), claimBuildFirst().message)`, which yields:

```
run `npm run build -w theme && npm run build -w web` before this suite
```

The suite's header is emphatic about *why* BA5 scans HTML as well as JS: `web/index.html` is one entry shared by both builds, so a JS-only scan would miss an unguarded profile block entirely.

### Two more conventions worth copying

**Lazy module imports for RED phases.** `bot/test/initdata.test.mjs` wraps its import in `loadValidator()` so a missing module produces one `FAIL` per case rather than a lost suite:

```js
let validatorPromise = null;
function loadValidator() {
  if (!validatorPromise) validatorPromise = import("../src/security/initdata.ts").then((m) => m.validateInitData);
  return validatorPromise;
}
```

**Self-checks that pin the test's own primitives.** `initdata.test.mjs` T7 asserts `node:crypto`'s HMAC output equals the RFC-4231 test-case-1 digest, so a broken signing path cannot produce false-green signing tests. `theme/scripts/audit.test.mjs` T9 runs a real controlled Tailwind build to prove the assertion isn't tautological.

### Anti-patterns to avoid

- **Letting a case reach the live Bot API.** `bot/test/webhook.test.mjs` documents the constraint at length: grammY's cloudflare-mod adapter calls `bot.init()` (a live `getMe`) *before* its own secret-token compare, so no committed case may let an update reach the framework with an armed secret. Armed-secret behaviour is verified by live smokes, not by the suite.
- **Writing fixtures into the repo.** `theme/scripts/audit.test.mjs` creates everything under `/tmp/opencode/audit-test/`; the bot/web suites keep fixtures in-process entirely.
- **Adding a test framework.** The zero-dependency property is load-bearing (decision `D-12`) and repeated across all three runners.

---

## Coverage requirements

**No coverage threshold is configured.** There is no `coverageThreshold`, no `nyc`/`c8` config, and no instrumentation anywhere — because there is no framework to instrument with.

What substitutes for coverage measurement:

| Mechanism | Where |
|---|---|
| **Existence assertions on the scanned tree** | `no-storage.test.mjs` asserts `files.length >= 12` before scanning (`"scanned a real tree"`), so the gate cannot pass vacuously on an empty directory. `copy-gate.test.mjs` COPY5 asserts `files.length >= 1`. |
| **Fixture self-checks** | `theme/scripts/audit.test.mjs` aborts with `fixture setup: … not found` if `theme/src/tokens.css` loses a key the fixtures depend on. |
| **Typecheck as a structural gate** | `npm run bot:typecheck` / `web:typecheck` catch what the suites would not: a type change in a shape the runtime assertions never inspect. |

If you need coverage numbers, you would have to add tooling — which contradicts the current design. Prefer adding a case.

---

## Visual capture workflow

Two Playwright scripts generate screenshots. They are **evidence generators, not tests** — they are not part of any `npm test` command and nothing in the suites depends on them. Both share the same shape: a docroot-guarded static server on `127.0.0.1:0`, a Playwright `chromium` launch, one `MIN_BYTES = 10240` blank bar per PNG, a `FAILED`/`MISSING`/`UNDERSIZED` report, and `exit(1)` if anything is unusable.

```bash
npm run capture       # → theme/scripts/screenshot.mjs → theme/screenshots/
npm run web:capture   # → web/scripts/capture.mjs      → web/screenshots/
```

Both output directories are gitignored — they are regenerated artifacts, never committed.

### `theme/scripts/screenshot.mjs`

Serves `theme/` as docroot, then captures:

| Output | Notes |
|---|---|
| `sheet-full.png` | Component sheet at 390px, full page, after `document.fonts.ready` |
| `mockup-<screen>.png` ×4 | The four mockup screens over `file://` — CDN Tailwind, so **network required**. On failure, the committed `screen.png` is copied in as fallback and logged as `FALLBACK` |
| `basemap-kl.png` | 1280×800, waits on the page's own `RAMS-BASEMAP-READY` console signal + 2s tile settle |

The mockups are read-only reference (decision `D-06`) and are never edited.

### `web/scripts/capture.mjs`

Rebuilds all three bundles first, then serves **`web/dist-harness/`**, not `web/dist`:

```bash
npm run build -w theme && npm run build -w web && npm run build:harness -w web
```

It then asserts in both directions before launching the browser — the harness bundle *must* contain `installMockWebApp`, and `web/dist` *must not* — and fails the run if either is wrong, because a wrong bundle means the capture would race forever.

| Output | Leg | Wait mode |
|---|---|---|
| `shell-boot-1.png` | `?mock=android&sessiondelay=1000` | `settle400` |
| `shell-boot-2.png` | `?mock=android&fixdelay=1200` | `settle400` |
| `shell-live-light.png` | `?mock=android&scheme=light` | `live` — races `ARAHARAH-LIVE` on console, 2s tile settle |
| `shell-live-dark.png` | `?mock=android&scheme=dark` | `live` |
| `shell-denied.png` | `?mock=tdesktop` | `denied` — waits `.lamp[data-state="denied"]`, 90s budget (the 60s chat-pin poll must elapse first) |
| `shell-chat-pin.png` | `?mock=tdesktop&pin=1` | `live` |
| `gallery-compare.png` | Side-by-side against `stitch_minimalist_route_navigator/navigate_map_overview/screen.png` | — |

Success prints `CAPTURE OK — 7/7 non-blank captures under web/screenshots/`.

Two capture legs double as runtime verification:

- **The theme honesty check** — the `live` legs read `document.documentElement.dataset.tgTheme` after the `ARAHARAH-LIVE` signal and throw if it does not equal the requested `colorScheme`. A themed screenshot cannot silently show the wrong theme.
- **The GEO-03 runtime probe** — after boot-to-live, `page.evaluate` reads `localStorage.length`, `sessionStorage` keys, `document.cookie`, and `indexedDB.databases()`, and fails the run on any non-empty app storage surface. One allowlisted exception: the official `telegram-web-app.js` SDK writes `__telegram__initParams`, which is the SDK's own launch bookkeeping and is not app data. Any *other* key fails.

A timeout is always a hard failure with a diagnostic naming which console signals fired — never a silent skip, never a blank-but-passing shot.

---

## CI integration

**No CI/CD pipeline is detected.** There is no `.github/workflows/`, no `.gitea/workflows/`, and no other workflow definition in the repository.

<!-- VERIFY: automated CI — no workflow directory exists in the working tree as of this writing. The repository is hosted on a private Gitea instance (`http://gitmirah:3000/c-fu/arah2.git`); if a server-side runner was configured outside the repository, this section is out of date. -->

The full green check a contributor must run by hand before pushing:

```bash
npm run bot:test
npm run web:test
npm test -w theme
npm run bot:typecheck
npm run web:typecheck
npm run audit          # token-drift audit on the real dist/rams.css
npm run build          # required before web:test's BA cases can pass
```

Order matters in one place: `web/test/build-artifact.test.mjs` reads `web/dist`, so **`npm run build` must run before `npm run web:test`** or BA1–BA6 all fail with the build-first message.

---

## Quick reference

| I want to… | Command |
|---|---|
| Run every bot test | `npm run bot:test` |
| Run every Mini App test | `npm run web:test` |
| Run the theme audit tests | `npm test -w theme` |
| Typecheck the Worker | `npm run bot:typecheck` |
| Typecheck the Mini App | `npm run web:typecheck` |
| Audit the real compiled CSS | `npm run audit` |
| Add a bot test | new `bot/test/<subject>.test.mjs` — pick up automatically |
| Add a Mini App test | new `web/test/<subject>.test.mjs` — pick up automatically |
| Regenerate design screenshots | `npm run capture` |
| Regenerate Mini App screenshots | `npm run web:capture` |

**Related docs:** [README](../README.md) for the command table and the HTTP surface these suites cover · [ARCHITECTURE](ARCHITECTURE.md) for why the Worker/Mini App split is what it is · [CONFIGURATION](CONFIGURATION.md) for the secrets the `bot:` tests substitute with `12345:TESTTOKEN`.