Arah2/docs/TESTING.md

Testing

arah² has no third-party test framework. Every suite in this repo is a hand-rolled, zero-dependency Node script that registers cases into a small array and drains it. The runners were written first in theme/, then copied verbatim into bot/ and web/ — the three share one shape, deliberately.

Beyond the suites, two additional gates stand in for CI (this repo has none — see CI integration): tsc --noEmit typechecks, and the theme's off-token audit is itself a tested CLI.

---

Contents

• Test framework and setup
• Running tests
• Writing new tests
• Test patterns
• Coverage requirements
• Visual capture workflow
• CI integration

---

Test framework and setup

What "framework" means here

Workspace	Runner	Harness	Registered via	Command
bot/	bot/test/runner.mjs	bot/test/harness.mjs	import { test } from "./harness.mjs"	npm run bot:test
web/	web/test/runner.mjs	web/test/harness.mjs	import { test } from "./harness.mjs"	npm run web:test
theme/	inline in theme/scripts/audit.test.mjs	local check() helper	check("name", cond, detail)	npm test -w theme

There is no jest, vitest, mocha, playwright-test, or any other runner in any package.json. The only test-relevant dev dependency anywhere is typescript (tsc --noEmit, used as a compile-time gate) and playwright (used by the capture scripts, not by any suite).

The harness (10 lines, duplicated per workspace)

bot/test/harness.mjs and web/test/harness.mjs are byte-identical:

const tests = [];
export function test(name, fn) {
  tests.push({ name, fn });
}
export const registry = tests;

test(name, fn) pushes a case. fn may be sync or async — the runner awaits every one.

The runner (47 lines, duplicated per workspace)

bot/test/runner.mjs and web/test/runner.mjs are also byte-identical in behaviour:

1. readdirSync(import.meta.dirname), filter *.test.mjs, sort.
2. Dynamic-import() each suite inside a try/catch. A suite that throws at load time is not fatal — it is pushed into the registry as a single failing case named suite load failed: <file>. The run always completes.
3. Drain registry in registration order: PASS <name> on stdout, or FAIL <name> plus the error stack on stderr.
4. Print [runner] <n> passed, <m> failed and set process.exitCode = 1 if anything failed.

Registration order is therefore: suite files in alphabetical order, cases in file order. There is no shuffling, no parallelism, no filtering, no watch mode.

TypeScript is loaded directly

Both bot/test/ and web/test/ await import("../src/....ts") directly against the shipped source — no build step, no transpile step, no fixtures directory. This works because Node strips types natively (the code comments call this "Node 24 type stripping").

Assertion style

Every suite in bot/ and web/ uses node:assert's strict mode, imported as:

import { strict as assert } from "node:assert";

In practice: assert.ok(...) for existence checks, assert.equal / assert.strictEqual for scalar equality, assert.deepEqual / assert.deepStrictEqual for structural equality, assert.match / assert.doesNotMatch for regex, and assert.throws / assert.doesNotThrow when a throw is the expected result. Every assertion carries a message string explaining the invariant being pinned.

theme/scripts/audit.test.mjs does not use node:assert at all — it defines a local check(name, cond, detail = '') that accumulates into passes[] / failures[], reports them all at the end, and exits 1 if any failed.

---

Running tests

Everything, per workspace

# from the repo root
npm run bot:test        # → node test/runner.mjs in bot/
npm run web:test        # → node test/runner.mjs in web/
npm test -w theme       # → node ./scripts/audit.test.mjs in theme/

There is no root-level npm test aggregate — run all three, or add one to package.json when you need it.

Running one suite

The runners auto-discover by filename and offer no filter flag. To run a single file, invoke it directly from inside the workspace:

cd bot && node test/initdata.test.mjs

This works because a suite file's top-level test(...) calls register into harness.mjs, and runner.mjs is what drains the registry — so importing a suite alone reports nothing. A single suite file run this way executes the module side effects (imports, fixture writes) but produces no pass/fail output. Always go through the runner:

cd bot && node test/runner.mjs        # all bot suites
cd web && node test/runner.mjs        # all web suites

theme is the exception — its suite is a standalone script and does report standalone:

cd theme && node scripts/audit.test.mjs

Typecheck gates

The bot/ and web/ suites do not compile. Run these alongside:

npm run bot:typecheck    # tsc --noEmit in bot/
npm run web:typecheck    # tsc --noEmit in web/

The theme audit as a standalone tool

The audit is both the thing under test and a useful CLI in its own right. Run it on the real build:

npm run audit            # → node ./scripts/audit.mjs ./dist/rams.css

Its exit codes are a contract: 0 clean, 1 violations (each printed as file:line: declaration), 2 token-sheet drift or error. See theme/scripts/audit.mjs for the full usage block and its scope adjudications.

graph TD
    ROOT["npm run bot:test / web:test / audit"] --> BR["bot/test/runner.mjs"]
    ROOT --> WR["web/test/runner.mjs"]
    ROOT --> TR["theme/scripts/audit.test.mjs"]

    BR --> BDISC["readdir *.test.mjs → dynamic import"]
    BDISC --> BREG["bot/test/harness.mjs registry"]
    BREG --> BCASES["commands · initdata · location-pin · privacy · sessions · webhook"]

    WR --> WDISC["readdir *.test.mjs → dynamic import"]
    WDISC --> WREG["web/test/harness.mjs registry"]
    WREG --> WCASES["build-artifact · copy-gate · location-chain · no-storage · session · theme"]

    TR --> TFIX["fixtures under /tmp/opencode/audit-test/"]
    TFIX --> TSPAWN["spawnSync scripts/audit.mjs"]
    TSPAWN --> TCHECK["check() → passes[] / failures[]"]
    TCHECK --> TREAL["also audits real dist/rams.css"]

    classDef entry fill:#EB8008,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
    classDef runner fill:#1A1A1A,stroke:#EB8008,stroke-width:2px,color:#F7F0E3
    classDef harness fill:#D9D2C6,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
    classDef theme fill:#F5EDE0,stroke:#736B1E,stroke-width:2px,color:#1E1B14

    class ROOT entry
    class BR,WR,TR runner
    class BDISC,BREG,WDISC,WREG harness
    class TFIX,TSPAWN,TCHECK,TREAL theme

---

Writing new tests

Naming and location

Rule	Value
File name	<subject>.test.mjs — the runner only discovers this suffix
Directory	bot/test/ or web/test/, flat, no subdirectories
Import the harness	import { test } from "./harness.mjs";
Case name	Prefix with a stable case code: "WK1 POST valid initData (under cap) → 200 {token: uuid}", "GEO3 no banned storage token…", "CS4 D-02: the command tree source has no reply-menu…". The code is the identity; the prose is the explanation.

A new file dropped into bot/test/ or web/test/ is picked up automatically on the next run — no manifest, no import list, no registration step anywhere.

The minimum shape

#!/usr/bin/env node
/*
 * <subject>.test.mjs — what this suite pins, and under which requirement id.
 * ------------------------------------------------------------------------
 *   <CODE>  one line per case, so a reader can see the contract without
 *           reading the code.
 */
import { strict as assert } from "node:assert";
import { test } from "./harness.mjs";

const { someExport } = await import("../src/some/module.ts");

test("<CODE> <what is asserted> <and why>", () => {
  assert.equal(someExport("input"), "expected", "why this matters");
});

test("<CODE> async path", async () => {
  const res = await someAsync();
  assert.ok(res.ok, `expected ok:true, got ${JSON.stringify(res)}`);
});

For theme/, add cases to theme/scripts/audit.test.mjs using the local helper instead:

{
  const r = runAudit([dirtyFixture]);
  check('T2 dirty stylesheet → exit 1', r.status === 1, `status=${r.status} error=${r.error}`);
}

Note the theme suite uses a bare block ({ … }) rather than a named function, and relies on passes[]/failures[] instead of exceptions.

Where the source of truth lives

Every suite header comment lists its cases up front. That header is the documentation — keep it in sync when adding a case. Several suites reference the decision ids (D-02, PRIV-02, GEO-03, CS1, COPY5, WK3) that the case is pinning; reuse that convention.

---

Test patterns

The repo has converged on four recognisable patterns. Follow the closest precedent rather than inventing a new one.

1. Injected fakes over mocks and over the real platform

bot/test/sessions.test.mjs and bot/test/webhook.test.mjs both hand-roll a ~20-line KVNamespace fake rather than depend on miniflare or wrangler:

function makeKV() {
  const map = new Map();
  const ttlLog = [];
  return {
    async get(key) { return map.has(key) ? map.get(key) : null; },
    async put(key, value, opts) {
      ttlLog.push({ key, expirationTtl: opts?.expirationTtl ?? null });
      map.set(key, value);
    },
    async delete(key) { map.delete(key); },
    map,
    ttlLog,
  };
}

Two variants exist and the difference matters:

• Asserting fake (sessions.test.mjs) — put() itself throws if expirationTtl is missing or non-finite, making the TTL law unbreakable from the inside.
• Recording fake (webhook.test.mjs, location-pin.test.mjs) — put() records everything into ttlLog and the case asserts on the log afterwards.

2. Global stubs: install, then restore in finally

Node has no DOM, no navigator, no document. Suites stub them per case and always restore:

function stubNavigator(geolocationOrNull) {
  const original = Object.getOwnPropertyDescriptor(globalThis, "navigator");
  Object.defineProperty(globalThis, "navigator", { value: /* … */, configurable: true, writable: true });
  return () => {
    if (original) Object.defineProperty(globalThis, "navigator", original);
    else delete globalThis.navigator;
  };
}

// in the case
const restoreNav = stubNavigator({ geolocation: true });
try {
  // …
} finally {
  restoreNav();
}

web/test/theme.test.mjs does the same for globalThis.document (installDocumentStub() returns the dataset object, which is the assertion target). The headers are explicit about why: "a leaked stub would silently make a later pass/fail for the wrong reason."

web/test/location-chain.test.mjs also stubs globalThis.fetch. bot/test/location-pin.test.mjs patches a different layer entirely — grammY's node shim reads node-fetch's cached default export, so createRequire reaches node-fetch and swaps .default for the duration of the case.

3. Source-introspection gates ("grep the shipped source")

Structural claims — "no menu surface exists", "copy lives only in strings.ts", "nothing calls setStyle" — are pinned by reading the file, not by mocking behaviour. This is the most distinctive pattern in the repo and it appears in both workspaces:

• bot/test/commands.test.mjs CS4 — asserts /keyboard/i does not appear in any command source.
• bot/test/privacy.test.mjs PG1 — every line containing .put( must also contain expirationTtl referencing a SCREAMING_CASE constant.
• bot/test/privacy.test.mjs PG2 — no persistence-shaped export names exist.
• web/test/copy-gate.test.mjs COPY5 — no shell module inlines user-facing copy.
• web/test/no-storage.test.mjs GEO3 — no banned storage token appears in web/src code.
• web/test/session.test.mjs SESS4, web/test/theme.test.mjs PX02 case 3 — same shape.

Every one of these gates strips comments first. The convention is two replaces, repeated verbatim across files:

const code = src
  .replace(/\/\*[\s\S]*?\*\//g, " ")          // block comments
  .replace(/(^|[^:\\])\/\/[^\n]*/g, "$1");    // line comments (skips URLs)

The reason is documented in no-storage.test.mjs: web/src/mock/telegram-mock.ts documents the storage ban in prose, and prose is not a call site.

4. Build-artifact gates (assert on the shipped bundle)

web/test/build-artifact.test.mjs reads web/dist — the production build only, never web/dist-harness/ — and proves what actually ships:

Case	Assertion
BA1	No emitted .js/.css references a third-party CDN host
BA2	At least one self-hosted .woff2 was copied through
BA3	A hashed MapLibre worker chunk exists under dist/assets
BA4	The built JS carries the inlined Rams style JSON
BA5	installMockWebApp / __ARAHARAH_MOCK__ appear in no emitted .js or .html
BA6	No emitted HTML carries the runner page's fetch stubs

These cases require web/dist to exist. Each one starts with assert.ok(existsSync(DIST), claimBuildFirst().message), which yields:

run `npm run build -w theme && npm run build -w web` before this suite

The suite's header is emphatic about why BA5 scans HTML as well as JS: web/index.html is one entry shared by both builds, so a JS-only scan would miss an unguarded profile block entirely.

Two more conventions worth copying

Lazy module imports for RED phases. bot/test/initdata.test.mjs wraps its import in loadValidator() so a missing module produces one FAIL per case rather than a lost suite:

let validatorPromise = null;
function loadValidator() {
  if (!validatorPromise) validatorPromise = import("../src/security/initdata.ts").then((m) => m.validateInitData);
  return validatorPromise;
}

Self-checks that pin the test's own primitives. initdata.test.mjs T7 asserts node:crypto's HMAC output equals the RFC-4231 test-case-1 digest, so a broken signing path cannot produce false-green signing tests. theme/scripts/audit.test.mjs T9 runs a real controlled Tailwind build to prove the assertion isn't tautological.

Anti-patterns to avoid

• Letting a case reach the live Bot API. bot/test/webhook.test.mjs documents the constraint at length: grammY's cloudflare-mod adapter calls bot.init() (a live getMe) before its own secret-token compare, so no committed case may let an update reach the framework with an armed secret. Armed-secret behaviour is verified by live smokes, not by the suite.
• Writing fixtures into the repo. theme/scripts/audit.test.mjs creates everything under /tmp/opencode/audit-test/; the bot/web suites keep fixtures in-process entirely.
• Adding a test framework. The zero-dependency property is load-bearing (decision D-12) and repeated across all three runners.

---

Coverage requirements

No coverage threshold is configured. There is no coverageThreshold, no nyc/c8 config, and no instrumentation anywhere — because there is no framework to instrument with.

What substitutes for coverage measurement:

Mechanism	Where
Existence assertions on the scanned tree	no-storage.test.mjs asserts files.length >= 12 before scanning ("scanned a real tree"), so the gate cannot pass vacuously on an empty directory. copy-gate.test.mjs COPY5 asserts files.length >= 1.
Fixture self-checks	theme/scripts/audit.test.mjs aborts with fixture setup: … not found if theme/src/tokens.css loses a key the fixtures depend on.
Typecheck as a structural gate	npm run bot:typecheck / web:typecheck catch what the suites would not: a type change in a shape the runtime assertions never inspect.

If you need coverage numbers, you would have to add tooling — which contradicts the current design. Prefer adding a case.

---

Visual capture workflow

Two Playwright scripts generate screenshots. They are evidence generators, not tests — they are not part of any npm test command and nothing in the suites depends on them. Both share the same shape: a docroot-guarded static server on 127.0.0.1:0, a Playwright chromium launch, one MIN_BYTES = 10240 blank bar per PNG, a FAILED/MISSING/UNDERSIZED report, and exit(1) if anything is unusable.

npm run capture       # → theme/scripts/screenshot.mjs → theme/screenshots/
npm run web:capture   # → web/scripts/capture.mjs      → web/screenshots/

Both output directories are gitignored — they are regenerated artifacts, never committed.

theme/scripts/screenshot.mjs

Serves theme/ as docroot, then captures:

Output	Notes
sheet-full.png	Component sheet at 390px, full page, after document.fonts.ready
mockup-<screen>.png ×4	The four mockup screens over file:// — CDN Tailwind, so network required. On failure, the committed screen.png is copied in as fallback and logged as FALLBACK
basemap-kl.png	1280×800, waits on the page's own RAMS-BASEMAP-READY console signal + 2s tile settle

The mockups are read-only reference (decision D-06) and are never edited.

web/scripts/capture.mjs

Rebuilds all three bundles first, then serves web/dist-harness/, not web/dist:

npm run build -w theme && npm run build -w web && npm run build:harness -w web

It then asserts in both directions before launching the browser — the harness bundle must contain installMockWebApp, and web/dist must not — and fails the run if either is wrong, because a wrong bundle means the capture would race forever.

Output	Leg	Wait mode
shell-boot-1.png	?mock=android&sessiondelay=1000	settle400
shell-boot-2.png	?mock=android&fixdelay=1200	settle400
shell-live-light.png	?mock=android&scheme=light	live — races ARAHARAH-LIVE on console, 2s tile settle
shell-live-dark.png	?mock=android&scheme=dark	live
shell-denied.png	?mock=tdesktop	denied — waits .lamp[data-state="denied"], 90s budget (the 60s chat-pin poll must elapse first)
shell-chat-pin.png	?mock=tdesktop&pin=1	live
gallery-compare.png	Side-by-side against stitch_minimalist_route_navigator/navigate_map_overview/screen.png	—

Success prints CAPTURE OK — 7/7 non-blank captures under web/screenshots/.

Two capture legs double as runtime verification:

• The theme honesty check — the live legs read document.documentElement.dataset.tgTheme after the ARAHARAH-LIVE signal and throw if it does not equal the requested colorScheme. A themed screenshot cannot silently show the wrong theme.
• The GEO-03 runtime probe — after boot-to-live, page.evaluate reads localStorage.length, sessionStorage keys, document.cookie, and indexedDB.databases(), and fails the run on any non-empty app storage surface. One allowlisted exception: the official telegram-web-app.js SDK writes __telegram__initParams, which is the SDK's own launch bookkeeping and is not app data. Any other key fails.

A timeout is always a hard failure with a diagnostic naming which console signals fired — never a silent skip, never a blank-but-passing shot.

---

CI integration

No CI/CD pipeline is detected. There is no .github/workflows/, no .gitea/workflows/, and no other workflow definition in the repository.

The full green check a contributor must run by hand before pushing:

npm run bot:test
npm run web:test
npm test -w theme
npm run bot:typecheck
npm run web:typecheck
npm run audit          # token-drift audit on the real dist/rams.css
npm run build          # required before web:test's BA cases can pass

Order matters in one place: web/test/build-artifact.test.mjs reads web/dist, so npm run build must run before npm run web:test or BA1–BA6 all fail with the build-first message.

---

Quick reference

I want to…	Command
Run every bot test	npm run bot:test
Run every Mini App test	npm run web:test
Run the theme audit tests	npm test -w theme
Typecheck the Worker	npm run bot:typecheck
Typecheck the Mini App	npm run web:typecheck
Audit the real compiled CSS	npm run audit
Add a bot test	new bot/test/<subject>.test.mjs — pick up automatically
Add a Mini App test	new web/test/<subject>.test.mjs — pick up automatically
Regenerate design screenshots	npm run capture
Regenerate Mini App screenshots	npm run web:capture

Related docs: README for the command table and the HTTP surface these suites cover · ARCHITECTURE for why the Worker/Mini App split is what it is · CONFIGURATION for the secrets the bot: tests substitute with 12345:TESTTOKEN.