c-fu/arah2
1
0
Fork 1
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add sume

Browse Source
This commit is contained in:
araharah-executor
2026-10-23 19:18:43 +08:00
parent 1890b5b1ca
commit af3edefbc6
9 changed files with 3027 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/TESTING.md
+446
View File
@@ -0,0 +1,446 @@
<!-- generated-by: gsd-doc-writer -->
# Testing
arah² has no third-party test framework. Every suite in this repo is a hand-rolled, zero-dependency Node script that registers cases into a small array and drains it. The runners were written first in `theme/`, then copied verbatim into `bot/` and `web/` — the three share one shape, deliberately.
Beyond the suites, two additional gates stand in for CI (this repo has none — see [CI integration](#ci-integration)): `tsc --noEmit` typechecks, and the theme's off-token audit is itself a tested CLI.
---
## Contents
- [Test framework and setup](#test-framework-and-setup)
- [Running tests](#running-tests)
- [Writing new tests](#writing-new-tests)
- [Test patterns](#test-patterns)
- [Coverage requirements](#coverage-requirements)
- [Visual capture workflow](#visual-capture-workflow)
- [CI integration](#ci-integration)
---
## Test framework and setup
### What "framework" means here
| Workspace | Runner | Harness | Registered via | Command |
|---|---|---|---|---|
| `bot/` | `bot/test/runner.mjs` | `bot/test/harness.mjs` | `import { test } from "./harness.mjs"` | `npm run bot:test` |
| `web/` | `web/test/runner.mjs` | `web/test/harness.mjs` | `import { test } from "./harness.mjs"` | `npm run web:test` |
| `theme/` | inline in `theme/scripts/audit.test.mjs` | local `check()` helper | `check("name", cond, detail)` | `npm test -w theme` |
There is **no jest, vitest, mocha, playwright-test, or any other runner** in any `package.json`. The only test-relevant dev dependency anywhere is `typescript` (`tsc --noEmit`, used as a compile-time gate) and `playwright` (used by the capture scripts, not by any suite).
### The harness (10 lines, duplicated per workspace)
`bot/test/harness.mjs` and `web/test/harness.mjs` are byte-identical:
```js
const tests = [];
export function test(name, fn) {
tests.push({ name, fn });
}
export const registry = tests;
```
`test(name, fn)` pushes a case. `fn` may be sync or `async` — the runner `await`s every one.
### The runner (47 lines, duplicated per workspace)
`bot/test/runner.mjs` and `web/test/runner.mjs` are also byte-identical in behaviour:
1. `readdirSync(import.meta.dirname)`, filter `*.test.mjs`, sort.
2. Dynamic-`import()` each suite inside a `try/catch`. A suite that throws at load time is **not** fatal — it is pushed into the registry as a single failing case named `suite load failed: <file>`. The run always completes.
3. Drain `registry` in registration order: `PASS <name>` on stdout, or `FAIL <name>` plus the error stack on stderr.
4. Print `[runner] <n> passed, <m> failed` and set `process.exitCode = 1` if anything failed.
Registration order is therefore: **suite files in alphabetical order, cases in file order**. There is no shuffling, no parallelism, no filtering, no watch mode.
### TypeScript is loaded directly
Both `bot/test/` and `web/test/` `await import("../src/....ts")` directly against the shipped source — no build step, no transpile step, no fixtures directory. This works because Node strips types natively (the code comments call this "Node 24 type stripping"). <!-- VERIFY: minimum Node.js version for native `.ts` import support — no `engines` field or `.nvmrc` is committed; the test suites themselves reference "Node 24 type stripping". -->
### Assertion style
Every suite in `bot/` and `web/` uses **`node:assert`'s strict mode**, imported as:
```js
import { strict as assert } from "node:assert";
```
In practice: `assert.ok(...)` for existence checks, `assert.equal` / `assert.strictEqual` for scalar equality, `assert.deepEqual` / `assert.deepStrictEqual` for structural equality, `assert.match` / `assert.doesNotMatch` for regex, and `assert.throws` / `assert.doesNotThrow` when a throw is the expected result. Every assertion carries a message string explaining the invariant being pinned.
`theme/scripts/audit.test.mjs` does not use `node:assert` at all — it defines a local `check(name, cond, detail = '')` that accumulates into `passes[]` / `failures[]`, reports them all at the end, and exits 1 if any failed.
---
## Running tests
### Everything, per workspace
```bash
# from the repo root
npm run bot:test # → node test/runner.mjs in bot/
npm run web:test # → node test/runner.mjs in web/
npm test -w theme # → node ./scripts/audit.test.mjs in theme/
```
There is no root-level `npm test` aggregate — run all three, or add one to `package.json` when you need it.
### Running one suite
The runners auto-discover by filename and offer no filter flag. To run a single file, invoke it directly from inside the workspace:
```bash
cd bot && node test/initdata.test.mjs
```
This works because a suite file's top-level `test(...)` calls register into `harness.mjs`, and `runner.mjs` is what drains the registry — so importing a suite alone reports nothing. **A single suite file run this way executes the module side effects (imports, fixture writes) but produces no pass/fail output.** Always go through the runner:
```bash
cd bot && node test/runner.mjs # all bot suites
cd web && node test/runner.mjs # all web suites
```
`theme` is the exception — its suite is a standalone script and *does* report standalone:
```bash
cd theme && node scripts/audit.test.mjs
```
### Typecheck gates
The `bot/` and `web/` suites do not compile. Run these alongside:
```bash
npm run bot:typecheck # tsc --noEmit in bot/
npm run web:typecheck # tsc --noEmit in web/
```
### The theme audit as a standalone tool
The audit is both the thing under test and a useful CLI in its own right. Run it on the real build:
```bash
npm run audit # → node ./scripts/audit.mjs ./dist/rams.css
```
Its exit codes are a contract: `0` clean, `1` violations (each printed as `file:line: declaration`), `2` token-sheet drift or error. See [theme/scripts/audit.mjs](theme/scripts/audit.mjs) for the full usage block and its scope adjudications.
```mermaid
graph TD
ROOT["npm run bot:test / web:test / audit"] --> BR["bot/test/runner.mjs"]
ROOT --> WR["web/test/runner.mjs"]
ROOT --> TR["theme/scripts/audit.test.mjs"]
BR --> BDISC["readdir *.test.mjs → dynamic import"]
BDISC --> BREG["bot/test/harness.mjs registry"]
BREG --> BCASES["commands · initdata · location-pin · privacy · sessions · webhook"]
WR --> WDISC["readdir *.test.mjs → dynamic import"]
WDISC --> WREG["web/test/harness.mjs registry"]
WREG --> WCASES["build-artifact · copy-gate · location-chain · no-storage · session · theme"]
TR --> TFIX["fixtures under /tmp/opencode/audit-test/"]
TFIX --> TSPAWN["spawnSync scripts/audit.mjs"]
TSPAWN --> TCHECK["check() → passes[] / failures[]"]
TCHECK --> TREAL["also audits real dist/rams.css"]
classDef entry fill:#EB8008,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
classDef runner fill:#1A1A1A,stroke:#EB8008,stroke-width:2px,color:#F7F0E3
classDef harness fill:#D9D2C6,stroke:#1A1A1A,stroke-width:2px,color:#1A1A1A
classDef theme fill:#F5EDE0,stroke:#736B1E,stroke-width:2px,color:#1E1B14
class ROOT entry
class BR,WR,TR runner
class BDISC,BREG,WDISC,WREG harness
class TFIX,TSPAWN,TCHECK,TREAL theme
```
---
## Writing new tests
### Naming and location
| Rule | Value |
|---|---|
| File name | `<subject>.test.mjs` — the runner only discovers this suffix |
| Directory | `bot/test/` or `web/test/`, flat, no subdirectories |
| Import the harness | `import { test } from "./harness.mjs";` |
| Case name | Prefix with a stable case code: `"WK1 POST valid initData (under cap) → 200 {token: uuid}"`, `"GEO3 no banned storage token…"`, `"CS4 D-02: the command tree source has no reply-menu…"`. The code is the identity; the prose is the explanation. |
A new file dropped into `bot/test/` or `web/test/` is picked up automatically on the next run — no manifest, no import list, no registration step anywhere.
### The minimum shape
```js
#!/usr/bin/env node
/*
* <subject>.test.mjs — what this suite pins, and under which requirement id.
* ------------------------------------------------------------------------
* <CODE> one line per case, so a reader can see the contract without
* reading the code.
*/
import { strict as assert } from "node:assert";
import { test } from "./harness.mjs";
const { someExport } = await import("../src/some/module.ts");
test("<CODE> <what is asserted> <and why>", () => {
assert.equal(someExport("input"), "expected", "why this matters");
});
test("<CODE> async path", async () => {
const res = await someAsync();
assert.ok(res.ok, `expected ok:true, got ${JSON.stringify(res)}`);
});
```
For `theme/`, add cases to `theme/scripts/audit.test.mjs` using the local helper instead:
```js
{
const r = runAudit([dirtyFixture]);
check('T2 dirty stylesheet → exit 1', r.status === 1, `status=${r.status} error=${r.error}`);
}
```
Note the theme suite uses a **bare block** (`{ … }`) rather than a named function, and relies on `passes[]`/`failures[]` instead of exceptions.
### Where the source of truth lives
Every suite header comment lists its cases up front. That header is the documentation — keep it in sync when adding a case. Several suites reference the decision ids (`D-02`, `PRIV-02`, `GEO-03`, `CS1`, `COPY5`, `WK3`) that the case is pinning; reuse that convention.
---
## Test patterns
The repo has converged on four recognisable patterns. Follow the closest precedent rather than inventing a new one.
### 1. Injected fakes over mocks and over the real platform
`bot/test/sessions.test.mjs` and `bot/test/webhook.test.mjs` both hand-roll a ~20-line `KVNamespace` fake rather than depend on `miniflare` or `wrangler`:
```js
function makeKV() {
const map = new Map();
const ttlLog = [];
return {
async get(key) { return map.has(key) ? map.get(key) : null; },
async put(key, value, opts) {
ttlLog.push({ key, expirationTtl: opts?.expirationTtl ?? null });
map.set(key, value);
},
async delete(key) { map.delete(key); },
map,
ttlLog,
};
}
```
Two variants exist and the difference matters:
- **Asserting fake** (`sessions.test.mjs`) — `put()` itself throws if `expirationTtl` is missing or non-finite, making the TTL law unbreakable from the inside.
- **Recording fake** (`webhook.test.mjs`, `location-pin.test.mjs`) — `put()` records everything into `ttlLog` and the case asserts on the log afterwards.
### 2. Global stubs: install, then restore in `finally`
Node has no DOM, no `navigator`, no `document`. Suites stub them per case and always restore:
```js
function stubNavigator(geolocationOrNull) {
const original = Object.getOwnPropertyDescriptor(globalThis, "navigator");
Object.defineProperty(globalThis, "navigator", { value: /* … */, configurable: true, writable: true });
return () => {
if (original) Object.defineProperty(globalThis, "navigator", original);
else delete globalThis.navigator;
};
}
// in the case
const restoreNav = stubNavigator({ geolocation: true });
try {
// …
} finally {
restoreNav();
}
```
`web/test/theme.test.mjs` does the same for `globalThis.document` (`installDocumentStub()` returns the `dataset` object, which *is* the assertion target). The headers are explicit about why: "a leaked stub would silently make a later pass/fail for the wrong reason."
`web/test/location-chain.test.mjs` also stubs `globalThis.fetch`. `bot/test/location-pin.test.mjs` patches a different layer entirely — grammY's node shim reads node-fetch's cached `default` export, so `createRequire` reaches `node-fetch` and swaps `.default` for the duration of the case.
### 3. Source-introspection gates ("grep the shipped source")
Structural claims — "no menu surface exists", "copy lives only in `strings.ts`", "nothing calls `setStyle`" — are pinned by *reading the file*, not by mocking behaviour. This is the most distinctive pattern in the repo and it appears in both workspaces:
- `bot/test/commands.test.mjs` CS4 — asserts `/keyboard/i` does not appear in any command source.
- `bot/test/privacy.test.mjs` PG1 — every line containing `.put(` must also contain `expirationTtl` referencing a `SCREAMING_CASE` constant.
- `bot/test/privacy.test.mjs` PG2 — no persistence-shaped export names exist.
- `web/test/copy-gate.test.mjs` COPY5 — no shell module inlines user-facing copy.
- `web/test/no-storage.test.mjs` GEO3 — no banned storage token appears in `web/src` code.
- `web/test/session.test.mjs` SESS4, `web/test/theme.test.mjs` PX02 case 3 — same shape.
**Every one of these gates strips comments first.** The convention is two replaces, repeated verbatim across files:
```js
const code = src
.replace(/\/\*[\s\S]*?\*\//g, " ") // block comments
.replace(/(^|[^:\\])\/\/[^\n]*/g, "$1"); // line comments (skips URLs)
```
The reason is documented in `no-storage.test.mjs`: `web/src/mock/telegram-mock.ts` documents the storage ban *in prose*, and prose is not a call site.
### 4. Build-artifact gates (assert on the shipped bundle)
`web/test/build-artifact.test.mjs` reads `web/dist` — the **production** build only, never `web/dist-harness/` — and proves what actually ships:
| Case | Assertion |
|---|---|
| BA1 | No emitted `.js`/`.css` references a third-party CDN host |
| BA2 | At least one self-hosted `.woff2` was copied through |
| BA3 | A hashed MapLibre worker chunk exists under `dist/assets` |
| BA4 | The built JS carries the inlined Rams style JSON |
| BA5 | `installMockWebApp` / `__ARAHARAH_MOCK__` appear in **no** emitted `.js` **or** `.html` |
| BA6 | No emitted HTML carries the runner page's fetch stubs |
These cases require `web/dist` to exist. Each one starts with `assert.ok(existsSync(DIST), claimBuildFirst().message)`, which yields:
```
run `npm run build -w theme && npm run build -w web` before this suite
```
The suite's header is emphatic about *why* BA5 scans HTML as well as JS: `web/index.html` is one entry shared by both builds, so a JS-only scan would miss an unguarded profile block entirely.
### Two more conventions worth copying
**Lazy module imports for RED phases.** `bot/test/initdata.test.mjs` wraps its import in `loadValidator()` so a missing module produces one `FAIL` per case rather than a lost suite:
```js
let validatorPromise = null;
function loadValidator() {
if (!validatorPromise) validatorPromise = import("../src/security/initdata.ts").then((m) => m.validateInitData);
return validatorPromise;
}
```
**Self-checks that pin the test's own primitives.** `initdata.test.mjs` T7 asserts `node:crypto`'s HMAC output equals the RFC-4231 test-case-1 digest, so a broken signing path cannot produce false-green signing tests. `theme/scripts/audit.test.mjs` T9 runs a real controlled Tailwind build to prove the assertion isn't tautological.
### Anti-patterns to avoid
- **Letting a case reach the live Bot API.** `bot/test/webhook.test.mjs` documents the constraint at length: grammY's cloudflare-mod adapter calls `bot.init()` (a live `getMe`) *before* its own secret-token compare, so no committed case may let an update reach the framework with an armed secret. Armed-secret behaviour is verified by live smokes, not by the suite.
- **Writing fixtures into the repo.** `theme/scripts/audit.test.mjs` creates everything under `/tmp/opencode/audit-test/`; the bot/web suites keep fixtures in-process entirely.
- **Adding a test framework.** The zero-dependency property is load-bearing (decision `D-12`) and repeated across all three runners.
---
## Coverage requirements
**No coverage threshold is configured.** There is no `coverageThreshold`, no `nyc`/`c8` config, and no instrumentation anywhere — because there is no framework to instrument with.
What substitutes for coverage measurement:
| Mechanism | Where |
|---|---|
| **Existence assertions on the scanned tree** | `no-storage.test.mjs` asserts `files.length >= 12` before scanning (`"scanned a real tree"`), so the gate cannot pass vacuously on an empty directory. `copy-gate.test.mjs` COPY5 asserts `files.length >= 1`. |
| **Fixture self-checks** | `theme/scripts/audit.test.mjs` aborts with `fixture setup: … not found` if `theme/src/tokens.css` loses a key the fixtures depend on. |
| **Typecheck as a structural gate** | `npm run bot:typecheck` / `web:typecheck` catch what the suites would not: a type change in a shape the runtime assertions never inspect. |
If you need coverage numbers, you would have to add tooling — which contradicts the current design. Prefer adding a case.
---
## Visual capture workflow
Two Playwright scripts generate screenshots. They are **evidence generators, not tests** — they are not part of any `npm test` command and nothing in the suites depends on them. Both share the same shape: a docroot-guarded static server on `127.0.0.1:0`, a Playwright `chromium` launch, one `MIN_BYTES = 10240` blank bar per PNG, a `FAILED`/`MISSING`/`UNDERSIZED` report, and `exit(1)` if anything is unusable.
```bash
npm run capture # → theme/scripts/screenshot.mjs → theme/screenshots/
npm run web:capture # → web/scripts/capture.mjs → web/screenshots/
```
Both output directories are gitignored — they are regenerated artifacts, never committed.
### `theme/scripts/screenshot.mjs`
Serves `theme/` as docroot, then captures:
| Output | Notes |
|---|---|
| `sheet-full.png` | Component sheet at 390px, full page, after `document.fonts.ready` |
| `mockup-<screen>.png` ×4 | The four mockup screens over `file://` — CDN Tailwind, so **network required**. On failure, the committed `screen.png` is copied in as fallback and logged as `FALLBACK` |
| `basemap-kl.png` | 1280×800, waits on the page's own `RAMS-BASEMAP-READY` console signal + 2s tile settle |
The mockups are read-only reference (decision `D-06`) and are never edited.
### `web/scripts/capture.mjs`
Rebuilds all three bundles first, then serves **`web/dist-harness/`**, not `web/dist`:
```bash
npm run build -w theme && npm run build -w web && npm run build:harness -w web
```
It then asserts in both directions before launching the browser — the harness bundle *must* contain `installMockWebApp`, and `web/dist` *must not* — and fails the run if either is wrong, because a wrong bundle means the capture would race forever.
| Output | Leg | Wait mode |
|---|---|---|
| `shell-boot-1.png` | `?mock=android&sessiondelay=1000` | `settle400` |
| `shell-boot-2.png` | `?mock=android&fixdelay=1200` | `settle400` |
| `shell-live-light.png` | `?mock=android&scheme=light` | `live` — races `ARAHARAH-LIVE` on console, 2s tile settle |
| `shell-live-dark.png` | `?mock=android&scheme=dark` | `live` |
| `shell-denied.png` | `?mock=tdesktop` | `denied` — waits `.lamp[data-state="denied"]`, 90s budget (the 60s chat-pin poll must elapse first) |
| `shell-chat-pin.png` | `?mock=tdesktop&pin=1` | `live` |
| `gallery-compare.png` | Side-by-side against `stitch_minimalist_route_navigator/navigate_map_overview/screen.png` | — |
Success prints `CAPTURE OK — 7/7 non-blank captures under web/screenshots/`.
Two capture legs double as runtime verification:
- **The theme honesty check** — the `live` legs read `document.documentElement.dataset.tgTheme` after the `ARAHARAH-LIVE` signal and throw if it does not equal the requested `colorScheme`. A themed screenshot cannot silently show the wrong theme.
- **The GEO-03 runtime probe** — after boot-to-live, `page.evaluate` reads `localStorage.length`, `sessionStorage` keys, `document.cookie`, and `indexedDB.databases()`, and fails the run on any non-empty app storage surface. One allowlisted exception: the official `telegram-web-app.js` SDK writes `__telegram__initParams`, which is the SDK's own launch bookkeeping and is not app data. Any *other* key fails.
A timeout is always a hard failure with a diagnostic naming which console signals fired — never a silent skip, never a blank-but-passing shot.
---
## CI integration
**No CI/CD pipeline is detected.** There is no `.github/workflows/`, no `.gitea/workflows/`, and no other workflow definition in the repository.
<!-- VERIFY: automated CI — no workflow directory exists in the working tree as of this writing. The repository is hosted on a private Gitea instance (`http://gitmirah:3000/c-fu/arah2.git`); if a server-side runner was configured outside the repository, this section is out of date. -->
The full green check a contributor must run by hand before pushing:
```bash
npm run bot:test
npm run web:test
npm test -w theme
npm run bot:typecheck
npm run web:typecheck
npm run audit # token-drift audit on the real dist/rams.css
npm run build # required before web:test's BA cases can pass
```
Order matters in one place: `web/test/build-artifact.test.mjs` reads `web/dist`, so **`npm run build` must run before `npm run web:test`** or BA1–BA6 all fail with the build-first message.
---
## Quick reference
| I want to… | Command |
|---|---|
| Run every bot test | `npm run bot:test` |
| Run every Mini App test | `npm run web:test` |
| Run the theme audit tests | `npm test -w theme` |
| Typecheck the Worker | `npm run bot:typecheck` |
| Typecheck the Mini App | `npm run web:typecheck` |
| Audit the real compiled CSS | `npm run audit` |
| Add a bot test | new `bot/test/<subject>.test.mjs` — pick up automatically |
| Add a Mini App test | new `web/test/<subject>.test.mjs` — pick up automatically |
| Regenerate design screenshots | `npm run capture` |
| Regenerate Mini App screenshots | `npm run web:capture` |
**Related docs:** [README](../README.md) for the command table and the HTTP surface these suites cover · [ARCHITECTURE](ARCHITECTURE.md) for why the Worker/Mini App split is what it is · [CONFIGURATION](CONFIGURATION.md) for the secrets the `bot:` tests substitute with `12345:TESTTOKEN`.