juxtapo/pak-jabit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
a58a61b98c61a707d1d14a827d7fea157433b36c
pak-jabit/docs/pak-jabit-manifest.md
T
juxtapoandClaude Opus 5 a58a61b98c Pak Jabit: full stack — storefront, Spree customisations, catalogue tooling 
A South African men's apparel store on Spree 5.6 + Next.js 15. 160 products
live, three markets, live Malaysian courier rates, customer-owned carts.

What is here:
  storefront/  the Next.js app
  backend/     ONLY the custom Spree code. Spree itself is a prebuilt image
               that mounts no app code, so every file here needs an explicit
               volume entry in docker-compose.yml or it is silently ignored.
  catalogue/   the scripts that loaded and maintain the products
  docs/        manifest, mock registry, napkins

Real .env files are committed deliberately. This lives on the house's private
Gitea, and a clone that runs beats a clone that needs a secrets handover.
LANDMINE.md maps every key, what it opens, and what must be rotated before this
repo goes anywhere less private.

The admin API key used to be a literal in six catalogue scripts. They now read
SPREE_ADMIN_KEY from the environment and refuse to run without it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-15 22:09:55 +08:00

347 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Pak Jabit Store — Project Manifest
## 🏗️ Architecture
Restructured 2026-09-15: backend and frontend now live on separate VMs, both
under jux-lab.com. Vercel is no longer in the path.
```
shop.jux-lab.com api.jux-lab.com + admin.jux-lab.com
(celest-proto, 34.21.217.116) (pak-jabit-backend, 35.247.170.3)
├── Next.js 16.3.5 storefront ├── Caddy (SSL)
│ systemd: pak-jabit-store └── Docker Compose
│ :3200 behind Caddy ├── Spree 5.6.1 (Rails 8.1.3)
├── Tailwind 4 ├── PostgreSQL 18
└── calls api.jux-lab.com └── Mailpit
```
Also on celest-proto (unrelated): jux-lab.com landing, mmo.jux-lab.com,
vibein.jux-lab.com, purge relay.
## 🔑 Credentials & Connections
| What | Value |
|------|-------|
| **Backend VM** | `pak-jabit-backend` · asia-southeast1-a · e2-medium · 35.247.170.3 |
| **Frontend VM** | `celest-proto` · asia-southeast1-a · e2-custom-4-8192 · 34.21.217.116 |
| **SSH** | `ssh -i /root/.ssh/google_compute_engine ijat@<ip>` |
| **Storefront** | https://shop.jux-lab.com |
| **Spree Admin** | https://admin.jux-lab.com (bare host redirects to the panel) |
| **Admin Login** | juxtapo.opus@gmail.com / ayambasuhkaki |
| **Storefront API** | https://api.jux-lab.com/api/v3/store/ |
| **Admin API** | https://api.jux-lab.com/api/v3/admin/ |
| **Publishable key** | pk_RVVf1fTUW1T4nW9FTFACm92D (header: X-Spree-Api-Key) |
| **Secret key (admin)** | sk_ntDFxFE1KwiDwqGzpCBGTJ8N — all scopes |
| **API Media** | `?expand=media` on product endpoint returns full gallery |
| **Store Currency** | **ZAR** (switched 2026-09-15). Supported: ZAR, USD, MYR — one `spree_markets` row per currency; the `spree_stores.default_currency` COLUMN is ignored, `Spree::Stores::Markets` overrides the reader |
| **Channel (Online Store)** | ch_UkLWZg9DAJ |
| **GitHub Repo** | https://github.com/juxtapo9090/pak-jabit-store (private) |
| **GCloud Project** | project-19f6479b-f9db-4588-92a |
| **DNS** | Cloud DNS zone `jux-lab-com`, same project |
| **Firewall Rule** | `allow-spree` (tcp:80,443) — applies to all instances. :3000 closed 2026-09-15; Caddy reaches Spree on 127.0.0.1:3000 |
## 🐳 Docker on the backend VM
```bash
ssh -i /root/.ssh/google_compute_engine ijat@35.247.170.3
cd ~/pak-jabit
docker compose up -d # start
docker compose down # stop
docker logs pak-jabit-web-1 # spree logs
docker exec pak-jabit-web-1 bin/rails runner "..." # rails console (~30s boot)
```
⚠️ `docker compose restart` does NOT re-read `.env`. Use `docker compose up -d`
after any env change.
## 🖥️ Storefront on celest-proto
```bash
ssh ijat@34.21.217.116
cd ~/pak-jabit-store
npm run build && sudo systemctl restart pak-jabit-store
sudo journalctl -u pak-jabit-store -f
```
⚠️ Delete `.env.local` before building — Next.js gives it precedence over
`.env.production`, and the Vercel-era copy points at the old backend.
## 📦 Products
- **160 products** live (30 original + 130 imported 2026-09-15)
- Scraped from Zalora (source prices are **MYR**), 500+ images in ActiveStorage
- Priced from source at the live rate 2026-09-15: ZAR R99.26-R3185.09 · USD $6.11-$196.02 · MYR RM24.90-RM799.00
- Imported via the Spree Admin API v3 (`scraped-catalogue/bulk_import.py`)
- Brands: G-Shock, H&M, Lacoste, Tommy Hilfiger, Cotton On, GAP, Giordano,
Superdry, Under Armour, Calvin Klein, Camel Active, Nike, Urban Revivo,
Trendyol, Swiss Polo, and more
### 🔴 Five gates a product must pass to appear on the storefront
Learned the hard way twice — 130 products invisible on 2026-09-15 morning, and
again that evening when Image Studio created products that looked perfect and
showed nothing. The scope is `Spree::Product.available(Time.current, currency)`,
defined in `spree_core-5.6.1/app/models/concerns/spree/product_scopes.rb:315`.
Read it rather than guessing; every gate below is a clause in it.
1. **`status: 'active'`** and not discontinued.
2. **A price row in the store currency.** `POST /products` silently ignores a
scalar `"price"`. It needs `"prices": [{"amount": N, "currency": "ZAR"}]`.
The clause is `with_currency(currency)`, which joins
`variants_including_master: :prices` — no row in that exact currency, no
product.
3. **A `Spree::ProductPublication` row for the channel.** Visibility is
**per-channel**, not per-product: `for_channel` INNER JOINs
`spree_product_publications`, so a product with no publication row is dropped
whatever its status. Channels here are `1 Online Store` and `2 Wholesale`.
Creating a product in Rails does **not** create this row.
4. **Purchasable.** Either stock on hand, or `track_inventory: false` on the
master **plus a stock item per active location**. `track_inventory: false`
alone is not enough — with zero stock items `in_stock?` is false,
`purchasable?` is false, and the product is excluded.
5. **Timestamps at or before `cutoff.beginning_of_minute`.** The scope truncates
the cutoff DOWN to the minute, so `published_at = Time.current` reads as the
*future* and the product stays hidden until the next minute ticks over. Set
`Time.current.beginning_of_minute` or it looks broken for up to 59 seconds
and then mysteriously fixes itself.
`Spree::Admin::ImageStudioController#build_new_product` does all five and then
re-checks `available?` before reporting success — copy that shape.
## 🎨 Design Tokens (THANDO)
| Token | Value |
|-------|-------|
| ink | #0E0D0B |
| ochre | #C8873A |
| bone | #EDE6DA |
| bone-light | #F5F1EB |
| ochre-dark | #A86E2D |
| Font | Instrument Sans (Google Fonts) |
| Default theme | Dark |
## ☁️ NextCloudPi (Team File Sharing)
| What | Value |
|------|-------|
| **URL** | https://cloudmi/ |
| **WebDAV** | https://cloudmi/remote.php/dav/files/juxtapo/ |
| **User** | juxtapo |
| **App Password** | Bk7qS-Pe5zk-7gWoY-kpc6E-KWYky |
| **Box** | C-fu's mirah (192.168.1.22) |
## 🚚 EasyParcel (MY Shipping)
| What | Value |
|------|-------|
| **Developer Hub** | https://developer.easyparcel.com |
| **App Name** | Pak Jabit Store |
| **Client ID** | 5be15621-f711-485b-bc79-15256747ce65 |
| **Client Secret** | 9a1e611e-c4e2-4d1a-87c8-f695c1323e63 |
| **Account** | Muhammad Aizat (#10935549) |
| **Auth** | OAuth 2.0 |
| **API Docs** | https://developer.easyparcel.com (Open API Documentation) |
| **Sandbox** | Available (connect from app config page) |
| **Redirect URI** | `https://shop.jux-lab.com/api/easyparcel/callback` — must match the developer hub exactly |
| **Token store** | `~/pak-jabit-store/.easyparcel-token.json` (chmod 600) |
| **Sender address** | `STORE_POSTCODE=10150` / `STORE_STATE=MY-07` (Penang) in `.env.production` |
| **Coverage** | All 16 MY states verified live — 20 couriers peninsular, 14-15 Sabah/Sarawak |
| **Re-auth** | Open the callback URL in a browser; EasyParcel asks for login, then writes the token |
### EasyParcel token handling (fixed 2026-09-15)
The access token lives 10 hours and **cannot** be extended — that is EasyParcel's
server-side policy, and it does not need extending: `getToken()` renews it with the
refresh token automatically, and writes the rotated refresh token back.
🔴 **The original code kept the token in a module-level variable**, so every restart,
redeploy or serverless cold start dropped it and the next quote died with "run OAuth
flow first". On Vercel it would have looked random. Now persisted to disk — verified
by restarting the service mid-session and getting 20 live quotes straight after.
🔴 **`req.nextUrl.origin` is wrong behind Caddy** — it resolved to `localhost:3200`
and produced a redirect URI EasyParcel would reject. Use `APP_BASE_URL` instead.
## ⚠️ Known Issues / TODO
- [x] ~~Currency USD → ZAR~~ done 2026-09-15, all 160 products in ZAR/USD/MYR
- [ ] Payment gateway not wired (Payfast — see napkin-payment-courier.md)
- [ ] Courier: EasyParcel is in for MY; Bob Go still needed for SA
- [ ] No product variants (size/colour) — every product is master-only
- [ ] `track_inventory: false` on all products — fine for demo, not for real stock
- [ ] Admin panel has no lockout or rate limiting — Devise `lockable` is off and
there is no rack-attack, so login attempts are unlimited. Worth an IP
allowlist or basic auth on the admin.jux-lab.com Caddy block before this
goes anywhere real
- [ ] Old Spree containers still on celest-proto (stopped, volumes intact) —
delete once the new backend has proven itself for a few days
- [ ] No backups on the backend VM — daily snapshots exist for celest-proto only
## 🗺️ Planned — customer data & image pipeline
### Customer account data (UI only — the data layer already exists)
Spree ships this; we just haven't surfaced it in the storefront.
| Need | Where it already lives |
|---|---|
| Saved addresses | `spree_addresses` · `GET/POST/PATCH/DELETE /api/v3/store/customers/me/addresses` |
| Saved cards | `spree_credit_cards` · `GET /api/v3/store/customers/me/credit_cards` |
🔴 **Never store raw card data.** `spree_credit_cards` holds `last_digits`,
`cc_type`, `month`/`year`, and `gateway_customer_id` / `gateway_payment_profile_id`
— a **gateway token**, not the PAN, and no CVV column at all. That is the
PCI-correct design and it is already correct. Cards only populate once a gateway
(Payfast) is wired, so this sequences after payment.
### Photoroom batch pipeline
✅ **Built 2026-09-15 evening.** Two ways in, same pipe:
- **Image Studio** — custom Spree admin page at `/admin/image_studio`. Upload
photos from the device (multiple, `accept="image/*"` so a phone opens the
camera roll) → Photoroom → before/after review → attach to an existing product
or create a new listing. Nothing touches the catalogue until Apply; discarding
is just navigating away, since the result is held as an unattached blob.
- **CLI** — `scraped-catalogue/photoroom_to_spree.py --slug <slug> [--live]
[--position 1]`. Same pipe, scriptable, with a read-back check.
🔴 **Batch is NOT built.** One at a time, capped at `MAX_PHOTOS = 8` per request
because each image is a 3-5s synchronous round trip and more starts risking the
web server timeout. A real batch needs a background job.
**Keys** live at `/root/.photoroom/credentials` (chmod 600) on the house box, and
`PHOTOROOM_API_KEY` in the backend `.env`. Currently the **sandbox** key —
1000/month free, every output watermarked. The live key has **10 images/month**
on the free tier (9 left as of 2026-09-15); the 7-day trial is the *web app*, a
separate meter from API credits, and its Batch tool does 250 images at once
through the browser without touching API quota.
⚠️ Photoroom adds little to the scraped catalogue — those images are already
studio shots on white, and the only real inconsistency is framing (2 aspect
ratios, 4 sizes across 160 products). Its value is phone photos of new stock,
which is what Image Studio is for. A free Pillow square-and-pad would fix the
existing 160.
## 📜 Session Log — 2026-09-14 (midnight build)
1. Discovered old `spree-app` VM deleted, `celest-proto` available with jux-lab.com
2. Installed Docker, cloned spree_starter, `docker compose up`
3. Created admin user, loaded 30 products via Rails script
4. Uploaded 76 product images to Spree ActiveStorage
5. Published products to "Online Store" channel (`Spree::ProductPublication`)
6. Scaffolded Next.js storefront with THANDO design tokens
7. Wired storefront to Spree API v3 (`X-Spree-Api-Key`, `?expand=media`)
8. Deployed to Vercel (pak-jabit-store-juxtapo9090.vercel.app)
9. Pushed to GitHub (juxtapo9090/pak-jabit-store)
10. Luc found `?expand=media` param for full image gallery
11. Built Zalora-style vertical thumbnail gallery (ProductImages component)
12. Hardcoded dark theme as default
## 📜 Session Log — 2026-09-15 (restructure)
1. Found the "deleted" `spree-app` VM alive in the `jux-mmo` project — stopped,
then deleted it. Its admin key never matched because each Spree instance
HMACs tokens with its own `SECRET_KEY_BASE`.
2. Imported 130 products via the Spree Admin API v3 (proper API, not Rails
runner — the presentation story is "automated import pipeline").
3. Spent an hour on 130 invisible products. Three stacked causes, all silent:
no `published_at`, not purchasable, and no price row at all.
4. Built `pak-jabit-backend` VM, migrated Postgres + 207MB ActiveStorage.
Kept `SECRET_KEY_BASE` so API keys and image URLs stayed valid.
5. Split the domain: api + admin → backend VM, shop → celest-proto.
6. Moved the storefront off Vercel onto celest-proto (systemd + Caddy).
A leftover `.env.local` from Vercel silently built it against the old
backend twice before we caught it.
7. Deleted the Vercel project. One project, two VMs, one domain.
8. Added `admin.jux-lab.com` — panel split from the API by hostname, so it can be
locked down separately later. Tested in a real browser: no bounce back to api.
9. Security pass: admin email off `spree@example.com`, password off `spree123`,
and closed `:3000` at the firewall (it was open to the internet over plain
HTTP, bypassing Caddy entirely).
10. Cart proven end-to-end on the new stack: create → add 2x item → $5,154.24.
## 📜 Session Log — 2026-09-15 (evening: ZAR, currency switcher, Image Studio, fail2ban)
1. **Backups** — `Projects/PeakyBuildr/backup-pak-jabit.sh`, nightly 04:30 via
`pak-jabit-backup.timer` on the house box → `/root/Backups/pak-jabit`.
pg_dump + ActiveStorage rsync + `.env` + compose + **the bind-mounted custom
code** (none of which is in the database). Restore proven into a throwaway
PG18 cluster: 160 products, 3200 blobs, 0 blobs missing their file.
`RESTORE.md` sits beside the dumps.
2. **The 23x price bug** — `fix_prices.py:6` set `MYR_TO_USD = 4.8` and
*multiplied*. 4.8 is the USD→MYR rate; MYR→USD is a divide. Every product was
listed at 4.8² = **23.04x** its real price (a MYR 537 watch at $2,577).
Repriced all 160 from source at the live rate.
3. **USD → ZAR** — store now quotes rand. 160 rows each in ZAR/USD/MYR.
4. **Currency switcher** on the storefront (ZAR · USD · MYR), URL-param driven so
the server fetches real Spree price rows — nothing converted in the browser.
The cart deliberately ignores it and stays on the store default.
5. **Image Studio** — custom admin page, see the Photoroom section above.
6. **fail2ban** — three jails, see Security below.
## 🔒 Security (2026-09-15 evening)
**fail2ban**, enabled at boot on the backend VM, banning at iptables so offenders
never reach Caddy or Rails:
| Jail | Trigger | Ban |
|---|---|---|
| `caddy-admin-login` | 8 POSTs to `/admin_users/sign_in` in 10 min | 24h |
| `caddy-scan` | 25 × 4xx in 10 min | 1h |
| `sshd` | 5 failed SSH in 10 min | 24h |
`ignoreip = 127.0.0.1/8 ::1 34.21.217.116 27.125.248.0/24` — celest-proto must
never be banned (it pulls product images constantly; banning it takes the shop
down), and the /24 is abang's ISP block, which his public IP rotates inside.
An allports ban on that range would also kill SSH to the box.
Caddy has **no `rate_limit` module** in the standard build, which is why this is
fail2ban and not a Caddy directive.
🔴 **Caddy had no access log until tonight** — errors only, so successful
requests left no trace whatsoever. Now JSON to journald (a file under
`/var/log/caddy/` fails to open even with the directory writable, and Caddy
fails the *whole service* on start rather than skipping the log — it dropped the
site for about a minute finding that out).
⚠️ **Unresolved:** `42.188.122.106` (Windows, Brave 153, carrying a session
cookie) requested `/admin/api_keys`, `/admin/store/edit` and `/admin/admin_users`.
Abang's browser is Linux on `27.125.248.0/24` — different machine. Those specific
requests 502'd mid-restart, but with no access logs before tonight any earlier
success is invisible. Most likely a colleague, since abang expects colleagues to
have access. `sign_in_count` is 0 but Devise trackable looks disabled, so that
number proves nothing.
🔴 **There is exactly ONE admin account**, shared. Every session is
indistinguishable, which is precisely why the above cannot be attributed. Give
each person their own account (Settings → Users) before more people get access.
## 💣 Landmines hit today
Every one of these returned success while being wrong. Worth knowing before
touching this stack again.
| Landmine | What it looked like | The actual cause |
|---|---|---|
| **Scalar `price` is silently dropped** | `POST /products` returns 200, product exists, price is `null` | Spree wants `prices: [{amount, currency}]`. No price row in the store currency → excluded from `available(Time.current, currency)` → invisible on the storefront, no error anywhere |
| **`published_at` left null** | `add_products` returns `{product_count: 130}` | The publication row is created but not live unless you pass `published_at` |
| **Not purchasable** | Product active, available, published | No stock and `track_inventory: true`. Needs stock or `track_inventory: false` |
| **`.env.local` beats `.env.production`** | Build succeeds, only 30 of 160 pages | Next.js precedence. The Vercel-era file pointed at the old backend, which was still running and still serving 30 products — so the build read stale data and reported nothing wrong. Cost two full rebuilds |
| **`docker compose restart` ignores `.env`** | Env edited, behaviour unchanged | `restart` reuses the existing container config. Needs `up -d` |
| **Admin API key looked broken** | `find_by_secret_token` → no match, every time | The key was copied from a *different* Spree instance. Each one HMACs tokens with its own `SECRET_KEY_BASE` |
| **"Deleted" VM was still running** | Manifest said deleted; nothing in the project list | It was in a different GCP project (`jux-mmo`), still billing |
| **OAuth token in a module variable** | Shipping worked, then randomly didn't | Process memory dies on restart/redeploy/cold start. Persist the refresh token to disk or DB |
| **`req.nextUrl.origin` behind a proxy** | OAuth redirect_uri came out as `localhost:3200` | Next.js sees the internal listener, not the public host. Use an explicit `APP_BASE_URL` |
| **100KB API body cap** | Image upload → `request_too_large` on a 115KB jpg | `Spree::Api::Config[:max_request_body_size]` default. 🔴 **The fix recorded here was never live.** `config/initializers/api_body_limit.rb` existed on the host with the 10MB value but the container never loaded it — the compose file mounts no app code, so the file was dead. Fixed by bind-mounting it (2026-09-15 evening). A runtime assignment also does nothing, but not for the reason written before: it isn't read at boot, it just isn't *persisted*, so it only affects the process that ran it |
| **Prebuilt image mounts no app code** | Initializer written, never ran | `ghcr.io/spree/spree` ships its own `/rails`. Anything custom needs an explicit bind mount in `docker-compose.yml`. This silently killed the 10MB body-limit fix for a whole day |
| **`add_routes` from an initializer is a no-op** | Sidebar entry appears, path helper does not exist | Spree draws engine routes BEFORE `config/initializers` run. Routes must go in `config/routes.rb` |
| **New asset defaults to position 1** | Appended an image, it became the product thumbnail | Spree defaults `position` to 1. A watermarked sandbox image went live as a public thumbnail this way. Always send an explicit position |
| **`max(:position)` on `media` misses half the gallery** | Appended image landed mid-gallery at position 2 | A gallery is the UNION of product-level `media` and legacy assets on the master variant. Clear both |
| **`product.stores << store`** | `NoMethodError` on a Relation | Products `belongs_to :store` (singular) in 5.6. Use `store:` on create |
| **`published_at = Time.current` hides the product** | Created, active, priced, stocked — invisible for up to 59s | The scope compares against `cutoff.beginning_of_minute`, truncating DOWN. Use `Time.current.beginning_of_minute` |
| **`curl -o` loses binary bodies here** | HTTP 200, `content-length: 758983`, zero-byte file | The rtk proxy layer eats it. Use Python for binary downloads on the house box |
Full write-up: `/mnt/playground/Server_files/Debug-Logs/2026-09-15_celeste_pak-jabit-restructure-and-invisible-products.md`
Reference in New Issue View Git Blame Copy Permalink