Gladex Agent Logs

+- **Two `gladex` invocations sharing one workspace `context.db` collided with `SQLITE_BUSY`** — the item queued since 0.4.11. SQLite's default is *no* busy handler (0 ms), so the second process failed on the very first lock instead of waiting for it.
+  - New `commands/db.go`: `openDB(path)` wraps `sql.Open` and appends `?_busy_timeout=5000` to the DSN; `dbBusyTimeoutMS = 5000` is the single knob.
+  - All **26** production `sql.Open("sqlite", dbPath)` call sites across 11 files (`root` ×5, `workflow` ×6, `trust` ×5, `serve` ×2, `mcp_server` ×2, `context_diff`, `context_export`, `context_import`, `status`, `templates`, `doctor`) now call `openDB()`, so no command can silently reopen the pre-fix path. The now-unused `"database/sql"` import was dropped from the 6 files that only needed it for `sql.Open`.
+  - **Why the DSN and not a `PRAGMA` executed after opening**: `busy_timeout` is a per-connection property and `database/sql` opens a *new* underlying connection whenever the pool grows — a post-open exec would have covered only the first connection and failed exactly where it matters. `modernc.org/sqlite` applies DSN parameters on every `newConn` and strips the query off a plain filename, so no `file:` prefix is needed and the path itself is untouched.
+  - Deliberately **not** bundled in: a journal-mode change. `serve` already sets `journal_mode=WAL`; converting CLI-only workspaces to WAL is a separate, riskier change and was not part of the queued item.
+
+### Added
+- **`db_test.go` — 4 tests**: (a) `TestOpenDBAppliesBusyTimeout` asserts the PRAGMA on **two distinct pooled connections** (the first handle is checked out and held so the second one must open a fresh underlying connection); (b) `TestOpenDBIsLoadBearing` — control asserting the driver default really is `0`, so (a) cannot pass vacuously; (c) `TestSecondWriterWaitsInsteadOfSQLITE_BUSY` — end-to-end reproduction: a holder keeps a write transaction open 300 ms while a second pool must block ≥250 ms, then succeed and land both rows; (d) `TestOpenDBStaysLazy` — the helper only decorates the DSN, so `locateDB`/`msgNoWorkspace` still own the missing-workspace error path.
+
+### Verified
+- **Mutation check**: dropping `?_busy_timeout=` from `openDB` makes (a) and (c) **FAIL** (`busy_timeout=0`; second writer returns instantly instead of blocking); restoring it returns green — so the tests are load-bearing, not decorative.
+- **Live two-process proof against the deployed dev binary**: a Python `BEGIN IMMEDIATE` holder parks the write lock for 1.2 s; `gladex context push` **blocks ~0.95 s and then succeeds (exit 0)** — reproduced twice. Control: a raw writer with `timeout=0` in the identical scenario dies instantly with `database is locked`, i.e. exactly the pre-fix behaviour.
+- **Suites**: Go **177 PASS / 0 FAIL** in `cmd/gladex/commands` (163 top-level + 14 subtests) plus **11** in `cmd/gladex`; `go vet` reports only the accepted pre-existing `serve_test.go:520` warning. PHP **182** (72+49+27+16+11+7) + shell **26** → **208 passed, 0 failed**.
+- **Deploy**: rebuilt with `build.sh` → `0.2.0 / commit d545d47 / 2026-09-24T03:36:28Z`, deployed with an atomic `cp` + `mv -f` (in-place overwrite answers `ETXTBSY`), dev first, prod after the reviewer gate. All **four** copies md5-identical `9cfe2f84f10520db039f87b63f438313`; `/stats` shows `Go CLI 0.2.0 (sha: d545d47)` on dev, prod **and** public `https://gladex.de/stats`; `/healthz` **200/200**, public `healthz` **200**, public `download/gladex` **200** with bytes md5-identical to the built artifact.
+- **Reviewer gate (standing rule)**: the reviewer subagent read the diff, re-ran the Go + PHP + shell suites and the binary/health sanity checks independently → **APPROVE**; the prod copy happened only after that verdict.
+- STEP 0: **0 unread** `investor_to_agent` rows (dev **and** prod), **0 unhandled** `INBOX.md` entries (65/65 carry HANDLED) → no reply rows owed this run. No DNS write, no config change, no service restart, no API key, no spend (2026-09: 5.00 / 1.50 / 3.50, **model spend 0.00**, `*-free` only).
+
+### Notes
+- Still queued, unchanged: observe today's 08:06–08:51 CEST identity shifts; the A/B/C public-gating decision sits with the investor (msg #57); SOA MNAME placeholder stays a provider-panel action; `mailboxes/*` still holds no Dispatcher assignments (0 pending).
+
 ## [0.4.17] - 2026-09-24 — friendlier `gladex status` outside a workspace
 
 ### Fixed
 - **`gladex status` in a non-workspace directory leaked a raw SQLite error** (the item queued at the end of 0.4.15): `failed to query contexts: unable to open database file (14)`. Root cause: `findDB()` walks up looking for `.gladex/context.db` and, finding none, **silently fell back to the relative path `.gladex/context.db`**. `sql.Open()` therefore *succeeded* (opening a nonexistent file is lazy in SQLite) and the failure only surfaced later at the first `QueryRow`, phrased as an internal database error rather than a missing workspace.

> build · mimo-v2.6-flash-free
Appending this run's PROGRESS entry:
← Edit repo/agent-logs/PROGRESS.md
Index: repo/agent-logs/PROGRESS.md
===================================================================
--- repo/agent-logs/PROGRESS.md
+++ repo/agent-logs/PROGRESS.md
@@ -3116,4 +3116,82 @@
 **Next (queued):** A/B/C public-gating decision still with the investor (sent #57) —
 untouched; SOA MNAME placeholder still provider-panel-only. Then: watch today's
 scheduled identity shifts (08:06–08:51) fire with the fixed wrapper and check
 mailboxes/* for the first real assignments.
+
+## 2026-09-24T03:45:00Z — STEP 0 clean + shipped the queued SQLITE_BUSY fix: busy_timeout on every DB open (0.4.18)
+
+**STEP 0 (investor first)**: `investor_to_agent AND read=0` in
+`/opt/startup/dev/data/messages.db` → **0 rows**, same query in
+`/opt/startup/prod/data/messages.db` → **0 rows**; `/data/repo/INBOX.md` →
+**0 unhandled** (`grep '^## '` = 65 entries, all carrying a HANDLED marker, the
+prompt's RECENT INBOX block re-checked line by line — team page, /team 302,
+Dispatcher+roster, webmail fixes, LE certs, MX, relay probe all already
+~~HANDLED~~). **Nothing owed, so no `agent_to_investor` reply rows were inserted
+this run.** The unread rows that do exist are *my own outgoing* messages (11,
+incl. the #57 A/B/C escalation awaiting the investor) — not mine to answer.
+
+**This run's one step — the oldest still-open queued item (0.4.11): `SQLITE_BUSY`
+/ missing `busy_timeout`.**
+- **Root cause**: SQLite's default busy handler is *none* (0 ms), so when two
+  `gladex` processes shared one workspace `context.db` the second failed on its
+  very first lock with `database is locked (5) (SQLITE_BUSY)` instead of waiting.
+- **Fix**: new `app/src/go/cmd/gladex/commands/db.go` with `openDB(path)` =
+  `sql.Open("sqlite", path+"?_busy_timeout=5000")` (`dbBusyTimeoutMS` single
+  knob); all **26** production call sites in 11 files switched to it, unused
+  `database/sql` imports removed in the 6 files that no longer reference `sql.`.
+  **The value rides the DSN on purpose**: `busy_timeout` is per connection and
+  `database/sql` opens new underlying connections as the pool grows, so a
+  post-open `PRAGMA` would have covered only the first connection and failed
+  precisely under contention. `modernc.org/sqlite` applies DSN params on every
+  `newConn` and strips the query from a plain filename (no `file:` prefix needed).
+  Journal mode deliberately left alone — `serve` already sets WAL; converting
+  CLI-only workspaces is a bigger, separate change.
+- **Tests (4 new, `db_test.go`)**: busy_timeout observed on **two distinct pooled
+  connections** (first checked out and *held* so the second must open a fresh
+  one); a **control** asserting the driver default is `0` so the first test
+  cannot pass vacuously; the **end-to-end reproduction** (holder keeps a write tx
+  open 300 ms → second pool must block ≥250 ms, then succeed with both rows);
+  and a laziness contract (helper only decorates the DSN, `msgNoWorkspace` still
+  owns the missing-workspace error).
+- **Mutation-verified**: removing `?_busy_timeout=` from `openDB` makes the two
+  key tests **FAIL** (`busy_timeout=0`; second writer returns instantly),
+  restoring it → green (checked both directions, `db.go` md5 identical after restore).
+
+**Verified (all green)**
+- Go: **177 PASS / 0 FAIL** in `cmd/gladex/commands` (163 top-level + 14 subtests)
+  + **11** in `cmd/gladex`; `go vet` → only the accepted pre-existing
+  `serve_test.go:520` warning. PHP **182** (72+49+27+16+11+7) + shell **26**
+  → **208 passed, 0 failed**.
+- **Live two-process proof with the deployed binary**: Python `BEGIN IMMEDIATE`
+  holder parks the lock 1.2 s → `gladex context push` **blocks ~0.95 s then
+  succeeds (exit 0)**, twice in a row; **control** — a raw writer with
+  `timeout=0` in the identical scenario dies instantly with `database is locked`
+  (the exact pre-fix symptom, so the test scenario is real).
+- **Deploy (reviewer-gated)**: `build.sh` → `0.2.0 / commit d545d47 /
+  2026-09-24T03:36:28Z`, atomic `cp`+`mv -f` to dev, then the **reviewer
+  subagent** read the diff and re-ran all three suites + binary/health checks
+  itself → **APPROVE**, only then prod. All **four** copies md5-identical
+  `9cfe2f84f10520db039f87b63f438313`; `/stats` → `Go CLI 0.2.0 (sha: d545d47)`
+  on dev, prod **and** public `https://gladex.de/stats`; healthz **200/200**,
+  public `healthz` **200**, public `download/gladex` **200** with md5-identical bytes.
+- **Health/system**: units `investor-app-dev`, `investor-app-prod`, `apache2`,
+  `postfix`, `dovecot`, `git-daemon` **active**; mail queue empty; `mailboxes/*`
+  holds only `.gitkeep` → **0 pending Dispatcher assignments**; all 6 identity
+  timers armed (next 08:06 / 08:11 / 08:15 / 08:20 / 08:25 / 08:51 CEST).
+- **No spend, no secrets**: budget 2026-09 → 5.00 / 1.50 / 3.50, **model spend
+  0.00** (`*-free` only); no DNS write, no service restart, no config change,
+  no API key touched; no secret/credential read, printed or committed.
+
+**Docs**: CHANGELOG **0.4.17 → 0.4.18**; this entry. Commit + push after this write.
+
+**NEEDS-INVESTOR (unchanged, still open)**: the **A/B/C public-gating decision**
+(msg #57, still unread by the investor) and the **SOA MNAME placeholder**
+(`a.misconfigured.dns.server.invalid.` — provider panel only; my granted DNS
+scope is A/AAAA/CNAME/TXT and `pdns-api.py` exposes no SOA command). Neither
+blocks this run's work.
+
+**Next (queued)**: (1) watch the 08:06–08:51 CEST identity shifts fire unattended
+and check `mailboxes/*` for the first real Dispatcher assignments; (2) A/B/C +
+SOA stay with the investor/operator; (3) optional hardening to consider next —
+carry the same `busy_timeout` into the PHP-side SQLite opens (investor app +
+messages DB) if two writers ever contend there.

> build · mimo-v2.6-flash-free
Committing and pushing:
$ cd /data/repo && git add -A && git status --short && git commit -q -m "chore: 0.4.18 rebuild + redeploy (4 copies md5 9cfe2f84, commit d545d47) + CHANGELOG/PROGRESS — busy_timeout fix reviewer-approved" && git log --oneline -3 && git push origin main 2>&1 | tail -3
M  CHANGELOG.md
M  agent-logs/PROGRESS.md
M  app/bin/gladex
M  app/src/go/gladex
ee58379 chore: 0.4.18 rebuild + redeploy (4 copies md5 9cfe2f84, commit d545d47) + CHANGELOG/PROGRESS — busy_timeout fix reviewer-approved
d545d47 fix: workspace context.db opens now carry a 5s busy_timeout (queued SQLITE_BUSY since 0.4.11)
9ef364f docs: Dispatcher shift observed end-to-end — fixed wrapper proven (transcript 1827→26852B, no WIP theft), persona step verified 21/21, no assignments (no gaps); STEP 0 clean (0 unread); §9.2 trust dashboard already shipped
To git://git.gladex.de/gladex.git
   9ef364f..ee58379  main -> main
> build · mimo-v2.6-flash-free
Final verification sweep (system status + re-check for any message that arrived during the run):
$ sqlite3 /opt/startup/dev/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; sqlite3 /opt/startup/prod/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; grep '^## ' /data/repo/INBOX.md | grep -vc HANDLED; cd /data/repo && git status --short | wc -l; git log origin/main..main --oneline | wc -l; curl -s -o /dev/null -w 'dev:%{http_code} ' http://127.0.0.1:8000/healthz; curl -s -o /dev/null -w 'prod:%{http_code}\n' http://127.0.0.1:8001/healthz; ./tools/budget-show 2>&1 | head -8
0
0
0
0
0
dev:200 prod:200
=== Budget Status ===
Month:       2026-09
Allowance:   5.00 €
Spent:       1.50 (gladex.de — investor-approved 2026-09-20, purchase pending at registrar) €
Remaining:   3.50 €

Log:
  - 2026-09-20: init, no domains ordered yet
$ cd /data/repo && ./tools/system-status 2>&1 | tail -30
=== Gladex System Status ===

  investor-app-dev          [OK] active
  investor-app-prod         [OK] active
  git-daemon                [OK] active
  apache2                   [OK] active
  certbot-renew             [OK] active
  watchdog                  [OK] active
  healthz-dev               [OK] 200
  healthz-prod              [OK] 200
  download                  [OK] 200
  api-health                [OK] 200
  api-uptime                [OK] 200
  api-endpoints             [OK] 200
  api-changelog             [OK] 200
  api-stats                 [OK] 200
  DNS:gladex.de             [OK] 77.90.15.49
  DNS:dev.gladex.de         [OK] 77.90.15.49
  DNS:git.gladex.de         [OK] 77.90.15.49
  DNS:info.gladex.de        [OK] 77.90.15.49
  DNS:stats.gladex.de       [OK] 77.90.15.49
  DNS:team.gladex.de        [OK] 77.90.15.49
  tls-cert-expiry           [OK] 89d left
  investor-messages         [OK] 0 unread
  git-tree                  [OK] clean
  go-tests                  [OK] passing

Overall: ALL SYSTEMS HEALTHY
> build · mimo-v2.6-flash-free
Run complete. Summary:

## STEP 0 — investor first
- `investor_to_agent AND read=0` → **0 rows** in **both** dev and prod `messages.db`
- `INBOX.md` → **0 unhandled** entries (65/65 carry HANDLED; the prompt's RECENT INBOX block re-checked — team page, `/team` 302, Dispatcher roster, webmail fixes, LE certs, MX, relay probe all already handled)
- **No reply rows owed**, so none were inserted. The unread rows that exist are my own outgoing messages (incl. #57 A/B/C escalation awaiting the investor) — not mine to answer. Nothing to reply to before work, per the rule.

## This run's step — shipped the oldest open queued item: `SQLITE_BUSY` / missing `busy_timeout` (queued since 0.4.11)

**Fix** (`d545d47`): new `app/src/go/cmd/gladex/commands/db.go` with `openDB()` appending `?_busy_timeout=5000` to the DSN; all **26** production `sql.Open` call sites across 11 files now route through it (unused imports dropped in 6).

The value deliberately rides the **DSN**, not a post-open PRAGMA: `busy_timeout` is per-connection and `database/sql` opens new connections as the pool grows — a PRAGMA would have covered only the first one and failed exactly under contention.

**Verification**
- 4 new tests in `db_test.go`, **mutation-verified**: removing the DSN param makes two of them FAIL (`busy_timeout=0`, second writer returns instantly), restoring → green
- **Live two-process proof** on the deployed binary: a `BEGIN IMMEDIATE` holder parks the lock 1.2 s → `gladex context push` **blocks ~0.95 s then succeeds** (twice); control writer with `timeout=0` dies instantly with `database is locked` — the exact old symptom
- Suites: Go **177+11 pass / 0 fail**, PHP **182**, shell **26** → **208/0**; `go vet` only the accepted pre-existing warning
- **Reviewer gate** (standing rule): diff + all three suites re-run independently → **APPROVE**, then prod was copied
- Deploy: 4 copies md5-identical `9cfe2f84…`, `/stats` → `sha: d545d47` on dev/prod/public, healthz **200/200**, public `healthz` + `download` **200**

**Closing state**: `system-status` **ALL SYSTEMS HEALTHY** (24 checks), git clean + pushed to origin, budget 5.00 / 1.50 / 3.50 with **model spend 0.00** (`*-free` only), no DNS write, no config change, no secrets touched. CHANGELOG **0.4.18** + PROGRESS entry committed.

**Still blocked on others**: A/B/C public-gating decision (msg #57, with investor), SOA MNAME (provider panel only), and the 08:06–08:51 CEST identity shifts (wall-clock).
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

[Fri Sep 25 01:42:50 2026] 127.0.0.1:37802 Accepted
[Fri Sep 25 01:42:50 2026] 127.0.0.1:37802 Closing
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37814 Accepted
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37814 Closing
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37818 Accepted
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37818 Closing
[Fri Sep 25 01:42:54 2026] 127.0.0.1:37824 Accepted
[Fri Sep 25 01:42:54 2026] 127.0.0.1:37824 Closing
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44098 Accepted
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44098 Closing
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44106 Accepted
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44106 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44120 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44120 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44134 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44134 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44150 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44150 Closing
[Fri Sep 25 01:43:04 2026] 127.0.0.1:44164 Accepted
[Fri Sep 25 01:43:04 2026] 127.0.0.1:44164 Closing
[Fri Sep 25 01:43:04 2026] 127.0.0.1:38886 Accepted
[Fri Sep 25 01:43:04 2026] 127.0.0.1:38886 Closing
[Fri Sep 25 01:43:05 2026] 127.0.0.1:38896 Accepted
[Fri Sep 25 01:43:05 2026] 127.0.0.1:38896 Closing
[Fri Sep 25 01:43:09 2026] 127.0.0.1:38912 Accepted
[Fri Sep 25 01:43:09 2026] 127.0.0.1:38912 Closing
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38918 Accepted
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38918 Closing
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38934 Accepted
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38934 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53584 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53584 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53596 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53596 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53610 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53610 Closing
[Fri Sep 25 01:43:39 2026] 127.0.0.1:34760 Accepted
[Fri Sep 25 01:43:39 2026] 127.0.0.1:34760 Closing
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34764 Accepted
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34764 Closing
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34780 Accepted
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34780 Closing
[Fri Sep 25 01:44:35 2026] 127.0.0.1:51444 Accepted
[Fri Sep 25 01:44:35 2026] 127.0.0.1:51444 Closing
[Fri Sep 25 01:44:35 2026] 127.0.0.1:51450 Accepted
[Fri Sep 25 01:44:35 2026] 127.0.0.1:51450 Closing
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51462 Accepted
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51462 Closing
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51470 Accepted
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51470 Closing
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51474 Accepted
[Fri Sep 25 01:44:38 2026] 127.0.0.1:51474 Closing
[Fri Sep 25 01:44:41 2026] 127.0.0.1:51486 Accepted
[Fri Sep 25 01:44:41 2026] 127.0.0.1:51486 Closing
[Fri Sep 25 01:44:42 2026] 127.0.0.1:51490 Accepted
[Fri Sep 25 01:44:42 2026] 127.0.0.1:51490 Closing
[Fri Sep 25 01:44:42 2026] 127.0.0.1:51494 Accepted
[Fri Sep 25 01:44:42 2026] 127.0.0.1:51494 Closing
[Fri Sep 25 01:45:07 2026] 127.0.0.1:53326 Accepted