Gladex Agent Logs

Agent run logs & app logs · env: prod · LAN-only investor surface

Overview
Run logs573 files, 20.9 MB
Latest run logrun-20260926-205455-171.log
Log directory/data/agent-logs
App log directory/opt/startup/prod/logs
Run logs (newest first, last 50)
FileSizeModified (UTC)
run-20260926-205455-171.log 132 KB 2026-09-26 19:09:01
run-20260926-202137-170.log 215 KB 2026-09-26 18:44:55
run-20260926-194055-169.log 257 KB 2026-09-26 18:11:37
run-20260926-191403-168.log 160 KB 2026-09-26 17:30:55
run-20260926-180537-167.log 281 KB 2026-09-26 17:04:03
run-20260926-170523-166.log 164 KB 2026-09-26 15:55:37
run-20260926-162230-165.log 178 KB 2026-09-26 14:55:23
run-20260926-154050-164.log 198 KB 2026-09-26 14:12:30
run-20260926-153049-163.log 153 B 2026-09-26 13:30:50
run-20260926-152049-162.log 153 B 2026-09-26 13:20:49
run-20260926-151048-161.log 153 B 2026-09-26 13:10:49
run-20260926-150047-160.log 153 B 2026-09-26 13:00:48
run-20260926-145046-159.log 153 B 2026-09-26 12:50:47
run-20260926-144046-158.log 153 B 2026-09-26 12:40:46
run-20260926-143045-157.log 153 B 2026-09-26 12:30:46
run-20260926-142044-156.log 153 B 2026-09-26 12:20:45
run-20260926-141044-155.log 153 B 2026-09-26 12:10:44
run-20260926-140043-154.log 153 B 2026-09-26 12:00:44
run-20260926-135042-153.log 190 B 2026-09-26 11:50:43
run-20260926-134042-152.log 153 B 2026-09-26 11:40:42
run-20260926-133041-151.log 153 B 2026-09-26 11:30:42
run-20260926-132040-150.log 190 B 2026-09-26 11:20:41
run-20260926-131039-149.log 153 B 2026-09-26 11:10:40
run-20260926-130039-148.log 153 B 2026-09-26 11:00:39
run-20260926-125038-147.log 190 B 2026-09-26 10:50:39
run-20260926-124037-146.log 153 B 2026-09-26 10:40:38
run-20260926-123037-145.log 153 B 2026-09-26 10:30:37
run-20260926-122036-144.log 190 B 2026-09-26 10:20:37
run-20260926-121035-143.log 190 B 2026-09-26 10:10:36
run-20260926-120035-142.log 153 B 2026-09-26 10:00:35
run-20260926-115034-141.log 153 B 2026-09-26 09:50:34
run-20260926-114033-140.log 153 B 2026-09-26 09:40:34
run-20260926-113032-139.log 153 B 2026-09-26 09:30:33
run-20260926-112032-138.log 153 B 2026-09-26 09:20:32
run-20260926-111031-137.log 153 B 2026-09-26 09:10:32
run-20260926-110026-136.log 153 B 2026-09-26 09:00:31
run-20260926-105025-135.log 153 B 2026-09-26 08:50:26
run-20260926-104024-134.log 190 B 2026-09-26 08:40:25
run-20260926-103023-133.log 153 B 2026-09-26 08:30:24
run-20260926-102023-132.log 153 B 2026-09-26 08:20:23
run-20260926-101022-131.log 190 B 2026-09-26 08:10:23
run-20260926-100021-130.log 153 B 2026-09-26 08:00:22
run-20260926-095021-129.log 153 B 2026-09-26 07:50:21
run-20260926-090029-128.log 230 KB 2026-09-26 07:40:21
run-20260926-081623-127.log 209 KB 2026-09-26 06:50:29
run-20260926-073109-126.log 146 KB 2026-09-26 06:06:23
run-20260926-061035-125.log 341 KB 2026-09-26 05:21:09
run-20260926-052113-124.log 352 KB 2026-09-26 04:00:35
run-20260926-043030-123.log 311 KB 2026-09-26 03:11:13
run-20260926-032802-122.log 338 KB 2026-09-26 02:20:30
Tail โ€” run-20260926-194055-169.log (last 200 lines)
> build ยท mimo-v2.6-flash-free
โ† Edit repo/tools/REGISTRY.md
Index: repo/tools/REGISTRY.md
===================================================================
--- repo/tools/REGISTRY.md
+++ repo/tools/REGISTRY.md
@@ -736,9 +736,9 @@
 
 **`error_code`** (present on every JSON output, `null` on a verdict):
 - `public_ip_unavailable` - every candidate answer was rejected; see `rejected[]` for why (`fetch_failed` | `empty` | `bad_json` | `not_an_ip` | `ipv6_not_comparable_to_A`)
 - `no_a_records` - a query **completed** and the answer was empty
-- `dns_lookup_failed` - no A query ever completed (dig missing, unreachable, timed out)
+- `dns_lookup_failed` - no A query ever completed (dig missing, unreachable, timed out) **or** dig's stdout carried a `;;` transport diagnostic โ€” dig writes those to stdout with stderr empty (measured 2026-09-26), and their presence means the query did not complete even when dig's exit code looked like success
 - `internal_error` - unexpected failure; a traceback never escapes (JSON in `--format json`, stderr otherwise)
 
 **Evidence rule (why `1` is hard to reach by accident)**: a drift verdict needs **two validated IPv4 literals** โ€” the egress answer and a `dig` answer line. An HTML/rate-limit body, an empty body, an IPv6 answer (not comparable to A), a CNAME/hostname line and a resolver that never completed a query are *errors*, not votes: they exit `3` with `drift: null` and can never produce `drift: true`. A bad answer from one service also never aborts the check โ€” the next service is tried. **CGNAT/private IPv4 egress is deliberately still a verdict** (`100.64.0.0/10`, RFC1918): that address *is* the tunnel-down signal `ip-drift-cron` classifies.
 

โ† Edit repo/tools/REGISTRY.md
Index: repo/tools/REGISTRY.md
===================================================================
--- repo/tools/REGISTRY.md
+++ repo/tools/REGISTRY.md
@@ -765,8 +765,9 @@
 **Use case**: Weekly cron (see `ip-drift-cron`) to detect that our egress no longer matches DNS โ€” most commonly a WireGuard tunnel drop. Part of the agent's weekly IP drift duty per investor directive. Detection only; changing DNS is a separate, deliberate step.
 
 **Tests**:
 - `bash tests/test_ip_drift_check.sh` โ†’ **133 assertions**, hermetic: a scenario-driven IP-echo server on `127.0.0.1` (via `IPDRIFT_IP_URLS`, per-path scenarios: ok / html / empty / IPv6 / CGNAT / 500 / JSON-without-ip) plus a scenario `dig` stub on PATH (`ok`/`cname`/`nodata`/`netfail`/`sleep`) โ€” no network is touched and no real service sees our egress IP. Covers the arg/`--help` contract (`--timeout 0`/`-1`/`abc` โ†’ `2`, invalid `--domain`/`--dns-server` โ†’ `2`, stdout empty on arg errors), the JSON/human contracts and key set, **unvalidated answers never deciding** (HTML โ†’ `3` not `1`, failover to the next service, empty/500 handling), **IPv6 โ†’ `3` not `1`** while **CGNAT stays a verdict**, **`dig` answers filtered to IPv4 literals** (a CNAME target โ†’ `no_a_records`, never drift), the **`no_a_records` vs `dns_lookup_failed`** split (plus missing `dig`), `--timeout` reaching dig as `+time=N +tries=1` and bounding a hanging lookup, `--dns-server` pass-through (single call, `@server`), static read-only/docs guards and the REGISTRY contract, one guarded live case, and **6 mutations** (M1 timeout gate, M2 domain gate, M3 IP-literal gate, M4 address-family gate, M5 IPv4-only DNS filter, M6 failure classification) โ€” each precondition-asserted to occur exactly once, must RUN and give the wrong verdict, with a surgical control on the happy path.
+- `bash tests/test_dig_tools_transport.sh` โ†’ **80 assertions** (shared with `dns-verify` and `domain-availability-check`): three dig transport shapes against this tool with the public-IP leg served from `127.0.0.1` โ€” all three โ†’ exit **3**, `error_code` `dns_lookup_failed`, `drift` `null`, **no `;;` in `dns_records` and none echoed into the output**. Pre-fix the rc-9 shapes already passed (the `returncode != 0` gate plus the IPv4-literal filter were sound); the masked shape reported `no_a_records` = *"query completed with no answer"* for a query that never completed, which mutant M4 must reproduce.
 - **Pre-fix replay**: `IPDRIFT_CHECK=<pre-fix blob> IPDRIFT_LEGACY=1 bash tests/test_ip_drift_check.sh` rewrites only the three hardcoded service URL literals into the sandbox (the pre-fix tool has no hook) โ†’ **51 passed / 66 failed**; the suite **refuses to run (exit 2)** against an unhooked tool instead of contacting the real internet.
 - Live: `ip-drift-check --format json --domain gladex.de` โ†’ exit 0, `drift:false` (tunnel up); `--help` โ†’ 0.
 
 **Status**: Built โœ… (hardened 2026-09-24: validated answers + arg contract + first test suite)

> build ยท mimo-v2.6-flash-free
$ cd /data/repo && tail -3 agent-logs/PROGRESS.md | cut -c1-200; echo "=== lines: $(wc -l < agent-logs/PROGRESS.md)"; git status --porcelain
- **Staging hazard had a victim again, and it was me (third occurrence)**: at 17:33Z `git status --porcelain` returned **only** `M CHANGELOG.md` and `M agent-logs/PROGRESS.md` โ€” the two paths this r
- **Still blocked (investor-owned, unchanged)**: NEEDS-INVESTOR **ยง14 cloud admin credentials** (ONE shared set for Nextcloud + Immich โ€” blocks INBOX line 452's six accounts and the test-photo uplo
- **Next-candidate queued, not actioned**: carry items (2)โ€“(9), (11), (14), (15), **(17)** and **(18)** from the 17:05Z entry unchanged โ€” item **(19) struck โ€” ACTIONED by `[0.4.77]` this run, de
=== lines: 5411
 M CHANGELOG.md
 M tools/REGISTRY.md
 M tools/dns-verify
 M tools/domain-availability-check
 M tools/ip-drift-check
?? tests/test_dig_tools_transport.sh
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && tail -1 agent-logs/PROGRESS.md
- **Next-candidate queued, not actioned**: carry items (2)โ€“(9), (11), (14), (15), **(17)** and **(18)** from the 17:05Z entry unchanged โ€” item **(19) struck โ€” ACTIONED by `[0.4.77]` this run, delivered as a TEST rather than the lint rule it asked for**. **(20) new from this step**: the same guard only exists inside `tests/`, so it fails at *regression* time, not at *commit* time โ€” `repo-lint` already reads committed blobs for exactly this reason (its own docstring: a worktree lint is blind to the commit), so a `repo-lint` rule running the same four verdicts over `tools/system-status`'s blob would refuse the commit that introduces a second reader instead of leaving it to the next regression run; it was not bolted onto this step because `tests/test_repo_lint.sh` is 1731 lines of pinned behaviour. **(21) new from this step**: `dns-verify`, `domain-availability-check` and `ip-drift-check` also shell out to dig and **none of the three mentions `;;` diagnostics anywhere** โ€” `ip-drift-check` reads as structurally safe on inspection (`returncode != 0 โ†’ no reading`, and only lines parsing as an IPv4 literal can become a value), but the other two were **not** audited and must not inherit the reputation by association: one suite that plants the transport shape (`;; communications error โ€ฆ` + exit 9) against each of them decides whether any of them can publish a resolver complaint as a fact.
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && ./tools/repo-lint --format json 2>/dev/null | python3 -c 'import json,sys; d=json.load(sys.stdin); print("repo-lint:", {k:d.get(k) for k in ("ok","files_total","failures","changelog_headings","changelog_unique")})'; ./tools/queue-source-check; echo "queue rc=$?"; ./tools/source-sync-check 2>&1 | tail -3; echo "sync rc=$?"
repo-lint: {'ok': True, 'files_total': 188, 'failures': [], 'changelog_headings': None, 'changelog_unique': None}
queue-source-check: OK - one queue: [0.4.78] pointer-only, 111 item line(s) frozen across 36 section(s), 54 PROGRESS bullet(s)
queue rc=0
  prod examples/workflows       8 files  OK
  prod src/php                 13 files  OK
result: in sync โ€” 42 file(s) across 2 env(s) (exit 0)
sync rc=0
$ cd /data/repo && for t in test_app_version test_cli_version; do printf "%s: " "$t"; php tests/$t.php 2>&1 | tail -2 | tr '\n' ' '; echo; done; printf "test_changelog_api: "; php tests/test_changelog_api.php 2>&1 | tail -2 | tr '\n' ' '; echo; printf "test_queue_source: "; bash tests/test_queue_source.sh 2>&1 | tail -1
test_app_version:  === Results: 39 passed, 0 failed === 
test_cli_version:  === Results: 35 passed, 0 failed === 
test_changelog_api:  === Results: 86 passed, 0 failed === 
test_queue_source: ===============================
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && bash tests/test_queue_source.sh 2>&1 | grep -E "Results|passed" | tail -2; echo "rc=${PIPESTATUS[0]}"
queue-source-check suite: 107 passed, 0 failed
rc=0
> build ยท mimo-v2.6-flash-free
โ† Edit repo/agent-logs/PROGRESS.md
Index: repo/agent-logs/PROGRESS.md
===================================================================
--- repo/agent-logs/PROGRESS.md
+++ repo/agent-logs/PROGRESS.md
@@ -5408,4 +5408,20 @@
 - **Safety**: model spend **0.00** (`*-free` only), **no money moved** (`BUDGET.md` untouched: **1.50 spent / 3.50 remaining**), **zero DNS writes** (no `pdns-api.py` call), no paid API key configured, **no secret read or printed** (`/root/.pdns-token` untouched, no credential in any prompt/thread/commit), **no service restarted, no certificate touched, no promote executed, Docker stacks and both investor apps untouched**. The only live I/O: the read-only probes in the STEP-0 reply, `--help`/`--bogus`, and the dashboard's own read-only checks during regression.
 - **Staging hazard had a victim again, and it was me (third occurrence)**: at 17:33Z `git status --porcelain` returned **only** `M CHANGELOG.md` and `M agent-logs/PROGRESS.md` โ€” the two paths this run had already finished (`tests/test_system_status_dig_single_point.sh`, `tools/REGISTRY.md`) were gone. `git log -- tests/โ€ฆ tools/REGISTRY.md` named the taker: **`ccaaa56 "run 168: 2026-09-26T17:30Z"`** swept them (487 + 28 lines) with its own `git add` while the suite was still under test. Content is byte-identical to what this run verified (`git diff HEAD` on both paths is empty, i.e. worktree == HEAD), nothing was lost, and it is **recorded rather than re-committed under my own message** โ€” the same call `[0.4.76]` made when `cd81110` did it. This commit therefore stages **only its own two paths, read immediately after that check, explicitly by path, never `git add -A`**.
 - **Still blocked (investor-owned, unchanged)**: NEEDS-INVESTOR **ยง14 cloud admin credentials** (ONE shared set for Nextcloud + Immich โ€” blocks INBOX line 452's six accounts and the test-photo upload), **#57 public investor-route gating A/B/C**, **SOA MNAME** (`a.misconfigured.dns.server.invalid.`), the **mail password rotation** approval, and the two `photos.`/`cloud.` exposure questions (public https from outside this container; the separate nft DNAT of `:8080`/`:2283`).
 - **Next-candidate queued, not actioned**: carry items (2)โ€“(9), (11), (14), (15), **(17)** and **(18)** from the 17:05Z entry unchanged โ€” item **(19) struck โ€” ACTIONED by `[0.4.77]` this run, delivered as a TEST rather than the lint rule it asked for**. **(20) new from this step**: the same guard only exists inside `tests/`, so it fails at *regression* time, not at *commit* time โ€” `repo-lint` already reads committed blobs for exactly this reason (its own docstring: a worktree lint is blind to the commit), so a `repo-lint` rule running the same four verdicts over `tools/system-status`'s blob would refuse the commit that introduces a second reader instead of leaving it to the next regression run; it was not bolted onto this step because `tests/test_repo_lint.sh` is 1731 lines of pinned behaviour. **(21) new from this step**: `dns-verify`, `domain-availability-check` and `ip-drift-check` also shell out to dig and **none of the three mentions `;;` diagnostics anywhere** โ€” `ip-drift-check` reads as structurally safe on inspection (`returncode != 0 โ†’ no reading`, and only lines parsing as an IPv4 literal can become a value), but the other two were **not** audited and must not inherit the reputation by association: one suite that plants the transport shape (`;; communications error โ€ฆ` + exit 9) against each of them decides whether any of them can publish a resolver complaint as a fact.
+
+## 2026-09-26T18:05Z main-loop run โ€” STEP 0 answered FIRST (0 unread in both DBs; fresh-probe reply **dev 75 / prod 41**); queue item **(21) EXECUTED**: the dig transport-shape audit of the last three readers โ€” measured first, the invariant then enforced in all three tools, `tests/test_dig_tools_transport.sh` **80/0**, pre-fix replay **58 passed / 13 failed**, regression **50 suites / 4054 passed / 0 failed**
+
+- **STEP 0 (first action, before any product work)**: `investor_to_agent` unread = **0 dev / 0 prod**, measured on both live SQLite DBs, so there was no row to mark read and **nothing was marked read that was not mine**. INBOX.md read end to end: every entry struck `~~HANDLED~~` except **one** โ€” line 452, the six identity accounts on Nextcloud + Immich โ€” open because it is **blocked**, not missed. A reply was still written and inserted this run (**dev 75 / prod 41**) instead of leaving STEP 0 at "nothing to do", because it carries *this run's* probes rather than carried ones: `/data/shared/cloud-admin.secret` **absent**, Nextcloud `status.php` โ†’ `{"installed":false}` **v34.0.4**, Immich `/api/server/ping` โ†’ `{"res":"pong"}` (**5/5 containers up**), `https://gladex.de`/`dev`/`photos`/`cloud` all **200**, `tls-check` โ†’ **9/9 OK** (87d + 89d + 89d), units `investor-app-dev`, `investor-app-prod`, `git-daemon`, `certbot.timer`, `postfix`, `dovecot`, `docker`, `agent-loop-watchdog.timer` **all active**, budget **1.50 / 3.50**, spend **0.00**. The one ask (REPORT.md ยง14) restated with both hand-over routes, and the five investor-owned items (public https from outside the container, `:8080`/`:2283` DNAT, #57 A/B/C, SOA MNAME, mail rotation) restated as theirs. No credential invented, no account created, no password in the thread, the prompt or the commit.
+- **The measurement, taken before touching any code**: `dig +short @203.0.113.99 +time=2 +tries=1 A example.com 2>/dev/null` โ†’ `;; communications error to 203.0.113.99#53: timed out` + `;; no servers could be reached`, **rc=9**, **0 bytes on stderr**; the same call through a pipeline (`| head`) โ†’ **rc=0**, because the pipeline's status is `head`'s. Two facts, and `[0.4.73]`/`[0.4.76]` were both paid for with them. The audit question was then answerable instead of arguable โ€” and it immediately produced the finding that mattered: **all three suites claiming to cover these tools had planted the complaint on stderr** (`test_dns_verify.sh:78`, `test_domain_availability.sh:86`, `test_ip_drift_check.sh:231`, each `โ€ฆ >&2`), so `netfail โ†’ exit 4` had been proven for a shape dig never produces, and the shape it does produce was covered by nothing.
+- **The audit, tool by tool โ€” can any of them publish a resolver complaint as a fact?** *Through dig's exit code: no, all three already gated on it (that half of the hypothesis held). Through a masked one: yes in two of three.* **`dns-verify`** read its failure detail from **stderr**, so the real shape reported `dig exit 9` seven times and never said why; with the exit code masked it set `has_records: true`, published **14 records** (2 `;;` lines ร— 7 types) and **exited 0**. **`domain-availability-check`** classified through `failure_kind(code, err)` โ€” same wrong stream, so a dig **timeout** could only ever be called `network`, and its own suite's assertion `in ("timeout","network")` was written to accept either and therefore could not tell; masked, the complaint was parsed as an **NS record**, and with RDAP and whois failing (DNS the only vote) the tool answered **TAKEN, exit 3** off a resolver's complaint. **`ip-drift-check`** was sound through both the exit-code gate and the IPv4-literal filter โ€” it never published one โ€” but the masked shape returned `([], True)` โ†’ `no_a_records`, whose message is *"query completed with no answer"*: a completed answer it had not completed, in the tool whose verdict is written to the cron alert log unattended.
+- **The change (one rule, three sites)**: *a `;;` line is never a value, and its presence means the query did not complete.* `dns-verify` gained `split_dig_output()` โ€” the single place raw stdout becomes answers vs diagnostics, with `parse_dig_output()` routed through it so no future caller can turn a complaint into a record โ€” and `verify_dns()` now fails a query on `code != 0 **or** diagnostics`; `domain-availability-check` gained the same split in `check_dns()` (a complaint now marks the check **failed**, excluded from the vote, rather than reading as "no records here"); `ip-drift-check`'s `dig_query()` returns `([], False)` when a diagnostic is present. Both failure-detail paths now read **whichever stream carried the message** (stderr if written, else dig's own `;;` line, else `dig exit <code>`), which is what turns `domain-availability-check`'s DNS classification from a permanent `network` into the measured **`timeout`** โ€” and both are confined to the failure branch, so a complaint is quoted *as a failure* and never stored as an answer. Exit-code tables and verdict rules are untouched: no caller's contract moved.
+- **Tests โ€” `tests/test_dig_tools_transport.sh`, 80 assertions / 4 mutants / ~2.5s / hermetic** (scenario `dig`+`curl`+`whois` stubs on PATH plus a `127.0.0.1` IP-echo server via `IPDRIFT_IP_URLS`; no network, no live DNS, no real RDAP/whois). Three shapes ร— three tools: **`transport`** (measured: `;;` on stdout, stderr empty, rc 9), **`transport-err`** (`;;` on stderr, rc 9 โ€” so no verdict may depend on which stream the stub chose), **`masked`** (`;;` on stdout with rc 0: real dig does **not** produce this, it is the wrapper-masked shape, labelled as such in the header rather than sold as a measurement). Section 0 pins the **stub itself** to the measurement (stdout diagnostics, stderr exactly empty, rc 9) because a stub that drifted to stderr would make every "real shape" assertion vacuous while staying green. `diag_in()` returns `no-json`, never `no`, so unparsable output *fails* "no `;;` anywhere" instead of satisfying it; every mutant must produce **non-empty output** first; every precondition asserts its needle occurs **exactly once** and that the mutant is **byte-different** โ€” the `[0.4.72]` lesson applied before the first mutation, not after the first false green. Mutants: M1 drop `or diagnostics` (masked โ†’ 3, not 4), M2 stop rejecting `;;` in `dns-verify` (masked โ†’ 14 published records), M3 same in `domain-availability-check` (masked โ†’ **TAKEN, exit 3**), M4 accept a complaint-laden stdout as a completed answer in `ip-drift-check` (masked โ†’ `no_a_records`) โ€” **all four caught**.
+- **Pre-fix replay (the suite is only worth what it fails on)**: the three `HEAD` blobs through the same suite with `DIG_SHAPE_SKIP_MUTATIONS=1` โ†’ **58 passed / 13 failed**, per tool **7 / 5 / 1** = `dns-verify` (2 wrong-stream detail + 5 masked), `domain-availability-check` (5 masked), `ip-drift-check` (1 masked). The four mutation preconditions print an explicit `skip - โ€ฆ pre-fix replay, nothing to mutate` rather than being counted as defects, and the flag is opt-in: without it a missing needle still **fails**, so the suite cannot quietly stop mutating itself.
+- **Regression (authoritative)**: `./tools/regression-run --format json` at 18:05Z โ†’ **50 suites, 4054 passed, 0 failed, 0 skipped, exit 0**. Closure is arithmetic: **49 + 1 = 50** and **3974 + 80 = 4054** โ€” exactly this suite added, not one assertion moved in any other suite. The three pre-existing suites over the same tools re-ran individually *before* the regression: `test_dns_verify` **40/0**, `test_domain_availability` **140/0**, `test_ip_drift_check` **133/0** โ€” unchanged by the edits, i.e. the fixes moved no pinned expectation. `bash -n` clean; `python3 -m py_compile` clean on all three tools; `regression-run --list` discovers the new suite.
+- **Docs**: `CHANGELOG.md` gained **`[0.4.78]` parked at the bottom** like `[0.4.29]`โ€“`[0.4.77]`, with its pointer-only `### Queue` (so `queue-source-check` stays green and `## [0.4.28]` remains the entry the two version suites key off); `tools/REGISTRY.md` updated for all three tools โ€” `dns-verify`'s `errors`/`network_errors` contract rewritten to the both-streams + "never in `records`" rule, a **`dig transport contract (2026-09-26)`** paragraph added to `domain-availability-check`, `ip-drift-check`'s `dns_lookup_failed` bullet widened to the diagnostic case, and a `test_dig_tools_transport.sh` bullet added under each tool's Tests. Measured **after** those edits and **before** this entry was appended: `repo-lint` **ok, 188 files, failures []**; `queue-source-check` **OK, `[0.4.78]` pointer-only, 111 item lines frozen across 36 sections**; `source-sync-check` **in sync, 42 files / 2 envs**; `test_app_version` **39/0**, `test_cli_version` **35/0**, `test_changelog_api` **86/0**, `test_queue_source` **107/0**.
+- **Deliberate non-changes**: the three older suites' `>&2` stubs were **left as they are** โ€” they prove a contract about a shape that is also real for other tools and are pinned at 313 assertions between them; rather than rewrite them, the new suite plants **both** shapes against all three tools, so the stderr shape stays covered and the stdout shape is no longer uncovered. Nothing under `tools/system-status` was touched (`[0.4.77]`'s single-call-site guard still governs it), and no `--help` text, exit code or verdict rule changed.
+- **Safety**: model spend **0.00** (`*-free` only), **no money moved** (`BUDGET.md` untouched: **1.50 spent / 3.50 remaining**), **zero DNS writes** (no `pdns-api.py` call โ€” the one measurement asked a `203.0.113.99` TEST-NET address a single question and wrote nothing anywhere), no paid API key configured, **no secret read or printed** (`/root/.pdns-token` untouched, no credential in any prompt/thread/commit), **no service restarted, no certificate touched, no promote executed, Docker stacks and both investor apps untouched**. The only live I/O: the read-only probes in the STEP-0 reply, that one `dig` measurement, `--help` runs, and the read-only checks the live dashboard performs during regression.
+- **Staging hazard: not this time โ€” recorded because it has bitten three times**: `git status --porcelain` read immediately before staging โ†’ **exactly this run's six paths** (`M CHANGELOG.md`, `M tools/REGISTRY.md`, `M tools/dns-verify`, `M tools/domain-availability-check`, `M tools/ip-drift-check`, `?? tests/test_dig_tools_transport.sh`), no other identity's WIP present, staged **explicitly by path, never `git add -A`**.
+- **Still blocked (investor-owned, unchanged)**: NEEDS-INVESTOR **ยง14 cloud admin credentials** (ONE shared set for Nextcloud + Immich โ€” blocks INBOX line 452's six accounts and the test-photo upload), **#57 public investor-route gating A/B/C**, **SOA MNAME** (`a.misconfigured.dns.server.invalid.`), the **mail password rotation** approval, and the two `photos.`/`cloud.` exposure questions (public https from outside this container; the separate nft DNAT of `:8080`/`:2283`).
+- **Next-candidate queued, not actioned**: carry items (2)โ€“(9), (11), (14), (15), (17), (18) and **(20)** from the 17:35Z entry unchanged โ€” item **(21) struck โ€” ACTIONED by `[0.4.78]` this run** (audit + enforcement + suite; the answer to its question is in the CHANGELOG entry: not through the exit code, yes through a masked one, and nobody knew because every suite had planted the complaint on stderr). **(22) new from this step**: `[0.4.77]`'s analyzer proves dig is executed in exactly one place **inside `tools/system-status`** and nothing reads any other file โ€” this run's own suite header had to admit that a *fourth* tool shelling out to dig would be covered by nobody, so the same state machine should run over all of `tools/` (or a `repo-lint` rule listing the four known sites) and refuse a new reader that does not go through a `;;`-rejecting split; not bolted onto this step because the analyzer is deliberately scoped to one file's bounds and `tests/test_repo_lint.sh` is 1731 lines of pinned behaviour. **(23) new from this step**: `dns-verify`'s `errors[]` now carries dig's raw `;; โ€ฆ` text (deliberate, asserted to be the only place it may appear), but a machine consumer grepping `errors` for a hostname can still meet a resolver's address there โ€” a structured `error_kind` + `detail` pair, the shape `domain-availability-check` already has (`error` โˆˆ `timeout`|`network`), would keep the readable line for humans and give parsers something to key on without parsing prose.

> build ยท mimo-v2.6-flash-free
$ cd /data/repo && ./tools/repo-lint 2>&1 | tail -3; echo "lint rc=$?"; ./tools/queue-source-check; echo "queue rc=$?"; ./tools/source-sync-check 2>&1 | tail -1; echo "=== status ==="; git status --porcelain
go-compile: 45 module file(s) compile clean (1.781s)
  changelog-version: 82 changelog version heading(s), 82 unique
result: all 133 linted file(s) parse clean (exit 0)
lint rc=0
queue-source-check: OK - one queue: [0.4.78] pointer-only, 111 item line(s) frozen across 36 section(s), 55 PROGRESS bullet(s)
queue rc=0
result: in sync โ€” 42 file(s) across 2 env(s) (exit 0)
=== status ===
 M CHANGELOG.md
 M agent-logs/PROGRESS.md
 M tools/REGISTRY.md
 M tools/dns-verify
 M tools/domain-availability-check
 M tools/ip-drift-check
?? tests/test_dig_tools_transport.sh
> build ยท mimo-v2.6-flash-free
โ† Edit repo/agent-logs/PROGRESS.md
Index: repo/agent-logs/PROGRESS.md
===================================================================
--- repo/agent-logs/PROGRESS.md
+++ repo/agent-logs/PROGRESS.md
@@ -5421,7 +5421,7 @@
 - **Regression (authoritative)**: `./tools/regression-run --format json` at 18:05Z โ†’ **50 suites, 4054 passed, 0 failed, 0 skipped, exit 0**. Closure is arithmetic: **49 + 1 = 50** and **3974 + 80 = 4054** โ€” exactly this suite added, not one assertion moved in any other suite. The three pre-existing suites over the same tools re-ran individually *before* the regression: `test_dns_verify` **40/0**, `test_domain_availability` **140/0**, `test_ip_drift_check` **133/0** โ€” unchanged by the edits, i.e. the fixes moved no pinned expectation. `bash -n` clean; `python3 -m py_compile` clean on all three tools; `regression-run --list` discovers the new suite.
 - **Docs**: `CHANGELOG.md` gained **`[0.4.78]` parked at the bottom** like `[0.4.29]`โ€“`[0.4.77]`, with its pointer-only `### Queue` (so `queue-source-check` stays green and `## [0.4.28]` remains the entry the two version suites key off); `tools/REGISTRY.md` updated for all three tools โ€” `dns-verify`'s `errors`/`network_errors` contract rewritten to the both-streams + "never in `records`" rule, a **`dig transport contract (2026-09-26)`** paragraph added to `domain-availability-check`, `ip-drift-check`'s `dns_lookup_failed` bullet widened to the diagnostic case, and a `test_dig_tools_transport.sh` bullet added under each tool's Tests. Measured **after** those edits and **before** this entry was appended: `repo-lint` **ok, 188 files, failures []**; `queue-source-check` **OK, `[0.4.78]` pointer-only, 111 item lines frozen across 36 sections**; `source-sync-check` **in sync, 42 files / 2 envs**; `test_app_version` **39/0**, `test_cli_version` **35/0**, `test_changelog_api` **86/0**, `test_queue_source` **107/0**.
 - **Deliberate non-changes**: the three older suites' `>&2` stubs were **left as they are** โ€” they prove a contract about a shape that is also real for other tools and are pinned at 313 assertions between them; rather than rewrite them, the new suite plants **both** shapes against all three tools, so the stderr shape stays covered and the stdout shape is no longer uncovered. Nothing under `tools/system-status` was touched (`[0.4.77]`'s single-call-site guard still governs it), and no `--help` text, exit code or verdict rule changed.
 - **Safety**: model spend **0.00** (`*-free` only), **no money moved** (`BUDGET.md` untouched: **1.50 spent / 3.50 remaining**), **zero DNS writes** (no `pdns-api.py` call โ€” the one measurement asked a `203.0.113.99` TEST-NET address a single question and wrote nothing anywhere), no paid API key configured, **no secret read or printed** (`/root/.pdns-token` untouched, no credential in any prompt/thread/commit), **no service restarted, no certificate touched, no promote executed, Docker stacks and both investor apps untouched**. The only live I/O: the read-only probes in the STEP-0 reply, that one `dig` measurement, `--help` runs, and the read-only checks the live dashboard performs during regression.
-- **Staging hazard: not this time โ€” recorded because it has bitten three times**: `git status --porcelain` read immediately before staging โ†’ **exactly this run's six paths** (`M CHANGELOG.md`, `M tools/REGISTRY.md`, `M tools/dns-verify`, `M tools/domain-availability-check`, `M tools/ip-drift-check`, `?? tests/test_dig_tools_transport.sh`), no other identity's WIP present, staged **explicitly by path, never `git add -A`**.
+- **Staging hazard: not this time โ€” recorded because it has bitten three times**: `git status --porcelain` read immediately before staging โ†’ **exactly this run's six paths** (`M CHANGELOG.md`, `M tools/REGISTRY.md`, `M tools/dns-verify`, `M tools/domain-availability-check`, `M tools/ip-drift-check`, `?? tests/test_dig_tools_transport.sh`), no other identity's WIP present; re-read after this entry was appended โ†’ **seven paths**, the seventh being `M agent-logs/PROGRESS.md` (this entry itself). Staged **explicitly by path, never `git add -A`**.
 - **Still blocked (investor-owned, unchanged)**: NEEDS-INVESTOR **ยง14 cloud admin credentials** (ONE shared set for Nextcloud + Immich โ€” blocks INBOX line 452's six accounts and the test-photo upload), **#57 public investor-route gating A/B/C**, **SOA MNAME** (`a.misconfigured.dns.server.invalid.`), the **mail password rotation** approval, and the two `photos.`/`cloud.` exposure questions (public https from outside this container; the separate nft DNAT of `:8080`/`:2283`).
 - **Next-candidate queued, not actioned**: carry items (2)โ€“(9), (11), (14), (15), (17), (18) and **(20)** from the 17:35Z entry unchanged โ€” item **(21) struck โ€” ACTIONED by `[0.4.78]` this run** (audit + enforcement + suite; the answer to its question is in the CHANGELOG entry: not through the exit code, yes through a masked one, and nobody knew because every suite had planted the complaint on stderr). **(22) new from this step**: `[0.4.77]`'s analyzer proves dig is executed in exactly one place **inside `tools/system-status`** and nothing reads any other file โ€” this run's own suite header had to admit that a *fourth* tool shelling out to dig would be covered by nobody, so the same state machine should run over all of `tools/` (or a `repo-lint` rule listing the four known sites) and refuse a new reader that does not go through a `;;`-rejecting split; not bolted onto this step because the analyzer is deliberately scoped to one file's bounds and `tests/test_repo_lint.sh` is 1731 lines of pinned behaviour. **(23) new from this step**: `dns-verify`'s `errors[]` now carries dig's raw `;; โ€ฆ` text (deliberate, asserted to be the only place it may appear), but a machine consumer grepping `errors` for a hostname can still meet a resolver's address there โ€” a structured `error_kind` + `detail` pair, the shape `domain-availability-check` already has (`error` โˆˆ `timeout`|`network`), would keep the readable line for humans and give parsers something to key on without parsing prose.

> build ยท mimo-v2.6-flash-free
$ cd /data/repo && git config user.name; git config user.email; echo "=== remote ==="; git remote -v | head -4; echo "=== ahead/behind ==="; git status -sb | head -2
Atlas
atlas@gladex.de
=== remote ===
origin	git://git.gladex.de/gladex.git (fetch)
origin	git://git.gladex.de/gladex.git (push)
=== ahead/behind ===
## main...origin/main
 M CHANGELOG.md
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && git add CHANGELOG.md agent-logs/PROGRESS.md tools/REGISTRY.md tools/dns-verify tools/domain-availability-check tools/ip-drift-check tests/test_dig_tools_transport.sh && git status --porcelain && git commit -q -m "test/tools: [0.4.78] queue item (21) โ€” a ';;' line is never a value, and its presence means the query did not complete: dig's real transport shape measured (diagnostics on STDOUT, 0 bytes stderr, rc 9) while all three suites claiming to cover these tools planted it on STDERR; enforced in dns-verify + domain-availability-check + ip-drift-check, new tests/test_dig_tools_transport.sh 80 assertions / 4 mutants / 3 shapes x 3 tools, pre-fix replay 58/13 (7/5/1), regression 50 suites / 4054 passed / 0 failed; STEP 0 answered first (0 unread both DBs, reply dev 75 / prod 41)" && git log --oneline -1
M  CHANGELOG.md
M  agent-logs/PROGRESS.md
A  tests/test_dig_tools_transport.sh
M  tools/REGISTRY.md
M  tools/dns-verify
M  tools/domain-availability-check
M  tools/ip-drift-check
6b7aeba test/tools: [0.4.78] queue item (21) โ€” a ';;' line is never a value, and its presence means the query did not complete: dig's real transport shape measured (diagnostics on STDOUT, 0 bytes stderr, rc 9) while all three suites claiming to cover these tools planted it on STDERR; enforced in dns-verify + domain-availability-check + ip-drift-check, new tests/test_dig_tools_transport.sh 80 assertions / 4 mutants / 3 shapes x 3 tools, pre-fix replay 58/13 (7/5/1), regression 50 suites / 4054 passed / 0 failed; STEP 0 answered first (0 unread both DBs, reply dev 75 / prod 41)
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && timeout 60 git push origin main 2>&1 | tail -5; echo "push rc=$?"; git status -sb | head -1; git ls-remote origin main 2>&1 | head -2
To git://git.gladex.de/gladex.git
   9f9bcec..6b7aeba  main -> main
push rc=0
## main...origin/main
6b7aebacd359649641e64568749600c256e5e3cf	refs/heads/main
> build ยท mimo-v2.6-flash-free
$ cd /data/repo && sqlite3 /opt/startup/dev/data/messages.db "SELECT id, ts FROM messages WHERE direction='agent_to_investor' ORDER BY id DESC LIMIT 1;"; sqlite3 /opt/startup/prod/data/messages.db "SELECT id, ts FROM messages WHERE direction='agent_to_investor' ORDER BY id DESC LIMIT 1;"; ls -l /data/shared/cloud-admin.secret 2>&1 | tail -1
75|2026-09-26T17:43:25Z
41|2026-09-26T17:43:25Z
ls: cannot access '/data/shared/cloud-admin.secret': No such file or directory
$ cd /data/repo && bash tests/test_dig_tools_transport.sh 2>&1 | tail -2; git status --porcelain | head; echo "tree: $(git status --porcelain | wc -l) modified"; echo "=== unread now ==="; sqlite3 /opt/startup/dev/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; sqlite3 /opt/startup/prod/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"
=== Results: 80 passed, 0 failed ===
tree: 0 modified
=== unread now ===
0
0
> build ยท mimo-v2.6-flash-free
$ docker ps --format '{{.Names}} {{.Status}}' 2>&1 | head -10; echo "count=$(docker ps -q 2>/dev/null | wc -l)"
gladex-immich-server Up 14 hours (healthy)
gladex-immich-db Up 14 hours (healthy)
gladex-immich-ml Up 14 hours (healthy)
gladex-immich-redis Up 14 hours
gladex-nextcloud Up 14 hours
count=5
> build ยท mimo-v2.6-flash-free
Run complete. Summary:

## STEP 0 (answered first, before any product work)
- **Unread `investor_to_agent` = 0 dev / 0 prod** โ€” measured on both live DBs, so nothing was marked read that wasn't mine.
- INBOX.md read end to end: every entry struck `~~HANDLED~~` except **line 452** (six identities on Nextcloud + Immich) โ€” open because it is **blocked**, not missed.
- A fresh-probe reply was still written and inserted (**dev 75 / prod 41**) rather than leaving STEP 0 at "nothing to do": `/data/shared/cloud-admin.secret` absent ยท Nextcloud `installed:false` v34.0.4 ยท Immich `pong` ยท 5/5 containers up ยท `https://` apex/dev/photos/cloud all **200** ยท `tls-check` 9/9 (87d + 91d/89d) ยท 8 units active ยท budget **1.50 / 3.50**, spend **0.00**. ยง14 restated as the single blocking ask; the five investor-owned items restated as theirs.

## The step: queue item (21) โ€” dig transport-shape audit, executed as `[0.4.78]`
**Measurement first:** `dig โ€ฆ @203.0.113.99` โ†’ both `;;` lines on **stdout**, **0 bytes on stderr**, **rc=9** (and rc=0 through a pipeline). That immediately produced the real finding: **all three suites claiming to cover these tools planted the complaint on stderr** (`test_dns_verify.sh:78`, `test_domain_availability.sh:86`, `test_ip_drift_check.sh:231`), so the shape dig actually produces was covered by nothing.

**Audit answer** โ€” *can any of them publish a resolver complaint as a fact?* Not through the exit code (all three already gated on it); **yes through a masked one, in two of three**:
- `dns-verify`: detail read from stderr โ†’ real shape said only `dig exit 9`; masked shape published **14 records** and **exited 0**.
- `domain-availability-check`: a dig timeout could only ever be classed `network`; masked, the complaint became an **NS record** โ†’ with RDAP/whois down, **TAKEN, exit 3**.
- `ip-drift-check`: never published one (rc gate + IPv4 filter โ€” that half held), but masked โ†’ `no_a_records` = *"query completed with no answer"* for a query that never completed.

**Fix:** one rule, three sites โ€” *a `;;` line is never a value, and its presence means the query did not complete*; failure detail now read from whichever stream carried the message (failure branch only). Exit codes and verdict rules untouched.

**Proof:** `tests/test_dig_tools_transport.sh` โ†’ **80/0** (3 shapes ร— 3 tools, 4 mutants all caught, `no-json` never counts as "no", mutants must be non-empty and byte-different) ยท pre-fix replay against `HEAD` blobs โ†’ **58 passed / 13 failed** (7/5/1) ยท the three pre-existing suites **40/0, 140/0, 133/0** ยท **regression 50 suites / 4054 passed / 0 failed** (49+1, 3974+80) ยท repo-lint / queue-source / source-sync all green.

Committed **`6b7aeba`** (staged by explicit path, seven paths, no other identity's WIP) and **pushed** to `origin` (`9f9bcec..6b7aeba`), identity `Atlas <atlas@gladex.de>`. No DNS write, no service restarted, no key read, no spend; tree clean, unread still 0/0 at end of run.

**Still investor-owned:** ยง14 cloud admin credentials (blocks line 452), public https from outside the container, `:8080`/`:2283` DNAT, #57 A/B/C, SOA MNAME, mail-rotation approval. **Queued next:** (22) extend `[0.4.77]`'s analyzer to all of `tools/` so a fourth dig caller can't land uncovered; (23) structured `error_kind`+`detail` for `dns-verify`.
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

App log tail โ€” prod-8001.log (last 60 lines)
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47428 Accepted
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47428 Closing
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47442 Accepted
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47442 Closing
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47450 Accepted
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47450 Closing
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47462 Accepted
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47462 Closing
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47464 Accepted
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47464 Closing
[Sat Sep 26 21:03:18 2026] 127.0.0.1:47472 Accepted
[Sat Sep 26 21:03:19 2026] 127.0.0.1:47472 Closing
[Sat Sep 26 21:03:19 2026] 127.0.0.1:47488 Accepted
[Sat Sep 26 21:03:19 2026] 127.0.0.1:47488 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44384 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44384 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44394 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44394 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44400 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44400 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44402 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44402 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44412 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44412 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44414 Accepted
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44414 Closing
[Sat Sep 26 21:03:50 2026] 127.0.0.1:44430 Accepted
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44430 Closing
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44442 Accepted
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44442 Closing
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44452 Accepted
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44452 Closing
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44458 Accepted
[Sat Sep 26 21:03:51 2026] 127.0.0.1:44458 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48898 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48898 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48914 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48914 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48928 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48928 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48934 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48934 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48936 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48936 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48950 Accepted
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48950 Closing
[Sat Sep 26 21:04:32 2026] 127.0.0.1:48966 Accepted
[Sat Sep 26 21:04:33 2026] 127.0.0.1:48966 Closing
[Sat Sep 26 21:04:33 2026] 127.0.0.1:48978 Accepted
[Sat Sep 26 21:04:33 2026] 127.0.0.1:48978 Closing
[Sat Sep 26 21:04:33 2026] 127.0.0.1:48990 Accepted
[Sat Sep 26 21:04:34 2026] 127.0.0.1:48990 Closing
[Sat Sep 26 21:04:34 2026] 127.0.0.1:48992 Accepted
[Sat Sep 26 21:04:34 2026] 127.0.0.1:48992 Closing
[Sat Sep 26 21:08:29 2026] 127.0.0.1:45578 Accepted
[Sat Sep 26 21:08:29 2026] 127.0.0.1:45578 Closing
[Sat Sep 26 21:09:09 2026] 127.0.0.1:37496 Accepted
[Sat Sep 26 21:09:09 2026] 127.0.0.1:37496 Closing
[Sat Sep 26 21:09:37 2026] 127.0.0.1:43786 Accepted

Generated 2026-09-26 19:09:37 UTC · Gladex.de