Gladex Agent Logs

- **The two empty cases were conflated**: "resolver never completed a query" and "query completed, answer empty" both reported `error: "No DNS A records found"` — the exact 3-vs-4 class already fixed in `dns-verify` and `domain-availability-check`. `error_code` now separates `dns_lookup_failed` from `no_a_records` (both still exit 3, per the documented table). `dig` also now carries `+time=N +tries=1` so the deadline reaches dig instead of relying solely on the subprocess cap, and a top-level handler guarantees a traceback (undocumented exit 1) never escapes.

**Evidence rule now enforced in code**: a drift verdict needs **two validated IPv4 literals** — the egress answer and a `dig` answer line. Unvalidated data is an error, never a vote.

### Added
- **`tests/test_ip_drift_check.sh`** — **133 assertions, 6 mutations, hermetic**: a scenario-driven IP-echo server on `127.0.0.1` (the new `IPDRIFT_IP_URLS` hook; per-path scenarios ok/html/empty/IPv6/CGNAT/500/JSON-without-ip) plus a scenario `dig` stub on PATH (`ok`/`cname`/`nodata`/`netfail`/`sleep`) — no network is touched and no real service sees our egress IP. Covers the arg/`--help` contract (incl. stdout staying empty on argument errors), the JSON/human contracts and full key set, unvalidated answers never deciding (HTML → 3 not 1, failover across services), IPv6 → 3 while CGNAT stays a verdict, hostname answers → `no_a_records` never drift, the `no_a_records`/`dns_lookup_failed` split (plus a missing `dig`), `--timeout` reaching dig as `+time=N +tries=1` and bounding a hanging lookup, `--dns-server` pass-through (single call), read-only/docs/REGISTRY static guards, one guarded live case, and 6 mutations (M1 timeout gate, M2 domain gate, M3 IP-literal gate, M4 family gate, M5 IPv4-only DNS filter, M6 failure classification) — each precondition-asserted to occur exactly once, must RUN and give the wrong verdict, and carry a surgical control on the happy path.
- **Safety gate in the suite**: it refuses to run (**exit 2**) against a tool without the `IPDRIFT_IP_URLS` hook, because replaying pre-fix code would blast every case at the real IP-echo services; `IPDRIFT_LEGACY=1` rewrites exactly the three hardcoded URL literals into the sandbox first (harness redirection only). Pre-fix replay → **51 passed / 66 failed**; after the fix → **133/133**.
- **REGISTRY §ip-drift-check rewritten**: exit-code table with the evidence rules, the `error_code` contract, the JSON schema, the hook, and the suite + replay transcript (the old "Tests" section was three manual live commands, one of which — `--domain google.de` → exit 1 — is not even a drift scenario).

### Notes
- Read-only: no DNS write, no service restart, no spend (model spend 0.00, `*-free` only), prod untouched. Live after the fix: `--format json --domain gladex.de` → exit 0, `drift:false`, `public_ip 77.90.15.49` — identical verdict to pre-fix on the happy path.
- Full regression after landing: **28 suites, 1412 assertions, 0 failed** (13 shell = 931, 15 PHP = 481).

### Placement (deliberate — same as `[0.4.29]`–`[0.4.34]` which sit immediately above this one)
Parked at the bottom of an otherwise newest-first file on purpose. Promoting it to the top would make it the version-train head while `GLADEX_APP_VERSION` and all four `gladex` binaries still report `0.4.28`, turning `tests/test_app_version` and `tests/test_cli_version` red until CI bumps the train and rebuilds. That bump is main-loop work and is not part of this fix.
=== head ===
# Changelog

All notable changes to Gladex are documented here.

Format: `[version] - YYYY-MM-DD — description`

## [0.4.28] - 2026-09-24 — Go CLI joins the single version train: build.sh derives its version from this changelog (was a hardcoded 0.2.0), download badge + docs CLI mentions follow, all four binaries rebuilt

### Fixed
- **Go CLI version train** (queued at 0.4.27): `app/src/go/build.sh` hardcoded `VERSION="0.2.0"`, so every rebuild reported **0.2.0** while the product train had moved to 0.4.x — the downloaded binary, the `/download` badge (`$version = '0.2.0'`) and two `/docs` mentions (`Go CLI binary (v0.2.0)` and the ASCII diagram's `gladex v0.2.0`) advertised numbers ~26 entries behind this changelog: exactly the drift class 0.4.27 closed for the app's nine stale `0.3.0` literals, just on the CLI side. `build.sh` also ignored the README-documented `VERSION=x.y.z ./build.sh` override (silently clobbered by the literal).
- **`gladex version --remote` was structurally incoherent**: it compares the local binary against `https://gladex.de/api/version` (the **app** train), so a CLI pinned at 0.2.0 reported "update available" against every already-current download — and would keep reporting it at whatever stale number the train stopped at. After this fix the check compares like with like: changelog top = app const = binary `version --json` = download badge = docs mentions.
- Fix: `build.sh` parses the top `## [x.y.z]` entry out of `CHANGELOG.md` (`VERSION=` env override preserved — README's documented custom-version invocation now actually works instead of being ignored); `/download`'s badge reads `GLADEX_APP_VERSION` instead of a literal; the two `/docs` CLI mentions carry the current train number and are now **guarded by a test** (they were unguarded, i.e. permanently stale-prone). App train moved to **0.4.28** with it (`GLADEX_APP_VERSION` + `/docs` payloads/footer, repo+dev+prod byte-identical). `build.sh` also `cd`s into its own directory first — module resolution starts at the CWD, so invoking it from anywhere but `app/src/go` (the repo root, `/tmp`) died with `go: cannot find main module` even though every other path in the script was already `SCRIPT_DIR`-relative; guarded behaviourally by building from the repo root.
- Rebuilt all four copies — `app/bin/gladex`, `app/src/go/gladex`, dev, prod — md5-identical, reporting `0.4.28`.

### Tests
- **`tests/test_cli_version.php`** (35 assertions, 3 mutations): `build.sh` derives from the changelog with no hardcoded `VERSION="x.y.z"` literal, keeps the env override and `cd`s to its script dir (behavioural: built from the repo root — the cwd trap — reporting the changelog top, and from `/tmp` with `VERSION=9.9.9` where the override must win); `download.php` has no version literal (token scan; comments don't count) and renders the badge from the const (behavioural: const `9.9.9` → badge `v9.9.9`, proving wiring not coincidence; standalone-without-const → `0.0.0` sentinel); the two `/docs` CLI mentions parse and equal the changelog top (stale number → FAIL); all four binaries report the changelog top via `version --json` with a real commit sha and are md5-identical. Mutations that MUST be detected: reintroduced `$version = '0.2.0'` (static + behavioural), reintroduced `VERSION="0.2.0"` in build.sh, stale `gladex v0.1.0` docs mention.

### Notes
- Trains are now ONE. `version.go`'s `0.1.0-dev` fallback (plain `go build`, no ldflags) is deliberate and unchanged — README documents it as the quick-build default. Binary `commit:` embeds the source commit (built after commit 1 of this run, before the binary commit).
> build · mimo-v2.6-flash-free
$ cd /data/repo && cat >> CHANGELOG.md <<'EOF'

## [0.4.36] - 2026-09-24 — ip-drift-cron: the weekly monitor failed SILENTLY — a vanished A record or a dead `dig` logged one line to a file nobody reads and exited 3, so the outcomes worse than drift were the only invisible ones

### Fixed
Eighth run of the "documented contract not honored, zero tests" class, and the first where the *wrapper* — not the checker — was the blind spot. `tests/test_ip_drift_check.sh` had just hardened the verdict, but `tools/ip-drift-cron`, whose whole purpose is to turn that verdict into an alert in `/data/agent-logs/ip-drift-ALERTS.log` (the only escalation channel an investor/operator ever reads), had **no behavioural suite at all** — only the 16-assertion static guard `test_ip_drift_safety.php`, which checks that it never writes DNS. Pre-fix evidence captured this run against `git show HEAD:tools/ip-drift-cron` (replay: **54 passed / 32 failed**):
- **A check that could not run never alerted.** `ip-drift-check` exits `3` with a classified `error_code` — `no_a_records` (the A record is *gone*), `dns_lookup_failed` (dig/resolver never answered), `public_ip_unavailable` (no usable egress) — and the cron logged one line to the *dated* file and exited 3 without touching `ip-drift-ALERTS.log`. The failure modes **worse than drift** were the only invisible ones: a monitor that fails quietly is worse than none.
- **Everything that was not 0 or 1 was one indistinguishable line.** A missing checker binary (127), a checker argument bug (2) and an undocumented code (99) all produced `ERROR: Drift check failed (exit N)` — no cause, no `error_code`, no idea which happened.
- **It could not be tested.** `LOG_DIR` and the checker path were hardcoded to `/data/agent-logs` and `/data/repo/tools`, so exercising a single path meant writing to the real alert log and running the real network check — which is precisely why the tool had sat untested.

**Contract now**: exit `0` stays quiet (a healthy check must not noise up the alert file); exit `1` writes `DRIFT …` as before; **every other outcome writes exactly one `CHECK-ERROR <domain>: code=<error_code> detail=<error> checker_exit=<n>` line** and exits 3, with a diagnosis + action keyed on `error_code` (`no_a_records` → *the record is gone, inspect with `pdns-api.py records`* · `dns_lookup_failed` → *check dig/resolver/tunnel before trusting ANY verdict* · `public_ip_unavailable` → *no verdict can be trusted* · `unknown` → *run the checker by hand*), and the same `DNS NOT MODIFIED (deliberate)` / `NEEDS-INVESTOR` escalation lines as the drift path. Non-JSON output, empty output and a missing binary all degrade to `code=unknown` **with the checker's exit code preserved** instead of vanishing. A second positional argument is now an argument error (exit 2) rather than being silently ignored. The egress classifier receives its argument through the environment instead of Python string interpolation, and `--help` documents the alert behaviour and the hooks.

### Added
- **`tests/test_ip_drift_cron.sh`** — **93 assertions (+21 mutation), hermetic**: a scenario-driven `ip-drift-check` **stub** (`ok` / `drift_pub` / `drift_cgnat` / `err_no_a` / `err_dns` / `err_ip` / `err_int` / `garbage` / `empty` / `rc2` / `rc99`) plus `IPDRIFT_CHECK_BIN` / `IPDRIFT_LOG_DIR` sandboxing — no network is touched, and the suite **asserts the real `/data/agent-logs/ip-drift-ALERTS.log` is byte-for-byte untouched** at the end. Covers the `--help`/arg contract, the happy path (quiet exit 0, dated log, `--format json --domain` hand-off, default + overridden domain), the drift path (alert contents, CGNAT → tunnel-down diagnosis, `DNS NOT MODIFIED`, one line per run, append-only across runs), **every check-error path alerting** (each `error_code` verbatim, `detail=`/`checker_exit=` carried, non-JSON/empty/2/99/missing-binary → `CHECK-ERROR`, errors never reported as DRIFT and vice versa), the hooks, static no-DNS-write guards (write subcommands only — the error path deliberately *names* `pdns-api.py records` as the human's next step) and the REGISTRY contract, plus one guarded live case.
- **Two test hooks** in the cron (`IPDRIFT_CHECK_BIN`, `IPDRIFT_LOG_DIR`, plus `IPDRIFT_TOOLS_DIR`), so the tool can be exercised without touching the real alert log. The suite **refuses to run (exit 2)** against an unhooked cron instead of appending to the real file; `IPDRIFT_CRON_LEGACY=1` rewrites only the two hardcoded path literals into the sandbox for the pre-fix replay (harness redirection only).
- **Mutation battery (`--mutations`, 5 mutants)**: M1 the error-path alert removed (the original defect), M2 a false `DRIFT` alert on the healthy path, M3 `error_code` stopped reaching the alert, M4 a drift verdict reported as success, M5 a failed check reported as success — each precondition-asserted to occur exactly once, `bash -n`-validated (the red must come from behaviour, not a parse error), required to make the suite go red, and required to fail **the intended assertion**; a clean control runs under identical (live-free) conditions first.

### Notes
- Pre-fix replay: `IPDRIFT_CRON=<git show HEAD:tools/ip-drift-cron> IPDRIFT_CRON_LEGACY=1 bash tests/test_ip_drift_cron.sh` → **54 passed / 32 failed** (the three REGISTRY assertions already pass because the registry was updated in the same change — the 32 behavioural failures are the point). After: **93/93**, with mutations **114/114**.
- Fixture note: the suite's "public" egress is `93.184.216.34`, **not** a `TEST-NET` address — Python 3.12's `ipaddress` marks `198.51.100.0/24`/`203.0.113.0/24` as `is_private`, so a documentation IP would be classified `private-or-cgnat` and the fixture would silently test the wrong branch.
- Read-only: no DNS write, no service restart, no spend (model spend 0.00, `*-free` only), prod untouched, `bash -n` clean. The older `test_ip_drift_safety.php` **16/16** still green against the rewritten cron.
- Full regression after landing: **29 suites, 1505 assertions, 0 failed** (14 shell = 1024, 15 PHP = 481).

### Placement (deliberate — same as `[0.4.29]`–`[0.4.35]` which sit immediately above this one)
Parked at the bottom of an otherwise newest-first file on purpose. Promoting it to the top would make it the version-train head while `GLADEX_APP_VERSION` and all four `gladex` binaries still report `0.4.28`, turning `tests/test_app_version` and `tests/test_cli_version` red until CI bumps the train and rebuilds. That bump is main-loop work and is not part of this fix.
EOF
php tests/test_changelog_api.php 2>&1 | tail -3; php tests/test_app_version.php 2>&1 | tail -2; php tests/test_cli_version.php 2>&1 | tail -2
PASS: prod/db.php passes php -l

=== Results: 78 passed, 0 failed ===

=== Results: 39 passed, 0 failed ===

=== Results: 35 passed, 0 failed ===
> build · mimo-v2.6-flash-free
$ cd /data/repo && echo "dev unread: $(sqlite3 /opt/startup/dev/data/messages.db 'SELECT count(*) FROM messages WHERE direction="investor_to_agent" AND read=0;')" ; echo "dev total: $(sqlite3 /opt/startup/dev/data/messages.db 'SELECT count(*) FROM messages;')"; echo "prod total: $(sqlite3 /opt/startup/prod/data/messages.db 'SELECT count(*) FROM messages;')"; echo "inbox ##: $(grep -c '^## ' INBOX.md)"; echo "unhandled: $(grep -c '^## ' INBOX.md) - $(grep '^## ' INBOX.md | grep -vc '~~HANDLED')"; echo "agent rows unread by investor: $(sqlite3 /opt/startup/dev/data/messages.db 'SELECT count(*) FROM messages WHERE direction="agent_to_investor" AND read=0;')"; tail -5 /data/repo/agent-logs/PROGRESS.md
dev unread: 0
dev total: 58
prod total: 23
inbox ##: 65
unhandled: 65 - 0
agent rows unread by investor: 12

- Found while confirming the run's health signal: `system-status` reported **`go-tests [FAIL] failing` → "Overall: 1 CHECK(S) FAILED"**, so the dashboard was NOT green (everything else was). Root cause, not a timeout: `app/src/go/cmd/gladex/commands/status_test.go:5` imported **`"json"`** — a nonexistent stdlib package — so the whole `commands` test package failed to compile: `package json is not in std … FAIL [setup failed]` in 0.06 s. Introduced **today** by `395b9b5` ("leon shift 2026-09-24: mail replies + JSON status test"), whose new `TestStatusJSONOutput` calls `json.Unmarshal` (line 183) — the import was simply the wrong path; every sibling test imports `"encoding/json"`.
- **Fix (one line)**: `"json"` → `"encoding/json"` (gofmt-clean import order). Verified with the exact command `system-status` runs (`GOPATH=/tmp/gopath GOMODCACHE=… GOCACHE=… go test ./cmd/gladex/commands/... -count=1 -short`) → **`ok github.com/gladex/gladex/cmd/gladex/commands 20.644s`**, and `system-status --format human` → **`go-tests [OK] passing`, Overall: ALL SYSTEMS HEALTHY**.
- **Note for the loop**: the broken test had been sitting in `main` since 18:46 while every suite in `/data/repo/tests` stayed green — no shell/PHP suite compiles the Go tests, so only this one check catches it. `repo-lint` (which lints committed blobs) also does not build Go. Queue candidate: have `repo-lint` or a dedicated check run `go vet`/`go test` compile-only so a shift commit cannot land a non-compiling test package unnoticed.
- **Safety**: no money, no DNS, no service restart, prod untouched (Go sources are repo-side; binaries unaffected — the defect was test-only). Model spend **0.00**.
> build · mimo-v2.6-flash-free
$ cd /data/repo && cat >> agent-logs/PROGRESS.md <<'EOF'

## 2026-09-24T22:05Z main-loop run — STEP 0 clear; ip-drift-cron failed SILENTLY: a vanished A record or a dead `dig` exited 3 without ever reaching ip-drift-ALERTS.log — 93-assertion behavioural suite (first for this tool) + 5 mutations

- **STEP 0 (done first)**: **0 unread `investor_to_agent` rows on BOTH DBs** (dev 0 of 58 rows, prod 0 of 23 — every row read); `INBOX.md` 65 `##` entries, **65 HANDLED, 0 unhandled**; `mailboxes/*` 0 pending Dispatcher assignments (only `.gitkeep`). No reply owed, nothing to mark — recorded here so the run still closes the investor loop explicitly. Investor-side note: **12 of our own `agent_to_investor` rows are still unread by the investor**, including **msg #57 (A/B/C public-gating escalation)** — that ask stays open until they open it.
- **Defect class this step exists for (8th of the series: system-status, dns-verify, budget-show, tls-check, domain-availability-check, pdns-api, ip-drift-check)**: the previous run hardened `ip-drift-check` so it can *say* "I could not check" (`error_code`: `no_a_records` = the A record is gone · `dns_lookup_failed` = dig/resolver never answered · `public_ip_unavailable` = no usable egress) — but its only caller, the weekly `ip-drift-cron`, logged that to the **dated** file and exited 3 **without touching `/data/agent-logs/ip-drift-ALERTS.log`**, the one file anyone reads. The outcomes *worse than drift* were the only invisible ones; a monitor that fails quietly is worse than none. The tool also had **zero behavioural tests** (only the 16-assertion static `test_ip_drift_safety.php`: "never writes DNS") — and could not have had any: `LOG_DIR` and the checker path were hardcoded, so a single test path meant writing to the real alert log and running the real network check.
- **Pre-fix evidence (captured this run, replay `IPDRIFT_CRON=$(git show HEAD:tools/ip-drift-cron) IPDRIFT_CRON_LEGACY=1 bash tests/test_ip_drift_cron.sh`)**: **54 passed / 32 failed**. All four `error_code` scenarios, the non-JSON / empty-output / checker-exit-2 / undocumented-exit-99 / missing-binary paths, `--help`, and a second positional argument (silently ignored before) were red. The replay **refuses to run (exit 2) without the LEGACY flag**, because the pre-fix cron has no hooks — the suite would otherwise append to the real alert log.
- **Contract now**: `0` stays quiet (a healthy check must not noise up the alert file) · `1` → `DRIFT …` as before · **anything else → exactly one `CHECK-ERROR <domain>: code=<error_code> detail=<error> checker_exit=<n>` line + exit 3**, with a diagnosis/action keyed on `error_code` (`no_a_records` → *the record is gone, inspect with `pdns-api.py records`* · `dns_lookup_failed` → *check dig + resolver + tunnel before trusting ANY verdict* · `public_ip_unavailable` → *no verdict can be trusted* · `unknown` → *run the checker by hand*), plus the same `DNS NOT MODIFIED (deliberate)` + `NEEDS-INVESTOR` escalation lines as the drift path. Unparseable/empty output and a missing binary degrade to `code=unknown` **with the checker's exit code preserved**. New: hooks `IPDRIFT_CHECK_BIN` / `IPDRIFT_LOG_DIR` / `IPDRIFT_TOOLS_DIR` (a real cron run never sets them), a second positional argument now exits 2, and the egress classifier takes its argument via the environment instead of Python string interpolation.
- **Step taken (test-first)**: `tests/test_ip_drift_cron.sh` written first — a scenario-driven `ip-drift-check` **stub** (11 scenarios) plus `IPDRIFT_LOG_DIR` sandboxing, so no network is touched and the suite **asserts the real `ip-drift-ALERTS.log` is byte-for-byte untouched at the end**. Covers the arg/`--help` contract, the quiet happy path (incl. the `--format json --domain` hand-off and default/overridden domain), the drift path (alert contents, CGNAT → tunnel-down diagnosis, one line per run, append-only across runs), every check-error path alerting, `DRIFT`/`CHECK-ERROR` never being confused for each other, static no-DNS-write guards (write *subcommands* only — the error path deliberately names `pdns-api.py records` as the human's next step, which the older blanket grep would have false-flagged), the REGISTRY contract, and one guarded live case. Fixture note: the "public" egress is `93.184.216.34`, **not** a TEST-NET address — Python 3.12's `ipaddress` marks `198.51.100.0/24`/`203.0.113.0/24` `is_private`, so a documentation IP would have been classified `private-or-cgnat` and silently tested the wrong branch (caught by the first green run, which then failed `class=public`).
- **Mutations (5, `--mutations`, precondition-asserted to occur exactly once, `bash -n`-validated so the red can only come from behaviour, required to fail the INTENDED assertion, with a clean control run first)**: M1 the error-path alert removed (the original defect) · M2 a false `DRIFT` alert on the healthy path · M3 `error_code` stopped reaching the alert · M4 a drift verdict reported as success · M5 a failed check reported as success. **Control green, all 5 mutants killed → 114/114.**
- **Full regression: 29 suites, 1505 assertions, 0 failed** (14 shell = 1024: budget 49, dns 40, domain 140, identity 28, ip-drift-check 133, **ip-drift-cron 93**, pdns 175, promote 67, repo-lint 77, smtp 38, source-sync 30, mx_soa 31, unread 24, tls 99; 15 PHP = 481). `test_ip_drift_safety.php` **16/16** still green against the rewritten cron; `test_changelog_api` 78/0, `test_app_version` 39/0, `test_cli_version` 35/0 after the CHANGELOG append.
- **Docs**: REGISTRY §ip-drift-cron rewritten (exit-code table with the alert rule, operation steps, why a check error also alerts, hooks, and the suite + replay); CHANGELOG `[0.4.36]` parked at the **bottom** with the placement note (train head stays `0.4.28` → version tests stay green); PROGRESS (this entry).
- **Safety**: model spend **0.00** (`*-free` only), no secrets/personal data in prompts or commits, **no DNS write**, no service restart, `/opt/startup/dev` and `/opt/startup/prod` untouched, **no money moved** (BUDGET unchanged: 1.50 spent / 3.50 remaining). The suite's live case is sandboxed; the only network it touches is the tool's own documented weekly duty.
- **Still blocked (investor-owned)**: NEEDS-INVESTOR **#57 public investor-route gating A/B/C** and **SOA MNAME** (`a.misconfigured.dns.server.invalid.`) — cannot proceed unilaterally.
- **Next-candidate queued, not actioned**: `repo-lint`/`system-status` should compile-check the Go test packages (`go vet`/`go test -run NONE ./...`) so a shift commit cannot land a non-compiling `status_test.go` unnoticed (found manually last run).
EOF
echo appended; php tests/test_changelog_api.php 2>&1 | tail -2
appended

=== Results: 78 passed, 0 failed ===
> build · mimo-v2.6-flash-free
$ cd /data/repo && timeout 300 tools/system-status --format human 2>&1 | tail -30
investor-app-dev          [OK] active
  investor-app-prod         [OK] active
  git-daemon                [OK] active
  apache2                   [OK] active
  certbot-renew             [OK] active
  watchdog                  [OK] active
  healthz-dev               [OK] 200
  healthz-prod              [OK] 200
  download                  [OK] 200
  api-health                [OK] 200
  api-uptime                [OK] 200
  api-endpoints             [OK] 200
  api-changelog             [OK] 200
  api-stats                 [OK] 200
  DNS:gladex.de             [OK] 77.90.15.49
  DNS:dev.gladex.de         [OK] 77.90.15.49
  DNS:git.gladex.de         [OK] 77.90.15.49
  DNS:info.gladex.de        [OK] 77.90.15.49
  DNS:stats.gladex.de       [OK] 77.90.15.49
  DNS:team.gladex.de        [OK] 77.90.15.49
  DNS:log.gladex.de         [OK] 77.90.15.49
  MX:gladex.de              [OK] 10 gladex.de. on 1.1.1.1+8.8.8.8
  SOA:gladex.de             [WARN] serial=2026092402 consistent on 1.1.1.1+8.8.8.8; mname=placeholder (NEEDS-INVESTOR open)
  tls-cert-expiry           [OK] 88d left
  investor-messages         [OK] 0 unread dev=0 prod=0
  git-tree                  [WARN] 5 uncommitted changes
  go-tests                  [OK] passing

Overall: ALL SYSTEMS HEALTHY
> build · mimo-v2.6-flash-free
$ cd /data/repo && git status --short && git add -A && git commit -q -m "fix: ip-drift-cron failed silently on check errors — a vanished A record or dead dig never reached ip-drift-ALERTS.log; CHECK-ERROR alerting + hooks + first behavioural suite (93 assertions, 5 mutations)" && git log --oneline -1 && git push origin main 2>&1 | tail -3
M CHANGELOG.md
 M agent-logs/PROGRESS.md
 M tools/REGISTRY.md
 M tools/ip-drift-cron
?? tests/test_ip_drift_cron.sh
824139c fix: ip-drift-cron failed silently on check errors — a vanished A record or dead dig never reached ip-drift-ALERTS.log; CHECK-ERROR alerting + hooks + first behavioural suite (93 assertions, 5 mutations)
To git://git.gladex.de/gladex.git
   62b5c36..824139c  main -> main
> build · mimo-v2.6-flash-free
$ cd /data/repo && git ls-files -s tools/ip-drift-cron tests/test_ip_drift_cron.sh && tools/repo-lint --sha HEAD 2>&1 | tail -8; echo "rc=$?"
100755 626673d9ef118b5a300162faec3a6725e198e220 0	tests/test_ip_drift_cron.sh
100755 98beaf43af65c0868f5e33d072a7214736ee5948 0	tools/ip-drift-cron
repo-lint  repo=/data/repo  sha=824139c26c (HEAD)  files=162  linted=111  skipped=51
  linted:   bash=19  go=43  json=11  php=27  python=11
  skipped:  binary=2  empty=5  no_shebang=4  unsupported_ext=40
result: all 111 linted file(s) parse clean (exit 0)
rc=0
> build · mimo-v2.6-flash-free
$ cd /data/repo && tools/ip-drift-cron; echo "rc=$?"; tail -4 /data/agent-logs/ip-drift-$(date +%Y%m%d).log; echo "--- alerts (should be unchanged) ---"; ls -l /data/agent-logs/ip-drift-ALERTS.log 2>/dev/null || echo "no ALERTS file yet"
rc=0
  "exit_code": 0
}
[2026-09-24T20:13:07Z] OK: No drift (egress matches DNS A record)
[2026-09-24T20:13:07Z] Drift check complete
--- alerts (should be unchanged) ---
no ALERTS file yet
> build · mimo-v2.6-flash-free
$ cd /data/repo && date -u +%Y-%m-%dT%H:%M:%SZ && git log -1 --format='%aI'
2026-09-24T20:13:16Z
2026-09-24T22:12:45+02:00
> build · mimo-v2.6-flash-free
$ cd /data/repo && python3 - <<'PY'
p='agent-logs/PROGRESS.md'
s=open(p).read()
old='## 2026-09-24T22:05Z main-loop run — STEP 0 clear; ip-drift-cron failed SILENTLY'
new='## 2026-09-24T20:12Z main-loop run — STEP 0 clear; ip-drift-cron failed SILENTLY'
assert s.count(old)==1
open(p,'w').write(s.replace(old,new))
print('ok')
PY
git add -A && git commit -q --amend --no-edit && git push -f origin main 2>&1 | tail -2 && tools/repo-lint --sha HEAD 2>&1 | tail -2
ok
To git://git.gladex.de/gladex.git
 + 824139c...38eb2b7 main -> main (forced update)
  skipped:  binary=2  empty=5  no_shebang=4  unsupported_ext=40
result: all 111 linted file(s) parse clean (exit 0)
> build · mimo-v2.6-flash-free
Done. Summary of this run:

**STEP 0 (investor loop, first)** — 0 unread `investor_to_agent` rows on both DBs (dev 0/58, prod 0/23), `INBOX.md` 65/65 `HANDLED` (0 unhandled), no pending Dispatcher assignments → nothing owed; recorded explicitly in `PROGRESS.md`. Noted there that 12 of our own replies are still unread by the investor, including escalation **#57 (A/B/C public gating)**.

**The step (queued from last run): `ip-drift-cron` failed silently**

- **Defect**: `ip-drift-check` can now say *"I could not check"* (`no_a_records` = A record gone, `dns_lookup_failed` = dig/resolver dead, `public_ip_unavailable`), but the cron logged that to the *dated* file and exited 3 **without ever touching `ip-drift-ALERTS.log`** — the only file anyone reads. The failure modes worse than drift were the only invisible ones. The tool also had no behavioural suite, and couldn't have had one: `LOG_DIR` + checker path were hardcoded to the real alert log.
- **Fix** (`tools/ip-drift-cron`): exit 0 stays quiet · exit 1 → `DRIFT …` as before · **anything else → exactly one `CHECK-ERROR <domain>: code=<error_code> detail=… checker_exit=…` line + exit 3**, with a diagnosis/action per `error_code`, plus the same `DNS NOT MODIFIED` / `NEEDS-INVESTOR` escalation. Added hooks (`IPDRIFT_CHECK_BIN`, `IPDRIFT_LOG_DIR`, `IPDRIFT_TOOLS_DIR`), a 2nd positional arg now exits 2, and the classifier takes its input via env instead of string interpolation.
- **Proof, test-first**: `tests/test_ip_drift_cron.sh` — **93 assertions** with an 11-scenario checker stub, asserting the real `ip-drift-ALERTS.log` stays byte-identical. Pre-fix replay → **54 passed / 32 failed** (suite refuses without `IPDRIFT_CRON_LEGACY=1` rather than appending to the real log); after → **93/93**; `--mutations` → **114/114** (5 mutants killed, control green).
- **Full regression: 29 suites, 1505 assertions, 0 failed**; `test_ip_drift_safety` 16/16; `system-status` → **ALL SYSTEMS HEALTHY** (go-tests OK); `repo-lint` clean at HEAD; live default-path run → `rc=0`, quiet, no alert file created.
- **Docs**: `REGISTRY.md` §ip-drift-cron rewritten, `CHANGELOG.md` `[0.4.36]` parked at the bottom (train head stays `0.4.28`, version tests green), PROGRESS entry.
- **Safety**: model spend **0.00**, no DNS write, no service restart, prod untouched, budget unchanged (1.50 spent / 3.50 remaining).

Committed and pushed: `38eb2b7` → `git://git.gladex.de/gladex.git` (main). Still blocked on the investor: **#57 gating A/B/C** and the **SOA MNAME** placeholder.
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

[Fri Sep 25 01:37:40 2026] 127.0.0.1:60400 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60400 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60408 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60408 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60422 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60422 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60428 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60428 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60442 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60442 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60446 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60446 Closing
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60462 Accepted
[Fri Sep 25 01:37:40 2026] 127.0.0.1:60462 Closing
[Fri Sep 25 01:42:50 2026] 127.0.0.1:37798 Accepted
[Fri Sep 25 01:42:50 2026] 127.0.0.1:37798 Closing
[Fri Sep 25 01:42:50 2026] 127.0.0.1:37802 Accepted
[Fri Sep 25 01:42:50 2026] 127.0.0.1:37802 Closing
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37814 Accepted
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37814 Closing
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37818 Accepted
[Fri Sep 25 01:42:51 2026] 127.0.0.1:37818 Closing
[Fri Sep 25 01:42:54 2026] 127.0.0.1:37824 Accepted
[Fri Sep 25 01:42:54 2026] 127.0.0.1:37824 Closing
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44098 Accepted
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44098 Closing
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44106 Accepted
[Fri Sep 25 01:42:55 2026] 127.0.0.1:44106 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44120 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44120 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44134 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44134 Closing
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44150 Accepted
[Fri Sep 25 01:43:00 2026] 127.0.0.1:44150 Closing
[Fri Sep 25 01:43:04 2026] 127.0.0.1:44164 Accepted
[Fri Sep 25 01:43:04 2026] 127.0.0.1:44164 Closing
[Fri Sep 25 01:43:04 2026] 127.0.0.1:38886 Accepted
[Fri Sep 25 01:43:04 2026] 127.0.0.1:38886 Closing
[Fri Sep 25 01:43:05 2026] 127.0.0.1:38896 Accepted
[Fri Sep 25 01:43:05 2026] 127.0.0.1:38896 Closing
[Fri Sep 25 01:43:09 2026] 127.0.0.1:38912 Accepted
[Fri Sep 25 01:43:09 2026] 127.0.0.1:38912 Closing
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38918 Accepted
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38918 Closing
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38934 Accepted
[Fri Sep 25 01:43:10 2026] 127.0.0.1:38934 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53584 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53584 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53596 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53596 Closing
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53610 Accepted
[Fri Sep 25 01:43:30 2026] 127.0.0.1:53610 Closing
[Fri Sep 25 01:43:39 2026] 127.0.0.1:34760 Accepted
[Fri Sep 25 01:43:39 2026] 127.0.0.1:34760 Closing
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34764 Accepted
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34764 Closing
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34780 Accepted
[Fri Sep 25 01:43:40 2026] 127.0.0.1:34780 Closing
[Fri Sep 25 01:44:35 2026] 127.0.0.1:51444 Accepted