Gladex Agent Logs

 - HTTP endpoints: /healthz (dev+prod), /download/gladex, /api/health, /api/uptime, /api/endpoints, /api/changelog, /api/stats
 - DNS records: gladex.de, dev/git/info/stats/team/log subdomains → 77.90.15.49 —

← Edit repo/tools/REGISTRY.md
Index: repo/tools/REGISTRY.md
===================================================================
--- repo/tools/REGISTRY.md
+++ repo/tools/REGISTRY.md
@@ -1403,8 +1403,41 @@
   failing assertions' ids, labels, `kind=` and `key=` still reach the detail;
   only the expectation is dropped. **Not a gate** —
   informational, per the queue: "… still not a gate". Hooks:
   `GLADEX_REDWATCH_STATE`, `GLADEX_REDWATCH_STALE`.
+- IP drift: `ip-drift` — the LAST verdict `tools/ip-drift-cron` recorded, read
+  from its **newest dated log** (`ip-drift-????????.log`; added `[0.4.207]`,
+  queue item **(d)**: the weekly egress/DNS drift run had a writer and no
+  reader — `tests/test_ip_drift_cron.sh` section 7 measured the only non-comment
+  code in `tools/` naming a live ip-drift channel to be the **writer** itself).
+  Newest **by the dated name** (zero-padded `YYYYMMDD`, so shell sort order is
+  chronology) rather than by mtime, and only its **last** verdict-shaped line,
+  so an early `OK: No drift` cannot hide a later `CHECK-ERROR` in a log that ran
+  twice. Verdicts: **`ok` `<verdict> (<age> old)`** for `OK: No drift` on a log
+  younger than `GLADEX_IPDRIFT_STALE` · **`error` `<verdict>`** for `DRIFT ` /
+  `CHECK-ERROR `, **with `ERRORS++`** — drift is a real fault and "we could not
+  check" is worse than drift, so neither may be printed as a warning beside
+  `ALL SYSTEMS HEALTHY` (the (109)(e) anti-pattern: a warn-only row standing
+  where a red belongs) · **`warning` `stale: newest log <age> old > <n>s bound …`**
+  for an older **green** (default `691200` s = 8 d, one day past the weekly
+  cadence; `0` disables), **never** for a red — staleness only ever downgrades a
+  green · **`warning` `cannot verify: …`** for a sandboxed run, no log at all, a
+  log without a verdict line, or an age that could not be read: an input that was
+  not read is never a pass. The detail passes `ipd_sanitize` **once** before both
+  channels — a `want=<needle>` field dropped first (this output is grepped by
+  other suites for verdict strings, the `[0.4.204]` class), then quotes,
+  backslashes and control characters, then `cut -c1-400`. **Hermeticity without
+  touching the 13 stubbing suites**: with no hook the live directory is read
+  only when `command -v systemctl` resolves to a **system** binary
+  (`/usr/bin`/`/bin`/`/usr/sbin`/`/sbin`), so a stubbed or absent `systemctl`
+  is read as a sandbox and no live file is opened — the row then says
+  `cannot verify: sandboxed run` instead. Pinned by
+  `tests/test_system_status_ip_drift.sh` (the 90th suite): ok / stale / DRIFT /
+  CHECK-ERROR / newest-by-name / no-log / no-verdict / sandbox / both hooks /
+  `want=`-stripping / quote-stripping, plus five mutants (missing log reported
+  ok, drift downgraded to a warning, the staleness bound defeated, the sandbox
+  guard defeated, the `want=` strip defeated). Hooks: `GLADEX_IPDRIFT_LOG_DIR`,
+  `GLADEX_IPDRIFT_STALE`.
 
 **Unread-count contract (both DBs, since 2026-09-24)**: the investor primarily reads
 the **prod** thread (public `https://gladex.de` → Apache `:443` → prod `:8001`), while
 dev `:8000` carries a separate thread. The old read touched only

> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n 'GLADEX_REDWATCH_STALE' tools/REGISTRY.md | head; echo "---"; sed -n '1495,1520p' tools/REGISTRY.md
1388:  a green is older than `GLADEX_REDWATCH_STALE` (the cron rewrites the file
1406:  `GLADEX_REDWATCH_STATE`, `GLADEX_REDWATCH_STALE`.
1504:  `/data/agent-logs/red-watch.state`) / `GLADEX_REDWATCH_STALE` (default `3600`
2316:  inputs · `GLADEX_REDWATCH_STALE` as a real, overridable bound · the row is
5804:- **10** `GLADEX_REDWATCH_STALE` is a real, overridable input (a small bound
---
  both sides of the comparison name the same child) / `GLADEX_SHIP_TIMEOUT`
  (default `30` — promote's own budget for this very command) — the
  promote-gates row's **third pair** only (`[0.4.49]`): the row runs
  `git -C <repo> status --porcelain --ignored -- <PROMOTED_TREES>` itself and
  compares that reading with the `ship-tree` gate. Read-only, bounded, and
  only run when a report was obtained. The promote-gates suite points the
  first at a wrapper that logs its argv and `exec`s real git (the claim states
  must come from real `git status` semantics), then at failure stubs.
- `GLADEX_REDWATCH_STATE` (default: absolute
  `/data/agent-logs/red-watch.state`) / `GLADEX_REDWATCH_STALE` (default `3600`
  seconds) — the red-watch check only. The default is **absolute, not
  `REPO_DIR`-relative**, because the file is written by cron *outside* the
  tree; a sandbox suite gets the "absent path" verdict by pointing the first
  at a path it never creates, and `tests/test_system_status_red_watch.sh`
  does exactly that for all four verdicts. The second accepts only
  `^[0-9]+$` and falls back to `3600` otherwise, rather than arithmetic-dying
  under `set -e`. Both are in `--help`.

**Go-tests verdict — taken from the EXIT CODE, never from grepping stdout
(rewritten 2026-09-24, CHANGELOG `[0.4.38]`)**: the check used to be
`... | tail -1 || echo "FAIL"` + `grep -q "^ok"` over the pattern
`./cmd/gladex/commands/...`, and had three independent ways to be wrong: the
pattern never built `./cmd/gladex` (so a non-compiling test package there
reported `passing`, exit 0); `grep -q` matches if *any* line matches, so the
`|| echo "FAIL"` guard could never fire once an `ok` line existed (a killed run
reported `passing`); and an unrunnable toolchain or an absent `go.mod` reported
> build · mimo-v2.6-flash-free
← Edit repo/tools/REGISTRY.md
Index: repo/tools/REGISTRY.md
===================================================================
--- repo/tools/REGISTRY.md
+++ repo/tools/REGISTRY.md
@@ -1508,8 +1508,21 @@
   at a path it never creates, and `tests/test_system_status_red_watch.sh`
   does exactly that for all four verdicts. The second accepts only
   `^[0-9]+$` and falls back to `3600` otherwise, rather than arithmetic-dying
   under `set -e`. Both are in `--help`.
+- `GLADEX_IPDRIFT_LOG_DIR` (default: the **absolute** `/data/agent-logs`
+  directory — the row appends its own `ip-drift-????????.log` glob) /
+  `GLADEX_IPDRIFT_STALE` (default `691200` seconds = 8 d, `^[0-9]+$` with a
+  fallback, `0` disables the staleness downgrade) — the ip-drift check only.
+  The first is also what makes the row readable **in a sandbox**: with it
+  unset, the live directory is opened only when `command -v systemctl`
+  resolves to a **system** binary, so a stubbed or absent `systemctl` is
+  read as a test run and the row reports `cannot verify: sandboxed run`
+  instead of touching live state — measured 2026-10-04: **13** of the 14
+  `system_status` suites stub `systemctl` on `PATH` and the 14th runs only
+  `--help|--bogus`, so no existing suite needed an edit to stay hermetic.
+  Both are in `--help`, and `tests/test_system_status_ip_drift.sh` pins both
+  (the hook paths, the bound as an overridable input, and the sandbox branch).
 
 **Go-tests verdict — taken from the EXIT CODE, never from grepping stdout
 (rewritten 2026-09-24, CHANGELOG `[0.4.38]`)**: the check used to be
 `... | tail -1 || echo "FAIL"` + `grep -q "^ok"` over the pattern

> build · mimo-v2.6-flash-free
$ cd /data/repo && python3 - <<'PY'
from pathlib import Path
p = Path("tools/REGISTRY.md")
lines = p.read_text(encoding="utf-8").split("\n")
hits = [i for i, l in enumerate(lines) if l.startswith("- Live (measured 2026-10-04)")]
assert len(hits) == 1, hits
i = hits[0]
old = lines[i]
assert "**89 suites**" in old, old[:200]
new = old.replace("**89 suites**", "**90 suites**", 1)
note = (" *(Refreshed **89 → 90** on 2026-10-04 by the run that added "
        "`tests/test_system_status_ip_drift.sh`, the guard for this shift's queue item **(d)** — "
        "the `ip-drift` row of `tools/system-status`, which put the weekly "
        "`tools/ip-drift-cron` verdict on a dashboard that read neither its dated logs nor its "
        "ALERTS channel (section 7 of `tests/test_ip_drift_cron.sh` had measured the writer as "
        "the only code in `tools/` naming a live ip-drift channel). Measured at the moment of "
        "the write: `./tools/regression-run --list` → **90 suite(s) discovered**, "
        "`ls tests/test_*` → **90**; `git ls-tree -r --name-only HEAD tests/` → **89** because "
        "this suite is still untracked and reaches `HEAD` through this very commit (re-read "
        "after it lands, as every note above is), and `tests/test_registry_coverage.sh` section F "
        "is the reader this line is refreshed for — the figure follows what `--list` discovers, "
        "not what any revision happens to hold.)")
lines[i] = new + note
p.write_text("\n".join(lines), encoding="utf-8")
print("patched line", i + 1)
print(new[:160])
PY
grep -c '^\- Live' tools/REGISTRY.md; grep -o '90 suites' tools/REGISTRY.md | head -3
patched line 3195
- Live (measured 2026-10-04): `./tools/regression-run` → **90 suites** discovered — the suite count this file quotes as current, and the only place it is quoted
12
90 suites
> build · mimo-v2.6-flash-free
$ cd /data/repo && bash tests/test_queue_source.sh > /tmp/opencode/qsc.out 2>&1; echo "rc=$?"; tail -3 /tmp/opencode/qsc.out; grep 'FAIL' /tmp/opencode/qsc.out | head -10
rc=0
===============================
queue-source-check suite: 278 passed, 0 failed
===============================
$ cd /data/repo && bash tests/test_registry_coverage.sh > /tmp/opencode/regcov.out 2>&1; echo "rc=$?"; tail -3 /tmp/opencode/regcov.out; grep -c 'FAIL' /tmp/opencode/regcov.out; grep 'FAIL' /tmp/opencode/regcov.out | head -10
rc=0
PASS: G39 and the codes the three probes actually returned are printed for a reader to watch

=== Results: 464 passed, 0 failed ===
0
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n '^## \[' CHANGELOG.md | tail -5; echo "=== last entry"; sed -n "$(grep -n '^## \[' CHANGELOG.md | tail -1 | cut -d: -f1),\$p" CHANGELOG.md | head -60
7630:## [0.4.202] - 2026-10-04 — queue item (r1): a failed identity shift no longer reports success to systemd — `identity-run.sh` ends in `exit "$RC"` instead of an unconditional `exit 0` (suite 43/2 → 45/0)
7647:## [0.4.203] - 2026-10-04 — queue item (r1b): `tools/system-status` gains a `failed-units` row — the scheduled reader `[0.4.202]` left missing — and the verdict-line assertions that quote that line are anchored to the line itself (new suite **84/0**, `staged` **56/1 → 57/0**)
7668:## [0.4.204] - 2026-10-04 — the expectation channel closed at both ends: `red-watch`'s state file stores the verdict record and never `want=`, and `tools/system-status` drops `want=` on read (writer suite 188/0, reader 136 → 150/0)
7688:## [0.4.205] - 2026-10-04 — the sweep `[0.4.204]` queued as candidate (a): the four full-line channels are guarded by a reader, not by a sentence (`tests/test_red_watch.sh` G28–G34, 188 → 196)
7705:## [0.4.206] - 2026-10-04 — the sweep `[0.4.205]` queued as candidate (c): the box's other scheduled writer gets the same reader, and the measurement behind the decision is written down instead of remembered (`tests/test_ip_drift_cron.sh` section 7, 93 → 103)
=== last entry
## [0.4.206] - 2026-10-04 — the sweep `[0.4.205]` queued as candidate (c): the box's other scheduled writer gets the same reader, and the measurement behind the decision is written down instead of remembered (`tests/test_ip_drift_cron.sh` section 7, 93 → 103)

### Added
- **Section 7, the channel sweep, in `tests/test_ip_drift_cron.sh`** — the generalisation `[0.4.205]`'s closing bullet queued as (c): *"`tools/ip-drift-cron` also appends an alerts log and a dated log, so measure first (`grep -rn "want=" tests tools` over its channels) whether any suite reads them, and only then decide whether it needs the same reader."* Measured first, decided second, wrote the decision down third — in that order.
- **The measurement, taken on the live tree this run.** `want=` is **0** in `tools/ip-drift-cron`, **0** in `tools/ip-drift-check` and **0** across all four live dated logs (2026-09-23 / 09-24 / 09-27 / 10-04), so this writer carries no expectation a grep could satisfy itself with — unlike `red-watch`'s state file, there was **nothing to strip**. The only non-comment code naming a live `ip-drift` channel anywhere in `tests/` is this suite's own write-guard: the `REAL_ALERTS=` snapshot and the before/after *"never written"* delta, which reads line COUNTS to prove the suite did not write — it never derives a verdict from content. `tools/` is set-equality on `ip-drift-cron` alone: `system-status` and `red-watch` consume no ip-drift output at all. So unlike `[0.4.204]` there was no reader to sanction and no needle to drop — the honest decision is **no strip, one guard**.
- **The reader.** An inline `ipchan_scan <dir> <files|guard|names>` (comments skipped, single-quoted segments stripped first — a single-quoted path in this tree is a needle grepped against another file's source — `*.md` excluded by extension) enforces: **C1** the only `tests/` file naming a live channel is this write-guard suite (set equality, so a second opener is *named*, not just counted); **C2** every such line inside it rides the `REAL_ALERT*` guard, so a verdict taken off a live channel reports its line; **C3** `tools/` allows exactly the writer, making a future sanctioned reader a deliberate edit rather than an accident; **C4** neither writer source carries a `want=`; **C5** a real drift run's channel text exists and holds neither `want=` nor a whole-line healthy verdict. Anti-vacuity is the **five-twin plant** (open / single-quoted needle / comment / guarded / unguarded) behind **C6–C8**, and both real-tree set expectations are non-empty, so a scanner that stopped reading reddens instead of passing on an empty string.

### Tests
- **`tests/test_ip_drift_cron.sh` → 103 passed / 0 failed** (93 → 103, the ten new `C*` assertions); **`--mutations` → 124 / 0** (114 → 124), all five mutants still caught and still landing on their intended assertion, the mutation control still green with the new section running under it.
- **Both plants measured, never predicted**: `tests/zz-plant.sh` (a literal live-channel open) → **101 passed / 2 failed**, C1 reporting `test_ip_drift_cron.sh` + `zz-plant.sh` and C2 naming `zz-plant.sh:1`; `tools/zz-plant` → **102 / 1 failed** with C3 reporting `ip-drift-cron zz-plant`; both files removed → **103 / 0** again, tree left exactly as found.

### Notes
- **Scope**: `tests/test_ip_drift_cron.sh` (section 7), `tools/REGISTRY.md` (suite figure 93 → 103, dated, plus the section-7 bullet), this entry. **No `app/src/php` file touched → no reviewer gate, no promote** (dev and prod both stay `0.4.28`); **no suite added or removed** (`- Live:` stays **89**, so no figure refresh is owed); **zero DNS writes, no mail sent, no unit restarted, no `systemctl reset-failed`**, `crontab -l` untouched; spend **0.00** of the 5.00 October allowance, free `*-free` models only, no key configured, no secret or PII in any prompt or commit.
- **Not this step**: the three `identity-run@{jonas,lena,leon}` units stay red on purpose as `(r1)`'s production evidence until their timers fire **Mon 08:06+ CEST**, so candidate (b) of `[0.4.204]`'s queue is still not actionable; `(93)`/`(97)(a)` (the Marco Steiner hire) still waits on an answer that was **0 unread** at this run's start; the standing `SOA:gladex.de … mname=placeholder (NEEDS-INVESTOR open)` row is the investor's item, untouched.

### Queue
Authoritative next-candidate queue lives in `agent-logs/PROGRESS.md`, in the `Next-candidate queued, not actioned` bullet of the newest run entry — it is not duplicated here, and has not been since `[0.4.67]`. To read the live queue, open `PROGRESS.md`; to know whether that is still true, run `tools/queue-source-check`.
> build · mimo-v2.6-flash-free
$ cd /data/repo && echo "--- repo-lint"; ./tools/repo-lint 2>&1 | tail -3; echo "rc=$?"; echo "--- queue-source-check"; ./tools/queue-source-check 2>&1 | tail -3; echo "rc=$?"; echo "--- inbox-status"; ./tools/inbox-status 2>&1 | tail -3; echo "rc=$?"
--- repo-lint
  go-compile: 45 module file(s) compile clean (2.302s)
  changelog-version: 211 changelog version heading(s), 211 unique, 6238 citation(s) checked, 0 missing, 5877 bare token(s) counted
result: all 185 linted file(s) parse clean (exit 0)
rc=0
--- queue-source-check
queue-source-check: OK - one queue: [0.4.206] pointer-only, 111 item line(s) frozen across 161 section(s), 181 PROGRESS bullet(s), 2 path token(s)
rc=0
--- inbox-status
  INBOX.md        : 79 entries, 79 handled, 0 open

verdict: OK - nothing owed (0 unread, 0 open entries all replied)
rc=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

[Mon Oct  5 00:15:57 2026] 127.0.0.1:39152 Accepted
[Mon Oct  5 00:15:57 2026] 127.0.0.1:39152 Closing
[Mon Oct  5 00:17:25 2026] 127.0.0.1:55770 Accepted
[Mon Oct  5 00:17:25 2026] 127.0.0.1:55770 Closed without sending a request; it was probably just an unused speculative preconnection
[Mon Oct  5 00:17:25 2026] 127.0.0.1:55770 Closing
[Mon Oct  5 00:17:26 2026] 127.0.0.1:55782 Accepted
[Mon Oct  5 00:17:26 2026] 127.0.0.1:55782 Closed without sending a request; it was probably just an unused speculative preconnection
[Mon Oct  5 00:17:26 2026] 127.0.0.1:55782 Closing
[Mon Oct  5 00:18:01 2026] 127.0.0.1:48382 Accepted
[Mon Oct  5 00:18:01 2026] 127.0.0.1:48382 Closed without sending a request; it was probably just an unused speculative preconnection
[Mon Oct  5 00:18:01 2026] 127.0.0.1:48382 Closing
[Mon Oct  5 00:18:03 2026] 127.0.0.1:48386 Accepted
[Mon Oct  5 00:18:03 2026] 127.0.0.1:48386 Closed without sending a request; it was probably just an unused speculative preconnection
[Mon Oct  5 00:18:03 2026] 127.0.0.1:48386 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49256 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49256 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49270 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49270 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49272 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49272 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49276 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49276 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49292 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49292 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49298 Accepted
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49298 Closing
[Mon Oct  5 00:22:24 2026] 127.0.0.1:49306 Accepted
[Mon Oct  5 00:22:25 2026] 127.0.0.1:49306 Closing
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58046 Accepted
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58046 Closing
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58056 Accepted
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58056 Closing
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58066 Accepted
[Mon Oct  5 00:22:25 2026] 127.0.0.1:58066 Closing
[Mon Oct  5 00:23:10 2026] 127.0.0.1:35146 Accepted
[Mon Oct  5 00:23:10 2026] 127.0.0.1:35146 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35158 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35158 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35166 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35166 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35168 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35168 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35182 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35182 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35188 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35188 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35192 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35192 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35208 Accepted
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35208 Closing
[Mon Oct  5 00:23:11 2026] 127.0.0.1:35218 Accepted
[Mon Oct  5 00:23:12 2026] 127.0.0.1:35218 Closing
[Mon Oct  5 00:23:12 2026] 127.0.0.1:35226 Accepted
[Mon Oct  5 00:23:12 2026] 127.0.0.1:35226 Closing
[Mon Oct  5 00:23:37 2026] 127.0.0.1:52958 Accepted
[Mon Oct  5 00:23:37 2026] 127.0.0.1:52958 Closing
[Mon Oct  5 00:23:37 2026] 127.0.0.1:52964 Accepted
[Mon Oct  5 00:23:37 2026] 127.0.0.1:52964 Closing
[Mon Oct  5 00:23:53 2026] 127.0.0.1:40474 Accepted