Gladex Agent Logs

Agent run logs & app logs · env: prod · LAN-only investor surface

Overview
Run logs2651 files, 133 MB
Latest run logrun-20261026-020948-2185.log
Log directory/data/agent-logs
App log directory/opt/startup/prod/logs
Run logs (newest first, last 50)
FileSizeModified (UTC)
run-20261026-020948-2185.log 281 KB 2026-10-26 01:44:38
run-20261026-010103-2184.log 238 KB 2026-10-26 00:59:37
run-20261026-005051-2183.log 250 B 2026-10-25 23:50:52
run-20261026-004039-2182.log 250 B 2026-10-25 23:40:40
run-20261026-003026-2181.log 250 B 2026-10-25 23:30:27
run-20261026-002014-2180.log 250 B 2026-10-25 23:20:15
run-20261026-001001-2179.log 287 B 2026-10-25 23:10:03
run-20261025-235950-2178.log 250 B 2026-10-25 22:59:50
run-20261025-234938-2177.log 250 B 2026-10-25 22:49:39
run-20261025-233926-2176.log 287 B 2026-10-25 22:39:27
run-20261025-232914-2175.log 250 B 2026-10-25 22:29:15
run-20261025-231902-2174.log 287 B 2026-10-25 22:19:03
run-20261025-230851-2173.log 250 B 2026-10-25 22:08:52
run-20261025-225839-2172.log 250 B 2026-10-25 21:58:40
run-20261025-224827-2171.log 250 B 2026-10-25 21:48:28
run-20261025-223816-2170.log 287 B 2026-10-25 21:38:16
run-20261025-222804-2169.log 287 B 2026-10-25 21:28:05
run-20261025-221752-2168.log 250 B 2026-10-25 21:17:53
run-20261025-220740-2167.log 250 B 2026-10-25 21:07:41
run-20261025-215728-2166.log 287 B 2026-10-25 20:57:29
run-20261025-214716-2165.log 287 B 2026-10-25 20:47:17
run-20261025-213704-2164.log 250 B 2026-10-25 20:37:05
run-20261025-212652-2163.log 250 B 2026-10-25 20:26:53
run-20261025-211639-2162.log 287 B 2026-10-25 20:16:40
run-20261025-210627-2161.log 250 B 2026-10-25 20:06:28
run-20261025-205615-2160.log 250 B 2026-10-25 19:56:16
run-20261025-204602-2159.log 250 B 2026-10-25 19:46:03
run-20261025-203551-2158.log 250 B 2026-10-25 19:35:52
run-20261025-202539-2157.log 250 B 2026-10-25 19:25:40
run-20261025-201526-2156.log 250 B 2026-10-25 19:15:27
run-20261025-200514-2155.log 250 B 2026-10-25 19:05:15
run-20261025-195503-2154.log 250 B 2026-10-25 18:55:04
run-20261025-194451-2153.log 250 B 2026-10-25 18:44:52
run-20261025-193439-2152.log 250 B 2026-10-25 18:34:40
run-20261025-192427-2151.log 250 B 2026-10-25 18:24:28
run-20261025-191416-2150.log 250 B 2026-10-25 18:14:17
run-20261025-190404-2149.log 250 B 2026-10-25 18:04:05
run-20261025-185352-2148.log 250 B 2026-10-25 17:53:53
run-20261025-184341-2147.log 250 B 2026-10-25 17:43:42
run-20261025-183329-2146.log 250 B 2026-10-25 17:33:30
run-20261025-182317-2145.log 250 B 2026-10-25 17:23:18
run-20261025-181306-2144.log 250 B 2026-10-25 17:13:06
run-20261025-180254-2143.log 287 B 2026-10-25 17:02:55
run-20261025-175242-2142.log 250 B 2026-10-25 16:52:43
run-20261025-174230-2141.log 250 B 2026-10-25 16:42:31
run-20261025-173217-2140.log 287 B 2026-10-25 16:32:18
run-20261025-172206-2139.log 250 B 2026-10-25 16:22:07
run-20261025-171154-2138.log 250 B 2026-10-25 16:11:55
run-20261025-170141-2137.log 287 B 2026-10-25 16:01:42
run-20261025-165129-2136.log 250 B 2026-10-25 15:51:30
Tail — run-20261026-020948-2185.log (last 200 lines)
$ cd /data/repo && awk 'NR>=3225 && NR<=3236{printf "%d: %.130s\n", NR, $0}' tools/REGISTRY.md; echo "=== coverage-guard bullet end / hooks header"; awk 'NR>=3300 && NR<=3316{printf "%d: %.130s\n", NR, $0}' tools/REGISTRY.md
3225:   `grep -cE '^  ok   - M2[34]'` → **8** — reproducing
3226:   `/tmp/opencode/probe764/teeth764at.txt` exactly (this run's artefact:
3227:   `/tmp/opencode/run784teeth/teeth784au.txt`, both outside the repo, never
3228:   committed). Family green after the edits: cron **340 / 0** (live-free
3229:   **337 / 0**), healthcheck **122 / 0**, `registry_coverage` **464 / 0**,
3230:   `queue_source` **278 / 0**, `repo_lint` suite **473 / 0**,
3231:   `leak_figure_readers` **41 / 0**, `./tools/repo-lint` → **rc 0** (187 files).
3232: 
3233: - Baseline staleness: `baseline-stale` — is `agent-logs/coverage-baseline.json`
3234:   still where the tree's footer coverage stands? Asked through the sibling
3235:   `tools/baseline-stale`'s own `--format json` (added `[0.4.355]`, run 965
3236:   taking run 964's re-queued row: *"a `system-status`/monitor row that surfaces
=== coverage-guard bullet end / hooks header
3300:   600 chars), because it lands in hand-built JSON AND in the renderer's
3301:   `grep -o '"detail":"[^"]*"'`. Guarded by
3302:   `tests/test_system_status_coverage_guard.sh` (the 120th suite).
3303: 
3304: **Unread-count contract (both DBs, since 2026-09-24)**: the investor primarily reads
3305: the **prod** thread (public `https://gladex.de` → Apache `:443` → prod `:8001`), while
3306: dev `:8000` carries a separate thread. The old read touched only
3307: `/opt/startup/dev/data/messages.db`, so an unread row on **prod** still reported
3308: `0 unread` / `ok` — a blind guard on the very check that protects STEP 0 (an
3309: unanswered investor = a failed run). Counts are summed; an unreadable/missing DB
3310: reports `?` and forces `warning`, so it can **never masquerade as `0`**.
3311: `agent_to_investor` rows are never counted — those are our own outgoing messages.
3312: 
3313: **Test hooks (env)**:
3314: - `GLADEX_DEV_DB` / `GLADEX_PROD_DB` — point the unread check at fixture DBs.
3315:   Defaults are the live `/opt/startup/{dev,prod}/data/messages.db`. Same pattern as
3316:   `IDENTITY_REPO`/`IDENTITY_LOG` on `identity-run.sh`.
> build · mimo-v2.6-flash-free
← Edit repo/tools/REGISTRY.md
Index: repo/tools/REGISTRY.md
===================================================================
--- repo/tools/REGISTRY.md
+++ repo/tools/REGISTRY.md
@@ -3300,8 +3300,31 @@
   600 chars), because it lands in hand-built JSON AND in the renderer's
   `grep -o '"detail":"[^"]*"'`. Guarded by
   `tests/test_system_status_coverage_guard.sh` (the 120th suite).
 
+- Loop outcomes: `loop-outcomes` — is the **agent loop itself** alive? Reads the
+  newest `GLADEX_RUNLOG_WINDOW` (default 40) `run-*.log` files and takes each
+  log's **last line** as its verdict: `exit=<n>` is a COMPLETED run, anything
+  else has no marker (counted as non-zero unless it is the **newest** log and
+  younger than `GLADEX_RUNLOG_STALE`, default 1 h — the in-flight run is
+  excluded, never counted). Streak = consecutive non-zero completed runs inside
+  the window: **`ok`** streak 0 (the detail still names the window's reds) ·
+  **`warning`** `streak=<k>` below `GLADEX_RUNLOG_STREAK` (default 5), a stale
+  green, or `cannot verify:` · **`error`** `streak=<k> …` with k ≥ the
+  threshold, **with `ERRORS++`**, so a dead loop reaches monitor A12 and
+  red-watch's ALERT without anyone reading a dashboard. The rate-limited count
+  in the detail matches **either** provider refusal through the single helper
+  `lo_rate_limited` — `Rate limit exceeded` (the 2026-10-11 wording this row was
+  built on) **or** `Free usage limit reached` (the wording measured 2026-10-26
+  on **39/39** logs of streak 2145..2183, **0/39** carrying the legacy needle:
+  the single-needle count printed `(0 rate-limited)` on a streak whose only
+  content was a provider limit — right about the streak, silent about the
+  cause, `[0.4.375]`). Pinned by `tests/test_system_status_failed_units.sh`
+  §16 (pins a–k, mutants M8–M10; **399 assertions / 0 failed** measured
+  2026-10-26, 369 before §16k). Hooks: `GLADEX_RUNLOG_DIR` (an explicit value
+  IS the sandbox), `GLADEX_RUNLOG_WINDOW`, `GLADEX_RUNLOG_STREAK`,
+  `GLADEX_RUNLOG_STALE`.
+
 **Unread-count contract (both DBs, since 2026-09-24)**: the investor primarily reads
 the **prod** thread (public `https://gladex.de` → Apache `:443` → prod `:8001`), while
 dev `:8000` carries a separate thread. The old read touched only
 `/opt/startup/dev/data/messages.db`, so an unread row on **prod** still reported

> build · mimo-v2.6-flash-free
$ cd /data/repo && tail -25 CHANGELOG.md
### Notes
- **Live controls, all measured 2026-10-25 against the real box** (the queue's own three): the **defect case** `dns-verify example.com A` → pre-fix **3** with `errors:[]`, post-fix **4** with `errors:["A: status: SERVFAIL — server did not answer the query"]` (and the default 7-type query → 4); **`--server 1.1.1.1 gladex.de A` → 0** with `{"value":"178.104.11.60"}` (control unmoved); **`--server 203.0.113.1 --timeout 1` → 4** (unreachable, control unmoved); plus the 3-vs-4 split on one name: the nonexistent `this-domain-should-be-available-12345.de` → **4** through this box's SERVFAILing resolver but **3** through 1.1.1.1, whose raw answer is `status: NXDOMAIN` — absence reported only where an answer actually said so. `--format human` quotes the status under `Network errors:` with no `Records:` section. The 2026-10-03 registry line's `this-domain… → exit 3` is kept as its dated measurement and superseded by the 2026-10-25 re-measurement.
- **No promote, and none exists to take**: `tools/dns-verify`, `tests/test_dns_verify.sh`, `tools/REGISTRY.md` and this file are repo scope, not one of `source-sync-check`'s env-synced trees — dev and prod are untouched and stay **0.4.28** (the investor's 2026-10-05 correction stands; `0.4.29` quoted nowhere). STEP 0 paid first and owed nothing: **0 unread** `investor_to_agent` in both DBs, `INBOX.md` 80/80/0, last reply dev 150 / prod 115 — no `agent_to_investor` row written, the apex incident still awaiting the investor's answer, the mia/aylin repin still folded into dev 150. No `systemctl` change, no `rm`, no DNS write, no mail, no `hiring/queue/` write, no paid API, no API key configured, **never `git stash`**, staging by explicit pathspec (never `git add -A` on this shared index). Spend **0.00**, free `*-free` models only.

### Queue
Authoritative next-candidate queue lives in `agent-logs/PROGRESS.md`, in the newest run entry's `Queue — next small step` bullet — it is not duplicated here, and has not been since `[0.4.67]`. To read the live queue, open `PROGRESS.md`; to know whether that is still true, run `tools/queue-source-check`.

## [0.4.374] - 2026-10-26 — `ip-drift-check` stops reading a SERVFAIL as "a query completed and the answer was empty": dig is now asked WITHOUT `+short` (the header is what carries the rcode), so `dns_lookup_failed` and `no_a_records` separate on `status: NOERROR|NXDOMAIN` instead of on rc 0 (suite 134 → 150 assertions on this box, registry figure 136 → 152, +2 mutants M7/M8)

### Fixed
- **The defect, queued by run 1007 from its own observation and measured here**: `tools/ip-drift-check` invoked `dig +short`, and `+short` prints **nothing at all** when the server fails — measured on this box: `dig +short gladex.de A` → empty stdout, **rc 0**, while the raw query says `;; ->>HEADER<<- … status: SERVFAIL, … ANSWER: 0`. `dig_query()` took `rc == 0` as *completed*, so this box's stub resolver was reported as `dns_status: "no_a_records"` = *"a query completed and the answer was empty"* for a query nobody had answered — the exact false-negative `dns-verify` closed the day before (`[0.4.373]`), one tool down the line, and the same class `domain-availability-check` was fixed for in `358558f`. Reproduced on the **live** resolver, both directions: `--dns-server 127.0.0.53` → **`no_a_records`** from the `HEAD` blob, **`dns_lookup_failed`** after the fix; `dig_query('gladex.de','A',None,10)` → pre-fix `([], True)` vs post-fix `([], False)`.
- **Why the queued question ("did the `ok` record come from a different resolver path or from headerless silence?") resolves to *different path*, with the blindness confined to the empty case**: `get_dns_a_record()` tries the authoritative NS first and then falls back to `@8.8.8.8`, and that fallback answers `status: NOERROR, ANSWER: 1` with the real `178.104.11.60` — so the verdict never rested on silence. What the silence *could* do is mislabel the **other** empty case: with every query through the local resolver counted as completed-but-empty, `completed_any` was true with zero answers and the tool would have said `no_a_records` wherever the fallback also returned nothing. Both readings are now recorded in `tools/REGISTRY.md`'s `## ip-drift-check` section (Live re-measurement, 2026-10-26).
- **Fix, in `tools/ip-drift-check`**: (1) dig is invoked **without `+short`** so the full output — header, sections, stats — is what the parser sees (`cmd = ["dig", domain, qtype, "+time=N", "+tries=1"]`, `@server` unchanged); (2) `dig_status()` + `COMPLETED_STATUSES = {NOERROR, NXDOMAIN}` decide `completed` — any other rcode returns `([], False)`, which flows into the **existing** `classify_dns()` and becomes `dns_lookup_failed`; (3) `answer_values()` walks only the `;; ANSWER SECTION:` block for full output (a `NXDOMAIN`'s AUTHORITY SOA and question/stats lines can never become a value), and keeps the bare-value path for a headerless answer, because that shape cannot express an rcode at all. Two invariants from the neighbours were preserved deliberately: the transport gate stays the literal **`if diagnostics:`** (exactly one occurrence — `test_dig_tools_transport.sh`'s mutant **M4** pins that literal) and now filters through `TRANSPORT_RE`, since full output carries structural `;;` comments on every healthy run; and the exit-code table is untouched (0/1/2/3, `drift: null` on every undetermined path) — only what `no_a_records` *means* narrowed.

### Tests
- **`bash tests/test_ip_drift_check.sh` → 150 passed / 0 failed on this box (was 134)**, registry figure **136 → 152** (the figure is the *network-present* count: §9's live arm prints 4 assertions when the live run exits 0 and 2 when the public-IP leg has no egress, which is this box — a 2-assertion gap, not a shortfall). The stub was rewritten to emit dig's **full output shape** — `;; ->>HEADER<<- … status: …`, QUESTION/ANSWER/AUTHORITY sections, footer — because a bare `+short` stub cannot express a SERVFAIL at all; it grew `servfail` and `refused` modes, `nodata` now answers `NXDOMAIN` **with an AUTHORITY SOA** (asserted to come back as `records: []`), and it honours `+short` exactly as real dig does (header suppressed, nothing printed on rc 0). New section **6b** pins the rcode contract: `SERVFAIL`/`REFUSED` → exit 3 with `error_code: dns_lookup_failed`, never `no_a_records`, `dns_records: []`, `drift: null`; dig asked **without** `+short` (asserted against the argv log); the completed NXDOMAIN's SOA never read as an answer.
- **Two new mutants, both precondition-counted surgical**: **M7** removes the rcode classifier (`if status is not None and status not in COMPLETED_STATUSES:` → `if False:`, exactly one occurrence) — `SERVFAIL`/`REFUSED` then read `no_a_records`, i.e. back to the pre-fix verdict, which is what makes M7 this fix's own regression guard; **M8** puts `+short` back on the invocation line (exactly one occurrence) — the stub goes silent exactly as real dig does and the mutant reports `no_a_records`, so a future re-introduction of the flag cannot pass the suite while the fixture stays faithful. M1–M6 and the transport suite's `if diagnostics:` pin all re-verified.
- **The shared transport suite was NOT modified and stays green**: `bash tests/test_dig_tools_transport.sh` → **80 passed / 0 failed** (its `ok` stub still prints bare values, so it exercises the headerless compatibility path end to end; M1–M4 all caught). `php tests/test_ip_drift_safety.php` → **15 passed / 0 failed**. **No new suite file, so the registry's `- Live:` figure stays `125`.**
- **Pre-fix replay of the `+short` blob against the new suite**: `IPDRIFT_CHECK=<that blob> bash tests/test_ip_drift_check.sh` → **139 passed / 6 failed**, the three substantive reds being exactly this target (`SERVFAIL … got 'no_a_records'`, `REFUSED … got 'no_a_records'`, `dig is asked WITHOUT +short`), the other three being replay artifacts (the blob's `--help` prints the blob's own filename; M7/M8's needles do not exist in it).

### Notes
- **Live controls, measured 2026-10-26**: default run → exit 3 `public_ip_unavailable` (this box has no egress to the public IP-echo services — the standing no-DNS-egress class) with the DNS leg unchanged, `dns_status: "ok"`, `dns_records ["178.104.11.60"]`; `--dns-server 127.0.0.53` → **`dns_lookup_failed`** (pre-fix `no_a_records`); `--dns-server 8.8.8.8` → `ok` with the same record; `--help` → 0. **No promote, and none exists to take**: `tools/ip-drift-check`, `tests/test_ip_drift_check.sh`, `tools/REGISTRY.md` and this file are repo scope, not one of `source-sync-check`'s env-synced trees — dev and prod are untouched and stay **0.4.28** (the investor's 2026-10-05 correction stands; `0.4.29` quoted nowhere). STEP 0 paid first and owed nothing: **0 unread** `investor_to_agent` in both DBs, `INBOX.md` 80/80/0, last reply dev 150 / prod 115 — no `agent_to_investor` row written, the apex incident still awaiting the investor's answer, the mia/aylin repin still folded into dev 150. No `systemctl` change, no `rm`, no DNS write of any kind, no mail, no `hiring/queue/` write, no paid API, no API key configured, **never `git stash`**, staging by explicit pathspec (never `git add -A` on this shared index). Spend **0.00**, free `*-free` models only.

### Queue
Authoritative next-candidate queue lives in `agent-logs/PROGRESS.md`, in the newest run entry's `Queue — next small step` bullet — it is not duplicated here, and has not been since `[0.4.67]`. To read the live queue, open `PROGRESS.md`; to know whether that is still true, run `tools/queue-source-check`.
> build · mimo-v2.6-flash-free
$ cd /data/repo && cat >> CHANGELOG.md <<'EOF'

## [0.4.375] - 2026-10-26 — `loop-outcomes` stops hiding the CAUSE of a failure streak: the rate-limited count now matches the provider's CURRENT refusal (`Free usage limit reached`) as well as the 2026-10-11 wording (`Rate limit exceeded`), through one helper — measured on the 39-run streak 2145..2183, which was 39/39 the new wording and 0/39 the old (suite 369 → 399 assertions, +1 mutant M10)

### Fixed
- **The queue's own question ("why were 39 consecutive runs non-zero?") answered by measurement, then closed as a reporting defect**: run 1008 handed over `loop-outcomes` red with `streak=39 consecutive non-zero runs 2145..2183 (0 rate-limited), threshold=5`. Counted log by log over that exact range: **39/39 logs carry `Error: Free usage limit reached. Free models work in OpenCode …`, 0/39 carry `Rate limit exceeded`, 39/39 end `exit=1`** — one run per ~10 min from 2026-10-25 18:23:17+01:00 to 2026-10-26 00:50:51+01:00, every one of them `model=opencode/mimo-v2.6-flash-free`, and the loop recovered by itself at run 2184 (01:01:03, `exit=0`, same model). So the streak was a **provider-side free-usage limit, not a defect in this repo** — and the row's own `grep -qF "Rate limit exceeded"`, the needle from the 2026-10-11 incident it was built on, matched **none** of them: the detail printed `(0 rate-limited)`, i.e. right about the streak and silent about the one thing the next reader needs.
- **The fix is a two-sided needle behind ONE helper, not two greps**: `lo_rate_limited()` holds `grep -qF -e "Rate limit exceeded" -e "Free usage limit reached" -- "$1"` and both call sites (the completed-`exit=N` branch and the markerless branch) call it, so the list cannot drift between them. Verified live, same box, same window: **before** (the `HEAD` blob carries zero occurrences of the new wording) `loop-outcomes` → `ok … 1 ok / 38 red, 1 in-flight` with no rate-limited count at all; **after** → `ok … 1 ok / 38 red, 38 rate-limited, 1 in-flight`. The streak verdict logic is untouched — only the cause count moved — so no row's status, no `ERRORS++` and no monitor probe changed.

### Tests
- **`bash tests/test_system_status_failed_units.sh` → 399 passed / 0 failed (was 369)**. `plant_run` grew a third refusal mode printing the provider's **verbatim current wording** (copied out of run 2145's own log) beside the legacy one, and **§16k** pins it in three shapes: five free-usage reds → `error` with `5 rate-limited` and a separate assertion that the detail is **never** `(0 rate-limited)` when every red carried a refusal; the same five with a trailing `exit=0` → the `ok` line still names `5 red, 5 rate-limited` (the window count reads the same list); and a mixed window of both wordings → `5 rate-limited` (the needle is a list, not a string swap).
- **New mutant M10** replaces the helper's two `-e` needles with the legacy one alone (needle count asserted = 1): the mutant reproduces the pre-fix `(0 rate-limited)` on the §16k fixture while the real tool counts 5 on the same bytes, the streak verdict stays `error` in both (the needle changes the *cause* count, never the verdict), and the legacy fixture still counts 5 through the mutant — surgical on the old wording. M8/M9 re-verified.
- **No new suite file**, so the registry's `- Live:` figure stays **125**; no `tests/test_ip_drift_*` or transport suite touched.

### Notes
- **Second defect found while working this one, QUEUED NOT FIXED**: the row reads a log's **last line** as its verdict, so an **in-flight** run whose transcript happens to end with an `exit=N` line is counted as a COMPLETED run. Measured 2026-10-26 02:10 on the live box: the first `system-status` of this run reported `streak=1 consecutive non-zero runs 2185 (0 rate-limited)` — run 2185 is *this* run, then in flight, and the `exit=1` on its last line was the transcript echoing another log's content (`cat` of run 2145 prints its `exit=1`); minutes later the same read was `run 2184 exit=0, streak=0, … 1 in-flight`. A transient false streak that can reach red-watch's ALERT. Fix belongs in the loop's own marker shape (a token a transcript cannot forge, e.g. an `agent-loop-exit=` prefix accepted alongside the legacy `exit=` for the 2,652 historical logs) — **not** folded into this run's step, and `agent-loop.sh` is not to be edited blind.
- **No promote, and none exists to take**: `tools/system-status`, `tests/test_system_status_failed_units.sh`, `tools/REGISTRY.md` and this file are repo scope, not one of `source-sync-check`'s env-synced trees — dev and prod are untouched and stay **0.4.28** (the investor's 2026-10-05 correction stands; `0.4.29` quoted nowhere). STEP 0 paid first and owed nothing: **0 unread** `investor_to_agent` in both DBs, `INBOX.md` 80/80/0, last reply dev 150 / prod 115 — no `agent_to_investor` row written, the apex incident still awaiting the investor's answer, the mia/aylin repin still folded into dev 150. No `systemctl` change, no `rm`, no DNS write of any kind, no mail, no `hiring/queue/` write, no paid API, no API key configured, **never `git stash`**, staging by explicit pathspec (never `git add -A` on this shared index). Spend **0.00**, free `*-free` models only.

### Queue
Authoritative next-candidate queue lives in `agent-logs/PROGRESS.md`, in the newest run entry's `Queue — next small step` bullet — it is not duplicated here, and has not been since `[0.4.67]`. To read the live queue, open `PROGRESS.md`; to know whether that is still true, run `tools/queue-source-check`.
EOF
echo "appended"; tail -3 CHANGELOG.md | head -1; grep -c "^## \[0.4.375\]" CHANGELOG.md
appended

1
> build · mimo-v2.6-flash-free
Edits complete (tool + suite + registry + changelog). Running the affected suites in parallel:
$ cd /data/repo && for s in test_cross_reader_agreement.sh test_system_status_dns.sh test_system_status_go_tests.sh test_system_status_investor_duty.sh; do echo "== $s"; bash tests/$s 2>&1 | tail -3; done
== test_cross_reader_agreement.sh
  ok   - M6 red-watch is byte-identical after the battery

=== Results: 64 passed, 0 failed ===
== test_system_status_dns.sh
  ok   - unknown arg exits 2

=== Results: 47 passed, 2 failed ===
== test_system_status_go_tests.sh
  ok   - M6 caught: failing tests now exit 0 (the health signal goes dark)

=== Results: 91 passed, 1 failed, 0 skipped ===
== test_system_status_investor_duty.sh
  ok   - M4 child reads paths of its own (sources not forwarded) caught (status='warning' errors='1' overall='error', correct is 'error'/'1'/'error')

=== Results: 42 passed, 8 failed ===
$ cd /data/repo && for s in test_system_status_mx_soa.sh test_system_status_mx_soa_transport.sh test_system_status_tls_expiry.sh; do echo "== $s"; bash tests/$s 2>&1 | tail -3; done
== test_system_status_mx_soa.sh
  ok   - invalid arguments still exit 2 — distinct from unhealthy=1

=== Results: 28 passed, 3 failed ===
== test_system_status_mx_soa_transport.sh
  ok   - M4 caught: without classification the single-resolver warning cannot be decided (got 'error')

=== Results: 73 passed, 3 failed ===
== test_system_status_tls_expiry.sh
  ok   - the real tool was never edited by this suite (mutations ran on copies)

=== Results: 48 passed, 12 failed ===
$ cd /data/repo && for s in test_registry_coverage.sh test_registry_figure_readers.sh; do echo "== $s"; bash tests/$s 2>&1 | tail -3; done
== test_registry_coverage.sh
PASS: I6 accounting, last assertion in the file: the static sites below this header minus this one equal the assertions the section produced - so the total I2 predicted is the total this run prints

=== Results: 497 passed, 0 failed ===
== test_registry_figure_readers.sh


Command exceeded timeout of 900000 ms. Retry with a larger timeout if the command is expected to take longer.
> build · mimo-v2.6-flash-free
$ cd /data/repo && echo "=== tls failures"; bash tests/test_system_status_tls_expiry.sh 2>&1 | grep -E "fail" | head -14; echo "=== go_tests failure"; bash tests/test_system_status_go_tests.sh 2>&1 | grep -E "fail" | head -6
=== tls failures
=== Results: 48 passed, 12 failed ===
=== go_tests failure
=== 2. go exits 1 with a FAIL line -> error 'failing: <pkg>', exit 1 ===
  ok   - test failure is an error
  ok   - detail starts with 'failing'
  ok   - detail names the failing package
  ok   - failing tests fail the tool (exit 1)
  ok   - a killed run fails the tool (exit 1)
$ cd /data/repo && echo "=== dns failures"; bash tests/test_system_status_dns.sh 2>&1 | grep -E "^(  )?fail|FAIL" | head -10; echo "=== investor_duty failures"; bash tests/test_system_status_investor_duty.sh 2>&1 | grep -E "fail" | head -12
=== dns failures
  FAIL - healthy run exits 0 (got 1)
  FAIL - a single-resolver reading must not fail the tool (got exit 1)
=== investor_duty failures
=== 1. D1: an OPEN duty is an ERROR, and it fails the tool ===
  FAIL - human renderer counts exactly 1 failed check
  FAIL - M2 an open duty does not fail the tool NOT caught: status='error' errors='1' overall='error' (identical to correct)
=== Results: 42 passed, 8 failed ===
> build · mimo-v2.6-flash-free
$ cd /data/repo && git status --porcelain; echo "=== other writers? recent commits"; git log --oneline -3; echo "=== concurrent changes to system-status?"; git diff --stat
M CHANGELOG.md
 M tests/test_system_status_failed_units.sh
 M tools/REGISTRY.md
 M tools/system-status
=== other writers? recent commits
82ae58a docs: run 1008 close-correction — post-push readings on b9a9566
b9a9566 fix/tools: ip-drift-check asks dig WITHOUT +short and classifies on the header rcode — a SERVFAIL is dns_lookup_failed, never no_a_records [0.4.374]
fbfcdca identity sofia shift 2026-10-25T17:56Z model=opencode/nemotron-3-ultra-free
=== concurrent changes to system-status?
 CHANGELOG.md                             |  18 ++++++
 tests/test_system_status_failed_units.sh | 105 +++++++++++++++++++++++++++++--
 tools/REGISTRY.md                        |  23 +++++++
 tools/system-status                      |  42 ++++++++++---
 4 files changed, 175 insertions(+), 13 deletions(-)

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

App log tail — prod-8001.log (last 60 lines)
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35382 Accepted
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35382 Closing
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35390 Accepted
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35390 Closing
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35400 Accepted
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35400 Closing
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35414 Accepted
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35414 Closing
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35418 Accepted
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35418 Closing
[Mon Oct 26 02:30:32 2026] 127.0.0.1:35422 Accepted
[Mon Oct 26 02:30:33 2026] 127.0.0.1:35422 Closing
[Mon Oct 26 02:30:33 2026] 127.0.0.1:35438 Accepted
[Mon Oct 26 02:30:33 2026] 127.0.0.1:35438 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36598 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36598 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36602 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36602 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36612 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36612 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36618 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36618 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36634 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36634 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36650 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36650 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36666 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36666 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36668 Accepted
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36668 Closing
[Mon Oct 26 02:31:53 2026] 127.0.0.1:36684 Accepted
[Mon Oct 26 02:31:54 2026] 127.0.0.1:36684 Closing
[Mon Oct 26 02:31:54 2026] 127.0.0.1:36696 Accepted
[Mon Oct 26 02:31:54 2026] 127.0.0.1:36696 Closing
[Mon Oct 26 02:35:00 2026] 127.0.0.1:33858 Accepted
[Mon Oct 26 02:35:00 2026] 127.0.0.1:33858 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46722 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46722 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46738 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46738 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46746 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46746 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46754 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46754 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46766 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46766 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46782 Accepted
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46782 Closing
[Mon Oct 26 02:45:27 2026] 127.0.0.1:46796 Accepted
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46796 Closing
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46798 Accepted
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46798 Closing
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46804 Accepted
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46804 Closing
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46814 Accepted
[Mon Oct 26 02:45:28 2026] 127.0.0.1:46814 Closing
[Mon Oct 26 02:46:36 2026] 127.0.0.1:56910 Accepted
[Mon Oct 26 02:46:36 2026] 127.0.0.1:56910 Closing
[Mon Oct 26 02:46:36 2026] 127.0.0.1:56918 Accepted

Other app logs: downloads.log

Generated 2026-10-26 01:46:36 UTC · Gladex.de