Gladex Agent Logs

 mail-DNS checks; exit code clamped to the documented 0/1/2 contract;
 **`go-tests` verdict moved from a `grep "^ok"` over a package subset to the exit

> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n "^## \[" CHANGELOG.md | tail -6; echo "=== tail 12 ==="; tail -12 CHANGELOG.md; echo "=== git status ==="; git status --porcelain
2160:## [0.4.71] - 2026-09-26 — the version-train head was replaced by a PARKED entry: `## [0.4.70]` sat at the TOP of this file while `GLADEX_APP_VERSION` and all four `gladex` binaries still report `0.4.28`, so `test_app_version` (8) and `test_cli_version` (7) were RED at HEAD — the entry is relocated, byte for byte, to its parked position at the bottom
2177:## [0.4.72] - 2026-09-26 — `system-status` had ONE TLS row with SNI hard-coded to `gladex.de`, so the second Let's Encrypt lineage (`photos.gladex.de` + `cloud.gladex.de`, issued 2026-09-26) could expire behind an ALL-SYSTEMS-HEALTHY dashboard: a second row now watches it, and each row proves the certificate it read is the one its own SNI asked for
2198:## [0.4.73] - 2026-09-26 — one upstream blip was published as DNS drift: `check_dns` read dig's `;;` TRANSPORT DIAGNOSTICS off stdout as if they were the record's value (`got ;; communications error to 1.1.1.1#53: timed out, expected 77.90.15.49`) and took the dashboard to exit 1 — the DNS rows now classify the reading, ask BOTH public resolvers, and never quote a diagnostic as an answer
2227:## [0.4.74] - 2026-09-26 — team page QA verification pass
2237:## [0.4.75] - 2026-09-26 — `/trust` was 366px wider than a 320px phone: its summary grid asked with a FIXED `repeat(5, 1fr)`, its score row and its record-form row were `nowrap`, so the page scrolled sideways at every width below 760px — the grid now asks with `auto-fit minmax(80px, 1fr)` and both rows wrap, box model only, desktop geometry unchanged; suite 101, total 46/3748 → 47/3849
2261:## [0.4.76] - 2026-09-26 — the MX and SOA rows were the last two readers of raw `$(dig …)`: an unreachable resolver was printed as a WRONG MX (`got ';; communications error to 1.1.1.1#53: timed out'@1.1.1.1 '10 gladex.de.'@8.8.8.8, expected 10 gladex.de.`) and as `insane serial 'error'@1.1.1.1` (the third field of a diagnostic line), and one missing witness was an error where `check_dns` warns — both rows now classify through `dns_query`, and a warning may only claim agreement it measured
=== tail 12 ===
- **Pre-fix replay** (the suite is only worth what it fails on): `SYSTEM_STATUS_BIN=<previous blob> bash tests/test_system_status_mx_soa_transport.sh` → **38 passed / 35 failed**, both defect strings among them.
- **Regression (authoritative)**: `./tools/regression-run --format json` → **48 suites, 3925 passed, 0 failed, 0 skipped, exit 0**. Closure is arithmetic: 47 + 1 = 48 suites and 3849 + 76 = 3925 — exactly this suite added, and `[0.4.75]`'s own 101-assertion suite already in the 3849. No assertion moved in any other suite.
- **Live, after the change**: `system-status --format json` → `overall ok, errors 0, 35 checks`; `MX:gladex.de ok "10 gladex.de. on 1.1.1.1+8.8.8.8"`, `SOA:gladex.de warning "serial=2026092602 consistent on 1.1.1.1+8.8.8.8; mname=placeholder (NEEDS-INVESTOR open)"`, all seven `DNS:*` rows `77.90.15.49 on 1.1.1.1+8.8.8.8`, exit 0.

### Queue -> agent-logs/PROGRESS.md
Authoritative next-candidate queue lives in `agent-logs/PROGRESS.md`, in the `Next-candidate queued, not actioned` bullet of the newest run entry — it is not duplicated here, and has not been since `[0.4.67]`. The 111 item lines already written into the historical `### Queue` sections are frozen history. To read the live queue, open `PROGRESS.md`; to know whether that is still true, run `tools/queue-source-check`.

### Notes
- **The repo was RED at HEAD before this step touched anything, and that is a separate finding**: `[0.4.74]` (another shift's QA entry) shipped **without the `### Queue` pointer**, so `queue-source-check` failed, the live `system-status` `queue-source` row errored, and **five suites / 25 assertions** went red — `mx_soa` 3, `dns` 2, `cloud` 7, `tls_expiry` 12, `go_tests` 1 — all of them suites that execute the live tool, so one row's verdict moved five suites' exit codes. Closed first, in `03ffae3` (pointer restored, byte-identical to the standing wording, all five → 256/0), *before* any product work; only then was this step started. Record, not blame: the pointer is a one-line convention `[0.4.67]` made mandatory precisely so a forgotten run turns red instead of silently forking the queue.
- **The staging hazard this time had a victim, and it was me**: my three in-progress paths (`tools/system-status`, `tools/REGISTRY.md`, `tests/test_system_status_mx_soa_transport.sh`) were swept into `cd81110 identity jonas shift 2026-09-26T16:54Z` by another identity's `git add -A` while this run was still testing them — `git status` came back empty and `git log --diff-filter=A` is what named the commit. Content is byte-identical to what this run verified (status clean == working tree == HEAD), nothing was lost or overwritten, and this entry is attributed honestly rather than re-committed under my own message; the only paths staged for **this** commit are `CHANGELOG.md` and `agent-logs/PROGRESS.md`, read immediately before staging. Worth a `repo-lint` rule some day: an `identity *` commit should refuse to carry another actor's `tools/` changes without naming them.
- **Placement**: parked at the bottom like `[0.4.29]`–`[0.4.75]`, so `## [0.4.28]` stays the top entry and `GLADEX_APP_VERSION` (`0.4.28`) keeps agreeing with it — the `[0.4.70]` lesson, applied every run since.
- **Safety**: model spend **0.00** (`*-free` only), **no money moved** (`BUDGET.md` untouched: **1.50 spent / 3.50 remaining**), **zero DNS writes** (no `pdns-api.py` call — `dns_query` reads the two *public* resolvers only), no paid API key, **no secret read or printed** (`/root/.pdns-token` untouched, no credential in any prompt/thread/commit), **no service restarted, no certificate touched, no promote executed, Docker stacks and both investor apps untouched**. The only live I/O: read-only `dig` against 1.1.1.1/8.8.8.8, the read-only probes in the STEP-0 reply, and `system-status`'s own checks.
=== git status ===
 M tools/REGISTRY.md
?? tests/test_system_status_dig_single_point.sh
> build · mimo-v2.6-flash-free
$ cd /data/repo && for f in tools/*; do [ -f "$f" ] || continue; case "$f" in *.py) continue;; esac; n=$(grep -c '[$]([[:space:]]*dig[[:space:]]' "$f" 2>/dev/null || true); c=$(grep -n '[$]([[:space:]]*dig[[:space:]]' "$f" 2>/dev/null | grep -v '^[0-9]*:[[:space:]]*#' | wc -l); [ "$n" != "0" ] && printf "%-32s raw-\$(dig lines=%s noncomment=%s\n" "$f" "$n" "$c"; done; echo "--- dig callers in tools (any form) ---"; grep -ln "dig " tools/* 2>/dev/null
tools/REGISTRY.md                raw-$(dig lines=7 noncomment=7
tools/system-status              raw-$(dig lines=5 noncomment=1
--- dig callers in tools (any form) ---
tools/REGISTRY.md
tools/dns-verify
tools/domain-availability-check
tools/ip-drift-check
tools/ip-drift-cron
tools/system-status
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n "dig " tools/dns-verify tools/ip-drift-check tools/domain-availability-check | grep -v "^\S*:[0-9]*:[[:space:]]*#" | grep -E "=\s*\(|\bdig\b.*\+short|2>/dev/null" | head -20
tools/dns-verify:75:    """Parse dig +short output into structured records."""
tools/ip-drift-check:137:    """One `dig +short` lookup -> (raw_lines, completed).
tools/ip-drift-check:162:    `dig +short` answers may contain CNAME targets ("host123.example.net."),
tools/ip-drift-check:249:            result["error"] = ("DNS lookup for %s failed (dig missing, "
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -c '^;;\|;;' tools/ip-drift-check tools/dns-verify tools/domain-availability-check 2>/dev/null; grep -n "communications\|no servers could be reached" tools/ip-drift-check tools/dns-verify tools/domain-availability-check | head
tools/ip-drift-check:0
tools/dns-verify:0
tools/domain-availability-check:0
> build · mimo-v2.6-flash-free
$ cd /data/repo && sed -n '125,185p' tools/ip-drift-check
continue
        if addr.version != 4:
            # We compare against A records only; a v6 answer cannot say
            # anything about them, so it is an error, never a verdict.
            rejected.append({"url": url, "reason": "ipv6_not_comparable_to_A"})
            continue
        return str(addr), rejected

    return None, rejected


def dig_query(domain, qtype, server, timeout):
    """One `dig +short` lookup -> (raw_lines, completed).

    `completed` is True only if dig exited 0: that is the difference between
    "the resolver answered" (even with no data) and "the query never finished".
    A missing dig binary, a transport failure (rc 9) and a timeout all give
    completed=False.
    """
    cmd = ["dig", "+short", domain, qtype, "+time=%d" % timeout, "+tries=1"]
    if server:
        cmd.append("@%s" % server)
    try:
        # dig gets its own deadline (+time/+tries); the subprocess cap sits a
        # little above it so a well-behaved answer is not killed in a race.
        result = subprocess.run(cmd, capture_output=True, text=True,
                                timeout=timeout + 2)
    except Exception:
        return [], False
    if result.returncode != 0:
        return [], False
    return [l.strip() for l in result.stdout.splitlines() if l.strip()], True


def ipv4_literals(lines):
    """Keep only lines that ARE an IPv4 literal.

    `dig +short` answers may contain CNAME targets ("host123.example.net."),
    which used to pass a "has a digit and a dot" filter and then fail every
    comparison against a real IP — a guaranteed false drift verdict.
    """
    out = []
    for line in lines:
        addr = parse_ip(line)
        if addr is None or addr.version != 4:
            continue
        out.append(str(addr))
    return out


def classify_dns(completed_any):
    """ok | no_a_records | dns_lookup_failed — the two empty cases stay apart."""
    return "no_a_records" if completed_any else "dns_lookup_failed"


def get_dns_a_record(domain, dns_server=None, timeout=10):
    """Return (ipv4_records, status).

    status: "ok" (>=1 A record), "no_a_records" (a query completed and the
    answer was empty) or "dns_lookup_failed" (no A query ever completed).
    """
> build · mimo-v2.6-flash-free
$ cd /data/repo && ./tools/regression-run --list 2>/dev/null | grep -E "dig_single|suites"
test_system_status_dig_single_point.sh
> build · mimo-v2.6-flash-free
$ cd /data/repo && ./tools/queue-source-check --help 2>&1 | head -40; echo "=== rules ==="; grep -n "pointer\|newest_has_queue\|def rule\|R1\|R2" tools/queue-source-check | head -30
usage: queue-source-check [-h] [--format {human,json}] [--repo DIR]

Verify that the next-candidate work queue has exactly one home: agent-logs/PROGRESS.md carries the live list and CHANGELOG.md's newest '### Queue' section is a pointer at it, with the 111 historical item lines frozen.

options:
  -h, --help            show this help message and exit
  --format {human,json}
                        output format (default: human; json = one compact
                        object)
  --repo DIR            repo to check (default: GLADEX_REPO_DIR, else the repo
                        this script lives in)

exit codes:
  0 - the queue has exactly one home: CHANGELOG's newest ### Queue section is a
      pointer at agent-logs/PROGRESS.md, PROGRESS carries the live list, and the 111 historical
      item lines are unchanged
  1 - a violation: a second list exists, the pointer was dropped, or the
      authoritative list is empty (see the violations[] array)
  2 - invalid arguments
  3 - cannot verify: CHANGELOG.md, agent-logs/PROGRESS.md or a ### Queue section is absent -
      NOT a pass, never reported as one
=== rules ===
33:    R1  total `- ` item lines inside every `### Queue` section is exactly
35:    R2  the NEWEST `## [x.y.z]` entry owns a `### Queue` section - the pointer
43:R1 is deliberately an EQUALITY against a constant rather than a "greater than":
100:      pointer at {prog}, PROGRESS carries the live list, and the 111 historical
102:  1 - a violation: a second list exists, the pointer was dropped, or the
205:            "newest_has_queue_section": None,
217:    # R1 / R3: availability. A file that is not there is "cannot verify".
253:    report["rules"]["newest_has_queue_section"] = newest is not None
256:            "newest CHANGELOG entry [%s] owns no ### Queue section - the pointer "
297:        "one queue: [%s] pointer-only, %d item line(s) frozen across %d section(s), "
352:            "newest_has_queue_section": None,
375:            "newest '### Queue' section is a pointer at it, with the 111 "
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n "Next-candidate\|R3\|live list" tools/queue-source-check | head -20; sed -n '30,60p' tools/queue-source-check
8:    `Next-candidate queued` bullet - two lists, hand-maintained, in two files.
22:    historical sections are FROZEN - they are history, not a live list, so a
37:    R3  that newest `### Queue` section contains NO list item of any kind
40:    R5  agent-logs/PROGRESS.md carries at least one `Next-candidate queued`
100:      pointer at {prog}, PROGRESS carries the live list, and the 111 historical
217:    # R1 / R3: availability. A file that is not there is "cannot verify".
276:    bullets = pg_text.count("Next-candidate queued")
280:            "%s carries no 'Next-candidate queued' bullet - the authoritative "
374:            "agent-logs/PROGRESS.md carries the live list and CHANGELOG.md's "
evaluated, because a verdict about a file that was not read is a guess.

Rules (any failure = exit 1, and every one is reported, not just the first):
    R1  total `- ` item lines inside every `### Queue` section is exactly
        FROZEN_ITEMS (111) - the historical snapshot neither grew nor shrank
    R2  the NEWEST `## [x.y.z]` entry owns a `### Queue` section - the pointer
        was not dropped by a run that forgot it
    R3  that newest `### Queue` section contains NO list item of any kind
        (`- `, `* `, `+ `, `1.`) - no second list may be written there
    R4  that newest `### Queue` section names `agent-logs/PROGRESS.md`
    R5  agent-logs/PROGRESS.md carries at least one `Next-candidate queued`
        bullet - the authoritative list still exists

R1 is deliberately an EQUALITY against a constant rather than a "greater than":
    a count that may only grow cannot see a history rewrite, and a count
    compared against nothing is the shape of arithmetic this repo has been
    removing since [0.4.65]. Changing FROZEN_ITEMS is a deliberate act - edit
    the constant and say so in the changelog entry that does it.

Fenced code blocks are NOT structure: an entry that quotes the shape of a
    `### Queue` section - as [0.4.67] does - must not gain a section from the
    quotation, and list items inside the fence do not count towards
    FROZEN_ITEMS either.

exit 3 is "cannot verify", never "ok": a sandbox repo with no CHANGELOG.md is
    an environment where the rule was not evaluated, and reporting that as a
    pass is how a check stops being a check.

`detail` in the JSON output never contains a `"` or a `\\`, so a shell caller
    can embed it in its own JSON object without re-escaping it.
> build · mimo-v2.6-flash-free
Waiting on the full regression run (started in background). Meanwhile, here's where the run stands:

- **STEP 0 done first**: `investor_to_agent` unread = **0 dev / 0 prod** (measured, both DBs), INBOX read — every entry struck `HANDLED` except line 452 (six cloud accounts), open because it's blocked on REPORT.md §14. Fresh-probe reply inserted: **dev 74 / prod 40**.
- **Step (queue item 19)**: new suite `tests/test_system_status_dig_single_point.sh` — **49 assertions, 0 failed, ~1.1s**. It's a static guard: `tools/system-status` may execute `dig` in exactly one place (`dns_query()`), enforced by a python state machine that tracks real quote/comment/heredoc state (a `grep -n dig` finds 14 occurrences — 11 comments + a 3-line `echo` string — while only 1 is an execution).
- **Pre-fix replays already measured**: `df567ec^` → 23/25 fail; `cd81110^` → 35/14 fail with `READERS must stay 1 (got '5')`.
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

[Sat Sep 26 19:28:06 2026] 127.0.0.1:38036 Accepted
[Sat Sep 26 19:28:06 2026] 127.0.0.1:38036 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41306 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41306 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41322 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41322 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41338 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41338 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41340 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41340 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41350 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41350 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41364 Accepted
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41364 Closing
[Sat Sep 26 19:28:22 2026] 127.0.0.1:41370 Accepted
[Sat Sep 26 19:28:23 2026] 127.0.0.1:41370 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44680 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44680 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44688 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44688 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44690 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44690 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44692 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44692 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44706 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44706 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44710 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44710 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44714 Accepted
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44714 Closing
[Sat Sep 26 19:28:38 2026] 127.0.0.1:44730 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44730 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44742 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44742 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44754 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44754 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44768 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44768 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44778 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44778 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44780 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44780 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44782 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44782 Closing
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44788 Accepted
[Sat Sep 26 19:28:39 2026] 127.0.0.1:44788 Closing
[Sat Sep 26 19:33:54 2026] 127.0.0.1:53108 Accepted
[Sat Sep 26 19:33:54 2026] 127.0.0.1:53108 Closing
[Sat Sep 26 19:34:02 2026] 127.0.0.1:52732 Accepted
[Sat Sep 26 19:34:02 2026] 127.0.0.1:52732 Closing
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52358 Accepted
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52358 Closing
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52360 Accepted
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52360 Closing
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52376 Accepted
[Sat Sep 26 19:34:37 2026] 127.0.0.1:52376 Closing
[Sat Sep 26 19:35:21 2026] 127.0.0.1:60986 Accepted
[Sat Sep 26 19:35:21 2026] 127.0.0.1:60986 Closing
[Sat Sep 26 19:38:23 2026] 127.0.0.1:60150 Accepted