Gladex Agent Logs

Agent run logs & app logs · env: prod · LAN-only investor surface

Overview
Run logs856 files, 42.6 MB
Latest run logrun-20261001-150856-454.log
Log directory/data/agent-logs
App log directory/opt/startup/prod/logs
Run logs (newest first, last 50)
FileSizeModified (UTC)
run-20261001-150856-454.log 153 B 2026-10-01 13:08:57
run-20261001-145849-453.log 153 B 2026-10-01 12:58:50
run-20261001-144842-452.log 153 B 2026-10-01 12:48:43
run-20261001-143835-451.log 153 B 2026-10-01 12:38:35
run-20261001-142828-450.log 190 B 2026-10-01 12:28:28
run-20261001-141820-449.log 153 B 2026-10-01 12:18:21
run-20261001-140813-448.log 153 B 2026-10-01 12:08:14
run-20261001-135806-447.log 153 B 2026-10-01 11:58:07
run-20261001-134759-446.log 153 B 2026-10-01 11:48:00
run-20261001-133752-445.log 153 B 2026-10-01 11:37:53
run-20261001-132745-444.log 153 B 2026-10-01 11:27:45
run-20261001-131738-443.log 190 B 2026-10-01 11:17:39
run-20261001-130731-442.log 153 B 2026-10-01 11:07:31
run-20261001-125724-441.log 153 B 2026-10-01 10:57:24
run-20261001-124717-440.log 153 B 2026-10-01 10:47:17
run-20261001-123709-439.log 153 B 2026-10-01 10:37:10
run-20261001-122702-438.log 153 B 2026-10-01 10:27:03
run-20261001-121655-437.log 153 B 2026-10-01 10:16:56
run-20261001-112609-436.log 264 KB 2026-10-01 10:06:49
run-20261001-104159-435.log 361 KB 2026-10-01 09:16:02
run-20261001-092806-434.log 353 KB 2026-10-01 08:31:53
run-20261001-080602-433.log 406 KB 2026-10-01 07:17:59
run-20261001-065553-432.log 373 KB 2026-10-01 05:55:55
run-20261001-050850-431.log 523 KB 2026-10-01 04:45:47
run-20261001-042252-430.log 198 KB 2026-10-01 02:58:43
run-20261001-032127-429.log 327 KB 2026-10-01 02:12:46
run-20261001-022005-428.log 263 KB 2026-10-01 01:11:21
run-20261001-020022-427.log 205 KB 2026-10-01 00:09:58
run-20261001-015015-426.log 153 B 2026-09-30 23:50:16
run-20261001-014008-425.log 153 B 2026-09-30 23:40:09
run-20261001-013001-424.log 153 B 2026-09-30 23:30:02
run-20261001-011954-423.log 153 B 2026-09-30 23:19:55
run-20261001-010947-422.log 153 B 2026-09-30 23:09:48
run-20261001-005940-421.log 153 B 2026-09-30 22:59:41
run-20261001-004933-420.log 153 B 2026-09-30 22:49:34
run-20261001-003926-419.log 153 B 2026-09-30 22:39:27
run-20261001-002920-418.log 190 B 2026-09-30 22:29:20
run-20261001-001913-417.log 153 B 2026-09-30 22:19:13
run-20261001-000906-416.log 153 B 2026-09-30 22:09:07
run-20260930-231805-415.log 582 KB 2026-09-30 21:59:00
run-20260930-223504-414.log 194 KB 2026-09-30 21:07:59
run-20260930-212614-413.log 270 KB 2026-09-30 20:24:58
run-20260930-201407-412.log 423 KB 2026-09-30 19:16:08
run-20260930-184629-411.log 463 KB 2026-09-30 18:04:01
run-20260930-165104-410.log 325 KB 2026-09-30 16:36:29
run-20260930-161443-409.log 189 KB 2026-09-30 14:41:04
run-20260930-160441-408.log 153 B 2026-09-30 14:04:43
run-20260930-155440-407.log 153 B 2026-09-30 13:54:41
run-20260930-154440-406.log 153 B 2026-09-30 13:44:40
run-20260930-153439-405.log 153 B 2026-09-30 13:34:40
Tail — run-20261001-104159-435.log (last 200 lines)
Overall: ALL SYSTEMS HEALTHY
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -rn "help.*wc -l\|wc -l.*help\|help_lines" tests/*.sh tests/*.php 2>/dev/null | head; echo "=== step0 recheck ==="; sqlite3 /opt/startup/dev/data/messages.db "SELECT COUNT(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; sqlite3 /opt/startup/prod/data/messages.db "SELECT COUNT(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"
=== step0 recheck ===
0
0
> build · mimo-v2.6-flash-free
$ cd /data/repo && ./tools/queue-source-check --help 2>&1 | head -50
usage: queue-source-check [-h] [--format {human,json}] [--repo DIR]

Verify that the next-candidate work queue has exactly one home: agent-logs/PROGRESS.md carries the live list and CHANGELOG.md's newest '### Queue' section is a pointer at it, with the 111 historical item lines frozen.

options:
  -h, --help            show this help message and exit
  --format {human,json}
                        output format (default: human; json = one compact
                        object)
  --repo DIR            repo to check (default: GLADEX_REPO_DIR, else the repo
                        this script lives in)

exit codes:
  0 - the queue has exactly one home: CHANGELOG's newest ### Queue section is a
      pointer at agent-logs/PROGRESS.md, PROGRESS carries the live list, and the 111 historical
      item lines are unchanged
  1 - a violation: a second list exists, the pointer was dropped, the
      authoritative list is empty, one entry owns two `### Queue` sections, or
      a `### Queue` heading was not read as a section (see the violations[]
      array - every cause is reported, not just the first)
  2 - invalid arguments
  3 - cannot verify: CHANGELOG.md, agent-logs/PROGRESS.md or a ### Queue section is absent -
      NOT a pass, never reported as one
> build · mimo-v2.6-flash-free
$ cd /data/repo && grep -n "frozen\|item_re\|ITEM" tools/queue-source-check | head -40
38:        FROZEN_ITEMS (111) - the historical snapshot neither grew nor shrank
77:    removing since [0.4.65]. Changing FROZEN_ITEMS is a deliberate act - edit
83:    FROZEN_ITEMS either.
121:FROZEN_ITEMS = 111
285:            "frozen_items_expected": FROZEN_ITEMS,
286:            "frozen_items_actual": None,
348:    # R1: the historical snapshot is frozen.
350:    report["rules"]["frozen_items_actual"] = total_items
351:    if total_items != FROZEN_ITEMS:
353:            "CHANGELOG.md ### Queue carries %d item line(s), frozen at %d"
354:            % (total_items, FROZEN_ITEMS)
427:        "one queue: [%s] pointer-only, %d item line(s) frozen across %d section(s), "
478:            "frozen_items_expected": FROZEN_ITEMS,
479:            "frozen_items_actual": None,
509:            "historical item lines frozen."
> build · mimo-v2.6-flash-free
$ cd /data/repo && sed -n '300,430p' tools/queue-source-check
"ok": False,
        "exit_code": EXIT_CANNOT,
    }

    # A1: availability. A file that is not there is "cannot verify".
    cl_text = read(changelog)
    if cl_text is None:
        report["error"] = "cannot verify: %s is absent or unreadable" % changelog
        report["detail"] = detail(report["error"])
        return report

    parsed = parse(cl_text)
    sections = parsed["sections"]
    report["rules"]["queue_sections"] = len(sections)

    # Census + per-entry tally (R6 / R7), both taken BEFORE the availability
    # returns: a reader that only reports what it could not judge is how a
    # number stops meaning anything. NUMBERING - one scheme, three documents:
    # the module docstring above, this file's comments and tools/REGISTRY.md
    # all number the rules A1-A3 (availability) and R1-R7 (the rules), and
    # tests/test_queue_source.sh section O asserts that every rule citation in
    # any of them names a rule the docstring actually declares - so a stale
    # citation is a red instead of a name that means two different rules.
    raw_headings, fenced_headings = count_queue_headings(cl_text)
    report["rules"]["queue_headings_raw"] = raw_headings
    report["rules"]["queue_headings_fenced"] = fenced_headings
    sections_per_version: dict = {}
    for section in sections:
        key = section["version"]
        sections_per_version[key] = sections_per_version.get(key, 0) + 1
    duplicates = [(ver, n) for ver, n in sections_per_version.items() if n > 1]
    report["rules"]["duplicate_queue_sections"] = len(duplicates)

    # A2: at least one `### Queue` section exists to judge.
    if not sections:
        report["error"] = "cannot verify: no ### Queue section in %s" % changelog
        report["detail"] = detail(report["error"])
        return report

    # A3: the authoritative file must be readable before its content is judged.
    pg_text = read(progress)
    if pg_text is None:
        report["error"] = "cannot verify: %s is absent or unreadable" % progress
        report["detail"] = detail(report["error"])
        return report

    violations: list[str] = []

    # R1: the historical snapshot is frozen.
    total_items = sum(s["items"] for s in sections)
    report["rules"]["frozen_items_actual"] = total_items
    if total_items != FROZEN_ITEMS:
        violations.append(
            "CHANGELOG.md ### Queue carries %d item line(s), frozen at %d"
            % (total_items, FROZEN_ITEMS)
        )

    # R2 / R3 / R4: the newest entry's section is a POINTER, not a list.
    newest = parsed["newest_section"]
    report["rules"]["newest_version"] = parsed["newest_version"]
    report["rules"]["newest_has_queue_section"] = newest is not None
    if newest is None:
        violations.append(
            "newest CHANGELOG entry [%s] owns no ### Queue section - the pointer "
            "was dropped" % (parsed["newest_version"] or "?")
        )
    else:
        body = "\n".join(newest["body"])
        items = newest["items"]
        report["rules"]["newest_queue_items"] = items
        report["rules"]["newest_points_at_progress"] = POINTER_NEEDLE in body
        if items:
            violations.append(
                "[%s] ### Queue lists %d item(s) - the queue lives in %s, not here"
                % (parsed["newest_version"] or "?", items, PROGRESS_REL)
            )
        if POINTER_NEEDLE not in body:
            violations.append(
                "[%s] ### Queue does not point at %s"
                % (parsed["newest_version"] or "?", PROGRESS_REL)
            )

    # R5: the authoritative list exists.
    bullets = count_queue_bullets(pg_text)
    report["rules"]["progress_bullets"] = bullets
    if bullets < 1:
        violations.append(
            "%s carries no 'Next-candidate queued' bullet - the authoritative "
            "queue is gone" % PROGRESS_REL
        )

    # R6 - one pointer per `## [x.y.z]` entry. Written after the
    # others because it is the rule that has to read what they structurally
    # cannot: a duplicate `### Queue` section carries no items (so the freeze
    # is unmoved) and is never the newest one judged (so the pointer rules
    # read the copy that was meant).
    for ver, count in duplicates:
        violations.append(
            "CHANGELOG.md entry [%s] owns %d '### Queue' sections - one "
            "pointer per entry" % (ver if ver else "?", count)
        )

    # R7 - the census and parse() walked the file separately and
    # disagree. The fenced term is what separates "a heading that was quoted"
    # from "a heading nobody read" - without it every dropped heading would
    # look like a fence.
    accounted = len(sections) + fenced_headings
    if raw_headings != accounted:
        violations.append(
            "CHANGELOG.md carries %d '### Queue' heading(s) but %d section(s) "
            "+ %d fenced = %d - a heading was dropped"
            % (raw_headings, len(sections), fenced_headings, accounted)
        )

    if violations:
        report["violations"] = violations
        report["error"] = "%d violation(s): %s" % (
            len(violations),
            "; ".join(violations),
        )
        report["detail"] = detail(report["error"])
        report["exit_code"] = EXIT_VIOLATION
        return report

    report["ok"] = True
    report["exit_code"] = EXIT_OK
    report["detail"] = detail(
        "one queue: [%s] pointer-only, %d item line(s) frozen across %d section(s), "
        "%d PROGRESS bullet(s)"
        % (
            report["rules"]["newest_version"],
> build · mimo-v2.6-flash-free
$ cd /tmp/opencode/c103 && sed 's/^Sequencing (why there is no --jobs flag):$/Sequencing renamed:/' /data/repo/tools/regression-run > renamed && chmod +x renamed && ./renamed --help >/dev/null 2>/tmp/opencode/c103/err; echo "exit=$?"; cat /tmp/opencode/c103/err; ./renamed --format json --nonsense 2>&1 >/dev/null | head -3
exit=3
regression-run: ERROR docstring section missing or empty: 'sequencing:' -> 'Sequencing (why there is no --jobs flag):'
> build · mimo-v2.6-flash-free
Refusal path confirmed live for the new pair (exit 3, names `'sequencing:'`). Waiting on the full regression run now.
$ cd /data/repo && ./tools/regression-run --list 2>/dev/null | tail -2; ls -la /tmp/opencode/m103/ 2>/dev/null
test_webmail_session_routing.php
result: 75 suite(s) discovered, none run (exit 0)
total 960
drwxr-xr-x     2 root root   4096 Oct  1 11:12 .
drwxr-xr-x 29773 root root 974848 Oct  1 11:15 ..
-rw-r--r--     1 root root      0 Oct  1 11:12 regression.err
-rw-r--r--     1 root root      0 Oct  1 11:12 regression.json
> build · mimo-v2.6-flash-free
Regression is running (75 suites, JSON written on completion). I'll resume when it finishes.
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

App log tail — prod-8001.log (last 60 lines)
[Thu Oct  1 13:33:53 2026] 127.0.0.1:37394 Accepted
[Thu Oct  1 13:33:53 2026] 127.0.0.1:37394 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37952 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37952 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37968 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37968 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37972 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37972 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37984 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37984 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37990 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37990 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37998 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:37998 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:38010 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:38010 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:38024 Accepted
[Thu Oct  1 13:54:03 2026] 127.0.0.1:38024 Closing
[Thu Oct  1 13:54:03 2026] 127.0.0.1:38036 Accepted
[Thu Oct  1 13:54:04 2026] 127.0.0.1:38036 Closing
[Thu Oct  1 13:54:04 2026] 127.0.0.1:38052 Accepted
[Thu Oct  1 13:54:04 2026] 127.0.0.1:38052 Closing
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54340 Accepted
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54340 Closing
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54354 Accepted
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54354 Closing
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54358 Accepted
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54358 Closing
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54372 Accepted
[Thu Oct  1 13:55:26 2026] 127.0.0.1:54372 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38498 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38498 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38500 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38500 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38516 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38516 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38524 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38524 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38528 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38528 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38538 Accepted
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38538 Closing
[Thu Oct  1 13:58:11 2026] 127.0.0.1:38546 Accepted
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38546 Closing
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38558 Accepted
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38558 Closing
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38564 Accepted
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38564 Closing
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38580 Accepted
[Thu Oct  1 13:58:12 2026] 127.0.0.1:38580 Closing
[Thu Oct  1 14:10:26 2026] 127.0.0.1:38968 Accepted
[Thu Oct  1 14:10:26 2026] 127.0.0.1:38968 Closing
[Thu Oct  1 14:10:27 2026] 127.0.0.1:38984 Accepted
[Thu Oct  1 14:10:27 2026] 127.0.0.1:38984 Closing
[Thu Oct  1 15:03:52 2026] 127.0.0.1:58926 Accepted
[Thu Oct  1 15:03:52 2026] 127.0.0.1:58926 Closing
[Thu Oct  1 15:09:00 2026] 127.0.0.1:49914 Accepted
[Thu Oct  1 15:09:00 2026] 127.0.0.1:49914 Closing
[Thu Oct  1 15:09:00 2026] 127.0.0.1:49916 Accepted

Generated 2026-10-01 13:09:00 UTC · Gladex.de