Gladex Agent Logs

echo "== repo =="; cd /data/repo && php tests/test_ts_ordering.php | tail -1
for e in dev prod; do echo "== $e =="; php /opt/startup/$e/tests/test_ts_ordering.php | tail -1; php /opt/startup/$e/tests/test_lan_middleware.php | tail -1; done
== repo ==
=== Results: 7 passed, 0 failed ===
== dev ==
=== Results: 7 passed, 0 failed ===
=== Results: 49 passed, 0 failed ===
== prod ==
=== Results: 7 passed, 0 failed ===
=== Results: 49 passed, 0 failed ===
> build · mimo-v2.6-flash-free
All green everywhere (7/7 + 49/49 on repo, dev, prod). Committing and documenting:
$ head -30 /data/repo/CHANGELOG.md
# Changelog

All notable changes to Gladex are documented here.

Format: `[version] - YYYY-MM-DD — description`

## [0.4.4] - 2026-09-23

### Added
- **`log.gladex.de` log viewer** (queued item b): new `app/src/php/log.php` — run-log list (name/size/mtime) from `/data/agent-logs/` + 200-line tail with 256KB read cap, plus env app-log tail from `/opt/startup/{dev,prod}/logs/`
- Path safety: basename allow-list `^run-\d{8}-\d{6}-\d+\.log$` — `?file=../../../../etc/passwd` and any non-matching name are rejected (verified); byte-count `strlen()`/`substr()` only (mbstring not installed — no `mb_*`)
- Routing: `/log` path route + host-based `log.gladex.de` block, both LAN-only (`isInvestorRoute` + explicit `REMOTE_ADDR` check with `denials.log`); `/log` added to the fallback-redirect exclusion so unknown-path → 302 behavior is unchanged
- DNS: `A log.gladex.de → 77.90.15.49` TTL 3600 (pdns API 204, zone serial 2026092305), dig-verified on both authoritative NS
- TLS: LE cert expanded to **7 SANs** (+log.gladex.de), expires 2026-12-22; `https://log.gladex.de/` 200, `http://` 301→https, apache reloaded, mail daemons unaffected

### Tests
- `tests/test_lan_middleware.php` extended: `/log` gated, `/team` + `/info` public, allow/deny sims → **49 passed, 0 failed** (repo, dev, prod)

### Security
- `/log` (run-log content) is reachable via the same public Apache ProxyPass path as the other investor routes — it inherits the open A/B/C decision from 0.4.3; no config change or spend made pending the investor's answer

## [0.4.3] - 2026-09-23

### Security
- **Known issue — escalated to the investor, open decision**: Apache `:443` terminates TLS and proxies everything (`ProxyPreserveHost On` + `ProxyPass / http://127.0.0.1:8001/`), so the app sees `REMOTE_ADDR=127.0.0.1` for every request arriving through the VPS and the LAN-only gate allows it. Proven world-reachable: `/api/messages` (all 21 chat messages), `/investor`, `/mailbox` read + send (public `POST /mailbox/send` → 303 with real local delivery), `/budget`, `/trust`, `/changelog`, `/docs`; live scanner traffic (439 `.env`, 61 cms-probe, 60 scanner-UA requests, 244 `GET /investor` → 200). Gate logic and middleware tests (44 green) are unaffected — a direct non-proxied attempt from the VPS IP is correctly DENIED.
- Not closed unilaterally: the operator's active session (Mac/iPhone via tunnel peer `10.2.3.1`) uses this exact path; blocking it would lock the investor out mid-session. Options sent to the investor (prod chat msg 22): **(A)** VPS XFF/PROXY-protocol + `mod_remoteip` trusting the tunnel peer, **(B)** Apache basic auth on investor paths, **(C)** VPS-side `:443` ACL. Recorded as a known limitation in `STRUCTURE.md`; no config change and no spend made pending the decision.

## [0.4.2] - 2026-09-23

### Added
$ cd /data/repo && git add -A && git commit -m "fix: message ordering — migrate legacy ts datestrings to epoch, numeric sort, regression test" && git log --oneline -1
[main c6d7e82] fix: message ordering — migrate legacy ts datestrings to epoch, numeric sort, regression test
 3 files changed, 81 insertions(+), 2 deletions(-)
 create mode 100644 tests/test_ts_ordering.php
c6d7e82 fix: message ordering — migrate legacy ts datestrings to epoch, numeric sort, regression test
> build · mimo-v2.6-flash-free
Committed. Adding CHANGELOG entry and PROGRESS.md log:
← Edit repo/CHANGELOG.md
Index: repo/CHANGELOG.md
===================================================================
--- repo/CHANGELOG.md
+++ repo/CHANGELOG.md
@@ -3,8 +3,20 @@
 All notable changes to Gladex are documented here.
 
 Format: `[version] - YYYY-MM-DD — description`
 
+## [0.4.5] - 2026-09-23
+
+### Fixed
+- **Chat message ordering (queued item c)**: `/api/messages` sorted lexicographically (`ORDER BY ts ASC`) over mixed `ts` formats — dev had 34 ISO/datestrings vs 23 epoch rows (prod 5/17), so all epoch rows (leading `1…`) sorted *before* every datestring (leading `2026…`) and chat chronology was scrambled across two eras.
+  - **Migration**: `UPDATE messages SET ts = strftime('%s', ts)` on non-epoch rows — dev 34 rows, prod 5 rows converted; row counts unchanged (57/22); pre-migration DB backups in `/data/shared/backups/messages-{dev,prod}-preTsFix-20260923-230104.db`.
+  - **Schema**: default `ts` is now `strftime('%s','now')` (was `datetime('now')`) so new rows can never be datestrings.
+  - **Query**: `ORDER BY CAST(ts AS INTEGER) ASC, id ASC` (numeric + deterministic tie-break).
+- New regression test `tests/test_ts_ordering.php` (7 assertions): schema default is 10-digit epoch, migration converts all 3 legacy formats, numeric ordering ascending, `/api/messages` query contract, JS `fmtTime` numeric branch. Resolves both repo (`app/src/php/`) and deployed (`src/php/`) schema paths.
+
+### Tests
+- `test_ts_ordering.php` **7/7** + `test_lan_middleware.php` **49/49**, green on repo, dev, prod.
+
 ## [0.4.4] - 2026-09-23
 
 ### Added
 - **`log.gladex.de` log viewer** (queued item b): new `app/src/php/log.php` — run-log list (name/size/mtime) from `/data/agent-logs/` + 200-line tail with 256KB read cap, plus env app-log tail from `/opt/startup/{dev,prod}/logs/`

> build · mimo-v2.6-flash-free
Appending the PROGRESS.md run entry:
$ date -u +%Y-%m-%dT%H:%M:%SZ; echo "---STEP0 recheck---"; sqlite3 /opt/startup/dev/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; sqlite3 /opt/startup/prod/data/messages.db "SELECT count(*) FROM messages WHERE direction='investor_to_agent' AND read=0;"; awk '/^## /{if ($0 !~ /HANDLED/) print}' /data/repo/INBOX.md | wc -l; ls /data/repo/mailboxes/*/ 2>/dev/null | grep -v '^$' | grep -v ':' | head
2026-09-23T21:04:13Z
---STEP0 recheck---
0
0
0
> build · mimo-v2.6-flash-free
STEP 0 recheck clean. Writing the PROGRESS entry:
$ cat >> /data/repo/agent-logs/PROGRESS.md << 'EOF'

---

## Run 2026-09-23T21:04:13Z — STEP 0 clear + chat ordering bug fixed (0.4.5)

**STEP 0 (investor first)**: queried dev AND prod `messages.db` — **0 unread**
`investor_to_agent` rows; `INBOX.md` — **0 unhandled** (all entries HANDLED);
`mailboxes/*` — no Dispatcher assignments. Nothing to reply to; nothing
unanswered. Re-checked again at exit: 0/0/0 still.

**This run's step — queued item (c): unixtimestamp ordering VERIFIED, bug FOUND + FIXED**

Verification of `ORDER BY ts ASC` on the live DBs exposed a real defect:

1. **Bug**: `ts` held MIXED formats — dev 34 ISO/datestrings (`2026-09-20T…`,
   `2026-09-20 …`) + 23 unix epochs; prod 5 + 17. Lexicographic sort put every
   epoch row (leading `1…`) BEFORE every datestring (leading `2026…`), so the
   chat rendered newest-era messages above older-era ones — chronology
   scrambled across two "eras" (RNK(id) vs RNK(ts) mismatched on **all 57** dev
   rows). Web-form sends (epoch, `(string)time()`) and legacy/default inserts
   (`datetime('now')`) were the two writers.
2. **Backups first**: pre-migration copies of both DBs →
   `/data/shared/backups/messages-{dev,prod}-preTsFix-20260923-230104.db`.
3. **Migration**: `UPDATE messages SET ts = strftime('%s', ts)` guarded to
   non-epoch rows → dev 34 rows, prod 5 rows converted; **0 non-epoch left**;
   row counts unchanged (57/22). All 3 legacy formats parse correctly
   (iso+offset, iso+Z, space-separator — all were UTC, container/PHP tz=UTC).
4. **Root-cause guards** (`app/src/php/schema.sql` + `app.php`):
   - default `ts` → `strftime('%s','now')` — new rows can never be datestrings;
   - query → `ORDER BY CAST(ts AS INTEGER) ASC, id ASC` (numeric + stable
     tie-break for same-second rows).
5. **Regression test** `tests/test_ts_ordering.php` (new, 7 assertions): schema
   default is 10-digit epoch; migration converts all 3 legacy formats; numeric
   ascending; `/api/messages` query contract; JS `fmtTime` numeric branch.
   Path-resolves both repo layout (`app/src/php/`) and env layout (`src/php/`).
6. **Deploy**: app.php + schema.sql + tests copied to `/opt/startup/{dev,prod}`,
   units restarted, both **active**.

**Verification (all green)**: post-migration RNK(id)↔RNK(ts) inversions **0/0**;
live `GET /api/messages` — dev 57 rows + prod 22 rows, all 10-digit epoch,
strictly ascending; tests **7/7** (ts) + **49/49** (lan) on repo, dev, prod;
healthz **200/200**; `https://gladex.de/` 200; frontend needs no change
(`fmtTime` already has the `/^\d+$/` epoch branch).

**Note (unchanged, pending investor)**: `https://gladex.de/api/messages`
returns 200 publicly — the A/B/C gating decision escalated in 0.4.3 is still
open; no gating config touched this run, per standing rule.

**Docs**: CHANGELOG 0.4.4 → **0.4.5** (bug + migration + guards + tests), this
entry. Commit `c6d7e82`.

**Budget**: untouched — month 2026-09, allowance 5.00, spent 1.50, remaining
3.50; model spend 0.00 (`*-free` only).

**Next (queued)**: (d) `ip-drift-check` review + any remaining STRUCTURE.md
mail-stack rows (snakeoil→LE already done per c70314e — verify recorded);
standing: awaiting investor A/B/C security decision.

**System status**: 0 unread investor_to_agent (dev+prod), 0 unhandled INBOX,
0 pending Dispatcher assignments, healthz 200/200, tests 7+49 / 0, units
active, git committed + pushed.
EOF
echo appended; tail -5 /data/repo/agent-logs/PROGRESS.md
appended
standing: awaiting investor A/B/C security decision.

**System status**: 0 unread investor_to_agent (dev+prod), 0 unhandled INBOX,
0 pending Dispatcher assignments, healthz 200/200, tests 7+49 / 0, units
active, git committed + pushed.
> build · mimo-v2.6-flash-free
Committing docs and pushing:
$ cd /data/repo && git add -A && git commit -m "docs: 0.4.5 chat ordering fix — CHANGELOG + PROGRESS (STEP 0 clear)" && git push origin main 2>&1 | tail -3; git log --oneline -3; git status --short | head
[main 6d1d0f9] docs: 0.4.5 chat ordering fix — CHANGELOG + PROGRESS (STEP 0 clear)
 2 files changed, 73 insertions(+)
To git://git.gladex.de/gladex.git
   cc45ccb..6d1d0f9  main -> main
6d1d0f9 docs: 0.4.5 chat ordering fix — CHANGELOG + PROGRESS (STEP 0 clear)
c6d7e82 fix: message ordering — migrate legacy ts datestrings to epoch, numeric sort, regression test
cc45ccb feat: log.gladex.de — A record + LAN-only /log viewer + cert 7 SANs
> build · mimo-v2.6-flash-free
Committed and pushed. Final health check:
$ curl -s -o /dev/null -w "dev=%{http_code} " http://127.0.0.1:8000/healthz; curl -s -o /dev/null -w "prod=%{http_code}\n" http://127.0.0.1:8001/healthz; systemctl is-active investor-app-dev investor-app-prod apache2 postfix dovecot | tr '\n' ' '; echo; cd /data/repo && git status --porcelain | wc -l
dev=200 prod=200
active active active active active 
0
> build · mimo-v2.6-flash-free
Run complete. Summary:

## STEP 0 — Investor: nothing owed
- Queried dev **and** prod `messages.db`: **0 unread** `investor_to_agent` rows
- `INBOX.md`: **0 unhandled** (all entries HANDLED)
- `mailboxes/*`: no Dispatcher assignments
- Re-checked at exit: still 0/0/0

## This run's step — queued item (c): chat ordering — verified, bug found, fixed (v0.4.5)

**The bug**: `/api/messages` sorted `ts` as text, but `ts` held **mixed formats** (dev: 34 datestrings + 23 epochs; prod: 5 + 17). Lexicographic sort put every epoch row (`1…`) *before* every datestring (`2026…`), scrambling chat chronology — RNK mismatched on all 57 dev rows.

**Fix** (root cause + data + guard rails):
1. **Backups** of both DBs → `/data/shared/backups/messages-{dev,prod}-preTsFix-20260923-230104.db`
2. **Migration**: `strftime('%s', ts)` on legacy rows → 34 dev + 5 prod converted, 0 non-epoch left, row counts unchanged (57/22)
3. **Schema default** → `strftime('%s','now')` — new rows can never be datestrings
4. **Query** → `ORDER BY CAST(ts AS INTEGER) ASC, id ASC` (numeric + stable tie-break)
5. **New regression test** `tests/test_ts_ordering.php` (7 assertions, resolves both repo and env layouts)

**Verified green**: 0 inversions post-migration; live API dev 57 + prod 22 rows, all 10-digit epoch, strictly ascending; tests **7/7 + 49/49** on repo, dev, prod; healthz **200/200**; all units active; frontend unchanged (`fmtTime` already handles epoch).

**Not touched**: the public-reachability of `/api/messages` (returns 200 via VPS) — that's the A/B/C gating decision escalated in 0.4.3, still awaiting the investor.

**Committed + pushed**: `c6d7e82` (fix), `6d1d0f9` (docs). Budget untouched (spent 1.50 / 3.50 left; model spend 0.00).
exit=0

Select another run log from the list above. Only files matching run-YYYYMMDD-HHMMSS-N.log are readable.

[Thu Sep 24 00:45:17 2026] 127.0.0.1:38480 Closing
[Thu Sep 24 00:45:53 2026] 127.0.0.1:53368 Accepted
[Thu Sep 24 00:45:53 2026] 127.0.0.1:53368 Closing
[Thu Sep 24 00:53:00 2026] 127.0.0.1:57694 Accepted
[Thu Sep 24 00:53:00 2026] 127.0.0.1:57694 Closing
[Thu Sep 24 00:53:00 2026] 127.0.0.1:57708 Accepted
[Thu Sep 24 00:53:00 2026] 127.0.0.1:57708 Closing
[Thu Sep 24 00:55:03 2026] 127.0.0.1:59628 Accepted
[Thu Sep 24 00:55:03 2026] 127.0.0.1:59628 Closing
[Thu Sep 24 00:55:03 2026] 127.0.0.1:59642 Accepted
[Thu Sep 24 00:55:03 2026] 127.0.0.1:59642 Closing
[Thu Sep 24 00:55:04 2026] 127.0.0.1:59646 Accepted
[Thu Sep 24 00:55:04 2026] 127.0.0.1:59646 Closing
[Thu Sep 24 00:55:08 2026] 127.0.0.1:50692 Accepted
[Thu Sep 24 00:55:08 2026] 127.0.0.1:50692 Closing
[Thu Sep 24 00:55:08 2026] 127.0.0.1:50706 Accepted
[Thu Sep 24 00:55:08 2026] 127.0.0.1:50706 Closing
[Thu Sep 24 00:56:17 2026] 127.0.0.1:41468 Accepted
[Thu Sep 24 00:56:17 2026] 127.0.0.1:41468 Closing
[Thu Sep 24 00:58:02 2026] 127.0.0.1:35814 Accepted
[Thu Sep 24 00:58:02 2026] 127.0.0.1:35814 Closing
[Thu Sep 24 00:59:31 2026] PHP 8.3.6 Development Server (http://0.0.0.0:8001) started
[Thu Sep 24 00:59:32 2026] 127.0.0.1:49864 Accepted
[Thu Sep 24 00:59:32 2026] 127.0.0.1:49864 Closing
[Thu Sep 24 00:59:32 2026] 127.0.0.1:49880 Accepted
[Thu Sep 24 00:59:32 2026] 127.0.0.1:49880 Closing
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59220 Accepted
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59220 Closing
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59222 Accepted
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59222 Closing
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59230 Accepted
[Thu Sep 24 01:05:48 2026] 127.0.0.1:59230 Closing
[Thu Sep 24 01:05:49 2026] 127.0.0.1:59244 Accepted
[Thu Sep 24 01:05:49 2026] 127.0.0.1:59244 Closing
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59256 Accepted
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59256 Closing
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59272 Accepted
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59272 Closing
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59286 Accepted
[Thu Sep 24 01:05:52 2026] 127.0.0.1:59286 Closing
[Thu Sep 24 01:05:53 2026] 127.0.0.1:59302 Accepted
[Thu Sep 24 01:05:53 2026] 127.0.0.1:59302 Closing
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51786 Accepted
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51786 Closing
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51790 Accepted
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51790 Closing
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51792 Accepted
[Thu Sep 24 01:05:57 2026] 127.0.0.1:51792 Closing
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51804 Accepted
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51804 Closing
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51806 Accepted
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51806 Closing
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51822 Accepted
[Thu Sep 24 01:06:02 2026] 127.0.0.1:51822 Closing
[Thu Sep 24 01:06:07 2026] 127.0.0.1:42830 Accepted
[Thu Sep 24 01:06:07 2026] 127.0.0.1:42830 Closing
[Thu Sep 24 01:06:07 2026] 127.0.0.1:42842 Accepted
[Thu Sep 24 01:06:07 2026] 127.0.0.1:42842 Closing
[Thu Sep 24 01:06:08 2026] 127.0.0.1:42852 Accepted