diff --git a/10-systems/hosts/gnbook.md b/10-systems/hosts/gnbook.md new file mode 100644 index 0000000..230b604 --- /dev/null +++ b/10-systems/hosts/gnbook.md @@ -0,0 +1,46 @@ +--- +title: VM gnbook (192.168.1.170) +type: host +ip: 192.168.1.170 +purpose: gnexus-book MCP knowledge-base server (Markdown docs + structured inventory) +owner: gmikcon +status: active +confidence: high +last_reviewed: 2026-11-10 +review_interval: 90d +source_of_truth: ssh-host-and-live-verified +--- + +# VM gnbook (192.168.1.170) + +## Overview + +- **Hostname**: gnbook, IP 192.168.1.170 (home LAN) +- **OS**: Ubuntu 22.04.5 LTS, kernel 5.15.0-202-generic +- **VM** on hypervisor HP ProLiant DL380 G6 (bserv 192.168.1.130) +- **Resources**: 2 vCPU, ~2 GiB RAM, 2 GiB swap, disk `/` = LVM 15.7 GB (single root FS, separate `/boot` 2 GB) +- **User**: ubuntu (sudo password in gnexus-creds secret `gnauth-gnbook-ssh`, id 11932ea2-998b-4b2c-a722-1bccc68454cd; same credentials cover gnauth 192.168.1.167). Root password differs from sudo password. +- **Service**: gnexus-book MCP server (python/uvicorn) — infrastructure knowledge base (Markdown docs + structured inventory) for MCP clients. + +## Disk profile (2026-11-10, measured) + +Small root FS — disk_high threshold (90%) events are expected to recur: + +| Consumer | Size | Note | +|---|---|---| +| `/usr/local/lib/ollama` | 3.5 GB | cuda_v12 2.5G + cuda_v13 953M kept simultaneously, vulkan 55M, bin 43M — biggest discretionary consumer | +| `/usr/lib` | 2.5 GB | stock firmware + kernel modules | +| `/swap.img` | 2.0 GB | expected | +| `/home/ubuntu` | 902 MB | claude 221M, nvm/node ~350M, repos 225M | +| `/var/lib/snapd` | ~533 MB | after cleanup; one disabled revision each remains | +| apt lists + cache | ~321 MB | lists; cache cleaned | +| `/var/log` | ~166 MB | after cleanup | +| `/opt/backups/gnexus-book` | 407 MB | existing backup dir | +| Docker | none | no /var/lib/docker | + +## Incident 2026-11-10: disk_high 90.9% + +- Panel event (server_id 18, mount `/`, value 90.9 ≥ threshold 90.0), ~06:42 UTC; at measurement moment panel showed 88.9% (panel % = used/(total−reserved)). +- Diagnosis (read-only SSH): disk **static** (+4 KB/min), no active filling; spike correlated with `unattended-upgr` activity. Journald user-1000 (the MCP server logs "POST /messages" every ~5 s) had grown `/var/log` to 382M; syslog jumped 0.4M → 13–14M since 2026-11-08. +- Cleanup applied by navi Eris (safe set only): `journalctl --vacuum-size=100M` (−216M), removed disabled snap revisions core20/core22/lxd (snapd 776M→533M), `apt-get clean` (−137M). Result: used 12.44 GB→11.81 GB (−628 MB), df 84%→80%, panel calc 79.0% — below threshold. +- **Left for owner decision**: (1) Ollama CUDA libs 3.5 GB — dropping cuda_v12 reclaims ~2.5 GB; (2) persistent journald cap `SystemMaxUse` (e.g. 100M) — the MCP request-log spam continues to grow (~10M/2d); (3) MCP server log verbosity. diff --git a/90-maintenance/pending-changes/20261110064915-a08935d7.json b/90-maintenance/pending-changes/20261110064915-a08935d7.json new file mode 100644 index 0000000..0cf9e09 --- /dev/null +++ b/90-maintenance/pending-changes/20261110064915-a08935d7.json @@ -0,0 +1,19 @@ +{ + "id": "20261110064915-a08935d7", + "kind": "doc", + "target": "10-systems/hosts/gnbook.md", + "summary": "New host doc gnbook: VM profile, disk breakdown after 2026-11-10 disk_high cleanup, pending owner decisions (ollama cuda_v12, journald cap, MCP log verbosity)", + "reason": "server_id 18 (gnbook) had no host doc; disk_high 90.9% incident 2026-11-10 diagnosed and cleaned by navi Eris — recording host profile, disk breakdown and pending owner decisions", + "payload": { + "content": "---\ntitle: VM gnbook (192.168.1.170)\ntype: host\nip: 192.168.1.170\npurpose: gnexus-book MCP knowledge-base server (Markdown docs + structured inventory)\nowner: gmikcon\nstatus: active\nconfidence: high\nlast_reviewed: 2026-11-10\nreview_interval: 90d\nsource_of_truth: ssh-host-and-live-verified\n---\n\n# VM gnbook (192.168.1.170)\n\n## Overview\n\n- **Hostname**: gnbook, IP 192.168.1.170 (home LAN)\n- **OS**: Ubuntu 22.04.5 LTS, kernel 5.15.0-202-generic\n- **VM** on hypervisor HP ProLiant DL380 G6 (bserv 192.168.1.130)\n- **Resources**: 2 vCPU, ~2 GiB RAM, 2 GiB swap, disk `/` = LVM 15.7 GB (single root FS, separate `/boot` 2 GB)\n- **User**: ubuntu (sudo password in gnexus-creds secret `gnauth-gnbook-ssh`, id 11932ea2-998b-4b2c-a722-1bccc68454cd; same credentials cover gnauth 192.168.1.167). Root password differs from sudo password.\n- **Service**: gnexus-book MCP server (python/uvicorn) — infrastructure knowledge base (Markdown docs + structured inventory) for MCP clients.\n\n## Disk profile (2026-11-10, measured)\n\nSmall root FS — disk_high threshold (90%) events are expected to recur:\n\n| Consumer | Size | Note |\n|---|---|---|\n| `/usr/local/lib/ollama` | 3.5 GB | cuda_v12 2.5G + cuda_v13 953M kept simultaneously, vulkan 55M, bin 43M — biggest discretionary consumer |\n| `/usr/lib` | 2.5 GB | stock firmware + kernel modules |\n| `/swap.img` | 2.0 GB | expected |\n| `/home/ubuntu` | 902 MB | claude 221M, nvm/node ~350M, repos 225M |\n| `/var/lib/snapd` | ~533 MB | after cleanup; one disabled revision each remains |\n| apt lists + cache | ~321 MB | lists; cache cleaned |\n| `/var/log` | ~166 MB | after cleanup |\n| `/opt/backups/gnexus-book` | 407 MB | existing backup dir |\n| Docker | none | no /var/lib/docker |\n\n## Incident 2026-11-10: disk_high 90.9%\n\n- Panel event (server_id 18, mount `/`, value 90.9 ≥ threshold 90.0), ~06:42 UTC; at measurement moment panel showed 88.9% (panel % = used/(total−reserved)).\n- Diagnosis (read-only SSH): disk **static** (+4 KB/min), no active filling; spike correlated with `unattended-upgr` activity. Journald user-1000 (the MCP server logs \"POST /messages\" every ~5 s) had grown `/var/log` to 382M; syslog jumped 0.4M → 13–14M since 2026-11-08.\n- Cleanup applied by navi Eris (safe set only): `journalctl --vacuum-size=100M` (−216M), removed disabled snap revisions core20/core22/lxd (snapd 776M→533M), `apt-get clean` (−137M). Result: used 12.44 GB→11.81 GB (−628 MB), df 84%→80%, panel calc 79.0% — below threshold.\n- **Left for owner decision**: (1) Ollama CUDA libs 3.5 GB — dropping cuda_v12 reclaims ~2.5 GB; (2) persistent journald cap `SystemMaxUse` (e.g. 100M) — the MCP request-log spam continues to grow (~10M/2d); (3) MCP server log verbosity.\n" + }, + "status": "applied", + "created_at": "2026-11-10T06:49:15Z", + "updated_at": "2026-11-10T06:49:17Z", + "applied_at": "2026-11-10T06:49:17Z", + "validation": { + "status": "ok", + "issue_count": 0, + "issues": [] + } +}