title: VM gitbucket (GitBucket + navi instance pilar) type: host ip: 192.168.1.156 purpose: "GitBucket git hosting + navi agent instance (pilar, profile navi_code)" owner: gmikcon status: active confidence: "100" last_reviewed: 2026-10-29 review_interval: 90d
KVM/QEMU VM on hp-proliant-dl380-g6 (192.168.1.130). Primary role: GitBucket git hosting (see applications/gitbucket.md). Since 2026-09-19 the VM also hosts the navi-1 agent instance pilar (profile navi_code), deployed by navi (rei) from the deploy branch.
gitbucket (service owner, home = /home/ubuntu), navi (navi instance, home = /home/navi), ubuntu (admin; SSH/sudo credentials — see gnexus-creds secret bserv@192.168.1.130)/mnt/main/vm_main/gitbucket (p1 root 28G — 21% used; p5 data 71G — 6% used; observed 2026-09-19)deploy, commit e276b1e)navi-postgres (pgvector/pgvector:pg16) on 127.0.0.1:5433 — port shifted from the default 5432 because GitBucket's system PostgreSQL 10 occupies 5432; DATABASE_URL in .env points to 5433peer list as pilar — 192.168.1.156:8099 [online]CC=gcc CXX=g++ before re-running install.sh (script is idempotent)navi has NOPASSWD sudoers (/etc/sudoers.d/navi) so install.sh's internal sudo works over non-interactive SSH; the navi user password matches the hypervisor bserv credential (gnexus-creds)Diagnosed 2026-10-29 10:41–10:48 UTC by navi rei (panel event #1100, GHard server id 44). The 2026-10-29 episode (cpu_high 90.5% at 10:40:57 UTC, peak load1 5.07 at 10:41:27, java 181.6% CPU) is the 7th episode of the same class since 2026-10-12: 12.10 crawler storm, 13.10 spike, 25.10 01:55, 26.10 17:44, 27.10 17:50, 28.10 21:33, 29.10 10:40.
New signature on 29.10: not a 60–90 s burst but a sustained ~4.5 h crawl (06:00–10:42 UTC): 5248 GET /root/<repo>/archive/<path>/<sha>.zip requests from 192.168.1.150 via the public route (journal volume per hour: 1413 lines 06:xx, 2891 08:xx, 1447 10:xx; 29 ERROR — Broken pipe / EofException / NPE in RepositoryViewerController.archiveRepository). Repos: vmk-demo-bot (incl. node_modules), navi-1 (incl. webclient/dist, vendor/gnexus-ui-kit), smart-home-server, vue-indexer. Load decayed 5.07 → 0.66 by 10:48 UTC after the crawl stopped (last archive request 10:42:10, zero since 10:44). Self-recovered, no restart, GitBucket HTTP 200 throughout, iowait 0, all 8 panel services up.
Cause (confirmed across all 7 episodes): bursts of GET /root/<repo>/archive/<path>/<sha>.zip (and a few /raw/... file downloads) through git.gnexus.space — GitBucket's web-UI "download directory as ZIP" endpoint. Clients abort the download mid-stream (EofException / Broken pipe in journalctl, logged as RepositoryViewerController - Catch unhandled error), but GitBucket keeps building the ZIP server-side — several concurrent archive builds saturate all 4 vCPUs. Baseline between episodes: CPU 10–27%, load1 0.2–1.5.
Traffic source (attributed in earlier sessions): requests arrive via the nginx reverse-proxy VM at 192.168.1.183 (note: inventory entry "internal-proxy-vps .226" is stale — .226 does not answer SSH/ICMP; actual proxy is .183). Client seen in nginx access.log: 192.168.1.150 (ovpn VM, SNAT) with browser UA rotation Chrome 142–145 Win/Mac (27.10, 29.10); crawler UAs GPTBot, meta-externalagent, Reflectionbot, ClaudeBot, SemrushBot, MJ12bot observed in the 25.10 storm. robots.txt is absent (GitBucket returns an HTML error page for it).
Triage recipe (next episode): journalctl -u gitbucket --since '<start>' | grep -o 'Request(GET [^)]*)' — if archive/*.zip bursts are present, it is this pattern; no restart needed, load decays within ~2 min after the burst ends (29.10: within ~7 min after the 4.5 h crawl ended). For attribution: nginx access.log on 192.168.1.183 (creds internal-proxy-vps-creds / bserv@192.168.1.130).
Not applied (owner decision pending, 7th episode): rate-limiting or blocking /archive/ for bots + robots.txt at the nginx proxy (192.168.1.183), or a GitBucket plugin/config change. Escalation criterion "bursts ≥2/day" is not formally exceeded (1 burst/day), but episodes now recur 4 consecutive days (26–29 Oct) and the 29.10 episode lasted ~4.5 h — recommend the owner decide on proxy-level rate-limiting/robots.txt. Related journals: workspace/incident_gitbucket_cpu_2026-10-12.md, workspace/incident_gitbucket_2026-10-25_load_high.md, workspace/incident_gitbucket_cpu_2026-10-27_crawler_archives.md.
ubuntu@192.168.1.156, credentials in gnexus-creds secret bserv@192.168.1.130; root SSH deniedlosetup -f --show -r -P /mnt/main/vm_main/gitbucket, then mount -o ro,noload)