| 2026-11-13 |
Refresh bserv access recipe (sshpass method, ssh_exec gotcha, wrong users) + dated state snapshot 2026-11-13
...
Проверка доступа 2026-11-13 заняла несколько неудачных SSH-попыток (ssh_exec без named connection, неверные пользователи). Документирую рабочий рецепт и текущий срез состояния.
|
| 2026-11-10 |
New host doc gnbook: VM profile, disk breakdown after 2026-11-10 disk_high cleanup, pending owner decisions (ollama cuda_v12, journald cap, MCP log verbosity)
...
server_id 18 (gnbook) had no host doc; disk_high 90.9% incident 2026-11-10 diagnosed and cleaned by navi Eris — recording host profile, disk breakdown and pending owner decisions
|
| 2026-11-08 |
keep-alive 20m + hive backup-роль qwen3.6:35b-128k-fit
...
Сессия navi Eris 2026-11-08: (1) OLLAMA_KEEP_ALIVE изменён -1 → 20m на .129 по решению владельца, подтверждено в environ живого процесса (MainPID 178780), бэкап override.conf.bak-rei-20261108-keepalive создан, cmp OK; (2) hive-конфиг v3 (PUT /config admin-ключом, hash sha256:2d099501…) назначил qwen3.6:35b-128k-fit backup-моделью route default (target home=192.168.1.129:11434) для всех профилей и всех 8 инстансов (ayame, Eris, Kael, melody, pilar, Proxy, tsukiko, zoe) — ack applied, restart_required пусто, на Eris llm применился живьём.
|
Добавлена приёмка под нагрузкой (тесты 3/2/6 параллельных — все ok, VRAM стабильна) и gotcha thinking-модели в док gmikcon-pc
...
Вторая волна приёмочных тестов 2026-11-08: стресс 3/2/6 параллельных запросов — без деградаций; добавлен gotcha thinking-модели (поле thinking, num_predict, eval_rate=0).
|
Добавлен хост gmikcon-pc (192.168.1.129) в инвентарь
...
Хост gmikcon-pc (192.168.1.129) добавлен в инвентарь по итогам сессии 73130412 «Оптимизация ollama под Qwen MoE»: Arch, Ryzen 5 3600, 62 GiB RAM, RTX 3090 24GB; Ollama 0.40.15, MoE-offload qwen3.6:35b-128k (N_CPU_MOE=19, KV q8_0, NUM_PARALLEL=3, ctx 131072, GPU_OVERHEAD 3 GiB), приёмочный тест 3 параллельных запроса пройден, single-stream 61.3 tok/s.
|
Добавлена секция Ollama MoE-инференс: версия 0.40.15, fit-модель, override.conf, N_CPU_MOE=19, результаты 61.3 tok/s и приёмки 3×128k
...
Итоги сессии 73130412 «Оптимизация ollama под Qwen MoE»: обновление до 0.40.15, fit-модель, override.conf, распределение N_CPU_MOE=19, результаты приёмочного теста.
|
Внешняя доступность сервисов задеплоена (171885e): env+public_url, пробы 7/7 up; в Synapse тип service/external_down + dispatcher; исправлены SSH-креды и IP прокси (.226→.183)
...
Доработка владельца задеплоена navi Eris 2026-11-08 (pull f14fb26..171885e + rebuild, пробы 7/7 up); приёмка и маршрут уведомлений в Synapse расширены и проверены E2E; SSH-строка дока ссылалась на несуществующий секрет — исправлено по факту входа; флот-таблица устарела (.226 → .183)
|
Split-horizon DNS on pfSense: apex Host Override + 17 explicit local-data records, verified live
...
New doc 10-systems/traffic-routes/split-horizon-dns-pfsense.md: apex via UI Host Override, 17 explicit local-data records in Custom Options (wildcard local-data verified not to match on this build), target 192.168.1.183, live verification 17/17 DNS + 16/17 HTTP (transcribe-demo 502, pre-existing), minecraft TCP 25565 ok, public zone unchanged.
|
Fix stale internal proxy LAN address .226 → .183 in Known Hosts and add external VPS public IP
...
Cross-check against 10-systems/servers/internal-proxy-vps.md (ssh-verified 2026-10-25): after the 2026-10-16/17 blackout the internal proxy VM moved from DHCP .226 to netplan-static .183; .226 no longer responds. The Known Hosts line still said .226 — a wrong target for the planned pfSense Host Overrides (gntodo 456).
|
| 2026-11-07 |
jellyfin: 2026-11-07 scan episode (ReadTimeout 30.5s + cpu_high 91%) triaged; Trickplay trigger still cleared; new recs: night-hour scan triggers, NetTera8 100% full
...
Episode 2026-11-07 triaged by rei via SSH (cpu_high 91%, load 77, 12 ffmpeg during library scan 17:21 UTC; task left to finish; Trickplay trigger verified still cleared; NetTera8 100% full)
|
ovpn #38 incident 2026-11-07: owner-ordered VM shutdown during blackout, verified healthy
...
ovpn-egress-vm: 2026-11-07 blackout incident — root cause owner-ordered virsh shutdown of all VMs (not power loss); post-boot health verified; socat unit name documented; panel alerts were delayed batch.
|
| 2026-11-06 |
gnexus-home: инцидент 2026-11-06 server_offline flap — первопричина IP-конфликт на .151 с фантомным WiFi-MAC 0e:c6:7d:38:8e:78; статические ARP-записи на .183/.176/госте, сайт восстановлен (502 → 200)
...
Реакция на событие hard-panel server_offline (server 42): установлен первопричина flap — IP-конфликт на 192.168.1.151 с фантомным WiFi-MAC; применены временные static-ARP записи, восстановлен публичный сайт.
|
| 2026-11-05 |
Update intel-nuc-shome inventory: SSH-verified hardware details
...
Applied pending inventory change 20261105231824-76ccaaa8 for intel-nuc-shome, verified via SSH dmidecode/lspci/lsblk by navi rei.
|
| 2026-11-04 |
Добавлен gotcha в bugtrail.md: после ребута хоста docker-стек поднимается ~15–20 мин, healthz может отдавать 502 — ждать перед эскалацией
...
Инцидент 2026-11-04: после массового ребута инфры healthz bugtrail отдавал 502 ~15–20 мин (docker daemon стартует позже хоста); verified rei — стек поднялся сам, gotcha задокументирован, чтобы следующая реакция не эскалировала преждевременно.
|
Добавлена секция инцидента 2026-11-04 (блэкаут стенда, волна server_offline ×18): хронология, верификация 10:16 UTC, вывод — самовосстановление. Kernel → 5.15.0-202, last_reviewed → 2026-11-04.
...
Реакция rei на hard-panel server_offline #46 (envelope 9b290f76, silent 353 мин): инцидент 2026-11-04 верифицирован, канонический отчёт в workspace navi-1; по конвенции книги хост-док получает датированную секцию; kernel/last_reviewed обновлены по live-данным.
|
bugtrail.md: add power-loss behaviour section from 2026-11-04 blackout (self-recovery, ~15 min health lag, verified by rei)
...
Observed during rei's reaction to the 2026-11-04 infra blackout (event adff2112, gnauth): stack self-recovered after power loss with ~15 min health lag; documenting expected behaviour so future reactions don't force-rebuild.
|
Дубли починены (mic-gate, drop-филты, hangover 500); прод: /tts/generate+пин+clean+seed42; v1 неразборчив — откат; GPU Xid56→ребут, boot129.sh
...
Итоги сессии: фикс дублей (эхо-петля), GPU-инцидент Xid56 + ребут, определение прод-профиля TTS (v1 неразборчив, откат на /tts/generate), boot129.sh
|
Разнос TTS лечён: /tts/generate, профиль temp 0.6/topk 100/min_p 0.15/seed 42 + guard; E2E чист
...
Диагноз жалобы владельца «в озвучке шумы»: разнос TTS при /v1/audio/speech. Источник: стресс-тест 12 генераций + Whisper-контроль 2026-11-04.
|
Обновить gmikcon-pc.md: RTX 3090 24GB, локальный Ollama (gemma4:e2b, qwen3.6:35b), оркестратор WS :9199 (VAD→STT→LLM→TTS, E2E_OK), демо-UI :9090, gotchas (LD_LIBRARY_PATH, setuptools<81, faillock)
...
Дополнение по итогам 2026-11-04: STT/оркестратор/демо-UI/локальный Ollama. Источник: ssh-host.
|
Добавить карточку хоста gmikcon PC (192.168.1.129): Arch-станция, zonos2.cpp TTS v0.5.2 на :1919, клонирование голоса, SSH-креды в gnexus-creds, верификация 2026-11-04
|
| 2026-11-03 |
Add gstation-tts node doc: pipeline layout, 7 verified traps, v8 run book and CER baseline.
...
Session 2026-11-03 verified live: HF_HOME trap broke two training launches; sparse blob, NUL logs, NFS-after-reboot and /tmp-wipe traps all hit in one night. Persisting the run book to gnexus-book so future sessions don't rediscover them.
|
| 2026-10-31 |
gitbucket.md: 9th crawler episode (31.10 03:30 UTC) + correction of ep8 timeline (one continuous ~3.5h crawl, ~127k req)
...
9th crawler episode on gitbucket (GHard #1152, 2026-10-31 03:30 UTC, cpu 97.1%): 4x GET /root/smart-home-server/archive/devices/*.zip (3x sha 44250a7a, 1x 8a945d0d), 8x EofException/Broken pipe, java 333.7% CPU (~3 ZIP builds), load peak 7.05, no restart, self-recovered 03:31:51. Correction: ep8 did not self-extinguish 00:09 — enumeration continued ~440-480/min through 00-02 (29k req/h), one continuous crawl 23:58-03:35+, ~126.8k req total, client 192.168.1.100% 192.168.1.150, GPTBot/1.4 + Chrome/Edge 142-145 rotation. Triage: read-only ssh ubuntu@192.168.1.156 (bserv@192.168.1.130 creds) and nginx access.log on 192.168.1.183. Incident journal: workspace/incident_gitbucket_2026-10-31_ep9_crawl.md.
|
gstation: новый хост — семейный игровой ПК на Bazzite (192.168.1.135)
...
Новый хост в инвентаре (hosts/gstation) + новый док 10-systems/hosts/gstation.md. Факты из SSH-сессий 2026-10-31: Bazzite 44.20261026, i5-3470/19GB/RX 580 2048SP, HDD NTFS /var/mnt/games, статический IP .135, GDM autologin, отключённые lock/hot corners/hotedge/welcome-тур (+welcome-fix autostart), ТВ underscan через GNOME underscanning (tvmode/tvtest), 35 игр через play.sh + Heroic couch, геймпад RZTK спарен (Home+X).
|
gitbucket: 8th archive-burst episode (30.10 23:57 UTC), nextcloud 61-s blip correlation confirmed twice, pending-decision counter updated
...
8th archive-burst episode 2026-10-30 23:57 UTC (panel #1136/#1137, reaction events 63c2c371 + 34367587): added episode to the list, recorded the 2nd confirmed 61-s nextcloud blip coincidence (27.10 + 30.10), updated pending-decision counter to 5 consecutive days (26–30 Oct). Sources: SSH triage on 192.168.1.156, nginx access.log on 192.168.1.183, hard-panel service_incidents.
|
gitbucket host doc: 8th archive-burst episode 2026-10-30 23:57 UTC (short burst, self-recovered ~7 min), GPTBot/1.4 crawl attribution, pending owner decision now 5 consecutive days
...
Episode 8 of the archive-burst class confirmed 2026-10-30 23:57 UTC (panel event #1136, diagnosed by navi rei via SSH + nginx logs): short burst of 20 archive/*.zip requests amid GPTBot crawl, peak load1 11.25, self-recovered ~7 min. Updates episode count, traffic-source note and pending owner decision (5 consecutive days).
|
gitbucket: 8th archive-crawler CPU episode (30.10 23:57 UTC), series 5 consecutive days
...
8th episode of the archive-crawler CPU class: 2026-10-30 23:57 UTC burst (cpu_high 91% #1136, load_high 2.11×cores #1137, cpu_recovered #1140), confirmed via read-only SSH journalctl on .156 — same /archive/*.zip signature, self-recovered in ~3.5 min. Series now 5 consecutive days (26–30 Oct).
|
| 2026-10-30 |
Document hpsa controller reset failure mode (DL380 G6) with 2026-10-30 incident
...
Reaction session: load_high on files (#36) and 5 other VMs traced to hpsa controller logical-volume resets on bserv (DL380 G6). Incident 2026-10-30 13:06-13:15 UTC, self-recovered. Documented signature, timeline, triage playbook.
|
Добавлен хост fuckjira (OpenProject 17, 192.168.1.175) в inventory hosts
...
Реакция на событие hard-panel container_added (bd255c26, openproject-web-1, server_id 29). Проверено: контейнер Up 13 days (healthy), стек OpenProject 17 цел; removed/added 13:11:41-42 UTC — артефакт ре-регистрации инвентаря панели на фоне load-всплеска 13:10-13:12 (hive ReadTimeout), реального рестарта не было (uptime непрерывный). SSH-доступа к хосту в gnexus-creds нет.
|
| 2026-10-29 |
gitbucket: 7th archive-crawl episode (29.10, ~4.5 h sustained, self-recovered)
...
Reaction session rei on GHard event #1100 (cpu_high 90.5%, server 44 gitbucket, 2026-10-29 10:40:57 UTC). SSH triage confirmed the known archive-crawl class (7th episode since 2026-10-12) with a new signature: sustained ~4.5 h crawl 06:00–10:42 UTC, 5248 /archive/*.zip requests from 192.168.1.150 via public route, 29 Broken pipe/EofException errors, load decayed 5.07→0.66 by 10:48 UTC, self-recovered, no restart needed. Known behavior section updated with 7th episode, new signature, and owner recommendation (4 consecutive days of episodes).
|
| 2026-10-28 |
gitbucket: corrected archive-burst section — 6th episode of class, client attributed, proxy is .183 not .226
...
Correction after cross-checking memory of prior sessions: the 28.10 episode is the 6th of the /archive/*.zip class since 12.10; client already attributed (192.168.1.150 via ovpn VM SNAT, browser UA rotation; crawler UAs in 25.10 storm); actual nginx proxy is 192.168.1.183 (inventory .226 entry stale — .226 does not answer SSH/ICMP). Removed the incorrect "client IPs not identified" statement.
|