diff --git a/40-inventory/hosts.yml b/40-inventory/hosts.yml index d2ccf3d..8578708 100644 --- a/40-inventory/hosts.yml +++ b/40-inventory/hosts.yml @@ -150,18 +150,29 @@ status: active type: server user: bserv - notes: 'AI services deployed 2026-09-20 (owner-approved): (1) Ollama 0.34.2, systemd - override /etc/systemd/system/ollama.service.d/override.conf: OLLAMA_HOST=0.0.0.0:11434, - FLASH_ATTENTION=1, KV_CACHE_TYPE=q8_0, CONTEXT_LENGTH=49152, MAX_LOADED_MODELS=1, - KEEP_ALIVE=-1; model qwen3.5:2b-q4_K_M (1.9GB, vision, 201 langs), use think:false - in requests; ~13 tok/s. (2) whisper.cpp built from /opt/whisper.cpp with CUDA - sm_50 (required gcc-10 host compiler fix for nvcc 11.5); systemd unit whisper-server.service: - whisper-server --host 0.0.0.0 --port 8008 -m models/ggml-large-v3-turbo-q5_0.bin - -t 4 -l auto -fa --vad -vm models/ggml-silero-v5.1.2.bin; HTTP POST /inference - multipart file=@; auto language detect, ru/uk verified. VRAM budget: ollama 2726MiB - + whisper ~855MiB = 3581/4096 MiB resident. Driver upgraded 535->580 live (rmmod/modprobe, - no reboot, 26 VMs unaffected). OOM fallback ladder: turbo->small q5_1, KV q8_0->q4_0, - ctx 48K->32K.' + notes: 'SSH access to the hypervisor: user bserv, password getroot88 (sudo same); + Navi ssh_exec cannot do password auth without a named connection in ssh_hosts.json + (none exists) — connect via `sshpass -p getroot88 ssh -o StrictHostKeyChecking=no + bserv@192.168.1.130`, sudo via `echo ''getroot88'' | sudo -S virsh ...`. Users + ubuntu/root do NOT work on the hypervisor. Full recipe: 10-systems/hardware/hp-proliant-dl380-g6.md. + AI services deployed 2026-09-20 (owner-approved): (1) Ollama 0.40.13, override.conf: + OLLAMA_HOST=0.0.0.0:11434, FLASH_ATTENTION=1, KV_CACHE_TYPE=q8_0, CONTEXT_LENGTH=49152, + MAX_LOADED_MODELS=1, KEEP_ALIVE=-1; resident qwen3.5:2b-q4_K_M, think:false, ~11.3 + tok/s. (2) whisper.cpp CUDA sm_50, whisper-server.service :8008, large-v3-turbo-q5_0, + ru/uk verified. EXPERIMENTS 2026-11-08 (navi Eris): lfm3-omni:2b (on disk) — text + 11.0 tok/s, ASR 12.6s vs whisper 8.4s, vision weak, TTS CUDA OOM (voices female/male, + no ru speech) — not viable on 4 GB. lfm3-vl:2b-q4_K_M (on disk, 1.5 GB) — RU A/B: + 16-17 tok/s (+40%), vision 3.8s vs 29.4s no object hallucinations, JSON cleaner, + strict IF failed for both. LANG TEST 2026-11-08 (5 tasks ru/uk, /tmp/bench_lang_*.txt): + languages officially — qwen3.5: 201 langs incl. uk+ru, published MMMLU 63.1 / + INCLUDE 55.4 / MMLU-ProX 52.3 / WMT24++ 45.8 / MAXIFE 60.6 (thinking); LFM3 family: + only 16 langs, RU yes, UK absent, no published multilingual knowledge evals. Live: + ru — qwen clean, lfm3-vl code-switching ("compression", "важна баланс", "бэков"); + uk — qwen fluent with minor slips, lfm3-vl grammatical but broke uk-to-ru translation + into garbled text; grammar-fix task lfm3-vl caught real error (згідно наказу -> + згідно з наказом), qwen failed it. Verdict: for ru/uk service LLM qwen3.5 stays + resident; lfm3-vl is disk option for vision/structured tasks. Resident decision + confirmed by owner via language criteria.' ram_gb: 120 gpu: NVIDIA Quadro K1200 (GM107GL, Maxwell sm_50, 4GB GDDR5, driver 580.178.04 non-open, CUDA 13.0 runtime); secondary AMD ES1000 embedded (server display only)