|
llm: routes that name a host, a backup and a window, and a profile that reads them
`llm.json` replaces `OLLAMA_BACKENDS_FILE`. It has targets (host and key environment variable) and routes, where a route is an ordered chain of `model @ target` entries, each with its own `context_window` and options. The `default` route now starts on `glm-5.3-flash:cloud` and backs it with `qwen3.6:35b-128k` on the home host, both at 131072 tokens, so a failover no longer changes the budget the compressor and `num_ctx` are accounted against. Embeddings get their own `embedding` route instead of a pinned host. The engine classifies a failure (network, quota, auth, server, missing model), cools that entry and moves the call down the chain; a background probe watches only degraded targets and lets the next turn return to the primary once it is healthy — it never interrupts a running turn. A profile joins a route with `llm_route`, and the loader now actually reads it: `llm_route` and `context_window` were declared on `AgentProfile` and written into every `config.json` while the loader skipped both, the same silent no-op that `is_admin_only` used to be, so every profile ran the legacy model list at the global window. The writer emits them too, or an admin edit would erase the route from disk. The turn's resolved entry travels to the UI: the first chunk is stamped with model and target, `model_info` carries `fallback` when a backup answered, and the chat header shows a chip only then. `serving_primary` compares the target as well as the model name, because a route may run one model on two hosts. `/health` gains an `llm` block with the active entry, target states and missing models. `deploy/ollama/` keeps the local model's definition honest: MTP off (slower at every context on this host), `presence_penalty 0` (the base model's 1.5 breaks agentic tool calls), and `num_gpu 40` — Ollama's automatic fit claims a 128k f16 slot fits and then dies mid-prefill with CUDA OOM; 40 of 42 layers leaves ~400 MiB and decodes at ~77 tok/s, 38 layers collapses to 48, and 39 OOMs. q8_0 KV measured 37 tok/s against 70 on f16 here, so the cache stays f16. |
|---|
|
|
| .env.example |
|---|
| .env.navi_code.example |
|---|
| deploy/ollama/Modelfile.qwen-128k 0 → 100644 |
|---|
| deploy/ollama/README.md 0 → 100644 |
|---|
| docs/api.md |
|---|
| docs/architecture.md |
|---|
| docs/config.md |
|---|
| docs/mechanics.md |
|---|
| docs/profiles.md |
|---|
| llm.json 0 → 100644 |
|---|
| navi/api/routes/admin.py |
|---|
| navi/api/routes/health.py |
|---|
| navi/config.py |
|---|
| navi/core/agent.py |
|---|
| navi/core/container.py |
|---|
| navi/core/events.py |
|---|
| navi/core/registry.py |
|---|
| navi/core/subagent_runner.py |
|---|
| navi/exceptions.py |
|---|
| navi/llm/base.py |
|---|
| navi/llm/config.py 0 → 100644 |
|---|
| navi/llm/fallback.py |
|---|
| navi/llm/health.py 0 → 100644 |
|---|
| navi/llm/ollama.py |
|---|
| navi/memory/_embeddings.py |
|---|
| navi/profiles/assistant/config.json |
|---|
| navi/profiles/base.py |
|---|
| navi/profiles/designer_3d/config.json |
|---|
| navi/profiles/developer/config.json |
|---|
| navi/profiles/discuss/config.json |
|---|
| navi/profiles/dispatcher/config.json |
|---|
| navi/profiles/loader.py |
|---|
| navi/profiles/modeler_3d/config.json |
|---|
| navi/profiles/navi_code/config.json |
|---|
| navi/profiles/server_admin/config.json |
|---|
| navi/tools/_internal/base.py |
|---|
| tests/unit/api/test_health.py |
|---|
| tests/unit/core/test_agent.py |
|---|
| tests/unit/core/test_registry.py |
|---|
| tests/unit/llm/test_config.py 0 → 100644 |
|---|
| tests/unit/llm/test_fallback.py |
|---|
| tests/unit/llm/test_health.py 0 → 100644 |
|---|
| tests/unit/llm/test_model_stamping.py |
|---|
| tests/unit/llm/test_ollama.py |
|---|
| tests/unit/memory/test_store.py |
|---|
| tests/unit/profiles/test_profile_loader.py |
|---|
| webclient/dist/assets/index-Bf7mBiaD.js 100644 → 0 |
|---|
|
Not supported
|
| webclient/dist/assets/index-DPNYPAd0.css 0 → 100644 |
|---|
| webclient/dist/assets/index-WziZgnPy.js 0 → 100644 |
|---|
|
Not supported
|
| webclient/dist/assets/index-uqZyYvXb.css 100644 → 0 |
|---|
| webclient/dist/index.html |
|---|
| webclient/dist/sw.js |
|---|
| webclient/src/components/chat/ChatHeader.vue |
|---|
| webclient/src/components/ui/ModelChip.vue 0 → 100644 |
|---|
| webclient/src/composables/useWebSocket.js |
|---|
| webclient/src/i18n/messages/en.js |
|---|
| webclient/src/i18n/messages/ru.js |
|---|
| webclient/src/i18n/messages/uk.js |
|---|
| webclient/src/stores/chat.js |
|---|
| webclient/tests/unit/stores/chat.test.js |
|---|