memory: a fact written under a new spelling of its key merges into the one that survives ...

The extractor invents a key every time it meets the same fact — `language`,
`preferred_language`, `ui_language` — and the upsert keys on the exact string, so
the base grew several spellings of each durable fact. On prod, 24 facts across 11
clusters share a key once noise prefixes are dropped, 10 of them under the exact
same value.

upsert_fact(dedupe=True) now compares the *tokens* of the normalized key — set
equality, not overlap, and never the value: `ui_language` and `language` are one
key while `dark` and `dark_ui` are two. The row already holding the writer's exact
key wins; otherwise the incoming one does. Losers are marked, not deleted: each
keeps its key and its value and gains `merged_into`, so the merge is reversible
with one UPDATE. Off by default, so the memory tool still writes exactly the row
it asked for.

Every read path filters `merged_into IS NULL`, and the mark is reachable only
where it is meant to be — get_all_facts/fact_count gained include_merged, and
GET /admin/memory?include_merged=true exposes it. search_facts deliberately has
no such flag: the injection path must not be able to see a hidden row. Forgetting
a key drops its hidden aliases first, so none is left pointing at nothing.

Three fixes that fell out of the same read:

- expires_at was overwritten with EXCLUDED.expires_at on every upsert, and the
  extractor always passes None — so it silently cleared any expiry the agent had
  set. Both DO UPDATE clauses now COALESCE.
- The ILIKE fallback was the one read path that served expired facts.
- The `list` action promised "all stored facts" and returned categories. The code
  was right and the description was wrong; the description now says so.

The extraction prompt stopped allowing infrastructure — servers, hosts, addresses,
running services — while the agent and the planner already forbade it; that
loophole is where 423 technical facts came from. It also forbids secrets outright,
and asks for stable snake_case keys. Expiry is now documented as being for facts
with a known end, not for facts you merely doubt.

Facts reach the prompt with their age ("updated 34d ago") alongside confidence:
nothing expires on a clock, so the injection says how old a fact is and lets the
agent weigh it.

mcp: every wait on the MCP path gets a bound, and a failed reload keeps what it had ...

reload_tools kept dying on prod, and only a restart cured it. Nothing on the
MCP path was bounded: ClientSession was built without read_timeout_seconds, so
the SDK waited forever (anyio.fail_after(None)); _send awaited a future with no
timeout; disconnect() awaited the shielded runner with no timeout; and
disconnect_all() walks the pool one client at a time, so a single wedged client
blocked every reload that came after it. A cancelled caller only cancelled
itself — the runner stayed wedged, which is why it was every reload, not one.

Bounds, in two deliberate layers. The SDK read timeout ends a request nobody
answers *inside* the runner, so the client stays usable; the caller-side bound
is the backstop for an op queued behind a stuck one, and it aborts the runner
rather than leaving it for the next command to queue behind. connect 60s,
list_tools 45s, call_tool 300s (SDK 240s), session read 30s, disconnect 15s,
each overridable, each logging the culprit.

disconnect() and disconnect_all() no longer swallow CancelledError — that is
exactly how the reload budget stopped being enforceable.

The registry is now rebuilt in one step: collect every server's tools first,
then commit the whole set in a single synchronous swap. clear_external_tools()
ran *before* the rebuild, so a rebuild that failed halfway left the model
holding MCP tool names that resolved to nothing. A failed pass now leaves the
previous registrations untouched, and the reload phase carries its own budget
so the tool always returns a report.

Two more defects fell out of the audit. The streamable_http transport was
handed a bare httpx.AsyncClient, whose 5s default read killed the GET stream of
an idle server — six of the nine prod servers are streamable_http. And
register_mcp_tools called log.warning() in a module where log was never
defined, a NameError on precisely the failure path it was meant to report.

profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

admin: POST /admin/tools/reload ...

reload_tools is granted to one profile only (tool_developer), so an admin
whose profile lacks it cannot reload at all — the tool answers "not
found" instead of reloading. The route calls the same reload_all() the
tool does, behind require_admin, for whoever is logged in as an admin.

An admin may read: ok, the tools loaded, the registry total, per-file
errors, names in enabled.json nothing answers to, and the MCP/providers
summary.

profiles: retire gate flags and retrain prompts for the plan tool ...

- drop planning_enabled / planning_mandatory / planning_phase2_enabled /
  observe_skips_plan_enabled / adaptive_replan_enabled from AgentProfile,
  the loader and admin serialization
- add `plan` to native tools of developer, navi_code, tool_developer,
  secretary, server_admin, modeler_3d — without the gate they would
  otherwise lose planning entirely
- system prompts: call plan for non-trivial multi-step work before
  execution; complex plans get presented for confirmation; stuck or goal
  changed -> plan with a reason

Автономность мелких моделей: OUTPUT DISCIPLINE, milestone-todo, перехват финала хода ...

Два последовательных захода над одной проблемой — мелкие модели (12–30B) в navi_code
теряют автономность на трудных шагах: «остановился поболтать» вместо действия и
слишком большое расстояние между пунктами плана.

Заход 1 (З1–З3):
- З1: OUTPUT DISCIPLINE в системном промпте navi_code — «act, don't announce»,
  без few-shot антипримеров. Контракт хода: ответ без tool_calls = конец хода, поэтому
  объявление намерения текстом убивает автономность; правила заставляют вызывать
  инструмент в том же ходе.
- З2: плоский todo + метка группы milestone + декомпозиция. _parse_plan_steps
  возвращает list[tuple[milestone, text]]; milestone — метка группировки (не сущность,
  без статуса), «done» вычисляется при рендеринге; подшаги = больше плоских шагов
  (без вложенности). TUI side-panel группирует по milestone (плоский фолбэк при пустом
  milestone). Plan depth: max 15→20 + правило декомпозиции.
- З3: adaptive re-plan «длинный шаг» — nudge «разбей шаг» при in_progress ≥ порога
  итераций без смены todo (порог 4, раньше общего anti-stall warning на 8).

Заход 2 (шаги 1–3, после cloud-теста 31b vs 12b):
Корневая структурная причина: весь спасательный механизм (anti-stall warning с явным
предложением reflect, adaptive re-plan) живёт только внутри tool-цикла — nudge
инжектируется в pre_turn *следующей* итерации, которой при «остановился поболтать»
нет (ход закрылся по return до post_turn). 31b застревала через «продолжаю
tool-итерации» → дожала до warning → спаслась; 12b — через «замолчала текстом» → мимо
всех nudge.

- Шаг 1: перехват финала хода. Если модель выдала bare-text, но в todo есть открытые
  шаги (pending/in_progress) и лимит не исчерпан — НЕ эмитить StreamEnd, а сохранить
  ассистентский текст в session.context, поставить системный nudge и continue
  (без StreamEnd, без workers — консистентно с multi-iteration tool-турами). Счётчик
  final_interceptions на AgentTurnContext, лимит final_intercept_limit (default 2),
  эскалация жёсткости (мягкий → «second stop»). has_open_steps в todo.py: пустой
  todo → False (защита casual-сообщений), failed/skipped терминальны. Профильные
  флаги final_intercept_enabled/limit в base.py + loader + admin.
- Шаг 2: жёсткий reflect-триггер в промпте — «~3 tool attempts on the same step
  without progress → call reflect IN THIS TURN (tool call, not reasoning aloud)».
- Шаг 3: открыть replan для застревания — «call replan when reflect showed the whole
  approach is dead (not one failed step, but the approach itself)».

Тесты: 874 passed, 1 skipped. Новые — has_open_steps (5), final intercept (5),
milestone-группировка, adaptive long-step nudge, парсер шагов с milestone-маркером.

cloud: navi_code model → gemma4:31b-cloud для тестирования догадки (31b признала
застревание, 12b — нет); .env cloud-host уже gitignored.

Co-Authored-By: Claude <noreply@anthropic.com>

agent: bounded autonomy — scope boundary + observe-vs-act ...

navi_code had unwanted "free flight": an observe request ("look at a
directory") triggered the full Phase 3 plan with milestones + auto-todo,
and goal_anchoring then drove the agent to finish those steps, climbing
into sibling projects and executing milestone docs it found.

Two toggleable, default-off profile flags (on for navi_code):

- scope_boundary_enabled: injects a standing system message keeping the
  agent within the literally requested scope; forbids acting on
  discovered TODO/roadmap/milestone docs (report only).
- observe_skips_plan_enabled: Phase 1 classifies MODE: observe|act; an
  observe request skips Phase 2/3 — no multi-step plan, no auto-todo, no
  "execute step by step" prompt. The agent just gathers info and answers.
  Independent of force_plan (observe on the first message still skips).

Free flight stays reproducible by flipping both flags off.

Co-Authored-By: Claude <noreply@anthropic.com>

Fix MCP names and profile API format consistency ...

- Replace stale mcp__navi_web__* / mcp__navi_3d__* names with canonical
  mcp__navi-web__* / mcp__navi-3d__* across prompts and key_tools.
- Update /agents/profiles and /admin/profiles endpoints to expose
  tools.agent / tools.subagent instead of deprecated enabled_tools fields.
- Update docs/mechanics.md to reference the new tools structure.
- Archive stale docs/visual.html.

Co-Authored-By: Claude <noreply@anthropic.com>

Fix 19 issues found in full codebase review ...

Backend:
- Stop session auth bypass: require auth for owned sessions, reject anonymous with 401
- upload_file: stream chunks directly to disk instead of buffering in RAM
- MCP config: validate name against path traversal regex
- auth deps: cleanup stale refresh locks periodically
- auth routes: expire mobile auth states after 10 min to prevent unbounded growth
- compressor: meta-summarize existing summaries before compression; preserve assistant content when tool_calls present; rewrite hard_truncate to keep whole turns
- orchestrator: configurable WS replay buffer size; async cleanup/remove_websocket/clear_busy; fix run_recall ContextVar order to avoid deadlock on _build_agent failure; await cleanup in finally
- agent: persist image_msg in session.messages; remove archived messages from session after archive; remove duplicate StreamStopped yield on tool stop
- websocket: try/except around create_task with cleanup on failure; await remove_websocket

Frontend:
- App.vue: hashchange listener lifecycle in onMounted/onUnmounted
- MessageList.vue: passive scroll, flash timeout cleanup, archive scroll snapshot
- InputBar.vue: 300 ms debounce on draft save to localStorage
- SessionList.vue: remove :key from DynamicScroller to avoid remount jitter

Tests: 422 passed, 1 skipped

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix broken MCP admin endpoints after deps refactor ...

Replace stale `_mcp_manager` module-level import with `get_mcp_manager()`
calls in admin endpoints. The module-level variable was removed during the
AppContainer refactor, causing 500 errors on:
- POST /admin/mcp/{server}/reconnect
- GET /admin/mcp/status
- POST /admin/mcp/test

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Replace global lazy singletons with explicit AppContainer + lifespan ...

- Create navi/core/container.py with AppContainer dataclass and create_container()
- Rewrite navi/api/deps.py: remove module-level singletons, add _container global
  fallback + set_container(), use _resolve_container() for all getters
- Replace @app.on_event with @asynccontextmanager lifespan in main.py
- Update routes to use Depends(get_scheduler) instead of calling get_scheduler()
- Fix FastAPI body parsing bug: remove dataclass parameters from Depends getters
  (FastAPI was treating AppContainer sub-dependencies as Body params, forcing
  embed=True on all endpoint body params and causing 422 errors)
- Update websocket.py to use _resolve_container() instead of get_registries()
- Update integration test fixtures to build AppContainer and call set_container()
- Remove obsolete tests/unit/test_startup.py (tests removed _on_startup)
- Fix test_scheduler_loop.py fixture (get_registries no longer exists)

All 387 tests pass (excluding websocket hang tests).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

feat: admin API endpoints for individual MCP server CRUD ...

Add granular REST endpoints for managing single MCP servers:
- GET  /admin/mcp/config/{server_name}
- PUT  /admin/mcp/config/{server_name}
- DELETE /admin/mcp/config/{server_name}

Bulk endpoints remain for backward compatibility.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add MCP admin endpoints for config, reconnect, status, test, and profile group management ...

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix ollama_backends / FallbackOllamaBackend issues ...

- registry.py: always use FallbackOllamaBackend (unified backend).
  Enables model priority lists in all deployments, not just multi-server.
- agent.py: add missing think=profile.think_enabled to run() (REST endpoint).
- compressor.py: fix model param type (str → list[str] | str | None).
- fallback.py: harden load_servers_from_file against missing/bad JSON files
  and entries without host. Add clear_blacklists() for manual reset.
- admin.py: add POST /admin/ollama/clear-blacklists endpoint.
- tech_debt_review: document dead stream() methods.
- tests: add tests for single-server fallback, bad file handling,
  missing host skipping, and blacklist clearing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add profile editing to admin panel ...

Backend:
- navi/profiles/loader.py: add save_profile_to_dir() to write config.json
  and system_prompt.txt back to disk
- navi/core/registry.py: add ProfileRegistry.update() for in-memory updates
- navi/api/routes/admin.py: new endpoints GET /admin/profiles/{id} and
  PUT /admin/profiles/{id} for reading and saving full profile config

Admin panel:
- Profiles table: add Edit button per row
- Drawer form with sections: Basic, Model & Generation, Thinking,
  Planning, Sub-agent, Tools, System Prompt
- All fields editable inline (text inputs, checkboxes, textareas for lists)
- Save via PUT request, updates both disk and in-memory registry without
  server restart

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add per-user sessions and memory filters in admin panel ...

Backend:
- GET /admin/memory now accepts user_id query param to filter facts
  for a specific user instead of returning the global view.

Frontend:
- Users table now has "Sessions" and "Memory" buttons per row.
- Clicking "Sessions" switches to Sessions tab and pre-fills search
  with the user id.
- Clicking "Memory" switches to Memory tab and passes user_id to the
  API, showing only that user's facts.
- Manual tab clicks reset filters (search / userId) so the view
  returns to unfiltered state.

Docs:
- docs/api.md updated with user_id param for GET /admin/memory.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix admin memory showing 0 facts; add pagination/search/sort ...

Bug fix:
- get_all_facts(user_id=None) was filtering for user_id IS NULL only,
  so admin panel showed 0 facts when facts had user_ids set.
- Added all_users parameter to get_all_facts and fact_count.
- Admin endpoint now passes all_users=True to return all facts.

Memory pagination:
- Extended get_all_facts with offset, search, sort_by, sort_order params.
- Extended fact_count with search and all_users params.
- /admin/memory endpoint now accepts limit, offset, search, sort_by,
  sort_order and returns {total, limit, offset, items}.

Admin panel frontend:
- Added server-side search with debounce for memory facts.
- Added sortable column headers (category, key, source, confidence, updated).
- Added pagination controls (prev/next, page size selector).
- Switched memory display from grouped tables to flat sortable table.

Tests:
- Added unit tests for get_all_facts(all_users=True) and
  fact_count(all_users=True).
- All 219 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add pagination, search, and sorting to admin sessions ...

Backend:
- Add count_all and search_list abstract methods to SessionStore
- Implement count_all and search_list in PgSessionStore (SQL with ILIKE)
- Implement count_all and search_list in InMemorySessionStore
- Update /admin/sessions to accept limit, offset, search, sort_by, sort_order
- Return {total, limit, offset, items} from /admin/sessions

Frontend:
- Add search input for sessions in admin panel
- Add clickable sortable column headers with asc/desc toggle
- Add pagination controls (prev/next, page size selector, item count)
- Debounce search input (300ms)

Tests:
- Add integration tests for pagination, offset, search, and sorting
- All 217 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Multi-user auth via gnexus-auth OAuth + hybrid role/permission model ...

- Integrate gnexus-auth-client-py (GAuthClient) for OAuth flow, token refresh,
  and webhook parsing
- Add navi/auth/ package: User model, Fernet encryptor, client singleton,
  deps (get_current_user, require_admin, require_permission)
- New tables: navi_users, user_auth_sessions (auto-created on startup)
- Session/memory isolation by user_id with legacy NULL support
- Cookie-based auth proxy: /auth/login, /callback, /logout, /me
- Webhook receiver /webhooks/gnexus-auth handling user events, global logout,
  session revocation, role/permission changes
- Admin endpoints (/admin/*) gated by role + permissions
- Webclient auth store with isAdmin/hasPermission guards
- Admin-only profile filtering in /agents/profiles
- 200/200 tests passing

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>