profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

tools, mcp: a description is a hook, not a manual ...

With 24 tools in a profile the schemas are the largest fixed block of every
request, and they are paid for whether or not the tool is called. Eleven of
them carried a manual's worth of prose in their `description` — examples,
error tables, "common mistakes", the mechanics of the file areas. Reading
them against the manuals first, the manuals already held all of it, and
richer: this was duplication sitting in the one place the model cannot avoid
reading, not knowledge that had nowhere else to live. So the descriptions
became selection hooks — what the tool is for, when to pick it over its
neighbour, and the one rule that cannot wait — and the detail moved to
tool_manual("<tool>"), which costs nothing until the tool is actually in
play.

filesystem, spawn_agent, schedule_recall, manage_recall, scratchpad, plan,
share_file, content_publish, reflect, todo and peer. Descriptions drop from
18.4 KB to 11.5 KB, those tools' schemas from 25.5 KB to 16.8 KB, and a
profile's whole native toolset from 38.3 KB to 29.6 KB (server_admin,
developer; secretary 33.7 → 25.0, navi_code 32.9 → 25.9, tool_developer
37.9 → 29.2, modeler_3d 33.4 → 24.8). Stated as a budget rather than a byte
count: roughly 4k tokens of the model's window per request, back.

Two things were deliberately not shortened. The filesystem edit ladder
(`edit` → `edit_lines` → `smart_edit`, last resort, costs an LLM call) and
todo's mandatory `validation` on `done` both exist to prevent an extra
round trip; a shorter description there would trade tokens for turns. What
todo lost is the prose around the rule, not the rule. `enabled.json` was
left alone too: weather, gmail and get_current_datetime are opt-in and the
DB shows 12, 16 and 40 calls, so they are opt-in tools that are actually
used.

peer had no manual at all, which is how this surfaced: its description was
the only documentation the tool had. manuals/peer.md is now written from
the tool and its /peer route — the ask/status/list actions, the fact that
the hive is only a phone book and the question travels peer-to-peer, the
one-concurrent-answer semaphore, the two independent recursion guards, and
the six error codes.

The MCP server instructions leave the system prompt the same way. They were
8 KB of always-present prose per request, most of it workflow detail the
model only needs when it is about to use that server. Each server now
contributes one line — a new optional `summary` field in mcp_servers.d/*.json,
defaulting to the first sentence of its `instructions` — and
tool_manual("<server>") returns the full text plus the tool names the server
declares. server_admin's MCP block goes from 8.3 KB to 1.3 KB, secretary's
from 8.1 to 1.2. tool_manual learned to answer for a server (a tool name
still wins over a server of the same name, and dash/underscore spelling is
tolerated), and the index names the servers alongside the tools.

One judgement call worth recording: gnexus-book's and navi-web's
instructions end in an absolute "NEVER bypass these tools with filesystem,
terminal or code_exec" rule. Moving that on demand would have been a
behavioural regression dressed as a token saving, so those two `summary`
fields carry the prohibition verbatim alongside the hook, and it stays
always visible.

docs/tools.md gains a section on the split (a description is paid for every
request, a manual only when the tool is used), which is where the next
person will look before trimming a description back into a manual.

Full suite green (1577 passed, 1 skipped). No frontend change and no new
dependency, so the deploy is a pull and a restart.

MCP settings tab: list every connected server, slot only where declared ...

The tab was empty on every install: it listed only servers whose config
declares a `user_key` slot, and no config declared one — which read as
"no MCP servers connected" even though five are wired to profiles.

- GET /mcp-keys now returns every server referenced by at least one
  profile, keyed ones first, with `accepts_user_key`, the slot location
  (null when there is none) and the profile ids that connect it. The
  per-user key store is skipped entirely when nothing has a slot.
- gnexus-creds declares `user_key: {header: Authorization, prefix:
  "Bearer "}` — it is the one server carrying a shared credential, so its
  personal-key field is now real: users with a key run under their own,
  users without one fall back to the shared default.
- The panel lists all servers (transport + profiles), dims the keyless
  rows, and shows a key input only for slotted ones, spelling out the
  shared-key fallback.

docs/api.md and docs/mcp.md updated; backend 1367 passed, webclient 148.

docs: mcp.md (per-user MCP keys BYOK), /mcp-keys endpoints, user_key config

docs: synapse integration guide, special sessions filter, new endpoints ...

- docs/synapse.md: full guide — gateway, reaction runner pipeline,
  notify tool, per-user settings, service sessions in the UI
- index/tools/config/api: entry point + built-in tools + env vars +
  GET /sessions special param + synapse REST/webhook endpoints

webclient: Backgrounds tab in artifacts panel + task toasts ...

- new Backgrounds tab: task list (running first, badges, per-tool icons)
  with a detail view (status, timestamps, sub-agent tokens, result/progress)
- GET /sessions/{id}/tasks snapshot endpoint: task_update events are not
  replayed on reconnect, so the client fetches the task list on session
  load/reload; live task_update entries are merged on top (chat.fetchTasks)
- terminal task_update no longer removes the entry — it marks it finished
  so the tab shows recent completions; _terminalTaskIds still blocks
  resurrection by a late running update
- toasts for background task start / finish (info / success / error) in
  the WS dispatch; silent for other sessions and unknown terminal ids
- persistent background-tasks chip removed (replaced by the tab); only
  the queued-messages chip stays
- fix invisible status text in terminal detail rows: filled .status-*
  backgrounds now scope to .terminal-status-badge only

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

port: 8000/8001 -> 8099/8098 everywhere ...

navi runs on shared servers where 8000/8001 are usually taken. New
defaults: API 8099, navi_ui MCP 8098. Touched: config defaults
(navi_port, navi_ui_mcp_port, public_url, gnauth_redirect_uri),
navi-server launcher docs, env.template/.env.example, install.sh
health-check fallback, terminal client base_url, webclient dev configs
(useWebSocket, contentLinks, vite proxy), android url hint, docs.
Also made the navi_ui FastMCP constructor port settings-driven instead
of a hardcoded 8001 (it was overridden at start anyway).

docs: sync with the plan rework and compression pipeline ...

- planning: plan tool flow (agent-invoked, no pre-turn gate), confirmation
  by COMPLEXITY, think=False + 20k-char Phase 1 window, retired flags
  removed from profiles/api docs
- compression: all four triggers through compress_and_save_session,
  real-baseline gates, head+tail summary input, _plan_compression shared
  decision, actual config defaults (0.90 threshold, 6000 summary tokens,
  32k input cap)

docs: index, profiles, tools, api actualization + archive historical docs ...

index.md: expand the file map to reference navi_code, navi_code_cli, store,
context_providers, eval_system, android-client, testing; point at archive/
for historical notes.

profiles.md: add the Compression section (compression_keep_recent /
compression_max_tokens / compression_prompt_file overrides), the
subagent_system_prompt field, and a note that AgentProfile allows unknown
keys (model_config extra=allow).

tools.md: rewrite the user-tools self-extension section — write_tool is
deprecated and removed; write the file via filesystem/code_exec and call
reload_tools instead.

api.md: add missing REST endpoints (GET /sessions/{id}/todos, GET
/sessions/{id}/files list, GET /sessions/{id}/messages/archive, POST
/sessions/{id}/stop) and the admin MCP management block (config CRUD,
reconnect, status, test, profile-mcp mapping, admin recalls); fix the
WS client→server table (type: message|compact, image limits 8/50MB);
add model_info / todo_updated / compression_started events; fix
recall_update action (add skipped), session_sync payload, context_compressed
threshold (70%).

Archive: move plan_01_god_object_agent.md, tech_debt_review_2026-04-29.md,
architecture_weak_spots.md, future_headless_nodes.md into docs/archive/
(historical design notes, no inbound links).

Co-Authored-By: Claude <noreply@anthropic.com>

docs: update docs for tools.agent/tools.subagent and Navi Code TUI ...

- profiles.md: document tools.agent/tools.subagent structure, deprecate enabled_tools
- tools.md: move web tools to MCP section, fix mcp__navi-web__* names, add create_mcp_server/test_mcp_tool/mcp_status/recall tools
- api.md: update /agents/profiles and /admin/profiles/{id} response schemas, fix MCP examples
- navi_code.md: correct navi_code tool list (MCP disabled)
- navi_code_cli.md: add /thinking, /compact, correct TUI layout
- index.md: add navi_code, terminal client, pydantic-settings
- testing.md: refresh test directory layout, replace stale coverage roadmap
- agent.md: mention tools.agent/tools.subagent in loop

Co-Authored-By: Claude <noreply@anthropic.com>

Add NAVI_AUTH_ENABLED switch for optional auth ...

- Add navi_auth_enabled setting (default true) to navi/config.py and .env.example
- When disabled, treat every request as anonymous admin user (id='anonymous')
- Create/update fixed anonymous navi_users row on startup
- Bypass OAuth/cookie/API-token resolution in navi/auth/deps.py
- Update /auth/status to return {enabled, configured}
- Log security warning on startup when auth is disabled
- Update webclient: skip fetchMe/login screen, show Local mode footer,
  expose /admin link, warn in API keys panel
- Rebuild webclient production bundle
- Add unit and integration tests for no-auth mode
- Update docs: auth.md, config.md, api.md, api_tokens.md, sessions.md,
  websocket.md, mechanics.md, index.md

Co-Authored-By: Claude <noreply@anthropic.com>

Fix settings route switching and document API token system ...

- App.vue: make route reactive via ref + getRouteFromHash() so
  #settings toggles work without page reload
- docs/api_tokens.md: new comprehensive API token auth doc
- docs/api.md: add /api-tokens REST endpoints
- docs/auth.md: add token flow architecture diagram and resolution order
- docs/websocket.md: add auth section with cookie vs query-param token
- docs/architecture.md: update to AgentSessionOrchestrator + ToolContext
- docs/tools.md: add gnexus-creds MCP tools
- docs/index.md: link to api_tokens.md
- Rebuild webclient dist

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

docs: document scheduled recall system ...

- Add docs/recall.md with full architecture, data model, tools, loop behavior
- Update docs/api.md with recall endpoints (GET/DELETE/POST recall) and recall_update WS event
- Update docs/websocket.md with recall_update frame and session_sync case #3
- Update docs/tools.md: add ScheduleRecallTool and ManageRecallTool
- Update docs/sessions.md: add is_recall message flag
- Update docs/index.md: link recall.md in file map

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename MCP tools to mcp:server:tool format and restore human-readable names ...

- Core naming: mcp_server_tool → mcp:server:tool (colon-delimited)
- navi-web tools: search/view/request → web_search/web_view/http_request
- navi-3d tools: compile_scad/render_stl/lint_scad (unchanged names)
- Updated all profile configs, system prompts, docs, manuals, tests
- Added new lint_scad.md manual
- Fixed modeler_3d prompt stale references (scad_lint, model_3d, render_3d)
- All 240 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up remaining web_search / web_view / http_request references ...

- Deleted stale source files: navi/tools/web_search.py, web_view.py,
  http_request.py (no longer imported or registered).
- Updated system_prompt.txt in all profiles (secretary, server_admin,
  discuss, modeler_3d subagent) to reference mcp_navi-web_* tool names.
- Updated key_tools in developer, secretary, server_admin config.json.
- Updated docs (tools.md, profiles.md, api.md) to reflect MCP migration.

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Audit and fix docs/ for accuracy against current codebase ...

Package A (API + WebSocket + Sessions):
- api.md: add missing endpoints (health/embed, auth/status, agents/prompts,
  agents/mcp_servers, sessions/content, 7 admin endpoints). Fix response
  schemas, auth requirements, query params. Update path formats.
- websocket.md: fix planning_status.phase type (int 1|2|3), add metadata
  to tool_call, message_index to stream_end, context_tokens/max_context_tokens
  to context_compressed.
- sessions.md: fix context_compression_threshold default (0.80→0.70),
  add list_page method, add metadata to Message flags.

Package B (Config + Profiles + Tools + Agent + Memory):
- config.md: remove stale ANTHROPIC_API_KEY description, add 12 missing
  env vars (embedding, web search, terminal_user_allowed_commands,
  context_providers_dir, gnauth fields, output_reserve_tokens).
  Fix OPENAI_BASE_URL type.
- profiles.md: fix max_iterations default (20→10), add top_k/top_p/num_thread,
  is_admin_only, context_providers, mcp_servers, subagent_think_enabled.
  Clarify subagent_tools fallback and step_validation_enabled.
- tools.md: replace 3 separate memory tools with unified `memory`, add
  `mcp_status`.
- agent.md: fix run_ephemeral return type (str→tuple), clarify DB reads.
- memory.md: fix search_facts/get_all_facts param order, fix MemoryStore
  init description.

236 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document Android OAuth bridge page flow ...

- docs/auth.md: describe browser vs Android WebView flows, bridge page
  mechanism, state metadata detection via NaviAndroid UA
- docs/api.md: add /auth/mobile-done endpoint reference
- docs/android-client.md: add OAuth/login section covering intent filter,
  CookieManager injection, deep-link handling in onCreate/onNewIntent

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add per-user sessions and memory filters in admin panel ...

Backend:
- GET /admin/memory now accepts user_id query param to filter facts
  for a specific user instead of returning the global view.

Frontend:
- Users table now has "Sessions" and "Memory" buttons per row.
- Clicking "Sessions" switches to Sessions tab and pre-fills search
  with the user id.
- Clicking "Memory" switches to Memory tab and passes user_id to the
  API, showing only that user's facts.
- Manual tab clicks reset filters (search / userId) so the view
  returns to unfiltered state.

Docs:
- docs/api.md updated with user_id param for GET /admin/memory.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Update docs for admin pagination, search, sorting and memory all_users ...

- docs/api.md:
  - GET /admin/sessions: document pagination (limit/offset), search,
    sort_by and sort_order query params; switch response to {total,
    limit, offset, items} envelope.
  - GET /admin/memory: same pagination/sort docs, {total, limit,
    offset, items} envelope.

- docs/memory.md:
  - get_all_facts signature now includes offset, search, sort_by,
    sort_order and all_users parameters.

- docs/sessions.md:
  - Document count_all and search_list store methods.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document multi-user auth system ...

- New docs/auth.md — comprehensive auth guide: OAuth flow, roles/permissions,
  webhooks, env vars, setup instructions, API reference, WebSocket auth,
  testing with mocked auth
- Update docs/config.md — add gnexus-auth OAuth env vars table
- Update docs/api.md — add /auth/*, /admin/*, /webhooks/gnexus-auth endpoints
- Update docs/index.md — add auth.md to file map
- Update docs/sessions.md — document user_id column and list_all filtering
- Update docs/memory.md — document user_id scoping in facts and summaries

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Use gemma4 cloud model by default

Comprehensive documentation update ...

- websocket.md: add heartbeat, session_sync, replay_start/end, planning_status,
  plan_ready (with is_subagent), stream_end extra fields, context_compressed summary,
  updated _AgentRun.events replay buffer, corrected reconnect section
- sessions.md: add name, planning_logs fields; message flags (is_plan, is_compression,
  is_summary, thinking); set_name store op; debug endpoints section
- api.md: full rewrite — add generate-name, planning, file download endpoints;
  all missing WS events; correct message field table with is_plan/is_compression
- tools.md: update user tools list with all current tools
- index.md: fix profiles list (smart_home → developer)
- CLAUDE.md: add Documentation section with table of all docs files
- NAVI.md: add architecture.md entry, improve websocket description

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

docs: add profile_switched WS event and switch_profile flow ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add API reference documentation ...

Covers: REST endpoints (health, profiles, tools, sessions, files,
messages), WebSocket protocol (all event types, client→server format,
typical event sequences), error codes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>