profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

profiles: tool_developer folds into developer ...

The profile was a duplicate on every axis we could measure. 22 of its 26
native tools were already developer's; the four it alone held —
reload_tools, create_mcp_server, test_mcp_tool, mcp_status — are 2.9 KB of
schema. Its model chain was the same seven models. Of its 14 KB prompt,
about 8 KB was copied verbatim from developer's (the whole `## Orchestration
model` block and everything from `## Editing policy` down), and most of the
remainder restated manuals/create_mcp_server.md, which already carried the
same ten-step workflow in more detail. It was not a specialisation, it was a
snapshot: `git log -S '"reload_tools"'` shows the tool lived in developer
until 61fa370 rewrote that profile around MCP and cut it off.

What kept it alive was a premise that no longer holds — that Navi's own
capabilities would be written as in-repo tools. They are MCP servers now,
and an MCP server is not a file in this repository with a life of its own:
it is an isolated process registered from mcp_servers.d/. So there is no
reason left for a profile whose only distinct feature is a toolset a general
developer profile can hold, and every reason to stop maintaining a second
prompt that drifts against the first.

- developer: + reload_tools, create_mcp_server, test_mcp_tool, mcp_status
  (24 → 28 native). Its sub-agent gains tool_manual, which is what it
  actually needed to reach the manual while writing a server — the previous
  tool_developer sub-agent had it, developer's did not.
- server_admin: + reload_tools only (24 → 25). Adding a third-party MCP
  server is something this profile does as often as developer does.
  Deliberately not to its sub-agent: reconnecting the MCP manager is a
  process-wide operation belonging to the main agent.
- The prompt and the manual took on what the deleted profile knew and
  create_mcp_server.md did not: reload_tools before the first test_mcp_tool
  (a freshly registered server is not connected, so the test fails and the
  iteration is wasted), the smoke test read by exit code — 124 means timeout
  killed a server still running, 0 means it exited on its own, usually a
  main() without parentheses — absolute command/cwd, mcp_status as discovery
  only, and the steps that stay inline instead of going to a sub-agent.

mcp_status and test_mcp_tool were built without an MCP manager, and their
fallback did `from navi.api.deps import _mcp_manager` — a name that does not
exist, so a live call raised ImportError rather than the intended "MCP
manager not available". The tools always passed a manager in tests, which is
why nothing caught it. Both now receive the manager at construction and fall
back to the live one lazily.

Sessions and profile_overrides are reassigned before the restart: agent.py
resolves the session's profile without a guard, so a deleted profile turns
every session that referenced it into an uncaught ProfileNotFound. Nothing
in the test suite pins the profile inventory, and profiles are read once at
import time — reload_tools does not re-read them — so this ships as a
restart, and the restart is also what makes it take effect.

agent: a profile switch now survives the turn that made it ...

switch_profile repoints the session with one narrow UPDATE, and the run's own
save() — which lands right after every tool batch, the switch's own batch
included — wrote its stale in-memory profile back over it. The post-turn
reload then read the row it had just clobbered, saw no change, and left the
turn bound to the old profile's tools: the agent called exactly the tools
switch_profile had just advertised ("Newly available: ssh_exec, terminal, …")
and got "tool not found" for every one of them. Prod shows both halves — the
session that reported two switches kept the profile it started with, and
agent.profile_reloaded appears zero times in the whole boot.

The reload now compares against the profile the run is bound to
(bound_profile_id), which set_profile cannot move, instead of against
session.profile_id, which the store either rewrites or mutates to the new
value — making the old guard false in both branches by construction. And
save() leaves the profile_id column alone on conflict: set_profile owns it
after creation, the same way name/created_at are owned elsewhere.

A tool the live tool_map does not hold is now logged (tool.not_found, with
profile and live tool count). It previously left no trace at all in the
journal, which is why this took a database dig to find.

Three tests, each verified to fail without its fix: the next LLM call is
offered the new profile's tools, save() does not rewrite profile_id, and a
missing tool is logged.

docs: sync with the plan rework and compression pipeline ...

- planning: plan tool flow (agent-invoked, no pre-turn gate), confirmation
  by COMPLEXITY, think=False + 20k-char Phase 1 window, retired flags
  removed from profiles/api docs
- compression: all four triggers through compress_and_save_session,
  real-baseline gates, head+tail summary input, _plan_compression shared
  decision, actual config defaults (0.90 threshold, 6000 summary tokens,
  32k input cap)

permissions: remove broken client-side gate; add backend design doc ...

The terminal-TUI permission gate was a race: _execute_tools_with_sink
emits ToolStarted and starts the tool on the backend immediately, so by
the time the client dialog appeared the destructive action had already
run -- 'deny' could only stop the session, not un-execute the tool. It
also existed only in the terminal TUI (webclient/android had nothing),
and the system-prompt 'Strict Confirmation' nudge was an unreliable
duplicate.

Remove the broken machinery for a clean slate and plan the real
replacement from zero:

- delete clients/terminal/tui/permissions.py, screens/permission_dialog.py
  and their tests
- strip tui_app.py (engine, _deny_tool, _show_permission_dialog,
  _confirm_shell_command, _stop_session_worker, on_permission_request,
  tool_started gate) and the PermissionRequest TUI event; !cmd now runs
  directly (user-typed, no agent gate)
- drop the 'Strict Confirmation' prompt rule from navi_code/developer/
  tool_developer
- add docs/permissions.md: authoritative backend gate design (engine +
  registry + event + endpoint + agent gate + postgres policy + global
  rules config, sub-agents gated) -- design only, not yet implemented
- update docs/index.md, navi_code_cli.md, profiles.md, testing.md

Until Phase 1 lands there is no destructive-action confirmation -- a
conscious period without the false security of the old race-gate.

docs: index, profiles, tools, api actualization + archive historical docs ...

index.md: expand the file map to reference navi_code, navi_code_cli, store,
context_providers, eval_system, android-client, testing; point at archive/
for historical notes.

profiles.md: add the Compression section (compression_keep_recent /
compression_max_tokens / compression_prompt_file overrides), the
subagent_system_prompt field, and a note that AgentProfile allows unknown
keys (model_config extra=allow).

tools.md: rewrite the user-tools self-extension section — write_tool is
deprecated and removed; write the file via filesystem/code_exec and call
reload_tools instead.

api.md: add missing REST endpoints (GET /sessions/{id}/todos, GET
/sessions/{id}/files list, GET /sessions/{id}/messages/archive, POST
/sessions/{id}/stop) and the admin MCP management block (config CRUD,
reconnect, status, test, profile-mcp mapping, admin recalls); fix the
WS client→server table (type: message|compact, image limits 8/50MB);
add model_info / todo_updated / compression_started events; fix
recall_update action (add skipped), session_sync payload, context_compressed
threshold (70%).

Archive: move plan_01_god_object_agent.md, tech_debt_review_2026-04-29.md,
architecture_weak_spots.md, future_headless_nodes.md into docs/archive/
(historical design notes, no inbound links).

Co-Authored-By: Claude <noreply@anthropic.com>

docs: document bounded autonomy + navi-code TUI features ...

- profiles.md: new "Bounded autonomy" section (scope_boundary_enabled,
  observe_skips_plan_enabled) + memory-facts scope filter; navi_code section
  and config.json example updated
- mechanics.md: Agent Loop / Planning Pipeline / Context Builder rows for
  scope boundary, observe short-circuit, MODE classification, scope filter
- navi_code.md: bounded autonomy note in profile section
- navi_code_cli.md: --raw and -h/--help params, multiline input, planning
  and spawn_agent cards, nested subagent styling, planning_status/plan_ready/
  turn_thinking rendering, tui/renderers/ in dev section

Co-Authored-By: Claude <noreply@anthropic.com>

docs: update docs for tools.agent/tools.subagent and Navi Code TUI ...

- profiles.md: document tools.agent/tools.subagent structure, deprecate enabled_tools
- tools.md: move web tools to MCP section, fix mcp__navi-web__* names, add create_mcp_server/test_mcp_tool/mcp_status/recall tools
- api.md: update /agents/profiles and /admin/profiles/{id} response schemas, fix MCP examples
- navi_code.md: correct navi_code tool list (MCP disabled)
- navi_code_cli.md: add /thinking, /compact, correct TUI layout
- index.md: add navi_code, terminal client, pydantic-settings
- testing.md: refresh test directory layout, replace stale coverage roadmap
- agent.md: mention tools.agent/tools.subagent in loop

Co-Authored-By: Claude <noreply@anthropic.com>

Update Navi Code profile: config, prompts, docs, and env example ...

- Sync navi_code profile tools with current codebase (memory, schedule_recall, manage_recall, switch_profile, list_profiles)

- Remove image_view and MCP tools from navi_code; keep local terminal focus

- Update system prompt and persona for terminal-first coding assistant

- Update docs/navi_code.md, docs/profiles.md, docs/config.md for new tool set

- Refresh .env.navi_code.example with current config variables

- Remove stale CLI prototype directories (cli/, navi_code_cli/)

Docs: actualize navi_code tool lists and key_tools ...

- docs/navi_code.md and docs/profiles.md now correctly list native tools,
  MCP navi-web tools, and excluded tools.
- image_view is documented as enabled for the agent (loads and describes
  images for the model, not shown to the user).
- web_search/http_request are documented as MCP navi-web tools, not native
  disabled tools.
- navi/profiles/navi_code/config.json: updated full_description.key_tools to
  match actual resolved tool names.

Tests: 459 passed, 1 skipped.

Co-Authored-By: Claude <noreply@anthropic.com>

Navi Code: Phase 2 — CLI terminal client, tests, docs ...

- Add clients/terminal package: config, state, REST API wrappers, WebSocket
  client, renderer, and click-based interactive CLI.
- Wire navi-code console script via pyproject.toml.
- Add unit and WebSocket integration tests for the terminal client.
- Update docs/profiles.md, docs/config.md, README.md with navi_code profile
  and default-profile instructions.
- Add docs/navi_code.md setup guide and docs/navi_code_cli.md usage reference.
- Fix lint in new test files and test_auth_disabled.py.

Tested: 459 passed, 1 skipped (excluded unrelated websocket test).

Co-Authored-By: Claude <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

docs/profiles: document subagent_tools MCP filtering behavior ...

Clarify that subagent_tools acts as a whitelist for MCP tools:
- Only mcp: entries explicitly listed in subagent_tools are exposed to sub-agents.
- If subagent_tools is non-empty and contains no mcp: entries, the sub-agent
  receives no MCP tools at all.

Add inherit_system_prompt and is_subagent_only mechanisms ...

- inherit_system_prompt: subagent parameter to prepend parent's system
  prompt as a base layer before subagent specialisation
- is_subagent_only: profile flag blocking switch_profile, allowing
  spawn_agent only; shown with [subagent only] tag in list_profiles
- Document both in docs/profiles.md and docs/tools.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename MCP tools to mcp:server:tool format and restore human-readable names ...

- Core naming: mcp_server_tool → mcp:server:tool (colon-delimited)
- navi-web tools: search/view/request → web_search/web_view/http_request
- navi-3d tools: compile_scad/render_stl/lint_scad (unchanged names)
- Updated all profile configs, system prompts, docs, manuals, tests
- Added new lint_scad.md manual
- Fixed modeler_3d prompt stale references (scad_lint, model_3d, render_3d)
- All 240 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up remaining web_search / web_view / http_request references ...

- Deleted stale source files: navi/tools/web_search.py, web_view.py,
  http_request.py (no longer imported or registered).
- Updated system_prompt.txt in all profiles (secretary, server_admin,
  discuss, modeler_3d subagent) to reference mcp_navi-web_* tool names.
- Updated key_tools in developer, secretary, server_admin config.json.
- Updated docs (tools.md, profiles.md, api.md) to reflect MCP migration.

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Audit and fix docs/ for accuracy against current codebase ...

Package A (API + WebSocket + Sessions):
- api.md: add missing endpoints (health/embed, auth/status, agents/prompts,
  agents/mcp_servers, sessions/content, 7 admin endpoints). Fix response
  schemas, auth requirements, query params. Update path formats.
- websocket.md: fix planning_status.phase type (int 1|2|3), add metadata
  to tool_call, message_index to stream_end, context_tokens/max_context_tokens
  to context_compressed.
- sessions.md: fix context_compression_threshold default (0.80→0.70),
  add list_page method, add metadata to Message flags.

Package B (Config + Profiles + Tools + Agent + Memory):
- config.md: remove stale ANTHROPIC_API_KEY description, add 12 missing
  env vars (embedding, web search, terminal_user_allowed_commands,
  context_providers_dir, gnauth fields, output_reserve_tokens).
  Fix OPENAI_BASE_URL type.
- profiles.md: fix max_iterations default (20→10), add top_k/top_p/num_thread,
  is_admin_only, context_providers, mcp_servers, subagent_think_enabled.
  Clarify subagent_tools fallback and step_validation_enabled.
- tools.md: replace 3 separate memory tools with unified `memory`, add
  `mcp_status`.
- agent.md: fix run_ephemeral return type (str→tuple), clarify DB reads.
- memory.md: fix search_facts/get_all_facts param order, fix MemoryStore
  init description.

236 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Allow subagents to use selected profiles

Add 3D modeling profile with model_3d and render_3d tools ...

New profile:
- modeler_3d: 3D model design for 3D printing (OpenSCAD → STL)

New tools:
- model_3d: compile .scad → binary STL via OpenSCAD CLI
- render_3d: render PNG previews from STL (up to 3 views, 400×300)

Both tools return openscad_not_found error if OpenSCAD is not installed.

Docs:
- README.md: add modeler_3d to profile table
- docs/profiles.md: add modeler_3d to active profiles
- docs/tools.md: add model_3d and render_3d to builtin tools
- manuals/model_3d.md, manuals/render_3d.md: full usage manuals
- modeler_3d system_prompt.txt: OpenSCAD-first workflow with dedicated tools

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Docs audit: fix outdated references (PostgreSQL, Vue client, profiles, tools) ...

- architecture.md: SQLite → PostgreSQL + pgvector
- README.md: vanilla JS → Vue 3 + Pinia, remove smart_home, update temps
- profiles.md: remove smart_home, add discuss, update temperatures
- tools.md: add missing built-in tools, update user tools list
- sessions.md: fix outdated references
- NAVI.md: sync with current stack
- persona.txt: update self-extension and content_publish guidance
- content_publish.md manual: reflect session_dir-based no-copy design
- content_store.py & content_publish.py: inline with docs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add structured planning review and adaptive depth

Add per-phase planning flags and planning_mandatory ...

- planning_mandatory: disables DIRECT shortcut, forces all phases to run
- planning_phase1_enabled / phase2_enabled / phase3_enabled: per-phase toggles
- planning_phase2_enabled replaces planning_reflect_enabled (migrated in loader with backward compat)
- Migrate all profile configs; rewrite docs/profiles.md as full config reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

WelcomeScreen polish, root-path fix, docs update ...

- Move mobile sidebar button to top-left corner (no header bar backdrop)
- Show WelcomeScreen on / with no hash instead of auto-loading first session
- Docs: document Ollama multi-server fallback, model priority lists, OLLAMA_BACKENDS_FILE

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Make server I/O non-blocking; update docs ...

- Wrap all heavy filesystem ops in asyncio.to_thread: filesystem tool
  (read/write/append/list/find/info/move/delete/query/smart_edit),
  image_view (read_bytes), share_file (shutil.copy2), write_tool
  (write_text, _register_user_tool), session_files (shutil.rmtree,
  iterdir), sessions upload endpoint (sync open/write → to_thread)
- Make delete_session_dir async; update its caller in sessions.py
- docs/config.md: fix wrong defaults (threshold 0.70, keep_recent 8),
  remove phantom SESSION_FILES_TTL_HOURS, add LLM timeouts, DATABASE_URL,
  PUBLIC_URL, Gmail, CONTEXT_SUMMARY_MAX_TOKENS sections
- docs/profiles.md: add missing tool_developer profile to table
- android-client: add WebView remote debugging; remove unused toolbar menu
- Remove stale helper scripts and test files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Use gemma4 cloud model by default

Add NAVI.md navigation hub and update core docs ...

NAVI.md: lightweight project entry point for Navi — server command,
key paths, doc map with query instructions, tool manual index.
All profiles read this; detailed content stays in docs/.

docs/agent.md: rewrite to cover 3-phase planning, all 10 thinking
mechanics flags, adaptive replan, anti-stall, goal anchoring.

docs/profiles.md: update AgentProfile fields (all flags), correct
profile list (secretary/server_admin/developer), JSON config format,
auto-discovery instead of manual registration.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add backend documentation ...

10 markdown files covering all backend subsystems: architecture,
agent loop, tool system, sessions, WebSocket protocol, profiles,
memory, config, and API reference. Structured for AI readability
with a single entry point (docs/index.md) and cross-references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>