profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

mcp: credentials leave the tree, and sub-agents lose the task board ...

Five tracked configs (gnexus-creds, gntodo, hard-panel, synapse, tgclient)
carried live bearer tokens — in git, in every clone, and handed to the admin
client by GET /admin/mcp/config. They now name a variable, and the value lives
in the service .env (chmod 600, untracked).

Substitution happens once, at transport open, next to the project-relative
path resolution: that is the single place a transport is built, so every
stored and serialised config stays placeholder-only and save_mcp_servers —
reached by create_mcp_server and PUT /admin/mcp/config — cannot write a secret
back into a tracked file. A variable that is not set refuses the connection
instead of dropping the header, which would fail open: a server may answer an
unauthenticated request as an anonymous user rather than returning 401.

Sub-agents also drop gntodo, gnexus-creds, tgclient and synapse from their
scopes, and the runner injects a scope anchor unconditionally — a sub-agent
asked to echo one word had picked a project off the task board and researched
it for forty iterations.

Docs: docs/mcp.md#secrets, a migration recipe in deploy/UPDATE.md, the variable
names in .env.example and deploy/env.template, and a regression guard over the
tracked configs.

The old values stay in git history; rotating all five at their sources is the
remediation, not a follow-up.

subagents, secrets: an honest result contract, and two leaks closed ...

A pass over the sub-agent system, plus the two findings from the previous one.

Leaks:

- `redact_args` / `is_sensitive_key` (navi/tools/_internal/redact.py) mask
  credential-shaped keys at every log site that prints tool arguments — by key
  name, for all tools, so a future MCP tool is covered without declaring
  anything. ssh_exec passwords were sitting in the prod journal in plaintext.
- `mcp_servers.d/*.json` are tracked in git and carried the scaffolding
  machine's absolute paths, so navi-3d/navi-web silently vanished from the
  toolset off that machine. They now hold project-relative paths, resolved in
  `McpClient.open_transport` — not at load time, because create_mcp_server
  round-trips every config file through save_mcp_servers.

Sub-agent result contract:

- `run_ephemeral` returns a `SubAgentOutcome` (text + status: ok, timeout,
  max_iterations, thinking_stall, user_stop, context_overflow) instead of
  `(text, bool)`. Every exit routes through `SubAgentRunner._finish`, so token
  accounting is never dropped and a partial run always carries a progress
  report — the stop-after-stream path used to return a bare string.
- `spawn_agent` renders its header from the status. Before this, every short
  run reached the parent as "hit iteration limit", and the parent repeats that
  diagnosis to the user — a timeout was reported as an iteration limit.
- `ContextTooLargeError` no longer kills a sub-agent: the parent compresses its
  own context, a sub-agent has no session to compress into, so it stops with
  `context_overflow` and hands back what it did rather than surfacing as
  "Sub-agent failed: …" with the whole run discarded.
- The result is capped at 8000 chars (head and tail kept) so a long run cannot
  flood the parent's context, and tool arguments in the progress report go
  through `redact_args`.
- The navi-3d calling convention left the runner for the server's own tool
  schemas — a generic runner should not know one MCP server's path shapes.
- Validation moved inside spawn_agent's try: a bad argument is a clean
  ToolResult the model can read, not an exception in the parent's loop.

Tests: 1636 passed, 1 skipped. New: SubAgentOutcome paths and headers, result
truncation, context overflow, clean argument failures.

profiles: tool_developer folds into developer ...

The profile was a duplicate on every axis we could measure. 22 of its 26
native tools were already developer's; the four it alone held —
reload_tools, create_mcp_server, test_mcp_tool, mcp_status — are 2.9 KB of
schema. Its model chain was the same seven models. Of its 14 KB prompt,
about 8 KB was copied verbatim from developer's (the whole `## Orchestration
model` block and everything from `## Editing policy` down), and most of the
remainder restated manuals/create_mcp_server.md, which already carried the
same ten-step workflow in more detail. It was not a specialisation, it was a
snapshot: `git log -S '"reload_tools"'` shows the tool lived in developer
until 61fa370 rewrote that profile around MCP and cut it off.

What kept it alive was a premise that no longer holds — that Navi's own
capabilities would be written as in-repo tools. They are MCP servers now,
and an MCP server is not a file in this repository with a life of its own:
it is an isolated process registered from mcp_servers.d/. So there is no
reason left for a profile whose only distinct feature is a toolset a general
developer profile can hold, and every reason to stop maintaining a second
prompt that drifts against the first.

- developer: + reload_tools, create_mcp_server, test_mcp_tool, mcp_status
  (24 → 28 native). Its sub-agent gains tool_manual, which is what it
  actually needed to reach the manual while writing a server — the previous
  tool_developer sub-agent had it, developer's did not.
- server_admin: + reload_tools only (24 → 25). Adding a third-party MCP
  server is something this profile does as often as developer does.
  Deliberately not to its sub-agent: reconnecting the MCP manager is a
  process-wide operation belonging to the main agent.
- The prompt and the manual took on what the deleted profile knew and
  create_mcp_server.md did not: reload_tools before the first test_mcp_tool
  (a freshly registered server is not connected, so the test fails and the
  iteration is wasted), the smoke test read by exit code — 124 means timeout
  killed a server still running, 0 means it exited on its own, usually a
  main() without parentheses — absolute command/cwd, mcp_status as discovery
  only, and the steps that stay inline instead of going to a sub-agent.

mcp_status and test_mcp_tool were built without an MCP manager, and their
fallback did `from navi.api.deps import _mcp_manager` — a name that does not
exist, so a live call raised ImportError rather than the intended "MCP
manager not available". The tools always passed a manager in tests, which is
why nothing caught it. Both now receive the manager at construction and fall
back to the live one lazily.

Sessions and profile_overrides are reassigned before the restart: agent.py
resolves the session's profile without a guard, so a deleted profile turns
every session that referenced it into an uncaught ProfileNotFound. Nothing
in the test suite pins the profile inventory, and profiles are read once at
import time — reload_tools does not re-read them — so this ships as a
restart, and the restart is also what makes it take effect.

config: capture the live server configuration ...

Profiles gain the gntodo / gnexus-book / gnexus-creds scopes for agent and
subagent runs, refreshed model lists, and write access where the live setup
has it. MCP server configs pick up their user_key slots, gnexus-book learns
delete_pending_change, and the hard-panel / synapse / tgclient servers join
the tree.

Merge remote-tracking branch 'origin/master' ...

# Conflicts:
#	webclient/vendor/gnexus-ui-kit/package-lock.json

synapse reactions W3: reaction runner, notify tool, gateway hookup ...

- navi/synapse/reactions.py: fire-and-forget reaction run — per-user
  reactions_enabled gate, one-shot dispatcher meta-pass (visible profiles
  only, hidden dispatcher, secretary fallback), special=True session
  headed by the resolved profile, headless agent run via the orchestrator
  run registry, finalise: app push per completion_notify +
  low-level reaction.finished/failed event to Synapse
- navi/synapse/outbound.py: source-side emit (syn_* key from env,
  silent when not configured)
- navi/tools/notify.py: notify tool — message + level (info/warning/
  intervention), both legs per push_target, honest delivered/skipped result
- push/service.py: notify_custom without cooldown
- gateway: verified + non-duplicate delivery schedules the reaction
- profiles: notify added to native tools (7 profiles)

synapse reactions W2: hidden dispatcher profile + synapse_instructions tool

Merge remote-tracking branch 'origin/master'

e2e fixes: tasks tool in profiles, bg timeout lift, stepIcon fix, queue semantics docs ...

E2E findings addressed:
- tasks tool was registered but not in any profile's tools.agent.native —
  added to all six profiles (agent could not check/wait/cancel bg tasks)
- detached terminal/code_exec/ssh_exec runs without explicit timeout are
  lifted to 300s: foreground defaults (20/30/60s) marked long commands
  'completed' with partial output while the process still ran
- ToolCard.vue: define stepIcon(status) — template referenced it but the
  function was missing (render crash on task_update step)
- message_queued reachability documented: WS read loop is sequential, the
  queue path is only reachable from a second socket/headless recall

Merge branch 'master' of https://git.gnexus.space/git/root/navi-1

.

swarm stage 2: peer-to-peer channel — /peer endpoints + peer tool ...

Server side (navi/api/routes/peer.py, port 8099):
- GET /peer/hello — open discovery ping (name, uuid prefix, version)
- GET /peer/status — PSK; identity, uptime, machine facts, hive view
- POST /peer/ask — PSK; one-shot agent run under PEER_ASK_PROFILE
  (default server_admin) answers, one concurrent ask at a time
- loop guard: answering agent runs without the peer tool (deterministic
  recursion cut) + own-uuid asks refused with 409
- verify_swarm_key in navi/swarm.py accepts .swarm-key.previous
  (rotation window, constant-time)

Client side (navi/tools/peer.py):
- peer tool: list (hive book, stale cache fallback marked), status,
  ask by swarm name; asks go direct peer-to-peer, hive never on the
  message path
- audit events on both sides (peer.ask_sent/received/answered/failed)

peer tool added to server_admin, navi_code, developer profiles.

.

profiles: retire gate flags and retrain prompts for the plan tool ...

- drop planning_enabled / planning_mandatory / planning_phase2_enabled /
  observe_skips_plan_enabled / adaptive_replan_enabled from AgentProfile,
  the loader and admin serialization
- add `plan` to native tools of developer, navi_code, tool_developer,
  secretary, server_admin, modeler_3d — without the gate they would
  otherwise lose planning entirely
- system prompts: call plan for non-trivial multi-step work before
  execution; complex plans get presented for confirmation; stuck or goal
  changed -> plan with a reason

permissions: remove broken client-side gate; add backend design doc ...

The terminal-TUI permission gate was a race: _execute_tools_with_sink
emits ToolStarted and starts the tool on the backend immediately, so by
the time the client dialog appeared the destructive action had already
run -- 'deny' could only stop the session, not un-execute the tool. It
also existed only in the terminal TUI (webclient/android had nothing),
and the system-prompt 'Strict Confirmation' nudge was an unreliable
duplicate.

Remove the broken machinery for a clean slate and plan the real
replacement from zero:

- delete clients/terminal/tui/permissions.py, screens/permission_dialog.py
  and their tests
- strip tui_app.py (engine, _deny_tool, _show_permission_dialog,
  _confirm_shell_command, _stop_session_worker, on_permission_request,
  tool_started gate) and the PermissionRequest TUI event; !cmd now runs
  directly (user-typed, no agent gate)
- drop the 'Strict Confirmation' prompt rule from navi_code/developer/
  tool_developer
- add docs/permissions.md: authoritative backend gate design (engine +
  registry + event + endpoint + agent gate + postgres policy + global
  rules config, sub-agents gated) -- design only, not yet implemented
- update docs/index.md, navi_code_cli.md, profiles.md, testing.md

Until Phase 1 lands there is no destructive-action confirmation -- a
conscious period without the false security of the old race-gate.

replan: integrated mid-task re-planning tool ...

Add a replan tool that re-runs the planner over the live session context
+ todo + scratchpad when the plan's structure is stale due to discoveries
(NOT failed steps -- that stays [Adaptive re-plan]). Integrated approach:
PlanningEngine.run gains is_replan/replan_context (suppresses DIRECT
shortcut and observe-skip, frames Phase 1 as a revision); ReplanRunner
packs reason/goal/todo/findings/errors and captures PlanReady; the tool
is exposed to navi_code/developer/tool_developer via a
current_replan_runner ContextVar set per-iteration in run_stream (correct
after switch_profile). New plan replaces the todo. Lazy events import
breaks the navi.tools -> navi.core -> navi.tools cycle.

profiles: port context-org/planning instructions to developer & tool_developer ...

Port the context-organization and planning-machinery framing accumulated
in navi_code to the developer and tool_developer profiles, adapted per
profile:

- Working state & memory (todo/scratchpad/context_transfer/schedule_recall/
  reflect/memory) replaces the stale "Context drift recovery" section.
- System signals subsection names the runtime-injected messages each profile
  actually receives: [Goal anchor], [Anti-stall warning], [Adaptive re-plan]
  (adaptive_replan is on for both), [Iteration N/M]. [Scope boundary] is
  omitted — scope_boundary_enabled is off on these profiles.
- Reading & searching, Editing policy (tool_developer), Safety Rules, Git
  discipline, and Project environments (isolated venv) added.
- developer: Workflow rewritten planner-aware with an observe carve-out
  (observe_skips_plan now on); Project knowledge replaced by docs-first
  Documentation; sub-agent briefing gains context_transfer + restricted-toolset
  bullets.
- tool_developer: keeps its MCP-specific 10-step workflow and prerequisites;
  sub-agent briefing gains context_transfer + restricted-toolset bullets
  (sub-agent cannot reload_tools/test_mcp_tool/mcp_status — run those inline).
- config: observe_skips_plan_enabled=true on both, so observe requests skip
  phase3 (no plan/todo for read/explain/inspect) — enables the Workflow
  observe carve-out and saves an LLM call on info requests.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem: make edit/edit_lines the default, smart_edit a last-resort fallback ...

Agent overused smart_edit (whole-file LLM call, weak context). Rewrite the
FilesystemTool description decision tree so edit (exact text) is the default and
edit_lines (by line numbers) is the deterministic option; smart_edit is reserved
for genuinely semantic changes that cannot be expressed as exact text or line
numbers. Strengthen the old_not_unique hint to steer back to edit/edit_lines
before smart_edit. Add an "Editing policy" section to the navi_code and
developer profile prompts reinforcing the same priority.

Co-Authored-By: Claude <noreply@anthropic.com>

profiles: add gemma4:12b-it-qat-128k as top-priority model ...

New local QAT model added at the head of the model priority list across
all profiles, so it is preferred first with cloud fallback.

Fix MCP names and profile API format consistency ...

- Replace stale mcp__navi_web__* / mcp__navi_3d__* names with canonical
  mcp__navi-web__* / mcp__navi-3d__* across prompts and key_tools.
- Update /agents/profiles and /admin/profiles endpoints to expose
  tools.agent / tools.subagent instead of deprecated enabled_tools fields.
- Update docs/mechanics.md to reference the new tools structure.
- Archive stale docs/visual.html.

Co-Authored-By: Claude <noreply@anthropic.com>

Clarify terminal description param and profile prompts ...

- terminal.py: stronger description for the 'description' field so
  the model knows it is REQUIRED for action=open and what to write
- server_admin and developer system prompts: add a short section
  explaining persistent terminal usage (open/close/list/status/send_input)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Prioritise gemma4 and add MCP tools to subagent scopes ...

- modeler_3d: gemma4:31b-cloud first (vision-capable), drop glm-5.1
- modeler_3d subagent: add navi-3d + navi-web MCP tools
- Adjust subagent tool scopes across profiles for explicit MCP access

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refactor profile tool config to explicit agent/subagent structure ...

Replaces the confusing mix of enabled_tools + mcp_servers + subagent_tools
with a single explicit structure:

    tools: {
      agent:   {native: [...], mcp: {server: [groups]}},
      subagent:{native: [...], mcp: {server: [groups]}}
    }

Why:
- Old fields mixed native and MCP names (mcp__server__tool) in one list,
  making it impossible to tell at a glance what a subagent actually gets.
- subagent_runner.py had 25 lines of runtime MCP filtering logic that
  was hard to follow and error-prone.

Changes:
- AgentProfile: add ToolConfig / ToolScopeConfig pydantic models,
  keep old fields (enabled_tools, mcp_servers, subagent_tools) for
  auto-migration via _migrate_tools validator.
- loader.py: read new "tools" key, auto-migrate legacy configs.
- agent.py: _tool_list now accepts ToolScopeConfig.
- subagent_runner.py: simplified — profile.get_subagent_tools() returns
  the exact scope, no runtime filtering needed.
- context_builder.py, list_tools.py, spawn_agent.py: updated to use
  profile.get_agent_tools() / get_subagent_tools().
- All 6 profile config.json files migrated to new schema.
- Secretary subagent now explicitly gets navi-web MCP tools for web search.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

cleanup: remove deprecated tools and orphaned memory tools ...

Removed (no profile used them, no cross-dependencies):
- write_tool.py, delete_tool.py, test_tool.py
- memory_save.py, memory_search.py, memory_forget.py

Updated:
- navi/core/registry.py — removed imports and registrations
- navi/tools/__init__.py — removed imports
- docs/tools.md — removed references, updated self-extension section to MCP
- navi/tools/tool_manual.py — updated example to create_mcp_server

Profile fixes:
- developer: +tool_manual, +ssh_exec
- discuss: +list_profiles
- tool_developer: +mcp_servers navi-web

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix profile MCP server groups to use group names instead of tool names ...

Profile configs were listing individual tool names (web_view, http_request)
in mcp_servers.navi-web instead of group names (browse, request). This caused
resolve_group() to return empty results, so MCP tools never appeared in the
profile's resolved tool list.

Also fixes modeler_3d stale tool references in system_prompt.txt.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename MCP tools to mcp:server:tool format and restore human-readable names ...

- Core naming: mcp_server_tool → mcp:server:tool (colon-delimited)
- navi-web tools: search/view/request → web_search/web_view/http_request
- navi-3d tools: compile_scad/render_stl/lint_scad (unchanged names)
- Updated all profile configs, system prompts, docs, manuals, tests
- Added new lint_scad.md manual
- Fixed modeler_3d prompt stale references (scad_lint, model_3d, render_3d)
- All 240 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up remaining web_search / web_view / http_request references ...

- Deleted stale source files: navi/tools/web_search.py, web_view.py,
  http_request.py (no longer imported or registered).
- Updated system_prompt.txt in all profiles (secretary, server_admin,
  discuss, modeler_3d subagent) to reference mcp_navi-web_* tool names.
- Updated key_tools in developer, secretary, server_admin config.json.
- Updated docs (tools.md, profiles.md, api.md) to reflect MCP migration.

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>