profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

subagents, secrets: an honest result contract, and two leaks closed ...

A pass over the sub-agent system, plus the two findings from the previous one.

Leaks:

- `redact_args` / `is_sensitive_key` (navi/tools/_internal/redact.py) mask
  credential-shaped keys at every log site that prints tool arguments — by key
  name, for all tools, so a future MCP tool is covered without declaring
  anything. ssh_exec passwords were sitting in the prod journal in plaintext.
- `mcp_servers.d/*.json` are tracked in git and carried the scaffolding
  machine's absolute paths, so navi-3d/navi-web silently vanished from the
  toolset off that machine. They now hold project-relative paths, resolved in
  `McpClient.open_transport` — not at load time, because create_mcp_server
  round-trips every config file through save_mcp_servers.

Sub-agent result contract:

- `run_ephemeral` returns a `SubAgentOutcome` (text + status: ok, timeout,
  max_iterations, thinking_stall, user_stop, context_overflow) instead of
  `(text, bool)`. Every exit routes through `SubAgentRunner._finish`, so token
  accounting is never dropped and a partial run always carries a progress
  report — the stop-after-stream path used to return a bare string.
- `spawn_agent` renders its header from the status. Before this, every short
  run reached the parent as "hit iteration limit", and the parent repeats that
  diagnosis to the user — a timeout was reported as an iteration limit.
- `ContextTooLargeError` no longer kills a sub-agent: the parent compresses its
  own context, a sub-agent has no session to compress into, so it stops with
  `context_overflow` and hands back what it did rather than surfacing as
  "Sub-agent failed: …" with the whole run discarded.
- The result is capped at 8000 chars (head and tail kept) so a long run cannot
  flood the parent's context, and tool arguments in the progress report go
  through `redact_args`.
- The navi-3d calling convention left the runner for the server's own tool
  schemas — a generic runner should not know one MCP server's path shapes.
- Validation moved inside spawn_agent's try: a bad argument is a clean
  ToolResult the model can read, not an exception in the parent's loop.

Tests: 1636 passed, 1 skipped. New: SubAgentOutcome paths and headers, result
truncation, context overflow, clean argument failures.

profiles: tool_developer folds into developer ...

The profile was a duplicate on every axis we could measure. 22 of its 26
native tools were already developer's; the four it alone held —
reload_tools, create_mcp_server, test_mcp_tool, mcp_status — are 2.9 KB of
schema. Its model chain was the same seven models. Of its 14 KB prompt,
about 8 KB was copied verbatim from developer's (the whole `## Orchestration
model` block and everything from `## Editing policy` down), and most of the
remainder restated manuals/create_mcp_server.md, which already carried the
same ten-step workflow in more detail. It was not a specialisation, it was a
snapshot: `git log -S '"reload_tools"'` shows the tool lived in developer
until 61fa370 rewrote that profile around MCP and cut it off.

What kept it alive was a premise that no longer holds — that Navi's own
capabilities would be written as in-repo tools. They are MCP servers now,
and an MCP server is not a file in this repository with a life of its own:
it is an isolated process registered from mcp_servers.d/. So there is no
reason left for a profile whose only distinct feature is a toolset a general
developer profile can hold, and every reason to stop maintaining a second
prompt that drifts against the first.

- developer: + reload_tools, create_mcp_server, test_mcp_tool, mcp_status
  (24 → 28 native). Its sub-agent gains tool_manual, which is what it
  actually needed to reach the manual while writing a server — the previous
  tool_developer sub-agent had it, developer's did not.
- server_admin: + reload_tools only (24 → 25). Adding a third-party MCP
  server is something this profile does as often as developer does.
  Deliberately not to its sub-agent: reconnecting the MCP manager is a
  process-wide operation belonging to the main agent.
- The prompt and the manual took on what the deleted profile knew and
  create_mcp_server.md did not: reload_tools before the first test_mcp_tool
  (a freshly registered server is not connected, so the test fails and the
  iteration is wasted), the smoke test read by exit code — 124 means timeout
  killed a server still running, 0 means it exited on its own, usually a
  main() without parentheses — absolute command/cwd, mcp_status as discovery
  only, and the steps that stay inline instead of going to a sub-agent.

mcp_status and test_mcp_tool were built without an MCP manager, and their
fallback did `from navi.api.deps import _mcp_manager` — a name that does not
exist, so a live call raised ImportError rather than the intended "MCP
manager not available". The tools always passed a manager in tests, which is
why nothing caught it. Both now receive the manager at construction and fall
back to the live one lazily.

Sessions and profile_overrides are reassigned before the restart: agent.py
resolves the session's profile without a guard, so a deleted profile turns
every session that referenced it into an uncaught ProfileNotFound. Nothing
in the test suite pins the profile inventory, and profiles are read once at
import time — reload_tools does not re-read them — so this ships as a
restart, and the restart is also what makes it take effect.

tools, mcp: a description is a hook, not a manual ...

With 24 tools in a profile the schemas are the largest fixed block of every
request, and they are paid for whether or not the tool is called. Eleven of
them carried a manual's worth of prose in their `description` — examples,
error tables, "common mistakes", the mechanics of the file areas. Reading
them against the manuals first, the manuals already held all of it, and
richer: this was duplication sitting in the one place the model cannot avoid
reading, not knowledge that had nowhere else to live. So the descriptions
became selection hooks — what the tool is for, when to pick it over its
neighbour, and the one rule that cannot wait — and the detail moved to
tool_manual("<tool>"), which costs nothing until the tool is actually in
play.

filesystem, spawn_agent, schedule_recall, manage_recall, scratchpad, plan,
share_file, content_publish, reflect, todo and peer. Descriptions drop from
18.4 KB to 11.5 KB, those tools' schemas from 25.5 KB to 16.8 KB, and a
profile's whole native toolset from 38.3 KB to 29.6 KB (server_admin,
developer; secretary 33.7 → 25.0, navi_code 32.9 → 25.9, tool_developer
37.9 → 29.2, modeler_3d 33.4 → 24.8). Stated as a budget rather than a byte
count: roughly 4k tokens of the model's window per request, back.

Two things were deliberately not shortened. The filesystem edit ladder
(`edit` → `edit_lines` → `smart_edit`, last resort, costs an LLM call) and
todo's mandatory `validation` on `done` both exist to prevent an extra
round trip; a shorter description there would trade tokens for turns. What
todo lost is the prose around the rule, not the rule. `enabled.json` was
left alone too: weather, gmail and get_current_datetime are opt-in and the
DB shows 12, 16 and 40 calls, so they are opt-in tools that are actually
used.

peer had no manual at all, which is how this surfaced: its description was
the only documentation the tool had. manuals/peer.md is now written from
the tool and its /peer route — the ask/status/list actions, the fact that
the hive is only a phone book and the question travels peer-to-peer, the
one-concurrent-answer semaphore, the two independent recursion guards, and
the six error codes.

The MCP server instructions leave the system prompt the same way. They were
8 KB of always-present prose per request, most of it workflow detail the
model only needs when it is about to use that server. Each server now
contributes one line — a new optional `summary` field in mcp_servers.d/*.json,
defaulting to the first sentence of its `instructions` — and
tool_manual("<server>") returns the full text plus the tool names the server
declares. server_admin's MCP block goes from 8.3 KB to 1.3 KB, secretary's
from 8.1 to 1.2. tool_manual learned to answer for a server (a tool name
still wins over a server of the same name, and dash/underscore spelling is
tolerated), and the index names the servers alongside the tools.

One judgement call worth recording: gnexus-book's and navi-web's
instructions end in an absolute "NEVER bypass these tools with filesystem,
terminal or code_exec" rule. Moving that on demand would have been a
behavioural regression dressed as a token saving, so those two `summary`
fields carry the prohibition verbatim alongside the hook, and it stays
always visible.

docs/tools.md gains a section on the split (a description is paid for every
request, a manual only when the tool is used), which is where the next
person will look before trimming a description back into a manual.

Full suite green (1577 passed, 1 skipped). No frontend change and no new
dependency, so the deploy is a pull and a restart.

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

navi_code: stop delegating code work to the developer profile ...

navi_code was systematically spawning sub-agents as `developer` instead of
itself, even though it already covers local code work. Root cause was a
three-channel nudge: the navi_code prompt recommended `developer` for "general
code work" and framed omit (-> navi_code) as the exception; spawn_agent's
description listed `developer` as the coding example and never mentioned
navi_code; the injected Available-profiles list shows developer's broad
"General-purpose software development" blurb. The planner locks profile_id at
plan time on those same signals, so the bias is baked in before execution.

Flip the default in all three places: for code work OMIT profile_id so the
sub-agent runs as the current profile (navi_code); set profile_id only for a
different specialisation (secretary/server_admin/tool_developer). developer is
not removed as a profile — it is just no longer recommended from navi_code.

Fix spawn_agent profile selection: stronger prompt + alias fallback ...

- Rewrite PROFILE SELECTION description to explicitly say: when the
  plan specifies a profile, you MUST pass profile_id. Only omit when
  the plan does NOT specify a profile.
- Add JSON examples showing correct profile_id usage.
- Add fallback for 'profile' parameter name (models sometimes use
  'profile' instead of 'profile_id' when the description says
  'Profile to use').\n\nCo-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Pass explicit ToolContext to tools instead of hidden ContextVars ...

Add ToolContext dataclass (session_id, event_sink, stop_event, model,
user_id, user_role, user_info) and thread it through the execution chain:
Agent._execute_tools_with_sink → ToolExecutor → tool.execute().

All ~25 tools updated to accept ctx parameter. Tools that previously
read ContextVar now prefer ctx when provided, falling back to
ContextVar for backward compatibility.

Tests updated to pass ToolContext explicitly — no more test fixtures
that set current_session_id / current_user_id ContextVars.

ContextVar setters remain as fallback for non-tool consumers
(ai_helper, context_builder, planning) and will be removed in a
follow-up refactor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refactor profile tool config to explicit agent/subagent structure ...

Replaces the confusing mix of enabled_tools + mcp_servers + subagent_tools
with a single explicit structure:

    tools: {
      agent:   {native: [...], mcp: {server: [groups]}},
      subagent:{native: [...], mcp: {server: [groups]}}
    }

Why:
- Old fields mixed native and MCP names (mcp__server__tool) in one list,
  making it impossible to tell at a glance what a subagent actually gets.
- subagent_runner.py had 25 lines of runtime MCP filtering logic that
  was hard to follow and error-prone.

Changes:
- AgentProfile: add ToolConfig / ToolScopeConfig pydantic models,
  keep old fields (enabled_tools, mcp_servers, subagent_tools) for
  auto-migration via _migrate_tools validator.
- loader.py: read new "tools" key, auto-migrate legacy configs.
- agent.py: _tool_list now accepts ToolScopeConfig.
- subagent_runner.py: simplified — profile.get_subagent_tools() returns
  the exact scope, no runtime filtering needed.
- context_builder.py, list_tools.py, spawn_agent.py: updated to use
  profile.get_agent_tools() / get_subagent_tools().
- All 6 profile config.json files migrated to new schema.
- Secretary subagent now explicitly gets navi-web MCP tools for web search.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add inherit_system_prompt and is_subagent_only mechanisms ...

- inherit_system_prompt: subagent parameter to prepend parent's system
  prompt as a base layer before subagent specialisation
- is_subagent_only: profile flag blocking switch_profile, allowing
  spawn_agent only; shown with [subagent only] tag in list_profiles
- Document both in docs/profiles.md and docs/tools.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Enhance native toolset and add persistent KV store ...

- Add PostgreSQL-backed KvStore (navi/store/) for session-scoped data.
- Migrate todo and scratchpad from in-memory dicts to KvStore.
- Filesystem: add copy, grep, diff actions; compress description.
- CodeExec: remove language param, expose working_dir in schema.
- ImageView: resize to 1024px JPEG + Content-Type guard for URLs.
- Memory list: return distinct categories instead of all facts.
- SSH: add scp action with upload/download support.
- Update CLAUDE.md (Postgres-only), docs/tools.md, add docs/store.md.
- Fix agent/planning/context_builder async signatures for todo helpers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refactor: move tool helper modules into _internal subpackage ...

Moves non-tool infrastructure out of navi/tools/ root so that only
actual tool classes live there:

  base.py           → _internal/base.py
  loader.py         → _internal/loader.py
  middleware.py     → _internal/middleware.py
  logging_middleware.py → _internal/logging_middleware.py
  _time_parser.py   → _internal/time_parser.py

All imports updated across core/, api/, mcp/, tools/, and tests/.
No proxy files remain in navi/tools/ root.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Wire MCP manager into sub-agent (spawn_agent) so MCP tools are available to sub-agents ...

- SpawnAgentTool: add mcp_manager parameter and pass it to Agent
- main.py: patch _mcp_manager into spawn_agent after MCP connect

Fixes: tool 'mcp_gnexus-book_*' not found in sub-agent sessions.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Allow subagents to use selected profiles

Simplify 3D SCAD subagent prompt

Fix profile prompt inconsistencies

Route subagent planning events into spawn_agent card in the UI ...

Previously PlanningStatus/PlanReady had no is_subagent flag, so subagent
planning spinners and plan cards rendered as top-level Navi planning UI.

Backend:
- Add is_subagent field to PlanningStatus and PlanReady events
- _run_planning accepts is_subagent param, passes it through all yields
- run_ephemeral calls _run_planning with is_subagent=True
- websocket.py forwards is_subagent in planning_status and plan_ready messages

Frontend (chat.js):
- onPlanningStatus: if is_subagent, set planningLabel on the last spawn_agent
  card instead of msg.statusLabel
- onPlanReady: if is_subagent, push plan into spawn card steps and clear
  planningLabel; otherwise behave as before

Frontend (ToolCard.vue):
- Render subagent-planning-indicator (spinner + label) when planningLabel set
- Render plan cards inside subagent steps using the same plan-card pattern

Also includes leftover session changes: spawn_agent default 40 in description
and manual, updated manual content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove context_transfer from all user-facing prompts — internal mechanism only ...

context_transfer is the scratchpad section name used internally by spawn_agent
to auto-inject parent state. Navi doesn't control it and doesn't need to know
about it. Removed from: persona, secretary, server_admin, spawn_agent description,
manual. Internal code (spawn_agent.py) still reads the section transparently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix core subagent misuse: enforce 1 plan step = 1 spawn_agent call ...

Root cause: nowhere was it stated that each AGENT step in the plan
maps to a separate spawn_agent call. Navi was bundling all AGENT steps
into a single call, dumping the full plan on one subagent.

spawn_agent description:
- Lead with: "Delegate EXACTLY ONE step of your plan"
- Explicit: "3 AGENT steps = 3 spawn_agent calls"
- Remove "multi-step sub-task" wording that invited bundling
- briefing: clarify as static context only (credentials, paths, instructions)
  Dynamic findings from prior steps → context_transfer, not briefing

Planning Phase 2 prompt:
- Add AGENT scoping rules: each step = one focused unit, not "do everything"
- Add good/bad examples of AGENT step granularity
- Show multiple AGENT steps in the format example

Secretary & server_admin system prompts:
- Add explicit 1:1 rule with counter-example
- Show correct multi-agent execution pattern with code example
- Clarify briefing vs context_transfer boundary everywhere

Persona:
- "ONE PLAN STEP = ONE spawn_agent CALL" as first sentence in SUB-AGENTS
- Field descriptions tightened: briefing = static, context_transfer = dynamic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Subagent system prompt rework: separate from parent, briefing as system context ...

run_ephemeral:
- Add briefing param (passed from spawn_agent, injected into system prompt)
- Subagent system prompt is now completely separate from parent's system_prompt:
    1. profile.subagent_system_prompt  (executor persona)
    2. custom_system_prompt            (role specialisation for this task)
    3. briefing                        (task context as system-level instruction)
  Fallback to profile.system_prompt only if subagent_system_prompt is not defined

spawn_agent:
- task → user message only (the goal)
- briefing → system prompt (credentials, context, instructions)
- system_prompt → role specialisation injected alongside briefing
- Removed old user-message composition (## Context / ## Task split)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix spawn_agent: restore briefing, fix status leakage, enable subagent planning ...

spawn_agent:
- Restore briefing param (task = goal, briefing = context — good separation)
- Add system_prompt as third param for role specialisation per task
- Remove [STATUS: ...] prefix that was leaking into Navi's responses and
  causing hallucination — replaced with natural-language headers that are
  less likely to be regurgitated verbatim
- completed → neutral header; limit_reached → explicit warning about incompleteness

Profiles:
- subagent_planning_enabled: false → true in all three profiles
  (planning is on by default, disable per-profile if needed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve subagent system: isolated tools, custom prompts, context transfer, timeout ...

AgentProfile:
- New fields: subagent_tools, subagent_planning_enabled, subagent_system_prompt
- loader.py: loads subagent_tools/subagent_planning_enabled from config.json,
  reads optional subagent_system_prompt.txt per profile

Profiles:
- Each profile now has a dedicated subagent_tools list (focused subset, no admin tools)
- subagent_planning_enabled: false (configurable per profile)
- New subagent_system_prompt.txt per profile with executor-focused instructions

run_ephemeral:
- Uses profile.subagent_tools instead of enabled_tools
- Builds subagent context without persona or profiles block (focused executor)
- Injects subagent_system_prompt after profile.system_prompt
- Accepts context_transfer: priming exchange injected before task message
- Wall-clock timeout (default 5 min) checked per iteration
- Returns (result_text, completed: bool) instead of bare string
- Optionally runs planning phase if profile.subagent_planning_enabled

spawn_agent:
- Removed briefing param; task is now fully self-contained
- Added system_prompt param: custom injected prompt for this specific task
- Auto-reads parent scratchpad context_transfer section via get_section()
- Result prefixed with [STATUS: completed|limit_reached]
- Timeout 300s

scratchpad:
- Added get_section(session_id, section) helper for cross-session reads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Terminal icon ⚡, subagent max_iterations 20→100 ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Major feature batch: visibility, planning, file uploads, streaming ...

- stream_complete(): streaming with tools for all LLM turns — thinking
  now streams as ThinkingDelta/ThinkingEnd in real-time during tool-
  selection turns, not just on the final response
- todo built-in tool: session-scoped plan manager (set/view/update/clear);
  persona + all profiles updated with mandatory planning instructions
- TurnThinking event: sub-agent thinking forwarded to parent sink as a
  collapsible block in the spawn_agent card
- File uploads: non-image files uploaded via XHR, shown as badges in
  message bubble; SVG treated as regular file (not base64 image)
- session_files: POST /sessions/{id}/files, TTL cleanup, forbidden exts
- WebSocket reconnect: _AgentRun broadcast pattern, re-attach mid-stream
- UI: favicon, sidebar logo, turn-thinking cards, subagent thinking blocks,
  token counter, draft persistence, file progress bar
- Removed AgentNote (content is always None alongside tool_calls)
- Ollama stream_complete: tool_calls captured from non-final chunk (done=False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

spawn_agent: fix model behavior — synthesis reminder + lower delegation threshold ...

- ToolResult.output now prefixed with explicit reminder that the user
  cannot see sub-agent output and the orchestrator must present findings
  in its own final response; reminder appears adjacent to the result so
  the model reads it in context
- spawn_agent description updated: lowers threshold from "3+" to "2+"
  tool calls, makes USER CANNOT SEE warning prominent, tightens wording
- persona.txt DELEGATION section: clearer rule (default to spawning for
  multi-step sub-tasks), explicit "never end turn after spawn results",
  removed abstract threshold in favour of concrete examples

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add spawn_agent: sub-agent delegation with isolated context ...

- Agent.run_ephemeral() — runs a sub-agent loop without a persistent
  session; accepts exclude_tools to block recursion; logs start/complete
- session_store made Optional in Agent.__init__ (None for ephemeral runs)
- SpawnAgentTool (navi/tools/spawn_agent.py): spawns an isolated Agent
  for a focused task; resolves profile from parent session via ContextVar;
  blocks spawn_agent recursion via exclude_tools=["spawn_agent"]
- build_default_registries() accepts session_store param; registers
  SpawnAgentTool after BackendRegistry is built (patches _backend_registry)
- deps.py passes _session_store to build_default_registries
- All profiles: spawn_agent added to enabled_tools, max_iterations 10→30
- persona.txt: DELEGATION section — when/how to use spawn_agent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>