Add explicit output token budget for summarizer (context_summary_max_tokens) ...

Previously there was no num_predict set for the summarization LLM call,
so Ollama used its server default (often 128 tokens — very short summaries).

- Add max_tokens param to LLMBackend.complete() and OllamaBackend (→ num_predict)
- Add context_summary_max_tokens: int = 1024 to config
- Thread it through compress_context() and CompressionWorker

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Expand summarization budget for better context quality ...

- _MAX_SUMMARY_INPUT_CHARS: 12k → 24k chars (2x input fed to summarizer)
- context_keep_recent: 10 → 8 turns (2 more turns go into each summary batch)
- Summarizer prompt: replace "Be brief" with "Be thorough" — capture code/config
  snippets and enough detail to continue the conversation without original messages

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Consolidate memory_search/save/forget into single memory tool ...

Three separate tools → one tool with action enum (save/search/forget/list).
Reduces tool-slot pressure; same functionality, same MemoryStore backend.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add memory_save tool for proactive fact persistence ...

Navi previously had no way to write to memory mid-conversation — she
could only search and forget. Facts were extracted automatically after
sessions went idle for 30+ min, so important context shared by the user
could be lost or delayed.

- New MemorySaveTool (navi/tools/memory_save.py): upsert a fact by
  category/key/value; overwrites existing key so no separate forget needed
- Registered as builtin alongside memory_search/memory_forget
- Added to all three profiles (secretary, server_admin, smart_home)
- persona.txt: explicit "call memory_save immediately when..." guidance
  so Navi saves stable facts as they arrive, not only post-session

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Expose compression summary as collapsible debug card in chat UI ...

ContextCompressed event now carries the full summary text produced by the
LLM. Compression notice in chat becomes a <details> element showing
message count (before→after) with the summary expandable on click.
Rendered as markdown via marked.js.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add share_file tool and session-lifetime file storage ...

Session file directories now live until the session is deleted, not
24h TTL. Cleanup loop only removes orphaned dirs (session gone from DB).

New share_file tool: copies any file to the session directory and returns
a clickable download URL. Navi can call this after generating any file
the user will want to keep.

New GET /sessions/{id}/files/{filename} endpoint serves files with
correct Content-Disposition (inline for images/HTML/PDF, attachment
for everything else).

Added PUBLIC_URL config key for building correct download links behind
reverse proxies.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve filesystem, web search, context guard, and subagent narration ...

filesystem: add find (glob), info (stat), move, append actions; read now
supports offset/limit with hard 1MB guard; list shows sizes, dates,
optional recursion.

web_search: retry DDG across auto/html/lite backends; add optional Brave
Search API and SearXNG fallbacks configured via .env.

agent: fix ContextTooLargeError to surface as Navi response instead of
raw system error; fix _check_context_size to calculate from remaining
budget (window - output_reserve) rather than a fixed 92% threshold.

persona: add ReAct narration instruction to subagent briefing template.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix LLM hang: stop button during prefill, context guard, timeouts ...

Root cause: during prefill (processing input tokens), Ollama emits no
HTTP chunks. The `async for chunk in stream_complete()` loop body never
executes, so stop_event is never checked — Stop button has no effect.
Same issue with complete() calls (planning, compression): blocking await
with no cancellation path.

Fixes:

_iter_stream_guarded() (agent.py, module-level):
  Wraps any stream_complete() generator. Polls stop_event every 1s while
  waiting for the next chunk using asyncio.wait() — so Stop works even
  during multi-minute prefill. On stop or timeout, calls aclose() on the
  generator which closes the HTTP connection to Ollama → generation halts
  → GPU drops to idle. Applied to both run_stream() and run_ephemeral().

_check_context_size() (Agent method):
  Estimates context tokens (chars/4 + 500 per image) before every LLM
  call. Raises ContextTooLargeError (new NaviError subclass) at 92% of
  ollama_num_ctx — before Ollama ever receives the request.

_run_planning() timeouts:
  Both complete() calls (phase 1 and 2) wrapped with asyncio.wait_for().
  Timeout logged and planning skipped gracefully — execution continues.

New config (config.py):
  llm_complete_timeout = 120s
  llm_stream_first_chunk_timeout = 180s  (prefill budget)
  llm_stream_chunk_timeout = 60s         (inter-token budget)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve planning: two-phase pipeline and orchestrator discipline ...

agent.py:
- _run_planning() now runs two sequential LLM calls:
  Phase 1 (analysis): reformulate task, identify subtasks and unknowns;
  skip immediately if DIRECT.
  Phase 2 (execution plan): assign each subtask an executor —
  TOOL/AGENT/SELF — using a structured ## Plan format.
  Phase 2 context = analysis (embedded in system prompt) + last user
  message only; full history excluded to keep focus on plan structure.
- Warn in logs when plan lacks TOOL/AGENT/SELF executor assignments.

persona.txt:
- MANDATORY sequence: step 0 = scratchpad init before anything else;
  todo tasks must mirror plan steps exactly (same order, same executors).
- PLAN → EXECUTION BINDING: explicit rule — never switch an AGENT step
  to inline execution silently.
- SCRATCHPAD: initialize sections at task start, not after first tool call;
  write context to scratchpad before briefing subagents.
- Fix typo in BRIEFING ("sub-lagent" → "sub-agent").
- Replace stale Knowledge Retrieval Protocol with accurate one-liner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

self edited

Review and tighten all system prompts ...

persona.txt:
- PLANNING: threshold now 'plan has 2+ steps' instead of '2+ tool calls'
- MEMORY: remove mandatory session-start memory_search (was conflicting with
  planning order); replace with contextual trigger rules
- SCRATCHPAD: add 'if you've written anything to it' qualifier before read
- DELEGATION: clarify sequential spawning is fine; tighten when-not-to-spawn

secretary: trim redundant execution discipline, add 'test in code_exec before
writing to disk' rule, profile-specific scratchpad sections

server_admin: add explicit diagnostic workflow (gather → diagnose → act),
profile-specific scratchpad sections, expanded safety and delegation guidance

smart_home: add 'read-before-act' rule (check entity state before modifying),
profile-specific scratchpad sections, tighten safety rules

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Strengthen Navi planning/delegation, unify toolsets, isolate subagent scratchpad ...

persona.txt:
- DELEGATION: 'default to spawning, not to doing inline' — stronger default,
  clearer triggers, explicit when-not-to-spawn rules
- PLANNING: ties automatic planning phase to mandatory todo(op='set') as first
  tool call; reconciles pre-loop plan with in-loop execution discipline
- SCRATCHPAD: new section — when to write, section naming conventions,
  mandatory read before final answer

Profiles (secretary, server_admin, smart_home):
- All three now share the same 18-tool set (each file independent)
- planning_enabled=True on all three
- scratchpad and web_search added to smart_home
- System prompts updated with scratchpad/todo execution discipline sections

agent.py run_ephemeral:
- Each subagent gets a unique session ID (subagent_<uuid>) for scratchpad
  isolation — parallel or sequential subagents no longer share working notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Skip planning phase for simple/direct requests ...

The planning prompt now asks the model to respond with "DIRECT" if the
request doesn't need multiple steps. Added a regex fallback: if the
response has no numbered steps it's also discarded. This prevents plan
cards appearing for conversational replies that would just duplicate
the final message.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add planning phase and scratchpad tool for smarter task execution ...

- ScratchpadTool: session-scoped working notepad with named sections
  (write/append/read/clear). Lets Navi capture intermediate findings
  between tool calls instead of losing track of them.

- Planning phase: when profile.planning_enabled=True, a fast pre-loop
  LLM call (think=False, no tools) outlines a numbered plan before
  any actions are taken. The plan is injected into session context as
  an assistant message so the model naturally continues from it.

- PlanReady event + plan_ready WebSocket message + plan card in UI
  (green-tinted, collapsible, mirroring thinking card design).

- secretary and server_admin profiles: planning_enabled=True,
  scratchpad added to enabled_tools, system prompts updated with
  explicit execution discipline instructions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix WebSocket state corruption preventing messages after first reply ...

Replace concurrent WS reads (_stream_recv + recv_task.cancel()) with
HTTP stop endpoint (POST /sessions/{id}/stop). Cancelling a background
receive_text() task corrupted Starlette's WS state, breaking all
subsequent receives. Now the WS has a single reader at all times.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add stop button and fix context compression hang ...

Stop generation:
- Client: send button toggles to red ■ during streaming; sends {type:stop} via WS
- Server: _stream_recv concurrently reads incoming messages during streaming using
  asyncio.wait — stop signal is handled immediately without polling
- Cooperative stop via asyncio.Event (current_stop_event ContextVar): agent breaks
  out of LLM async-for cleanly so aclose() fires → Ollama stream closes gracefully,
  model stays in VRAM. No task.cancel() which would eject the model.
- StreamStopped event propagates through run_stream/run_ephemeral; sub-agents stop
  via the same shared stop_event inherited through task context

Context compression fix:
- compress_context passes think=False to llm.complete() — no extended reasoning
  during summarization which caused GPU hang
- Input truncated to 12k chars before sending to summarizer
- LLMBackend.complete() / OllamaBackend.complete() accept think: bool | None override

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Terminal icon ⚡, subagent max_iterations 20→100 ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix ssh_exec: is_closing() → is_closed() (asyncssh API) ...

SSHClientConnection has no is_closing() — that's asyncio Transport.
The correct public method is is_closed().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix save(): persist profile_id to DB ...

profile_id was never included in the UPDATE statement — only set on
initial INSERT. Profile switches appeared to work in-memory but reverted
on page reload or server restart.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix profile switch: reload tools/schema after switch_profile tool call ...

switch_profile updates profile_id in DB, but run_stream() held a stale
local session object — the final save would overwrite the change, and
subsequent LLM calls in the same turn still used the old tool schemas.

After each tool-call iteration, compare DB profile_id with the local
session object. On mismatch: update session.profile_id, reload profile,
tools, tool_schemas, and llm backend so the next LLM call gets the
correct schema and the final save preserves the new profile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add ssh_exec to secretary profile ...

Was missing, causing 'tool not found' when Navi tried to SSH from secretary mode.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Dynamic system prompt — inject per-call instead of storing in context ...

System prompt is no longer stored in session.context. Instead,
_build_context() prepends the current profile's system prompt fresh on
every LLM call. This means profile switches take effect immediately on
the next message — no stale prompt lingering in stored context.

Also strips any existing system messages from context for migration
safety (old sessions that have one stored will still work).

_with_memory() removed, replaced by _build_context(context, profile, mem).
run_ephemeral() context no longer includes system message either.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix ssh_exec error reporting + circular import in switch_profile ...

ssh_exec:
- Non-zero exit status: output (stdout/stderr) now included in error
  field so model can actually see why the command failed
- All error cases: error field now contains human-readable message
  instead of short code (no_target, permission_denied, timeout)
- Named connections from ssh_hosts.json: default known_hosts=none
  same as ad-hoc connections (was missing, caused host key failures)

switch_profile:
- Move ProfileSwitched import inside execute() to break circular
  import: tools.__init__ → switch_profile → core.events → core.agent
  → core.registry → tools

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Profile switch: emit WS event so client updates UI immediately ...

ProfileSwitched event emitted by switch_profile tool via current_event_sink.
Client handles profile_switched: updates chat header, profile selector,
and local sessions[] — no page refresh needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add switch_profile tool for automatic profile switching ...

Navi can now switch her own profile mid-session when the task domain
changes. The new profile (tools + system prompt) takes effect from the
next user message. Injected with session_store + profile_registry like
SpawnAgentTool. Added to all profiles' enabled_tools and persona.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Major feature batch: visibility, planning, file uploads, streaming ...

- stream_complete(): streaming with tools for all LLM turns — thinking
  now streams as ThinkingDelta/ThinkingEnd in real-time during tool-
  selection turns, not just on the final response
- todo built-in tool: session-scoped plan manager (set/view/update/clear);
  persona + all profiles updated with mandatory planning instructions
- TurnThinking event: sub-agent thinking forwarded to parent sink as a
  collapsible block in the spawn_agent card
- File uploads: non-image files uploaded via XHR, shown as badges in
  message bubble; SVG treated as regular file (not base64 image)
- session_files: POST /sessions/{id}/files, TTL cleanup, forbidden exts
- WebSocket reconnect: _AgentRun broadcast pattern, re-attach mid-stream
- UI: favicon, sidebar logo, turn-thinking cards, subagent thinking blocks,
  token counter, draft persistence, file progress bar
- Removed AgentNote (content is always None alongside tool_calls)
- Ollama stream_complete: tool_calls captured from non-final chunk (done=False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Live tool visibility: pending cards, sub-agent step log ...

Backend:
- ToolStarted event: emitted before tool execution begins so client
  can render a pending card with spinner immediately
- ToolEvent gains is_subagent flag; ToolStarted same
- current_event_sink ContextVar in tools/base.py — run_stream() sets it
  to an asyncio.Queue before create_task(); run_ephemeral() reads it and
  puts ToolStarted/ToolEvent into the queue as each sub-agent step runs
- run_stream() tool loop: sequential execution via create_task() +
  polling drain loop (20ms sleep); yields ToolStarted → sub-agent events
  from sink → ToolEvent (completed) for each tool call
- run_ephemeral() rewritten to inline sequential tool execution with
  sink emission (replaces _execute_tool_calls gather)
- _run_single_tool() helper extracted for run_stream()
- websocket.py handles tool_started and adds is_subagent to tool_call

Frontend:
- appendPendingToolCard(): creates card with spinner; spawn_agent opens
  body immediately to show sub-agent log as it fills
- finalizeToolCard(): fills result, removes spinner, adds toggle; strips
  "[Sub-agent result — ...]" reminder prefix from displayed text
- appendSubagentStep() / finalizeSubagentStep(): live step log inside
  spawn_agent card — each sub-agent tool call gets a ↳ row
- app.js: tool_started → pending card; tool_call → finalize card;
  is_subagent routing to sub-step vs main card; abandonStream() resets
  pendingToolCard/pendingSubStep
- CSS: .spinner-inline for card headers; .subagent-log / .subagent-step
  for nested step display; .tool-body-open for always-open spawn_agent
  body; .tool-card.pending suppresses chevron

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

spawn_agent: fix model behavior — synthesis reminder + lower delegation threshold ...

- ToolResult.output now prefixed with explicit reminder that the user
  cannot see sub-agent output and the orchestrator must present findings
  in its own final response; reminder appears adjacent to the result so
  the model reads it in context
- spawn_agent description updated: lowers threshold from "3+" to "2+"
  tool calls, makes USER CANNOT SEE warning prominent, tightens wording
- persona.txt DELEGATION section: clearer rule (default to spawning for
  multi-step sub-tasks), explicit "never end turn after spawn results",
  removed abstract threshold in favour of concrete examples

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add spawn_agent: sub-agent delegation with isolated context ...

- Agent.run_ephemeral() — runs a sub-agent loop without a persistent
  session; accepts exclude_tools to block recursion; logs start/complete
- session_store made Optional in Agent.__init__ (None for ephemeral runs)
- SpawnAgentTool (navi/tools/spawn_agent.py): spawns an isolated Agent
  for a focused task; resolves profile from parent session via ContextVar;
  blocks spawn_agent recursion via exclude_tools=["spawn_agent"]
- build_default_registries() accepts session_store param; registers
  SpawnAgentTool after BackendRegistry is built (patches _backend_registry)
- deps.py passes _session_store to build_default_registries
- All profiles: spawn_agent added to enabled_tools, max_iterations 10→30
- persona.txt: DELEGATION section — when/how to use spawn_agent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

SSH connection pooling: per-session, 20-minute TTL ...

- Pool keyed by session_id:host:port:username — parallel sessions share
  no state even when targeting the same server
- Per-key asyncio.Lock prevents concurrent connection creation races
- TTL (20 min) and is_closing() checked on every access; expired/closed
  connections are evicted and replaced transparently
- On disconnect error during command execution: evict + retry once with
  fresh connection
- current_session_id ContextVar (set by Agent before tool calls) is read
  in ssh_exec to build the pool key without changing tool signatures

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>