Improve filesystem, web search, context guard, and subagent narration ...

filesystem: add find (glob), info (stat), move, append actions; read now
supports offset/limit with hard 1MB guard; list shows sizes, dates,
optional recursion.

web_search: retry DDG across auto/html/lite backends; add optional Brave
Search API and SearXNG fallbacks configured via .env.

agent: fix ContextTooLargeError to surface as Navi response instead of
raw system error; fix _check_context_size to calculate from remaining
budget (window - output_reserve) rather than a fixed 92% threshold.

persona: add ReAct narration instruction to subagent briefing template.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve planning: two-phase pipeline and orchestrator discipline ...

agent.py:
- _run_planning() now runs two sequential LLM calls:
  Phase 1 (analysis): reformulate task, identify subtasks and unknowns;
  skip immediately if DIRECT.
  Phase 2 (execution plan): assign each subtask an executor —
  TOOL/AGENT/SELF — using a structured ## Plan format.
  Phase 2 context = analysis (embedded in system prompt) + last user
  message only; full history excluded to keep focus on plan structure.
- Warn in logs when plan lacks TOOL/AGENT/SELF executor assignments.

persona.txt:
- MANDATORY sequence: step 0 = scratchpad init before anything else;
  todo tasks must mirror plan steps exactly (same order, same executors).
- PLAN → EXECUTION BINDING: explicit rule — never switch an AGENT step
  to inline execution silently.
- SCRATCHPAD: initialize sections at task start, not after first tool call;
  write context to scratchpad before briefing subagents.
- Fix typo in BRIEFING ("sub-lagent" → "sub-agent").
- Replace stale Knowledge Retrieval Protocol with accurate one-liner.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.

Review and tighten all system prompts ...

persona.txt:
- PLANNING: threshold now 'plan has 2+ steps' instead of '2+ tool calls'
- MEMORY: remove mandatory session-start memory_search (was conflicting with
  planning order); replace with contextual trigger rules
- SCRATCHPAD: add 'if you've written anything to it' qualifier before read
- DELEGATION: clarify sequential spawning is fine; tighten when-not-to-spawn

secretary: trim redundant execution discipline, add 'test in code_exec before
writing to disk' rule, profile-specific scratchpad sections

server_admin: add explicit diagnostic workflow (gather → diagnose → act),
profile-specific scratchpad sections, expanded safety and delegation guidance

smart_home: add 'read-before-act' rule (check entity state before modifying),
profile-specific scratchpad sections, tighten safety rules

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Strengthen Navi planning/delegation, unify toolsets, isolate subagent scratchpad ...

persona.txt:
- DELEGATION: 'default to spawning, not to doing inline' — stronger default,
  clearer triggers, explicit when-not-to-spawn rules
- PLANNING: ties automatic planning phase to mandatory todo(op='set') as first
  tool call; reconciles pre-loop plan with in-loop execution discipline
- SCRATCHPAD: new section — when to write, section naming conventions,
  mandatory read before final answer

Profiles (secretary, server_admin, smart_home):
- All three now share the same 18-tool set (each file independent)
- planning_enabled=True on all three
- scratchpad and web_search added to smart_home
- System prompts updated with scratchpad/todo execution discipline sections

agent.py run_ephemeral:
- Each subagent gets a unique session ID (subagent_<uuid>) for scratchpad
  isolation — parallel or sequential subagents no longer share working notes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add switch_profile tool for automatic profile switching ...

Navi can now switch her own profile mid-session when the task domain
changes. The new profile (tools + system prompt) takes effect from the
next user message. Injected with session_store + profile_registry like
SpawnAgentTool. Added to all profiles' enabled_tools and persona.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add workspace dir + clean up junk from project root ...

- workspace/ — persistent dir for Navi's long-term files (scripts, notes,
  data); excluded from git, Navi instructed to use it instead of project root
- .gitignore: session_files/ and workspace/* added
- persona.txt: WORKSPACE section pointing Navi to workspace/
- Deleted scanner.py, network_scan_results.txt, targets.txt (Navi artifacts)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Major feature batch: visibility, planning, file uploads, streaming ...

- stream_complete(): streaming with tools for all LLM turns — thinking
  now streams as ThinkingDelta/ThinkingEnd in real-time during tool-
  selection turns, not just on the final response
- todo built-in tool: session-scoped plan manager (set/view/update/clear);
  persona + all profiles updated with mandatory planning instructions
- TurnThinking event: sub-agent thinking forwarded to parent sink as a
  collapsible block in the spawn_agent card
- File uploads: non-image files uploaded via XHR, shown as badges in
  message bubble; SVG treated as regular file (not base64 image)
- session_files: POST /sessions/{id}/files, TTL cleanup, forbidden exts
- WebSocket reconnect: _AgentRun broadcast pattern, re-attach mid-stream
- UI: favicon, sidebar logo, turn-thinking cards, subagent thinking blocks,
  token counter, draft persistence, file progress bar
- Removed AgentNote (content is always None alongside tool_calls)
- Ollama stream_complete: tool_calls captured from non-final chunk (done=False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

spawn_agent: fix model behavior — synthesis reminder + lower delegation threshold ...

- ToolResult.output now prefixed with explicit reminder that the user
  cannot see sub-agent output and the orchestrator must present findings
  in its own final response; reminder appears adjacent to the result so
  the model reads it in context
- spawn_agent description updated: lowers threshold from "3+" to "2+"
  tool calls, makes USER CANNOT SEE warning prominent, tightens wording
- persona.txt DELEGATION section: clearer rule (default to spawning for
  multi-step sub-tasks), explicit "never end turn after spawn results",
  removed abstract threshold in favour of concrete examples

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add spawn_agent: sub-agent delegation with isolated context ...

- Agent.run_ephemeral() — runs a sub-agent loop without a persistent
  session; accepts exclude_tools to block recursion; logs start/complete
- session_store made Optional in Agent.__init__ (None for ephemeral runs)
- SpawnAgentTool (navi/tools/spawn_agent.py): spawns an isolated Agent
  for a focused task; resolves profile from parent session via ContextVar;
  blocks spawn_agent recursion via exclude_tools=["spawn_agent"]
- build_default_registries() accepts session_store param; registers
  SpawnAgentTool after BackendRegistry is built (patches _backend_registry)
- deps.py passes _session_store to build_default_registries
- All profiles: spawn_agent added to enabled_tools, max_iterations 10→30
- persona.txt: DELEGATION section — when/how to use spawn_agent

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add long-term user memory system ...

Architecture:
- navi/memory/store.py: MemoryStore backed by SQLite (memory_facts,
  memory_summary, session_memory_state tables in navi.db)
- navi/memory/extractor.py: LLM-based fact extraction from sessions +
  summary regeneration (triggered after session goes idle >30 min)
- Fact upsert uses UNIQUE(category, key) — same key always overwrites,
  no duplicates or stale contradictions
- Keyword search across category + key + value (LIKE-based, no extra deps)

Context injection:
- Memory summary injected as an ephemeral system message on every LLM call
  via Agent._with_memory() — never persisted to session.context

Tools (all profiles):
- memory_search(query): keyword search against fact DB; persona instructs
  model to call it at session start and before personal-context questions
- memory_forget(key, category?): delete a specific fact on user request

Extraction trigger:
- On new session creation, fire-and-forget background task checks all
  sessions idle >30 min with unprocessed messages → runs extraction

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix NAVI_PERSONA not loading: move persona to persona.txt ...

python-dotenv cannot parse multi-line env values that contain blank lines or
lines resembling key=value assignments (like the write_tool code examples).
The persona was silently dropped, leaving Navi with no identity.

Solution: navi_persona_file setting (default empty) — if set, the persona is
read from the specified file at startup via a model_validator. The .env now
points to persona.txt instead of embedding the multi-line string.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>