cleanup: remove deprecated tools and orphaned memory tools ...

Removed (no profile used them, no cross-dependencies):
- write_tool.py, delete_tool.py, test_tool.py
- memory_save.py, memory_search.py, memory_forget.py

Updated:
- navi/core/registry.py — removed imports and registrations
- navi/tools/__init__.py — removed imports
- docs/tools.md — removed references, updated self-extension section to MCP
- navi/tools/tool_manual.py — updated example to create_mcp_server

Profile fixes:
- developer: +tool_manual, +ssh_exec
- discuss: +list_profiles
- tool_developer: +mcp_servers navi-web

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix profile MCP server groups to use group names instead of tool names ...

Profile configs were listing individual tool names (web_view, http_request)
in mcp_servers.navi-web instead of group names (browse, request). This caused
resolve_group() to return empty results, so MCP tools never appeared in the
profile's resolved tool list.

Also fixes modeler_3d stale tool references in system_prompt.txt.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename MCP tools to mcp:server:tool format and restore human-readable names ...

- Core naming: mcp_server_tool → mcp:server:tool (colon-delimited)
- navi-web tools: search/view/request → web_search/web_view/http_request
- navi-3d tools: compile_scad/render_stl/lint_scad (unchanged names)
- Updated all profile configs, system prompts, docs, manuals, tests
- Added new lint_scad.md manual
- Fixed modeler_3d prompt stale references (scad_lint, model_3d, render_3d)
- All 240 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up remaining web_search / web_view / http_request references ...

- Deleted stale source files: navi/tools/web_search.py, web_view.py,
  http_request.py (no longer imported or registered).
- Updated system_prompt.txt in all profiles (secretary, server_admin,
  discuss, modeler_3d subagent) to reference mcp_navi-web_* tool names.
- Updated key_tools in developer, secretary, server_admin config.json.
- Updated docs (tools.md, profiles.md, api.md) to reflect MCP migration.

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rename navi-web MCP tools and compact search output ...

- web_search → search, web_view → view, http_request → request
  This removes the redundant 'web' prefix from tool names in the
  navi-web MCP server, making them cleaner: mcp_navi-web_search etc.

- Compact search output: output string now contains only
  [Source] + [N] Title — URL, without duplicating the body snippet.
  Full results with body/description remain in metadata for
  programmatic use.

- Updated mcp_servers.json groups and all profile configs.

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate web tools (web_search, web_view, http_request) to navi-web MCP server ...

New MCP server: mcp-servers/navi-web/
- app/search.py    — SearXNG primary, DDG fallback, Brave tertiary
- app/browse.py    — Playwright headless browser page extraction
- app/request.py   — httpx raw HTTP requests
- app/mcp_server.py — FastMCP entry point with stdio transport
- pyproject.toml   — deps: mcp, httpx, ddgs, playwright, pydantic

Changes:
- Added "navi-web" block to mcp_servers.json (stdio transport)
- Removed web_search, web_view, http_request from built-in registry
- Removed them from navi/tools/__init__.py
- Removed them from enabled_tools / subagent_tools in all profiles
- Added "navi-web": ["search", "browse", "request"] to mcp_servers
  in all profiles (developer, secretary, server_admin, tool_developer,
  modeler_3d, discuss)

Tests: 240 passed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Increase max_iterations across all profiles ...

- developer: 45 → 100
- discuss: 12 → 30
- modeler_3d: 50 → 70
- secretary: 35 → 70
- server_admin: 40 → 100
- tool_developer: 35 → 100

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Improve content publishing UX

Fix system prompt leakage into chat history; polish content cards ...

Backend:
- websocket.py + agent.py: separate user-visible display_message from
  LLM user_message. System hints (image/file attachments) no longer leak
  into session.messages and appear after page reload.
- main.py: add ensure_tables() on startup so session_content table is
  created before first publish.
- profiles: add kimi-k2.6:cloud to all model lists as fallback.

Frontend:
- ContentCard.vue: remove border-radius, add scrollbar styles, fix
  metadata fallback parsing so cards survive page reload.
- content-viewers/*.html: add matching scrollbar styles.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

changed llm

changed llm & new ollama param

changed model

Add structured planning review and adaptive depth

Tune profile sampling configs

temp -

Set temperature=1.0, top_k=64, top_p=0.95 for all profiles (Google recommended for gemma4) ...

Also fixes discuss profile memory tools: use combined "memory" tool name, not nonexistent split variants.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add per-phase planning flags and planning_mandatory ...

- planning_mandatory: disables DIRECT shortcut, forces all phases to run
- planning_phase1_enabled / phase2_enabled / phase3_enabled: per-phase toggles
- planning_phase2_enabled replaces planning_reflect_enabled (migrated in loader with backward compat)
- Migrate all profile configs; rewrite docs/profiles.md as full config reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add Ollama multi-server fallback with in-memory blacklisting ...

- New FallbackOllamaBackend (navi/llm/fallback.py): tries servers and
  models in priority order; on LLMConnectionError blacklists the server
  for the process lifetime, on LLMModelNotFoundError blacklists the
  (server, model) pair — eliminates latency from repeated failed probes
- OllamaBackend now raises typed LLMConnectionError / LLMModelNotFoundError
  instead of bare LLMBackendError; accepts list[str] | str | None for model
- AgentProfile.model changed from str to list[str] (str auto-normalised);
  all profiles updated to ["gemma4:31b-cloud", "gemma4:26b-a4b-it-q4_K_M"]
- New config field OLLAMA_BACKENDS_FILE: path to [{host, api_key?}] JSON;
  when set, registry creates FallbackOllamaBackend instead of OllamaBackend
- ollama_backends.json template added (gitignored — contains API key)
- current_model ContextVar type widened to list[str] | str | None

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Use gemma4 cloud model by default

Support Ollama Cloud API key

Agent improvements: mandatory planning, tool cleanup, smart_edit fixes ...

- Planning now mandatory on first message of every session (force_plan)
- RESOURCES, COMMITMENTS, ATOMICITY fields added to planning phase 1
- Todo auto-injected at iteration 0 so model tracks steps immediately
- Execution trigger injected after plan to prevent model treating plan as response
- Split developer profile: tool_developer (Navi tools) vs developer (general code)
- Simplified persona.txt: trimmed redundant content now handled by mechanics
- AIHelper.ask(): 120s timeout via asyncio.wait_for to prevent smart_edit hangs
- filesystem._smart_edit(): atomic write via temp file + os.replace()
- Removed 5 junk user tools (game project artifacts, trivial utilities)
- Removed instagram tools (to be rewritten); cleaned enabled.json

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add instagram_engine and instagram_viewer tools (Navi-generated) ...

Browser automation tools for scraping public Instagram profiles using
Playwright + stealth. Registered in enabled.json and developer profile.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove hello_world test tool and incomplete instagram_scraper ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Adaptive re-plan on todo step failure ...

When a todo step is newly marked failed, queue a targeted system message
for the next iteration prompting the model to revise its remaining pending
steps before continuing. Enabled by adaptive_replan_enabled flag (on by
default in developer profile). Zero overhead when no failure occurs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Autonomous reasoning improvements: budget, anchoring, anti-stall, validation ...

- AgentProfile: per-profile thinking mechanics flags (think_enabled,
  iteration_budget_enabled, goal_anchoring, anti_stall, step_validation,
  planning_reflect, adaptive_replan) — all profiles updated in config.json
- Iteration budget: inject remaining iterations into context so model knows
  when to wrap up; urgency levels at ≤7 and ≤3 remaining
- Goal anchoring: inject original goal + todo state every N iterations to
  prevent drift on long tasks
- Anti-stall: two signals — no todo progress for N iterations, or identical
  tool calls repeated N times; warning injected into context
- Todo step validation: marking done requires a validation field describing
  how result was verified; failed gets a soft nudge with tip for re-planning
- stream_complete: add think param to base class, ollama and openai backends
- Summarizer: raise max_tokens 1024→3000, expand system prompt with
  user-preferences section and verbatim-value instructions
- Compression card: persist to session.messages (is_compression flag on
  Message), show expandable summary in webclient with markdown body
- ToolResult.to_message_content: always include output on failure so
  tracebacks and error details reach the model (fixes silent Error: None)
- Developer profile: fix subagent profile secretary→developer, add write_tool
  to subagent_tools, clarify write_tool vs filesystem in system prompt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix spawn_agent: restore briefing, fix status leakage, enable subagent planning ...

spawn_agent:
- Restore briefing param (task = goal, briefing = context — good separation)
- Add system_prompt as third param for role specialisation per task
- Remove [STATUS: ...] prefix that was leaking into Navi's responses and
  causing hallucination — replaced with natural-language headers that are
  less likely to be regurgitated verbatim
- completed → neutral header; limit_reached → explicit warning about incompleteness

Profiles:
- subagent_planning_enabled: false → true in all three profiles
  (planning is on by default, disable per-profile if needed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve subagent system: isolated tools, custom prompts, context transfer, timeout ...

AgentProfile:
- New fields: subagent_tools, subagent_planning_enabled, subagent_system_prompt
- loader.py: loads subagent_tools/subagent_planning_enabled from config.json,
  reads optional subagent_system_prompt.txt per profile

Profiles:
- Each profile now has a dedicated subagent_tools list (focused subset, no admin tools)
- subagent_planning_enabled: false (configurable per profile)
- New subagent_system_prompt.txt per profile with executor-focused instructions

run_ephemeral:
- Uses profile.subagent_tools instead of enabled_tools
- Builds subagent context without persona or profiles block (focused executor)
- Injects subagent_system_prompt after profile.system_prompt
- Accepts context_transfer: priming exchange injected before task message
- Wall-clock timeout (default 5 min) checked per iteration
- Returns (result_text, completed: bool) instead of bare string
- Optionally runs planning phase if profile.subagent_planning_enabled

spawn_agent:
- Removed briefing param; task is now fully self-contained
- Added system_prompt param: custom injected prompt for this specific task
- Auto-reads parent scratchpad context_transfer section via get_section()
- Result prefixed with [STATUS: completed|limit_reached]
- Timeout 300s

scratchpad:
- Added get_section(session_id, section) helper for cross-session reads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Webclient UI improvements + backend fixes ...

Webclient:
- Draft persistence across page refreshes (localStorage, reactive watch)
- Image lightbox modal using UI kit classes on thumbnail click
- Copy button on user and assistant messages
- Selection reply toolbar: select assistant text → quote inserted into input
- User message rendering: proper HTML escaping, styled blockquote for > replies
- Markdown table fix: preprocessor to inject missing separator rows
- Planning status labels (rebuild dist)

Backend:
- Developer profile: enable subagent delegation, increase max_iterations to 35
- share_file: updated description + manual with absolute path requirement and URL sharing
- persona.txt: instructions for quote replies and GFM table format

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add reflect tool: three parallel expert perspectives ...

ReflectTool runs Critic / Pragmatist / Detailer advisors concurrently
via asyncio.gather() + AIHelper.ask(). Each role has a distinct system
prompt designed to produce genuinely different analysis:
- Critic: challenges assumptions, surfaces risks and logical gaps
- Pragmatist: finds the simplest path, cuts unnecessary complexity
- Detailer: spots missing requirements, edge cases, ambiguities

Parameters: situation (required), assumptions (required list — the key
input that forces Navi to surface implicit beliefs), tried (optional).

Registered as a builtin with AIHelper injection. Added to all three
profiles. Persona updated with guidance on when to use it (complex or
ambiguous tasks before planning, or when stuck mid-execution).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>