Add artifact source previews

Add 3D modeling profile with model_3d and render_3d tools ...

New profile:
- modeler_3d: 3D model design for 3D printing (OpenSCAD → STL)

New tools:
- model_3d: compile .scad → binary STL via OpenSCAD CLI
- render_3d: render PNG previews from STL (up to 3 views, 400×300)

Both tools return openscad_not_found error if OpenSCAD is not installed.

Docs:
- README.md: add modeler_3d to profile table
- docs/profiles.md: add modeler_3d to active profiles
- docs/tools.md: add model_3d and render_3d to builtin tools
- manuals/model_3d.md, manuals/render_3d.md: full usage manuals
- modeler_3d system_prompt.txt: OpenSCAD-first workflow with dedicated tools

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Improve content publishing UX

Clarify share file publishing boundaries

Docs audit: fix outdated references (PostgreSQL, Vue client, profiles, tools) ...

- architecture.md: SQLite → PostgreSQL + pgvector
- README.md: vanilla JS → Vue 3 + Pinia, remove smart_home, update temps
- profiles.md: remove smart_home, add discuss, update temperatures
- tools.md: add missing built-in tools, update user tools list
- sessions.md: fix outdated references
- NAVI.md: sync with current stack
- persona.txt: update self-extension and content_publish guidance
- content_publish.md manual: reflect session_dir-based no-copy design
- content_store.py & content_publish.py: inline with docs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content_publish manual and improve tool description ...

- Create manuals/content_publish.md with full usage guide,
  internal mechanics, supported types, and best practices
- Update ContentPublishTool description with usage guidelines
  and examples in the tool schema
- All tests pass (150 backend + 47 webclient)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rework content_publish: files live in session_dir, no copying ...

content_publish now registers existing session files for inline viewing
instead of copying them to navi/content/<uuid>/. This makes files
editable by the agent and immediately visible to the user via the
stable URL /sessions/{id}/files/{filename}.

- content_store.publish() → registers metadata, no shutil.copy2
- content_store.delete_content() → removes DB record only (file stays
  in session_dir; cleaned up by session lifecycle)
- content_store.cleanup_old() → deletes stale DB records
- ContentPublishTool:
  - accepts 'filename' instead of 'path'
  - looks up file in uploads/sessions/<id>/
  - returns clear error if file not found with instructions
  - includes full local path in output for future editing
- Added unique index on (session_id, filename) for idempotent re-publish
- Removed /content static mount from main.py (endpoint still works
  for legacy files; new files use /sessions/{id}/files/)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture cleanup: old_webclient, SSH shutdown, todo decoupling, ContextVar reset ...

- Remove old_webclient directory and /static mount from main.py
- Add shutdown handler in main.py to close all SSH pooled connections
- Decouple agent.py from todo module internals:
  - Add public API to navi/tools/todo.py: get_task_snapshot, get_failed_steps,
    get_progress_message, set_tasks, render_todo_lines
  - Replace all direct _plans/_STATUS_ICON/_Task imports in agent.py with API calls
- Wrap run_ephemeral in try/finally to restore _sid_var and _model_var after subagent

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content hosting system with inline viewers ...

Backend:
- Add navi/content/ directory for published files
- Add content_store.py with publish/list/delete/cleanup functions
- Add content_publish tool for publishing files as viewable content
- Add /content static file mount in main.py
- Add /content-viewers mount for viewer pages
- Extend ToolEvent with metadata field
- Forward metadata through websocket tool_call events
- Update Agent to include metadata in ToolEvent

Frontend:
- Add ContentCard.vue component for displaying published content
- Add viewer pages: stl.html (Three.js), svg.html, html.html, pdf.html
- Update AssistantMessage.vue to render ContentCard for content_publish
- Update chat store to preserve metadata in tool cards
- Update websocket protocol docs with metadata field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Wire pgvector semantic search into memory system ...

- Add vector(768) column + HNSW index to memory_facts

- Add LLMBackend.embed() with Ollama + fallback implementation

- MemoryStore: cosine-distance search with ILIKE fallback

- New memory tool params: source, confidence, expires_days, source_context

- Update extractor, sqlite_store, deps wiring

- Add pgvector to requirements

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stop image_view hallucinations on inline-attached images ...

The model was inventing fake paths/URLs (e.g. files.oaiusercontent.com,
/home/ubuntu/navi-1/input_file_0.png) and calling image_view on them
when the user attached an image directly in chat — the image was
already in the multimodal context, but the tool description and lack
of a signal pushed the model to "load" it anyway.

- websocket.py: when a user message has inline images, append a brief
  note that they are already in context.
- image_view.py: soften the description — keep proactive use for paths
  and URLs the model genuinely cannot see, but tell it inline images
  don't need this tool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Strengthen todo progress discipline

Tune reflect autonomy guidance

Fix profile prompt inconsistencies

filesystem mkdir: parents=False to prevent accidental path chain creation ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add mkdir action to filesystem tool (parents=True, exist_ok=True) ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add context providers: dynamic system message injection per LLM call ...

- navi/context_providers/ registry + built-in public_url provider (global, always injected)
- context_providers/ user directory, hot-reloaded via reload_tools
- AgentProfile.context_providers field for per-profile opt-in providers
- Agent._collect_context_injections() called before every tool-calling loop
- reload_tools now reloads both user tools and user context providers
- manuals/write_context_provider.md for Navi, docs/context_providers.md reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add Ollama multi-server fallback with in-memory blacklisting ...

- New FallbackOllamaBackend (navi/llm/fallback.py): tries servers and
  models in priority order; on LLMConnectionError blacklists the server
  for the process lifetime, on LLMModelNotFoundError blacklists the
  (server, model) pair — eliminates latency from repeated failed probes
- OllamaBackend now raises typed LLMConnectionError / LLMModelNotFoundError
  instead of bare LLMBackendError; accepts list[str] | str | None for model
- AgentProfile.model changed from str to list[str] (str auto-normalised);
  all profiles updated to ["gemma4:31b-cloud", "gemma4:26b-a4b-it-q4_K_M"]
- New config field OLLAMA_BACKENDS_FILE: path to [{host, api_key?}] JSON;
  when set, registry creates FallbackOllamaBackend instead of OllamaBackend
- ollama_backends.json template added (gitignored — contains API key)
- current_model ContextVar type widened to list[str] | str | None

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Make server I/O non-blocking; update docs ...

- Wrap all heavy filesystem ops in asyncio.to_thread: filesystem tool
  (read/write/append/list/find/info/move/delete/query/smart_edit),
  image_view (read_bytes), share_file (shutil.copy2), write_tool
  (write_text, _register_user_tool), session_files (shutil.rmtree,
  iterdir), sessions upload endpoint (sync open/write → to_thread)
- Make delete_session_dir async; update its caller in sessions.py
- docs/config.md: fix wrong defaults (threshold 0.70, keep_recent 8),
  remove phantom SESSION_FILES_TTL_HOURS, add LLM timeouts, DATABASE_URL,
  PUBLIC_URL, Gmail, CONTEXT_SUMMARY_MAX_TOKENS sections
- docs/profiles.md: add missing tool_developer profile to table
- android-client: add WebView remote debugging; remove unused toolbar menu
- Remove stale helper scripts and test files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Agent improvements: mandatory planning, tool cleanup, smart_edit fixes ...

- Planning now mandatory on first message of every session (force_plan)
- RESOURCES, COMMITMENTS, ATOMICITY fields added to planning phase 1
- Todo auto-injected at iteration 0 so model tracks steps immediately
- Execution trigger injected after plan to prevent model treating plan as response
- Split developer profile: tool_developer (Navi tools) vs developer (general code)
- Simplified persona.txt: trimmed redundant content now handled by mechanics
- AIHelper.ask(): 120s timeout via asyncio.wait_for to prevent smart_edit hangs
- filesystem._smart_edit(): atomic write via temp file + os.replace()
- Removed 5 junk user tools (game project artifacts, trivial utilities)
- Removed instagram tools (to be rewritten); cleaned enabled.json

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Restructure persona: concrete information gathering protocol ...

- Replace abstract CONTEXT FIRST principle with explicit numbered protocol
  (check summary → memory_search → NAVI.md → ask user)
- Move protocol to top of persona (after identity) so it has maximum weight
- NAVI.md section: remove READ instructions (now in protocol), keep only WRITE
- LONG-TERM MEMORY: remove "search proactively" line (now in protocol),
  keep only save/forget instructions
- memory_search description: remove misleading "call at start of each session"
  (summary is already auto-injected); clarify when to actually call it

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Autonomous reasoning improvements: budget, anchoring, anti-stall, validation ...

- AgentProfile: per-profile thinking mechanics flags (think_enabled,
  iteration_budget_enabled, goal_anchoring, anti_stall, step_validation,
  planning_reflect, adaptive_replan) — all profiles updated in config.json
- Iteration budget: inject remaining iterations into context so model knows
  when to wrap up; urgency levels at ≤7 and ≤3 remaining
- Goal anchoring: inject original goal + todo state every N iterations to
  prevent drift on long tasks
- Anti-stall: two signals — no todo progress for N iterations, or identical
  tool calls repeated N times; warning injected into context
- Todo step validation: marking done requires a validation field describing
  how result was verified; failed gets a soft nudge with tip for re-planning
- stream_complete: add think param to base class, ollama and openai backends
- Summarizer: raise max_tokens 1024→3000, expand system prompt with
  user-preferences section and verbatim-value instructions
- Compression card: persist to session.messages (is_compression flag on
  Message), show expandable summary in webclient with markdown body
- ToolResult.to_message_content: always include output on failure so
  tracebacks and error details reach the model (fixes silent Error: None)
- Developer profile: fix subagent profile secretary→developer, add write_tool
  to subagent_tools, clarify write_tool vs filesystem in system prompt

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Planning debug panel, todo auto-populate, scratchpad/persona improvements ...

- Planning debug panel: new Planning tab in debug/index.html shows raw
  phase 1/2 outputs and token counts per planning run, stored in
  session.planning_logs (new column in both SQLite and PostgreSQL)
- New GET /sessions/{id}/planning API endpoint
- PlanningDebugData internal event wires _run_planning() output into
  session storage; never forwarded to WebSocket clients
- Phase 3 (plan critic) disabled — to be reworked with reflect integration
- Todo tool: auto-populated from plan steps after phase 2; model only
  needs to call update/view, not set
- Scratchpad: clarified description and persona instructions; removed
  context_transfer from user-facing docs (internal mechanism only)
- web_search: switched to ddgs package, SearXNG as primary backend,
  DDG html-only fallback; added find_up action to filesystem tool
- Persona: added SCRATCHPAD and TODO sections with clear usage rules;
  added NAVI.md project context instructions
- chat.js: fixed subagent planning event fallthrough into parent UI;
  statusLabel cleared on first stream delta

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Route subagent planning events into spawn_agent card in the UI ...

Previously PlanningStatus/PlanReady had no is_subagent flag, so subagent
planning spinners and plan cards rendered as top-level Navi planning UI.

Backend:
- Add is_subagent field to PlanningStatus and PlanReady events
- _run_planning accepts is_subagent param, passes it through all yields
- run_ephemeral calls _run_planning with is_subagent=True
- websocket.py forwards is_subagent in planning_status and plan_ready messages

Frontend (chat.js):
- onPlanningStatus: if is_subagent, set planningLabel on the last spawn_agent
  card instead of msg.statusLabel
- onPlanReady: if is_subagent, push plan into spawn card steps and clear
  planningLabel; otherwise behave as before

Frontend (ToolCard.vue):
- Render subagent-planning-indicator (spinner + label) when planningLabel set
- Render plan cards inside subagent steps using the same plan-card pattern

Also includes leftover session changes: spawn_agent default 40 in description
and manual, updated manual content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Remove context_transfer from all user-facing prompts — internal mechanism only ...

context_transfer is the scratchpad section name used internally by spawn_agent
to auto-inject parent state. Navi doesn't control it and doesn't need to know
about it. Removed from: persona, secretary, server_admin, spawn_agent description,
manual. Internal code (spawn_agent.py) still reads the section transparently.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix core subagent misuse: enforce 1 plan step = 1 spawn_agent call ...

Root cause: nowhere was it stated that each AGENT step in the plan
maps to a separate spawn_agent call. Navi was bundling all AGENT steps
into a single call, dumping the full plan on one subagent.

spawn_agent description:
- Lead with: "Delegate EXACTLY ONE step of your plan"
- Explicit: "3 AGENT steps = 3 spawn_agent calls"
- Remove "multi-step sub-task" wording that invited bundling
- briefing: clarify as static context only (credentials, paths, instructions)
  Dynamic findings from prior steps → context_transfer, not briefing

Planning Phase 2 prompt:
- Add AGENT scoping rules: each step = one focused unit, not "do everything"
- Add good/bad examples of AGENT step granularity
- Show multiple AGENT steps in the format example

Secretary & server_admin system prompts:
- Add explicit 1:1 rule with counter-example
- Show correct multi-agent execution pattern with code example
- Clarify briefing vs context_transfer boundary everywhere

Persona:
- "ONE PLAN STEP = ONE spawn_agent CALL" as first sentence in SUB-AGENTS
- Field descriptions tightened: briefing = static, context_transfer = dynamic

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Subagent system prompt rework: separate from parent, briefing as system context ...

run_ephemeral:
- Add briefing param (passed from spawn_agent, injected into system prompt)
- Subagent system prompt is now completely separate from parent's system_prompt:
    1. profile.subagent_system_prompt  (executor persona)
    2. custom_system_prompt            (role specialisation for this task)
    3. briefing                        (task context as system-level instruction)
  Fallback to profile.system_prompt only if subagent_system_prompt is not defined

spawn_agent:
- task → user message only (the goal)
- briefing → system prompt (credentials, context, instructions)
- system_prompt → role specialisation injected alongside briefing
- Removed old user-message composition (## Context / ## Task split)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix spawn_agent: restore briefing, fix status leakage, enable subagent planning ...

spawn_agent:
- Restore briefing param (task = goal, briefing = context — good separation)
- Add system_prompt as third param for role specialisation per task
- Remove [STATUS: ...] prefix that was leaking into Navi's responses and
  causing hallucination — replaced with natural-language headers that are
  less likely to be regurgitated verbatim
- completed → neutral header; limit_reached → explicit warning about incompleteness

Profiles:
- subagent_planning_enabled: false → true in all three profiles
  (planning is on by default, disable per-profile if needed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Improve subagent system: isolated tools, custom prompts, context transfer, timeout ...

AgentProfile:
- New fields: subagent_tools, subagent_planning_enabled, subagent_system_prompt
- loader.py: loads subagent_tools/subagent_planning_enabled from config.json,
  reads optional subagent_system_prompt.txt per profile

Profiles:
- Each profile now has a dedicated subagent_tools list (focused subset, no admin tools)
- subagent_planning_enabled: false (configurable per profile)
- New subagent_system_prompt.txt per profile with executor-focused instructions

run_ephemeral:
- Uses profile.subagent_tools instead of enabled_tools
- Builds subagent context without persona or profiles block (focused executor)
- Injects subagent_system_prompt after profile.system_prompt
- Accepts context_transfer: priming exchange injected before task message
- Wall-clock timeout (default 5 min) checked per iteration
- Returns (result_text, completed: bool) instead of bare string
- Optionally runs planning phase if profile.subagent_planning_enabled

spawn_agent:
- Removed briefing param; task is now fully self-contained
- Added system_prompt param: custom injected prompt for this specific task
- Auto-reads parent scratchpad context_transfer section via get_section()
- Result prefixed with [STATUS: completed|limit_reached]
- Timeout 300s

scratchpad:
- Added get_section(session_id, section) helper for cross-session reads

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>