Add navi_ui form component with client-side validation ...

- Add Form UI component schema in navi/mcp/ui_server/components/form.py
- Support text, textarea, number, email, url, select, multiselect, checkbox, date fields
- Implement hidden user message mode for form submissions (is_display=False, is_context=True)
- Add WebSocket form_submit handling and _start_agent_run helper
- Render forms client-side with real-time JS validation in webclient
- Submit once guard via localStorage and replace form with read-only summary
- Provide wsSend via Vue provide/inject in ChatArea
- Add tests for backend form component, websocket form_submit, and frontend Form.vue
- Update docs/tools.md and docs/websocket.md
- Build production webclient dist

Co-Authored-By: Claude <noreply@anthropic.com>

Enable navi_ui card_grid for realtor profile ...

- Add navi_ui MCP group to realtor profile tools.
- Document card_grid payload shape in realtor system prompt.
- Add card_grid schema validation in navi/mcp/ui_server.py.
- Implement CardGrid.vue: 4-card responsive grid, image/meta/description,
  modal with full details and actions on click.
- Render UI components via dynamic component in UiComponentCard.vue.
- Tests: card_grid payload validation, chat store handler, CardGrid rendering
  and modal open.

Co-Authored-By: Claude <noreply@anthropic.com>

Add internal navi_ui MCP server for structured UI components ...

- Add navi/mcp/ui_server.py: FastMCP streamable_http server on port 8001
  exposing render_component(component_name, payload, session_id).
- Start server in main lifespan before container creation so McpManager can
  connect; wire orchestrator once container is ready; clean up on shutdown.
- Add env settings NAVI_UI_MCP_ENABLED/HOST/PORT.
- Add mcp_servers.d/navi_ui.json config with the 'ui' tool group.
- Frontend: dispatch ui_component websocket event, store in chat.js, render
  placeholder UiComponentCard inside AssistantMessage.vue.
- Unit tests for ui_server tool and chat.onUiComponent.

Co-Authored-By: Claude <noreply@anthropic.com>

Fix scratchpad tool: rename op→action, add examples, improve error messages ...

- Rename primary parameter from 'op' to 'action' for consistency with
  all other tools (terminal, filesystem, todo, etc.). Legacy 'op' still
  works as a fallback to avoid breaking old calls in compressed context.
- Add JSON examples directly in the tool description so the model sees
  the exact structure to produce.
- Improve all error messages to include the correct JSON example,
  making it obvious what the model did wrong.
- Add manuals/scratchpad.md for tool_manual support.
- Update and expand tests for new syntax and error cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Make shared files and published content publicly accessible ...

Remove auth requirements from:
- GET /sessions/{id}/files/{filename} — direct download links (session ID
  acts as unguessable capability token)
- GET /sessions/{id}/content — published inline content list

Both endpoints still verify session exists and protect against path
traversal. File upload and file listing remain auth-gated.

Update tests to match new signatures.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Move terminal_manager to _internal subpackage ...

- terminal_manager is an internal helper, not a tool
- Update imports in terminal.py, container.py, test_terminal.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix terminal review issues: security, lifecycle, reactivity ...

- TerminalManager.open now accepts exec_tokens to use create_subprocess_exec
  for restricted commands instead of always using shell
- Fix kill/terminate order: SIGTERM first, SIGKILL fallback
- Pop closed sessions from _sessions dict to prevent memory leak
- Add terminal_manager.shutdown() to AppContainer.shutdown()
- Wait for reader tasks in foreground open before returning output
- Add _MAX_TERMINALS_PER_SESSION limit (10)
- Wrap cleanup_idle tasks in _close_one_safe with error logging
- send_input catches BrokenPipeError/ConnectionResetError specifically
- Foreground terminals auto-close after gathering output
- Vue reactivity: replace terminals object immutably instead of mutating
- onTerminalClosed marks matching tool card as no longer pending
- Update tests for new behavior (foreground auto-close, max limit)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add persistent multi-session terminal tool with background support ...

- New TerminalManager module: named subprocess sessions per Navi session,
  background readers, event-sink streaming, idle auto-cleanup
- Refactor terminal tool to multi-action: run, open, close, list,
  status, send_input
- Add TerminalOutputDelta and TerminalClosed events for streaming
- Wire TerminalManager into AppContainer, orchestrator, and registry
- Persist session_metadata in Session model and pg_session_store
- Close all session terminals on session delete
- Webclient: handle terminal_output/terminal_closed WS events,
  display live terminal output in tool cards
- Update unit tests for new terminal actions

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix 19 issues found in full codebase review ...

Backend:
- Stop session auth bypass: require auth for owned sessions, reject anonymous with 401
- upload_file: stream chunks directly to disk instead of buffering in RAM
- MCP config: validate name against path traversal regex
- auth deps: cleanup stale refresh locks periodically
- auth routes: expire mobile auth states after 10 min to prevent unbounded growth
- compressor: meta-summarize existing summaries before compression; preserve assistant content when tool_calls present; rewrite hard_truncate to keep whole turns
- orchestrator: configurable WS replay buffer size; async cleanup/remove_websocket/clear_busy; fix run_recall ContextVar order to avoid deadlock on _build_agent failure; await cleanup in finally
- agent: persist image_msg in session.messages; remove archived messages from session after archive; remove duplicate StreamStopped yield on tool stop
- websocket: try/except around create_task with cleanup on failure; await remove_websocket

Frontend:
- App.vue: hashchange listener lifecycle in onMounted/onUnmounted
- MessageList.vue: passive scroll, flash timeout cleanup, archive scroll snapshot
- InputBar.vue: 300 ms debounce on draft save to localStorage
- SessionList.vue: remove :key from DynamicScroller to avoid remount jitter

Tests: 422 passed, 1 skipped

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add meta-summary for multi-level compression ...

When to_summarize contains multiple existing summary messages whose
combined length exceeds 8000 chars (~1/3 of max summarizer input),
run a quick meta-summary pass first to consolidate them into a single
compact summary before the main compression. This prevents information
loss when repeated compressions stack up long summary chains.

- _meta_summarize(): fast LLM pass (think=False, max_tokens=1500)
- compress_context(): detects >1 long summaries and triggers meta pass
- Graceful fallback: if meta-summary fails, continue with raw summaries
- 3 new unit tests: consolidation, skipped for short summaries, failure fallback

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Wire archive trigger into agent after compression ...

After _do_compress_and_save finishes, if the total persisted message count
(db_next_sequence) exceeds session_messages_window (default 1000), the agent
now calls archive_old_messages() to move older rows into
session_messages_archive.

Adds session_messages_window config and unit tests for archive SQL.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Review fixes: restore _build_sessions, fix flags, search filter, tests ...

- Restored _load_messages_map and _build_sessions helpers that were
  accidentally dropped in Phase 5 (list_all/list_page/search_list
  called them but they were missing, causing NameError at runtime)
- _build_sessions now filters messages by is_display so list methods
  return consistent display-only history like get()
- count_all/search_list EXISTS subquery now filters to is_display=true
  so search only matches visible chat messages
- Updated pg_session_store docstring to remove stale dual-write claim
- compressor summary_msg now defaults to is_display=False
- Added unit tests for message flags (agent, compressor, planning)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Phase 2: Dual-write with is_context/is_display flags on Message ...

- Message model gets is_context and is_display bools
- PgSessionStore.save() writes flags directly to session_messages
- Agent sets is_context=False on display-only messages, is_display=False on context-only
- Planning: plan context msg is_display=False, plan marker is_context=False
- Compression: summarized messages get is_context=False, summary added to messages with is_display=False
- Tests updated for extra user display+context messages per turn

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add subagent progress report on failure ...

When a subagent stops (timeout, max iterations, thinking stall, user stop),
it now returns a structured progress report built from its local message
context, so the parent agent knows what tools were called and what was
accomplished before the stop.

- Add _build_progress_report() to SubAgentRunner
- Report includes: turn number, assistant text, tool calls with results
- Prepended to result_text for every stop reason (completed also gets it)
- Updated test_run_ephemeral_complete to expect the report prefix

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix MCP tool spinner bug: match tool_started → tool_call by tool_call_id ...

- Add tool_call_id field to ToolStarted and ToolEvent dataclasses
- Pass tc.id as tool_call_id from agent.py, subagent_runner.py, and tool_executor.py
- Update frontend chat.js onToolStarted/onToolCall to match cards by toolCallId
  with fallback to name-matching for backward compatibility

Closes spinner issue where LLM short name ("search_docs") didn't match
resolved MCP name ("mcp__gnexus_book__search_docs").

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add auth resilience: user cache, retry, and API token fallback ...

- 30-second in-memory _user_cache to avoid hammering gnexus-auth
- _fetch_user_with_retry: one retry after 1.5s sleep on transient failure
- API token fallback when OAuth cookie is present but refresh fails
- Clear cache/locks in test fixture to prevent cross-test pollution
- Fix registry timeout test after lowering default to 90s

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Apply review fixes to API token auth system ...

Backend:
- navi/auth/deps.py: replace 3 DB round-trips with single JOIN query for
  token resolution; update last_used_at still separate (best-effort)
- navi/api/routes/api_tokens.py: replace asyncpg-specific "UPDATE 1"
  string check with RETURNING id fetchrow; increase token_prefix from
  8 to 12 chars for better visual identification; add security notes
- tests/unit/auth/test_api_tokens.py: update tests for JOIN query and
  RETURNING-based revoke

Frontend:
- webclient/src/components/settings/ShowTokenModal.vue: new modal that
  shows the plain token in a readonly field with copy button and
  explicit warning — replaces the transient toast notification
- webclient/src/components/settings/ApiKeysPanel.vue: use ShowTokenModal
- webclient/src/composables/useWebSocket.js: add security comment about
  localStorage XSS risk and query param log exposure

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add API token auth system for headless/micro clients ...

Backend:
- navi/auth/_ddl.py: add api_tokens table with boot-time migration
- navi/auth/deps.py: _resolve_user now falls back to X-Api-Token header
  and ?api_token query param for WebSocket auth
- navi/auth/__init__.py: add ApiToken pydantic model
- navi/api/routes/api_tokens.py: CRUD endpoints (POST/GET/DELETE)
- navi/main.py: wire api_tokens router

Frontend:
- webclient/src/App.vue: add #settings hash routing
- webclient/src/components/settings/: SettingsView, ApiKeysPanel,
  CreateKeyModal with copy-to-clipboard flow
- webclient/src/api/index.js: token CRUD API functions
- webclient/src/stores/apiTokens.js: Pinia store
- webclient/src/components/sidebar/AppSidebar.vue: settings link
- webclient/src/composables/useWebSocket.js: append ?api_token= when
  localStorage token is present

Tests:
- tests/unit/auth/test_api_tokens.py: 10 unit tests covering token
  resolution (header + query param), revoke, missing/revoked tokens,
  orphan users, and CRUD endpoints

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Unify in-memory session state in AgentSessionOrchestrator ...

Replace scattered _runs + _busy_sessions + _session_sockets with a
single _sessions: dict[str, SessionState] on the orchestrator.

- SessionState dataclass holds run, busy_event, and websockets
- _session_sockets module-level global removed from websocket.py;
  socket tracking moved into orchestrator (add/remove_websocket)
- Event bus subscriber _on_recall_update moved into orchestrator
- Per-session asyncio.Lock added to protect concurrent-run guard
- _cleanup() auto-removes empty SessionState entries

Tests updated to reference _sessions instead of legacy _runs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Pass explicit ToolContext to tools instead of hidden ContextVars ...

Add ToolContext dataclass (session_id, event_sink, stop_event, model,
user_id, user_role, user_info) and thread it through the execution chain:
Agent._execute_tools_with_sink → ToolExecutor → tool.execute().

All ~25 tools updated to accept ctx parameter. Tools that previously
read ContextVar now prefer ctx when provided, falling back to
ContextVar for backward compatibility.

Tests updated to pass ToolContext explicitly — no more test fixtures
that set current_session_id / current_user_id ContextVars.

ContextVar setters remain as fallback for non-tool consumers
(ai_helper, context_builder, planning) and will be removed in a
follow-up refactor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix auth race condition causing frequent logouts ...

Add per-session-id asyncio.Lock around token refresh to prevent
parallel requests from simultaneously refreshing the same token.
Re-read the session inside the lock so a second request can use
the token already refreshed by the first one.

Stop deleting the auth session on refresh failure — transient
errors (network, race condition, expired refresh token) were
wiping the session and forcing a full re-login.

+ tests for both behaviours.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix token counting: show only completion tokens, not cumulative prompt+completion ...

The token_count displayed next to assistant messages was summing
prompt_tokens + completion_tokens across ALL tool-calling iterations,
giving hundreds of thousands of tokens for multi-turn conversations.

Now:
- token_count (coins icon) = only completion tokens generated by the model
- context_tokens (ContextBar) = only prompt tokens (context size sent to LLM)

This gives users a realistic measure of how much the model actually generated.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

FallbackOllamaBackend: do not blacklist single server, empty file fallback ...

- When only one Ollama server is configured, LLMConnectionError no longer
  adds it to the dead-server blacklist. This fixes the bug where a
  transient failure permanently blocked all requests until server restart.
- LLMModelNotFoundError on a single server is also not blacklisted.
- _discover_backends now falls back to settings.ollama_host when the
  ollama_backends_file is empty, missing, or returns no valid servers.
- Added unit tests covering single-server no-blacklist, multi-server
  blacklist, model-not-found no-blacklist, and empty-file fallback.

400 passed, 1 skipped

McpTool: auto-inject session_id + normalize navi-3d paths ...

- McpTool.execute() now forces the real session_id from current_session_id
  ContextVar, preventing LLM hallucinations of wrong UUIDs (ghost-session bug).
- For navi-3d MCP server, source_path/output_path are normalized to basename
  to prevent double path nesting when the LLM passes full relative paths.
- Updated MCP tool descriptions to ask for filenames only.
- Added system prompt instructions in context_builder and subagent_runner
  reminding the model to pass bare filenames to navi-3d tools.

396 passed, 1 skipped

Extract single shared Database pool, eliminate 4 duplicated pool creations ...

- Create navi/db.py::Database managing one asyncpg pool
- KvStore, PgSessionStore, MemoryStore, RecallScheduler now accept pool in constructor
- AppContainer holds Database, shutdown closes one pool instead of 4
- create_container creates one pool and passes it to all stores
- All tests updated to set _initialized=True on fakes to skip DDL

392 passed, 1 skipped

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Make Settings immutable (frozen=True) and fix all test mutations ...

- Add frozen=True to SettingsConfigDict in navi/config.py
- Convert model_validator to mode="before" since mode="after" cannot mutate frozen instances
- Replace all field-level monkeypatches in tests with whole-Settings object replacement
- Ensure cross-module settings consistency (content_store, session_files, share_file, content_publish, filesystem)

392 passed, 1 skipped

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extract WebSocket business logic into AgentSessionOrchestrator ...

- Create navi/core/orchestrator.py with AgentSessionOrchestrator and SessionRun
- Orchestrator owns _runs, _busy_sessions, Agent creation, run_agent(), run_recall()
- Transport-agnostic: accepts notify callback from WebSocket handler
- WebSocket handler (websocket.py) now only does serialization/deserialization
- _fire_recall delegates to orchestrator.run_recall() instead of inline logic
- recall_scheduler_loop now accepts orchestrator parameter
- AppContainer gains .orchestrator field, created in create_container()
- deps.py: add get_orchestrator()
- Update integration tests for scheduler_loop and websocket unit tests

All 392 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add WebSocket handler unit tests ...

Tests for reconnect/replay, concurrent-run guard, event buffering,
and session_sync behavior after both normal and recall runs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Replace global lazy singletons with explicit AppContainer + lifespan ...

- Create navi/core/container.py with AppContainer dataclass and create_container()
- Rewrite navi/api/deps.py: remove module-level singletons, add _container global
  fallback + set_container(), use _resolve_container() for all getters
- Replace @app.on_event with @asynccontextmanager lifespan in main.py
- Update routes to use Depends(get_scheduler) instead of calling get_scheduler()
- Fix FastAPI body parsing bug: remove dataclass parameters from Depends getters
  (FastAPI was treating AppContainer sub-dependencies as Body params, forcing
  embed=True on all endpoint body params and causing 422 errors)
- Update websocket.py to use _resolve_container() instead of get_registries()
- Update integration test fixtures to build AppContainer and call set_container()
- Remove obsolete tests/unit/test_startup.py (tests removed _on_startup)
- Fix test_scheduler_loop.py fixture (get_registries no longer exists)

All 387 tests pass (excluding websocket hang tests).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>