fix(recall): stabilize scheduled callback system and improve UX ...

Backend fixes:
- stop_session now stops headless recall runs via _busy_sessions dict
- _fire_recall sets user ContextVars so tools work correctly
- MaxIterationsReached treated as success, not failure
- skip_next_recall uses GREATEST(trigger_at, now) for overdue recalls
- schedule_recall rejects past trigger times
- timezone offset double-adjustment fixed for aware datetimes
- _fire_recall registers _AgentRun for reconnect/replay support
- session_sync race with stream_start fixed

Frontend improvements:
- Recall banner moved to ChatHeader with live Cancel/Skip buttons
- Recall messages styled with is_recall flag and badge
- Real-time recall updates via WebSocket (recall_update events)
- Recall filter moved to sessions-header as toggle button
- Session list shows clock icon for sessions with pending recall
- Empty state messages for empty/filtered session lists
- Fixed missing api import in ChatHeader.vue

Tests:
- Updated scheduler_loop tests for _busy_sessions dict change

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Persist uploaded files in messages, live file tree updates, and UI polish ...

Backend:
- Add `files` field to `Message` model so uploaded file metadata survives page refresh
- Pass `files` through websocket handler → `agent.run_stream` / `agent.run`
- `list_tools`: make `profile_id` required; return error instead of all-tools fallback

Webclient:
- Call `fetchFiles()` after successful file upload for immediate Files tab update
- Live refresh file tree on filesystem (write/edit/append/mkdir/rm/cp/mv), terminal, and code_exec tool calls
- Add manual refresh button (desktop) and pull-to-refresh (mobile) to Files tab
- Fix live link updates: move regex creation inside per-message loop to avoid lastIndex state leak
- Restore full profile name text next to avatar in ChatHeader; hide avatar in header
- Fix mobile ArtifactsPanel: collapse tab text labels so close button fits with 3 tabs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix MCP tool lookup in websocket handler and tests ...

- websocket.py: pass mcp_manager to Agent(), with graceful fallback
  if MCP connection fails
- conftest.py: mock get_mcp_manager() in tests to prevent SSE
  connection attempts against real servers

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Propagate user profile to LLM context via current_user_info ContextVar ...

- Extend User model: username, first_name, last_name, phone, birth_date,
  country, city, locale (all from gnexus-auth profile)
- navi_users DDL: add new profile columns
- auth/deps + auth/callback: populate new fields on upsert
- /auth/me: return all profile fields
- Add current_user_info ContextVar for full user profile propagation
- websocket + messages: set current_user_info before agent.run()
- run_ephemeral: inherit and restore current_user_info
- ContextBuilder: _user_context_msg() injects [User context] with name,
  email, location, locale, role into LLM system messages
- _security_policy_msg: reads user_id/role from ContextVar directly

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add multi-user sandbox: filesystem, terminal, code_exec, security policy ...

- filesystem, share_file: sandbox non-admin users to user_data/<user_id>/
- terminal: working_dir sandbox + allowlist + dangerous pattern block for users
- code_exec: sandbox CWD and temp files to user_data/<user_id>/ for users
- context_builder: inject dynamic security policy into LLM context (user/admin)
- config: terminal_user_allowed_commands setting
- agent: wire user_id/user_role through ContextBuilder.build()
- base: add current_user_role ContextVar; run_ephemeral inherits role

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix stop_session 422: use get_current_user instead of get_current_user_ws ...

stop_session is an HTTP endpoint but was using Depends(get_current_user_ws),
whose websocket parameter caused FastAPI to demand it as a query param.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 — bypass FastAPI Depends for WS auth ...

- websocket.py: resolve user by calling get_current_user_ws directly
  inside the handler instead of using Depends(). This avoids FastAPI
  returning HTTP 403 on the upgrade request when auth resolution fails.
- websocket.py: accept WebSocket before access check, close with 4003
  for auth failures instead of HTTP 403.
- auth/deps.py: remove debug logging from get_current_user_ws.
- tests/conftest.py: monkeypatch get_current_user_ws directly since
  Depends() is no longer used on the WebSocket endpoint.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 and restore dependency resolution for auth ...

- websocket.py: restore Depends(get_current_user_ws) in endpoint signature
  so FastAPI dependency_overrides work correctly in tests
- websocket.py: accept WebSocket before access check; reject anonymous
  only for owned sessions, allow anonymous for legacy (user_id=None)
- auth/deps.py: add info-level logging to get_current_user_ws entry/exit

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 by accepting before access check ...

- websocket.py: move await websocket.accept() before check_session_access
  so auth failures close the WebSocket with 4003 instead of returning
  HTTP 403 on the upgrade request
- Reject anonymous WebSocket connections explicitly (4003)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix legacy session visibility and add WebSocket auth debug logging ...

- pg_session_store: remove OR user_id IS NULL from list_all/list_page
  so legacy sessions are no longer visible to all users
- auth/deps.py: add debug logging at every step of _resolve_user
- websocket.py: add debug logging at every stage of websocket_session

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Multi-user auth via gnexus-auth OAuth + hybrid role/permission model ...

- Integrate gnexus-auth-client-py (GAuthClient) for OAuth flow, token refresh,
  and webhook parsing
- Add navi/auth/ package: User model, Fernet encryptor, client singleton,
  deps (get_current_user, require_admin, require_permission)
- New tables: navi_users, user_auth_sessions (auto-created on startup)
- Session/memory isolation by user_id with legacy NULL support
- Cookie-based auth proxy: /auth/login, /callback, /logout, /me
- Webhook receiver /webhooks/gnexus-auth handling user events, global logout,
  session revocation, role/permission changes
- Admin endpoints (/admin/*) gated by role + permissions
- Webclient auth store with isAdmin/hasPermission guards
- Admin-only profile filtering in /agents/profiles
- 200/200 tests passing

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stability fixes batch — tech debt review 2026-04-29 ...

Critical:
- Concurrent WS run race guard (#1)
- Tool task cancellation on generator teardown (#2)
- StopAsyncIteration kills fallback chain (#3)
- Session loading race with _lastLoadId guard (#4)
- ContentCard .match() crash on non-string result (#5)
- Image data type guard in buildMessageList (#6)

High:
- Cap WS replay buffer at 500 events (#7)
- Deduplicate memory extraction task with asyncio.Lock (#9)
- TTL-based fallback blacklisting (5 min) (#10)
- Subagent tool exception isolation (#11)
- Inline image size/count validation on WS (#12)
- Clean up orphaned file on DB insert failure (#13)
- Deep watch streamingMsg for auto-scroll (#14)
- WS_SCHEME wss:// support for HTTPS (#15)
- Sending guard against duplicate message sends (#16)
- Global unhandledrejection listener in API layer (#17)

Medium:
- Cap planning_logs at 20 entries (#22)
- Store cleanup_loop task reference (#23)
- BaseException → Exception in _run_with_sentinel (#24)
- Propagate SystemExit in agent loop (#25)
- Configurable output_reserve_tokens (#26)
- Always reloadSession on session_sync (#30)
- FIFO queue for confirm dialogs (#31)
- Reset body.overflow on ImageLightbox unmount (#32)
- try/finally in fallback copy (#33)
- _isConnecting guard in WS send() (#34)

Low:
- Lazy-init deps.py singletons (#36)
- Replace __import__ with direct imports (#38)
- Preserve token count 0 in ollama.py (#39)
- Clear orphaned streamingMsg on reconnect reload (#43)
- Escape single quote in UserMessage (#44)
- Polyfill-free findLast replacement (#48)
- Match <table> tags with attributes in markdown (#49)
- Attach copy buttons only when msg.done (#50)
- Fix hasMeta falsy-0 bug (#53)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix system prompt leakage into chat history; polish content cards ...

Backend:
- websocket.py + agent.py: separate user-visible display_message from
  LLM user_message. System hints (image/file attachments) no longer leak
  into session.messages and appear after page reload.
- main.py: add ensure_tables() on startup so session_content table is
  created before first publish.
- profiles: add kimi-k2.6:cloud to all model lists as fallback.

Frontend:
- ContentCard.vue: remove border-radius, add scrollbar styles, fix
  metadata fallback parsing so cards survive page reload.
- content-viewers/*.html: add matching scrollbar styles.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content hosting system with inline viewers ...

Backend:
- Add navi/content/ directory for published files
- Add content_store.py with publish/list/delete/cleanup functions
- Add content_publish tool for publishing files as viewable content
- Add /content static file mount in main.py
- Add /content-viewers mount for viewer pages
- Extend ToolEvent with metadata field
- Forward metadata through websocket tool_call events
- Update Agent to include metadata in ToolEvent

Frontend:
- Add ContentCard.vue component for displaying published content
- Add viewer pages: stl.html (Three.js), svg.html, html.html, pdf.html
- Update AssistantMessage.vue to render ContentCard for content_publish
- Update chat store to preserve metadata in tool cards
- Update websocket protocol docs with metadata field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stop image_view hallucinations on inline-attached images ...

The model was inventing fake paths/URLs (e.g. files.oaiusercontent.com,
/home/ubuntu/navi-1/input_file_0.png) and calling image_view on them
when the user attached an image directly in chat — the image was
already in the multimodal context, but the tool description and lack
of a signal pushed the model to "load" it anyway.

- websocket.py: when a user message has inline images, append a brief
  note that they are already in context.
- image_view.py: soften the description — keep proactive use for paths
  and URLs the model genuinely cannot see, but tell it inline images
  don't need this tool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix websocket.py: unpack 4-tuple from get_registries(), pass cp_registry to Agent ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

WebSocket event replay buffer for disconnect resilience ...

On reconnect to an active agent run the server now replays all events
emitted since the turn started, then switches to live forwarding.
This eliminates the gap where tool cards, thinking blocks and stream
deltas were permanently lost after a network blip.

Server (_AgentRun):
- events: list[dict] buffers every serialised agent event
- broadcast() serialises and appends before putting in subscriber queues
- reconnect flow: subscribe → replay_count snapshot → stream_start →
  replay events[0:replay_count] → live _stream_to_client

Client:
- onStreamStart() removes the frozen ghost message instead of marking
  done=true, so replay cleanly rebuilds the message from scratch
- replayMode flag suppresses animations during replay
- onReplayStart/onReplayEnd handlers set/clear the flag and restore
  animate on the message once live events resume

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Route subagent planning events into spawn_agent card in the UI ...

Previously PlanningStatus/PlanReady had no is_subagent flag, so subagent
planning spinners and plan cards rendered as top-level Navi planning UI.

Backend:
- Add is_subagent field to PlanningStatus and PlanReady events
- _run_planning accepts is_subagent param, passes it through all yields
- run_ephemeral calls _run_planning with is_subagent=True
- websocket.py forwards is_subagent in planning_status and plan_ready messages

Frontend (chat.js):
- onPlanningStatus: if is_subagent, set planningLabel on the last spawn_agent
  card instead of msg.statusLabel
- onPlanReady: if is_subagent, push plan into spawn card steps and clear
  planningLabel; otherwise behave as before

Frontend (ToolCard.vue):
- Render subagent-planning-indicator (spinner + label) when planningLabel set
- Render plan cards inside subagent steps using the same plan-card pattern

Also includes leftover session changes: spawn_agent default 40 in description
and manual, updated manual content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Planning phases, context compression, and tool improvements ...

Agent:
- Planning now a 3-phase async generator: Analysis → Execution plan → AIHelper critic
- Yield PlanningStatus events before each phase (UI progress labels)
- Phase 1 runs with think=True for deeper analysis
- Phase 2 includes available tool list so executor assignments are accurate
- Phase 3: independent critic pass validates and corrects TOOL: names against real tool list
- Planning converted from list return to async generator (fixes token accounting)

Backend:
- Context compression threshold: 80% → 70% to trigger earlier
- Compressor summary prompt: structured sections (goal, work state, key facts, outputs, errors)
- Terminal output capped at 5000 chars to prevent context flooding
- Web search: region=wt-wt for DDG, country=ALL for Brave, language=all for SearxNG
- Scratchpad: mandate writing a 'goal' section at start of multi-step tasks
- secretary max_iterations: 40→25, temperature: 0.7→0.5
- server_admin max_iterations: 40→20

Webclient:
- ThinkingCard strips <thought> XML tags leaked by Ollama
- planning_status WS event wired to chat.onPlanningStatus()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix WS disconnect and missed stream on reconnect ...

Two related problems:
- During long AIHelper calls (non-streaming LLM), no data flows to the
  WebSocket and browsers drop the connection after ~30-60s of inactivity.
  Fixed with a 20s heartbeat: _stream_to_client now uses asyncio.wait_for
  and sends {"type":"heartbeat"} on timeout to keep the connection alive.

- After reconnect, if the agent finished while the client was offline,
  _runs no longer holds the session and no stream_start is sent. Client
  would reconnect silently with no response shown. Fixed by sending
  {"type":"session_sync"} on every new WS connection (after reattach
  completes or immediately when no run is active) so the client knows
  to reload session history.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add response metrics: elapsed time, tool calls, token count ...

Server:
- Message model: elapsed_seconds, tool_call_count, token_count fields
  (display-only, excluded from LLM context via exclude_none)
- StreamEnd event: carries same three fields
- agent.run_stream: tracks turn start time, counts ToolEvent completions,
  writes metrics onto the final assistant Message before saving to DB
- WebSocket: forwards metrics in stream_end payload

Client:
- chat.onStreamEnd: attaches elapsed_seconds, tool_call_count, token_count
  to the streaming message on completion
- buildMessageList: scans each assistant group for metrics from history
- AssistantMessage: renders .msg-meta-row below the response —
  timer icon + Xs · wrench icon + N tools · coins icon + Nk tokens · time
  (each item only shown if present; time pushed right via margin-left: auto)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Expose compression summary as collapsible debug card in chat UI ...

ContextCompressed event now carries the full summary text produced by the
LLM. Compression notice in chat becomes a <details> element showing
message count (before→after) with the summary expandable on click.
Rendered as markdown via marked.js.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add planning phase and scratchpad tool for smarter task execution ...

- ScratchpadTool: session-scoped working notepad with named sections
  (write/append/read/clear). Lets Navi capture intermediate findings
  between tool calls instead of losing track of them.

- Planning phase: when profile.planning_enabled=True, a fast pre-loop
  LLM call (think=False, no tools) outlines a numbered plan before
  any actions are taken. The plan is injected into session context as
  an assistant message so the model naturally continues from it.

- PlanReady event + plan_ready WebSocket message + plan card in UI
  (green-tinted, collapsible, mirroring thinking card design).

- secretary and server_admin profiles: planning_enabled=True,
  scratchpad added to enabled_tools, system prompts updated with
  explicit execution discipline instructions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix WebSocket state corruption preventing messages after first reply ...

Replace concurrent WS reads (_stream_recv + recv_task.cancel()) with
HTTP stop endpoint (POST /sessions/{id}/stop). Cancelling a background
receive_text() task corrupted Starlette's WS state, breaking all
subsequent receives. Now the WS has a single reader at all times.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add stop button and fix context compression hang ...

Stop generation:
- Client: send button toggles to red ■ during streaming; sends {type:stop} via WS
- Server: _stream_recv concurrently reads incoming messages during streaming using
  asyncio.wait — stop signal is handled immediately without polling
- Cooperative stop via asyncio.Event (current_stop_event ContextVar): agent breaks
  out of LLM async-for cleanly so aclose() fires → Ollama stream closes gracefully,
  model stays in VRAM. No task.cancel() which would eject the model.
- StreamStopped event propagates through run_stream/run_ephemeral; sub-agents stop
  via the same shared stop_event inherited through task context

Context compression fix:
- compress_context passes think=False to llm.complete() — no extended reasoning
  during summarization which caused GPU hang
- Input truncated to 12k chars before sending to summarizer
- LLMBackend.complete() / OllamaBackend.complete() accept think: bool | None override

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Profile switch: emit WS event so client updates UI immediately ...

ProfileSwitched event emitted by switch_profile tool via current_event_sink.
Client handles profile_switched: updates chat header, profile selector,
and local sessions[] — no page refresh needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Major feature batch: visibility, planning, file uploads, streaming ...

- stream_complete(): streaming with tools for all LLM turns — thinking
  now streams as ThinkingDelta/ThinkingEnd in real-time during tool-
  selection turns, not just on the final response
- todo built-in tool: session-scoped plan manager (set/view/update/clear);
  persona + all profiles updated with mandatory planning instructions
- TurnThinking event: sub-agent thinking forwarded to parent sink as a
  collapsible block in the spawn_agent card
- File uploads: non-image files uploaded via XHR, shown as badges in
  message bubble; SVG treated as regular file (not base64 image)
- session_files: POST /sessions/{id}/files, TTL cleanup, forbidden exts
- WebSocket reconnect: _AgentRun broadcast pattern, re-attach mid-stream
- UI: favicon, sidebar logo, turn-thinking cards, subagent thinking blocks,
  token counter, draft persistence, file progress bar
- Removed AgentNote (content is always None alongside tool_calls)
- Ollama stream_complete: tool_calls captured from non-final chunk (done=False)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Live tool visibility: pending cards, sub-agent step log ...

Backend:
- ToolStarted event: emitted before tool execution begins so client
  can render a pending card with spinner immediately
- ToolEvent gains is_subagent flag; ToolStarted same
- current_event_sink ContextVar in tools/base.py — run_stream() sets it
  to an asyncio.Queue before create_task(); run_ephemeral() reads it and
  puts ToolStarted/ToolEvent into the queue as each sub-agent step runs
- run_stream() tool loop: sequential execution via create_task() +
  polling drain loop (20ms sleep); yields ToolStarted → sub-agent events
  from sink → ToolEvent (completed) for each tool call
- run_ephemeral() rewritten to inline sequential tool execution with
  sink emission (replaces _execute_tool_calls gather)
- _run_single_tool() helper extracted for run_stream()
- websocket.py handles tool_started and adds is_subagent to tool_call

Frontend:
- appendPendingToolCard(): creates card with spinner; spawn_agent opens
  body immediately to show sub-agent log as it fills
- finalizeToolCard(): fills result, removes spinner, adds toggle; strips
  "[Sub-agent result — ...]" reminder prefix from displayed text
- appendSubagentStep() / finalizeSubagentStep(): live step log inside
  spawn_agent card — each sub-agent tool call gets a ↳ row
- app.js: tool_started → pending card; tool_call → finalize card;
  is_subagent routing to sub-step vs main card; abandonStream() resets
  pendingToolCard/pendingSubStep
- CSS: .spinner-inline for card headers; .subagent-log / .subagent-step
  for nested step display; .tool-body-open for always-open spawn_agent
  body; .tool-card.pending suppresses chevron

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>