Fix dead code in websocket.py — stream_start never sent on first message ...

websocket.py lines 301-308 were indented inside the `except` block after
`raise`, making them unreachable. This meant `stream_start` and
`_stream_to_client` never ran for the initial message send, so the client
saw no response until page reload (when the reconnect path took over).

Fix: dedent lines 301-308 so they execute after the try/except/finally
block on the normal success path.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix 19 issues found in full codebase review ...

Backend:
- Stop session auth bypass: require auth for owned sessions, reject anonymous with 401
- upload_file: stream chunks directly to disk instead of buffering in RAM
- MCP config: validate name against path traversal regex
- auth deps: cleanup stale refresh locks periodically
- auth routes: expire mobile auth states after 10 min to prevent unbounded growth
- compressor: meta-summarize existing summaries before compression; preserve assistant content when tool_calls present; rewrite hard_truncate to keep whole turns
- orchestrator: configurable WS replay buffer size; async cleanup/remove_websocket/clear_busy; fix run_recall ContextVar order to avoid deadlock on _build_agent failure; await cleanup in finally
- agent: persist image_msg in session.messages; remove archived messages from session after archive; remove duplicate StreamStopped yield on tool stop
- websocket: try/except around create_task with cleanup on failure; await remove_websocket

Frontend:
- App.vue: hashchange listener lifecycle in onMounted/onUnmounted
- MessageList.vue: passive scroll, flash timeout cleanup, archive scroll snapshot
- InputBar.vue: 300 ms debounce on draft save to localStorage
- SessionList.vue: remove :key from DynamicScroller to avoid remount jitter

Tests: 422 passed, 1 skipped

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add client-side image resize and server-side image token budgeting ...

Client (useFileUpload.js):
- Resize images to max 1024px on longest side via Canvas before upload
- JPEG quality 0.9, preserves EXIF orientation
- Reduces typical phone photos from 8 MB → ~400-900 KB

Server (websocket.py):
- Increase limit to 8 images, 50 MB total payload

Server (agent.py):
- Before adding user message to session.context, estimate how many images
  fit in remaining context (500 tok/img, up to 80% of max_context_tokens)
- Images that don't fit are saved as resized JPEGs to session_files/
- References to saved images are appended to the user message text
  so Navi can view them later via image_view tool
- session.messages keeps ALL images for full UI history

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix recall race, ContextVar leaks, dead code, and recall duplication ...

- run_recall: wrap busy check + create_run in session_lock to prevent
  race between scheduler and websocket handler
- run_recall: save ContextVar tokens and reset in finally to avoid
  leaking user context into subsequent background tasks
- websocket.py: reset user ContextVars in finally after run completes
- orchestrator.py: remove dead set_notify / _notify abstraction
- orchestrator.py: extract _finalize_recall to deduplicate success /
  MaxIterationsReached / Exception finalization blocks

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Unify in-memory session state in AgentSessionOrchestrator ...

Replace scattered _runs + _busy_sessions + _session_sockets with a
single _sessions: dict[str, SessionState] on the orchestrator.

- SessionState dataclass holds run, busy_event, and websockets
- _session_sockets module-level global removed from websocket.py;
  socket tracking moved into orchestrator (add/remove_websocket)
- Event bus subscriber _on_recall_update moved into orchestrator
- Per-session asyncio.Lock added to protect concurrent-run guard
- _cleanup() auto-removes empty SessionState entries

Tests updated to reference _sessions instead of legacy _runs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extract WebSocket business logic into AgentSessionOrchestrator ...

- Create navi/core/orchestrator.py with AgentSessionOrchestrator and SessionRun
- Orchestrator owns _runs, _busy_sessions, Agent creation, run_agent(), run_recall()
- Transport-agnostic: accepts notify callback from WebSocket handler
- WebSocket handler (websocket.py) now only does serialization/deserialization
- _fire_recall delegates to orchestrator.run_recall() instead of inline logic
- recall_scheduler_loop now accepts orchestrator parameter
- AppContainer gains .orchestrator field, created in create_container()
- deps.py: add get_orchestrator()
- Update integration tests for scheduler_loop and websocket unit tests

All 392 tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Replace global lazy singletons with explicit AppContainer + lifespan ...

- Create navi/core/container.py with AppContainer dataclass and create_container()
- Rewrite navi/api/deps.py: remove module-level singletons, add _container global
  fallback + set_container(), use _resolve_container() for all getters
- Replace @app.on_event with @asynccontextmanager lifespan in main.py
- Update routes to use Depends(get_scheduler) instead of calling get_scheduler()
- Fix FastAPI body parsing bug: remove dataclass parameters from Depends getters
  (FastAPI was treating AppContainer sub-dependencies as Body params, forcing
  embed=True on all endpoint body params and causing 422 errors)
- Update websocket.py to use _resolve_container() instead of get_registries()
- Update integration test fixtures to build AppContainer and call set_container()
- Remove obsolete tests/unit/test_startup.py (tests removed _on_startup)
- Fix test_scheduler_loop.py fixture (get_registries no longer exists)

All 387 tests pass (excluding websocket hang tests).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

fix(recall): stabilize scheduled callback system and improve UX ...

Backend fixes:
- stop_session now stops headless recall runs via _busy_sessions dict
- _fire_recall sets user ContextVars so tools work correctly
- MaxIterationsReached treated as success, not failure
- skip_next_recall uses GREATEST(trigger_at, now) for overdue recalls
- schedule_recall rejects past trigger times
- timezone offset double-adjustment fixed for aware datetimes
- _fire_recall registers _AgentRun for reconnect/replay support
- session_sync race with stream_start fixed

Frontend improvements:
- Recall banner moved to ChatHeader with live Cancel/Skip buttons
- Recall messages styled with is_recall flag and badge
- Real-time recall updates via WebSocket (recall_update events)
- Recall filter moved to sessions-header as toggle button
- Session list shows clock icon for sessions with pending recall
- Empty state messages for empty/filtered session lists
- Fixed missing api import in ChatHeader.vue

Tests:
- Updated scheduler_loop tests for _busy_sessions dict change

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Persist uploaded files in messages, live file tree updates, and UI polish ...

Backend:
- Add `files` field to `Message` model so uploaded file metadata survives page refresh
- Pass `files` through websocket handler → `agent.run_stream` / `agent.run`
- `list_tools`: make `profile_id` required; return error instead of all-tools fallback

Webclient:
- Call `fetchFiles()` after successful file upload for immediate Files tab update
- Live refresh file tree on filesystem (write/edit/append/mkdir/rm/cp/mv), terminal, and code_exec tool calls
- Add manual refresh button (desktop) and pull-to-refresh (mobile) to Files tab
- Fix live link updates: move regex creation inside per-message loop to avoid lastIndex state leak
- Restore full profile name text next to avatar in ChatHeader; hide avatar in header
- Fix mobile ArtifactsPanel: collapse tab text labels so close button fits with 3 tabs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix MCP tool lookup in websocket handler and tests ...

- websocket.py: pass mcp_manager to Agent(), with graceful fallback
  if MCP connection fails
- conftest.py: mock get_mcp_manager() in tests to prevent SSE
  connection attempts against real servers

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Propagate user profile to LLM context via current_user_info ContextVar ...

- Extend User model: username, first_name, last_name, phone, birth_date,
  country, city, locale (all from gnexus-auth profile)
- navi_users DDL: add new profile columns
- auth/deps + auth/callback: populate new fields on upsert
- /auth/me: return all profile fields
- Add current_user_info ContextVar for full user profile propagation
- websocket + messages: set current_user_info before agent.run()
- run_ephemeral: inherit and restore current_user_info
- ContextBuilder: _user_context_msg() injects [User context] with name,
  email, location, locale, role into LLM system messages
- _security_policy_msg: reads user_id/role from ContextVar directly

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add multi-user sandbox: filesystem, terminal, code_exec, security policy ...

- filesystem, share_file: sandbox non-admin users to user_data/<user_id>/
- terminal: working_dir sandbox + allowlist + dangerous pattern block for users
- code_exec: sandbox CWD and temp files to user_data/<user_id>/ for users
- context_builder: inject dynamic security policy into LLM context (user/admin)
- config: terminal_user_allowed_commands setting
- agent: wire user_id/user_role through ContextBuilder.build()
- base: add current_user_role ContextVar; run_ephemeral inherits role

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix stop_session 422: use get_current_user instead of get_current_user_ws ...

stop_session is an HTTP endpoint but was using Depends(get_current_user_ws),
whose websocket parameter caused FastAPI to demand it as a query param.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 — bypass FastAPI Depends for WS auth ...

- websocket.py: resolve user by calling get_current_user_ws directly
  inside the handler instead of using Depends(). This avoids FastAPI
  returning HTTP 403 on the upgrade request when auth resolution fails.
- websocket.py: accept WebSocket before access check, close with 4003
  for auth failures instead of HTTP 403.
- auth/deps.py: remove debug logging from get_current_user_ws.
- tests/conftest.py: monkeypatch get_current_user_ws directly since
  Depends() is no longer used on the WebSocket endpoint.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 and restore dependency resolution for auth ...

- websocket.py: restore Depends(get_current_user_ws) in endpoint signature
  so FastAPI dependency_overrides work correctly in tests
- websocket.py: accept WebSocket before access check; reject anonymous
  only for owned sessions, allow anonymous for legacy (user_id=None)
- auth/deps.py: add info-level logging to get_current_user_ws entry/exit

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix WebSocket 403 by accepting before access check ...

- websocket.py: move await websocket.accept() before check_session_access
  so auth failures close the WebSocket with 4003 instead of returning
  HTTP 403 on the upgrade request
- Reject anonymous WebSocket connections explicitly (4003)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix legacy session visibility and add WebSocket auth debug logging ...

- pg_session_store: remove OR user_id IS NULL from list_all/list_page
  so legacy sessions are no longer visible to all users
- auth/deps.py: add debug logging at every step of _resolve_user
- websocket.py: add debug logging at every stage of websocket_session

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Multi-user auth via gnexus-auth OAuth + hybrid role/permission model ...

- Integrate gnexus-auth-client-py (GAuthClient) for OAuth flow, token refresh,
  and webhook parsing
- Add navi/auth/ package: User model, Fernet encryptor, client singleton,
  deps (get_current_user, require_admin, require_permission)
- New tables: navi_users, user_auth_sessions (auto-created on startup)
- Session/memory isolation by user_id with legacy NULL support
- Cookie-based auth proxy: /auth/login, /callback, /logout, /me
- Webhook receiver /webhooks/gnexus-auth handling user events, global logout,
  session revocation, role/permission changes
- Admin endpoints (/admin/*) gated by role + permissions
- Webclient auth store with isAdmin/hasPermission guards
- Admin-only profile filtering in /agents/profiles
- 200/200 tests passing

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stability fixes batch — tech debt review 2026-04-29 ...

Critical:
- Concurrent WS run race guard (#1)
- Tool task cancellation on generator teardown (#2)
- StopAsyncIteration kills fallback chain (#3)
- Session loading race with _lastLoadId guard (#4)
- ContentCard .match() crash on non-string result (#5)
- Image data type guard in buildMessageList (#6)

High:
- Cap WS replay buffer at 500 events (#7)
- Deduplicate memory extraction task with asyncio.Lock (#9)
- TTL-based fallback blacklisting (5 min) (#10)
- Subagent tool exception isolation (#11)
- Inline image size/count validation on WS (#12)
- Clean up orphaned file on DB insert failure (#13)
- Deep watch streamingMsg for auto-scroll (#14)
- WS_SCHEME wss:// support for HTTPS (#15)
- Sending guard against duplicate message sends (#16)
- Global unhandledrejection listener in API layer (#17)

Medium:
- Cap planning_logs at 20 entries (#22)
- Store cleanup_loop task reference (#23)
- BaseException → Exception in _run_with_sentinel (#24)
- Propagate SystemExit in agent loop (#25)
- Configurable output_reserve_tokens (#26)
- Always reloadSession on session_sync (#30)
- FIFO queue for confirm dialogs (#31)
- Reset body.overflow on ImageLightbox unmount (#32)
- try/finally in fallback copy (#33)
- _isConnecting guard in WS send() (#34)

Low:
- Lazy-init deps.py singletons (#36)
- Replace __import__ with direct imports (#38)
- Preserve token count 0 in ollama.py (#39)
- Clear orphaned streamingMsg on reconnect reload (#43)
- Escape single quote in UserMessage (#44)
- Polyfill-free findLast replacement (#48)
- Match <table> tags with attributes in markdown (#49)
- Attach copy buttons only when msg.done (#50)
- Fix hasMeta falsy-0 bug (#53)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix system prompt leakage into chat history; polish content cards ...

Backend:
- websocket.py + agent.py: separate user-visible display_message from
  LLM user_message. System hints (image/file attachments) no longer leak
  into session.messages and appear after page reload.
- main.py: add ensure_tables() on startup so session_content table is
  created before first publish.
- profiles: add kimi-k2.6:cloud to all model lists as fallback.

Frontend:
- ContentCard.vue: remove border-radius, add scrollbar styles, fix
  metadata fallback parsing so cards survive page reload.
- content-viewers/*.html: add matching scrollbar styles.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content hosting system with inline viewers ...

Backend:
- Add navi/content/ directory for published files
- Add content_store.py with publish/list/delete/cleanup functions
- Add content_publish tool for publishing files as viewable content
- Add /content static file mount in main.py
- Add /content-viewers mount for viewer pages
- Extend ToolEvent with metadata field
- Forward metadata through websocket tool_call events
- Update Agent to include metadata in ToolEvent

Frontend:
- Add ContentCard.vue component for displaying published content
- Add viewer pages: stl.html (Three.js), svg.html, html.html, pdf.html
- Update AssistantMessage.vue to render ContentCard for content_publish
- Update chat store to preserve metadata in tool cards
- Update websocket protocol docs with metadata field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stop image_view hallucinations on inline-attached images ...

The model was inventing fake paths/URLs (e.g. files.oaiusercontent.com,
/home/ubuntu/navi-1/input_file_0.png) and calling image_view on them
when the user attached an image directly in chat — the image was
already in the multimodal context, but the tool description and lack
of a signal pushed the model to "load" it anyway.

- websocket.py: when a user message has inline images, append a brief
  note that they are already in context.
- image_view.py: soften the description — keep proactive use for paths
  and URLs the model genuinely cannot see, but tell it inline images
  don't need this tool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix websocket.py: unpack 4-tuple from get_registries(), pass cp_registry to Agent ...

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

WebSocket event replay buffer for disconnect resilience ...

On reconnect to an active agent run the server now replays all events
emitted since the turn started, then switches to live forwarding.
This eliminates the gap where tool cards, thinking blocks and stream
deltas were permanently lost after a network blip.

Server (_AgentRun):
- events: list[dict] buffers every serialised agent event
- broadcast() serialises and appends before putting in subscriber queues
- reconnect flow: subscribe → replay_count snapshot → stream_start →
  replay events[0:replay_count] → live _stream_to_client

Client:
- onStreamStart() removes the frozen ghost message instead of marking
  done=true, so replay cleanly rebuilds the message from scratch
- replayMode flag suppresses animations during replay
- onReplayStart/onReplayEnd handlers set/clear the flag and restore
  animate on the message once live events resume

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Route subagent planning events into spawn_agent card in the UI ...

Previously PlanningStatus/PlanReady had no is_subagent flag, so subagent
planning spinners and plan cards rendered as top-level Navi planning UI.

Backend:
- Add is_subagent field to PlanningStatus and PlanReady events
- _run_planning accepts is_subagent param, passes it through all yields
- run_ephemeral calls _run_planning with is_subagent=True
- websocket.py forwards is_subagent in planning_status and plan_ready messages

Frontend (chat.js):
- onPlanningStatus: if is_subagent, set planningLabel on the last spawn_agent
  card instead of msg.statusLabel
- onPlanReady: if is_subagent, push plan into spawn card steps and clear
  planningLabel; otherwise behave as before

Frontend (ToolCard.vue):
- Render subagent-planning-indicator (spinner + label) when planningLabel set
- Render plan cards inside subagent steps using the same plan-card pattern

Also includes leftover session changes: spawn_agent default 40 in description
and manual, updated manual content.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Planning phases, context compression, and tool improvements ...

Agent:
- Planning now a 3-phase async generator: Analysis → Execution plan → AIHelper critic
- Yield PlanningStatus events before each phase (UI progress labels)
- Phase 1 runs with think=True for deeper analysis
- Phase 2 includes available tool list so executor assignments are accurate
- Phase 3: independent critic pass validates and corrects TOOL: names against real tool list
- Planning converted from list return to async generator (fixes token accounting)

Backend:
- Context compression threshold: 80% → 70% to trigger earlier
- Compressor summary prompt: structured sections (goal, work state, key facts, outputs, errors)
- Terminal output capped at 5000 chars to prevent context flooding
- Web search: region=wt-wt for DDG, country=ALL for Brave, language=all for SearxNG
- Scratchpad: mandate writing a 'goal' section at start of multi-step tasks
- secretary max_iterations: 40→25, temperature: 0.7→0.5
- server_admin max_iterations: 40→20

Webclient:
- ThinkingCard strips <thought> XML tags leaked by Ollama
- planning_status WS event wired to chat.onPlanningStatus()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix WS disconnect and missed stream on reconnect ...

Two related problems:
- During long AIHelper calls (non-streaming LLM), no data flows to the
  WebSocket and browsers drop the connection after ~30-60s of inactivity.
  Fixed with a 20s heartbeat: _stream_to_client now uses asyncio.wait_for
  and sends {"type":"heartbeat"} on timeout to keep the connection alive.

- After reconnect, if the agent finished while the client was offline,
  _runs no longer holds the session and no stream_start is sent. Client
  would reconnect silently with no response shown. Fixed by sending
  {"type":"session_sync"} on every new WS connection (after reattach
  completes or immediately when no run is active) so the client knows
  to reload session history.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add response metrics: elapsed time, tool calls, token count ...

Server:
- Message model: elapsed_seconds, tool_call_count, token_count fields
  (display-only, excluded from LLM context via exclude_none)
- StreamEnd event: carries same three fields
- agent.run_stream: tracks turn start time, counts ToolEvent completions,
  writes metrics onto the final assistant Message before saving to DB
- WebSocket: forwards metrics in stream_end payload

Client:
- chat.onStreamEnd: attaches elapsed_seconds, tool_call_count, token_count
  to the streaming message on completion
- buildMessageList: scans each assistant group for metrics from history
- AssistantMessage: renders .msg-meta-row below the response —
  timer icon + Xs · wrench icon + N tools · coins icon + Nk tokens · time
  (each item only shown if present; time pushed right via margin-left: auto)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>