reload: pick up the new code, and drop MCP tools that are gone ...

reload_tools reported success while three separate things kept it from
doing what it says.

The bytecode cache is keyed on (mtime in whole seconds, file size), so a
tool edited to the same length inside the same second as its previous
load re-ran the OLD code — the reload was real, the new version was not
live. The loader now compiles the source itself instead of consulting
__pycache__ (importlib.invalidate_caches() does not help here).

MCP registrations only ever grew: register_mcp_tools called
register_external, and unregister_external was used in one place, so a
server removed from the config or a tool a server stopped exposing
stayed in the registry and failed only when the model called it. Reload
now clears external tools and rebuilds them.

enabled.json naming a tool that failed to load (the gmail/html2text case)
was visible only as a log line. It is now part of the report.

The reload itself moves to navi/core/reload.py, one implementation shared
by the tool and the admin route, so the two cannot leave different
toolsets behind. list_tools.py also read enabled.json through its own
cwd-relative path, which made it disagree with the real toolset, and the
class-based loader rejected execute(self, params, ctx=None) — the shape
every built-in uses.

switch_profile: run the switch alone, then the batch on the new tools ...

A tool called in the same batch as switch_profile was still dispatched against
the old tool_map and died with "tool 'X' not found" — reload_tools did, twice
in one session. The batch is now split: the switch runs first, its
ProfileSwitched event is watched for the target profile, the tools are
re-resolved from it, and the remaining calls run against the new set.

The turn's own profile binding is deliberately left alone — the
end-of-iteration reload in run_stream() is what rebinds profile/llm/schemas,
and pre-setting session.profile_id here would make it skip that step.

switch_profile: deliver profile_switched, report the new toolset ...

The tool took its sink as `ctx.event_sink if ctx else current_event_sink.get()`,
but the agent loop builds tool_ctx with event_sink=None — the ContextVar is the
real channel — so the branch always picked None, the event was silently
dropped, and the profile badge in the header never moved. plan.py already had
the fall-through (`if ctx and ctx.event_sink else current_event_sink.get()`);
this is the same fix.

The result also said nothing about what the switch changed, so the model kept
calling tools the target profile does not have (reload_tools after leaving
tool_developer). It now names the gained and lost tools, and the timing claim
is corrected: the new prompt and tools are in force from the next step of the
same turn, not "from the next message".

Allocate message sequence numbers in the DB, not in memory ...

save() numbered new messages from Session.db_next_sequence, a counter read
when the session was loaded. Any second writer of the same session handed
out the numbers it still believed were free, and the two inserts collided on
UNIQUE(session_id, sequence_number) — in production, when switch_profile
loaded the session mid-run and saved it back. The turn died with
"Internal error: duplicate key value violates unique constraint".

The range is now claimed inside save()'s transaction with a single
UPDATE ... RETURNING, so concurrent writers serialize on the session row,
and GREATEST() seeds the pre-counter sessions whose next_sequence is still 0
instead of relying on a racy max()+1 fallback in memory.

switch_profile itself no longer saves a session at all: it repoints the row
through a narrow set_profile() UPDATE, which keeps it out of the running
turn's way. It runs mid-turn on a session the turn still holds, so saving a
second copy from there was the collision in the first place.

mcp keys W2: per-user client clones, KeyResolver wiring, user_id in call path

synapse reactions W3: reaction runner, notify tool, gateway hookup ...

- navi/synapse/reactions.py: fire-and-forget reaction run — per-user
  reactions_enabled gate, one-shot dispatcher meta-pass (visible profiles
  only, hidden dispatcher, secretary fallback), special=True session
  headed by the resolved profile, headless agent run via the orchestrator
  run registry, finalise: app push per completion_notify +
  low-level reaction.finished/failed event to Synapse
- navi/synapse/outbound.py: source-side emit (syn_* key from env,
  silent when not configured)
- navi/tools/notify.py: notify tool — message + level (info/warning/
  intervention), both legs per push_target, honest delivered/skipped result
- push/service.py: notify_custom without cooldown
- gateway: verified + non-duplicate delivery schedules the reaction
- profiles: notify added to native tools (7 profiles)

synapse reactions W2: hidden dispatcher profile + synapse_instructions tool

synapse reactions W1: special sessions flag + per-user reaction settings and instruction history

webclient + navi: session title click-to-rename; empty rename regenerates via LLM

navi: session naming folds agent's final replies into context

navi: session delete covers pending (not-yet-persisted) sessions ...

Lazy persistence keeps never-messaged sessions out of the DB, so the bare
DELETE FROM sessions reported 0 rows for a very real session and the API
answered 404 'Session not found' — the webclient's list cleanup and the
welcome-screen redirect silently never ran.

compressor: single-summary invariant — old summaries always fold into the fresh one ...

partition_messages now strips is_summary messages out of the turn structure
before grouping: a summary is already-compressed history, not a user request.
Previously its role=user made it a turn head, so midturn compression kept the
old summary verbatim beside the fresh one (head kept + new summary added),
and adaptive swap resurrected summary turns whose text pattern-matches the
importance heuristics. This piled stale summary copies into the context
(one session accumulated 6 copies = 120k of 239k chars) and made compression
no-op for whole stretches before a lucky pass finally shrank it.

Also folds summaries on the early-exit path so two stale summaries
consolidate via the meta-summary, and fixes the demotion loop in
compress_and_save_session to cover session.context too: after a reload,
context-only rows (is_display=False summaries) were absent from
session.messages and never demoted, so is_context=True persisted in the DB
and the copies survived into the loaded context.

background tasks: review fixes B1-B11 batch ...

- stop mid-batch cancels in-flight tools (B1) and keeps real results
  of already-finished ones, mixing them with synthetic stopped notes
  in call order (B2)
- queued messages: headless drain publishes session_sync (B4), the
  run's teardown broadcasts session_sync to other sockets but not the
  owner socket (prevents double reload) (B5)
- task_update notifications are chained per task so late running
  updates can't overtake the terminal one (B7; client drops late
  re-flicker of a terminal task chip) (B8)
- client: ui_component results reference the owning message via
  card.parentMsg instead of a stale msg reference (B6)
- tasks cancel reports the real outcome after a bounded wait instead
  of an optimistic 'cancelled' (B9)
- session delete cancels its running background jobs and drops
  pending result notes (B10)
- subagent tool loop routes background:true calls through
  ToolExecutor._maybe_background like the main loop (B11)
- tests for all of the above; docs/tasks.md stop/cancel semantics

local mode isolation: anonymous user is a NULL-owner user, not an alias ...

Local mode (NAVI_AUTH_ENABLED=false) used an anonymous admin whose admin
role widened every session listing to ALL users' chats — acceptable only
while assuming a private database, but a real leak on any shared one.

- session listings (list_all/list_page/count_all/search_list): new
  scoping rule — non-admin with user_id=None sees only user_id IS NULL
  rows; named owner unchanged; admin unchanged
- session create/list routes: owner id is None in local mode (sessions
  are persisted ownerless), admin listing flag resolved only when auth
  is enabled
- check_session_access: local mode allows only NULL-owner sessions —
  a user-owned session by id is 403
- debug/admin endpoints (require_admin) stay reachable in local mode
- prod DB: 30 stray user_id='anonymous' rows reassigned to NULL owner

Tests: store scoping unit tests, local-mode integration tests (sidebar
filtering + access by id), updated check_session_access unit tests.
Full pytest 1239 passed, 1 skipped.

chat history pagination: paged load instead of full-session fetch ...

Backend:
- PgSessionStore.get_meta — light session head (sessions row + COUNT),
  no message payload
- PgSessionStore.get_history_page — page of display history (archive ∪
  hot UNION), newest-first page, oldest-first result, limit+1 peek for
  has_more; each message stamped metadata.display_index (ROW_NUMBER-1
  over the full display history) so client ids stay stable across
  paged loads; dangling-tool-call repair keeps real ranks by
  sequence-number lookup
- GET /sessions/{id}/meta and GET /sessions/{id}/messages/history
  (before_seq cursor, limit ≤200)

Webclient:
- loadSession/reloadSession fetch meta + newest 200-item page
  (Promise.all), archive state from the page itself
- buildMessageList ids (h_*) and rawIndices use the global
  display_index — feedback keys and search-jump stay stable when
  history is paged
- scroll-up loads older pages through the same history endpoint
  (archive table alone misses sessions whose threshold never moved)
- search-jump pulls pages until the target index is covered (bounded)

Tests: store unit tests (ranks, has_more peek, placeholder shift) +
vitest updates; full pytest 1231 passed, vitest 94 passed.

context: task-note messages survive build()'s system filter ...

E2E caught a real delivery bug: the completion note was drained and
persisted (agent.task_notes_drained logged) but build() filtered out all
system-role history from session.context, so the LLM never saw it — the
agent could only get detached results via tasks check. Notes now carry
metadata source=task_note (kept by build) and is_display=False (clients
already show live task_update events). Verified live: agent reads the
note verbatim without calling any tool.

e2e fixes: tasks tool in profiles, bg timeout lift, stepIcon fix, queue semantics docs ...

E2E findings addressed:
- tasks tool was registered but not in any profile's tools.agent.native —
  added to all six profiles (agent could not check/wait/cancel bg tasks)
- detached terminal/code_exec/ssh_exec runs without explicit timeout are
  lifted to 300s: foreground defaults (20/30/60s) marked long commands
  'completed' with partial output while the process still ran
- ToolCard.vue: define stepIcon(status) — template referenced it but the
  function was missing (render crash on task_update step)
- message_queued reachability documented: WS read loop is sequential, the
  queue path is only reachable from a second socket/headless recall

container: runtime-import PushService — master could not boot since the PWA commit ...

PushService was only imported under TYPE_CHECKING; create_container raised
NameError at startup. Caught by the first real boot (e2e) — the running
production container predates the PWA commit, so master was silently
un-launchable.

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

PWA: installable webclient, offline shell, web push ...

Installability:
- public/manifest.webmanifest (standalone, theme #16161E) + PNG icons
  generated from logo.svg (regular + maskable, served via /images mount)
- index.html: manifest link, theme-color, apple-touch-icon

Offline shell:
- hand-rolled sw.js (no workbox): navigation = network-first (3s race)
  with cached-shell fallback + background refresh (a stale cached shell
  would 404 on entry chunks after a deploy); /assets/* cache-first
  (content-hashed); /images/* cache-first capped; /api,/ws,/auth,/push,
  /content pass-through
- vite closeBundle plugin stamps __NAVI_BUILD_VERSION__ (digest of
  index.html + asset names) into dist/sw.js; sw.js served no-store so
  every deploy reactivates the SW and activation evicts old caches
- SW registration in main.js, PROD only (dev HMR untouched)
- OfflineBanner (useOnline composable) over the app shell

Web push (VAPID, pywebpush):
- navi/push/ package: push_subscriptions table (postgres, boot-time DDL),
  PushSubscriptionStore, PushService (async fan-out, to_thread sends,
  404/410 prunes dead endpoints, per-session cooldown)
- routes: GET /push/vapid-key, POST/DELETE /push/subscribe (auth-gated)
- trigger in orchestrator run_agent + run_recall: push on StreamEnd when
  no WebSocket client watches the session; fire-and-forget, never
  disturbs the run; anonymous fallback only when auth is off
- client: usePush composable + Notifications settings panel (enable/
  disable via PushManager.subscribe with the server VAPID key)
- notification click focuses the app at /#<session_id> (hash routing
  opens the right chat); payload body is a markdown-stripped <=140-char
  preview

NAVIVAPID keys empty = push fully disabled (graceful, like other optional
integrations). dist/ artifacts committed per repo convention.

Tests: pytest push store/service/routes/trigger (+23), vitest usePush
(83 webclient tests green). Full suite 1143 passed.

planning: fix LLM-output failure modes that killed every plan ...

Production planning_logs showed three ways a plan silently dies:

1. glm-5.3-flash wraps the whole answer in a structured envelope
   (response:unknown{value:...}<tool_call|>) — the planner only knew the
   gemma4 thought<channel|> artifact, so the wrapped analysis/plan failed
   to parse. The stripper now unwraps response:<type>{value:...} envelopes,
   including a truncated (unbalanced) one.
2. Empty content with spent completion tokens: glm on ollama-cloud puts
   the whole answer in the thinking channel on some calls. Both planning
   phases now fall back to thinking when content is empty; the debug log
   records which channel served (source: content|thinking).
3. phase3_timeout at the shared 120s LLM_COMPLETE_TIMEOUT: planning now
   has its own PLANNING_LLM_TIMEOUT_SEC (default 240).

Unit tests cover the envelope unwrap (balanced/truncated/inner braces),
the thinking fallback in both phases, and the clean-fail path.

swarm stage 2: peer-to-peer channel — /peer endpoints + peer tool ...

Server side (navi/api/routes/peer.py, port 8099):
- GET /peer/hello — open discovery ping (name, uuid prefix, version)
- GET /peer/status — PSK; identity, uptime, machine facts, hive view
- POST /peer/ask — PSK; one-shot agent run under PEER_ASK_PROFILE
  (default server_admin) answers, one concurrent ask at a time
- loop guard: answering agent runs without the peer tool (deterministic
  recursion cut) + own-uuid asks refused with 409
- verify_swarm_key in navi/swarm.py accepts .swarm-key.previous
  (rotation window, constant-time)

Client side (navi/tools/peer.py):
- peer tool: list (hive book, stale cache fallback marked), status,
  ask by swarm name; asks go direct peer-to-peer, hive never on the
  message path
- audit events on both sides (peer.ask_sent/received/answered/failed)

peer tool added to server_admin, navi_code, developer profiles.

swarm stage 1: instance identity, PSK, hive registry, announce loop ...

- navi/identity.py: adjective-animal names + uuid in instance.json
  (generated at install, renameable by hand)
- navi/swarm.py: HiveAnnouncer - periodic POST /announce to the hive
  with non-blocking reachability tracking (transitional logs, /health
  export, hive_status context provider reports outages to the agent)
- hive/: standalone FastAPI address book (SQLite, port 8087, PSK via
  X-Swarm-Key with .swarm-key.previous rotation window, TTL online
  status, host from client IP, port from payload). Not installed or
  started by default - run manually on the main server.
- ports moved to 8099 (API) / 8098 (UI MCP)
- install.sh: PYTHONIOENCODING=utf-8 in the systemd unit, generates
  instance.json and .swarm-key on fresh installs

compress: route every trigger path through compress_and_save_session ...

- CompressionWorker keeps the real-token gate (ctx.context_tokens) but now
  delegates to compress_and_save_session(reason="postturn") — it used to
  call compress_context directly with its own save logic and silently
  no-op'ed on any summarizer failure, leaving the session over the
  threshold until the next turn's gates
- pre-turn and /compact gates use real_baseline_estimate instead of the
  chars//3 heuristic (mid-turn already did), so code-heavy contexts
  compress in time instead of tripping ContextTooLargeError
- summarizer input keeps head + tail instead of head-only truncation,
  which silently dropped the newest summarized messages closest to the
  keep window
- critical tool results over the 4000-char budget keep head+tail halves
  instead of collapsing to the 800-char non-critical preview (a 3999-char
  read survived whole, a 4001-char one lost 98%)
- images count 500 tokens each, not 500 per message carrying them
- _plan_compression is the single partition decision shared by
  compress_context and would_compress, so the dry-run prediction cannot
  drift from the real path; the trigger reason is now logged
  (preturn/midturn/postturn/forced)

planning: replace the mandatory pre-turn gate with an agent-invoked plan tool ...

- Phase 2 (critique) retired; observe/MODE and the top-level DIRECT shortcut
  removed (sub-agents keep DIRECT for trivial subtasks)
- replan merged into plan: `reason` (+ updated_goal) packs the re-plan
  context, without a reason it is fresh planning
- both phases call the LLM with think=False — cloud reasoning models leak
  their chain-of-thought into structured output on non-streaming calls —
  and strip the gemma "thought<channel|>" content artifact
- Phase 1 conversation windowed to ~20k chars, newest-first with the
  original task pinned (28.7k-token prompts came back as 1-token output)
- confirmation by COMPLEXITY: the tool result tells the agent to present
  complex plans and wait; sub-agents keep the execute-now injection
- PlanningStatus/PlanReady reach the UI mid-turn via current_event_sink
  (ctx.event_sink is None in the agent loop — events silently never
  reached the WS before)
- agent gate, casual-message detector and adaptive re-plan nudges removed;
  anti_stall keeps only stall detection

Merge branch 'master' into feature/navi-code ...

# Conflicts:
#	navi/api/websocket.py
#	navi/config.py
#	navi/core/agent.py
#	navi/core/orchestrator.py
#	navi/main.py

security: critical batch 1 — RCE/XSS/CORS/webhook hardening ...

Backend:
- auth/deps: fix refresh-lock clock race (cleanup now monotonic like the cache)
- core/registry: add missing structlog logger (NameError on fallback path)
- config: GNAUTH_WEBHOOK_SECRET, NAVI_ALLOWED_ORIGINS (+list property)
- webhooks: HMAC-SHA256 signature verification (503 unconfigured in auth mode,
  unsigned+warning in no-auth mode, 403 bad/stale signature, 400 bad JSON)
- main: CORS from NAVI_ALLOWED_ORIGINS in auth mode (fail-fast on empty),
  eval router behind require_admin, /debug only registered in no-auth mode
- auth routes: mobile-done sid validation (32 hex) + CSP header + safe JS
  escaping (reflected XSS); Secure cookie flag via helper (https base URL)
- websocket: anonymous WS rejected in auth mode (closes legacy-session RCE);
  socket registered only after access checks; stop_session auth gate
- messages: REST agent start now takes session lock + busy flag (409 on
  active run), contextvars reset in finally

Webclient:
- useMarkdown: DOMPurify.sanitize on all rendered markdown (stored XSS),
  image-URL scheme whitelist, delegated error listener (no inline onerror)
- html.html artifact viewer: sandbox without allow-same-origin (opaque origin)
- tests: DOMPurify runs under jsdom (happy-dom Node.prototype.nodeName getter
  breaks DOMPurify); useWebSocket tests get localStorage stub + ui-kit alias

Tests: pytest 1049 passed, 1 skipped; vitest 61 passed

compress: raise budget 6000 + richer prompt detail + input headroom ...

Increase the compression summary budget and push the prompt toward more
detail and completeness, so the agent keeps more durable facts when old turns
are folded into a summary.

Budget (output):
- context_summary_max_tokens 4000 → 6000 (global default).
- navi_code compression_max_tokens 4000 → 6000 (profile override).

Prompt (base, all profiles):
- New 'Intermediate Findings' section: durable facts from read/grep/log/terminal
  that informed decisions (was forbidden by 'do not preserve intermediate
  reasoning'). Existing sections get richer instructions (Active Files: key
  symbol affected; Decisions: rationale + trigger; Completed: step + file +
  verification outcome; Errors: snippet + what was tried + outcome).
- Loosen 'tight bullet points' → 'specific and complete; prefer listing over
  generalizing — losing a fact costs more than a longer summary'.
- 'Do not write implementation code' → 'preserve short critical code verbatim
  (final signatures, one-line fixes, key config lines); drop long blocks'.

Input headroom (so the summarizer can SEE what it must preserve):
- Non-critical tool result preview 300 → 800 chars.
- _MAX_SUMMARY_INPUT_CHARS 24000 → 32000 (fits 65k–128k windows with 6k output).

Profile prompt (navi_code/compression_prompt.txt): preserve short diagnostic
snippets verbatim (exact error line, failing assertion, one-line fix) instead
of blanket 'do not preserve long output'; keep code signatures + short key
snippets, not just names.

Tests: Intermediate Findings section in base prompt; 'short critical code'
phrasing; non-critical preview keeps 800 chars; meta-summary threshold raised
(2×6000 chars). Fixed flaky test_attach_session_seeds_context_fill (relied on
_startup worker timing after Etap 4 lengthened attach; now explicit attach
like its sibling + mock get_todos to skip a 30s×2 real-network timeout).
Full suite: 1008 passed, 1 skipped.

terminal: session-broadcast sink + lifecycle events (Etap 1) ...

Persistent terminals emitted their stream only into a per-tool-call sink, which
closed once the open action returned — so a background dev server's output
stopped reaching the client immediately after open. TerminalClosed was never
emitted at all (the handler existed, the emitter did not).

Replace the per-tool event_sink with a single callback the orchestrator wires to
the session's WebSocket(s):

- TerminalManager: set_event_callback(cb) + _emit(session_id, event). Reader
  tasks and open/close deliver TerminalOutputDelta / TerminalOpened /
  TerminalClosed through the callback, which lives for the terminal's (and
  session's) lifetime — not the tool call's.
- New TerminalOpened event (terminal_opened: name/description/pid/background).
  TerminalClosed now actually emitted from _close_one (explicit/idle/shutdown/
  session_ended). Background output keeps streaming after open returns.
- terminal.py: drop the per-tool event_sink from _tm.open (no longer needed).
- orchestrator._on_terminal_event → _notify_session (per-session WS fan-out,
  same path recall/mcp updates use). container wires the callback after building
  the orchestrator.
- Events imported lazily inside terminal_manager methods to avoid a circular
  import (navi.core.events → navi.core.__init__ → agent → registry → navi.tools).

Tests: open emits TerminalOpened; background output streams via the callback
after open returns (gap-3 fix); close emits TerminalClosed; no callback does not
break readers. Existing terminal tests unchanged. Full suite: 968 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md (5 etaps, this is Etap 1).

mcp: dedicated runner task per client (fix cross-task cancel-scope on shutdown) ...

MCP client SDK transports (stdio/sse/streamable_http) + ClientSession are
anyio task groups whose cancel scopes require __aenter__/__aexit__ in the
SAME asyncio task. McpClient entered the transport in one task (lifespan
connect / health-check reconnect / request retry) and exited it in another
(lifespan teardown) -> RuntimeError: Attempted to exit cancel scope in a
different task than it was entered in.

Refactor McpClient to own a single long-running runner task that holds the
AsyncExitStack and performs ALL transport enter/exit + list_tools/call_tool.
The public async API (connect/disconnect/list_tools/call_tool/mark_disconnected)
just enqueues a _Cmd and awaits a Future, so callers from any task no longer
cross cancel-scope boundaries. connected/instructions mirror from the runner
onto the instance to stay sync-readable. disconnect() enqueues a stop command
and awaits shield(runner) so teardown isn't interrupted by lifespan cancel.

Also call mcp_manager.stop_health_check() BEFORE disconnect_all() in
AppContainer.shutdown() so the health-check task cannot enqueue onto a
client whose runner is being torn down. mark_disconnected() is now async
(queued) and its manager caller updated.

Regression test: connect in one task, list_tools in a second, disconnect in
a third — the exact scenario that raised the RuntimeError before.