context: task-note messages survive build()'s system filter ...

E2E caught a real delivery bug: the completion note was drained and
persisted (agent.task_notes_drained logged) but build() filtered out all
system-role history from session.context, so the LLM never saw it — the
agent could only get detached results via tasks check. Notes now carry
metadata source=task_note (kept by build) and is_display=False (clients
already show live task_update events). Verified live: agent reads the
note verbatim without calling any tool.

e2e fixes: tasks tool in profiles, bg timeout lift, stepIcon fix, queue semantics docs ...

E2E findings addressed:
- tasks tool was registered but not in any profile's tools.agent.native —
  added to all six profiles (agent could not check/wait/cancel bg tasks)
- detached terminal/code_exec/ssh_exec runs without explicit timeout are
  lifted to 300s: foreground defaults (20/30/60s) marked long commands
  'completed' with partial output while the process still ran
- ToolCard.vue: define stepIcon(status) — template referenced it but the
  function was missing (render crash on task_update step)
- message_queued reachability documented: WS read loop is sequential, the
  queue path is only reachable from a second socket/headless recall

container: runtime-import PushService — master could not boot since the PWA commit ...

PushService was only imported under TYPE_CHECKING; create_container raised
NameError at startup. Caught by the first real boot (e2e) — the running
production container predates the PWA commit, so master was silently
un-launchable.

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

PWA: installable webclient, offline shell, web push ...

Installability:
- public/manifest.webmanifest (standalone, theme #16161E) + PNG icons
  generated from logo.svg (regular + maskable, served via /images mount)
- index.html: manifest link, theme-color, apple-touch-icon

Offline shell:
- hand-rolled sw.js (no workbox): navigation = network-first (3s race)
  with cached-shell fallback + background refresh (a stale cached shell
  would 404 on entry chunks after a deploy); /assets/* cache-first
  (content-hashed); /images/* cache-first capped; /api,/ws,/auth,/push,
  /content pass-through
- vite closeBundle plugin stamps __NAVI_BUILD_VERSION__ (digest of
  index.html + asset names) into dist/sw.js; sw.js served no-store so
  every deploy reactivates the SW and activation evicts old caches
- SW registration in main.js, PROD only (dev HMR untouched)
- OfflineBanner (useOnline composable) over the app shell

Web push (VAPID, pywebpush):
- navi/push/ package: push_subscriptions table (postgres, boot-time DDL),
  PushSubscriptionStore, PushService (async fan-out, to_thread sends,
  404/410 prunes dead endpoints, per-session cooldown)
- routes: GET /push/vapid-key, POST/DELETE /push/subscribe (auth-gated)
- trigger in orchestrator run_agent + run_recall: push on StreamEnd when
  no WebSocket client watches the session; fire-and-forget, never
  disturbs the run; anonymous fallback only when auth is off
- client: usePush composable + Notifications settings panel (enable/
  disable via PushManager.subscribe with the server VAPID key)
- notification click focuses the app at /#<session_id> (hash routing
  opens the right chat); payload body is a markdown-stripped <=140-char
  preview

NAVIVAPID keys empty = push fully disabled (graceful, like other optional
integrations). dist/ artifacts committed per repo convention.

Tests: pytest push store/service/routes/trigger (+23), vitest usePush
(83 webclient tests green). Full suite 1143 passed.

planning: fix LLM-output failure modes that killed every plan ...

Production planning_logs showed three ways a plan silently dies:

1. glm-5.3-flash wraps the whole answer in a structured envelope
   (response:unknown{value:...}<tool_call|>) — the planner only knew the
   gemma4 thought<channel|> artifact, so the wrapped analysis/plan failed
   to parse. The stripper now unwraps response:<type>{value:...} envelopes,
   including a truncated (unbalanced) one.
2. Empty content with spent completion tokens: glm on ollama-cloud puts
   the whole answer in the thinking channel on some calls. Both planning
   phases now fall back to thinking when content is empty; the debug log
   records which channel served (source: content|thinking).
3. phase3_timeout at the shared 120s LLM_COMPLETE_TIMEOUT: planning now
   has its own PLANNING_LLM_TIMEOUT_SEC (default 240).

Unit tests cover the envelope unwrap (balanced/truncated/inner braces),
the thinking fallback in both phases, and the clean-fail path.

swarm names: single female name (user preference; collisions fixed by hand)

swarm names: two female names (Japanese/American/Spanish) instead of adjective-animal ...

Single names would collide across a swarm (peers are addressed by name),
a mixed pair like 'yuki-grace' or 'sofia-rin' gives 150x150 combinations.
Only affects NEW installs - existing instance.json names stay as they are.

swarm stage 2: peer-to-peer channel — /peer endpoints + peer tool ...

Server side (navi/api/routes/peer.py, port 8099):
- GET /peer/hello — open discovery ping (name, uuid prefix, version)
- GET /peer/status — PSK; identity, uptime, machine facts, hive view
- POST /peer/ask — PSK; one-shot agent run under PEER_ASK_PROFILE
  (default server_admin) answers, one concurrent ask at a time
- loop guard: answering agent runs without the peer tool (deterministic
  recursion cut) + own-uuid asks refused with 409
- verify_swarm_key in navi/swarm.py accepts .swarm-key.previous
  (rotation window, constant-time)

Client side (navi/tools/peer.py):
- peer tool: list (hive book, stale cache fallback marked), status,
  ask by swarm name; asks go direct peer-to-peer, hive never on the
  message path
- audit events on both sides (peer.ask_sent/received/answered/failed)

peer tool added to server_admin, navi_code, developer profiles.

swarm stage 1: instance identity, PSK, hive registry, announce loop ...

- navi/identity.py: adjective-animal names + uuid in instance.json
  (generated at install, renameable by hand)
- navi/swarm.py: HiveAnnouncer - periodic POST /announce to the hive
  with non-blocking reachability tracking (transitional logs, /health
  export, hive_status context provider reports outages to the agent)
- hive/: standalone FastAPI address book (SQLite, port 8087, PSK via
  X-Swarm-Key with .swarm-key.previous rotation window, TTL online
  status, host from client IP, port from payload). Not installed or
  started by default - run manually on the main server.
- ports moved to 8099 (API) / 8098 (UI MCP)
- install.sh: PYTHONIOENCODING=utf-8 in the systemd unit, generates
  instance.json and .swarm-key on fresh installs

port: 8000/8001 -> 8099/8098 everywhere ...

navi runs on shared servers where 8000/8001 are usually taken. New
defaults: API 8099, navi_ui MCP 8098. Touched: config defaults
(navi_port, navi_ui_mcp_port, public_url, gnauth_redirect_uri),
navi-server launcher docs, env.template/.env.example, install.sh
health-check fallback, terminal client base_url, webclient dev configs
(useWebSocket, contentLinks, vite proxy), android url hint, docs.
Also made the navi_ui FastMCP constructor port settings-driven instead
of a hardcoded 8001 (it was overridden at start anyway).

fix: ASCII-safe startup log messages + force UTF-8 stdout in the systemd unit ...

A unit on an old distro without a UTF-8 locale gets latin-1/ascii
stdout; structlog's print of the em-dash in the webclient-disabled
message raised UnicodeEncodeError and killed the app at startup
(Application startup failed, crash-loop restart counter 28). The unit
now sets PYTHONIOENCODING=utf-8 so any future non-ASCII log line is
safe too.

server: NAVI_WEBCLIENT_ENABLED gate + navi-server launcher ...

NAVI_WEBCLIENT_ENABLED=false removes every web-facing route (/, /assets,
/images, /content-viewers, /content, /admin panel, /debug*) and skips
the navi_ui MCP server — its only consumer is the webclient. The REST
API and WS the terminal client uses stay fully intact, and so does the
/admin/* JSON API (still gated by require_admin). Static mounts use
check_dir=False so a tree without webclient/dist imports fine.

New navi-server console script (navi/server.py): uvicorn launcher that
reads NAVI_HOST/NAVI_PORT from settings — the systemd unit needs no
hardcoded values.

Default stays true (full web UI), so nothing changes for dev setups;
subprocess route tests cover both modes.

mcp: a dead MCP endpoint no longer kills server startup ...

A failed transport connect inside the MCP SDK's anyio cancel scopes
surfaces as CancelledError without any task.cancel(). The runner mistook
that for a real teardown, cancelled the in-flight caller's future, and
McpManager.load_all (which only catches Exception) let it blow up the
whole lifespan startup — one unreachable server in mcp_servers.d/ meant
a dead Navi.

The runner now distinguishes real task cancellation
(asyncio.current_task().cancelling()) from the anyio artifact: the
latter is surfaced to the caller as a normal connect failure, the
server pool marks it disconnected and the health-check loop retries.
Also, when the runner dies for any reason, pending queue commands are
failed instead of leaving _send callers hanging forever.

compress: route every trigger path through compress_and_save_session ...

- CompressionWorker keeps the real-token gate (ctx.context_tokens) but now
  delegates to compress_and_save_session(reason="postturn") — it used to
  call compress_context directly with its own save logic and silently
  no-op'ed on any summarizer failure, leaving the session over the
  threshold until the next turn's gates
- pre-turn and /compact gates use real_baseline_estimate instead of the
  chars//3 heuristic (mid-turn already did), so code-heavy contexts
  compress in time instead of tripping ContextTooLargeError
- summarizer input keeps head + tail instead of head-only truncation,
  which silently dropped the newest summarized messages closest to the
  keep window
- critical tool results over the 4000-char budget keep head+tail halves
  instead of collapsing to the 800-char non-critical preview (a 3999-char
  read survived whole, a 4001-char one lost 98%)
- images count 500 tokens each, not 500 per message carrying them
- _plan_compression is the single partition decision shared by
  compress_context and would_compress, so the dry-run prediction cannot
  drift from the real path; the trigger reason is now logged
  (preturn/midturn/postturn/forced)

profiles: retire gate flags and retrain prompts for the plan tool ...

- drop planning_enabled / planning_mandatory / planning_phase2_enabled /
  observe_skips_plan_enabled / adaptive_replan_enabled from AgentProfile,
  the loader and admin serialization
- add `plan` to native tools of developer, navi_code, tool_developer,
  secretary, server_admin, modeler_3d — without the gate they would
  otherwise lose planning entirely
- system prompts: call plan for non-trivial multi-step work before
  execution; complex plans get presented for confirmation; stuck or goal
  changed -> plan with a reason

planning: replace the mandatory pre-turn gate with an agent-invoked plan tool ...

- Phase 2 (critique) retired; observe/MODE and the top-level DIRECT shortcut
  removed (sub-agents keep DIRECT for trivial subtasks)
- replan merged into plan: `reason` (+ updated_goal) packs the re-plan
  context, without a reason it is fresh planning
- both phases call the LLM with think=False — cloud reasoning models leak
  their chain-of-thought into structured output on non-streaming calls —
  and strip the gemma "thought<channel|>" content artifact
- Phase 1 conversation windowed to ~20k chars, newest-first with the
  original task pinned (28.7k-token prompts came back as 1-token output)
- confirmation by COMPLEXITY: the tool result tells the agent to present
  complex plans and wait; sub-agents keep the execute-now injection
- PlanningStatus/PlanReady reach the UI mid-turn via current_event_sink
  (ctx.event_sink is None in the agent loop — events silently never
  reached the WS before)
- agent gate, casual-message detector and adaptive re-plan nudges removed;
  anti_stall keeps only stall detection

Merge branch 'master' into feature/navi-code ...

# Conflicts:
#	navi/api/websocket.py
#	navi/config.py
#	navi/core/agent.py
#	navi/core/orchestrator.py
#	navi/main.py

navi_code: add gemma4 31b/26b qat models to the profile fallback chain

security: critical batch 1 — RCE/XSS/CORS/webhook hardening ...

Backend:
- auth/deps: fix refresh-lock clock race (cleanup now monotonic like the cache)
- core/registry: add missing structlog logger (NameError on fallback path)
- config: GNAUTH_WEBHOOK_SECRET, NAVI_ALLOWED_ORIGINS (+list property)
- webhooks: HMAC-SHA256 signature verification (503 unconfigured in auth mode,
  unsigned+warning in no-auth mode, 403 bad/stale signature, 400 bad JSON)
- main: CORS from NAVI_ALLOWED_ORIGINS in auth mode (fail-fast on empty),
  eval router behind require_admin, /debug only registered in no-auth mode
- auth routes: mobile-done sid validation (32 hex) + CSP header + safe JS
  escaping (reflected XSS); Secure cookie flag via helper (https base URL)
- websocket: anonymous WS rejected in auth mode (closes legacy-session RCE);
  socket registered only after access checks; stop_session auth gate
- messages: REST agent start now takes session lock + busy flag (409 on
  active run), contextvars reset in finally

Webclient:
- useMarkdown: DOMPurify.sanitize on all rendered markdown (stored XSS),
  image-URL scheme whitelist, delegated error listener (no inline onerror)
- html.html artifact viewer: sandbox without allow-same-origin (opaque origin)
- tests: DOMPurify runs under jsdom (happy-dom Node.prototype.nodeName getter
  breaks DOMPurify); useWebSocket tests get localStorage stub + ui-kit alias

Tests: pytest 1049 passed, 1 skipped; vitest 61 passed

navi_code: add web lookup, ssh, and image_view — grow beyond local-only ...

navi_code was strictly local: no web, no remote hosts, no image viewing. That
made it lean on the developer profile whenever a coding task needed a doc
lookup or a remote command. Bring those capabilities in so navi_code covers
full coding work itself, while staying terminal-first and local-by-default.

config: agent gains image_view, ssh_exec, and the navi-web MCP (search/browse/
request); subagent gains image_view + navi-web (no ssh_exec — sub-agents don't
reach remote hosts, matching developer). Description/short_description/
full_description updated to "terminal-first with web and remote access". Also
includes the gemma4:12b-it-qat-128k model in the model list (local 128k-context
option alongside 31b-cloud).

prompt: soften "Local-only, no remote hosts" to "local by default, with web
lookup (docs/APIs) and ssh (remote ops) when the task needs it". Add compact
Web lookup and Remote access sections under Execution environment, and update
the sub-agent toolset briefing to list image_view + web and to exclude ssh_exec.
scope_boundary still applies; ssh destructive/system-wide actions need user
confirmation. Kept additions minimal to avoid bloating the prompt for the 12b
model.

navi_code: stop delegating code work to the developer profile ...

navi_code was systematically spawning sub-agents as `developer` instead of
itself, even though it already covers local code work. Root cause was a
three-channel nudge: the navi_code prompt recommended `developer` for "general
code work" and framed omit (-> navi_code) as the exception; spawn_agent's
description listed `developer` as the coding example and never mentioned
navi_code; the injected Available-profiles list shows developer's broad
"General-purpose software development" blurb. The planner locks profile_id at
plan time on those same signals, so the bias is baked in before execution.

Flip the default in all three places: for code work OMIT profile_id so the
sub-agent runs as the current profile (navi_code); set profile_id only for a
different specialisation (secretary/server_admin/tool_developer). developer is
not removed as a profile — it is just no longer recommended from navi_code.

code_exec: agent-controlled timeout (30s default, 300s max) + language metadata ...

The 30s timeout was a hard constant; long computations or test suites hit it
with no escape hatch. Expose it as a `timeout` param (clamped 1-300s, default
30) like terminal's. Also stamp `language: "python"` into metadata so the
TUI renderer can highlight without hard-coding, and record `timeout` on the
timeout result for a dedicated timeout status card.

compress: raise budget 6000 + richer prompt detail + input headroom ...

Increase the compression summary budget and push the prompt toward more
detail and completeness, so the agent keeps more durable facts when old turns
are folded into a summary.

Budget (output):
- context_summary_max_tokens 4000 → 6000 (global default).
- navi_code compression_max_tokens 4000 → 6000 (profile override).

Prompt (base, all profiles):
- New 'Intermediate Findings' section: durable facts from read/grep/log/terminal
  that informed decisions (was forbidden by 'do not preserve intermediate
  reasoning'). Existing sections get richer instructions (Active Files: key
  symbol affected; Decisions: rationale + trigger; Completed: step + file +
  verification outcome; Errors: snippet + what was tried + outcome).
- Loosen 'tight bullet points' → 'specific and complete; prefer listing over
  generalizing — losing a fact costs more than a longer summary'.
- 'Do not write implementation code' → 'preserve short critical code verbatim
  (final signatures, one-line fixes, key config lines); drop long blocks'.

Input headroom (so the summarizer can SEE what it must preserve):
- Non-critical tool result preview 300 → 800 chars.
- _MAX_SUMMARY_INPUT_CHARS 24000 → 32000 (fits 65k–128k windows with 6k output).

Profile prompt (navi_code/compression_prompt.txt): preserve short diagnostic
snippets verbatim (exact error line, failing assertion, one-line fix) instead
of blanket 'do not preserve long output'; keep code signatures + short key
snippets, not just names.

Tests: Intermediate Findings section in base prompt; 'short critical code'
phrasing; non-critical preview keeps 800 chars; meta-summary threshold raised
(2×6000 chars). Fixed flaky test_attach_session_seeds_context_fill (relied on
_startup worker timing after Etap 4 lengthened attach; now explicit attach
like its sibling + mock get_todos to skip a 30s×2 real-network timeout).
Full suite: 1008 passed, 1 skipped.

todo: add op — append steps discovered mid-task (preserve statuses) ...

The todo tool only had set/view/update/clear. Adding a step that surfaced
mid-task meant either 'set' (which replaces the plan and resets every status
to pending) or 'replan' (1–3 LLM calls) — both heavy for a single new step, so
the agent usually skipped recording it. Add a cheap 'add' op:

- navi/tools/todo.py: op 'add' (tasks: list[str]) appends new _Task entries to
  the end of the existing plan in pending status; existing steps and their
  statuses are untouched. Requires an existing plan (points to 'set'
  otherwise). Schema enum + op description + tool description updated.
- system_prompt.txt: tell the agent to record new subtasks with 'todo add'
  right away (not held in head, not a replan); clarify the 'small todo edit'
  line — add via 'todo add', drop/merge/reorder via 'set' (re-apply statuses).
- renderers/todo.py: TodoStartedRenderer renders the add card
  ('→ todo · add (N)') listing the new pending steps, not a JSON dump.

Tests: add appends + preserves statuses / requires existing plan / requires
tasks; renderer add card + empty-tasks. Full suite: 1005 passed, 1 skipped.

terminal: REST list/close endpoints + async client helpers (Etap 2) ...

The /terminals modal and the status-bar count need a server source for the
session's persistent terminals (background dev servers, etc.) and a way to
force-close one. Add REST endpoints over the existing TerminalManager:

- GET /sessions/{id}/terminals → terminal_manager.list(session_id) →
  {session_id, terminals: [summary,...]}. Access-checked (navi.sessions.read_all).
- POST /sessions/{id}/terminals/{name}/close → terminal_manager.close(...)
  → {session_id, terminal_name, closed: bool}. Name is URL-encoded by the
  client (terminal names may contain spaces).
- clients/terminal/api.py (async): list_terminals(session_id),
  close_terminal(session_id, name) — for the TUI modal + status-bar seed.

Tests: list empty / list returns active / close routes to manager / 404 on
unknown session (fake TerminalManager injected into the container, mirroring the
kv_store injection pattern). Full suite: 972 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md — Etap 2 of 5.

terminal: session-broadcast sink + lifecycle events (Etap 1) ...

Persistent terminals emitted their stream only into a per-tool-call sink, which
closed once the open action returned — so a background dev server's output
stopped reaching the client immediately after open. TerminalClosed was never
emitted at all (the handler existed, the emitter did not).

Replace the per-tool event_sink with a single callback the orchestrator wires to
the session's WebSocket(s):

- TerminalManager: set_event_callback(cb) + _emit(session_id, event). Reader
  tasks and open/close deliver TerminalOutputDelta / TerminalOpened /
  TerminalClosed through the callback, which lives for the terminal's (and
  session's) lifetime — not the tool call's.
- New TerminalOpened event (terminal_opened: name/description/pid/background).
  TerminalClosed now actually emitted from _close_one (explicit/idle/shutdown/
  session_ended). Background output keeps streaming after open returns.
- terminal.py: drop the per-tool event_sink from _tm.open (no longer needed).
- orchestrator._on_terminal_event → _notify_session (per-session WS fan-out,
  same path recall/mcp updates use). container wires the callback after building
  the orchestrator.
- Events imported lazily inside terminal_manager methods to avoid a circular
  import (navi.core.events → navi.core.__init__ → agent → registry → navi.tools).

Tests: open emits TerminalOpened; background output streams via the callback
after open returns (gap-3 fix); close emits TerminalClosed; no callback does not
break readers. Existing terminal tests unchanged. Full suite: 968 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md (5 etaps, this is Etap 1).

mcp: dedicated runner task per client (fix cross-task cancel-scope on shutdown) ...

MCP client SDK transports (stdio/sse/streamable_http) + ClientSession are
anyio task groups whose cancel scopes require __aenter__/__aexit__ in the
SAME asyncio task. McpClient entered the transport in one task (lifespan
connect / health-check reconnect / request retry) and exited it in another
(lifespan teardown) -> RuntimeError: Attempted to exit cancel scope in a
different task than it was entered in.

Refactor McpClient to own a single long-running runner task that holds the
AsyncExitStack and performs ALL transport enter/exit + list_tools/call_tool.
The public async API (connect/disconnect/list_tools/call_tool/mark_disconnected)
just enqueues a _Cmd and awaits a Future, so callers from any task no longer
cross cancel-scope boundaries. connected/instructions mirror from the runner
onto the instance to stay sync-readable. disconnect() enqueues a stop command
and awaits shield(runner) so teardown isn't interrupted by lifespan cancel.

Also call mcp_manager.stop_health_check() BEFORE disconnect_all() in
AppContainer.shutdown() so the health-check task cannot enqueue onto a
client whose runner is being torn down. mark_disconnected() is now async
(queued) and its manager caller updated.

Regression test: connect in one task, list_tools in a second, disconnect in
a third — the exact scenario that raised the RuntimeError before.

compressor: target hysteresis — shrink to 65% after compression, not just below the trigger ...

Compression triggered at 90% but left the context just under the trigger, so a
fixed keep_recent (navi_code: 12 turns ≈ 104 messages) re-triggered a couple
of messages later — the context yo-yoed at the trigger line instead of gaining
headroom. Add context_compression_target (0.65): in turn-based (preturn) mode
compress_context shrinks keep_recent until the verbatim kept region fits 65% of
the window, folding the extra turns into the same single summary LLM call; a
safety net in compress_session token-budget-truncates if the kept region alone
still exceeds the target (midturn kept a huge in-flight turn, or the
keep_recent floor can't fit). Trigger 90% → target 65% leaves real headroom.

filesystem: strip trailing newline in _number_diff so diff lines aren't blank-separated ...

_unified_diff (used by edit_lines / smart_edit) feeds difflib with
"line + \n" so difflib does not emit a "\ No newline at end of file" marker.
With lineterm="" difflib keeps that trailing "\n" on content lines, so the
final "\n".join produced "\n\n" between every diff line — a blank line
between each, bloating the model-facing output and rendering with double
spacing in both clients. Strip the trailing "\n" in _number_diff before
processing; no-op for the _diff action, whose lines come from splitlines()
without trailing newlines.