deploy: auto-install docker compose v2 plugin when the host lacks it ...

Ubuntu 18.04's docker packages predate compose v2. The plugin is a
single static binary that runs on any distro with docker CLI >= 18.09,
so the script downloads it into /usr/local/lib/docker/cli-plugins
(system-wide — sudo docker compose sees it too) when docker compose is
unavailable, with the same offline pattern as the python tarball: a
pre-downloaded binary next to install.sh wins. If the CLI is too old
to support plugins at all, the error asks for docker --version.

deploy: standalone CPython fallback for old distros ...

venv cannot be shipped in the repo — it embeds absolute paths and
distro-specific compiled wheels (asyncpg, uvloop, cryptography), so it
dies on any other machine. Instead, when the host has no python >= 3.11
(Ubuntu 18.04: 3.6, 22.04: 3.10; system python must not be replaced),
install.sh fetches a standalone astral python-build-standalone CPython
3.12 into .python/ — self-contained, needs only glibc >= 2.17, works
on any distro. The tarball is cached next to the repo and reused; on
offline servers a pre-downloaded tarball next to install.sh wins.
Verified locally: tarball layout, venv creation, pip install.

deploy: pick the newest python >= 3.11 instead of hardcoding python3 ...

Ubuntu 22.04's system python3 is 3.10 and must not be replaced (apt
depends on it) — a deadsnakes install lives under python3.11/3.12.
The install now tries those names before plain python3 and builds
the venv with whichever qualifies, with a hint in the error path.

deploy: README opens with the full install breakdown

deploy: regenerate placeholder secrets from the deploy branch's .env on first install

deploy: one-command server bundle (dockerized postgres + systemd) ...

deploy/install.sh — everything in one command: prerequisite checks,
.env provisioning from env.template (generated DB password), dockerized
PostgreSQL (pgvector image, restart: always, bound to 127.0.0.1, vector
+ pg_trgm extensions), venv + pip install, a systemd unit with
Restart=always (the server lives from installation, per the roadmap
where these modules become each other's subagents), health check, and
navi-code/navi-server symlinks into PATH. Idempotent — safe to re-run.

Default deployed shape: web UI off, auth off, terminal client only.
Re-enabling the web panel later is editing .env + restart — nothing is
removed from the tree.

The deploy branch (next commit set) differs from master only by a
filled-in .env.

server: NAVI_WEBCLIENT_ENABLED gate + navi-server launcher ...

NAVI_WEBCLIENT_ENABLED=false removes every web-facing route (/, /assets,
/images, /content-viewers, /content, /admin panel, /debug*) and skips
the navi_ui MCP server — its only consumer is the webclient. The REST
API and WS the terminal client uses stay fully intact, and so does the
/admin/* JSON API (still gated by require_admin). Static mounts use
check_dir=False so a tree without webclient/dist imports fine.

New navi-server console script (navi/server.py): uvicorn launcher that
reads NAVI_HOST/NAVI_PORT from settings — the systemd unit needs no
hardcoded values.

Default stays true (full web UI), so nothing changes for dev setups;
subprocess route tests cover both modes.

mcp: a dead MCP endpoint no longer kills server startup ...

A failed transport connect inside the MCP SDK's anyio cancel scopes
surfaces as CancelledError without any task.cancel(). The runner mistook
that for a real teardown, cancelled the in-flight caller's future, and
McpManager.load_all (which only catches Exception) let it blow up the
whole lifespan startup — one unreachable server in mcp_servers.d/ meant
a dead Navi.

The runner now distinguishes real task cancellation
(asyncio.current_task().cancelling()) from the anyio artifact: the
latter is surfaced to the caller as a normal connect failure, the
server pool marks it disconnected and the health-check loop retries.
Also, when the runner dies for any reason, pending queue commands are
failed instead of leaving _send callers hanging forever.

docs: sync with the plan rework and compression pipeline ...

- planning: plan tool flow (agent-invoked, no pre-turn gate), confirmation
  by COMPLEXITY, think=False + 20k-char Phase 1 window, retired flags
  removed from profiles/api docs
- compression: all four triggers through compress_and_save_session,
  real-baseline gates, head+tail summary input, _plan_compression shared
  decision, actual config defaults (0.90 threshold, 6000 summary tokens,
  32k input cap)

compress: route every trigger path through compress_and_save_session ...

- CompressionWorker keeps the real-token gate (ctx.context_tokens) but now
  delegates to compress_and_save_session(reason="postturn") — it used to
  call compress_context directly with its own save logic and silently
  no-op'ed on any summarizer failure, leaving the session over the
  threshold until the next turn's gates
- pre-turn and /compact gates use real_baseline_estimate instead of the
  chars//3 heuristic (mid-turn already did), so code-heavy contexts
  compress in time instead of tripping ContextTooLargeError
- summarizer input keeps head + tail instead of head-only truncation,
  which silently dropped the newest summarized messages closest to the
  keep window
- critical tool results over the 4000-char budget keep head+tail halves
  instead of collapsing to the 800-char non-critical preview (a 3999-char
  read survived whole, a 4001-char one lost 98%)
- images count 500 tokens each, not 500 per message carrying them
- _plan_compression is the single partition decision shared by
  compress_context and would_compress, so the dry-run prediction cannot
  drift from the real path; the trigger reason is now logged
  (preturn/midturn/postturn/forced)

profiles: retire gate flags and retrain prompts for the plan tool ...

- drop planning_enabled / planning_mandatory / planning_phase2_enabled /
  observe_skips_plan_enabled / adaptive_replan_enabled from AgentProfile,
  the loader and admin serialization
- add `plan` to native tools of developer, navi_code, tool_developer,
  secretary, server_admin, modeler_3d — without the gate they would
  otherwise lose planning entirely
- system prompts: call plan for non-trivial multi-step work before
  execution; complex plans get presented for confirmation; stuck or goal
  changed -> plan with a reason

planning: replace the mandatory pre-turn gate with an agent-invoked plan tool ...

- Phase 2 (critique) retired; observe/MODE and the top-level DIRECT shortcut
  removed (sub-agents keep DIRECT for trivial subtasks)
- replan merged into plan: `reason` (+ updated_goal) packs the re-plan
  context, without a reason it is fresh planning
- both phases call the LLM with think=False — cloud reasoning models leak
  their chain-of-thought into structured output on non-streaming calls —
  and strip the gemma "thought<channel|>" content artifact
- Phase 1 conversation windowed to ~20k chars, newest-first with the
  original task pinned (28.7k-token prompts came back as 1-token output)
- confirmation by COMPLEXITY: the tool result tells the agent to present
  complex plans and wait; sub-agents keep the execute-now injection
- PlanningStatus/PlanReady reach the UI mid-turn via current_event_sink
  (ctx.event_sink is None in the agent loop — events silently never
  reached the WS before)
- agent gate, casual-message detector and adaptive re-plan nudges removed;
  anti_stall keeps only stall detection

tests: fake_run_agent accepts hidden kwarg (merged navi_ui form_submit path)

webclient: rebuild dist from merged source (navi_ui + XSS fixes); keep dist tracked

Merge branch 'master' into feature/navi-code ...

# Conflicts:
#	navi/api/websocket.py
#	navi/config.py
#	navi/core/agent.py
#	navi/core/orchestrator.py
#	navi/main.py

navi_code: add gemma4 31b/26b qat models to the profile fallback chain

security: critical batch 1 — RCE/XSS/CORS/webhook hardening ...

Backend:
- auth/deps: fix refresh-lock clock race (cleanup now monotonic like the cache)
- core/registry: add missing structlog logger (NameError on fallback path)
- config: GNAUTH_WEBHOOK_SECRET, NAVI_ALLOWED_ORIGINS (+list property)
- webhooks: HMAC-SHA256 signature verification (503 unconfigured in auth mode,
  unsigned+warning in no-auth mode, 403 bad/stale signature, 400 bad JSON)
- main: CORS from NAVI_ALLOWED_ORIGINS in auth mode (fail-fast on empty),
  eval router behind require_admin, /debug only registered in no-auth mode
- auth routes: mobile-done sid validation (32 hex) + CSP header + safe JS
  escaping (reflected XSS); Secure cookie flag via helper (https base URL)
- websocket: anonymous WS rejected in auth mode (closes legacy-session RCE);
  socket registered only after access checks; stop_session auth gate
- messages: REST agent start now takes session lock + busy flag (409 on
  active run), contextvars reset in finally

Webclient:
- useMarkdown: DOMPurify.sanitize on all rendered markdown (stored XSS),
  image-URL scheme whitelist, delegated error listener (no inline onerror)
- html.html artifact viewer: sandbox without allow-same-origin (opaque origin)
- tests: DOMPurify runs under jsdom (happy-dom Node.prototype.nodeName getter
  breaks DOMPurify); useWebSocket tests get localStorage stub + ui-kit alias

Tests: pytest 1049 passed, 1 skipped; vitest 61 passed

navi_code: add web lookup, ssh, and image_view — grow beyond local-only ...

navi_code was strictly local: no web, no remote hosts, no image viewing. That
made it lean on the developer profile whenever a coding task needed a doc
lookup or a remote command. Bring those capabilities in so navi_code covers
full coding work itself, while staying terminal-first and local-by-default.

config: agent gains image_view, ssh_exec, and the navi-web MCP (search/browse/
request); subagent gains image_view + navi-web (no ssh_exec — sub-agents don't
reach remote hosts, matching developer). Description/short_description/
full_description updated to "terminal-first with web and remote access". Also
includes the gemma4:12b-it-qat-128k model in the model list (local 128k-context
option alongside 31b-cloud).

prompt: soften "Local-only, no remote hosts" to "local by default, with web
lookup (docs/APIs) and ssh (remote ops) when the task needs it". Add compact
Web lookup and Remote access sections under Execution environment, and update
the sub-agent toolset briefing to list image_view + web and to exclude ssh_exec.
scope_boundary still applies; ssh destructive/system-wide actions need user
confirmation. Kept additions minimal to avoid bloating the prompt for the 12b
model.

navi_code: stop delegating code work to the developer profile ...

navi_code was systematically spawning sub-agents as `developer` instead of
itself, even though it already covers local code work. Root cause was a
three-channel nudge: the navi_code prompt recommended `developer` for "general
code work" and framed omit (-> navi_code) as the exception; spawn_agent's
description listed `developer` as the coding example and never mentioned
navi_code; the injected Available-profiles list shows developer's broad
"General-purpose software development" blurb. The planner locks profile_id at
plan time on those same signals, so the bias is baked in before execution.

Flip the default in all three places: for code work OMIT profile_id so the
sub-agent runs as the current profile (navi_code); set profile_id only for a
different specialisation (secretary/server_admin/tool_developer). developer is
not removed as a profile — it is just no longer recommended from navi_code.

tui: dedicated code_exec renderer — highlighted code + structured result ...

code_exec went through the generic tool renderer: the whole script was dumped
as an escaped JSON string (unreadable), and the result was a dim wall of text
with stdout/stderr fused by a [stderr] marker. Give it a first-class card:

started: Python syntax highlighting via the shared highlight_code (follows
Theme.code_theme), long scripts folded at 60 lines, working_dir/timeout shown
as compact key/values instead of JSON bulk.

result: exit code anchored in the title (exit N), stdout and stderr split into
separate blocks with stderr in the warning colour, and a dedicated "timeout Ns"
status (detected from metadata or, for legacy sessions, the output text).
Registered before the generic tool renderers (first accepting wins).

code_exec: agent-controlled timeout (30s default, 300s max) + language metadata ...

The 30s timeout was a hard constant; long computations or test suites hit it
with no escape hatch. Expose it as a `timeout` param (clamped 1-300s, default
30) like terminal's. Also stamp `language: "python"` into metadata so the
TUI renderer can highlight without hard-coding, and record `timeout` on the
timeout result for a dedicated timeout status card.

tui: lower default visible chat window 200->60 (faster long-session resume) ...

The cold cost is dominated by the one-shot mount of the whole visible
window at load/switch time. 200 _ChatItemView widgets each parse Markdown,
build Content, compute height and run layout in one pass. 60 cuts that
peak ~3.3x with no loss for streaming (signature cache + throttle + render
cache keep per-token cost O(1) once mounted). Model cap (max_visible_items
* 3) follows automatically: 600 -> 180. Older items collapse into the
existing "... N earlier messages not shown" hint.

compress: raise budget 6000 + richer prompt detail + input headroom ...

Increase the compression summary budget and push the prompt toward more
detail and completeness, so the agent keeps more durable facts when old turns
are folded into a summary.

Budget (output):
- context_summary_max_tokens 4000 → 6000 (global default).
- navi_code compression_max_tokens 4000 → 6000 (profile override).

Prompt (base, all profiles):
- New 'Intermediate Findings' section: durable facts from read/grep/log/terminal
  that informed decisions (was forbidden by 'do not preserve intermediate
  reasoning'). Existing sections get richer instructions (Active Files: key
  symbol affected; Decisions: rationale + trigger; Completed: step + file +
  verification outcome; Errors: snippet + what was tried + outcome).
- Loosen 'tight bullet points' → 'specific and complete; prefer listing over
  generalizing — losing a fact costs more than a longer summary'.
- 'Do not write implementation code' → 'preserve short critical code verbatim
  (final signatures, one-line fixes, key config lines); drop long blocks'.

Input headroom (so the summarizer can SEE what it must preserve):
- Non-critical tool result preview 300 → 800 chars.
- _MAX_SUMMARY_INPUT_CHARS 24000 → 32000 (fits 65k–128k windows with 6k output).

Profile prompt (navi_code/compression_prompt.txt): preserve short diagnostic
snippets verbatim (exact error line, failing assertion, one-line fix) instead
of blanket 'do not preserve long output'; keep code signatures + short key
snippets, not just names.

Tests: Intermediate Findings section in base prompt; 'short critical code'
phrasing; non-critical preview keeps 800 chars; meta-summary threshold raised
(2×6000 chars). Fixed flaky test_attach_session_seeds_context_fill (relied on
_startup worker timing after Etap 4 lengthened attach; now explicit attach
like its sibling + mock get_todos to skip a 30s×2 real-network timeout).
Full suite: 1008 passed, 1 skipped.

todo: add op — append steps discovered mid-task (preserve statuses) ...

The todo tool only had set/view/update/clear. Adding a step that surfaced
mid-task meant either 'set' (which replaces the plan and resets every status
to pending) or 'replan' (1–3 LLM calls) — both heavy for a single new step, so
the agent usually skipped recording it. Add a cheap 'add' op:

- navi/tools/todo.py: op 'add' (tasks: list[str]) appends new _Task entries to
  the end of the existing plan in pending status; existing steps and their
  statuses are untouched. Requires an existing plan (points to 'set'
  otherwise). Schema enum + op description + tool description updated.
- system_prompt.txt: tell the agent to record new subtasks with 'todo add'
  right away (not held in head, not a replan); clarify the 'small todo edit'
  line — add via 'todo add', drop/merge/reorder via 'set' (re-apply statuses).
- renderers/todo.py: TodoStartedRenderer renders the add card
  ('→ todo · add (N)') listing the new pending steps, not a JSON dump.

Tests: add appends + preserves statuses / requires existing plan / requires
tasks; renderer add card + empty-tasks. Full suite: 1005 passed, 1 skipped.

tui: terminals picker — ▶ / ■ status icons (terminal-native) ...

Replace the 🟢/⚪ coloured circles with ▶ (running) / ■ (stopped) — a
play/stop metaphor that fits a terminal process better than emoji circles.

Full suite: 1000 passed, 1 skipped.

tui: terminals picker — clearer two-line rows + Delete to close ...

- Each terminal row is now two lines: the name (bold, with status icon) on
  top, the description + pid + uptime as a dim meta line beneath, separated
  by '·' — easier to scan than a single run-on line.
- Close is bound to Delete/Backspace instead of Enter: Enter commonly means
  "open/select", Delete means "remove/close", which matches closing a
  terminal. Title updated: "Open terminals — Delete to close, Esc to cancel".
  No-op on an empty list.

Tests: close-on-delete (was close-on-enter), empty-list delete is a no-op.
Full suite: 1000 passed, 1 skipped.

tui: terminal tool_started renderer (action-aware card) ...

terminal tool_started cards used to fall through to the generic
ToolStartedRenderer (a raw JSON dump of the args — a wall of quoted strings for
a command or input). Add an action-aware started renderer, mirroring the result
renderer and the filesystem started renderer:

- Title: → terminal <action>.
- run: the command as a shell prompt in accent (the headline) + optional cwd.
- open: terminal name (accent) + description + background + command preview.
- close/status: terminal name. send_input: name + input preview.
- list: empty body (no per-call headline; the result card carries the table).

Registered before ToolStartedRenderer (first accepting wins).

Tests: started accepts gating + run/open/close/send_input/list cards;
render_plain covers the borderless reading-mode path. Full suite: 1000 passed,
1 skipped.

tui: /terminals command + modal to view and force-close terminals (Etap 5) ...

The final piece of the terminals plan: a modal to list the session's open
persistent terminals and force-close one.

- screens/terminals_picker.py: a modal (mirror of sessions_picker). Seeds the
  list from api.list_terminals on open, kept live by refresh_from_model — the
  app calls it as terminal_opened/closed events arrive while the modal is open
  (closed ones drop out, newly opened append). Each row shows status emoji +
  name (bold) + description + pid + uptime. Up/Down navigate, Enter force-closes
  the highlighted terminal via api.close_terminal (in a worker), Escape cancels.
  Empty list is a no-op for Enter (no crash, no close call).
- commands/builtin.py: TerminalsCommand (/terminals) opens the picker for the
  active session. Registered in the command registry.
- tui_app._open_terminals_picker pushes the screen; on_ws_event's terminal
  branch now also refreshes the picker live if it is the active screen.

Tests: list seed, Enter closes the highlighted terminal (api.close_terminal
called, drops from list), Escape cancels (dismiss None), live refresh filters
closed + adds opened, empty list Enter is a no-op. Input-box hint tests
adjusted for the new /t match order (terminals, thinking, themes). Full suite:
993 passed, 1 skipped.

Plan docs/terminal_tool_plan.md — Etap 5 of 5 (complete).

tui: handle terminal lifecycle events + status-bar count (Etap 4) ...

terminal_opened/output/closed WS events used to fall through to ChatModel's
unknown-event path (raw dict dumped into chat). Now they feed a dedicated
terminals state and a status-bar count, with no chat bubble:

- ChatModel: terminals dict (name → {description, pid, background, status,
  output_tail, closed}) updated by terminal_opened/output/closed in
  handle_ws_event (return None — no chat item). open_terminal_count property;
  seed_terminals(summaries) populates from a REST list (on attach). output_tail
  capped at 200 lines.
- tui_app.on_ws_event: a terminal-events branch forwards to the model (no _sync
  walk — background output can be frequent) and updates the StatusPanel count.
  attach_session seeds via api.list_terminals and sets the count (graceful 0 on
  failure).
- StatusPanel: the "Ctrl+P palette | /help commands" hint line is replaced with
  a live "Terminals: N" count (set_terminals_count). Key combos still live in
  the bottom StatusBar.
- Tests: mock_tui_api fixtures across 5 client test files gain an async
  fake_list_terminals so attach_session never hits a real network in tests.

Tests: ChatModel opened tracks state (no bubble) / output appends tail / unknown
ignored / closed decrements count / seed from REST; tui_app terminal events
update the status count with no chat bubble. Full suite: 988 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md — Etap 4 of 5.

tui: terminal tool-call renderer (action-aware) (Etap 3) ...

terminal tool_call results used to fall through to the generic
ToolResultRenderer (flat Text, no awareness of the action). Add an
action-aware renderer registered before the generic one, mirroring filesystem:

- run: command echoed as a shell prompt + dim output + an exit-code anchor
  (green on 0, red otherwise). A failed run still shows what ran and the exit
  code, not just the raw error.
- open: terminal name (accent) + description + background flag + pid.
- list / status: dim, truncated (200-line cap like the generic tool renderer).
- send_input / close: dim echo. open failure surfaces the reason (already
  exists / max reached). Other action failures show the message in red.

Registered before ToolResultRenderer in default_registry (first accepting
renderer wins). ChatPanel._item_msg already spreads the WS payload into meta,
so the renderer reads args/result/success/metadata directly.

Tests: accepts gating, run success/failure with exit code, open name/desc/pid,
list/status/send_input/close, run truncation; render_plain covers the
borderless reading-mode path. Full suite: 982 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md — Etap 3 of 5.