reload: pick up the new code, and drop MCP tools that are gone ...

reload_tools reported success while three separate things kept it from
doing what it says.

The bytecode cache is keyed on (mtime in whole seconds, file size), so a
tool edited to the same length inside the same second as its previous
load re-ran the OLD code — the reload was real, the new version was not
live. The loader now compiles the source itself instead of consulting
__pycache__ (importlib.invalidate_caches() does not help here).

MCP registrations only ever grew: register_mcp_tools called
register_external, and unregister_external was used in one place, so a
server removed from the config or a tool a server stopped exposing
stayed in the registry and failed only when the model called it. Reload
now clears external tools and rebuilds them.

enabled.json naming a tool that failed to load (the gmail/html2text case)
was visible only as a log line. It is now part of the report.

The reload itself moves to navi/core/reload.py, one implementation shared
by the tool and the admin route, so the two cannot leave different
toolsets behind. list_tools.py also read enabled.json through its own
cwd-relative path, which made it disagree with the real toolset, and the
class-based loader rejected execute(self, params, ctx=None) — the shape
every built-in uses.

switch_profile: deliver profile_switched, report the new toolset ...

The tool took its sink as `ctx.event_sink if ctx else current_event_sink.get()`,
but the agent loop builds tool_ctx with event_sink=None — the ContextVar is the
real channel — so the branch always picked None, the event was silently
dropped, and the profile badge in the header never moved. plan.py already had
the fall-through (`if ctx and ctx.event_sink else current_event_sink.get()`);
this is the same fix.

The result also said nothing about what the switch changed, so the model kept
calling tools the target profile does not have (reload_tools after leaving
tool_developer). It now names the gained and lost tools, and the timing claim
is corrected: the new prompt and tools are in force from the next step of the
same turn, not "from the next message".

Allocate message sequence numbers in the DB, not in memory ...

save() numbered new messages from Session.db_next_sequence, a counter read
when the session was loaded. Any second writer of the same session handed
out the numbers it still believed were free, and the two inserts collided on
UNIQUE(session_id, sequence_number) — in production, when switch_profile
loaded the session mid-run and saved it back. The turn died with
"Internal error: duplicate key value violates unique constraint".

The range is now claimed inside save()'s transaction with a single
UPDATE ... RETURNING, so concurrent writers serialize on the session row,
and GREATEST() seeds the pre-counter sessions whose next_sequence is still 0
instead of relying on a racy max()+1 fallback in memory.

switch_profile itself no longer saves a session at all: it repoints the row
through a narrow set_profile() UPDATE, which keeps it out of the running
turn's way. It runs mid-turn on a session the turn still holds, so saving a
second copy from there was the collision in the first place.

Await the session pool in notify and three neighbours ...

PgSessionStore._get_pool() is async, and four call sites passed its coroutine
straight into a store constructor, so the first query inside died with
"'coroutine' object has no attribute 'fetchrow'": notify always failed, the
reaction runner never got past reading its settings, synapse_instructions was
unusable, and the BYOK resolver caught the AttributeError and silently fell
back to the default credential.

The tests missed it because each one mocked the store or the pool provider
away; the new ones run on a fake asyncpg pool with nothing faked below the tool.

mcp keys: tool-management tools aligned with BYOK (test_mcp_tool runs on user key, mcp_status marks key slots)

synapse reactions W3: reaction runner, notify tool, gateway hookup ...

- navi/synapse/reactions.py: fire-and-forget reaction run — per-user
  reactions_enabled gate, one-shot dispatcher meta-pass (visible profiles
  only, hidden dispatcher, secretary fallback), special=True session
  headed by the resolved profile, headless agent run via the orchestrator
  run registry, finalise: app push per completion_notify +
  low-level reaction.finished/failed event to Synapse
- navi/synapse/outbound.py: source-side emit (syn_* key from env,
  silent when not configured)
- navi/tools/notify.py: notify tool — message + level (info/warning/
  intervention), both legs per push_target, honest delivered/skipped result
- push/service.py: notify_custom without cooldown
- gateway: verified + non-duplicate delivery schedules the reaction
- profiles: notify added to native tools (7 profiles)

synapse reactions W2: hidden dispatcher profile + synapse_instructions tool

background tasks: review fixes B1-B11 batch ...

- stop mid-batch cancels in-flight tools (B1) and keeps real results
  of already-finished ones, mixing them with synthetic stopped notes
  in call order (B2)
- queued messages: headless drain publishes session_sync (B4), the
  run's teardown broadcasts session_sync to other sockets but not the
  owner socket (prevents double reload) (B5)
- task_update notifications are chained per task so late running
  updates can't overtake the terminal one (B7; client drops late
  re-flicker of a terminal task chip) (B8)
- client: ui_component results reference the owning message via
  card.parentMsg instead of a stale msg reference (B6)
- tasks cancel reports the real outcome after a bounded wait instead
  of an optimistic 'cancelled' (B9)
- session delete cancels its running background jobs and drops
  pending result notes (B10)
- subagent tool loop routes background:true calls through
  ToolExecutor._maybe_background like the main loop (B11)
- tests for all of the above; docs/tasks.md stop/cancel semantics

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

swarm names: single female name (user preference; collisions fixed by hand)

swarm names: two female names (Japanese/American/Spanish) instead of adjective-animal ...

Single names would collide across a swarm (peers are addressed by name),
a mixed pair like 'yuki-grace' or 'sofia-rin' gives 150x150 combinations.
Only affects NEW installs - existing instance.json names stay as they are.

swarm stage 2: peer-to-peer channel — /peer endpoints + peer tool ...

Server side (navi/api/routes/peer.py, port 8099):
- GET /peer/hello — open discovery ping (name, uuid prefix, version)
- GET /peer/status — PSK; identity, uptime, machine facts, hive view
- POST /peer/ask — PSK; one-shot agent run under PEER_ASK_PROFILE
  (default server_admin) answers, one concurrent ask at a time
- loop guard: answering agent runs without the peer tool (deterministic
  recursion cut) + own-uuid asks refused with 409
- verify_swarm_key in navi/swarm.py accepts .swarm-key.previous
  (rotation window, constant-time)

Client side (navi/tools/peer.py):
- peer tool: list (hive book, stale cache fallback marked), status,
  ask by swarm name; asks go direct peer-to-peer, hive never on the
  message path
- audit events on both sides (peer.ask_sent/received/answered/failed)

peer tool added to server_admin, navi_code, developer profiles.

planning: replace the mandatory pre-turn gate with an agent-invoked plan tool ...

- Phase 2 (critique) retired; observe/MODE and the top-level DIRECT shortcut
  removed (sub-agents keep DIRECT for trivial subtasks)
- replan merged into plan: `reason` (+ updated_goal) packs the re-plan
  context, without a reason it is fresh planning
- both phases call the LLM with think=False — cloud reasoning models leak
  their chain-of-thought into structured output on non-streaming calls —
  and strip the gemma "thought<channel|>" content artifact
- Phase 1 conversation windowed to ~20k chars, newest-first with the
  original task pinned (28.7k-token prompts came back as 1-token output)
- confirmation by COMPLEXITY: the tool result tells the agent to present
  complex plans and wait; sub-agents keep the execute-now injection
- PlanningStatus/PlanReady reach the UI mid-turn via current_event_sink
  (ctx.event_sink is None in the agent loop — events silently never
  reached the WS before)
- agent gate, casual-message detector and adaptive re-plan nudges removed;
  anti_stall keeps only stall detection

navi_code: stop delegating code work to the developer profile ...

navi_code was systematically spawning sub-agents as `developer` instead of
itself, even though it already covers local code work. Root cause was a
three-channel nudge: the navi_code prompt recommended `developer` for "general
code work" and framed omit (-> navi_code) as the exception; spawn_agent's
description listed `developer` as the coding example and never mentioned
navi_code; the injected Available-profiles list shows developer's broad
"General-purpose software development" blurb. The planner locks profile_id at
plan time on those same signals, so the bias is baked in before execution.

Flip the default in all three places: for code work OMIT profile_id so the
sub-agent runs as the current profile (navi_code); set profile_id only for a
different specialisation (secretary/server_admin/tool_developer). developer is
not removed as a profile — it is just no longer recommended from navi_code.

code_exec: agent-controlled timeout (30s default, 300s max) + language metadata ...

The 30s timeout was a hard constant; long computations or test suites hit it
with no escape hatch. Expose it as a `timeout` param (clamped 1-300s, default
30) like terminal's. Also stamp `language: "python"` into metadata so the
TUI renderer can highlight without hard-coding, and record `timeout` on the
timeout result for a dedicated timeout status card.

todo: add op — append steps discovered mid-task (preserve statuses) ...

The todo tool only had set/view/update/clear. Adding a step that surfaced
mid-task meant either 'set' (which replaces the plan and resets every status
to pending) or 'replan' (1–3 LLM calls) — both heavy for a single new step, so
the agent usually skipped recording it. Add a cheap 'add' op:

- navi/tools/todo.py: op 'add' (tasks: list[str]) appends new _Task entries to
  the end of the existing plan in pending status; existing steps and their
  statuses are untouched. Requires an existing plan (points to 'set'
  otherwise). Schema enum + op description + tool description updated.
- system_prompt.txt: tell the agent to record new subtasks with 'todo add'
  right away (not held in head, not a replan); clarify the 'small todo edit'
  line — add via 'todo add', drop/merge/reorder via 'set' (re-apply statuses).
- renderers/todo.py: TodoStartedRenderer renders the add card
  ('→ todo · add (N)') listing the new pending steps, not a JSON dump.

Tests: add appends + preserves statuses / requires existing plan / requires
tasks; renderer add card + empty-tasks. Full suite: 1005 passed, 1 skipped.

terminal: session-broadcast sink + lifecycle events (Etap 1) ...

Persistent terminals emitted their stream only into a per-tool-call sink, which
closed once the open action returned — so a background dev server's output
stopped reaching the client immediately after open. TerminalClosed was never
emitted at all (the handler existed, the emitter did not).

Replace the per-tool event_sink with a single callback the orchestrator wires to
the session's WebSocket(s):

- TerminalManager: set_event_callback(cb) + _emit(session_id, event). Reader
  tasks and open/close deliver TerminalOutputDelta / TerminalOpened /
  TerminalClosed through the callback, which lives for the terminal's (and
  session's) lifetime — not the tool call's.
- New TerminalOpened event (terminal_opened: name/description/pid/background).
  TerminalClosed now actually emitted from _close_one (explicit/idle/shutdown/
  session_ended). Background output keeps streaming after open returns.
- terminal.py: drop the per-tool event_sink from _tm.open (no longer needed).
- orchestrator._on_terminal_event → _notify_session (per-session WS fan-out,
  same path recall/mcp updates use). container wires the callback after building
  the orchestrator.
- Events imported lazily inside terminal_manager methods to avoid a circular
  import (navi.core.events → navi.core.__init__ → agent → registry → navi.tools).

Tests: open emits TerminalOpened; background output streams via the callback
after open returns (gap-3 fix); close emits TerminalClosed; no callback does not
break readers. Existing terminal tests unchanged. Full suite: 968 passed, 1 skipped.

Plan: docs/terminal_tool_plan.md (5 etaps, this is Etap 1).

filesystem: strip trailing newline in _number_diff so diff lines aren't blank-separated ...

_unified_diff (used by edit_lines / smart_edit) feeds difflib with
"line + \n" so difflib does not emit a "\ No newline at end of file" marker.
With lineterm="" difflib keeps that trailing "\n" on content lines, so the
final "\n".join produced "\n\n" between every diff line — a blank line
between each, bloating the model-facing output and rendering with double
spacing in both clients. Strip the trailing "\n" in _number_diff before
processing; no-op for the _diff action, whose lines come from splitlines()
without trailing newlines.

Автономность мелких моделей: OUTPUT DISCIPLINE, milestone-todo, перехват финала хода ...

Два последовательных захода над одной проблемой — мелкие модели (12–30B) в navi_code
теряют автономность на трудных шагах: «остановился поболтать» вместо действия и
слишком большое расстояние между пунктами плана.

Заход 1 (З1–З3):
- З1: OUTPUT DISCIPLINE в системном промпте navi_code — «act, don't announce»,
  без few-shot антипримеров. Контракт хода: ответ без tool_calls = конец хода, поэтому
  объявление намерения текстом убивает автономность; правила заставляют вызывать
  инструмент в том же ходе.
- З2: плоский todo + метка группы milestone + декомпозиция. _parse_plan_steps
  возвращает list[tuple[milestone, text]]; milestone — метка группировки (не сущность,
  без статуса), «done» вычисляется при рендеринге; подшаги = больше плоских шагов
  (без вложенности). TUI side-panel группирует по milestone (плоский фолбэк при пустом
  milestone). Plan depth: max 15→20 + правило декомпозиции.
- З3: adaptive re-plan «длинный шаг» — nudge «разбей шаг» при in_progress ≥ порога
  итераций без смены todo (порог 4, раньше общего anti-stall warning на 8).

Заход 2 (шаги 1–3, после cloud-теста 31b vs 12b):
Корневая структурная причина: весь спасательный механизм (anti-stall warning с явным
предложением reflect, adaptive re-plan) живёт только внутри tool-цикла — nudge
инжектируется в pre_turn *следующей* итерации, которой при «остановился поболтать»
нет (ход закрылся по return до post_turn). 31b застревала через «продолжаю
tool-итерации» → дожала до warning → спаслась; 12b — через «замолчала текстом» → мимо
всех nudge.

- Шаг 1: перехват финала хода. Если модель выдала bare-text, но в todo есть открытые
  шаги (pending/in_progress) и лимит не исчерпан — НЕ эмитить StreamEnd, а сохранить
  ассистентский текст в session.context, поставить системный nudge и continue
  (без StreamEnd, без workers — консистентно с multi-iteration tool-турами). Счётчик
  final_interceptions на AgentTurnContext, лимит final_intercept_limit (default 2),
  эскалация жёсткости (мягкий → «second stop»). has_open_steps в todo.py: пустой
  todo → False (защита casual-сообщений), failed/skipped терминальны. Профильные
  флаги final_intercept_enabled/limit в base.py + loader + admin.
- Шаг 2: жёсткий reflect-триггер в промпте — «~3 tool attempts on the same step
  without progress → call reflect IN THIS TURN (tool call, not reasoning aloud)».
- Шаг 3: открыть replan для застревания — «call replan when reflect showed the whole
  approach is dead (not one failed step, but the approach itself)».

Тесты: 874 passed, 1 skipped. Новые — has_open_steps (5), final intercept (5),
milestone-группировка, adaptive long-step nudge, парсер шагов с milestone-маркером.

cloud: navi_code model → gemma4:31b-cloud для тестирования догадки (31b признала
застревание, 12b — нет); .env cloud-host уже gitignored.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem: show real file line numbers in unified diffs ...

_number_diff parses @@ hunk headers and prefixes each content line with its
real file line number: removed/context lines use the old (from) number,
added lines use the new (to) number, in `{marker} {num}│ {content}` form
(marker first so existing startswith highlighting stays valid; column
right-aligned to the largest line number). Hunk/file headers and the
`\ No newline` marker are left unchanged.

Applied in _unified_diff (covers edit / edit_lines / smart_edit) and the
`diff` action — a single server-side change, so both the TUI and the web
client receive numbered diffs in the tool result text.

TUI highlight_unified_diff renders the `{marker} {num}│` column dim and the
content in the marker color, so the number reads as meta, not as part of the
added/removed text. Lines without the prefix (standalone `diff` event, older
callers) are still colored whole.

Co-Authored-By: Claude <noreply@anthropic.com>

replan: integrated mid-task re-planning tool ...

Add a replan tool that re-runs the planner over the live session context
+ todo + scratchpad when the plan's structure is stale due to discoveries
(NOT failed steps -- that stays [Adaptive re-plan]). Integrated approach:
PlanningEngine.run gains is_replan/replan_context (suppresses DIRECT
shortcut and observe-skip, frames Phase 1 as a revision); ReplanRunner
packs reason/goal/todo/findings/errors and captures PlanReady; the tool
is exposed to navi_code/developer/tool_developer via a
current_replan_runner ContextVar set per-iteration in run_stream (correct
after switch_profile). New plan replaces the todo. Lazy events import
breaks the navi.tools -> navi.core -> navi.tools cycle.

recall: carry self-instruction (message) on the recall_update wire ...

The recall card could show call_type/trigger_at but not the self-instruction
(additional_context_message) — it was absent from RecallUpdate.to_wire, so
the user couldn't see what future-self was about to do at the scheduled or
fired moment. Extend the wire payload.

- events.RecallUpdate: add `message` field; to_wire emits "message".
- scheduler._publish_recall_update: accept and forward `message`.
- Publish sites carry message=recall.additional_context_message:
  schedule_recall (scheduled) and orchestrator._finalize_recall
  (rescheduled / fired / cancelled). manage_recall cancel/skip omit it
  (no recall object handy; already visible in the prior scheduled card).
- TUI RecallRenderer: preview the message (first line + "(+N lines)",
  capped at 80) on a `msg:` body line when present.
- Tests: RecallUpdate.to_wire carries message (defaults None); renderer
  preview (single/multiline/truncate/empty) and scheduled-card renders it.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem edit: render red/green diff in TUI without changing model context ...

edit is now the primary editing method, yet its output was a dry one-line
status ("Edited …: replaced X B with Y B") while edit_lines/smart_edit showed a
highlighted unified diff. Carry the unified diff in ToolResult.metadata["diff"]
(kept out of the model-facing output, so the agent's context is unchanged) and
render it in the TUI: dim summary line plus highlighted diff (green +, red -,
dim @@). Falls back to plain text when no diff metadata is present.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem: make edit/edit_lines the default, smart_edit a last-resort fallback ...

Agent overused smart_edit (whole-file LLM call, weak context). Rewrite the
FilesystemTool description decision tree so edit (exact text) is the default and
edit_lines (by line numbers) is the deterministic option; smart_edit is reserved
for genuinely semantic changes that cannot be expressed as exact text or line
numbers. Strengthen the old_not_unique hint to steer back to edit/edit_lines
before smart_edit. Add an "Editing policy" section to the navi_code and
developer profile prompts reinforcing the same priority.

Co-Authored-By: Claude <noreply@anthropic.com>

tui: show the changed step's text in the todo update call card ...

The todo update call card (→ todo · #3 → done) now shows the text of the
step being changed, plus the validation when present. The LLM's update args
carry only the index, not the task text, so the text is read from the
current plan row before the tool runs and attached to ToolStarted.metadata
(client-rendering only, backward-compatible).

Backend:
- events.ToolStarted gains a metadata dict (mirrors ToolEvent) → to_wire.
- navi/tools/todo: step_text_for_update(index, ctx) resolves the step text
  via _sid/_uid (so sub-agent isolation holds), started_metadata_for_call
  wraps it for both emit sites.
- agent.py (parent) and subagent_runner.py (sub-agent) enrich ToolStarted
  via the shared helper before emitting.

Renderer:
- TodoStartedRenderer update card reads msg.metadata.step_text → shows the
  step text + validation; falls back to 'no validation' on history replay.
- Removed the step-text line from the result card (it now lives in the call
  card) — result card is back to the compact 'plan · X/N done' summary.

Co-Authored-By: Claude <noreply@anthropic.com>

Isolate sub-agent todo + render live todo in TUI side panel ...

Phase 1 — sub-agent todo isolation (backend):
- Add current_todo_session_id ContextVar; subagent_runner scopes it to the
  sub-agent's ephemeral run id so its auto-populated plan and todo updates
  land in an isolated KV row instead of clobbering the parent session's todo
  (which the parent's goal-anchoring reads every iteration).
- todo._sid() and planning.set_tasks prefer current_todo_session_id; the
  parent run leaves it unset, so all existing todo consumers (anti-stall,
  goal anchor, get_progress_message) behave exactly as before.

Phase 2 — live todo in the TUI right column:
- TodoUpdated event + emit it from the agent loop after planning auto-populate
  and after each tool-execution turn.
- GET /sessions/{id}/todos reads the parent session's todo KV row (explicit
  user_id/session_id, optional injected kv).
- api.get_todos + TodoList/TodoPanel widgets: status-coloured markers
  (pending dim, in_progress accent+bold, done success, failed error, skipped
  dim), progress header, scrollable panel below the auto-height info block.
- Hybrid delivery: REST seeds the panel on attach/switch, todo_updated WS
  events update it live.

Sub-agent todos as nested sub-lists is deferred to a later phase.

Co-Authored-By: Claude <noreply@anthropic.com>

Navi Code: stop via Esc + project cwd propagation ...

- TUI: Esc stops active stream cooperatively via POST /sessions/{id}/stop
- TUI: render stream_stopped as status message
- CLI/WebSocket: send shell cwd in client->server message field
- Orchestrator stores cwd in session.session_metadata
- ContextBuilder injects [Working directory] into LLM context
- Agent sets current_working_directory ContextVar per turn
- tools/base: ToolContext gains cwd field
- filesystem/terminal/code_exec resolve relative paths against session cwd
- Add bin/navi-code wrapper for PATH symlink; document in README.md
- Update docs/websocket.md and tests

Full pytest: 544 passed, 1 skipped.

Co-Authored-By: Claude <noreply@anthropic.com>

Fix spawn_agent profile selection: stronger prompt + alias fallback ...

- Rewrite PROFILE SELECTION description to explicitly say: when the
  plan specifies a profile, you MUST pass profile_id. Only omit when
  the plan does NOT specify a profile.
- Add JSON examples showing correct profile_id usage.
- Add fallback for 'profile' parameter name (models sometimes use
  'profile' instead of 'profile_id' when the description says
  'Profile to use').\n\nCo-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Fix scratchpad tool: rename op→action, add examples, improve error messages ...

- Rename primary parameter from 'op' to 'action' for consistency with
  all other tools (terminal, filesystem, todo, etc.). Legacy 'op' still
  works as a fallback to avoid breaking old calls in compressed context.
- Add JSON examples directly in the tool description so the model sees
  the exact structure to produce.
- Improve all error messages to include the correct JSON example,
  making it obvious what the model did wrong.
- Add manuals/scratchpad.md for tool_manual support.
- Update and expand tests for new syntax and error cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Clarify terminal description param and profile prompts ...

- terminal.py: stronger description for the 'description' field so
  the model knows it is REQUIRED for action=open and what to write
- server_admin and developer system prompts: add a short section
  explaining persistent terminal usage (open/close/list/status/send_input)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>