filesystem: strip trailing newline in _number_diff so diff lines aren't blank-separated ...

_unified_diff (used by edit_lines / smart_edit) feeds difflib with
"line + \n" so difflib does not emit a "\ No newline at end of file" marker.
With lineterm="" difflib keeps that trailing "\n" on content lines, so the
final "\n".join produced "\n\n" between every diff line — a blank line
between each, bloating the model-facing output and rendering with double
spacing in both clients. Strip the trailing "\n" in _number_diff before
processing; no-op for the _diff action, whose lines come from splitlines()
without trailing newlines.

Автономность мелких моделей: OUTPUT DISCIPLINE, milestone-todo, перехват финала хода ...

Два последовательных захода над одной проблемой — мелкие модели (12–30B) в navi_code
теряют автономность на трудных шагах: «остановился поболтать» вместо действия и
слишком большое расстояние между пунктами плана.

Заход 1 (З1–З3):
- З1: OUTPUT DISCIPLINE в системном промпте navi_code — «act, don't announce»,
  без few-shot антипримеров. Контракт хода: ответ без tool_calls = конец хода, поэтому
  объявление намерения текстом убивает автономность; правила заставляют вызывать
  инструмент в том же ходе.
- З2: плоский todo + метка группы milestone + декомпозиция. _parse_plan_steps
  возвращает list[tuple[milestone, text]]; milestone — метка группировки (не сущность,
  без статуса), «done» вычисляется при рендеринге; подшаги = больше плоских шагов
  (без вложенности). TUI side-panel группирует по milestone (плоский фолбэк при пустом
  milestone). Plan depth: max 15→20 + правило декомпозиции.
- З3: adaptive re-plan «длинный шаг» — nudge «разбей шаг» при in_progress ≥ порога
  итераций без смены todo (порог 4, раньше общего anti-stall warning на 8).

Заход 2 (шаги 1–3, после cloud-теста 31b vs 12b):
Корневая структурная причина: весь спасательный механизм (anti-stall warning с явным
предложением reflect, adaptive re-plan) живёт только внутри tool-цикла — nudge
инжектируется в pre_turn *следующей* итерации, которой при «остановился поболтать»
нет (ход закрылся по return до post_turn). 31b застревала через «продолжаю
tool-итерации» → дожала до warning → спаслась; 12b — через «замолчала текстом» → мимо
всех nudge.

- Шаг 1: перехват финала хода. Если модель выдала bare-text, но в todo есть открытые
  шаги (pending/in_progress) и лимит не исчерпан — НЕ эмитить StreamEnd, а сохранить
  ассистентский текст в session.context, поставить системный nudge и continue
  (без StreamEnd, без workers — консистентно с multi-iteration tool-турами). Счётчик
  final_interceptions на AgentTurnContext, лимит final_intercept_limit (default 2),
  эскалация жёсткости (мягкий → «second stop»). has_open_steps в todo.py: пустой
  todo → False (защита casual-сообщений), failed/skipped терминальны. Профильные
  флаги final_intercept_enabled/limit в base.py + loader + admin.
- Шаг 2: жёсткий reflect-триггер в промпте — «~3 tool attempts on the same step
  without progress → call reflect IN THIS TURN (tool call, not reasoning aloud)».
- Шаг 3: открыть replan для застревания — «call replan when reflect showed the whole
  approach is dead (not one failed step, but the approach itself)».

Тесты: 874 passed, 1 skipped. Новые — has_open_steps (5), final intercept (5),
milestone-группировка, adaptive long-step nudge, парсер шагов с milestone-маркером.

cloud: navi_code model → gemma4:31b-cloud для тестирования догадки (31b признала
застревание, 12b — нет); .env cloud-host уже gitignored.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem: show real file line numbers in unified diffs ...

_number_diff parses @@ hunk headers and prefixes each content line with its
real file line number: removed/context lines use the old (from) number,
added lines use the new (to) number, in `{marker} {num}│ {content}` form
(marker first so existing startswith highlighting stays valid; column
right-aligned to the largest line number). Hunk/file headers and the
`\ No newline` marker are left unchanged.

Applied in _unified_diff (covers edit / edit_lines / smart_edit) and the
`diff` action — a single server-side change, so both the TUI and the web
client receive numbered diffs in the tool result text.

TUI highlight_unified_diff renders the `{marker} {num}│` column dim and the
content in the marker color, so the number reads as meta, not as part of the
added/removed text. Lines without the prefix (standalone `diff` event, older
callers) are still colored whole.

Co-Authored-By: Claude <noreply@anthropic.com>

replan: integrated mid-task re-planning tool ...

Add a replan tool that re-runs the planner over the live session context
+ todo + scratchpad when the plan's structure is stale due to discoveries
(NOT failed steps -- that stays [Adaptive re-plan]). Integrated approach:
PlanningEngine.run gains is_replan/replan_context (suppresses DIRECT
shortcut and observe-skip, frames Phase 1 as a revision); ReplanRunner
packs reason/goal/todo/findings/errors and captures PlanReady; the tool
is exposed to navi_code/developer/tool_developer via a
current_replan_runner ContextVar set per-iteration in run_stream (correct
after switch_profile). New plan replaces the todo. Lazy events import
breaks the navi.tools -> navi.core -> navi.tools cycle.

recall: carry self-instruction (message) on the recall_update wire ...

The recall card could show call_type/trigger_at but not the self-instruction
(additional_context_message) — it was absent from RecallUpdate.to_wire, so
the user couldn't see what future-self was about to do at the scheduled or
fired moment. Extend the wire payload.

- events.RecallUpdate: add `message` field; to_wire emits "message".
- scheduler._publish_recall_update: accept and forward `message`.
- Publish sites carry message=recall.additional_context_message:
  schedule_recall (scheduled) and orchestrator._finalize_recall
  (rescheduled / fired / cancelled). manage_recall cancel/skip omit it
  (no recall object handy; already visible in the prior scheduled card).
- TUI RecallRenderer: preview the message (first line + "(+N lines)",
  capped at 80) on a `msg:` body line when present.
- Tests: RecallUpdate.to_wire carries message (defaults None); renderer
  preview (single/multiline/truncate/empty) and scheduled-card renders it.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem edit: render red/green diff in TUI without changing model context ...

edit is now the primary editing method, yet its output was a dry one-line
status ("Edited …: replaced X B with Y B") while edit_lines/smart_edit showed a
highlighted unified diff. Carry the unified diff in ToolResult.metadata["diff"]
(kept out of the model-facing output, so the agent's context is unchanged) and
render it in the TUI: dim summary line plus highlighted diff (green +, red -,
dim @@). Falls back to plain text when no diff metadata is present.

Co-Authored-By: Claude <noreply@anthropic.com>

filesystem: make edit/edit_lines the default, smart_edit a last-resort fallback ...

Agent overused smart_edit (whole-file LLM call, weak context). Rewrite the
FilesystemTool description decision tree so edit (exact text) is the default and
edit_lines (by line numbers) is the deterministic option; smart_edit is reserved
for genuinely semantic changes that cannot be expressed as exact text or line
numbers. Strengthen the old_not_unique hint to steer back to edit/edit_lines
before smart_edit. Add an "Editing policy" section to the navi_code and
developer profile prompts reinforcing the same priority.

Co-Authored-By: Claude <noreply@anthropic.com>

tui: show the changed step's text in the todo update call card ...

The todo update call card (→ todo · #3 → done) now shows the text of the
step being changed, plus the validation when present. The LLM's update args
carry only the index, not the task text, so the text is read from the
current plan row before the tool runs and attached to ToolStarted.metadata
(client-rendering only, backward-compatible).

Backend:
- events.ToolStarted gains a metadata dict (mirrors ToolEvent) → to_wire.
- navi/tools/todo: step_text_for_update(index, ctx) resolves the step text
  via _sid/_uid (so sub-agent isolation holds), started_metadata_for_call
  wraps it for both emit sites.
- agent.py (parent) and subagent_runner.py (sub-agent) enrich ToolStarted
  via the shared helper before emitting.

Renderer:
- TodoStartedRenderer update card reads msg.metadata.step_text → shows the
  step text + validation; falls back to 'no validation' on history replay.
- Removed the step-text line from the result card (it now lives in the call
  card) — result card is back to the compact 'plan · X/N done' summary.

Co-Authored-By: Claude <noreply@anthropic.com>

Isolate sub-agent todo + render live todo in TUI side panel ...

Phase 1 — sub-agent todo isolation (backend):
- Add current_todo_session_id ContextVar; subagent_runner scopes it to the
  sub-agent's ephemeral run id so its auto-populated plan and todo updates
  land in an isolated KV row instead of clobbering the parent session's todo
  (which the parent's goal-anchoring reads every iteration).
- todo._sid() and planning.set_tasks prefer current_todo_session_id; the
  parent run leaves it unset, so all existing todo consumers (anti-stall,
  goal anchor, get_progress_message) behave exactly as before.

Phase 2 — live todo in the TUI right column:
- TodoUpdated event + emit it from the agent loop after planning auto-populate
  and after each tool-execution turn.
- GET /sessions/{id}/todos reads the parent session's todo KV row (explicit
  user_id/session_id, optional injected kv).
- api.get_todos + TodoList/TodoPanel widgets: status-coloured markers
  (pending dim, in_progress accent+bold, done success, failed error, skipped
  dim), progress header, scrollable panel below the auto-height info block.
- Hybrid delivery: REST seeds the panel on attach/switch, todo_updated WS
  events update it live.

Sub-agent todos as nested sub-lists is deferred to a later phase.

Co-Authored-By: Claude <noreply@anthropic.com>

Navi Code: stop via Esc + project cwd propagation ...

- TUI: Esc stops active stream cooperatively via POST /sessions/{id}/stop
- TUI: render stream_stopped as status message
- CLI/WebSocket: send shell cwd in client->server message field
- Orchestrator stores cwd in session.session_metadata
- ContextBuilder injects [Working directory] into LLM context
- Agent sets current_working_directory ContextVar per turn
- tools/base: ToolContext gains cwd field
- filesystem/terminal/code_exec resolve relative paths against session cwd
- Add bin/navi-code wrapper for PATH symlink; document in README.md
- Update docs/websocket.md and tests

Full pytest: 544 passed, 1 skipped.

Co-Authored-By: Claude <noreply@anthropic.com>

Fix spawn_agent profile selection: stronger prompt + alias fallback ...

- Rewrite PROFILE SELECTION description to explicitly say: when the
  plan specifies a profile, you MUST pass profile_id. Only omit when
  the plan does NOT specify a profile.
- Add JSON examples showing correct profile_id usage.
- Add fallback for 'profile' parameter name (models sometimes use
  'profile' instead of 'profile_id' when the description says
  'Profile to use').\n\nCo-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Fix scratchpad tool: rename op→action, add examples, improve error messages ...

- Rename primary parameter from 'op' to 'action' for consistency with
  all other tools (terminal, filesystem, todo, etc.). Legacy 'op' still
  works as a fallback to avoid breaking old calls in compressed context.
- Add JSON examples directly in the tool description so the model sees
  the exact structure to produce.
- Improve all error messages to include the correct JSON example,
  making it obvious what the model did wrong.
- Add manuals/scratchpad.md for tool_manual support.
- Update and expand tests for new syntax and error cases.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Clarify terminal description param and profile prompts ...

- terminal.py: stronger description for the 'description' field so
  the model knows it is REQUIRED for action=open and what to write
- server_admin and developer system prompts: add a short section
  explaining persistent terminal usage (open/close/list/status/send_input)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Move terminal_manager to _internal subpackage ...

- terminal_manager is an internal helper, not a tool
- Update imports in terminal.py, container.py, test_terminal.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix terminal review issues: security, lifecycle, reactivity ...

- TerminalManager.open now accepts exec_tokens to use create_subprocess_exec
  for restricted commands instead of always using shell
- Fix kill/terminate order: SIGTERM first, SIGKILL fallback
- Pop closed sessions from _sessions dict to prevent memory leak
- Add terminal_manager.shutdown() to AppContainer.shutdown()
- Wait for reader tasks in foreground open before returning output
- Add _MAX_TERMINALS_PER_SESSION limit (10)
- Wrap cleanup_idle tasks in _close_one_safe with error logging
- send_input catches BrokenPipeError/ConnectionResetError specifically
- Foreground terminals auto-close after gathering output
- Vue reactivity: replace terminals object immutably instead of mutating
- onTerminalClosed marks matching tool card as no longer pending
- Update tests for new behavior (foreground auto-close, max limit)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add persistent multi-session terminal tool with background support ...

- New TerminalManager module: named subprocess sessions per Navi session,
  background readers, event-sink streaming, idle auto-cleanup
- Refactor terminal tool to multi-action: run, open, close, list,
  status, send_input
- Add TerminalOutputDelta and TerminalClosed events for streaming
- Wire TerminalManager into AppContainer, orchestrator, and registry
- Persist session_metadata in Session model and pg_session_store
- Close all session terminals on session delete
- Webclient: handle terminal_output/terminal_closed WS events,
  display live terminal output in tool cards
- Update unit tests for new terminal actions

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix user tool execute() signature to accept ctx keyword argument ...

The tool_executor always passes ctx=ctx when calling tool.execute(),
but _try_module_level in loader.py created user tools with
_execute(self, params) only. This caused:

  Error: _execute() got an unexpected keyword argument 'ctx'

when calling get_current_datetime and any other module-level user tool.

- Add ctx=None parameter to the generated _execute wrapper
- Preserves backward compatibility with existing user tools

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add MCP health check loop, auto-reconnect, and system toast notifications ...

Backend:
- McpManager: keep all configured servers in the pool even if connect()
  fails at startup; health-check loop (30s) tries to reconnect dead servers
  and verifies live ones with list_tools()
- McpManager: set_on_server_connected callback re-registers tools when a
  dead server comes back online
- McpClient: add mark_disconnected() for silent drop detection
- McpStatusTool: skip list_tools() for disconnected servers
- Orchestrator: broadcast mcp_status_update to all WebSocket sessions
- New event type McpStatusUpdate with server_name, status, tool_count

Webclient:
- useWebSocket: handle mcp_status_update → toast.success/toast.error

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Pass explicit ToolContext to tools instead of hidden ContextVars ...

Add ToolContext dataclass (session_id, event_sink, stop_event, model,
user_id, user_role, user_info) and thread it through the execution chain:
Agent._execute_tools_with_sink → ToolExecutor → tool.execute().

All ~25 tools updated to accept ctx parameter. Tools that previously
read ContextVar now prefer ctx when provided, falling back to
ContextVar for backward compatibility.

Tests updated to pass ToolContext explicitly — no more test fixtures
that set current_session_id / current_user_id ContextVars.

ContextVar setters remain as fallback for non-tool consumers
(ai_helper, context_builder, planning) and will be removed in a
follow-up refactor.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refactor profile tool config to explicit agent/subagent structure ...

Replaces the confusing mix of enabled_tools + mcp_servers + subagent_tools
with a single explicit structure:

    tools: {
      agent:   {native: [...], mcp: {server: [groups]}},
      subagent:{native: [...], mcp: {server: [groups]}}
    }

Why:
- Old fields mixed native and MCP names (mcp__server__tool) in one list,
  making it impossible to tell at a glance what a subagent actually gets.
- subagent_runner.py had 25 lines of runtime MCP filtering logic that
  was hard to follow and error-prone.

Changes:
- AgentProfile: add ToolConfig / ToolScopeConfig pydantic models,
  keep old fields (enabled_tools, mcp_servers, subagent_tools) for
  auto-migration via _migrate_tools validator.
- loader.py: read new "tools" key, auto-migrate legacy configs.
- agent.py: _tool_list now accepts ToolScopeConfig.
- subagent_runner.py: simplified — profile.get_subagent_tools() returns
  the exact scope, no runtime filtering needed.
- context_builder.py, list_tools.py, spawn_agent.py: updated to use
  profile.get_agent_tools() / get_subagent_tools().
- All 6 profile config.json files migrated to new schema.
- Secretary subagent now explicitly gets navi-web MCP tools for web search.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Migrate MCP tool naming from mcp:server:tool to mcp__server__tool ...

The colon separator (mcp:server:tool) confuses many LLMs during
tool-calling because colons appear in schemas and URLs. Switch to
double-underscore separator (mcp__server__tool) for robust parsing.

Key changes:
- navi/mcp/tools.py: add build_mcp_name(), parse_mcp_name(), is_mcp_tool()
- navi/core/tool_executor.py: update _resolve_tool() with new helpers
  and legacy colon fallback for old sessions
- navi/core/tool_utils.py, subagent_runner.py: use build_mcp_name()
- navi/api/routes/{admin,agents}.py: prefix via build_mcp_name()
- navi/tools/{list_tools,reload_tools}.py: migrated
- All profile configs + system_prompt.txt: replace mcp: with mcp__
- manuals/{model_3d,lint_scad,render_3d,spawn_agent}.md: updated
- mcp_servers.d/gnexus-book.json: instructions updated
- docs/{api,profiles,tools,mechanics,visual.html}: updated
- tests: test_tool_executor.py and test_mcp.py aligned

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix session file URLs (add /api prefix) and WebGL error handling in STL viewer ...

Backend:
- _file_url in content_store.py now returns /api/sessions/... instead of /sessions/...
- share_file.py URL construction updated to include /api
- content_publish.py docstring updated to reflect correct endpoint

Frontend:
- contentLinks.js: avoid double /api prefix in dev mode when backend already returns it
- stl.html: replace throw err with return after showing WebGL fallback message
- Rebuild dist

Tests:
- Update expected URLs in test_content_store.py and test_share_file.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix review issues: KV-store NULL, SVG messages, filesystem tests + docs ...

Bugs fixed:
- filesystem.py: add missing `import re` (grep was broken in production)
- image_view.py: consistent SVG rejection message for URL and file paths
- store/__init__.py: normalize user_id None→'' to prevent duplicate rows
  in unique constraint for anonymous sessions; add DDL migration for
  existing NULL values

Tests:
- Add 10 unit tests for filesystem copy, grep, diff operations

Documentation:
- agent.md: document streaming guard wrapper, system prompt caching,
  ContextVar restoration in subagents
- tools.md: document middleware hooks
- websocket.md: document image upload limits and concurrent run guard
- store.md: document user_id normalization
- mechanics.md: mark newly-documented mechanics as documented

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix scratchpad get_section user_id and filesystem diff newline bugs ...

- get_section: use _uid() instead of None so multi-user lookups match
  the user_id-scoped writes done by the scratchpad tool execute()
- filesystem _diff: remove erroneous [l + \"\\n\"...] wrappers around
  splitlines() output; difflib.unified_diff already handles line lists

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add inherit_system_prompt and is_subagent_only mechanisms ...

- inherit_system_prompt: subagent parameter to prepend parent's system
  prompt as a base layer before subagent specialisation
- is_subagent_only: profile flag blocking switch_profile, allowing
  spawn_agent only; shown with [subagent only] tag in list_profiles
- Document both in docs/profiles.md and docs/tools.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Enhance native toolset and add persistent KV store ...

- Add PostgreSQL-backed KvStore (navi/store/) for session-scoped data.
- Migrate todo and scratchpad from in-memory dicts to KvStore.
- Filesystem: add copy, grep, diff actions; compress description.
- CodeExec: remove language param, expose working_dir in schema.
- ImageView: resize to 1024px JPEG + Content-Type guard for URLs.
- Memory list: return distinct categories instead of all facts.
- SSH: add scp action with upload/download support.
- Update CLAUDE.md (Postgres-only), docs/tools.md, add docs/store.md.
- Fix agent/planning/context_builder async signatures for todo helpers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

cleanup: remove deprecated tools and orphaned memory tools ...

Removed (no profile used them, no cross-dependencies):
- write_tool.py, delete_tool.py, test_tool.py
- memory_save.py, memory_search.py, memory_forget.py

Updated:
- navi/core/registry.py — removed imports and registrations
- navi/tools/__init__.py — removed imports
- docs/tools.md — removed references, updated self-extension section to MCP
- navi/tools/tool_manual.py — updated example to create_mcp_server

Profile fixes:
- developer: +tool_manual, +ssh_exec
- discuss: +list_profiles
- tool_developer: +mcp_servers navi-web

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

fix(recall): stabilize scheduled callback system and improve UX ...

Backend fixes:
- stop_session now stops headless recall runs via _busy_sessions dict
- _fire_recall sets user ContextVars so tools work correctly
- MaxIterationsReached treated as success, not failure
- skip_next_recall uses GREATEST(trigger_at, now) for overdue recalls
- schedule_recall rejects past trigger times
- timezone offset double-adjustment fixed for aware datetimes
- _fire_recall registers _AgentRun for reconnect/replay support
- session_sync race with stream_start fixed

Frontend improvements:
- Recall banner moved to ChatHeader with live Cancel/Skip buttons
- Recall messages styled with is_recall flag and badge
- Real-time recall updates via WebSocket (recall_update events)
- Recall filter moved to sessions-header as toggle button
- Session list shows clock icon for sessions with pending recall
- Empty state messages for empty/filtered session lists
- Fixed missing api import in ChatHeader.vue

Tests:
- Updated scheduler_loop tests for _busy_sessions dict change

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Refactor: move tool helper modules into _internal subpackage ...

Moves non-tool infrastructure out of navi/tools/ root so that only
actual tool classes live there:

  base.py           → _internal/base.py
  loader.py         → _internal/loader.py
  middleware.py     → _internal/middleware.py
  logging_middleware.py → _internal/logging_middleware.py
  _time_parser.py   → _internal/time_parser.py

All imports updated across core/, api/, mcp/, tools/, and tests/.
No proxy files remain in navi/tools/ root.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add self-recall (scheduled callback) system ...

Core features:
- schedule_recall tool: once/recurring/immediate callbacks
- manage_recall tool: cancel/skip/list scheduled recalls
- Natural-language time parser (ISO, relative, "tomorrow at 09:00")
- PostgreSQL-backed RecallScheduler with lazy pool init
- Background recall_scheduler_loop with asyncio.Semaphore(3)
- _busy_sessions guard prevents user messages during headless runs
- Agent.run() preserves thinking field for session history visibility
- API endpoints: GET/DELETE/POST for session recall, admin list
- Frontend: recall badge, filter, cancel/skip in sidebar and chat header
- Tests: parser, scheduler CRUD, tools, API, scheduler loop (53 tests)
- Manuals: schedule_recall.md and manage_recall.md

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>