navi: a token count that matches the model, a type for every file, and a socket that admits it is dead ...

Compression decided everything in chars // 3, and on this workload that sits
~1.8x below reality — 101k real prompt tokens against a ~56k estimate — so the
trigger, the target, the truncate budgets and the pre-call size check all ran
in a unit half the size of the real one. TokenCalibration now fits
overhead + chars / chars_per_token per model from the prompt_eval_count that
every call already reports, and a profile declares its own context_window
(sent to Ollama as num_ctx) instead of every model being held to OLLAMA_NUM_CTX.

Superseded summaries were never demoted in the database: save() only updates
flags for rows in session.messages, a summary is is_display=False, so it lived
in session.context alone and every reload resurrected it — prod sessions
carried up to 15 live summaries. PgSessionStore.set_context_flag() writes the
flip for those rows directly.

Five extension tables disagreed about what a file is. navi/content_types.py is
the one detector — signature, then extension, then a text probe — it decides
the MIME the download route serves, and the session file list now carries the
type the viewers branch on.

The idle receive loop answers client pings and sends its own heartbeat. A
half-open socket keeps readyState at OPEN and never fires close, so traffic
coming back is the only proof the connection still works.

e2e fixes: tasks tool in profiles, bg timeout lift, stepIcon fix, queue semantics docs ...

E2E findings addressed:
- tasks tool was registered but not in any profile's tools.agent.native —
  added to all six profiles (agent could not check/wait/cancel bg tasks)
- detached terminal/code_exec/ssh_exec runs without explicit timeout are
  lifted to 300s: foreground defaults (20/30/60s) marked long commands
  'completed' with partial output while the process still ran
- ToolCard.vue: define stepIcon(status) — template referenced it but the
  function was missing (render crash on task_update step)
- message_queued reachability documented: WS read loop is sequential, the
  queue path is only reachable from a second socket/headless recall

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

docs: sync with the plan rework and compression pipeline ...

- planning: plan tool flow (agent-invoked, no pre-turn gate), confirmation
  by COMPLEXITY, think=False + 20k-char Phase 1 window, retired flags
  removed from profiles/api docs
- compression: all four triggers through compress_and_save_session,
  real-baseline gates, head+tail summary input, _plan_compression shared
  decision, actual config defaults (0.90 threshold, 6000 summary tokens,
  32k input cap)

docs(websocket): /compact control message, new events, fixes ...

- Document the {"type":"compact"} forced-compression control message
  (bypasses threshold, no stream_start, rejected if a run is active,
  TUI /compact + Ctrl+X C).
- Fix image limits: 8 images, 50 MB total payload (was 10 / 5 MB each).
- tool_started/tool_call: document metadata + tool_call_id fields.
- recall_update action: add skipped (scheduled|cancelled|skipped|fired|
  rescheduled).
- session_sync: document the session_id/profile_id payload.
- Add missing events: model_info, todo_updated, terminal_output,
  terminal_closed.
- Document close code 4003 (auth/access denied) alongside 4004.

Co-Authored-By: Claude <noreply@anthropic.com>

docs(websocket): record deferred WS auth-message hardening ...

Move task #248 (accept the API token as the first WS frame instead of a
?api_token query param, which leaks into access logs) into the websocket
docs as a Future work section with rationale, protocol sketch, and scope.

Navi Code: stop via Esc + project cwd propagation ...

- TUI: Esc stops active stream cooperatively via POST /sessions/{id}/stop
- TUI: render stream_stopped as status message
- CLI/WebSocket: send shell cwd in client->server message field
- Orchestrator stores cwd in session.session_metadata
- ContextBuilder injects [Working directory] into LLM context
- Agent sets current_working_directory ContextVar per turn
- tools/base: ToolContext gains cwd field
- filesystem/terminal/code_exec resolve relative paths against session cwd
- Add bin/navi-code wrapper for PATH symlink; document in README.md
- Update docs/websocket.md and tests

Full pytest: 544 passed, 1 skipped.

Co-Authored-By: Claude <noreply@anthropic.com>

Add NAVI_AUTH_ENABLED switch for optional auth ...

- Add navi_auth_enabled setting (default true) to navi/config.py and .env.example
- When disabled, treat every request as anonymous admin user (id='anonymous')
- Create/update fixed anonymous navi_users row on startup
- Bypass OAuth/cookie/API-token resolution in navi/auth/deps.py
- Update /auth/status to return {enabled, configured}
- Log security warning on startup when auth is disabled
- Update webclient: skip fetchMe/login screen, show Local mode footer,
  expose /admin link, warn in API keys panel
- Rebuild webclient production bundle
- Add unit and integration tests for no-auth mode
- Update docs: auth.md, config.md, api.md, api_tokens.md, sessions.md,
  websocket.md, mechanics.md, index.md

Co-Authored-By: Claude <noreply@anthropic.com>

Document mcp_status_update WebSocket event

Fix settings route switching and document API token system ...

- App.vue: make route reactive via ref + getRouteFromHash() so
  #settings toggles work without page reload
- docs/api_tokens.md: new comprehensive API token auth doc
- docs/api.md: add /api-tokens REST endpoints
- docs/auth.md: add token flow architecture diagram and resolution order
- docs/websocket.md: add auth section with cookie vs query-param token
- docs/architecture.md: update to AgentSessionOrchestrator + ToolContext
- docs/tools.md: add gnexus-creds MCP tools
- docs/index.md: link to api_tokens.md
- Rebuild webclient dist

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extract ContextCompressor, fix STL viewer, expand test suite, add architecture audit docs ...

- Extract ContextCompressor from agent.py (Step 1 of god-object refactor)
- Add retry + hard-truncate fallback logic to ContextCompressor
- Add unit tests: agent loop (14), compressor (18), KV store (8),
  auth encrypt (3), auth deps (13), todo/scratchpad/image_view/memory
- Fix WebGL STL viewer: allow-same-origin sandbox + graceful fallback
- Add CompressionStarted event and client-side compression notice
- Add docs/architecture_weak_spots.md and plan_01_god_object_agent.md
- Update test_events.py and test_agent_context_size.py for new logic

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix review issues: KV-store NULL, SVG messages, filesystem tests + docs ...

Bugs fixed:
- filesystem.py: add missing `import re` (grep was broken in production)
- image_view.py: consistent SVG rejection message for URL and file paths
- store/__init__.py: normalize user_id None→'' to prevent duplicate rows
  in unique constraint for anonymous sessions; add DDL migration for
  existing NULL values

Tests:
- Add 10 unit tests for filesystem copy, grep, diff operations

Documentation:
- agent.md: document streaming guard wrapper, system prompt caching,
  ContextVar restoration in subagents
- tools.md: document middleware hooks
- websocket.md: document image upload limits and concurrent run guard
- store.md: document user_id normalization
- mechanics.md: mark newly-documented mechanics as documented

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

docs: document scheduled recall system ...

- Add docs/recall.md with full architecture, data model, tools, loop behavior
- Update docs/api.md with recall endpoints (GET/DELETE/POST recall) and recall_update WS event
- Update docs/websocket.md with recall_update frame and session_sync case #3
- Update docs/tools.md: add ScheduleRecallTool and ManageRecallTool
- Update docs/sessions.md: add is_recall message flag
- Update docs/index.md: link recall.md in file map

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Audit and fix docs/ for accuracy against current codebase ...

Package A (API + WebSocket + Sessions):
- api.md: add missing endpoints (health/embed, auth/status, agents/prompts,
  agents/mcp_servers, sessions/content, 7 admin endpoints). Fix response
  schemas, auth requirements, query params. Update path formats.
- websocket.md: fix planning_status.phase type (int 1|2|3), add metadata
  to tool_call, message_index to stream_end, context_tokens/max_context_tokens
  to context_compressed.
- sessions.md: fix context_compression_threshold default (0.80→0.70),
  add list_page method, add metadata to Message flags.

Package B (Config + Profiles + Tools + Agent + Memory):
- config.md: remove stale ANTHROPIC_API_KEY description, add 12 missing
  env vars (embedding, web search, terminal_user_allowed_commands,
  context_providers_dir, gnauth fields, output_reserve_tokens).
  Fix OPENAI_BASE_URL type.
- profiles.md: fix max_iterations default (20→10), add top_k/top_p/num_thread,
  is_admin_only, context_providers, mcp_servers, subagent_think_enabled.
  Clarify subagent_tools fallback and step_validation_enabled.
- tools.md: replace 3 separate memory tools with unified `memory`, add
  `mcp_status`.
- agent.md: fix run_ephemeral return type (str→tuple), clarify DB reads.
- memory.md: fix search_facts/get_all_facts param order, fix MemoryStore
  init description.

236 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Comprehensive documentation update ...

- websocket.md: add heartbeat, session_sync, replay_start/end, planning_status,
  plan_ready (with is_subagent), stream_end extra fields, context_compressed summary,
  updated _AgentRun.events replay buffer, corrected reconnect section
- sessions.md: add name, planning_logs fields; message flags (is_plan, is_compression,
  is_summary, thinking); set_name store op; debug endpoints section
- api.md: full rewrite — add generate-name, planning, file download endpoints;
  all missing WS events; correct message field table with is_plan/is_compression
- tools.md: update user tools list with all current tools
- index.md: fix profiles list (smart_home → developer)
- CLAUDE.md: add Documentation section with table of all docs files
- NAVI.md: add architecture.md entry, improve websocket description

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add backend documentation ...

10 markdown files covering all backend subsystems: architecture,
agent loop, tool system, sessions, WebSocket protocol, profiles,
memory, config, and API reference. Structured for AI readability
with a single entry point (docs/index.md) and cross-references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>