navi: a token count that matches the model, a type for every file, and a socket that admits it is dead ...

Compression decided everything in chars // 3, and on this workload that sits
~1.8x below reality — 101k real prompt tokens against a ~56k estimate — so the
trigger, the target, the truncate budgets and the pre-call size check all ran
in a unit half the size of the real one. TokenCalibration now fits
overhead + chars / chars_per_token per model from the prompt_eval_count that
every call already reports, and a profile declares its own context_window
(sent to Ollama as num_ctx) instead of every model being held to OLLAMA_NUM_CTX.

Superseded summaries were never demoted in the database: save() only updates
flags for rows in session.messages, a summary is is_display=False, so it lived
in session.context alone and every reload resurrected it — prod sessions
carried up to 15 live summaries. PgSessionStore.set_context_flag() writes the
flip for those rows directly.

Five extension tables disagreed about what a file is. navi/content_types.py is
the one detector — signature, then extension, then a text probe — it decides
the MIME the download route serves, and the session file list now carries the
type the viewers branch on.

The idle receive loop answers client pings and sends its own heartbeat. A
half-open socket keeps readyState at OPEN and never fires close, so traffic
coming back is the only proof the connection still works.

synapse: a reaction continues its conversation, and the request becomes a record ...

Every Synapse event used to open its own service session, so two messages of
one Telegram chat landed in two unrelated chats. The dispatcher now returns a
`session_key` — derived from the payload according to the user's routing
document — and a later event carrying the same key joins the session that
opened it, within `reaction_session_ttl_minutes` of idle time (0 restores the
old always-fresh behaviour). A session that is still running is never joined:
`create_run` overwrites `state.run`, so a second run would orphan the first
one's subscribers — a busy thread gets a fresh session instead. When the
dispatcher names a different profile for the event, the session's profile is
switched through `set_profile`, never through `save()`, which does not rewrite
`name` or `profile_id`: the thread keeps its name and its history.

What the user reads in the transcript is now only the request. The event
record is a system message (`metadata.source == "synapse_event"`) drawn by a
new SynapseEventNotice.vue in the visual language of the recall badge, and the
framing plus the JSON envelope become a hidden user turn written once, on the
first event of a thread — not on every one. The instructions from Settings
reach the model through a new `current_reaction_instructions` ContextVar and
the `[Reaction session]` block of the system prompt, so they appear in no
transcript at all. The stored system record still has to reach the model, so
ContextBuilder's allowlist becomes `_LLM_SYSTEM_SOURCES = {task_note,
synapse_event}`.

The dispatcher gets a document of its own — how to route, rather than what to
do — stored beside the reaction instructions, versioned separately, and edited
in its own field in the settings panel. `synapse_instruction_versions` gains a
`doc` discriminator, so existing rows keep `'reaction'` and their history
verbatim. The new columns are added by `_MIGRATE`, and the index over `doc`
lives there too: `_DDL` runs first, so an index over a column the migration has
not added yet kills the whole batch on an existing database — which is what
the prod database is. A guard test keeps it that way.

Tests: 1776 passed, 1 skipped (backend), 242 passed (webclient, 28 files).

Allocate message sequence numbers in the DB, not in memory ...

save() numbered new messages from Session.db_next_sequence, a counter read
when the session was loaded. Any second writer of the same session handed
out the numbers it still believed were free, and the two inserts collided on
UNIQUE(session_id, sequence_number) — in production, when switch_profile
loaded the session mid-run and saved it back. The turn died with
"Internal error: duplicate key value violates unique constraint".

The range is now claimed inside save()'s transaction with a single
UPDATE ... RETURNING, so concurrent writers serialize on the session row,
and GREATEST() seeds the pre-counter sessions whose next_sequence is still 0
instead of relying on a racy max()+1 fallback in memory.

switch_profile itself no longer saves a session at all: it repoints the row
through a narrow set_profile() UPDATE, which keeps it out of the running
turn's way. It runs mid-turn on a session the turn still holds, so saving a
second copy from there was the collision in the first place.

synapse reactions W1: special sessions flag + per-user reaction settings and instruction history

local mode isolation: anonymous user is a NULL-owner user, not an alias ...

Local mode (NAVI_AUTH_ENABLED=false) used an anonymous admin whose admin
role widened every session listing to ALL users' chats — acceptable only
while assuming a private database, but a real leak on any shared one.

- session listings (list_all/list_page/count_all/search_list): new
  scoping rule — non-admin with user_id=None sees only user_id IS NULL
  rows; named owner unchanged; admin unchanged
- session create/list routes: owner id is None in local mode (sessions
  are persisted ownerless), admin listing flag resolved only when auth
  is enabled
- check_session_access: local mode allows only NULL-owner sessions —
  a user-owned session by id is 403
- debug/admin endpoints (require_admin) stay reachable in local mode
- prod DB: 30 stray user_id='anonymous' rows reassigned to NULL owner

Tests: store scoping unit tests, local-mode integration tests (sidebar
filtering + access by id), updated check_session_access unit tests.
Full pytest 1239 passed, 1 skipped.

Add persistent multi-session terminal tool with background support ...

- New TerminalManager module: named subprocess sessions per Navi session,
  background readers, event-sink streaming, idle auto-cleanup
- Refactor terminal tool to multi-action: run, open, close, list,
  status, send_input
- Add TerminalOutputDelta and TerminalClosed events for streaming
- Wire TerminalManager into AppContainer, orchestrator, and registry
- Persist session_metadata in Session model and pg_session_store
- Close all session terminals on session delete
- Webclient: handle terminal_output/terminal_closed WS events,
  display live terminal output in tool cards
- Update unit tests for new terminal actions

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add archive message pagination, configurable WS replay buffer ...

Backend:
- Add archive_threshold to Session model and getSession response
- Add next_before_seq to archive endpoint for cursor pagination
- Make WS replay buffer size configurable via WS_REPLAY_BUFFER_SIZE

Webclient:
- Add getArchivedMessages API function
- Add archive pagination state and loadArchivedMessages to chat store
- MessageList: auto-load older messages on scroll-to-top with scroll
  position preservation and loading spinner

Docs: update config.md with new env vars

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add session message archive table + global sequence_number tracking ...

Schema:
- session_messages_archive — identical structure, stores old messages.
- sessions.next_sequence    — monotonic seq counter per session.
- sessions.archive_threshold — split point between hot and archive.

Behaviour:
- get() / _build_sessions() load only seq >= archive_threshold (hot).
- save() UPDATEs existing rows (seq >= 0) and INSERTs new ones (seq = -1)
  with auto-assigned sequence_number = next_sequence, next_sequence+1, ...
- archive_old_messages() moves a batch to archive and bumps the threshold.

This keeps the hot table bounded so list/get RAM stays flat regardless
of total session history size.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Implement delta-save for session messages ...

Replace full DELETE/INSERT with efficient delta writes:
- Track db_message_count on Session (how many rows already persisted).
- On save(): UPDATE mutable flags for existing rows, DELETE only extras
  (race guard), INSERT new messages via executemany.
- Reduces DB write amplification from O(N) to O(delta) per turn.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add session search across messages with backend + frontend ...

Backend:
- GET /sessions now accepts `search` query param
- _session_summary computes match_indices and match_preview from messages
- pg_session_store: messages ILIKE added to search_list and count_all
- In-memory store: search_list also filters by message content

Frontend:
- AppSidebar: search toggle icon + debounced input, Ctrl+K shortcut
- SessionItem: highlight matching text, show match_preview from search
- SessionList: pass searchQuery to SessionItem
- SessionsStore: searchQuery/searchActive state, setSearch/clearSearch
- API layer: getSessions accepts search param
- MessageList: scroll to target message + brightness flash animation
- ChatStore: loadSession accepts targetMessageIndex, scrollToMessageIndex ref
- CSS: msg-flash keyframe animation in app.scss
- Tests updated for new getSessions signature

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add pagination, search, and sorting to admin sessions ...

Backend:
- Add count_all and search_list abstract methods to SessionStore
- Implement count_all and search_list in PgSessionStore (SQL with ILIKE)
- Implement count_all and search_list in InMemorySessionStore
- Update /admin/sessions to accept limit, offset, search, sort_by, sort_order
- Return {total, limit, offset, items} from /admin/sessions

Frontend:
- Add search input for sessions in admin panel
- Add clickable sortable column headers with asc/desc toggle
- Add pagination controls (prev/next, page size selector, item count)
- Debounce search input (300ms)

Tests:
- Add integration tests for pagination, offset, search, and sorting
- All 217 tests pass

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Multi-user auth via gnexus-auth OAuth + hybrid role/permission model ...

- Integrate gnexus-auth-client-py (GAuthClient) for OAuth flow, token refresh,
  and webhook parsing
- Add navi/auth/ package: User model, Fernet encryptor, client singleton,
  deps (get_current_user, require_admin, require_permission)
- New tables: navi_users, user_auth_sessions (auto-created on startup)
- Session/memory isolation by user_id with legacy NULL support
- Cookie-based auth proxy: /auth/login, /callback, /logout, /me
- Webhook receiver /webhooks/gnexus-auth handling user events, global logout,
  session revocation, role/permission changes
- Admin endpoints (/admin/*) gated by role + permissions
- Webclient auth store with isAdmin/hasPermission guards
- Admin-only profile filtering in /agents/profiles
- 200/200 tests passing

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Paginate session list loading

Planning debug panel, todo auto-populate, scratchpad/persona improvements ...

- Planning debug panel: new Planning tab in debug/index.html shows raw
  phase 1/2 outputs and token counts per planning run, stored in
  session.planning_logs (new column in both SQLite and PostgreSQL)
- New GET /sessions/{id}/planning API endpoint
- PlanningDebugData internal event wires _run_planning() output into
  session storage; never forwarded to WebSocket clients
- Phase 3 (plan critic) disabled — to be reworked with reflect integration
- Todo tool: auto-populated from plan steps after phase 2; model only
  needs to call update/view, not set
- Scratchpad: clarified description and persona instructions; removed
  context_transfer from user-facing docs (internal mechanism only)
- web_search: switched to ddgs package, SearXNG as primary backend,
  DDG html-only fallback; added find_up action to filesystem tool
- Persona: added SCRATCHPAD and TODO sections with clear usage rules;
  added NAVI.md project context instructions
- chat.js: fixed subagent planning event fallthrough into parent UI;
  statusLabel cleared on first stream delta

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Add session name generation via LLM ...

Backend:
- Session model gets name: str | None field
- SQLite migration: ADD COLUMN name TEXT
- PostgreSQL: ADD COLUMN IF NOT EXISTS name TEXT (applied on pool init)
- SessionStore: add set_name() abstract method, implemented in all stores
- navi/core/name_generator.py: LLM worker that reads user messages and
  returns a 3–6 word title or None if content isn't substantial yet
- POST /sessions/{id}/generate-name endpoint: fires LLM, saves and
  returns name; skips if session already named or has no user messages
- GET /sessions and GET /sessions/{id} now include name field

Client:
- api.generateSessionName(id) — calls the new endpoint
- sessions store: updateName(id, name) mutation
- chat store: after stream_end, _tryGenerateName() runs fire-and-forget;
  skips silently if session already has a name or if request fails
- SessionItem already displays session.name (falls back to id prefix)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Fix context loss: ensure system prompt is always present in LLM context ...

Replaced `if not session.context:` with a role-based check so the system
message is inserted whenever it is absent — not just for brand-new sessions.

Root cause: backward-compat sessions (context column was empty) had their
context initialised from session.messages, which never contains a system
message. The old check (`if not session.context:`) saw a non-empty list and
skipped the system prompt, so every subsequent request ran without it —
Navi had no persona and no profile instructions.

Also add context_token_count field to Session (follow-up for token counter
fix — persistence wiring comes in next commit).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Separate display history from LLM context; formalize worker system ...

Architecture change:
- session.messages: full display history, never modified by compression
- session.context: what the LLM sees, may be compressed by workers
- System messages go only into context (not display history)
- Image injections (synthetic) go only into context
- User/assistant/tool messages go into both

SQLite: add context column with backward-compat migration
(empty context → initialized from messages on load)

Workers (navi/workers/):
- Worker ABC + WorkerContext + WorkerResult (base.py)
- CompressionWorker: compresses session.context when above threshold
- build_default_workers() returns [CompressionWorker()]
- Agent accepts workers list, runs them after StreamEnd
- Workers injected via deps.py get_workers() (lru_cached singleton)
- WebSocket agent construction also receives workers

Compressor: compress_context() now takes context[], not messages[]
Config: context_keep_recent 6 → 10
Agent: _run_workers() collects events from all workers and yields them

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Server review fixes: profile model routing, sorting, datetime, cleanup ...

- LLMBackend.complete/stream: add model param; OllamaBackend uses it
  over self.model, enabling per-profile model selection
- BackendRegistry.get(): remove unused model param
- Agent: pass profile.model to complete() and stream()
- Profiles: correct model to gemma4:e2b-it-q8_0 (was leftover e4b)
- InMemorySessionStore.list_all(): fix sort (pinned+newest first,
  was pinned+oldest) — now consistent with SQLite ORDER BY
- session.py, sqlite_session_store.py: datetime.utcnow() →
  datetime.now(timezone.utc) (deprecated since Python 3.12)
- _base_options(): accept temperature param, remove dead default
- deps.py: rename _registries → get_registries (public API)
- websocket.py: update import accordingly

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Pin sessions + larger code font ...

Server:
- Session.pinned field (bool, default False)
- SqliteSessionStore: ALTER TABLE migration for existing DBs, set_pinned(),
  ORDER BY pinned DESC, last_active DESC
- PATCH /sessions/{id}/pin endpoint

Client:
- Pin/unpin button appears on hover (📌/📍)
- Pinned sessions get blue left border and always-visible pin icon
- Local sort mirrors server sort (pinned first)
- Code blocks font-size bumped from 0.82em to 0.9em

Initial implementation of the agent system core ...

- FastAPI server with REST API and WebSocket streaming
- Modular LLM backend abstraction (Ollama implemented, OpenAI stub)
- Tool system: web_search (ddgs), filesystem, http_request, code_exec, terminal
- Agent profiles: smart_home, server_admin, secretary
- Tool-calling loop with concurrent tool execution
- In-memory session store with SessionStore ABC for future persistence
- Registry pattern for tools, profiles, and backends
- Orchestrator stub as foundation for multi-agent scenarios