profiles: restricted profiles for ordinary users, and a role gate that holds ...

`is_admin_only` was checked in one place out of nine and was not read from
config.json at all, so all seven profiles were reachable by every account with
role `user`. The flag now lives in config.json — the file is the baseline, a
`profile_overrides` row still wins on top of it — and one predicate,
`admin_only_blocked`, is the single place the rule is expressed. The nine
surfaces that list, switch to, spawn or resolve a profile all consult it:
`POST /sessions`, the WebSocket, switch_profile, list_profiles, the system
prompt's "Available profiles" block, spawn_agent and the Synapse reaction
runner. The prompt cache is now keyed by (profile, role), so a user's prompt
can never be served an admin's profile list.

The seven existing profiles (developer, discuss, dispatcher, modeler_3d,
navi_code, secretary, server_admin) are marked admin-only. Three new ones take
their place for ordinary users: assistant, designer_3d and coder. They share one
native tool set — ssh_exec, peer, reload_tools, create_mcp_server, test_mcp_tool,
image_view and gmail are withheld — and differ only in system prompt, model and
MCP groups. navi-web's raw `request` group, and the whole of gnexus-creds and
tgclient, are withheld too.

MCP per-user keys gain the missing half of the rule: a server that declares a
`user_key` slot is refused to anyone but an admin who has no personal key, and
is left out of their tool list entirely, instead of quietly falling back to the
owner's credential and appearing as a tool that cannot work. The refusal names
the server and points at Settings.

Also closes `GET /agents/prompts`, which served every profile's system prompt to
anyone, with no user dependency at all.

The accepted residual risk is written down in docs/profiles.md: the working
directory is a convention, not a sandbox.

docs: fix stale VAPID env names (NAVI_VAPID_* -> NAVI_PUSH_VAPID_*, per config.py)

docs: mcp.md (per-user MCP keys BYOK), /mcp-keys endpoints, user_key config

docs: synapse integration guide, special sessions filter, new endpoints ...

- docs/synapse.md: full guide — gateway, reaction runner pipeline,
  notify tool, per-user settings, service sessions in the UI
- index/tools/config/api: entry point + built-in tools + env vars +
  GET /sessions special param + synapse REST/webhook endpoints

e2e fixes: tasks tool in profiles, bg timeout lift, stepIcon fix, queue semantics docs ...

E2E findings addressed:
- tasks tool was registered but not in any profile's tools.agent.native —
  added to all six profiles (agent could not check/wait/cancel bg tasks)
- detached terminal/code_exec/ssh_exec runs without explicit timeout are
  lifted to 300s: foreground defaults (20/30/60s) marked long commands
  'completed' with partial output while the process still ran
- ToolCard.vue: define stepIcon(status) — template referenced it but the
  function was missing (render crash on task_update step)
- message_queued reachability documented: WS read loop is sequential, the
  queue path is only reachable from a second socket/headless recall

agent parallelism: background tools, bg spawn_agent, parallel tool batches, message queue ...

- backgroundable tools (terminal/ssh_exec/peer/spawn_agent/code_exec) detach
  via TaskManager with per-session/global/spawn caps, rate limit and TTL
- completion delivery both ways: task_update event (out-of-band) + pending
  notes injected into the next turn; tasks tool (list/check/wait/cancel)
- parallel tool-call batches (profile-level gate, off by default) with
  ToolStarted up-front, tagged event mux, call-order results, one save,
  plus dangling tool_call repair on session load
- user message queue instead of busy error: message_queued frame,
  back-to-back drain on the same socket, headless fallback on disconnect
- pin mcp<2 (v2 renames FastMCP with breaking API changes)
- docs: tasks.md (new), websocket/api/agent/config updates, tasks manual,
  spawn_agent background param, persona contract section

PWA: installable webclient, offline shell, web push ...

Installability:
- public/manifest.webmanifest (standalone, theme #16161E) + PNG icons
  generated from logo.svg (regular + maskable, served via /images mount)
- index.html: manifest link, theme-color, apple-touch-icon

Offline shell:
- hand-rolled sw.js (no workbox): navigation = network-first (3s race)
  with cached-shell fallback + background refresh (a stale cached shell
  would 404 on entry chunks after a deploy); /assets/* cache-first
  (content-hashed); /images/* cache-first capped; /api,/ws,/auth,/push,
  /content pass-through
- vite closeBundle plugin stamps __NAVI_BUILD_VERSION__ (digest of
  index.html + asset names) into dist/sw.js; sw.js served no-store so
  every deploy reactivates the SW and activation evicts old caches
- SW registration in main.js, PROD only (dev HMR untouched)
- OfflineBanner (useOnline composable) over the app shell

Web push (VAPID, pywebpush):
- navi/push/ package: push_subscriptions table (postgres, boot-time DDL),
  PushSubscriptionStore, PushService (async fan-out, to_thread sends,
  404/410 prunes dead endpoints, per-session cooldown)
- routes: GET /push/vapid-key, POST/DELETE /push/subscribe (auth-gated)
- trigger in orchestrator run_agent + run_recall: push on StreamEnd when
  no WebSocket client watches the session; fire-and-forget, never
  disturbs the run; anonymous fallback only when auth is off
- client: usePush composable + Notifications settings panel (enable/
  disable via PushManager.subscribe with the server VAPID key)
- notification click focuses the app at /#<session_id> (hash routing
  opens the right chat); payload body is a markdown-stripped <=140-char
  preview

NAVIVAPID keys empty = push fully disabled (graceful, like other optional
integrations). dist/ artifacts committed per repo convention.

Tests: pytest push store/service/routes/trigger (+23), vitest usePush
(83 webclient tests green). Full suite 1143 passed.

planning: fix LLM-output failure modes that killed every plan ...

Production planning_logs showed three ways a plan silently dies:

1. glm-5.3-flash wraps the whole answer in a structured envelope
   (response:unknown{value:...}<tool_call|>) — the planner only knew the
   gemma4 thought<channel|> artifact, so the wrapped analysis/plan failed
   to parse. The stripper now unwraps response:<type>{value:...} envelopes,
   including a truncated (unbalanced) one.
2. Empty content with spent completion tokens: glm on ollama-cloud puts
   the whole answer in the thinking channel on some calls. Both planning
   phases now fall back to thinking when content is empty; the debug log
   records which channel served (source: content|thinking).
3. phase3_timeout at the shared 120s LLM_COMPLETE_TIMEOUT: planning now
   has its own PLANNING_LLM_TIMEOUT_SEC (default 240).

Unit tests cover the envelope unwrap (balanced/truncated/inner braces),
the thinking fallback in both phases, and the clean-fail path.

swarm names: single female name (user preference; collisions fixed by hand)

swarm names: two female names (Japanese/American/Spanish) instead of adjective-animal ...

Single names would collide across a swarm (peers are addressed by name),
a mixed pair like 'yuki-grace' or 'sofia-rin' gives 150x150 combinations.
Only affects NEW installs - existing instance.json names stay as they are.

swarm stage 2: peer-to-peer channel — /peer endpoints + peer tool ...

Server side (navi/api/routes/peer.py, port 8099):
- GET /peer/hello — open discovery ping (name, uuid prefix, version)
- GET /peer/status — PSK; identity, uptime, machine facts, hive view
- POST /peer/ask — PSK; one-shot agent run under PEER_ASK_PROFILE
  (default server_admin) answers, one concurrent ask at a time
- loop guard: answering agent runs without the peer tool (deterministic
  recursion cut) + own-uuid asks refused with 409
- verify_swarm_key in navi/swarm.py accepts .swarm-key.previous
  (rotation window, constant-time)

Client side (navi/tools/peer.py):
- peer tool: list (hive book, stale cache fallback marked), status,
  ask by swarm name; asks go direct peer-to-peer, hive never on the
  message path
- audit events on both sides (peer.ask_sent/received/answered/failed)

peer tool added to server_admin, navi_code, developer profiles.

swarm stage 1: instance identity, PSK, hive registry, announce loop ...

- navi/identity.py: adjective-animal names + uuid in instance.json
  (generated at install, renameable by hand)
- navi/swarm.py: HiveAnnouncer - periodic POST /announce to the hive
  with non-blocking reachability tracking (transitional logs, /health
  export, hive_status context provider reports outages to the agent)
- hive/: standalone FastAPI address book (SQLite, port 8087, PSK via
  X-Swarm-Key with .swarm-key.previous rotation window, TTL online
  status, host from client IP, port from payload). Not installed or
  started by default - run manually on the main server.
- ports moved to 8099 (API) / 8098 (UI MCP)
- install.sh: PYTHONIOENCODING=utf-8 in the systemd unit, generates
  instance.json and .swarm-key on fresh installs

port: 8000/8001 -> 8099/8098 everywhere ...

navi runs on shared servers where 8000/8001 are usually taken. New
defaults: API 8099, navi_ui MCP 8098. Touched: config defaults
(navi_port, navi_ui_mcp_port, public_url, gnauth_redirect_uri),
navi-server launcher docs, env.template/.env.example, install.sh
health-check fallback, terminal client base_url, webclient dev configs
(useWebSocket, contentLinks, vite proxy), android url hint, docs.
Also made the navi_ui FastMCP constructor port settings-driven instead
of a hardcoded 8001 (it was overridden at start anyway).

deploy: one-command server bundle (dockerized postgres + systemd) ...

deploy/install.sh — everything in one command: prerequisite checks,
.env provisioning from env.template (generated DB password), dockerized
PostgreSQL (pgvector image, restart: always, bound to 127.0.0.1, vector
+ pg_trgm extensions), venv + pip install, a systemd unit with
Restart=always (the server lives from installation, per the roadmap
where these modules become each other's subagents), health check, and
navi-code/navi-server symlinks into PATH. Idempotent — safe to re-run.

Default deployed shape: web UI off, auth off, terminal client only.
Re-enabling the web panel later is editing .env + restart — nothing is
removed from the tree.

The deploy branch (next commit set) differs from master only by a
filled-in .env.

docs: sync with the plan rework and compression pipeline ...

- planning: plan tool flow (agent-invoked, no pre-turn gate), confirmation
  by COMPLEXITY, think=False + 20k-char Phase 1 window, retired flags
  removed from profiles/api docs
- compression: all four triggers through compress_and_save_session,
  real-baseline gates, head+tail summary input, _plan_compression shared
  decision, actual config defaults (0.90 threshold, 6000 summary tokens,
  32k input cap)

docs: actualize compression mechanics + real-token baseline ...

mechanics.md: rewrite the Context Compression catalog (adaptive turn
importance, intra-turn fallback, would_compress guard, token-budget
hard-truncate, per-message view truncation, profile-aware overrides,
real-token baseline estimator, meta-summary, archive-on-compress,
CompressionStarted/ContextCompressed events); fix Agent Loop rows
(run() delegates to run_stream, planning entry condition, pre/mid-turn
gates, check_context_size, forced /compact, estimate chars//3 in
compressor.py); fix WS image limits (8 / 50MB) and add /compact control
message; rebuild the Undocumented Mechanics Summary from current ❌ rows;
refresh the cross-reference index.

config.md: LLM_STREAM_FIRST_CHUNK_TIMEOUT 180→90; add
CONTEXT_MESSAGE_TOKEN_BUDGET.

agent.md: run() runs planning; full run_ephemeral signature + compact_stream
entry point; cwd ContextVar; planning entry condition; loop pseudocode
(midturn compression, check_context_size, ModelInfo, real baseline);
streaming guard 90s; workers carry profile; pre-turn estimate-based gate;
per-message view truncation.

sessions.md: full Message flag table (is_context/is_display/is_compression_critical
…); three compression trigger points + forced /compact; real-baseline
estimate; check_context_size guard; expanded algorithm (adaptive partitioning,
intra-turn fallback, meta-summary, 24k cap, 4000-char critical preview,
token-budget hard-truncate fallback, archive-on-compress, baseline clear);
compression events; SessionStore archive methods + list_page profile_id.

Co-Authored-By: Claude <noreply@anthropic.com>

compressor: structured summaries, profile-aware compression, adaptive keep_recent ...

- Replace free-form summary with strict Markdown template (Goal, Active Files,
  Decisions, Completed Work, Pending Work/Todo, Errors, Key Values).
- Keep filesystem/code_exec/terminal tool results and messages with
  is_compression_critical=True verbatim during compression instead of 300-char truncation.
- Make compression profile-aware: AgentProfile gains compression_keep_recent,
  compression_max_tokens, compression_prompt_file. navi_code uses dedicated
  compression prompt and larger keep_recent/max_tokens.
- Adaptive partition_messages(): important turns (user corrections, errors,
  critical tools) survive longer; filler/social turns compress sooner.
- Increase default context_summary_max_tokens from 3000 to 4000.
- Propagate active profile changes to ContextCompressor and SubAgentRunner.

Co-Authored-By: Claude <noreply@anthropic.com>

Update Navi Code profile: config, prompts, docs, and env example ...

- Sync navi_code profile tools with current codebase (memory, schedule_recall, manage_recall, switch_profile, list_profiles)

- Remove image_view and MCP tools from navi_code; keep local terminal focus

- Update system prompt and persona for terminal-first coding assistant

- Update docs/navi_code.md, docs/profiles.md, docs/config.md for new tool set

- Refresh .env.navi_code.example with current config variables

- Remove stale CLI prototype directories (cli/, navi_code_cli/)

Navi Code: Phase 2 — CLI terminal client, tests, docs ...

- Add clients/terminal package: config, state, REST API wrappers, WebSocket
  client, renderer, and click-based interactive CLI.
- Wire navi-code console script via pyproject.toml.
- Add unit and WebSocket integration tests for the terminal client.
- Update docs/profiles.md, docs/config.md, README.md with navi_code profile
  and default-profile instructions.
- Add docs/navi_code.md setup guide and docs/navi_code_cli.md usage reference.
- Fix lint in new test files and test_auth_disabled.py.

Tested: 459 passed, 1 skipped (excluded unrelated websocket test).

Co-Authored-By: Claude <noreply@anthropic.com>

Add NAVI_AUTH_ENABLED switch for optional auth ...

- Add navi_auth_enabled setting (default true) to navi/config.py and .env.example
- When disabled, treat every request as anonymous admin user (id='anonymous')
- Create/update fixed anonymous navi_users row on startup
- Bypass OAuth/cookie/API-token resolution in navi/auth/deps.py
- Update /auth/status to return {enabled, configured}
- Log security warning on startup when auth is disabled
- Update webclient: skip fetchMe/login screen, show Local mode footer,
  expose /admin link, warn in API keys panel
- Rebuild webclient production bundle
- Add unit and integration tests for no-auth mode
- Update docs: auth.md, config.md, api.md, api_tokens.md, sessions.md,
  websocket.md, mechanics.md, index.md

Co-Authored-By: Claude <noreply@anthropic.com>

Add archive message pagination, configurable WS replay buffer ...

Backend:
- Add archive_threshold to Session model and getSession response
- Add next_before_seq to archive endpoint for cursor pagination
- Make WS replay buffer size configurable via WS_REPLAY_BUFFER_SIZE

Webclient:
- Add getArchivedMessages API function
- Add archive pagination state and loadArchivedMessages to chat store
- MessageList: auto-load older messages on scroll-to-top with scroll
  position preservation and loading spinner

Docs: update config.md with new env vars

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Audit and fix docs/ for accuracy against current codebase ...

Package A (API + WebSocket + Sessions):
- api.md: add missing endpoints (health/embed, auth/status, agents/prompts,
  agents/mcp_servers, sessions/content, 7 admin endpoints). Fix response
  schemas, auth requirements, query params. Update path formats.
- websocket.md: fix planning_status.phase type (int 1|2|3), add metadata
  to tool_call, message_index to stream_end, context_tokens/max_context_tokens
  to context_compressed.
- sessions.md: fix context_compression_threshold default (0.80→0.70),
  add list_page method, add metadata to Message flags.

Package B (Config + Profiles + Tools + Agent + Memory):
- config.md: remove stale ANTHROPIC_API_KEY description, add 12 missing
  env vars (embedding, web search, terminal_user_allowed_commands,
  context_providers_dir, gnauth fields, output_reserve_tokens).
  Fix OPENAI_BASE_URL type.
- profiles.md: fix max_iterations default (20→10), add top_k/top_p/num_thread,
  is_admin_only, context_providers, mcp_servers, subagent_think_enabled.
  Clarify subagent_tools fallback and step_validation_enabled.
- tools.md: replace 3 separate memory tools with unified `memory`, add
  `mcp_status`.
- agent.md: fix run_ephemeral return type (str→tuple), clarify DB reads.
- memory.md: fix search_facts/get_all_facts param order, fix MemoryStore
  init description.

236 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document multi-user auth system ...

- New docs/auth.md — comprehensive auth guide: OAuth flow, roles/permissions,
  webhooks, env vars, setup instructions, API reference, WebSocket auth,
  testing with mocked auth
- Update docs/config.md — add gnexus-auth OAuth env vars table
- Update docs/api.md — add /auth/*, /admin/*, /webhooks/gnexus-auth endpoints
- Update docs/index.md — add auth.md to file map
- Update docs/sessions.md — document user_id column and list_all filtering
- Update docs/memory.md — document user_id scoping in facts and summaries

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clarify share file publishing boundaries

Update deployment docs — PostgreSQL-only, new .env vars, pg_trgm notes ...

- README.md: remove SQLite, add PostgreSQL setup (Ubuntu/Arch/Docker),
  add pg_trgm notes, update .env example
- docs/config.md: remove DB_PATH, add OLLAMA_REQUEST_TIMEOUT / OPENAI_MODEL /
  OPENAI_BASE_URL, update example .env
- .env.example: rewrite with all current variables and comments

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

WelcomeScreen polish, root-path fix, docs update ...

- Move mobile sidebar button to top-left corner (no header bar backdrop)
- Show WelcomeScreen on / with no hash instead of auto-loading first session
- Docs: document Ollama multi-server fallback, model priority lists, OLLAMA_BACKENDS_FILE

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Make server I/O non-blocking; update docs ...

- Wrap all heavy filesystem ops in asyncio.to_thread: filesystem tool
  (read/write/append/list/find/info/move/delete/query/smart_edit),
  image_view (read_bytes), share_file (shutil.copy2), write_tool
  (write_text, _register_user_tool), session_files (shutil.rmtree,
  iterdir), sessions upload endpoint (sync open/write → to_thread)
- Make delete_session_dir async; update its caller in sessions.py
- docs/config.md: fix wrong defaults (threshold 0.70, keep_recent 8),
  remove phantom SESSION_FILES_TTL_HOURS, add LLM timeouts, DATABASE_URL,
  PUBLIC_URL, Gmail, CONTEXT_SUMMARY_MAX_TOKENS sections
- docs/profiles.md: add missing tool_developer profile to table
- android-client: add WebView remote debugging; remove unused toolbar menu
- Remove stale helper scripts and test files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Use gemma4 cloud model by default

Support Ollama Cloud API key

Add backend documentation ...

10 markdown files covering all backend subsystems: architecture,
agent loop, tool system, sessions, WebSocket protocol, profiles,
memory, config, and API reference. Structured for AI readability
with a single entry point (docs/index.md) and cross-references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>