MCP (Model Context Protocol)

How Navi connects to external MCP servers, how their tools are exposed, and how per-user credentials (BYOK) work.

Configs

Each server lives in its own file mcp_servers.d/<name>.json:

{
  "transport": "streamable_http",
  "url": "https://creds.example.com/mcp",
  "headers": { "Authorization": "Bearer ${NAVI_MCP_GNEXUS_CREDS_TOKEN}" },
  "groups": {
    "creds": ["search_secrets", "get_secret", "reveal_secret"]
  },
  "instructions": "MANDATORY: ...",
  "user_key": { "header": "Authorization", "prefix": "Bearer " }
}

The legacy monolithic mcp_servers.json is auto-migrated to mcp_servers.d/ on first load. Edits take effect on server restart or a reload_tools-triggered McpManager.reload_all().

Heads-up for admin config edits: PUT /admin/agents/mcp-servers round-trips the whole model — write back the config you read, or a user_key slot added by hand is silently erased.

Secrets

mcp_servers.d/*.json is tracked in git, so a credential written into one is in history for good — and GET /admin/mcp/config hands the same value to the admin client. A config therefore never carries a live credential: it names the variable that holds it.

"headers": { "Authorization": "Bearer ${NAVI_MCP_GNTODO_TOKEN}" }

• Only headers and env values are scanned, and only the braced ${NAME} form — a bare $NAME is left alone, so header values that contain $ survive.
• The value comes from the process environment over the project's .env (chmod 600, untracked; environment variable beats the file, as with Settings). A blank NAME= in .env counts as missing. .env is re-read on every (re)connect, so a rotated token is picked up without a restart — the canonical store for the value itself is gnexus-creds (see the platform's secrets rules; deploy/env.template lists the variable names).
• A missing variable fails the connection with ValueError naming the server and the variable, rather than dropping the header — a server may answer an unauthenticated request as an anonymous user instead of returning 401, so silently dropping would fail open. McpManager.load_all isolates a failing server, so this shows up as one broken entry in /mcp/status.
• The explicit escape hatch is ${NAME:-fallback} (fallback may be empty). It is for a setting that tunes a server — a SearXNG URL — where being unset costs one backend instead of every tool the server provides. Writing the fallback down is the point: it is a decision in the config, not an accident of a variable nobody set. A credential never takes this form.
• Resolution happens at transport open (navi/mcp/client.py), next to the project-relative path resolution. This is what keeps every stored and serialised config placeholder-only: save_mcp_servers — reached by create_mcp_server and by PUT /admin/mcp/config — cannot write a secret back into a tracked file, and a GET→PUT round-trip is lossless.

A user key (user_key) is injected first and overwrites the header, so a per-user credential simply replaces the placeholder — no substitution happens.

navi-web settings

The search/browse server has no user key, but it still has runtime settings of its own, and they reach it through the same placeholder mechanism. A stdio server does not inherit the Navi process's environment: the SDK passes a safe allowlist (HOME, PATH, USER, …) plus the config's env block, so a variable that is only in .env is invisible to it unless the config maps it.

Without NAVI_WEB_SEARXNG_URL the server starts straight at the ddgs merge — the engines pinned in search._DDG_BACKENDS (DuckDuckGo, Mojeek, Yahoo, Wikipedia, Grokipedia; Yandex and Brave are excluded on purpose — Yandex by the owner's choice, Brave because brave.com answers nothing from this host). There is no keyed third backend: the Brave Search API refused to issue a key (403 at signup), so the chain is SearXNG → ddgs and nothing else. It is slower and the first thing to be rate-limited. The ${SEARXNG_URL:-} fallback is what keeps a host with no SearXNG working: an unset value costs the backend, not the server's three tools. navi.config.Settings has no search fields: SEARXNG_URL is read here, not by Navi itself.

Two details in the SearXNG request are deliberate. The engines list is pinned (google,duckduckgo) because the instance's own defaults are mostly suspended, and bing is excluded: from a host it cannot match, Bing answers with trending/homepage blocks, and a non-empty list of noise otherwise wins the chain over a working fallback. The language is all, not a fixed tag — a pinned uk-UA made the instance's DuckDuckGo engine return CAPTCHA on every request (kl=ua-uk) and skewed English queries. The same trap exists in ddgs v9: backend takes engine names, and an unknown name does not fail — it silently means "all engines".

Batching is the latency lever

web_search takes a list of up to three queries, web_view a list of up to three URLs; the items run concurrently, so the call costs the slowest item instead of the sum — three pages measured 17.3 s one by one and 1.9 s together, three searches 3.5 s and 0.6 s. The caps live in the server: three items, deduplicated, extras dropped with a note in the output.

The parameter is an array in the schema, and that is the load-bearing part. It used to be str | list[str], which the model sees as anyOf: [string, array]; it took the first branch every time — 20 of 20 recorded web_search calls and 8 of 8 web_view calls carried a scalar, across sessions where three searches on one topic were issued one after another. Prose did not move it: the same recommendation already stood in the parameter description, the tool docstring and the summary line. Annotating the parameter as list[str] with minItems/maxItems and an examples entry removes the choice; a BeforeValidator still folds a bare "query": "…" into a one-item list, so a laxer caller is not refused. The minItems/maxItems ride in json_schema_extra rather than in pydantic's min_length/max_length on purpose: the cap is advisory, and four queries still reach the code that runs three and names the rest.

The model is pointed at the feature three times — by the parameter description and examples in the schema, by summary in mcp_servers.d/navi-web.json, which is the line that reaches the system prompt (instructions carries the long form for tool_manual), and by the persona's parallelism rule, which now separates "several items in one call" from "several calls in one message". The wording is a trigger, not a speed plea: batch when the lookups are already known to be independent, never a follow-up whose query depends on the previous answer.

A per-item deadline is deliberate. One hung backend or page is named in the output rather than erasing what the others already returned, and the tool raises — isError in the journal — only when nothing at all came back.

This is not Navi's parallel_tool_calls: that flag runs a whole batch of independent tool calls concurrently for every tool of a profile, terminal included. The server-side fan-out is scoped to these two tools, and its caps and budgets are enforced where the work happens.

What web_view hands back

The page is rendered in Chromium, then trafilatura picks the main content out by text density and the server renders that fragment as markdown: # headings, fenced code with its own indentation, table rows joined by |. This replaced innerText after a tag kill-list, which could not tell navigation from prose — 462 of the asyncio page's 1018 lines were shorter than 40 characters — collapsed tables into one line per thousand characters (news.ycombinator.com came back as four lines), marked no headings and dropped every link.

Links become a Links on this page appendix rather than [label](url) inline: at most 40, absolute, deduplicated, with self-anchors and hide/vote/login chrome filtered out. Inline links cost 3.5k characters on the docs page alone, and trafilatura drops the # from any heading that carries an anchor when they are on. Text past 20k characters is cut at a paragraph boundary, and the note names the sections that did not fit.

A page whose content is its thread — a forum, a Q&A, a code review — comes back as a bare header when comments are excluded, so when the extractor keeps less than a quarter of the page's visible text it runs again with comments. If it still finds nothing worth sending, the tool falls back to the visible-text read; metadata.extractor says which path answered (trafilatura or innerText), and metadata.links counts the appendix.

The server's own venv

Each in-repo server runs from mcp-servers/<name>/.venv, which is separate from Navi's and is what mcp_servers.d/<name>.json points command at: ./mcp-servers/navi-web/.venv/bin/python -m app.mcp_server with its working directory there. Navi's venv does not carry those dependencies, so a dependency added to mcp-servers/<name>/pyproject.toml needs an install into that venv — deploy/sync-mcp-venvs.sh does it for every server, and deploy/install.sh calls it. It is stamped by a hash of each pyproject.toml, so an unchanged server is skipped; run it on its own after a git pull (see deploy/UPDATE.md).

bash deploy/sync-mcp-venvs.sh    # all servers, no-op for unchanged ones
sudo systemctl restart navi

For a one-off install outside a deploy checkout, the same thing by hand:

VIRTUAL_ENV=mcp-servers/navi-web/.venv uv pip install -e mcp-servers/navi-web

Skipping it is silent: the module does not import, McpManager cannot connect, and build_tool_list reports the server's tools as missing — the model simply never sees them. /admin/mcp/status and the MCP server %r failed to connect line in the journal are where it shows up.

Runtime

• McpManager (navi/mcp/manager.py) holds one McpClient per server, reconnects dead ones in a 30 s health-check loop, and is the single call_tool gateway. Connection headers/env are fixed at transport open (navi/mcp/client.py) — a shared client cannot switch credentials per call.
• Tool names follow mcp__<server>__<tool>; profiles select servers via tools.agent.mcp / tools.subagent.mcp group maps. Registered MCP tools are process-wide, regardless of which profile uses them.
• McpTool.execute forwards ctx.user_id (falling back to the current_user_id ContextVar) into manager.call_tool(..., user_id=...).

Per-user keys (BYOK)

A config with a user_key slot lets each user bring their own credential:

"user_key": { "header": "Authorization", "prefix": "Bearer " }

or, for stdio servers:

"user_key": { "env": "API_KEY" }

Exactly one destination (validator navi/mcp/config.py::McpUserKey); a header is only valid on HTTP transports, an env only on stdio.

Resolution flow (McpManager._client_for), given the caller's user_id and role:

1. The config has no user_key slot → the default shared client. The resolver and the database are never touched.
2. The config has a slot and the caller has a saved key → a per-user McpClient clone built from McpServerConfig.with_user_key(key) (a deep copy with the key injected into the header/env destination). Clones are cached per key snapshot: a changed key transparently rebuilds the client.
3. The config has a slot and the caller has no key:
   • role == "admin" → the default shared client with the credential from the config file. An admin's calls behave byte-for-byte as before BYOK existed — the shared credential is theirs.
   • any other role → refused, with a RuntimeError naming the server and pointing at Settings → MCP keys. There is no quiet fallback to the owner's credential: that fallback is what made six of the owner's servers reachable by every account.

KeyResolver.resolve reads the user's key (30 s in-memory TTL; every failure — DB error, missing NAVI_AUTH_ENCRYPTION_KEY — logs a warning and resolves to None, i.e. "no key", which for a non-admin is a refusal rather than a fallback).

Visibility — a gated server is absent, not merely refused

A server the manager would refuse must not appear in the tool list either, or the model sees a tool it can never call, retries it, and records a broken tool instead of a boundary. build_tool_list is synchronous while the key store is not, so the allowed set is computed once per run and only read afterwards:

• Agent._apply_mcp_scope (called at the top of run_stream) asks McpManager.visible_servers(role, user_id) and puts the answer in the current_allowed_mcp_servers ContextVar (navi/tools/_internal/base.py). It returns two tokens and both are reset on the stop and max-iterations exit paths; the normal-completion path returns without resetting, which is harmless because every run sets the variables again before anything reads them.
• build_tool_list (navi/core/tool_utils.py) skips any server outside that set.
• role == "admin", or no MCP layer at all → the ContextVar is None, meaning "no restriction", and nothing changes for the owner.
• Fail closed: if visible_servers raises, the set is empty rather than unrestricted.

Why a server is absent is part of the prompt. Absence alone left the model knowing the server existed, holding none of its tools, and having nothing to tell the user. _apply_mcp_scope therefore also derives the withheld servers — byok_servers() − allowed, the complement of what it kept, not a second read of the key store that could disagree with the first — and puts them in current_keyless_mcp_servers. ContextBuilder._mcp_context_msg names each of them in the [MCP servers] system message, followed by the reason and a pointer at Settings → MCP keys, so the assistant can explain the missing key itself instead of silently lacking the tools. The variable defaults to empty, so a run that never set it — a peer run out of a fresh API task — claims nothing.

Saving a key in the panel takes effect on the next turn — the resolver's on_change callback clears the manager's visibility cache along with the affected clients.

Cache bounds: hard LRU cap of 32 per-user clients; per-user clients idle longer than 30 minutes are reaped by the health-check tick. Per-user clients are outside the health-check's reconnect logic — they reconnect lazily with backoff when next called. disconnect_all (reload/shutdown) tears user clients down as well.

Key changes (PUT/DELETE) fire the resolver's on_change callback, which drops affected per-user clients and the visibility cache immediately.

Storage

mcp_user_keys table (postgres, navi/mcp/_ddl.py), Fernet-encrypted via NAVI_AUTH_ENCRYPTION_KEY (navi/mcp/keystore.py::McpKeyStore). One row per (user_id, server_name); deleting a user cascades. Keys never leave the backend through logs or REST responses.

Anon mode (NAVI_AUTH_ENABLED=false) is one shared local user — its saved key effectively becomes a system-level override, and requires NAVI_AUTH_ENCRYPTION_KEY to be set (the Fernet encryptor refuses an empty key).

REST

Settings UI surface — see api.md: GET /mcp-keys (every profile-referenced server, keyed ones flagged, no key values), PUT /mcp-keys/{server} (save/invalidate), DELETE /mcp-keys/{server} (reset to default). The webclient lists all of them: rows for servers without a user_key slot are read-only context, and only slotted servers get a key input. Servers that declare an Authorization-style header carry a shared default in the config — as a ${...} placeholder, not a literal (see Secrets) — which an admin keeps using when they have no personal key, and which an ordinary user cannot reach at all.