memory: a fact written under a new spelling of its key merges into the one that survives ...

The extractor invents a key every time it meets the same fact — `language`,
`preferred_language`, `ui_language` — and the upsert keys on the exact string, so
the base grew several spellings of each durable fact. On prod, 24 facts across 11
clusters share a key once noise prefixes are dropped, 10 of them under the exact
same value.

upsert_fact(dedupe=True) now compares the *tokens* of the normalized key — set
equality, not overlap, and never the value: `ui_language` and `language` are one
key while `dark` and `dark_ui` are two. The row already holding the writer's exact
key wins; otherwise the incoming one does. Losers are marked, not deleted: each
keeps its key and its value and gains `merged_into`, so the merge is reversible
with one UPDATE. Off by default, so the memory tool still writes exactly the row
it asked for.

Every read path filters `merged_into IS NULL`, and the mark is reachable only
where it is meant to be — get_all_facts/fact_count gained include_merged, and
GET /admin/memory?include_merged=true exposes it. search_facts deliberately has
no such flag: the injection path must not be able to see a hidden row. Forgetting
a key drops its hidden aliases first, so none is left pointing at nothing.

Three fixes that fell out of the same read:

- expires_at was overwritten with EXCLUDED.expires_at on every upsert, and the
  extractor always passes None — so it silently cleared any expiry the agent had
  set. Both DO UPDATE clauses now COALESCE.
- The ILIKE fallback was the one read path that served expired facts.
- The `list` action promised "all stored facts" and returned categories. The code
  was right and the description was wrong; the description now says so.

The extraction prompt stopped allowing infrastructure — servers, hosts, addresses,
running services — while the agent and the planner already forbade it; that
loophole is where 423 technical facts came from. It also forbids secrets outright,
and asks for stable snake_case keys. Expiry is now documented as being for facts
with a known end, not for facts you merely doubt.

Facts reach the prompt with their age ("updated 34d ago") alongside confidence:
nothing expires on a clock, so the injection says how old a fact is and lets the
agent weigh it.

Audit and fix docs/ for accuracy against current codebase ...

Package A (API + WebSocket + Sessions):
- api.md: add missing endpoints (health/embed, auth/status, agents/prompts,
  agents/mcp_servers, sessions/content, 7 admin endpoints). Fix response
  schemas, auth requirements, query params. Update path formats.
- websocket.md: fix planning_status.phase type (int 1|2|3), add metadata
  to tool_call, message_index to stream_end, context_tokens/max_context_tokens
  to context_compressed.
- sessions.md: fix context_compression_threshold default (0.80→0.70),
  add list_page method, add metadata to Message flags.

Package B (Config + Profiles + Tools + Agent + Memory):
- config.md: remove stale ANTHROPIC_API_KEY description, add 12 missing
  env vars (embedding, web search, terminal_user_allowed_commands,
  context_providers_dir, gnauth fields, output_reserve_tokens).
  Fix OPENAI_BASE_URL type.
- profiles.md: fix max_iterations default (20→10), add top_k/top_p/num_thread,
  is_admin_only, context_providers, mcp_servers, subagent_think_enabled.
  Clarify subagent_tools fallback and step_validation_enabled.
- tools.md: replace 3 separate memory tools with unified `memory`, add
  `mcp_status`.
- agent.md: fix run_ephemeral return type (str→tuple), clarify DB reads.
- memory.md: fix search_facts/get_all_facts param order, fix MemoryStore
  init description.

236 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Update docs for admin pagination, search, sorting and memory all_users ...

- docs/api.md:
  - GET /admin/sessions: document pagination (limit/offset), search,
    sort_by and sort_order query params; switch response to {total,
    limit, offset, items} envelope.
  - GET /admin/memory: same pagination/sort docs, {total, limit,
    offset, items} envelope.

- docs/memory.md:
  - get_all_facts signature now includes offset, search, sort_by,
    sort_order and all_users parameters.

- docs/sessions.md:
  - Document count_all and search_list store methods.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document multi-user auth system ...

- New docs/auth.md — comprehensive auth guide: OAuth flow, roles/permissions,
  webhooks, env vars, setup instructions, API reference, WebSocket auth,
  testing with mocked auth
- Update docs/config.md — add gnexus-auth OAuth env vars table
- Update docs/api.md — add /auth/*, /admin/*, /webhooks/gnexus-auth endpoints
- Update docs/index.md — add auth.md to file map
- Update docs/sessions.md — document user_id column and list_all filtering
- Update docs/memory.md — document user_id scoping in facts and summaries

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture fixes batch — NaN validation, ILIKE indexes, prompt cache, N+1 batching ...

- _vector_to_str: reject NaN/Inf via math.isfinite() to avoid invalid pgvector syntax
- memory DDL: add pg_trgm + GIN trigram indexes on category/key/value for fast ILIKE fallback
- _build_system_prompt: cache per-profile to avoid rebuilding every iteration
- backfill_embeddings: batch UPDATEs via executemany instead of N+1 loop

No new Python deps; pg_trgm is a PostgreSQL extension auto-created on startup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up SQLite references in docs and prompts after removal ...

- docs/index.md: PostgreSQL via asyncpg
- docs/memory.md: drop aiosqlite fallback mention
- docs/sessions.md: replace SqliteSessionStore with PgSessionStore docs
- webclient/docs/architecture.md: PostgreSQL only
- navi/profiles/tool_developer/system_prompt.txt: remove aiosqlite from available imports
- .gitignore: add .codex and *.db
- No remaining sqlite/aiosqlite/db_path references in codebase

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Update memory docs to reflect pgvector + dedicated embedding backend ...

- Add dedicated embedding backend section (.env variables)
- Add backfill_embeddings script documentation
- Update storage methods: upsert_fact generates embeddings, search_facts
  uses vector search with cosine distance fallback to ILIKE
- Update extractor process: tool calls/results in transcript, source/confidence
- Replace memory_search/memory_forget references with unified memory tool

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document pgvector migration in memory system ...

- Add PostgreSQL vs SQLite comparison table

- Document migrate_pgvector.py usage

- Update storage section to mention asyncpg + aiosqlite

Add backend documentation ...

10 markdown files covering all backend subsystems: architecture,
agent loop, tool system, sessions, WebSocket protocol, profiles,
memory, config, and API reference. Structured for AI readability
with a single entry point (docs/index.md) and cross-references.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>