Add webclient unit tests (Vitest) — 47 tests ...

- Vitest + @vue/test-utils + happy-dom setup
- api/index.test.js: 8 tests for fetch wrapper, verbs, errors, FormData
- stores/chat.test.js: 23 tests for buildMessageList, WS handlers, load/clear
- stores/sessions.test.js: 6 tests for fetch, create, delete, pin, preview
- stores/profiles.test.js: 3 tests for fetch, auto-select, lookup
- composables/useWebSocket.test.js: 7 tests with MockWebSocket
- Export buildMessageList from chat store so tests can verify message parsing
- Update docs/testing.md with webclient stack, layout, and commands

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add 62 tests across all planned phases + fix integration flakiness ...

- Phase 3: 19 API route integration tests (health, agents, sessions, messages)
- Phase 3: 7 WebSocket integration tests (connect, send, replay, invalid, stop)
- Phase 4: 9 agent tests (_check_context_size, _iter_stream_guarded)
- Phase 4: 5 planning tests (_parse_plan_steps)
- Phase 5: 22 tool tests (filesystem 13, code_exec 5, terminal 4)
- Fix flaky integration tests by patching module-level singletons
  (_session_store, _registries, _workers) instead of getter functions,
  because FastAPI Depends() captures the original function at import time.
- Update docs/testing.md coverage table (150 total tests)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Phase 2 — Memory store + extractor unit tests ...

- tests/conftest_factory.py: add FakeRecord, FakeConnection, FakePool, make_store_with_pool
- tests/unit/memory/test_store.py: 18 tests for upsert, vector/ILIKE search, delete, list, count, summary, session state, backfill
- tests/unit/memory/test_extractor.py: 11 tests for JSON extraction, invalid JSON, empty response, tool result truncation, summary regeneration
- docs/testing.md: update coverage status, add FakePool usage example

88 total tests passing (59 + 29).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Bootstrap test suite — Phase 1 unit tests ...

- docs/testing.md: testing strategy, mock strategy, phase breakdown
- tests/conftest.py: autouse fixture to reset navi.config.settings per test
- tests/conftest_factory.py: FakeLLMBackend, FakeTool, make_profile, make_registry helpers
- tests/unit/core/test_events.py: wire serialization for all 15 event dataclasses
- tests/unit/core/test_compressor.py: should_compress, partition_messages, format_for_summary, compress_context
- tests/unit/core/test_registry.py: ToolRegistry, ProfileRegistry, BackendRegistry
- tests/unit/core/test_context_builder.py: system prompt caching, persona injection, goal anchor, iteration budget
- tests/unit/profiles/test_base.py: Pydantic model coercion, defaults, extra fields
- navi/core/context_builder.py: use module-level `import navi.config` instead of `from navi.config import settings` so tests can swap the singleton

59 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Split memory/store.py into focused mixins ...

- _ddl.py: table creation (conditional on pgvector/pg_trgm)
- _embeddings.py: EmbeddingMixin — pgvector checks, embed generation, backfill
- _facts.py: FactMixin — upsert, search (vector + ILIKE), delete, list, count
- _summary.py: SummaryMixin — get/set conversation summary
- _session_state.py: SessionStateMixin — extraction tracking per session
- store.py: reduced to ~60-line facade composing all mixins

No external API changes — MemoryStore remains the single interface.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extract PlanningEngine, ContextBuilder, ToolExecutor from agent.py ...

- navi/core/planning.py: new 3-phase planning pipeline (~390 lines)
- navi/core/context_builder.py: system prompt caching, memory/context injection, goal anchoring (~160 lines)
- navi/core/tool_executor.py: tool execution with middleware chain (~150 lines)
- navi/core/agent.py: reduced from ~1420 to ~770 lines; delegates to extracted classes

All compilation verified.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Update deployment docs — PostgreSQL-only, new .env vars, pg_trgm notes ...

- README.md: remove SQLite, add PostgreSQL setup (Ubuntu/Arch/Docker),
  add pg_trgm notes, update .env example
- docs/config.md: remove DB_PATH, add OLLAMA_REQUEST_TIMEOUT / OPENAI_MODEL /
  OPENAI_BASE_URL, update example .env
- .env.example: rewrite with all current variables and comments

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture fixes batch — NaN validation, ILIKE indexes, prompt cache, N+1 batching ...

- _vector_to_str: reject NaN/Inf via math.isfinite() to avoid invalid pgvector syntax
- memory DDL: add pg_trgm + GIN trigram indexes on category/key/value for fast ILIKE fallback
- _build_system_prompt: cache per-profile to avoid rebuilding every iteration
- backfill_embeddings: batch UPDATEs via executemany instead of N+1 loop

No new Python deps; pg_trgm is a PostgreSQL extension auto-created on startup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix content_store._get_db_pool after lazy deps refactor ...

_get_db_pool() imported _memory_store directly from deps.py, but it
is now None at import time (lazy init). Call get_memory_store() instead
to trigger initialization before accessing _get_pool().

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture cleanup: old_webclient, SSH shutdown, todo decoupling, ContextVar reset ...

- Remove old_webclient directory and /static mount from main.py
- Add shutdown handler in main.py to close all SSH pooled connections
- Decouple agent.py from todo module internals:
  - Add public API to navi/tools/todo.py: get_task_snapshot, get_failed_steps,
    get_progress_message, set_tasks, render_todo_lines
  - Replace all direct _plans/_STATUS_ICON/_Task imports in agent.py with API calls
- Wrap run_ephemeral in try/finally to restore _sid_var and _model_var after subagent

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up SQLite references in docs and prompts after removal ...

- docs/index.md: PostgreSQL via asyncpg
- docs/memory.md: drop aiosqlite fallback mention
- docs/sessions.md: replace SqliteSessionStore with PgSessionStore docs
- webclient/docs/architecture.md: PostgreSQL only
- navi/profiles/tool_developer/system_prompt.txt: remove aiosqlite from available imports
- .gitignore: add .codex and *.db
- No remaining sqlite/aiosqlite/db_path references in codebase

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Remove SQLite legacy support ...

SQLite is no longer supported; PostgreSQL is now required.

- Delete navi/core/sqlite_session_store.py
- Delete navi/memory/sqlite_store.py
- Remove SqliteSessionStore from navi/core/__init__.py exports
- deps.py: drop SQLite fallback, raise RuntimeError if DATABASE_URL missing
- config.py: remove db_path setting
- pyproject.toml & requirements.txt: drop aiosqlite dependency
- .gitignore: remove navi.db entry
- tech_debt_review_2026-04-29.md: mark #8 as REMOVED

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stability fixes batch — tech debt review 2026-04-29 ...

Critical:
- Concurrent WS run race guard (#1)
- Tool task cancellation on generator teardown (#2)
- StopAsyncIteration kills fallback chain (#3)
- Session loading race with _lastLoadId guard (#4)
- ContentCard .match() crash on non-string result (#5)
- Image data type guard in buildMessageList (#6)

High:
- Cap WS replay buffer at 500 events (#7)
- Deduplicate memory extraction task with asyncio.Lock (#9)
- TTL-based fallback blacklisting (5 min) (#10)
- Subagent tool exception isolation (#11)
- Inline image size/count validation on WS (#12)
- Clean up orphaned file on DB insert failure (#13)
- Deep watch streamingMsg for auto-scroll (#14)
- WS_SCHEME wss:// support for HTTPS (#15)
- Sending guard against duplicate message sends (#16)
- Global unhandledrejection listener in API layer (#17)

Medium:
- Cap planning_logs at 20 entries (#22)
- Store cleanup_loop task reference (#23)
- BaseException → Exception in _run_with_sentinel (#24)
- Propagate SystemExit in agent loop (#25)
- Configurable output_reserve_tokens (#26)
- Always reloadSession on session_sync (#30)
- FIFO queue for confirm dialogs (#31)
- Reset body.overflow on ImageLightbox unmount (#32)
- try/finally in fallback copy (#33)
- _isConnecting guard in WS send() (#34)

Low:
- Lazy-init deps.py singletons (#36)
- Replace __import__ with direct imports (#38)
- Preserve token count 0 in ollama.py (#39)
- Clear orphaned streamingMsg on reconnect reload (#43)
- Escape single quote in UserMessage (#44)
- Polyfill-free findLast replacement (#48)
- Match <table> tags with attributes in markdown (#49)
- Attach copy buttons only when msg.done (#50)
- Fix hasMeta falsy-0 bug (#53)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add SVG/HTML/XML tag formatting rule to persona ...

Prevents model from generating doubled/escaped tags like <<svgsvg>
by explicitly instructing single-angle-bracket markup in code output.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix system prompt leakage into chat history; polish content cards ...

Backend:
- websocket.py + agent.py: separate user-visible display_message from
  LLM user_message. System hints (image/file attachments) no longer leak
  into session.messages and appear after page reload.
- main.py: add ensure_tables() on startup so session_content table is
  created before first publish.
- profiles: add kimi-k2.6:cloud to all model lists as fallback.

Frontend:
- ContentCard.vue: remove border-radius, add scrollbar styles, fix
  metadata fallback parsing so cards survive page reload.
- content-viewers/*.html: add matching scrollbar styles.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content hosting system with inline viewers ...

Backend:
- Add navi/content/ directory for published files
- Add content_store.py with publish/list/delete/cleanup functions
- Add content_publish tool for publishing files as viewable content
- Add /content static file mount in main.py
- Add /content-viewers mount for viewer pages
- Extend ToolEvent with metadata field
- Forward metadata through websocket tool_call events
- Update Agent to include metadata in ToolEvent

Frontend:
- Add ContentCard.vue component for displaying published content
- Add viewer pages: stl.html (Three.js), svg.html, html.html, pdf.html
- Update AssistantMessage.vue to render ContentCard for content_publish
- Update chat store to preserve metadata in tool cards
- Update websocket protocol docs with metadata field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Update memory docs to reflect pgvector + dedicated embedding backend ...

- Add dedicated embedding backend section (.env variables)
- Add backfill_embeddings script documentation
- Update storage methods: upsert_fact generates embeddings, search_facts
  uses vector search with cosine distance fallback to ILIKE
- Update extractor process: tool calls/results in transcript, source/confidence
- Replace memory_search/memory_forget references with unified memory tool

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add dedicated CPU embedding server for memory backfill ...

- Install Ollama CPU-only on 192.168.1.168 server
- Pull nomic-embed-text:latest on server
- Create systemd service ollama-embed.service (0.0.0.0:11434)
- Add embedding_ollama_host / embedding_ollama_api_key to config.py
- Update deps.py to build separate embedding backend when host configured
- Update backfill_embeddings.py to use dedicated embedding backend
- Add _generate_embeddings batch helper and backfill_embeddings to store.py
- Backfilled 119 existing facts with embeddings

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Enrich memory extractor with tool calls/results in transcript ...

- _EXTRACT_SYSTEM now explains 4 transcript entry types and instructs
  LLM to trust tool results over chat, return source/source_context
- _extract_facts builds tool_call_map, appends [Tool call] and
  [Tool result] lines with truncation (500/200 chars)
- Transcript capped at 12k chars (head+tail, drop middle)
- Parse source/source_context from LLM response; map confidence:
  tool_call/auto_discovery=95, user_explicit=90, default=70
- Add TODO comment about deferred semantic deduplication

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Document pgvector migration in memory system ...

- Add PostgreSQL vs SQLite comparison table

- Document migrate_pgvector.py usage

- Update storage section to mention asyncpg + aiosqlite

Add pgvector migration script for memory_facts ...

- ALTER TABLE memory_facts: embedding, source, confidence, expires_at, source_context

- CREATE INDEX: hnsw(embedding), expires, source+category

- Safe to run multiple times (IF NOT EXISTS)

- Reads DATABASE_URL from settings

Wire pgvector semantic search into memory system ...

- Add vector(768) column + HNSW index to memory_facts

- Add LLMBackend.embed() with Ollama + fallback implementation

- MemoryStore: cosine-distance search with ILIKE fallback

- New memory tool params: source, confidence, expires_days, source_context

- Update extractor, sqlite_store, deps wiring

- Add pgvector to requirements

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Slim eval rubric to 3 levels with one reference per axis ...

Five anchors per axis (10/30/50/75/100, even after the earlier shift)
were both redundant and amplified the model's snap-to-round-numbers
prior. Cut to three level descriptions per axis (weak / typical /
strong) with a single non-round reference score (53) on `typical`.
Re-state the scale as open-ended with no upper bound to make the
"future Navi may exceed past ceilings" intent explicit.

- rubric_v1.yaml: anchors → levels (5 → 3 per axis), reference score
  53 only on typical, scale framed as fully open-ended.
- judge.py: render_rubric_for_prompt walks the new `levels` shape and
  surfaces the reference score only when present.
- expert prompts (strict_critic, pragmatist, tech_lead): drop the
  example output blocks (their concrete numbers were misleading the
  judges), rewrite the scale paragraph for the new structure.
- schema.py: docstring no longer pins ">100" as the open-scale marker.

User intent: dynamics, not absolute scores. Weekly aggregates over
three averaged experts smooth individual snap-to-5 into continuous
trends; the rubric is a calibration aid, not a grading ceiling.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fight rubric-anchor snapping in eval judges ...

Judges were clustering scores onto the rubric's round anchor values
(30, 50, 75, 100) instead of producing fine-grained continuous scores,
which made small differences between sessions invisible.

- rubric_v1.yaml: shift anchors off round numbers (33/51/77/102),
  reframe the scale as open-ended integers ≥ 0 with illustrative level
  descriptions, and tell judges explicitly not to round to anchors.
- expert prompts (strict_critic, pragmatist, tech_lead): mirror the
  scale framing and add an example output with deliberately non-round
  scores between anchors.
- judge.py: bump expert temperature 0.2 → 0.5 so the judges produce
  more varied, non-deterministic scores.

Old v1 evaluations in the DB are not comparable to new ones; user
intends to wipe and re-run from scratch, so versions are not bumped.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stop image_view hallucinations on inline-attached images ...

The model was inventing fake paths/URLs (e.g. files.oaiusercontent.com,
/home/ubuntu/navi-1/input_file_0.png) and calling image_view on them
when the user attached an image directly in chat — the image was
already in the multimodal context, but the tool description and lack
of a signal pushed the model to "load" it anyway.

- websocket.py: when a user message has inline images, append a brief
  note that they are already in context.
- image_view.py: soften the description — keep proactive use for paths
  and URLs the model genuinely cannot see, but tell it inline images
  don't need this tool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rewrite eval_system.md as user guide; preserve original spec as eval_system_design.md ...

- docs/eval_system.md: replaced stale spec with current user guide
  covering UI tabs, CLI, rubric, experts, versioning, API
- docs/eval_system_design.md: preserved original design spec for
  reference

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add eval system Phase 5 — debug UI ...

Self-contained SPA at /debug/eval (route already wired in 8e0eed6).
Single index.html in the existing debug/ style — vanilla JS, embedded
CSS, no framework, no build step. Four tabs:

- Sessions — filterable table (profile / status / limit), eval status
  pill, headline avg scores, click-through to detail
- Detail — session metadata + every stored eval run, axes laid out as
  axis × expert grids with inline averages, expert comments, button to
  re-evaluate this single session
- Stats — weekly per-axis means table, optional complexity-bucket split
- Run — form to trigger any scope (unevaluated / single / all), live
  status panel polling /eval/run/{id} every 2.5s, run history with
  click-to-attach

Hash routing: #detail/<session_id> deep-links to a session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

changed llm

Add eval system Phase 4 — read endpoints and background runner ...

REST surface for the debug UI:
- GET /eval/sessions  — overview list with eval status / latest avg /
  feedback counts (single SQL: sessions ⨝ feedback ⨝ latest run)
- GET /eval/sessions/{id} — session detail with all evaluations
- GET /eval/stats — weekly per-axis means; optional complexity-bucket split
- POST /eval/run — fire-and-forget background eval, returns run_id
- GET /eval/run/{id}, GET /eval/runs — poll progress and history

Pulled the runner loop out of cli into runner.py so both the CLI and
the REST endpoint share the same loop. State for in-flight runs lives
in an in-memory registry (single-process, cleared on restart).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>