Initialize embedding backend before health check

Clarify share file publishing boundaries

Docs audit: fix outdated references (PostgreSQL, Vue client, profiles, tools) ...

- architecture.md: SQLite → PostgreSQL + pgvector
- README.md: vanilla JS → Vue 3 + Pinia, remove smart_home, update temps
- profiles.md: remove smart_home, add discuss, update temperatures
- tools.md: add missing built-in tools, update user tools list
- sessions.md: fix outdated references
- NAVI.md: sync with current stack
- persona.txt: update self-extension and content_publish guidance
- content_publish.md manual: reflect session_dir-based no-copy design
- content_store.py & content_publish.py: inline with docs

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content_publish manual and improve tool description ...

- Create manuals/content_publish.md with full usage guide,
  internal mechanics, supported types, and best practices
- Update ContentPublishTool description with usage guidelines
  and examples in the tool schema
- All tests pass (150 backend + 47 webclient)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Rework content_publish: files live in session_dir, no copying ...

content_publish now registers existing session files for inline viewing
instead of copying them to navi/content/<uuid>/. This makes files
editable by the agent and immediately visible to the user via the
stable URL /sessions/{id}/files/{filename}.

- content_store.publish() → registers metadata, no shutil.copy2
- content_store.delete_content() → removes DB record only (file stays
  in session_dir; cleaned up by session lifecycle)
- content_store.cleanup_old() → deletes stale DB records
- ContentPublishTool:
  - accepts 'filename' instead of 'path'
  - looks up file in uploads/sessions/<id>/
  - returns clear error if file not found with instructions
  - includes full local path in output for future editing
- Added unique index on (session_id, filename) for idempotent re-publish
- Removed /content static mount from main.py (endpoint still works
  for legacy files; new files use /sessions/{id}/files/)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Bootstrap test suite — Phase 1 unit tests ...

- docs/testing.md: testing strategy, mock strategy, phase breakdown
- tests/conftest.py: autouse fixture to reset navi.config.settings per test
- tests/conftest_factory.py: FakeLLMBackend, FakeTool, make_profile, make_registry helpers
- tests/unit/core/test_events.py: wire serialization for all 15 event dataclasses
- tests/unit/core/test_compressor.py: should_compress, partition_messages, format_for_summary, compress_context
- tests/unit/core/test_registry.py: ToolRegistry, ProfileRegistry, BackendRegistry
- tests/unit/core/test_context_builder.py: system prompt caching, persona injection, goal anchor, iteration budget
- tests/unit/profiles/test_base.py: Pydantic model coercion, defaults, extra fields
- navi/core/context_builder.py: use module-level `import navi.config` instead of `from navi.config import settings` so tests can swap the singleton

59 tests passing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Split memory/store.py into focused mixins ...

- _ddl.py: table creation (conditional on pgvector/pg_trgm)
- _embeddings.py: EmbeddingMixin — pgvector checks, embed generation, backfill
- _facts.py: FactMixin — upsert, search (vector + ILIKE), delete, list, count
- _summary.py: SummaryMixin — get/set conversation summary
- _session_state.py: SessionStateMixin — extraction tracking per session
- store.py: reduced to ~60-line facade composing all mixins

No external API changes — MemoryStore remains the single interface.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Extract PlanningEngine, ContextBuilder, ToolExecutor from agent.py ...

- navi/core/planning.py: new 3-phase planning pipeline (~390 lines)
- navi/core/context_builder.py: system prompt caching, memory/context injection, goal anchoring (~160 lines)
- navi/core/tool_executor.py: tool execution with middleware chain (~150 lines)
- navi/core/agent.py: reduced from ~1420 to ~770 lines; delegates to extracted classes

All compilation verified.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture extensibility — event bus, middleware, auto-discovery, Pydantic profiles ...

- EventBus: async pub/sub for AgentEvents, WebSocket subscribes instead of direct yield
- Declarative serialization: AgentEvent.to_wire() on all event types
- Auto-discovery for LLM backends (_discover_backends) and workers (scan navi/workers/*.py)
- AgentProfile: Pydantic BaseModel with extra='allow', @field_validator for model coercion
- Tool middleware chain: pre/post execute hooks via ToolRegistry.add_middleware()
- LoggingMiddleware: built-in, logs every tool call
- Fix pg_trgm DDL: conditional GIN indexes via DO $$ block, no CREATE EXTENSION
- New files: event_bus.py, middleware.py, logging_middleware.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture fixes batch — NaN validation, ILIKE indexes, prompt cache, N+1 batching ...

- _vector_to_str: reject NaN/Inf via math.isfinite() to avoid invalid pgvector syntax
- memory DDL: add pg_trgm + GIN trigram indexes on category/key/value for fast ILIKE fallback
- _build_system_prompt: cache per-profile to avoid rebuilding every iteration
- backfill_embeddings: batch UPDATEs via executemany instead of N+1 loop

No new Python deps; pg_trgm is a PostgreSQL extension auto-created on startup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix content_store._get_db_pool after lazy deps refactor ...

_get_db_pool() imported _memory_store directly from deps.py, but it
is now None at import time (lazy init). Call get_memory_store() instead
to trigger initialization before accessing _get_pool().

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Architecture cleanup: old_webclient, SSH shutdown, todo decoupling, ContextVar reset ...

- Remove old_webclient directory and /static mount from main.py
- Add shutdown handler in main.py to close all SSH pooled connections
- Decouple agent.py from todo module internals:
  - Add public API to navi/tools/todo.py: get_task_snapshot, get_failed_steps,
    get_progress_message, set_tasks, render_todo_lines
  - Replace all direct _plans/_STATUS_ICON/_Task imports in agent.py with API calls
- Wrap run_ephemeral in try/finally to restore _sid_var and _model_var after subagent

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Clean up SQLite references in docs and prompts after removal ...

- docs/index.md: PostgreSQL via asyncpg
- docs/memory.md: drop aiosqlite fallback mention
- docs/sessions.md: replace SqliteSessionStore with PgSessionStore docs
- webclient/docs/architecture.md: PostgreSQL only
- navi/profiles/tool_developer/system_prompt.txt: remove aiosqlite from available imports
- .gitignore: add .codex and *.db
- No remaining sqlite/aiosqlite/db_path references in codebase

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Remove SQLite legacy support ...

SQLite is no longer supported; PostgreSQL is now required.

- Delete navi/core/sqlite_session_store.py
- Delete navi/memory/sqlite_store.py
- Remove SqliteSessionStore from navi/core/__init__.py exports
- deps.py: drop SQLite fallback, raise RuntimeError if DATABASE_URL missing
- config.py: remove db_path setting
- pyproject.toml & requirements.txt: drop aiosqlite dependency
- .gitignore: remove navi.db entry
- tech_debt_review_2026-04-29.md: mark #8 as REMOVED

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stability fixes batch — tech debt review 2026-04-29 ...

Critical:
- Concurrent WS run race guard (#1)
- Tool task cancellation on generator teardown (#2)
- StopAsyncIteration kills fallback chain (#3)
- Session loading race with _lastLoadId guard (#4)
- ContentCard .match() crash on non-string result (#5)
- Image data type guard in buildMessageList (#6)

High:
- Cap WS replay buffer at 500 events (#7)
- Deduplicate memory extraction task with asyncio.Lock (#9)
- TTL-based fallback blacklisting (5 min) (#10)
- Subagent tool exception isolation (#11)
- Inline image size/count validation on WS (#12)
- Clean up orphaned file on DB insert failure (#13)
- Deep watch streamingMsg for auto-scroll (#14)
- WS_SCHEME wss:// support for HTTPS (#15)
- Sending guard against duplicate message sends (#16)
- Global unhandledrejection listener in API layer (#17)

Medium:
- Cap planning_logs at 20 entries (#22)
- Store cleanup_loop task reference (#23)
- BaseException → Exception in _run_with_sentinel (#24)
- Propagate SystemExit in agent loop (#25)
- Configurable output_reserve_tokens (#26)
- Always reloadSession on session_sync (#30)
- FIFO queue for confirm dialogs (#31)
- Reset body.overflow on ImageLightbox unmount (#32)
- try/finally in fallback copy (#33)
- _isConnecting guard in WS send() (#34)

Low:
- Lazy-init deps.py singletons (#36)
- Replace __import__ with direct imports (#38)
- Preserve token count 0 in ollama.py (#39)
- Clear orphaned streamingMsg on reconnect reload (#43)
- Escape single quote in UserMessage (#44)
- Polyfill-free findLast replacement (#48)
- Match <table> tags with attributes in markdown (#49)
- Attach copy buttons only when msg.done (#50)
- Fix hasMeta falsy-0 bug (#53)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Fix system prompt leakage into chat history; polish content cards ...

Backend:
- websocket.py + agent.py: separate user-visible display_message from
  LLM user_message. System hints (image/file attachments) no longer leak
  into session.messages and appear after page reload.
- main.py: add ensure_tables() on startup so session_content table is
  created before first publish.
- profiles: add kimi-k2.6:cloud to all model lists as fallback.

Frontend:
- ContentCard.vue: remove border-radius, add scrollbar styles, fix
  metadata fallback parsing so cards survive page reload.
- content-viewers/*.html: add matching scrollbar styles.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add content hosting system with inline viewers ...

Backend:
- Add navi/content/ directory for published files
- Add content_store.py with publish/list/delete/cleanup functions
- Add content_publish tool for publishing files as viewable content
- Add /content static file mount in main.py
- Add /content-viewers mount for viewer pages
- Extend ToolEvent with metadata field
- Forward metadata through websocket tool_call events
- Update Agent to include metadata in ToolEvent

Frontend:
- Add ContentCard.vue component for displaying published content
- Add viewer pages: stl.html (Three.js), svg.html, html.html, pdf.html
- Update AssistantMessage.vue to render ContentCard for content_publish
- Update chat store to preserve metadata in tool cards
- Update websocket protocol docs with metadata field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add dedicated CPU embedding server for memory backfill ...

- Install Ollama CPU-only on 192.168.1.168 server
- Pull nomic-embed-text:latest on server
- Create systemd service ollama-embed.service (0.0.0.0:11434)
- Add embedding_ollama_host / embedding_ollama_api_key to config.py
- Update deps.py to build separate embedding backend when host configured
- Update backfill_embeddings.py to use dedicated embedding backend
- Add _generate_embeddings batch helper and backfill_embeddings to store.py
- Backfilled 119 existing facts with embeddings

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Enrich memory extractor with tool calls/results in transcript ...

- _EXTRACT_SYSTEM now explains 4 transcript entry types and instructs
  LLM to trust tool results over chat, return source/source_context
- _extract_facts builds tool_call_map, appends [Tool call] and
  [Tool result] lines with truncation (500/200 chars)
- Transcript capped at 12k chars (head+tail, drop middle)
- Parse source/source_context from LLM response; map confidence:
  tool_call/auto_discovery=95, user_explicit=90, default=70
- Add TODO comment about deferred semantic deduplication

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Add pgvector migration script for memory_facts ...

- ALTER TABLE memory_facts: embedding, source, confidence, expires_at, source_context

- CREATE INDEX: hnsw(embedding), expires, source+category

- Safe to run multiple times (IF NOT EXISTS)

- Reads DATABASE_URL from settings

Wire pgvector semantic search into memory system ...

- Add vector(768) column + HNSW index to memory_facts

- Add LLMBackend.embed() with Ollama + fallback implementation

- MemoryStore: cosine-distance search with ILIKE fallback

- New memory tool params: source, confidence, expires_days, source_context

- Update extractor, sqlite_store, deps wiring

- Add pgvector to requirements

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Stop image_view hallucinations on inline-attached images ...

The model was inventing fake paths/URLs (e.g. files.oaiusercontent.com,
/home/ubuntu/navi-1/input_file_0.png) and calling image_view on them
when the user attached an image directly in chat — the image was
already in the multimodal context, but the tool description and lack
of a signal pushed the model to "load" it anyway.

- websocket.py: when a user message has inline images, append a brief
  note that they are already in context.
- image_view.py: soften the description — keep proactive use for paths
  and URLs the model genuinely cannot see, but tell it inline images
  don't need this tool.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

changed llm

Add eval system Phase 1 — message feedback signal ...

Spec at docs/eval_system.md describes the full LLM-as-judge plan;
this commit lands only the in-app feedback layer:
- debug/eval/ Python package with EvalDB (asyncpg) and FastAPI router
  exposing /eval/feedback (set / clear / list)
- message_feedback postgres table keyed by (session_id, message_index)
- thumbs up / down on each completed assistant block in the webclient,
  optimistic update with rollback on failure

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

changed llm & new ollama param

changed model

Strengthen todo progress discipline

Add structured planning review and adaptive depth

Tune reflect autonomy guidance

Improve compression and memory prompts