# Memory System

Long-term user memory: facts extracted from conversations, stored in SQLite, injected into every session.

## PostgreSQL + pgvector (semantic search)

When `DATABASE_URL` is set, the memory system uses **PostgreSQL with pgvector** for semantic search via embeddings.

| Feature | SQLite | PostgreSQL |
|---|---|---|
| Storage | File-based | Server |
| Semantic search | No | Yes (cosine distance on `vector(768)`) |
| Embeddings | None | Generated via Ollama (`nomic-embed-text:latest`) |
| Metadata | `category, key, value` | `+ source, confidence, expires_at, source_context` |

---

## Schema migration

When upgrading the memory system to a new schema (e.g. adding pgvector columns), run:

```bash
.venv/bin/python navi/memory/migrate_pgvector.py
```

This script:
1. Verifies the `vector` extension is installed in PostgreSQL
2. Adds missing columns: `embedding`, `source`, `confidence`, `expires_at`, `last_verified_at`, `source_context`
3. Creates indexes: `hnsw(embedding)`, `expires`, `source+category`

Safe to run multiple times — all operations use `IF NOT EXISTS`.

---

## Storage (`navi/memory/store.py`)

Three tables in the database:

| Table | Purpose |
|---|---|
| `memory_facts` | Individual facts: `(category, key, value)` — unique on `(category, key)` |
| `memory_summary` | Single-row narrative summary generated from all facts |
| `session_memory_state` | Tracks which sessions have been processed (by `extracted_at`) |

`MemoryStore` is initialized synchronously (creates tables), all operations are async via asyncpg (PostgreSQL) or aiosqlite (SQLite fallback).

### Key operations

| Method | Description |
|---|---|
| `upsert_fact(category, key, value)` | Insert or update a fact |
| `search_facts(query, limit=15)` | Full-text search across category/key/value (OR across terms) |
| `delete_fact(key, category=None)` | Delete by key, optionally filtered by category |
| `get_all_facts(limit=None)` | All facts ordered by `(category, updated_at DESC)` |
| `get_summary()` | Current narrative summary text |
| `set_summary(content)` | Replace the summary |
| `mark_session_extracted(session_id)` | Record extraction timestamp |
| `get_extracted_at(session_id)` | Check if/when a session was processed |

---

## Automatic extraction (`navi/memory/extractor.py`)

Facts are extracted from stale sessions automatically.

**Trigger:** `POST /sessions` (create new session) fires `_process_stale_sessions()` as a background task.

**Stale criterion:** `session.last_active < now - 30 minutes` AND not yet extracted (or extracted before last activity).

**Extraction process:**
1. Render conversation as plain text.
2. Call LLM with an extraction prompt: "extract facts the user shared about themselves, their preferences, projects, and environment."
3. Parse the response as `category: key = value` lines.
4. Upsert each fact into `memory_facts`.
5. Regenerate `memory_summary` from all current facts.
6. Mark session as extracted.

---

## Memory injection into agent context

At the start of each `run_stream()` / `run()` / `run_ephemeral()` call, `_memory_msg()` is called:

```python
async def _memory_msg(self) -> Message | None:
    summary = await self._memory.get_summary()
    if not summary:
        return None
    return Message(role="system", content=f"## What I remember about the user\n\n{summary}")
```

This message is inserted after the main system message but before conversation history. The agent reads it on every turn.

---

## Memory tools

**`memory_search`** — searches facts by keyword query. Returns matching facts with category/key/value. Agent should call this when the user mentions something personal that may already be known.

**`memory_forget`** — deletes facts matching a key (optionally filtered by category). Agent calls this when the user explicitly asks to forget something or when a fact is clearly outdated.

---

## Memory usage guidelines (from persona)

Call `memory_search` when:
- The user mentions something personal (location, project, preference, recurring task).
- About to make an assumption about the user's environment or preferences — verify first.
- The user asks about something helped with before.

Do NOT call `memory_search` reflexively at the start of every session — only when context warrants it.

Call `memory_forget` only when the user explicitly asks, or when a stored fact is clearly wrong or outdated.