diff --git a/docs/memory.md b/docs/memory.md index b400040..baf52a8 100644 --- a/docs/memory.md +++ b/docs/memory.md @@ -24,7 +24,7 @@ | Storage | File-based | Server | | Semantic search | No | Yes (cosine distance on `vector(768)`) | | Embeddings | None | Generated via Ollama (`nomic-embed-text:latest`) | -| Metadata | `category, key, value` | `+ embedding, source, confidence, expires_at, source_context` | +| Metadata | `category, key, value` | `+ embedding, source, confidence, expires_at, source_context, merged_into` | ### Dedicated embedding backend @@ -50,7 +50,7 @@ This script: 1. Verifies the `vector` extension is installed in PostgreSQL -2. Adds missing columns: `embedding`, `source`, `confidence`, `expires_at`, `last_verified_at`, `source_context` +2. Adds missing columns: `embedding`, `source`, `confidence`, `expires_at`, `last_verified_at`, `source_context`, `merged_into` 3. Creates indexes: `hnsw(embedding)`, `expires`, `source+category`, `pg_trgm` GIN indexes for ILIKE fallback Safe to run multiple times — all operations use `IF NOT EXISTS`. @@ -83,21 +83,23 @@ `user_id` references `navi_users(id)` with `ON DELETE CASCADE`. Facts and summaries are scoped per user. Admin with `navi.memory.read_all` can pass `user_id=None` for global search. +`merged_into` is a nullable pointer to the row that superseded this one — the extractor writes the same fact under a new spelling of the key, and the older row is hidden rather than deleted, so the merge can be undone with `UPDATE memory_facts SET merged_into = NULL`. Every read path filters `merged_into IS NULL`; `get_all_facts(include_merged=True)` and `fact_count(include_merged=True)` are the only ways to see a hidden row, and `GET /admin/memory?include_merged=true` exposes them. + `MemoryStore` lazily creates tables on first async operation via `_get_pool()`. All operations are async via asyncpg (PostgreSQL). ### Key operations | Method | Description | |---|---| -| `upsert_fact(..., user_id=None)` | Insert or update a fact scoped to user | +| `upsert_fact(..., user_id=None, dedupe=False)` | Insert or update a fact scoped to user. With `dedupe=True` (the extractor) a row already holding the same key under a different spelling is hidden behind the surviving one. | | `search_facts(query, user_id=None, limit=15)` | **Vector search first** (cosine distance, cutoff 0.3), then ILIKE fallback. `user_id=None` requires admin permission for global search. | -| `delete_fact(key, category=None, user_id=None)` | Delete by key, optionally filtered by category and user | -| `get_all_facts(user_id=None, all_users=False, limit=None, offset=0, search=None, sort_by="category", sort_order="desc")` | All facts ordered by `sort_by`. Pass `all_users=True` for admin global view. | +| `delete_fact(key, category=None, user_id=None)` | Delete by key, optionally filtered by category and user. Drops the hidden rows pointing at it first, so none is left orphaned. | +| `get_all_facts(user_id=None, all_users=False, limit=None, offset=0, search=None, sort_by="category", sort_order="desc", include_merged=False)` | All facts ordered by `sort_by`. Pass `all_users=True` for admin global view. | | `get_summary(user_id=None)` | Current narrative summary text for user | | `set_summary(content, user_id=None)` | Replace the summary for user | | `mark_session_extracted(session_id)` | Record extraction timestamp | | `get_extracted_at(session_id)` | Check if/when a session was processed | -| `backfill_embeddings(batch_size=8)` | Generate embeddings for facts with `embedding IS NULL` | +| `backfill_embeddings(batch_size=8)` | Generate embeddings for facts with `embedding IS NULL` (hidden rows skipped) | --- @@ -119,7 +121,7 @@ 3. Call LLM with extraction prompt: "extract stable facts about the user." 4. Parse JSON array with fields: `category`, `key`, `value`, `source`, `source_context`. 5. Map confidence: `tool_call`/`auto_discovery` → 95, `user_explicit` → 90, default → 70. -6. Upsert each fact into `memory_facts`. +6. Upsert each fact into `memory_facts` with `dedupe=True`: a fact already stored under a differently spelled key (`language` vs `preferred_language`) is merged instead of duplicated. 7. Regenerate `memory_summary` from all current facts. 8. Mark session as extracted. @@ -139,6 +141,12 @@ This message is inserted after the main system message but before conversation history. The agent reads it on every turn. +Individual facts reach the prompt as their own list (`navi/core/context_builder.py`), one line each, carrying both the confidence and the age of the fact — a stale fact then reads as stale instead of as current: + +``` +• [preferences] editor → neovim (confidence: 90%, updated 34d ago) +``` + --- ## Memory tool (`navi/tools/memory.py`) @@ -150,7 +158,7 @@ | `save` | Upsert a fact with `category`, `key`, `value`, `source`, `confidence`, `expires_days`, `source_context` | | `search` | Find facts by keyword query (vector search → ILIKE fallback) | | `forget` | Delete a fact by `key` (optionally filtered by `category`) | -| `list` | Return all stored facts | +| `list` | Return the categories that hold facts, not the facts themselves | **Sources:** `conversation`, `tool_call`, `auto_discovery`, `user_explicit` diff --git a/manuals/memory.md b/manuals/memory.md index 1c2186e..7be719e 100644 --- a/manuals/memory.md +++ b/manuals/memory.md @@ -16,7 +16,7 @@ | `value` | `save` | The fact, one concise plain-text statement. | | `source` | `save` | `conversation` (default) \| `tool_call` \| `auto_discovery` \| `user_explicit`. | | `confidence` | `save` | 0–100, default 70. | -| `expires_days` | `save` | Days until the fact expires. Omit for "never". | +| `expires_days` | `save` | Set only when the thing itself ends — a trial, a temporary host. Omit for everything else. | | `source_context` | `save` | Provenance, e.g. `found via ip addr on localhost`. | Categories: `profile` = who they are, `preferences` = likes/dislikes, `technical` = OS/tools/servers, `projects` = ongoing work, `other` = everything else. @@ -33,7 +33,7 @@ {"action": "save", "category": "technical", "key": "prod_server_ip", "value": "Production runs on 192.168.1.168 (branch master, docker postgres)", - "source": "tool_call", "confidence": 95, "expires_days": 7, + "source": "tool_call", "confidence": 95, "source_context": "found via ssh_exec hostname/ip addr"} ``` @@ -47,7 +47,7 @@ | Read it on the web | `auto_discovery` | 50 | | Inferred / guessed | `conversation` | 30 | -**System facts expire.** An IP, a running service, a host's disk layout — set `expires_days: 7`. A stale host address is worse than no address, because it is trusted. Durable preferences and identity never expire. +**Expiry is for facts with a known end, not for facts you merely doubt.** Set `expires_days` when the thing itself finishes — a trial that ends on a date, a host provisioned for a week. An address or a running service does not expire on a schedule: it is wrong the moment it changes, and only the tool that observes it can know that. A fact with no expiry is corrected by saving over it, which is what search-then-save is for. ### `search` Requires `query`. Returns up to 15 matching facts, each with its category, key, value and provenance (`src`, `conf`, `ctx`). This is the read path — search before saving, so you update an existing key instead of inventing a near-duplicate. diff --git a/navi/api/routes/admin.py b/navi/api/routes/admin.py index f5181d9..d0bd235 100644 --- a/navi/api/routes/admin.py +++ b/navi/api/routes/admin.py @@ -210,8 +210,14 @@ sort_by: str = "updated_at", sort_order: str = "desc", user_id: str | None = None, + include_merged: bool = False, ): - """Return memory facts with pagination, search and sorting.""" + """Return memory facts with pagination, search and sorting. + + `include_merged` also returns the facts hidden behind a merge loser, each + carrying the id of the fact it points at — this is the only place the mark + is visible, and therefore the only way to undo one. + """ from navi.api.deps import get_memory_store memory = get_memory_store() @@ -223,11 +229,13 @@ sort_order=sort_order, user_id=user_id, all_users=user_id is None, + include_merged=include_merged, ) total = await memory.fact_count( user_id=user_id, all_users=user_id is None, search=search or None, + include_merged=include_merged, ) return {"total": total, "limit": limit, "offset": offset, "items": facts} diff --git a/navi/core/context_builder.py b/navi/core/context_builder.py index 3d15488..b26a1fc 100644 --- a/navi/core/context_builder.py +++ b/navi/core/context_builder.py @@ -3,6 +3,7 @@ Extracted from agent.py to reduce the Agent class surface area. """ +from datetime import datetime, timezone from pathlib import Path from typing import TYPE_CHECKING @@ -15,6 +16,27 @@ log = structlog.get_logger() +def _fact_age(updated_at: str) -> str: + """How old a fact is, so a stale one reads as stale. + + Nothing in memory expires on a clock — a fact is wrong when the world moves, + not when a timer fires — so the injection says how old it is and lets the + agent weigh it. A missing or unparseable timestamp simply says nothing. + """ + try: + then = datetime.fromisoformat(updated_at) + except (TypeError, ValueError): + return "" + if then.tzinfo is None: + then = then.replace(tzinfo=timezone.utc) + days = (datetime.now(timezone.utc) - then).days + if days <= 0: + return "today" + if days < 30: + return f"{days}d ago" + return f"{days // 30}mo ago" + + # Roles whose content may be head/tail-truncated when a single message is too # large for the context window. User messages carry intent and are never # truncated; system/summary messages are already compact. @@ -242,7 +264,12 @@ lines = ["Known facts about the user:"] for f in new_facts: conf = f.get("confidence", 70) - lines.append(f"• [{f['category']}] {f['key']} → {f['value']} (confidence: {conf}%)") + age = _fact_age(str(f.get("updated_at", ""))) + age_note = f", updated {age}" if age else "" + lines.append( + f"• [{f['category']}] {f['key']} → {f['value']} " + f"(confidence: {conf}%{age_note})" + ) result = "\n".join(lines) log.info("memory_facts_msg.injected", lines=len(new_facts), length=len(result)) return Message(role="system", content=result) diff --git a/navi/memory/_ddl.py b/navi/memory/_ddl.py index 53c4b49..22798c0 100644 --- a/navi/memory/_ddl.py +++ b/navi/memory/_ddl.py @@ -14,6 +14,9 @@ # Migration: add user_id to existing tables (nullable for legacy) "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS user_id TEXT REFERENCES navi_users(id) ON DELETE CASCADE", "ALTER TABLE memory_summary ADD COLUMN IF NOT EXISTS user_id TEXT REFERENCES navi_users(id) ON DELETE CASCADE", + # Merge losers: a fact hidden behind a canonical one written under a different + # spelling of the same key. Nullable, filtered with IS NULL on read. + "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS merged_into TEXT", # Migrate unique constraint from (category, key) to (user_id, category, key) "DO $$ BEGIN IF EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'memory_facts_category_key_key') THEN ALTER TABLE memory_facts DROP CONSTRAINT memory_facts_category_key_key; END IF; END $$;", "DO $$ BEGIN IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname = 'memory_facts_user_cat_key') THEN ALTER TABLE memory_facts ADD CONSTRAINT memory_facts_user_cat_key UNIQUE (user_id, category, key); END IF; END $$;", @@ -35,6 +38,7 @@ expires_at TIMESTAMPTZ, last_verified_at TIMESTAMPTZ, source_context TEXT, + merged_into TEXT, UNIQUE(user_id, category, key) )""" % embedding_col, "CREATE INDEX IF NOT EXISTS idx_memory_facts_expires ON memory_facts (expires_at) WHERE expires_at IS NOT NULL", diff --git a/navi/memory/_embeddings.py b/navi/memory/_embeddings.py index 6dcaeff..5af1839 100644 --- a/navi/memory/_embeddings.py +++ b/navi/memory/_embeddings.py @@ -96,7 +96,8 @@ async with pool.acquire() as conn: while True: rows = await conn.fetch( - "SELECT id, value FROM memory_facts WHERE embedding IS NULL LIMIT $1", + "SELECT id, value FROM memory_facts " + "WHERE embedding IS NULL AND merged_into IS NULL LIMIT $1", batch_size, ) if not rows: diff --git a/navi/memory/_facts.py b/navi/memory/_facts.py index 57ece27..f9be273 100644 --- a/navi/memory/_facts.py +++ b/navi/memory/_facts.py @@ -15,9 +15,14 @@ _SEPARATORS = re.compile(r"[-_/\\.]") _NOISE = re.compile(r"[^\w\s]", re.UNICODE) +_KEY_SPLIT = re.compile(r"[-_/.\s]+") _AUTO_DUMP_THRESHOLD = 60 _VECTOR_CUTOFF_DISTANCE = 0.3 +# Leading tokens that carry no meaning of their own: `preferred_language`, +# `ui_language` and `language` are one key, and the extractor writes all three. +_MERGE_PREFIX_TOKENS = frozenset({"preferred", "preference", "pref", "user", "my", "ui", "response"}) + def _normalize_query(query: str) -> list[str]: q = _SEPARATORS.sub(" ", query) @@ -25,6 +30,19 @@ return [t for t in q.lower().split() if len(t) > 1] +def _key_tokens(key: str) -> frozenset[str]: + """The tokens of a fact key, with *leading* noise words dropped. + + Leading only, and that is load-bearing: `ui_language` is the language key with a + noise prefix, but `dark_ui` is a name of its own and must not collapse into + `dark`. A rule that strips `ui` wherever it stands cannot tell those apart. + """ + tokens = [t for t in _KEY_SPLIT.split(key.strip().lower()) if len(t) > 1] + while tokens and tokens[0] in _MERGE_PREFIX_TOKENS: + tokens.pop(0) + return frozenset(tokens) + + class FactMixin: """Fact storage operations. @@ -45,12 +63,26 @@ confidence: int = 70, expires_at: datetime | None = None, source_context: str = "", + dedupe: bool = False, ) -> None: + """Write a fact, keyed on `(user_id, category, key)`. + + With `dedupe`, the same fact already stored under a different spelling of + this key is hidden behind the row that survives — see `_merge_key_aliases`. + Off by default, so callers that own their keys (the `memory` tool) write + exactly the row they asked for. + """ now = datetime.now(timezone.utc) embedding = await self._generate_embedding(value) vec_str = _vector_to_str(embedding) if embedding else None pool = await self._get_pool() - async with pool.acquire() as conn: + fact_id = str(uuid.uuid4()) + async with pool.acquire() as conn, conn.transaction(): + # The dedupe path marks losers before the row they point at exists, so a + # failed insert would leave them hidden behind nothing. The plain path + # needs no boundary of its own, and taking one anyway costs nothing. + if dedupe: + await self._merge_key_aliases(conn, category, key, user_id, fact_id) if vec_str: await conn.execute( """INSERT INTO memory_facts @@ -64,9 +96,9 @@ embedding = EXCLUDED.embedding, source = EXCLUDED.source, confidence = EXCLUDED.confidence, - expires_at = EXCLUDED.expires_at, + expires_at = COALESCE(EXCLUDED.expires_at, memory_facts.expires_at), source_context = EXCLUDED.source_context""", - str(uuid.uuid4()), user_id, category, key, value, now, now, + fact_id, user_id, category, key, value, now, now, source_session_id, vec_str, source, confidence, expires_at, source_context, ) else: @@ -81,12 +113,75 @@ source_session_id = EXCLUDED.source_session_id, source = EXCLUDED.source, confidence = EXCLUDED.confidence, - expires_at = EXCLUDED.expires_at, + expires_at = COALESCE(EXCLUDED.expires_at, memory_facts.expires_at), source_context = EXCLUDED.source_context""", - str(uuid.uuid4()), user_id, category, key, value, now, now, + fact_id, user_id, category, key, value, now, now, source_session_id, source, confidence, expires_at, source_context, ) + async def _merge_key_aliases( + self, conn, category: str, key: str, user_id: str | None, new_id: str + ) -> None: + """Hide the same fact already stored under a different spelling of this key. + + The exact-key upsert cannot see `preferred_language` when `language` is what + is already there, which is how the base grew 11 spellings of the same handful + of keys. Compared on key tokens and **not** on value: `ui_language` and + `language` are one key while `dark` and `dark_ui` are two, and that is set + equality, not overlap. Values are deliberately left out of the decision: the + survivor is the row that already holds the key the writer used (else the + incoming one), and since the upsert writes the fresh value into it, the + fresher value is what remains readable either way. + + Only *leading* noise tokens are dropped (see `_key_tokens`), so a trailing + one survives: `dark_ui` keeps its `ui` and stays apart from `dark`, while + `ui_dark` loses it and does not. That asymmetry is the price of reading `ui` + as noise in `ui_language` and as part of the name in `dark_ui`; no rule reads + it both ways. + + Marked, never deleted: a loser keeps its own key and its own value, so the + merge is reversible with `UPDATE ... SET merged_into = NULL`. + """ + tokens = _key_tokens(key) + if not tokens: + return + # Hidden rows are read too: one of them may hold this exact key, and then the + # upsert below will land on it through ON CONFLICT rather than inserting the + # row we are about to point everything at. + rows = await conn.fetch( + "SELECT id, key, merged_into FROM memory_facts " + "WHERE user_id IS NOT DISTINCT FROM $1 AND category = $2 AND id <> $3", + user_id, category, new_id, + ) + exact = [r["id"] for r in rows if r["key"] == key] + # Only live rows are worth hiding; the rest are already behind something. + aliases = [ + r["id"] for r in rows + if r["key"] != key and r["merged_into"] is None and _key_tokens(r["key"]) == tokens + ] + winner = exact[0] if exact else new_id + if exact: + # A row with this exact key is what the ON CONFLICT will update, so it + # becomes live again — and it has to be un-hidden for that, or the write + # lands in a row no read can see and the losers get pointed at a row that + # was never inserted. + await conn.execute( + "UPDATE memory_facts SET merged_into = NULL WHERE id = ANY($1::text[])", exact + ) + if not aliases: + return + # The second clause re-points the losers of an earlier merge, so pointers + # never form a chain through a row that is itself hidden. + await conn.execute( + "UPDATE memory_facts SET merged_into = $2 " + "WHERE (id = ANY($1::text[]) OR merged_into = ANY($1::text[])) AND id <> $2", + aliases, winner, + ) + log.info( + "memory.key_alias_merged", + category=category, key=key[:60], merged=len(aliases), winner=winner, + ) + async def search_facts(self, query: str, user_id: str | None = None, limit: int = 15) -> list[dict]: # 1. Try vector search if pgvector + embedding backend are available if self._embedding_backend and await self._has_pgvector(): @@ -99,10 +194,11 @@ if user_id is None: rows = await conn.fetch( """SELECT id, category, key, value, updated_at, - source, confidence, expires_at, source_context, + source, confidence, expires_at, source_context, merged_into, embedding <=> $1::vector AS distance FROM memory_facts WHERE (expires_at IS NULL OR expires_at > now()) + AND merged_into IS NULL AND user_id IS NULL ORDER BY embedding <=> $1::vector LIMIT $2""", @@ -111,10 +207,11 @@ else: rows = await conn.fetch( """SELECT id, category, key, value, updated_at, - source, confidence, expires_at, source_context, + source, confidence, expires_at, source_context, merged_into, embedding <=> $1::vector AS distance FROM memory_facts WHERE (expires_at IS NULL OR expires_at > now()) + AND merged_into IS NULL AND user_id = $3 ORDER BY embedding <=> $1::vector LIMIT $2""", @@ -157,6 +254,9 @@ user_filter = "user_id IS NULL" if user_id is None else f"user_id = ${len(base_params) + 1}" if user_id is not None: base_params.append(user_id) + # This branch used to be the only read path that served expired facts, and + # hidden merge losers used to be served by all of them. + user_filter += " AND merged_into IS NULL AND (expires_at IS NULL OR expires_at > now())" limit_idx = len(base_params) + 1 and_where = " AND ".join(term_conds) @@ -166,7 +266,7 @@ async with pool.acquire() as conn: rows = await conn.fetch( f"SELECT id, category, key, value, updated_at, source, confidence, " - f"expires_at, source_context FROM memory_facts " + f"expires_at, source_context, merged_into FROM memory_facts " f"WHERE {user_filter} AND {and_where} ORDER BY updated_at DESC LIMIT ${limit_idx}", *base_params, limit, ) @@ -178,7 +278,7 @@ ) rows = await conn.fetch( f"SELECT id, category, key, value, updated_at, source, confidence, " - f"expires_at, source_context, ({score_expr}) AS score " + f"expires_at, source_context, merged_into, ({score_expr}) AS score " f"FROM memory_facts WHERE {user_filter} AND ({or_where}) " f"ORDER BY score DESC, updated_at DESC LIMIT ${limit_idx}", *base_params, limit, @@ -187,7 +287,8 @@ async def delete_fact(self, key: str, category: str | None = None, user_id: str | None = None) -> int: pool = await self._get_pool() - async with pool.acquire() as conn: + async with pool.acquire() as conn, conn.transaction(): + await self._delete_merged_dependents(conn, key, category, user_id) if category: if user_id is None: result = await conn.execute( @@ -210,6 +311,31 @@ ) return int(result.split()[1]) + async def _delete_merged_dependents( + self, conn, key: str, category: str | None, user_id: str | None + ) -> None: + """Forget drops the hidden aliases of the keys it is about to delete. + + A merge loser points at its winner; deleting the winner on its own would + leave those rows hidden with nothing left to point at and no way back to + them. They are deleted, not un-hidden — the user asked for the key gone. + """ + conditions = ["LOWER(key) = LOWER($1)"] + args: list = [key] + if category: + args.append(category) + conditions.append(f"LOWER(category) = LOWER(${len(args)})") + if user_id is None: + conditions.append("user_id IS NULL") + else: + args.append(user_id) + conditions.append(f"user_id = ${len(args)}") + await conn.execute( + "DELETE FROM memory_facts WHERE merged_into IN " + f"(SELECT id FROM memory_facts WHERE {' AND '.join(conditions)})", + *args, + ) + async def get_all_facts( self, user_id: str | None = None, @@ -219,10 +345,11 @@ sort_by: str = "category", sort_order: str = "desc", all_users: bool = False, + include_merged: bool = False, ) -> list[dict]: q = ( "SELECT id, category, key, value, updated_at, source, confidence, " - "expires_at, source_context FROM memory_facts " + "expires_at, source_context, merged_into FROM memory_facts " ) params: list = [] param_idx = 0 @@ -239,6 +366,8 @@ conditions.append("user_id IS NULL") else: conditions.append(f"user_id = {add_param(user_id)}") + if not include_merged: + conditions.append("merged_into IS NULL") if search: like = f"%{search}%" conditions.append( @@ -271,6 +400,7 @@ user_id: str | None = None, all_users: bool = False, search: str | None = None, + include_merged: bool = False, ) -> int: q = "SELECT COUNT(*) FROM memory_facts" params: list = [] @@ -281,6 +411,8 @@ else: conditions.append(f"user_id = ${len(params) + 1}") params.append(user_id) + if not include_merged: + conditions.append("merged_into IS NULL") if search: like = f"%{search}%" conditions.append( @@ -299,11 +431,13 @@ async with pool.acquire() as conn: if user_id is None: rows = await conn.fetch( - "SELECT DISTINCT category FROM memory_facts WHERE user_id IS NULL ORDER BY category" + "SELECT DISTINCT category FROM memory_facts " + "WHERE user_id IS NULL AND merged_into IS NULL ORDER BY category" ) else: rows = await conn.fetch( - "SELECT DISTINCT category FROM memory_facts WHERE user_id = $1 ORDER BY category", + "SELECT DISTINCT category FROM memory_facts " + "WHERE user_id = $1 AND merged_into IS NULL ORDER BY category", user_id, ) return [r["category"] for r in rows] @@ -324,4 +458,5 @@ "confidence": row.get("confidence", 70), "expires_at": expires_str, "source_context": row.get("source_context", ""), + "merged_into": row.get("merged_into"), } diff --git a/navi/memory/extractor.py b/navi/memory/extractor.py index 2170b52..a45005d 100644 --- a/navi/memory/extractor.py +++ b/navi/memory/extractor.py @@ -30,7 +30,7 @@ Extract facts that are: - Persistent characteristics: name, age, location, occupation, family situation -- Technical environment: OS, tools, servers, devices, IPs, running services +- Stable technical traits: operating system, primary editor, main language - Preferences: communication style, coding habits, likes/dislikes, workflow patterns - Ongoing projects or goals @@ -42,6 +42,14 @@ - Infrastructure inventory, service topology, network routes, server facts, or local operational notes - Already-known facts that appear in the transcript +Never record a secret — no passwords, tokens, API keys or private keys, in any +form, not even quoted from a tool result. Record where a secret lives, not what +it is. + +Keys are lowercase snake_case and must be stable across sessions: reuse the exact +key already used in the transcript for the same fact rather than inventing a new +spelling of it. + For each fact, indicate its source: - "conversation" — the user explicitly stated it or it was inferred from chat - "tool_call" — discovered through a tool execution result @@ -173,10 +181,6 @@ confidence = {"tool_call": 95, "auto_discovery": 95, "user_explicit": 90}.get(source, 70) if key and value: - # TODO: Semantic deduplication — before upsert, search for semantically - # similar facts and merge/update instead of creating duplicates. - # Problem: LLM generates different keys for the same fact across sessions. - # Solution: vector search + similarity threshold before upsert. await store.upsert_fact( category=category, key=key, @@ -186,6 +190,7 @@ source=source, confidence=confidence, source_context=source_context, + dedupe=True, ) count += 1 diff --git a/navi/memory/migrate_pgvector.py b/navi/memory/migrate_pgvector.py index edb9c93..1a5e675 100644 --- a/navi/memory/migrate_pgvector.py +++ b/navi/memory/migrate_pgvector.py @@ -5,7 +5,7 @@ What it does: 1. Verifies pgvector extension is installed -2. Adds missing columns to memory_facts: embedding, source, confidence, expires_at, last_verified_at, source_context +2. Adds missing columns to memory_facts: embedding, source, confidence, expires_at, last_verified_at, source_context, merged_into 3. Creates indexes: hnsw on embedding, expires (partial), source+category Safe to run multiple times — all operations use IF NOT EXISTS. @@ -34,6 +34,7 @@ "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS expires_at TIMESTAMPTZ", "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS last_verified_at TIMESTAMPTZ", "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS source_context TEXT", + "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS merged_into TEXT", ] ) diff --git a/navi/tools/memory.py b/navi/tools/memory.py index 5a7a71c..00466b9 100644 --- a/navi/tools/memory.py +++ b/navi/tools/memory.py @@ -15,8 +15,9 @@ name = "memory" description = ( "Manage long-term memory about the user — facts that survive across sessions. " - "Actions: save (upsert a fact), search (find facts by query), forget (delete by key), list (all facts). " - "When saving system facts (IPs, hosts, services), set source='tool_call', confidence=95, and expires_days=7." + "Actions: save (upsert a fact), search (find facts by query), forget (delete by key), " + "list (which categories hold facts). " + "Never save a secret — password, token, key. Save where it lives, not what it is." ) parameters = { "type": "object", @@ -28,7 +29,7 @@ "save — upsert a fact (overwrites existing key). " "search — find facts by keyword query. " "forget — delete a fact by key. " - "list — return all stored facts." + "list — the categories that hold facts, not the facts themselves." ), }, "query": { @@ -72,7 +73,10 @@ }, "expires_days": { "type": "integer", - "description": "save only: how many days this fact stays valid. Null = never expires.", + "description": ( + "save only: set it only when the thing itself ends — a trial, a " + "temporary host. Not for facts you merely doubt. Omit for everything else." + ), }, "source_context": { "type": "string", diff --git a/tests/unit/core/test_context_builder.py b/tests/unit/core/test_context_builder.py index b568838..a8547fe 100644 --- a/tests/unit/core/test_context_builder.py +++ b/tests/unit/core/test_context_builder.py @@ -1,8 +1,9 @@ """Unit tests for ContextBuilder.""" import asyncio +from datetime import datetime, timedelta, timezone -from navi.core.context_builder import ContextBuilder, _fact_value_is_abs_path_outside +from navi.core.context_builder import ContextBuilder, _fact_age, _fact_value_is_abs_path_outside from navi.llm.base import Message from tests.conftest_factory import make_profile, make_profile_registry @@ -269,6 +270,41 @@ return {"id": fid, "category": "technical", "key": key, "value": value, "confidence": 80} +class TestFactAge: + """Nothing in memory expires on a clock, so the injection reports the age + instead and lets the agent weigh a stale fact as stale. + """ + + def _ago(self, **delta): + now = datetime.now(timezone.utc) + return (now - timedelta(**delta)).isoformat() + + def test_today(self): + assert _fact_age(self._ago(hours=3)) == "today" + + def test_days(self): + assert _fact_age(self._ago(days=12)) == "12d ago" + + def test_months(self): + assert _fact_age(self._ago(days=95)) == "3mo ago" + + def test_naive_timestamp_is_read_as_utc(self): + """asyncpg hands back an aware datetime, but a row written by hand may not.""" + naive = (datetime.now(timezone.utc) - timedelta(days=5)).replace(tzinfo=None) + assert _fact_age(naive.isoformat()) == "5d ago" + + def test_a_missing_or_broken_timestamp_says_nothing(self): + assert _fact_age("") == "" + assert _fact_age("not a date") == "" + assert _fact_age(None) == "" + + def test_the_age_reaches_the_injected_line(self): + fact = {**_fact("os", "Linux"), "updated_at": self._ago(days=34)} + builder = ContextBuilder(profile_registry=make_profile_registry(), memory_store=FakeMemoryStore([fact])) + msg = asyncio.run(builder._memory_facts_msg("what do you know about me")) + assert "(confidence: 80%, updated 1mo ago)" in msg.content + + class TestMemoryFactsScopeFilter: def _builder(self, facts): return ContextBuilder(profile_registry=make_profile_registry(), memory_store=FakeMemoryStore(facts)) diff --git a/tests/unit/memory/test_extractor.py b/tests/unit/memory/test_extractor.py index dcb0b82..daed237 100644 --- a/tests/unit/memory/test_extractor.py +++ b/tests/unit/memory/test_extractor.py @@ -20,6 +20,7 @@ '[{"category": "profile", "key": "name", "value": "Eugene", "source": "conversation"}]' ]) conn = FakeConnection() + conn.enqueue([]) # no key aliases to merge conn.enqueue("INSERT 0 1") store = make_store_with_pool(conn) session = FakeSession([ @@ -27,8 +28,68 @@ ]) count = await _extract_facts(session, backend, "test-model", store) assert count == 1 - assert conn.calls[0][0] == "execute" - assert "memory_facts" in conn.calls[0][1] + assert conn.calls[0][0] == "fetch" + assert "user_id IS NOT DISTINCT FROM $1" in conn.calls[0][1] + assert conn.calls[1][0] == "execute" + assert "memory_facts" in conn.calls[1][1] + + async def test_merges_a_key_written_under_a_new_spelling(self): + """The extractor invents `preferred_language` for a fact it already stored as + `language`, and the exact-key upsert cannot see it — that is how the base + grew 11 spellings of the same handful of keys. + """ + backend = FakeLLMBackend(responses=[ + '[{"category": "preferences", "key": "preferred_language", "value": "ru"}]' + ]) + conn = FakeConnection() + conn.enqueue([FakeRecord(id="old-1", key="language", merged_into=None)]) + conn.enqueue("UPDATE 1") # the mark, written before the row it points at + conn.enqueue("INSERT 0 1") # the new row itself + store = make_store_with_pool(conn) + session = FakeSession([Message(role="user", content="I prefer Russian")]) + count = await _extract_facts(session, backend, "test-model", store) + assert count == 1 + + op, query, args = conn.calls[1] + assert op == "execute" + assert "SET merged_into" in query + inserted_id = conn.calls[2][2][0] + assert args == (["old-1"], inserted_id) # the alias points at the fresher row + + async def test_merging_a_key_that_is_already_hidden(self): + """The second write of a spelling that was hidden by the first is the case + the store tests cover in isolation; here it has to survive the extractor's + whole path, because this is the shape the prod database was found in. + """ + backend = FakeLLMBackend(responses=[ + '[{"category": "preferences", "key": "language", "value": "ru"}]' + ]) + conn = FakeConnection() + conn.enqueue([ + FakeRecord(id="exact-1", key="language", merged_into="live-1"), + FakeRecord(id="live-1", key="preferred_language", merged_into=None), + ]) + conn.enqueue("UPDATE 1") # un-hide the row the upsert lands on + conn.enqueue("UPDATE 1") # hide the spelling that was live + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + session = FakeSession([Message(role="user", content="I prefer Russian")]) + assert await _extract_facts(session, backend, "test-model", store) == 1 + assert "SET merged_into = NULL" in conn.calls[1][1] + assert conn.calls[2][2] == (["live-1"], "exact-1") + + async def test_leaves_a_different_key_alone(self): + """Token equality, not overlap: `dark` and `dark_ui` are two keys.""" + backend = FakeLLMBackend(responses=[ + '[{"category": "preferences", "key": "dark_ui", "value": "true"}]' + ]) + conn = FakeConnection() + conn.enqueue([FakeRecord(id="old-1", key="dark", merged_into=None)]) + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + session = FakeSession([Message(role="user", content="hi")]) + await _extract_facts(session, backend, "test-model", store) + assert not any("SET merged_into" in c[1] for c in conn.calls) async def test_invalid_json_returns_zero(self): backend = FakeLLMBackend(responses=["not json"]) @@ -52,6 +113,7 @@ '[{"category": "profile", "key": "name", "value": "Eugene"}, "not a dict"]' ]) conn = FakeConnection() + conn.enqueue([]) conn.enqueue("INSERT 0 1") store = make_store_with_pool(conn) session = FakeSession([Message(role="user", content="hi")]) @@ -63,6 +125,7 @@ '[{"category": "profile", "key": "name", "value": "Eugene"}]' ]) conn = FakeConnection() + conn.enqueue([]) conn.enqueue("INSERT 0 1") store = make_store_with_pool(conn) session = FakeSession([Message(role="user", content="hi")]) @@ -74,13 +137,14 @@ '[{"category": "technical", "key": "ip", "value": "10.0.0.1", "source": "tool_call"}]' ]) conn = FakeConnection() + conn.enqueue([]) conn.enqueue("INSERT 0 1") store = make_store_with_pool(conn) session = FakeSession([Message(role="user", content="hi")]) count = await _extract_facts(session, backend, "test-model", store) assert count == 1 # Confidence for tool_call is 95 - call_args = conn.calls[0][2] + call_args = conn.calls[1][2] assert 95 in call_args # confidence parameter async def test_truncates_long_tool_results(self): @@ -125,6 +189,7 @@ "Summary: Eugene is the user.", # regenerate summary ]) conn = FakeConnection() + conn.enqueue([]) # upsert_fact: no key aliases conn.enqueue("INSERT 0 1") # upsert_fact conn.enqueue("OK") # mark_session_extracted conn.enqueue([ # get_all_facts in _regenerate_summary diff --git a/tests/unit/memory/test_memory_ddl.py b/tests/unit/memory/test_memory_ddl.py new file mode 100644 index 0000000..f9d31e1 --- /dev/null +++ b/tests/unit/memory/test_memory_ddl.py @@ -0,0 +1,40 @@ +"""Unit tests for the memory DDL — the only path that reaches an existing database. + +There is no migration framework here: `MemoryStore._get_pool()` runs `_build_ddl()` +on every process start, and on a database that already has the table only the +`ADD COLUMN IF NOT EXISTS` statements do anything. A column that is declared in the +CREATE TABLE but forgotten in that block exists on a fresh database and nowhere else. +""" + +import re + +from navi.memory._ddl import _build_ddl + + +def _facts_table(ddl: list[str]) -> str: + for stmt in ddl: + block = re.search(r"CREATE TABLE IF NOT EXISTS memory_facts \((.*?)\n\s*\)", stmt, re.DOTALL) + if block: + return block.group(1) + raise AssertionError("memory_facts is no longer declared in _build_ddl") + + +def test_merged_into_is_migrated_and_declared(): + for pgvector in (False, True): + ddl = _build_ddl(pgvector) + assert any( + "ALTER TABLE memory_facts ADD COLUMN IF NOT EXISTS merged_into" in stmt + for stmt in ddl + ), f"The migration never adds merged_into (pgvector={pgvector})" + assert "merged_into" in _facts_table(ddl) + + +def test_no_index_on_the_migrated_column(): + """Almost every read filters `merged_into IS NULL`, so an index over it cannot be + selective — and one placed in the CREATE list would run before the ALTER that + adds the column, failing the whole batch on an existing database. + """ + for pgvector in (False, True): + for stmt in _build_ddl(pgvector): + if "CREATE INDEX" in stmt: + assert "merged_into" not in stmt, f"an index mentions merged_into: {stmt[:80]}" diff --git a/tests/unit/memory/test_store.py b/tests/unit/memory/test_store.py index b5f53ac..eeb6af3 100644 --- a/tests/unit/memory/test_store.py +++ b/tests/unit/memory/test_store.py @@ -5,9 +5,40 @@ import pytest from navi.config import settings +from navi.memory._facts import _key_tokens from tests.conftest_factory import FakeConnection, FakeRecord, FakePool, make_store_with_pool +class TestKeyTokens: + """`preferred_language`, `ui_language` and `language` are one key while `dark` + and `dark_ui` are two — hence set equality, not overlap. + """ + + def test_prefixes_are_dropped(self): + assert _key_tokens("preferred_language") == _key_tokens("language") + assert _key_tokens("ui_language") == _key_tokens("language") + assert _key_tokens("response_language") == _key_tokens("language") + + def test_word_order_of_meaningful_tokens_does_not_matter(self): + assert _key_tokens("primary_editor") == _key_tokens("editor_primary") + + def test_a_trailing_noise_word_survives(self): + """Noise is dropped from the front only. `ui_language` loses its `ui`; + `dark_ui` keeps it, which is what keeps that key apart from `dark`. The + cost is that `ui_dark` does lose it and does collapse into `dark`.""" + assert _key_tokens("ui_language") == _key_tokens("language") + assert _key_tokens("dark_ui") == frozenset({"dark", "ui"}) + assert _key_tokens("ui_dark") == frozenset({"dark"}) + + def test_overlap_without_equality_is_not_a_match(self): + assert _key_tokens("dark") != _key_tokens("dark_ui") + assert _key_tokens("primary_os") != _key_tokens("os") + + def test_a_key_of_pure_noise_has_no_tokens(self): + assert _key_tokens("") == frozenset() + assert _key_tokens("ui_") == frozenset() + + class TestUpsertFact: async def test_calls_execute(self): conn = FakeConnection() @@ -37,6 +68,217 @@ assert highest_placeholder == len(columns) == len(args) +class TestExpirySurvivesAnExtractorWrite: + """`SET expires_at = EXCLUDED.expires_at` let the extractor, which always passes + None, wipe an expiry the agent had set on the same key. + """ + + async def test_plain_branch_coalesces(self): + conn = FakeConnection() + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="technical", key="ip", value="10.0.0.1") + assert "COALESCE(EXCLUDED.expires_at, memory_facts.expires_at)" in conn.calls[0][1] + + async def test_embedding_branch_coalesces_too(self): + conn = FakeConnection() + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + store._pgvector_checked = True + store._pgvector_available = True + store._embedding_backend = object() + + async def _fake_embed(text: str): + return [0.1] * 768 + + store._generate_embedding = _fake_embed + await store.upsert_fact(category="technical", key="ip", value="10.0.0.1") + assert "COALESCE(EXCLUDED.expires_at, memory_facts.expires_at)" in conn.calls[0][1] + + +class TestMergeKeyAliases: + """Key drift is what the extractor produces on its own: it writes the same fact + under a new spelling, and the exact-key upsert cannot see the old one. Prod had + 11 such clusters / 24 rows out of 1017 facts. + """ + + async def test_plain_upsert_never_looks_for_aliases(self): + conn = FakeConnection() + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="profile", key="name", value="Eugene") + assert [c[0] for c in conn.calls] == ["execute"] + + async def test_marks_the_older_spelling(self): + conn = FakeConnection() + conn.enqueue([FakeRecord(id="old-1", key="language", merged_into=None)]) + conn.enqueue("UPDATE 1") # the mark, written before the row it points at + conn.enqueue("INSERT 0 1") # the new row itself + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="preferred_language", + value="ru", dedupe=True) + + assert [c[0] for c in conn.calls] == ["fetch", "execute", "execute"] + _, query, args = conn.calls[1] + assert "SET merged_into" in query + inserted_id = conn.calls[2][2][0] # the id the INSERT carries + assert args == (["old-1"], inserted_id) + + async def test_repoints_the_losers_of_an_earlier_merge(self): + """Otherwise the loser of a loser keeps pointing at a row that is itself hidden.""" + conn = FakeConnection() + conn.enqueue([FakeRecord(id="old-1", key="language", merged_into=None)]) + conn.enqueue("UPDATE 2") + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="preferred_language", + value="ru", dedupe=True) + assert "merged_into = ANY($1::text[])" in conn.calls[1][1] + + async def test_an_exact_key_stays_the_winner(self): + """An exact key is the ON CONFLICT target, so the freshest value lands in it + and only the other spelling is hidden.""" + conn = FakeConnection() + conn.enqueue([ + FakeRecord(id="exact-1", key="language", merged_into=None), + FakeRecord(id="old-1", key="preferred_language", merged_into=None), + ]) + conn.enqueue("UPDATE 1") + conn.enqueue("UPDATE 1") + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="language", + value="ru", dedupe=True) + assert conn.calls[2][2] == (["old-1"], "exact-1") + + async def test_an_exact_key_that_is_hidden_is_unhidden(self): + """`language` was hidden behind `preferred_language`, and now the extractor + writes `language` again. The upsert lands on that hidden row through + ON CONFLICT — the id we generated is never inserted — so unless it is + un-hidden the write goes somewhere no read can see it, and the loser is + pointed at a row that does not exist. + """ + conn = FakeConnection() + conn.enqueue([ + FakeRecord(id="exact-1", key="language", merged_into="live-1"), + FakeRecord(id="live-1", key="preferred_language", merged_into=None), + ]) + conn.enqueue("UPDATE 1") # un-hide the row the upsert will land on + conn.enqueue("UPDATE 1") # hide the spelling that was live + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="language", + value="ru", dedupe=True) + + unhide, mark = conn.calls[1], conn.calls[2] + assert "SET merged_into = NULL" in unhide[1] + assert unhide[2] == (["exact-1"],) + assert mark[2] == (["live-1"], "exact-1") + + async def test_a_hidden_row_is_not_hidden_twice(self): + """The losers of an earlier merge are already behind something; only the + live spellings are the ones this write has to hide.""" + conn = FakeConnection() + conn.enqueue([ + FakeRecord(id="hidden-1", key="response_language", merged_into="live-1"), + FakeRecord(id="live-1", key="preferred_language", merged_into=None), + ]) + conn.enqueue("UPDATE 1") + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="language", value="ru", + dedupe=True) + _, _, args = conn.calls[1] + assert args[0] == ["live-1"] # hidden-1 is re-pointed by the second clause + + async def test_a_different_key_is_left_alone(self): + conn = FakeConnection() + conn.enqueue([FakeRecord(id="old-1", key="dark", merged_into=None)]) + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="preferences", key="dark_ui", value="true", + dedupe=True) + assert [c[0] for c in conn.calls] == ["fetch", "execute"] + + async def test_candidates_are_scoped_to_user_and_category(self): + conn = FakeConnection() + conn.enqueue([]) + conn.enqueue("INSERT 0 1") + store = make_store_with_pool(conn) + await store.upsert_fact(category="technical", key="os", value="Arch", + user_id="u1", dedupe=True) + _, query, args = conn.calls[0] + assert "user_id IS NOT DISTINCT FROM $1" in query + assert args[:2] == ("u1", "technical") + + +class TestMergeFilterOnReads: + """A merged loser that some read can still see is no better than a deleted row + nobody can undelete — the mark only means something while it hides the row. + """ + + async def test_vector_search_hides_merged(self): + conn = FakeConnection() + conn.enqueue([FakeRecord(id="1", category="profile", key="name", value="Eugene", + updated_at=None, source="conversation", confidence=90, + expires_at=None, source_context="", distance=0.1)]) + store = make_store_with_pool(conn) + store._pgvector_checked = True + store._pgvector_available = True + store._embedding_backend = object() + + async def _fake_embed(text: str): + return [0.1] * 768 + + store._generate_embedding = _fake_embed + await store.search_facts("name") + assert "merged_into IS NULL" in conn.calls[0][1] + + async def test_ilike_fallback_hides_merged_and_expired(self): + """This branch was also the one read path that served expired facts.""" + conn = FakeConnection() + conn.enqueue(100) # fact_count, above the auto-dump threshold + conn.enqueue([]) + conn.enqueue([]) + store = make_store_with_pool(conn) + store._pgvector_checked = True + store._pgvector_available = False + await store.search_facts("eugene") + for call in (conn.calls[1], conn.calls[2]): + assert "merged_into IS NULL" in call[1] + assert "expires_at" in call[1] + + async def test_get_all_facts_hides_merged_unless_asked(self): + conn = FakeConnection() + conn.enqueue([]) + store = make_store_with_pool(conn) + await store.get_all_facts() + assert "merged_into IS NULL" in conn.calls[0][1] + + conn2 = FakeConnection() + conn2.enqueue([]) + store2 = make_store_with_pool(conn2) + await store2.get_all_facts(include_merged=True) + # The column is selected either way — the admin route needs to see the mark + # to undo a merge. Only the WHERE changes. + assert "merged_into" in conn2.calls[0][1] + assert "merged_into IS NULL" not in conn2.calls[0][1] + + async def test_categories_hide_merged(self): + conn = FakeConnection() + conn.enqueue([]) + store = make_store_with_pool(conn) + await store.get_categories() + assert "merged_into IS NULL" in conn.calls[0][1] + + async def test_backfill_skips_merged(self): + conn = FakeConnection() + conn.enqueue([]) + store = make_store_with_pool(conn) + await store.backfill_embeddings() + assert "merged_into IS NULL" in conn.calls[0][1] + + class TestEmbeddingInputLength: """An input past the model's window came back as a 400 and took the whole embedding with it — 41 reaction-session prompts did that on prod, each one @@ -141,25 +383,41 @@ class TestDeleteFact: async def test_by_key(self): conn = FakeConnection() + conn.enqueue("DELETE 0") # hidden aliases of the key conn.enqueue("DELETE 1") store = make_store_with_pool(conn) count = await store.delete_fact("name") assert count == 1 - assert "DELETE FROM memory_facts" in conn.calls[0][1] + assert "DELETE FROM memory_facts" in conn.calls[1][1] async def test_by_key_returns_zero(self): conn = FakeConnection() conn.enqueue("DELETE 0") + conn.enqueue("DELETE 0") store = make_store_with_pool(conn) count = await store.delete_fact("missing") assert count == 0 async def test_by_key_and_category(self): conn = FakeConnection() + conn.enqueue("DELETE 0") conn.enqueue("DELETE 0 1") store = make_store_with_pool(conn) count = await store.delete_fact("name", category="profile") - assert "category" in conn.calls[0][1] + assert "category" in conn.calls[1][1] + + async def test_purges_the_aliases_of_the_deleted_key(self): + """A merge loser points at its winner. Deleting the winner alone would leave + those rows hidden with nothing to point at and no way back to them. + """ + conn = FakeConnection() + conn.enqueue("DELETE 2") + conn.enqueue("DELETE 1") + store = make_store_with_pool(conn) + await store.delete_fact("language", category="preferences") + op, query, _args = conn.calls[0] + assert op == "execute" + assert "merged_into IN (SELECT id FROM memory_facts" in query class TestGetAllFacts: @@ -204,7 +462,16 @@ conn.enqueue(100) store = make_store_with_pool(conn) assert await store.fact_count(all_users=True) == 100 - assert "WHERE" not in conn.calls[0][1] + # No user filter, but merged losers are still counted out of the live total. + assert "user_id" not in conn.calls[0][1] + assert "merged_into IS NULL" in conn.calls[0][1] + + async def test_include_merged(self): + conn = FakeConnection() + conn.enqueue(100) + store = make_store_with_pool(conn) + assert await store.fact_count(include_merged=True) == 100 + assert "merged_into" not in conn.calls[0][1] class TestSummary: