Testing Strategy

Backend stack

• pytest + pytest-asyncio (asyncio_mode = auto)
• pytest-mock — mocker fixture
• httpx — TestClient for FastAPI routes
• asgi-lifespan — lifespan management in integration tests

Web client stack

• Vitest + @vue/test-utils — Vue 3 component and composable testing
• happy-dom — lightweight DOM environment
• Pinia testing utils — @pinia/testing for store mocks

Directory layout

tests/                          # Backend (pytest)
├── conftest.py
├── conftest_factory.py
├── unit/
│   ├── api/
│   │   └── test_session_files.py # upload/download file endpoint logic
│   ├── core/
│   │   ├── test_events.py         # 17 tests
│   │   ├── test_context_builder.py
│   │   ├── test_compressor.py
│   │   ├── test_registry.py       # registries, backend discovery, context provider registry
│   │   ├── test_planning.py
│   │   ├── test_agent_context_size.py
│   │   └── test_agent_stream_guard.py
│   ├── llm/
│   │   └── test_ollama.py         # timeout/error classification + fallback timeout wiring
│   ├── memory/
│   │   ├── test_store.py
│   │   └── test_extractor.py
│   ├── tools/
│   │   ├── test_filesystem.py
│   │   ├── test_code_exec.py
│   │   ├── test_terminal.py
│   │   ├── test_share_file.py
│   │   └── test_content_publish.py
│   ├── profiles/
│   │   └── test_base.py
│   ├── config/
│       └── test_settings.py
│   ├── test_content_store.py
│   └── test_startup.py
├── integration/
│   ├── conftest.py
│   ├── test_api_routes.py
│   └── test_websocket.py
└── e2e/
    └── test_chat_flow.py

webclient/tests/                # Web client (Vitest)
├── unit/
│   ├── api/
│   │   └── index.test.js          # 8 tests — request helper, verbs, errors, FormData
│   ├── stores/
│   │   ├── chat.test.js           # 23 tests — buildMessageList, WS handlers, session load
│   │   ├── sessions.test.js       # 6 tests — fetch, create, delete, pin sorting
│   │   └── profiles.test.js       # 3 tests — fetch, selection, lookup
│   └── composables/
│       └── useWebSocket.test.js   # 7 tests — connect, dispatch, reconnect

Mock strategy

LLM — FakeLLMBackend

FakeLLMBackend cycles through a list of pre-defined responses and optionally emits ToolCallRequest objects. This lets us test the agent loop and planning without real Ollama.

from tests.conftest_factory import FakeLLMBackend

backend = FakeLLMBackend(
    responses=["Hello", "DIRECT"],
    tool_calls=[None, None],
    thinking=["Hmm", None],
)
resp = await backend.complete([])  # → LLMResponse(content="Hello")

PostgreSQL — FakePool / FakeConnection

Unit tests mock asyncpg.Pool via an in-memory FakePool/FakeConnection. Integration tests may use a real Postgres instance via TEST_DATABASE_URL.

from tests.conftest_factory import FakeConnection, FakeRecord, make_store_with_pool

conn = FakeConnection()
conn.enqueue(42)  # fetchval result
conn.enqueue([FakeRecord(id="1", key="name", value="Eugene")])  # fetch result
store = make_store_with_pool(conn)

Coverage status

Coverage roadmap

This is the living plan for what still needs tests. Keep it updated whenever a bug is fixed, a new module is added, or a planned area becomes covered.

Status meanings:

• ✅ Covered enough for current risk
• 🟨 Basic coverage exists, important edge cases remain
• ⬜ Not covered yet
• 🔴 Regression target from a real bug

Phase 7 — Recent Regression Coverage

Phase 8 — Agent Loop Behavior

Phase 9 — Memory And Embeddings

Phase 10 — WebSocket And API Lifecycles

Phase 11 — Frontend

Running tests

# Backend tests
pytest                              # all backend tests
pytest tests/unit                   # unit only
pytest -v tests/unit/core           # verbose
pytest -v tests/unit/core/test_events.py::TestToolStarted::test_to_wire  # single test
TEST_DATABASE_URL=postgresql://... pytest tests/integration

# Web client tests (run from webclient/)
cd webclient && npm test             # all webclient tests
npx vitest run tests/unit/api       # single directory
npx vitest run -t "buildMessageList" # filter by test name

Adding a new test

1. Create file in the appropriate tests/unit/ or tests/integration/ directory.
2. Use async def for async tests — pytest-asyncio handles the rest.
3. Import helpers from tests.conftest_factory for fakes.
4. Mutations to navi.config.settings are reset automatically by the autouse fixture in conftest.py.

Guidelines

• Mock at boundaries: LLM calls → FakeLLMBackend, DB → FakePool, filesystem → tmp_path.
• Avoid real network: Never hit Ollama, OpenAI, or DuckDuckGo in unit tests.
• Avoid real DB in unit tests: Use in-memory mocks; real Postgres only in tests/integration/.
• Keep tests deterministic: No randomness, no time-dependent logic without monkeypatching datetime.now.