Todo store — plans that outlive the process

The todo tool's plan lives in its own PostgreSQL table, one row per (user_id, session_id). It used to be an opaque JSON string inside session_store (scope='todo'); this page covers why it moved and what the row now carries.

Why a separate table

The old row already survived restarts — the problem was that it could not tell a live plan from one a crashed run abandoned. Reads were untyped JSON parsed on every call, and there was no timestamp the recovery path could trust. The dedicated row adds a revision counter and an updated_at that the agent can reason about, plus room for per-step provenance that protocols will use.

Schema

CREATE TABLE session_todos (
    user_id    TEXT NOT NULL DEFAULT '',
    session_id TEXT NOT NULL,
    tasks      JSONB NOT NULL DEFAULT '[]'::jsonb,
    revision   INTEGER NOT NULL DEFAULT 0,
    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (user_id, session_id)
);

tasks is an array of objects, one per step:

Rows written before these fields existed load with their defaults, so no backfill is needed.

revision starts at 0 and increments on every save. It is not a concurrency control yet — it exists so the recovery note can fire once per plan version.

Migration

PgTodoStore creates the table and runs the migration lazily on first use (same pattern as KvStore). The migration copies every session_store row with scope='todo' into session_todos and then deletes the source rows:

• ON CONFLICT DO NOTHING makes the copy idempotent, so a plan written to the new table is never overwritten by a stale KV copy;
• deleting the source is what stops a plan the user cleared from being resurrected on the next process start.

If session_store does not exist yet, the migration is logged and skipped — the table simply starts empty. Running the old code after this migration will not see existing plans; that is the cost of the cut-over and the reason it is done in one release.

Crash recovery

When the plan reminder is built (get_progress_message), a plan with a step still in_progress whose row was last updated before this process started is flagged once:

Recovery: this plan was last changed at … and step N is still 'in_progress'. If no run is working on it right now, a previous run stopped mid-step — check the actual state … before continuing.

The note is keyed by (session_id, revision), so it appears once per plan version and not on every iteration. A plan updated by the current process gets no note. The wording is deliberately conditional ("if no run is working on it right now") because a multi-worker deployment can legitimately have another process writing the row.

Wiring

PgTodoStore(pool) is created in navi/core/container.py, stored on AppContainer.todo_store, passed into build_default_registries() and onward to TodoTool. todo.set_store() makes it the module-global backend. If no store is wired, TodoTool falls back to KvTodoStore over the shared KvStore — that path is what unit tests use and what an instance that has not migrated yet would still read.

load_tasks_for(session_id, user_id, kv=None, store=None) resolves in order: explicit store, explicit kv, module-global store. The REST endpoint GET /sessions/{id}/todos passes both (the container's store, and the KV store as a fallback), so its payload shape is unchanged.

Helpers for protocols

navi/tools/todo.py exposes two internals that the protocol work builds on:

• insert_steps(session_id, steps, after=None, source="protocol", protocol_run_id=…, step_ids=…) splices a block into the existing plan after the last step the agent reached (or after an explicit index) and returns the new 1-based indexes. Existing steps and statuses are preserved — no re-issue of the whole plan via set.
• _current_position(tasks) finds that insertion point (last done / in_progress / failed step).