| 2026-04-26 |
Add eval system Phase 5 — debug UI
...
Self-contained SPA at /debug/eval (route already wired in 8e0eed6).
Single index.html in the existing debug/ style — vanilla JS, embedded
CSS, no framework, no build step. Four tabs:
- Sessions — filterable table (profile / status / limit), eval status
pill, headline avg scores, click-through to detail
- Detail — session metadata + every stored eval run, axes laid out as
axis × expert grids with inline averages, expert comments, button to
re-evaluate this single session
- Stats — weekly per-axis means table, optional complexity-bucket split
- Run — form to trigger any scope (unevaluated / single / all), live
status panel polling /eval/run/{id} every 2.5s, run history with
click-to-attach
Hash routing: #detail/<session_id> deep-links to a session.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 26 Apr
|
Add eval system Phase 4 — read endpoints and background runner
...
REST surface for the debug UI:
- GET /eval/sessions — overview list with eval status / latest avg /
feedback counts (single SQL: sessions ⨝ feedback ⨝ latest run)
- GET /eval/sessions/{id} — session detail with all evaluations
- GET /eval/stats — weekly per-axis means; optional complexity-bucket split
- POST /eval/run — fire-and-forget background eval, returns run_id
- GET /eval/run/{id}, GET /eval/runs — poll progress and history
Pulled the runner loop out of cli into runner.py so both the CLI and
the REST endpoint share the same loop. State for in-flight runs lives
in an in-memory registry (single-process, cleared on restart).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 26 Apr
|

Add eval system Phase 3 — judge runner end to end
...
Fills in the stubs from Phase 2:
- judge.render_session: full transcript with tool_call/tool_result folding,
reactions inlined per assistant block, planning_logs appendix, no
compression-summary substitution
- judge.run_expert: real LLM call, fence-tolerant JSON parse, single retry
with corrective nudge on schema or parse error
- judge.evaluate_session: asyncio.gather across the three experts
- db.EvalDB: insert_evaluation_run (txn), list_evaluations,
evaluated_session_ids, feedback_by_index helper
- cli `run` (filters: --session, --since, --limit, --re-evaluate-all,
--dry-run, --model, --backend) and `show` (groups by eval_run_id, prints
per-expert axes plus averaged scores)
Verified end-to-end against a real 10-message secretary session:
all three experts returned valid JSON first try; spread between strict
critic and the others surfaced as expected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 26 Apr
|
Add eval system Phase 2 — rubric, expert prompts, judge skeleton
...
Drafts the v1 rubric (7 axes, anchors at 10/30/50/75/100, open scale),
three independent expert prompts (strict_critic / pragmatist / tech_lead)
that all return the same JSON shape, and the orchestration scaffolding:
schema.py (pydantic models), judge.py (rubric loader, score averaging,
fence-tolerant JSON parser, new_run_metadata), cli.py with argparse for
run / show / stats. Real LLM calls and transcript rendering land in
Phase 3 — the stubs raise NotImplementedError.
`python -m debug.eval` works as the entry point. Anchor `examples` are
left empty for now; user fills them with real session_ids later without
bumping rubric_version.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 26 Apr
|
Add eval system Phase 1 — message feedback signal
...
Spec at docs/eval_system.md describes the full LLM-as-judge plan;
this commit lands only the in-app feedback layer:
- debug/eval/ Python package with EvalDB (asyncpg) and FastAPI router
exposing /eval/feedback (set / clear / list)
- message_feedback postgres table keyed by (session_id, message_index)
- thumbs up / down on each completed assistant block in the webclient,
optimistic update with rollback on failure
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 26 Apr
|
| 2026-04-21 |
Add instagram_engine and instagram_viewer tools (Navi-generated)
...
Browser automation tools for scraping public Instagram profiles using
Playwright + stealth. Registered in enabled.json and developer profile.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 21 Apr
|
| 2026-04-20 |

Planning debug panel, todo auto-populate, scratchpad/persona improvements
...
- Planning debug panel: new Planning tab in debug/index.html shows raw
phase 1/2 outputs and token counts per planning run, stored in
session.planning_logs (new column in both SQLite and PostgreSQL)
- New GET /sessions/{id}/planning API endpoint
- PlanningDebugData internal event wires _run_planning() output into
session storage; never forwarded to WebSocket clients
- Phase 3 (plan critic) disabled — to be reworked with reflect integration
- Todo tool: auto-populated from plan steps after phase 2; model only
needs to call update/view, not set
- Scratchpad: clarified description and persona instructions; removed
context_transfer from user-facing docs (internal mechanism only)
- web_search: switched to ddgs package, SearXNG as primary backend,
DDG html-only fallback; added find_up action to filesystem tool
- Persona: added SCRATCHPAD and TODO sections with clear usage rules;
added NAVI.md project context instructions
- chat.js: fixed subagent planning event fallthrough into parent UI;
statusLabel cleared on first stream delta
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 20 Apr
|
| 2026-04-17 |
Add Prompts and Tools tabs to debug page
...
Backend:
- GET /agents/prompts — returns full built system prompt for every
profile, broken into sections (persona / profile / profiles block)
with char/token counts; mirrors Agent._build_system_prompt() exactly
- GET /agents/tools — now includes parameters schema alongside name
and description
Debug page:
- Tab bar: Context / Prompts / Tools
- Prompts tab: profile sidebar + collapsible sections per prompt part
(persona, profile prompt, profiles block), togglable tools list
- Tools tab: searchable list of all tools with description and
parameter table (name, type, description, required marker)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 17 Apr
|
Add standalone debug page at /debug
...
Replaces the old_webclient/debug.html with a proper self-contained
tool at debug/index.html. New features over the old page:
- Sidebar session list with profile, message count, pin indicator
- Auto-refresh toggle (3s interval)
- Refresh button
- Renders thinking blocks, is_plan and is_summary tags
- Shows tool call name on tool result messages
- Clickable image thumbnails (open full-size)
- All new fields from the current LLM context API
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Eugene Sukhodolskiy
committed
on 17 Apr
|