diff --git a/00-overview/postgres-migration-roadmap.md b/00-overview/postgres-migration-roadmap.md index 7f2c099..bc57005 100644 --- a/00-overview/postgres-migration-roadmap.md +++ b/00-overview/postgres-migration-roadmap.md @@ -480,6 +480,10 @@ | 2026-11-19 | Phase 4: gnexus-auth integration | OAuth 2.0 PKCE login/callback, `/me`, locale override, logout, HMAC-verified and idempotent webhooks; auth disabled when client credentials are absent | | 2026-11-19 | Phase 4: route guard and actor | Anonymous requests rejected when auth is enabled; actor recorded on changesets, document versions and inventory versions | | 2026-11-19 | Phase 4: tests | Full suite: 66 passed, 1 known stale-freshness failure | +| 2026-11-19 | Phase 5: streamable HTTP MCP | Guarded `/mcp-protocol/` endpoint via `app.mcp_http`; stdio remains for tests/local development | +| 2026-11-19 | Phase 5: personal keys | `mcp_tokens` with hash + hint, one active key per (`user_id`, `consumer`), session key API, `/internal/keys` provisioning, rate limits | +| 2026-11-19 | Phase 5: actor and revocation | Mutating tools accept `is_user`; REST accepts `X-Actor`; blocked users lose all keys; role changes mark keys stale | +| 2026-11-19 | Phase 5: tests | Full suite: 75 passed, 1 known stale-freshness failure | Notes: @@ -494,7 +498,8 @@ - Phase 4 is scaffolded without production credentials. Real login needs an OAuth client registered by the owner in gnexus-auth (`client_id`, `client_secret`, redirect URI) and its webhook secret, all supplied through `.env`/gnexus-creds. -- Role-specific publish policy and MCP key revocation on user block remain open; the - current rule is "authenticated and not blocked may write". -- The next phase is Phase 5: MCP streamable HTTP, per-user keys, provisioning, and - key revocation on user block. +- Role-specific publish policy remains open; the current rule is "authenticated and + not blocked may write". MCP keys carry `read`/`write` scopes. +- The Vue UI does not yet expose the MCP keys page; the API is ready. +- The next phase is Phase 6: UI parity, auth gate, i18n, history/diff and the MCP keys + page. diff --git a/CLAUDE.md b/CLAUDE.md index ba9b6bc..c680610 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,7 +42,14 @@ .venv/bin/python -m app.mcp_server ``` -MCP environment: `GNEXUS_BOOK_REPO_ROOT` (defaults to the repo root derived from `server/app/config.py`) and `GNEXUS_BOOK_MCP_TRANSPORT` (`stdio` | `sse` | `streamable-http`). +MCP environment: `GNEXUS_BOOK_REPO_ROOT` (defaults to the repo root derived from `server/app/config.py`) and `GNEXUS_BOOK_MCP_TRANSPORT` (`stdio` | `sse` | `streamable-http`). Production agents use streamable HTTP at `/mcp-protocol/` with a personal `mcp_*` key: + +```bash +cd server +GNEXUS_BOOK_MCP_HOST=0.0.0.0 GNEXUS_BOOK_MCP_PORT=8001 .venv/bin/python -m app.mcp_http +``` + +Keys are managed through `/mcp-keys` (session) and `/internal/keys` (service secret, not exposed by nginx); `mcp_tokens` stores only a SHA-256 hash and a tail hint, never plaintext. ### PostgreSQL read backend (Phase 1) @@ -95,7 +102,7 @@ - **`validation.py`** — the full-repo gate used before apply/commit: schemas parse as Draft 2020-12, every inventory file validates against its schema (`SCHEMA_BY_INVENTORY`), Markdown has required frontmatter, ids are unique per inventory file, and a regex scan detects raw secret assignments/private-key blocks. The separate inventory `docs`-link validator (`_validate_inventory_doc_links`) is intentionally disabled; freshness still reports those missing targets. - **`pending_changes.py`** — the write path. A proposal is a JSON file in `90-maintenance/pending-changes/`; `apply()` writes a Markdown doc or patches/creates an inventory item, runs whole-repo validation, and restores the previous file content if validation fails. Only `doc` and `inventory-item` kinds can be applied; full-file inventory replacement is not implemented. - **`git_adapter.py`** — status/diff/commit. Commit requires an explicit file list, rejects absolute/traversal/denied paths, runs validation first, stages only the named files, and creates a local commit only (never pushes). -- **`auth/`** — gnexus-auth integration: OAuth 2.0 PKCE login/callback, hashed browser sessions, user mirror, HMAC webhook handling, a pure-ASGI session guard, and the actor context used by writes. Authentication is disabled unless `GNEXUS_BOOK_AUTH_CLIENT_ID`, `..._CLIENT_SECRET`, `..._BASE_URL`, and `..._REDIRECT_URI` are set; disabled mode is intentional for local development. The guard exempts `/health`, `/me`, `/auth/*`, and `/webhooks/gnexus-auth*`. The `gnexus-gauth` client library is a pinned git dependency. +- **`auth/`** — gnexus-auth integration: OAuth 2.0 PKCE login/callback, hashed browser sessions, user mirror, HMAC webhook handling, a pure-ASGI session guard, personal MCP key management (`mcp_keys.py`, `mcp_routes.py`), the MCP bearer guard (`mcp_guard.py`), and the actor context used by writes. Authentication is disabled unless `GNEXUS_BOOK_AUTH_CLIENT_ID`, `..._CLIENT_SECRET`, `..._BASE_URL`, and `..._REDIRECT_URI` are set; disabled mode is intentional for local development. The guard exempts `/health`, `/me`, `/auth/*`, and `/webhooks/gnexus-auth*`. The `gnexus-gauth` client library is a pinned git dependency. - **`storage/`** — the PostgreSQL layer introduced for the migration. `models.py` defines the schema (documents/versions/blobs, inventory items/versions, changesets, users, audit), `db_repositories.py` implements the read interfaces, `pending_changes_db.py` implements draft/apply changesets transactionally, `documents.py` provides history/version/diff/restore, `validation.py` validates the database, `importer.py` copies the repository into PostgreSQL, `parity.py` compares both backends, and `cli.py` exposes `import`, `parity`, and `counts`. `factory.py` selects docs/inventory/pending repositories from `GNEXUS_BOOK_STORAGE_BACKEND`. Alembic lives in `server/alembic/`. In `db` mode all knowledge writes go through changesets; `commit_changes` is a no-op. Validation, freshness, and relationships use the selected backend. ### MCP server @@ -121,6 +128,6 @@ Tests run from `server/` with `pythonpath = ["."]`. Most tests read the real repository, so they assert on real inventory/docs content; fixture-based tests use `Settings(tmp_path)` and copy schemas from the real repo. -Database tests use `GNEXUS_BOOK_TEST_DATABASE_URL` (default `postgresql+psycopg://gmikcon@127.0.0.1:5433/gnexus_book_test`) and skip when PostgreSQL is unavailable. `tests/test_db_storage.py` covers read parity, `tests/test_db_changesets.py` covers versioning/changesets/history/diff/restore, `tests/test_auth.py` covers login, sessions, `/me`, the route guard and signed webhooks with mocked gnexus-auth HTTP calls, and `tests/test_migrations.py` applies the Alembic migration to a temporary database. +Database tests use `GNEXUS_BOOK_TEST_DATABASE_URL` (default `postgresql+psycopg://gmikcon@127.0.0.1:5433/gnexus_book_test`) and skip when PostgreSQL is unavailable. `tests/test_db_storage.py` covers read parity, `tests/test_db_changesets.py` covers versioning/changesets/history/diff/restore, `tests/test_auth.py` covers login, sessions, `/me`, the route guard and signed webhooks with mocked gnexus-auth HTTP calls, `tests/test_mcp_keys.py` covers internal provisioning, the user key API, the MCP bearer guard, rate limits, actor selection and key revocation on user block, and `tests/test_migrations.py` applies the Alembic migration to a temporary database. `tests/test_docs_repository.py::test_freshness_report_is_clean_for_knowledge_docs` is date-dependent: it fails once documents pass their `review_interval`. When it fails, refresh `last_reviewed` only for docs whose content was actually reviewed, or treat the failure as genuine staleness. diff --git a/server/README.md b/server/README.md index a94239d..71174b7 100644 --- a/server/README.md +++ b/server/README.md @@ -88,17 +88,49 @@ It should be treated by AI agents as infrastructure memory. Agents should consult it proactively for infrastructure-specific work rather than waiting for the user to explicitly ask them to "look in the knowledge base". -Run over stdio: +Production agents use **streamable HTTP** at `/mcp-protocol/` with a personal +`mcp_*` key: ```bash cd server -.venv/bin/python -m app.mcp_server +GNEXUS_BOOK_MCP_HOST=0.0.0.0 GNEXUS_BOOK_MCP_PORT=8001 .venv/bin/python -m app.mcp_http ``` -Optional environment: +`python -m app.mcp_server` still runs stdio for local development and tests; SSE is a +legacy fallback only. `GNEXUS_BOOK_MCP_TRANSPORT=streamable-http` delegates to the same +guarded HTTP app. + +Environment: - `GNEXUS_BOOK_REPO_ROOT=/path/to/gnexus-book` - `GNEXUS_BOOK_MCP_TRANSPORT=stdio|sse|streamable-http` +- `GNEXUS_BOOK_MCP_HOST`, `GNEXUS_BOOK_MCP_PORT` +- `GNEXUS_BOOK_MCP_TOKEN` — optional static break-glass token, never a fallback for personal keys +- `GNEXUS_BOOK_MCP_REQUEST_RATE_LIMIT` — per-key requests per minute (default 300) +- `GNEXUS_BOOK_INTERNAL_KEY_SECRET`, `GNEXUS_BOOK_INTERNAL_ALLOWLIST` — provisioning API protection + +### Personal keys and provisioning + +- `mcp_tokens` stores only a SHA-256 hash and a tail hint; one active key per + (`user_id`, `consumer`), revocation is `revoked_at`. +- Mutating tools accept `is_user`; it changes `actor_kind` from `agent` to `user` while + keeping the key label and `via=mcp`. +- A read-only (`scopes` without `write`) key receives `{"error": 403, ...}` as data. + +Session-protected key management: + +- `GET /mcp-keys` (admins may pass `?all=true`) +- `POST /mcp-keys` — plaintext returned once +- `POST /mcp-keys/{key_id}/rotate` +- `DELETE /mcp-keys/{key_id}` + +Service-to-service provisioning (not exposed through nginx; requires +`X-Service-Key`/Bearer service secret and allowlisted source): + +- `POST /internal/keys` +- `GET /internal/keys?user_id=&consumer=` +- `POST /internal/keys/{key_id}/rotate` +- `DELETE /internal/keys/{key_id}` Core tools: @@ -113,15 +145,15 @@ - `check_freshness()` - `git_status()` - `git_diff(files?)` -- `propose_doc_change(path, content, summary, reason?)` -- `propose_inventory_item_change(inventory_type, item_id, patch, summary, reason?, mode?)` -- `apply_pending_change(change_id)` -- `commit_changes(summary, files, details?)` -- `update_doc(path, content, summary, reason?)` +- `propose_doc_change(path, content, summary, reason?, is_user?)` +- `propose_inventory_item_change(inventory_type, item_id, patch, summary, reason?, mode?, is_user?)` +- `apply_pending_change(change_id, is_user?)` +- `commit_changes(summary, files, details?, is_user?)` +- `update_doc(path, content, summary, reason?, is_user?)` - `document_history(path)` - `document_version(path, version_no)` - `document_diff(path, from_version?, to_version?)` -- `restore_document_version(path, version_no, summary?)` +- `restore_document_version(path, version_no, summary?, is_user?)` Agent usage policy: @@ -152,6 +184,14 @@ - `GET /auth/login` - `GET /auth/callback` - `POST /auth/logout` +- `GET /mcp-keys` +- `POST /mcp-keys` +- `POST /mcp-keys/{key_id}/rotate` +- `DELETE /mcp-keys/{key_id}` +- `POST /internal/keys` +- `GET /internal/keys` +- `POST /internal/keys/{key_id}/rotate` +- `DELETE /internal/keys/{key_id}` - `PATCH /me` - `POST /webhooks/gnexus-auth` - `GET /docs` @@ -185,7 +225,8 @@ - Database mode publishes on apply; `POST /commit` is only available with the file backend. - Authentication is gnexus-auth SSO when configured; without client credentials the service runs open for local development. - Role-specific publish policy is not implemented yet: any authenticated, non-blocked user may write. -- MCP personal keys and key revocation on user block are not implemented yet. +- The MCP keys page in the Vue UI is still pending; the API is available. +- Key scopes are currently `read` and `write`; finer-grained scopes are not implemented. - File-backend `POST /commit` creates a local Git commit only. It does not push. - Commit requests must provide an explicit file list. - Validation uses JSON Schema 2020-12 for inventory records in both backends.