# Gnexus Book Server

FastAPI backend and MCP server for reading and maintaining Gnexus Book documentation and inventory.

## Development

From the repository root:

```bash
cd server
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
uvicorn app.main:app --reload --host 127.0.0.1 --port 8080
```

## PostgreSQL storage

The read path can be served from PostgreSQL instead of files:

```bash
# From the repository root: start a local development cluster on 127.0.0.1:5433
./scripts/dev-db.sh start

cd server
export GNEXUS_BOOK_DATABASE_URL="$(../scripts/dev-db.sh url)"
.venv/bin/alembic upgrade head
.venv/bin/python -m app.storage.cli import --replace
.venv/bin/python -m app.storage.cli parity

GNEXUS_BOOK_STORAGE_BACKEND=db .venv/bin/python -m uvicorn app.main:app --port 3005
```

- The default backend is `file`; set `GNEXUS_BOOK_STORAGE_BACKEND=db` to use PostgreSQL.
- `import --replace` re-imports the current repository; `parity` compares file and
  database read results and exits non-zero on a difference.
- `scripts/backup-db.sh` writes a custom-format dump to the off-host NFS by default;
  `scripts/restore-db.sh` requires `GNEXUS_BOOK_RESTORE_YES=1` before replacing data.
- Documents and inventory are read and written through PostgreSQL. Every write creates
  immutable versions grouped into a changeset; applying a changeset publishes it.
  History, version reads, diff, and restore are available in database mode.
- Validation and freshness use the database when the backend is `db`.
- In database mode `POST /commit` is blocked: applying a changeset is the publish step.
  `commit_changes` in MCP is a documented no-op.
- Database tests use `GNEXUS_BOOK_TEST_DATABASE_URL` (default
  `postgresql+psycopg://gmikcon@127.0.0.1:5433/gnexus_book_test`) and are skipped when
  PostgreSQL is unavailable.

### Migration and cutover

`python -m app.storage.cli migrate --replace --manifest /tmp/report.json` imports the
file repository, validates the database, verifies the content manifest, and reconciles
documents, search, inventory, relationships, validation and freshness. It exits
non-zero on any discrepancy.

`python -m app.storage.cli assign-owner --user-json /path/to/owner.json` creates or
updates the owner mirror, assigns unowned documents and inventory items, and writes an
`ownership.assigned` audit entry. `ownership` prints owned/unowned counts. Historical
import authorship is preserved; ownership is stored separately in
`documents.owner_user_id` and `inventory_items.owner_user_id`.

For a rehearsed or real cutover use the scripts:

```bash
# Rehearsal / cutover against a target database
GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://USER@HOST:PORT/gnexus_book \
  ./scripts/cutover-db.sh --replace

# Verify that the file backend is still readable for rollback
./scripts/rollback-cutover.sh
```

`cutover-db.sh` applies migrations, imports, verifies, reconciles and writes an
off-host dump. It does not switch the running service: set
`GNEXUS_BOOK_STORAGE_BACKEND=db` (and the database URL) in `server/.env`, then restart
the backend and MCP units. The systemd units load `server/.env` through
`EnvironmentFile`, and the Git checkout remains untouched as the rollback source.
See `../30-runbooks/cutover-postgres.md` for the full procedure.

### Operations

- `scripts/backup-db.sh` writes an off-host custom dump, verifies it, applies
  retention and alerts through Synapse on failure when `SYNAPSE_URL` /
  `SYNAPSE_API_KEY` are configured.
- `gnexus-book-backup.timer` runs the backup daily; the monthly
  `gnexus-book-restore-drill.timer` is enabled after a successful manual drill.
- `scripts/restore-drill.sh [dump]` restores into a temporary database, validates it
  and drops it.
- `scripts/healthcheck.sh` probes `GNEXUS_BOOK_HEALTH_URL` (default
  `http://127.0.0.1:8000/health`), checks `status` and `checks.db`, and checks
  `GNEXUS_BOOK_DISK_PATH` against the warning and critical percentage thresholds.
- `gnexus-book-healthcheck.timer` runs the probe every five minutes.
- `scripts/notify-synapse.sh` sends a Synapse event envelope; it is used by the backup,
  restore-drill and healthcheck failure paths.
- `deploy.sh` sources `server/.env`, runs `alembic upgrade head` when the `db` backend
  is active, refreshes systemd units and restarts backend, MCP and frontend. Set
  `GNEXUS_BOOK_SKIP_KIT_PULL=1` when the sibling UI kit is bundle-managed and has no
  GitBucket credentials.

`GNEXUS_BOOK_FILE_BACKEND_FROZEN=true` freezes file-backend writes after a database
cutover; create/apply/delete and Git commit then fail with a clear error while reads
remain available for rollback.

Full procedure: `../30-runbooks/postgres-operations.md`.

## Authentication (gnexus-auth)

Authentication is disabled unless the gnexus-auth client is configured. When the
following variables are present, the service guards every route except `/health`,
`/me`, `/auth/*`, and `/webhooks/gnexus-auth*`:

```text
GNEXUS_BOOK_AUTH_CLIENT_ID
GNEXUS_BOOK_AUTH_CLIENT_SECRET
GNEXUS_BOOK_AUTH_BASE_URL
GNEXUS_BOOK_AUTH_REDIRECT_URI
GNEXUS_BOOK_AUTH_WEBHOOK_SECRET
GNEXUS_BOOK_AUTH_SESSION_COOKIE      # default gnb_session
GNEXUS_BOOK_AUTH_STATE_COOKIE        # default gnb_oauth_state
GNEXUS_BOOK_AUTH_COOKIE_SECURE       # default true for https redirect URIs
GNEXUS_BOOK_AUTH_SESSION_TTL_SECONDS # default 604800
```

Endpoints:

- `GET /me` — `{auth_enabled, user, locale, locale_effective}`; returns `user: null`
  when there is no session.
- `GET /auth/login` — redirects to gnexus-auth with OAuth PKCE.
- `GET /auth/callback` — exchanges the code, mirrors the `sub` user, and sets an
  httpOnly session cookie.
- `POST /auth/logout` — revokes the session and clears the cookie.
- `PATCH /me` — stores the per-user locale override (`en`, `uk`, `ru`, or empty).
- `POST /webhooks/gnexus-auth` and `/webhooks/gnexus-auth/` — HMAC-verified
  gnexus-auth events, idempotent by event id.

Sessions are stored as SHA-256 hashes in `user_sessions`. OAuth state is bound to a
short-lived HttpOnly cookie (`GNEXUS_BOOK_AUTH_STATE_COOKIE`, default
`gnb_oauth_state`); the callback rejects a missing or mismatched state cookie. Webhook
signatures follow gnexus-auth's `t=<timestamp>,v1=<hmac>` scheme over
`<timestamp>.<body>`. The `/internal/keys` provisioning API is blocked in nginx and
rate-limited before service-secret validation.

Current scope: user mirroring, sessions, `/me`, webhook handling, actor attribution,
MCP key management, and key revocation on user block. Role-specific publish policy is
still open.

### Local gnexus-auth testing

`server/.env` (gitignored, `chmod 600`) holds the runtime configuration. The local
scripts load it automatically:

```bash
./scripts/dev-server.sh   # backend on 127.0.0.1:3005, database + SSO
./scripts/dev-mcp.sh      # MCP streamable HTTP on 127.0.0.1:8001
```

The systemd units load the same file via `EnvironmentFile`. The local client, test
user, verification steps and production differences are documented in
[../30-runbooks/local-auth-testing.md](../30-runbooks/local-auth-testing.md).

## MCP Server

The MCP server exposes the same repository, inventory, validation, pending-change, and Git commit logic through MCP tools and resources.

It should be treated by AI agents as infrastructure memory. Agents should consult it proactively for infrastructure-specific work rather than waiting for the user to explicitly ask them to "look in the knowledge base".

Production agents use **streamable HTTP** at `/mcp-protocol/` with a personal
`mcp_*` key:

```bash
cd server
GNEXUS_BOOK_MCP_HOST=0.0.0.0 GNEXUS_BOOK_MCP_PORT=8001 .venv/bin/python -m app.mcp_http
```

`python -m app.mcp_server` still runs stdio for local development and tests. SSE is not
served by the production MCP process; production clients use streamable HTTP only.
`GNEXUS_BOOK_MCP_TRANSPORT=streamable-http` delegates to the same guarded HTTP app.
`GNEXUS_BOOK_MCP_TOKEN` is an optional static break-glass token; normal access uses a
personal `mcp_*` key.

Environment:

- `GNEXUS_BOOK_REPO_ROOT=/path/to/gnexus-book`
- `GNEXUS_BOOK_MCP_TRANSPORT=stdio|sse|streamable-http`
- `GNEXUS_BOOK_MCP_HOST`, `GNEXUS_BOOK_MCP_PORT`
- `GNEXUS_BOOK_MCP_ALLOWED_HOSTS` — Host values accepted by FastMCP; include the public host behind nginx
- `GNEXUS_BOOK_MCP_ALLOWED_ORIGINS` — allowed Origin values for browser-based clients; include the public HTTPS origin
- `GNEXUS_BOOK_MCP_TOKEN` — optional static break-glass token, never a fallback for personal keys
- `GNEXUS_BOOK_MCP_REQUEST_RATE_LIMIT` — per-key requests per minute (default 300)
- `GNEXUS_BOOK_INTERNAL_KEY_SECRET`, `GNEXUS_BOOK_INTERNAL_ALLOWLIST` — provisioning API protection

### Personal keys and provisioning

- `mcp_tokens` stores only a SHA-256 hash and a tail hint; one active key per
  (`user_id`, `consumer`), revocation is `revoked_at`.
- Mutating tools accept `is_user`; it changes `actor_kind` from `agent` to `user` while
  keeping the key label and `via=mcp`.
- A read-only (`scopes` without `write`) key receives `{"error": 403, ...}` as data.

Session-protected key management:

- `GET /mcp-keys` (admins may pass `?all=true`)
- `POST /mcp-keys` — plaintext returned once
- `POST /mcp-keys/{key_id}/rotate`
- `DELETE /mcp-keys/{key_id}`

Service-to-service provisioning (not exposed through nginx; requires
`X-Service-Key`/Bearer service secret and allowlisted source):

- `POST /internal/keys`
- `GET /internal/keys?user_id=&consumer=`
- `POST /internal/keys/{key_id}/rotate`
- `DELETE /internal/keys/{key_id}`

Core tools:

- `get_agent_usage_guide()`
- `search_docs(query)`
- `read_doc(path)`
- `list_docs()`
- `list_inventory(inventory_type?)`
- `get_inventory_item(inventory_type, item_id)`
- `get_relationships()`
- `validate_repository()`
- `check_freshness()`
- `git_status()`
- `git_diff(files?)`
- `propose_doc_change(path, content, summary, reason?, is_user?)`
- `propose_inventory_item_change(inventory_type, item_id, patch, summary, reason?, mode?, is_user?)`
- `apply_pending_change(change_id, is_user?)`
- `commit_changes(summary, files, details?, is_user?)`
- `update_doc(path, content, summary, reason?, is_user?)`
- `document_history(path)`
- `document_version(path, version_no)`
- `document_diff(path, from_version?, to_version?)`
- `restore_document_version(path, version_no, summary?, is_user?)`

Agent usage policy:

- Call `get_agent_usage_guide()` when unsure how to use the knowledge tools.
- Use MCP before answering questions about hosts, VPS, VMs, networks, domains, routes, services, deployments, GitBucket repositories, or smart-home infrastructure.
- Use `search_docs` first for broad lookup, then `read_doc`, inventory tools, and `get_relationships` for authoritative context.
- Use `check_freshness` when stale information matters.
- Use write tools to update documentation after discovering relevant new facts.
- Never store raw passwords, tokens, private keys, recovery codes, session cookies, or secret config values.

Detailed rules: `../90-maintenance/agent-mcp-usage.md`.

Useful resources:

- `gnexus-book://docs`
- `gnexus-book://agent-guide`
- `gnexus-book://inventory`
- `gnexus-book://relationships`
- `gnexus-book://freshness`
- `gnexus-book://validation`
- `gnexus-book://git/status`

## Current API

- Swagger UI: `GET /api-docs`
- `GET /health`
- `GET /me`
- `GET /auth/login`
- `GET /auth/callback`
- `POST /auth/logout`
- `GET /mcp-keys`
- `POST /mcp-keys`
- `POST /mcp-keys/{key_id}/rotate`
- `DELETE /mcp-keys/{key_id}`
- `POST /internal/keys`
- `GET /internal/keys`
- `POST /internal/keys/{key_id}/rotate`
- `DELETE /internal/keys/{key_id}`
- `PATCH /me`
- `POST /webhooks/gnexus-auth`
- `GET /docs`
- `GET /docs/read?path=...`
- `GET /documents/{path}/history`
- `GET /documents/{path}/versions/{version_no}`
- `GET /documents/{path}/diff?from=&to=`
- `POST /documents/{path}/restore/{version_no}?summary=...`
- `GET /search?q=...`
- `GET /inventory`
- `GET /inventory/{type}`
- `GET /inventory/{type}/{id}`
- `GET /traffic-routes`
- `GET /relationships`
- `GET /health/freshness`
- `GET /validate`
- `GET /changes`
- `GET /changes/{id}`
- `POST /changes`
- `POST /changes/{id}/apply`
- `GET /git/status`
- `GET /git/diff`
- `POST /commit`

Inventory parsing requires `PyYAML`.

## Current Limitations

- `POST /changes` creates pending change records in the file backend or changesets in the database backend.
- After the production cutover, the file backend is frozen for writes and its legacy
  pending-change JSON was archived; database changesets are the active proposal path.
- `POST /changes/{id}/apply` applies `kind=doc` and `kind=inventory-item` changes transactionally after validation.
- Database mode publishes on apply; `POST /commit` is only available with the file backend.
- Authentication is gnexus-auth SSO when configured; without client credentials the service runs open for local development.
- Role-specific publish policy is not implemented yet: any authenticated, non-blocked user may write.
- Key scopes are currently `read` and `write`; finer-grained scopes are not implemented.
- File-backend `POST /commit` creates a local Git commit only. It does not push.
- Commit requests must provide an explicit file list.
- Validation uses JSON Schema 2020-12 for inventory records in both backends.
- Search keeps a substring/line-match path for exact response parity; a PostgreSQL
  full-text index is deferred to a performance phase.

## Validation

`GET /validate` checks:

- schema validity for `schemas/*.json`;
- required Markdown frontmatter;
- inventory YAML against JSON Schema;
- inventory `docs` links;
- duplicate inventory item IDs;
- common raw secret assignment patterns in Markdown, YAML, JSON, and dotenv-style files.

Inventory includes broad records for hosts, services, domains, traffic routes, endpoints, integrations, projects, databases, backups, networks, hardware, and virtual machines.

`POST /changes/{id}/apply` and `POST /commit` both rely on validation to block unsafe repository states.
