Newer
Older
gnexus-book / server / README.md
@Eugene Sukhodolskiy Eugene Sukhodolskiy 1 day ago 11 KB docs: document local gnexus-auth testing

Gnexus Book Server

FastAPI backend and MCP server for reading and maintaining Gnexus Book documentation and inventory.

Development

From the repository root:

cd server
python3 -m venv .venv
. .venv/bin/activate
pip install -e ".[dev]"
uvicorn app.main:app --reload --host 127.0.0.1 --port 8080

PostgreSQL storage

The read path can be served from PostgreSQL instead of files:

# From the repository root: start a local development cluster on 127.0.0.1:5433
./scripts/dev-db.sh start

cd server
export GNEXUS_BOOK_DATABASE_URL="$(../scripts/dev-db.sh url)"
.venv/bin/alembic upgrade head
.venv/bin/python -m app.storage.cli import --replace
.venv/bin/python -m app.storage.cli parity

GNEXUS_BOOK_STORAGE_BACKEND=db .venv/bin/python -m uvicorn app.main:app --port 3005
  • The default backend is file; set GNEXUS_BOOK_STORAGE_BACKEND=db to use PostgreSQL.
  • import --replace re-imports the current repository; parity compares file and database read results and exits non-zero on a difference.
  • scripts/backup-db.sh writes a custom-format dump to the off-host NFS by default; scripts/restore-db.sh requires GNEXUS_BOOK_RESTORE_YES=1 before replacing data.
  • Documents and inventory are read and written through PostgreSQL. Every write creates immutable versions grouped into a changeset; applying a changeset publishes it. History, version reads, diff, and restore are available in database mode.
  • Validation and freshness use the database when the backend is db.
  • In database mode POST /commit is blocked: applying a changeset is the publish step. commit_changes in MCP is a documented no-op.
  • Database tests use GNEXUS_BOOK_TEST_DATABASE_URL (default postgresql+psycopg://gmikcon@127.0.0.1:5433/gnexus_book_test) and are skipped when PostgreSQL is unavailable.

Migration and cutover

python -m app.storage.cli migrate --replace --manifest /tmp/report.json imports the file repository, validates the database, verifies the content manifest, and reconciles documents, search, inventory, relationships, validation and freshness. It exits non-zero on any discrepancy.

For a rehearsed or real cutover use the scripts:

# Rehearsal / cutover against a target database
GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://USER@HOST:PORT/gnexus_book \
  ./scripts/cutover-db.sh --replace

# Verify that the file backend is still readable for rollback
./scripts/rollback-cutover.sh

cutover-db.sh applies migrations, imports, verifies, reconciles and writes an off-host dump. It does not switch the running service: set GNEXUS_BOOK_STORAGE_BACKEND=db (and the database URL) in server/.env, then restart the backend and MCP units. The systemd units load server/.env through EnvironmentFile, and the Git checkout remains untouched as the rollback source. See ../30-runbooks/cutover-postgres.md for the full procedure.

Authentication (gnexus-auth)

Authentication is disabled unless the gnexus-auth client is configured. When the following variables are present, the service guards every route except /health, /me, /auth/*, and /webhooks/gnexus-auth*:

GNEXUS_BOOK_AUTH_CLIENT_ID
GNEXUS_BOOK_AUTH_CLIENT_SECRET
GNEXUS_BOOK_AUTH_BASE_URL
GNEXUS_BOOK_AUTH_REDIRECT_URI
GNEXUS_BOOK_AUTH_WEBHOOK_SECRET
GNEXUS_BOOK_AUTH_SESSION_COOKIE      # default gnb_session
GNEXUS_BOOK_AUTH_COOKIE_SECURE       # default true for https redirect URIs
GNEXUS_BOOK_AUTH_SESSION_TTL_SECONDS # default 604800

Endpoints:

  • GET /me — {auth_enabled, user, locale, locale_effective}; returns user: null when there is no session.
  • GET /auth/login — redirects to gnexus-auth with OAuth PKCE.
  • GET /auth/callback — exchanges the code, mirrors the sub user, and sets an httpOnly session cookie.
  • POST /auth/logout — revokes the session and clears the cookie.
  • PATCH /me — stores the per-user locale override (en, uk, ru, or empty).
  • POST /webhooks/gnexus-auth and /webhooks/gnexus-auth/ — HMAC-verified gnexus-auth events, idempotent by event id.

Sessions are stored as SHA-256 hashes in user_sessions. Webhook signatures follow gnexus-auth's t=<timestamp>,v1=<hmac> scheme over <timestamp>.<body>.

Current scope: user mirroring, sessions, /me, webhook handling, actor attribution, MCP key management, and key revocation on user block. Role-specific publish policy is still open.

Local gnexus-auth testing

server/.env (gitignored, chmod 600) holds the runtime configuration. The local scripts load it automatically:

./scripts/dev-server.sh   # backend on 127.0.0.1:3005, database + SSO
./scripts/dev-mcp.sh      # MCP streamable HTTP on 127.0.0.1:8001

The systemd units load the same file via EnvironmentFile. The local client, test user, verification steps and production differences are documented in ../30-runbooks/local-auth-testing.md.

MCP Server

The MCP server exposes the same repository, inventory, validation, pending-change, and Git commit logic through MCP tools and resources.

It should be treated by AI agents as infrastructure memory. Agents should consult it proactively for infrastructure-specific work rather than waiting for the user to explicitly ask them to "look in the knowledge base".

Production agents use streamable HTTP at /mcp-protocol/ with a personal mcp_* key:

cd server
GNEXUS_BOOK_MCP_HOST=0.0.0.0 GNEXUS_BOOK_MCP_PORT=8001 .venv/bin/python -m app.mcp_http

python -m app.mcp_server still runs stdio for local development and tests; SSE is a legacy fallback only. GNEXUS_BOOK_MCP_TRANSPORT=streamable-http delegates to the same guarded HTTP app.

Environment:

  • GNEXUS_BOOK_REPO_ROOT=/path/to/gnexus-book
  • GNEXUS_BOOK_MCP_TRANSPORT=stdio|sse|streamable-http
  • GNEXUS_BOOK_MCP_HOST, GNEXUS_BOOK_MCP_PORT
  • GNEXUS_BOOK_MCP_TOKEN — optional static break-glass token, never a fallback for personal keys
  • GNEXUS_BOOK_MCP_REQUEST_RATE_LIMIT — per-key requests per minute (default 300)
  • GNEXUS_BOOK_INTERNAL_KEY_SECRET, GNEXUS_BOOK_INTERNAL_ALLOWLIST — provisioning API protection

Personal keys and provisioning

  • mcp_tokens stores only a SHA-256 hash and a tail hint; one active key per (user_id, consumer), revocation is revoked_at.
  • Mutating tools accept is_user; it changes actor_kind from agent to user while keeping the key label and via=mcp.
  • A read-only (scopes without write) key receives {"error": 403, ...} as data.

Session-protected key management:

  • GET /mcp-keys (admins may pass ?all=true)
  • POST /mcp-keys — plaintext returned once
  • POST /mcp-keys/{key_id}/rotate
  • DELETE /mcp-keys/{key_id}

Service-to-service provisioning (not exposed through nginx; requires X-Service-Key/Bearer service secret and allowlisted source):

  • POST /internal/keys
  • GET /internal/keys?user_id=&consumer=
  • POST /internal/keys/{key_id}/rotate
  • DELETE /internal/keys/{key_id}

Core tools:

  • get_agent_usage_guide()
  • search_docs(query)
  • read_doc(path)
  • list_docs()
  • list_inventory(inventory_type?)
  • get_inventory_item(inventory_type, item_id)
  • get_relationships()
  • validate_repository()
  • check_freshness()
  • git_status()
  • git_diff(files?)
  • propose_doc_change(path, content, summary, reason?, is_user?)
  • propose_inventory_item_change(inventory_type, item_id, patch, summary, reason?, mode?, is_user?)
  • apply_pending_change(change_id, is_user?)
  • commit_changes(summary, files, details?, is_user?)
  • update_doc(path, content, summary, reason?, is_user?)
  • document_history(path)
  • document_version(path, version_no)
  • document_diff(path, from_version?, to_version?)
  • restore_document_version(path, version_no, summary?, is_user?)

Agent usage policy:

  • Call get_agent_usage_guide() when unsure how to use the knowledge tools.
  • Use MCP before answering questions about hosts, VPS, VMs, networks, domains, routes, services, deployments, GitBucket repositories, or smart-home infrastructure.
  • Use search_docs first for broad lookup, then read_doc, inventory tools, and get_relationships for authoritative context.
  • Use check_freshness when stale information matters.
  • Use write tools to update documentation after discovering relevant new facts.
  • Never store raw passwords, tokens, private keys, recovery codes, session cookies, or secret config values.

Detailed rules: ../90-maintenance/agent-mcp-usage.md.

Useful resources:

  • gnexus-book://docs
  • gnexus-book://agent-guide
  • gnexus-book://inventory
  • gnexus-book://relationships
  • gnexus-book://freshness
  • gnexus-book://validation
  • gnexus-book://git/status

Current API

  • Swagger UI: GET /api-docs
  • GET /health
  • GET /me
  • GET /auth/login
  • GET /auth/callback
  • POST /auth/logout
  • GET /mcp-keys
  • POST /mcp-keys
  • POST /mcp-keys/{key_id}/rotate
  • DELETE /mcp-keys/{key_id}
  • POST /internal/keys
  • GET /internal/keys
  • POST /internal/keys/{key_id}/rotate
  • DELETE /internal/keys/{key_id}
  • PATCH /me
  • POST /webhooks/gnexus-auth
  • GET /docs
  • GET /docs/read?path=...
  • GET /documents/{path}/history
  • GET /documents/{path}/versions/{version_no}
  • GET /documents/{path}/diff?from=&to=
  • POST /documents/{path}/restore/{version_no}?summary=...
  • GET /search?q=...
  • GET /inventory
  • GET /inventory/{type}
  • GET /inventory/{type}/{id}
  • GET /traffic-routes
  • GET /relationships
  • GET /health/freshness
  • GET /validate
  • GET /changes
  • GET /changes/{id}
  • POST /changes
  • POST /changes/{id}/apply
  • GET /git/status
  • GET /git/diff
  • POST /commit

Inventory parsing requires PyYAML.

Current Limitations

  • POST /changes creates pending change records in the file backend or changesets in the database backend.
  • POST /changes/{id}/apply applies kind=doc and kind=inventory-item changes transactionally after validation.
  • Database mode publishes on apply; POST /commit is only available with the file backend.
  • Authentication is gnexus-auth SSO when configured; without client credentials the service runs open for local development.
  • Role-specific publish policy is not implemented yet: any authenticated, non-blocked user may write.
  • The MCP keys page in the Vue UI is still pending; the API is available.
  • Key scopes are currently read and write; finer-grained scopes are not implemented.
  • File-backend POST /commit creates a local Git commit only. It does not push.
  • Commit requests must provide an explicit file list.
  • Validation uses JSON Schema 2020-12 for inventory records in both backends.
  • Search keeps a substring/line-match path for exact response parity; a PostgreSQL full-text index is deferred to a performance phase.

Validation

GET /validate checks:

  • schema validity for schemas/*.json;
  • required Markdown frontmatter;
  • inventory YAML against JSON Schema;
  • inventory docs links;
  • duplicate inventory item IDs;
  • common raw secret assignment patterns in Markdown, YAML, JSON, and dotenv-style files.

Inventory includes broad records for hosts, services, domains, traffic routes, endpoints, integrations, projects, databases, backups, networks, hardware, and virtual machines.

POST /changes/{id}/apply and POST /commit both rely on validation to block unsafe repository states.