diff --git a/00-overview/postgres-migration-roadmap.md b/00-overview/postgres-migration-roadmap.md index f2bcba8..d612cdc 100644 --- a/00-overview/postgres-migration-roadmap.md +++ b/00-overview/postgres-migration-roadmap.md @@ -487,6 +487,11 @@ | 2026-11-19 | Phase 6: shell and auth gate | Drawer navigation, `GnPageHeader` on every page, gnexus-auth gate, settings, logout | | 2026-11-19 | Phase 6: pages and i18n | Docs search/history/diff/restore, inventory JSON/raw, changesets/apply, status reports, MCP keys page, en/uk/ru | | 2026-11-19 | Phase 6: verification | Production build plus headless Chromium smoke in file mode, database mode and auth-gate mode; no browser console errors | +| 2026-11-19 | Phase 7: importer v1 | Source revision and content-manifest digest, post-import validation, database manifest verification, drift detection | +| 2026-11-19 | Phase 7: reconciliation | File/DB comparison now covers documents, search, inventory, relationships, validation and freshness | +| 2026-11-19 | Phase 7: cutover tooling | `cutover-db.sh`, `rollback-cutover.sh`, `30-runbooks/cutover-postgres.md`, systemd `EnvironmentFile` | +| 2026-11-19 | Phase 7: rehearsal | Fresh `gnexus_book_cutover`: 30 documents, manifest match, 0 reconciliation differences, off-host dump, API smoke, file rollback verified; rehearsal database dropped | +| 2026-11-19 | Phase 7: tests | Full suite: 78 passed, 1 known stale-freshness failure | Notes: @@ -505,5 +510,12 @@ not blocked may write". MCP keys carry `read`/`write` scopes. - The Vue UI now covers the current functionality, the auth gate, i18n and the MCP keys page. Real gnexus-auth credentials are still required to exercise live login. -- The next phase is Phase 7: importer v1, dual-run reconciliation, cutover rehearsal - and the final migration of the existing knowledge base. +- Phase 7 tooling and the cutover rehearsal are complete. The actual production flip + is a deployment action: set `GNEXUS_BOOK_STORAGE_BACKEND=db` and the database URL in + `server/.env`, restart the backend and MCP units, then run the post-cutover checks in + [cutover-postgres.md](../30-runbooks/cutover-postgres.md). The Git checkout is + untouched and remains the rollback source. +- The retained cutover evidence is the migration report under + `~/backup/gnexus-book/` and the off-host dump on NetTera8. +- The next step is connecting the local gnexus-auth instance (the same system as + production) and verifying login, callback, webhooks and actor attribution end to end. diff --git a/30-runbooks/cutover-postgres.md b/30-runbooks/cutover-postgres.md new file mode 100644 index 0000000..dc3945b --- /dev/null +++ b/30-runbooks/cutover-postgres.md @@ -0,0 +1,88 @@ +--- +owner: gmikcon +status: draft +last_reviewed: 2026-11-19 +review_interval: 90d +confidence: high +source_of_truth: migration-runbook +--- + +# PostgreSQL cutover runbook + +Move Gnexus Book from the file backend to PostgreSQL. The Git checkout is never +modified by this procedure; it remains the rollback source until the cutover is +accepted. + +## Preconditions + +- The pre-cutover backup has been copied off-host and restored at least once. +- `validate_repository` is clean on the file backend. +- The target PostgreSQL database exists and is reachable. +- The target is either empty or the operator explicitly passes `--replace`. +- The owner has chosen a maintenance window and confirmed the cutover. +- Real gnexus-auth credentials are configured before exposing the service outside + the LAN; database cutover and authentication cutover can be separate steps. + +## Rehearsal + +Always rehearse on a disposable database first: + +```bash +createdb gnexus_book_cutover +GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://USER@HOST:PORT/gnexus_book_cutover \ + ./scripts/cutover-db.sh --replace +``` + +The script runs the importer, validates the imported database, reconciles file and +database reads (documents, search, inventory, relationships, validation, freshness), +and writes a migration report. It must exit `0`. + +## Cutover + +1. Announce the maintenance window and the rollback plan. +2. Freeze writes to the file backend (stop the backend service or put it in + maintenance mode). +3. Run the cutover against the target database: + + ```bash + GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://USER@HOST:PORT/gnexus_book \ + ./scripts/cutover-db.sh --replace + ``` + +4. Set the service environment: + + ```text + GNEXUS_BOOK_STORAGE_BACKEND=db + GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://... + ``` + +5. Restart `gnexus-book-backend` and `gnexus-book-mcp`. +6. Run the post-cutover checks below. +7. Unfreeze writes and announce completion. + +## Post-cutover checks + +- `GET /health` reports `storage: db` and `checks.db.status: ok`. +- `GET /docs`, `/search`, `/inventory`, `/relationships`, `/validate`, + `/health/freshness` return the expected data. +- `GET /documents//history` returns imported version 1. +- A test changeset can be proposed and applied, with validation passing. +- The UI loads documents, history, inventory and status. +- A fresh off-host dump exists and its checksum verifies. + +## Rollback + +The file backend remains intact. To roll back: + +1. Set `GNEXUS_BOOK_STORAGE_BACKEND=file`. +2. Restart `gnexus-book-backend` and `gnexus-book-mcp`. +3. Run `scripts/rollback-cutover.sh` to verify file validation and document reads. +4. If the database must be restored separately, use `scripts/restore-db.sh` with the + pre-cutover dump and `GNEXUS_BOOK_RESTORE_YES=1`. + +## After acceptance + +- Keep the pre-cutover dump and the Git archive for the agreed retention period. +- Remove the file write path only after the database backend has run without incident. +- Record the cutover in gnexus-book and update the service page with backup and + restore instructions. diff --git a/CLAUDE.md b/CLAUDE.md index 975971c..efe9e56 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,6 +62,11 @@ .venv/bin/alembic upgrade head .venv/bin/python -m app.storage.cli import --replace .venv/bin/python -m app.storage.cli parity # exits 1 on a file/DB difference +.venv/bin/python -m app.storage.cli migrate --replace --manifest /tmp/report.json + +# Rehearsed/real cutover and rollback verification +../scripts/cutover-db.sh --replace +../scripts/rollback-cutover.sh GNEXUS_BOOK_STORAGE_BACKEND=db .venv/bin/python -m uvicorn app.main:app --port 3005 ``` diff --git a/server/README.md b/server/README.md index 71174b7..60c23df 100644 --- a/server/README.md +++ b/server/README.md @@ -46,6 +46,31 @@ `postgresql+psycopg://gmikcon@127.0.0.1:5433/gnexus_book_test`) and are skipped when PostgreSQL is unavailable. +### Migration and cutover + +`python -m app.storage.cli migrate --replace --manifest /tmp/report.json` imports the +file repository, validates the database, verifies the content manifest, and reconciles +documents, search, inventory, relationships, validation and freshness. It exits +non-zero on any discrepancy. + +For a rehearsed or real cutover use the scripts: + +```bash +# Rehearsal / cutover against a target database +GNEXUS_BOOK_DATABASE_URL=postgresql+psycopg://USER@HOST:PORT/gnexus_book \ + ./scripts/cutover-db.sh --replace + +# Verify that the file backend is still readable for rollback +./scripts/rollback-cutover.sh +``` + +`cutover-db.sh` applies migrations, imports, verifies, reconciles and writes an +off-host dump. It does not switch the running service: set +`GNEXUS_BOOK_STORAGE_BACKEND=db` (and the database URL) in `server/.env`, then restart +the backend and MCP units. The systemd units load `server/.env` through +`EnvironmentFile`, and the Git checkout remains untouched as the rollback source. +See `../30-runbooks/cutover-postgres.md` for the full procedure. + ## Authentication (gnexus-auth) Authentication is disabled unless the gnexus-auth client is configured. When the