diff --git a/10-systems/services/gnexus-creds.md b/10-systems/services/gnexus-creds.md index 15c2a96..28584c5 100644 --- a/10-systems/services/gnexus-creds.md +++ b/10-systems/services/gnexus-creds.md @@ -17,7 +17,7 @@ - **Port**: 80 (External) $\to$ 8000 (Internal) - **Database**: PostgreSQL 17 (Containerized) - **Repository**: https://git.gnexus.space/git/root/gnexus-creds.git (branch master) -- **Current version**: 53acfb4 (2026-10-03, updated by rei; Bearer auth fails closed — a dead token no longer rides the session cookie; previous 287b7a0: session death drops the SPA onto the sign-in screen without a reload; 27f95b4: webhook fixes — trailing-slash target, 401 on bad signature, global_logout ends sessions, revoked tokens visible in popup + tokens table, extension 0.2.2 + design pack v24; 543d6ae: splash on sign-in, logout to internal sign-in, extension 0.2.1; 4c6a4f7 2026-10-01: PWA shell, biometric reveal gate, ui-kit 1.0) +- **Current version**: 8a4814a (2026-10-03, updated by rei; 5 commits: salted scrypt master KDF + AAD-bound ciphertexts + hardened destructive endpoints + OAuth state cleanup + MCP arg parsing + frontend review fixes; extension 0.2.4; alembic 0002_notes_widen applied; crypto migration run: 2 user keys rewrapped, 130 field ciphertexts rebound, idempotency verified; previous 53acfb4: Bearer auth fails closed; 287b7a0: session death → sign-in without reload; 27f95b4: webhook fixes, revoked tokens UI, extension 0.2.2 + pack v24; 543d6ae: splash on sign-in, extension 0.2.1; 4c6a4f7 2026-10-01: PWA shell, biometric reveal gate, ui-kit 1.0) ## Users (verified 2026-10-02) - Two accounts exist: goldmikki@gmail.com (Eugene Sukhodolskiy, owns ALL secrets) and lnikita2o1o@gmail.com (Popelous, registered 2026-09-18 via SSO, owns 0 secrets, 1 session + 1 encryption key). @@ -43,17 +43,29 @@ - `ip_address` in audit for UI traffic via https://creds.gnexus.space shows the internal reverse-proxy IP (192.168.1.226, [[internal-proxy-vps]]), not the real client IP — attribute by `user_agent`, not by IP. - Full reconciliation 2026-10-02: exactly 2 audit titles reference secrets absent from the DB — "git.gnexus.space — root" and "git.gnexus.space — testing", both deleted 2026-05-28 via UI (Chrome/Linux, through the proxy). "gntodo SSO client + MCP token (tasks.gnexus.space)" audit title refers to a since-renamed/split secret (now "gntodo SSO client + webhook secret" + "gntodo MCP token"). No phantom reveals, no unexplained deletions. +## Crypto migration 2026-10-03 (commit 8a4814a, DONE) +- New crypto: master key derived via salted scrypt (`GNEXUS_CREDS_MASTER_KEY_SALT`, mandatory in production), field ciphertexts AAD-bound to `::`, hardened destructive endpoints, OAuth state cleanup, MCP arg parsing fixes. +- Migration steps executed on live data (order matters): + 1. Full pre-backup: .env + compose to ~/backups/ AND full DB dump via /root/gnexus-creds-backups/backup.sh (db-20261003-102144.sql.gz). + 2. Generated salt: `python3 -c "import secrets; print(secrets.token_urlsafe(16))"`, appended `GNEXUS_CREDS_MASTER_KEY_SALT=` to .env (master key itself unchanged). Backup of pre-salt .env: .env.pre-salt. + 3. Added `GNEXUS_CREDS_CREDS_CORS_ORIGINS=["https://creds.gnexus.space"]` — production validator now REJECTS `*` cors_origins (also validates auth_base_url/auth_redirect_uri/mcp_resource_url must be https). + 4. `docker compose up -d --build` → app healthy. + 5. `docker compose exec -T app alembic upgrade head` → 0001_initial → 0002_notes_widen (secrets.notes 140→255 chars). + 6. Crypto migration: scripts/migrate-crypto.py is NOT in the image (Dockerfile copies only gnexus_creds/alembic/extensions/frontend) — `docker compose exec -T app mkdir -p /app/scripts && docker cp scripts/migrate-crypto.py gnexus-creds-app-1:/app/scripts/` then `docker compose exec -T app python scripts/migrate-crypto.py --dry-run` (2 keys, 130 ciphertexts) then real run (keys rewrapped 2, field ciphertexts rebound 130), then dry-run again → 0 remaining (idempotent). +- Production validator now enforces: master_key_salt present, cors_origins explicit list, https URLs. App crash-loops with pydantic ValidationError if any missing — check `docker compose logs app` first. + ## Backups (set up 2026-10-02 by rei, owner-requested after false data-loss alarm) - **Script**: `/root/gnexus-creds-backups/backup.sh` (root-only, 700) on gnexus-creds-host. - **What**: `pg_dump` of the gnexus_creds DB from the running container (gzip) + snapshot of `.env` — the dump is undecryptable without `GNEXUS_CREDS_MASTER_KEY` from `.env`, so both are snapshotted together. - **Schedule**: root crontab, daily 04:00 UTC (07:00 Europe/Kyiv), log: `/root/gnexus-creds-backups/backup.log` + cron.log. - **Retention**: 10 days (find -mtime +10 -delete for both db-*.sql.gz and env-*.bak). - **Sanity check**: dump must be non-empty and contain `CREATE TABLE public.secrets`, otherwise exit 1. -- **Verified**: manual run 2026-10-02 12:18 UTC, EXIT=0, db-20261002-121808.sql.gz 64080 bytes + env-20261002-121808.bak 874 bytes. +- **Verified**: manual run 2026-10-02 12:18 UTC, EXIT=0, db-20261002-121808.sql.gz 64080 bytes + env-20261002-121808.bak 874 bytes. Cron runs confirmed (env-20261003-040001.bak). Pre-crypto-migration dump: db-20261003-102144.sql.gz 66000 bytes. - **Restore sketch**: stop app → recreate empty DB (or drop schema) → `zcat db-*.sql.gz | docker compose exec -T db psql -U gnexus_creds -d gnexus_creds` → restore `.env` → `docker compose up -d app`. Dumps are --clean/--if-exists, so they can be replayed over an existing schema. ## Extension deployment (updated 2026-10-03) - Release zips are COMMITTED to the repo (`extensions/dist/gnexus-creds-extension-{chrome,firefox}-.zip`) and the Dockerfile does `COPY extensions ./extensions` — so `docker compose up -d --build` deploys the released extension automatically. +- Current release: 0.2.4 (chrome 5062904 bytes, firefox 5061964 bytes, verified inside container 2026-10-03). - The host has NO node/npm and NO frontend/node_modules. If a host-side rebuild is ever needed (owner explicitly asks to rebuild the extension), run node in a throwaway container: `docker run --rm -v /home/ubuntu/gnexus-creds:/w -w /w/extensions/extension node:22-bookworm-slim bash -lc "cd ../../frontend && npm ci --no-audit --no-fund && cd ../extensions/extension && make lib && make chrome"` (two separate `make` invocations — `make lib chrome` in one line fails). Verify with python zipfile CRC comparison against the released zip; git status in extensions/ must stay clean. - Design pack (`extensions/pack/`, packVersion) is served to installed extensions for hot-update of styling/detection tables without re-releasing. - Extension endpoints require auth (401 without session/token) — verify deployment by checking files inside the container, not by curling the API. @@ -62,6 +74,7 @@ - Environment variables are managed via `.env` file in `/home/ubuntu/gnexus-creds`. - Production mode enabled (`GNEXUS_CREDS_ENV=production`). - Secrets (`SECRET_KEY`, `MCP_TOKEN`) are randomly generated. +- New 2026-10-03: `GNEXUS_CREDS_MASTER_KEY_SALT` (mandatory, set 2026-10-03), `GNEXUS_CREDS_CORS_ORIGINS=["https://creds.gnexus.space"]`. ## Local runtime adaptation (do not overwrite) `docker-compose.yml` has an uncommitted local modification that matches the live stack: @@ -76,5 +89,5 @@ - **Readiness Check**: `GET /ready` - **Migrations**: `docker compose run --rm app alembic upgrade head` - **Logs**: `docker compose logs -f app` -- **Update procedure** (verified 2026-10-01 ×1 and 2026-10-02 ×6): backup `.env` + `docker-compose.yml` to `~/backups/` $\to$ stash compose $\to$ `git pull --ff-only origin master` $\to$ stash pop $\to$ `sudo docker compose up -d --build` $\to$ `alembic upgrade head` (only if migrations exist) $\to$ check `/health`, `/ready` and https://creds.gnexus.space/health. +- **Update procedure** (verified 2026-10-01 ×1 and 2026-10-02/03 ×7): backup `.env` + `docker-compose.yml` to `~/backups/` $\to$ stash compose $\to$ `git pull --ff-only origin master` $\to$ stash pop $\to$ `sudo docker compose up -d --build` $\to$ `alembic upgrade head` (only if migrations exist) $\to$ check `/health`, `/ready` and https://creds.gnexus.space/health. If app crash-loops with pydantic ValidationError — new production validator requirement, add missing .env var (see Crypto migration section). - **SSH access**: ubuntu@192.168.1.173, password same as navi swarm password (see gnexus-creds secret "navi (192.168.1.168) — SSH ubuntu"); sudo same password. No dedicated secret for this host yet. diff --git a/90-maintenance/pending-changes/20261003102605-bcc028f6.json b/90-maintenance/pending-changes/20261003102605-bcc028f6.json new file mode 100644 index 0000000..bbbf8e8 --- /dev/null +++ b/90-maintenance/pending-changes/20261003102605-bcc028f6.json @@ -0,0 +1,19 @@ +{ + "id": "20261003102605-bcc028f6", + "kind": "doc", + "target": "10-systems/services/gnexus-creds.md", + "summary": "Version 8a4814a: crypto migration done (salted KDF + AAD), alembic 0002, extension 0.2.4, new env vars (salt, CORS), update count ×7", + "reason": "Prod update 2026-10-03 10:21-10:30 UTC to 8a4814a: 5 commits with crypto changes (salted scrypt master KDF, AAD-bound ciphertexts, hardened endpoints). Required: new GNEXUS_CREDS_MASTER_KEY_SALT env var + explicit CORS origins (production validator), alembic 0002_notes_widen, crypto migration script (not in image — docker cp needed): 2 keys rewrapped, 130 ciphertexts rebound, idempotency verified. Extension 0.2.4 deployed. App crash-looped twice during setup (missing salt, missing CORS) — documented.", + "payload": { + "content": "---\nowner: gmikcon\nstatus: active\nlast_reviewed: 2026-10-03\nreview_interval: 90d\nconfidence: high\nsource_of_truth: deployment-verified\n---\n\n# gnexus-creds\n\nPersonal secret storage service with REST API, MCP HTTP/SSE adapter, and Vue UI.\n\n## Deployment\n- **Host**: [[gnexus-creds-host]] (192.168.1.173)\n- **Method**: Docker Compose\n- **Port**: 80 (External) $\\to$ 8000 (Internal)\n- **Database**: PostgreSQL 17 (Containerized)\n- **Repository**: https://git.gnexus.space/git/root/gnexus-creds.git (branch master)\n- **Current version**: 8a4814a (2026-10-03, updated by rei; 5 commits: salted scrypt master KDF + AAD-bound ciphertexts + hardened destructive endpoints + OAuth state cleanup + MCP arg parsing + frontend review fixes; extension 0.2.4; alembic 0002_notes_widen applied; crypto migration run: 2 user keys rewrapped, 130 field ciphertexts rebound, idempotency verified; previous 53acfb4: Bearer auth fails closed; 287b7a0: session death → sign-in without reload; 27f95b4: webhook fixes, revoked tokens UI, extension 0.2.2 + pack v24; 543d6ae: splash on sign-in, extension 0.2.1; 4c6a4f7 2026-10-01: PWA shell, biometric reveal gate, ui-kit 1.0)\n\n## Users (verified 2026-10-02)\n- Two accounts exist: goldmikki@gmail.com (Eugene Sukhodolskiy, owns ALL secrets) and lnikita2o1o@gmail.com (Popelous, registered 2026-09-18 via SSO, owns 0 secrets, 1 session + 1 encryption key).\n- Secrets are per-user (`secrets.user_id` → users, ON DELETE CASCADE); users cannot see each other's secrets.\n\n## Token revocation semantics (updated 2026-10-03, commit 53acfb4)\n- Revocation is immediate server-side: `authenticate_api_token` filters `revoked_at IS NULL` (services.py); `last_used_at` freezes at the moment of revocation.\n- Since 53acfb4 a presented Bearer header is FINAL: an invalid or revoked token answers 401 immediately (with `access.failed` audit, key=`bearer:<12-char prefix>`); cookie auth is only for requests WITHOUT a Bearer header (the SPA). Previously an invalid Bearer fell through to the session cookie in the same request — a revoked token kept \"working\" inside a logged-in browser (channel=ui), masking revocation. Regression test added (tests/test_auth.py).\n- The extension sends ONLY the Bearer header (fetch without credentials:include, no cookies). After revocation it gets 401 → background sets `tokenInvalid` flag → popup shows the invalid-token plate on next load. Until then the popup may render the secrets list from `chrome.storage.session` cache (TTL 5 min, refresh alarm every 15 min) — stale display, not live access.\n- How to actually verify a revocation: audit `access.failed` with key=`bearer:` (the revoked token being rejected), and frozen `last_used_at`.\n\n## Visibility semantics (important, caused a false \"data loss\" alarm 2026-10-02)\n- MCP `search_secrets` returns ONLY secrets with `allow_mcp=true` (by design, commit 2082db2: secrets created via MCP always get allow_mcp=true; secrets created via UI/extension default to allow_mcp=false).\n- MCP search default limit is 20 (max 50, paginate with offset).\n- Web UI lists 20 secrets per page with pagination (since commit aae771f, predates 2026-10 updates).\n- As of 2026-10-02 (later count): 71 secrets total (69 actual + 2 archived), all owned by goldmikki@gmail.com; 20 actual with allow_mcp=true, 49 with allow_mcp=false (mostly personal logins created via extension since May 2026). Nothing was lost during the 2026-10-01/02 updates: .env byte-identical to pre-update backup, zero app errors, only 2 deletions ever (2026-05-28).\n- Extension-created secrets are stored with `source` = site domain (44 domain-like sources, all alive). The extension creates them via REST (audit channel=rest); there is no channel=extension in audit.\n\n## Audit semantics (verified 2026-10-02, resolved a second false alarm)\n- Audit events keep a `metadata` snapshot (title, tags, category) forever, including after deletion — deleted secrets remain visible in the audit log under their last known title. Deletion code: `gnexus_creds/services.py` (`audit(..., action=\"secret.deleted\", metadata={\"snapshot\": ...})`).\n- Renaming a secret does NOT rewrite old audit metadata — each event shows the title as of that moment.\n- Archiving hides a secret from the default UI list, but it stays in the DB (`archived=true`) and its audit history remains.\n- `ip_address` in audit for UI traffic via https://creds.gnexus.space shows the internal reverse-proxy IP (192.168.1.226, [[internal-proxy-vps]]), not the real client IP — attribute by `user_agent`, not by IP.\n- Full reconciliation 2026-10-02: exactly 2 audit titles reference secrets absent from the DB — \"git.gnexus.space — root\" and \"git.gnexus.space — testing\", both deleted 2026-05-28 via UI (Chrome/Linux, through the proxy). \"gntodo SSO client + MCP token (tasks.gnexus.space)\" audit title refers to a since-renamed/split secret (now \"gntodo SSO client + webhook secret\" + \"gntodo MCP token\"). No phantom reveals, no unexplained deletions.\n\n## Crypto migration 2026-10-03 (commit 8a4814a, DONE)\n- New crypto: master key derived via salted scrypt (`GNEXUS_CREDS_MASTER_KEY_SALT`, mandatory in production), field ciphertexts AAD-bound to `::`, hardened destructive endpoints, OAuth state cleanup, MCP arg parsing fixes.\n- Migration steps executed on live data (order matters):\n 1. Full pre-backup: .env + compose to ~/backups/ AND full DB dump via /root/gnexus-creds-backups/backup.sh (db-20261003-102144.sql.gz).\n 2. Generated salt: `python3 -c \"import secrets; print(secrets.token_urlsafe(16))\"`, appended `GNEXUS_CREDS_MASTER_KEY_SALT=` to .env (master key itself unchanged). Backup of pre-salt .env: .env.pre-salt.\n 3. Added `GNEXUS_CREDS_CREDS_CORS_ORIGINS=[\"https://creds.gnexus.space\"]` — production validator now REJECTS `*` cors_origins (also validates auth_base_url/auth_redirect_uri/mcp_resource_url must be https).\n 4. `docker compose up -d --build` → app healthy.\n 5. `docker compose exec -T app alembic upgrade head` → 0001_initial → 0002_notes_widen (secrets.notes 140→255 chars).\n 6. Crypto migration: scripts/migrate-crypto.py is NOT in the image (Dockerfile copies only gnexus_creds/alembic/extensions/frontend) — `docker compose exec -T app mkdir -p /app/scripts && docker cp scripts/migrate-crypto.py gnexus-creds-app-1:/app/scripts/` then `docker compose exec -T app python scripts/migrate-crypto.py --dry-run` (2 keys, 130 ciphertexts) then real run (keys rewrapped 2, field ciphertexts rebound 130), then dry-run again → 0 remaining (idempotent).\n- Production validator now enforces: master_key_salt present, cors_origins explicit list, https URLs. App crash-loops with pydantic ValidationError if any missing — check `docker compose logs app` first.\n\n## Backups (set up 2026-10-02 by rei, owner-requested after false data-loss alarm)\n- **Script**: `/root/gnexus-creds-backups/backup.sh` (root-only, 700) on gnexus-creds-host.\n- **What**: `pg_dump` of the gnexus_creds DB from the running container (gzip) + snapshot of `.env` — the dump is undecryptable without `GNEXUS_CREDS_MASTER_KEY` from `.env`, so both are snapshotted together.\n- **Schedule**: root crontab, daily 04:00 UTC (07:00 Europe/Kyiv), log: `/root/gnexus-creds-backups/backup.log` + cron.log.\n- **Retention**: 10 days (find -mtime +10 -delete for both db-*.sql.gz and env-*.bak).\n- **Sanity check**: dump must be non-empty and contain `CREATE TABLE public.secrets`, otherwise exit 1.\n- **Verified**: manual run 2026-10-02 12:18 UTC, EXIT=0, db-20261002-121808.sql.gz 64080 bytes + env-20261002-121808.bak 874 bytes. Cron runs confirmed (env-20261003-040001.bak). Pre-crypto-migration dump: db-20261003-102144.sql.gz 66000 bytes.\n- **Restore sketch**: stop app → recreate empty DB (or drop schema) → `zcat db-*.sql.gz | docker compose exec -T db psql -U gnexus_creds -d gnexus_creds` → restore `.env` → `docker compose up -d app`. Dumps are --clean/--if-exists, so they can be replayed over an existing schema.\n\n## Extension deployment (updated 2026-10-03)\n- Release zips are COMMITTED to the repo (`extensions/dist/gnexus-creds-extension-{chrome,firefox}-.zip`) and the Dockerfile does `COPY extensions ./extensions` — so `docker compose up -d --build` deploys the released extension automatically.\n- Current release: 0.2.4 (chrome 5062904 bytes, firefox 5061964 bytes, verified inside container 2026-10-03).\n- The host has NO node/npm and NO frontend/node_modules. If a host-side rebuild is ever needed (owner explicitly asks to rebuild the extension), run node in a throwaway container: `docker run --rm -v /home/ubuntu/gnexus-creds:/w -w /w/extensions/extension node:22-bookworm-slim bash -lc \"cd ../../frontend && npm ci --no-audit --no-fund && cd ../extensions/extension && make lib && make chrome\"` (two separate `make` invocations — `make lib chrome` in one line fails). Verify with python zipfile CRC comparison against the released zip; git status in extensions/ must stay clean.\n- Design pack (`extensions/pack/`, packVersion) is served to installed extensions for hot-update of styling/detection tables without re-releasing.\n- Extension endpoints require auth (401 without session/token) — verify deployment by checking files inside the container, not by curling the API.\n\n## Configuration\n- Environment variables are managed via `.env` file in `/home/ubuntu/gnexus-creds`.\n- Production mode enabled (`GNEXUS_CREDS_ENV=production`).\n- Secrets (`SECRET_KEY`, `MCP_TOKEN`) are randomly generated.\n- New 2026-10-03: `GNEXUS_CREDS_MASTER_KEY_SALT` (mandatory, set 2026-10-03), `GNEXUS_CREDS_CORS_ORIGINS=[\"https://creds.gnexus.space\"]`.\n\n## Local runtime adaptation (do not overwrite)\n`docker-compose.yml` has an uncommitted local modification that matches the live stack:\n- service renamed `postgres` $\\to$ `db` (container names gnexus-creds-db-1 / app-1 depend on it);\n- port mapping `80:8000` (upstream default is `8000:8000`);\n- app healthcheck uses python urllib instead of curl (curl not in image);\n- `restart: unless-stopped` on both services.\nPreserve it across updates: `git stash push docker-compose.yml` $\\to$ pull $\\to$ `git stash pop`.\n\n## Operations\n- **Health Check**: `GET /health`\n- **Readiness Check**: `GET /ready`\n- **Migrations**: `docker compose run --rm app alembic upgrade head`\n- **Logs**: `docker compose logs -f app`\n- **Update procedure** (verified 2026-10-01 ×1 and 2026-10-02/03 ×7): backup `.env` + `docker-compose.yml` to `~/backups/` $\\to$ stash compose $\\to$ `git pull --ff-only origin master` $\\to$ stash pop $\\to$ `sudo docker compose up -d --build` $\\to$ `alembic upgrade head` (only if migrations exist) $\\to$ check `/health`, `/ready` and https://creds.gnexus.space/health. If app crash-loops with pydantic ValidationError — new production validator requirement, add missing .env var (see Crypto migration section).\n- **SSH access**: ubuntu@192.168.1.173, password same as navi swarm password (see gnexus-creds secret \"navi (192.168.1.168) — SSH ubuntu\"); sudo same password. No dedicated secret for this host yet." + }, + "status": "applied", + "created_at": "2026-10-03T10:26:05Z", + "updated_at": "2026-10-03T10:26:07Z", + "applied_at": "2026-10-03T10:26:07Z", + "validation": { + "status": "ok", + "issue_count": 0, + "issues": [] + } +}