navi-1 / hive /
@Eugene Sukhodolskiy Eugene Sukhodolskiy authored 20 hours ago
..
README.md hive: versioned central config with layered targeting 20 hours ago
__init__.py swarm stage 1: instance identity, PSK, hive registry, announce loop 1 month ago
app.py hive: versioned central config with layered targeting 20 hours ago
config_store.py hive: versioned central config with layered targeting 20 hours ago
db.py hive: versioned central config with layered targeting 20 hours ago
hive.service swarm stage 1: instance identity, PSK, hive registry, announce loop 1 month ago
README.md

hive — the swarm address book

A tiny standalone service for the main server. Every Navi instance in the swarm announces itself here every couple of minutes (name, instance id, address, machine info), and agents ask the book who is currently alive and where. The free-form meta field is the seed of a future mini knowledge base about the fleet (host, OS, cores, RAM — whatever the machine reports).

The hive is not part of the Navi server and never starts by default — install.sh installs nothing hive-related. It runs on one machine with a stable address (e.g. the main server: local 192.168.1.168:8087).

Run it

On the main server, from the repo root (same clone, same venv — zero extra dependencies):

# quick check
.venv/bin/uvicorn hive.app:app --host 0.0.0.0 --port 8087

# as a systemd unit:
sed -e "s|EDIT_ME|$HOME|g" hive/hive.service | sudo tee /etc/systemd/system/hive.service
sudo systemctl daemon-reload
sudo systemctl enable --now hive

--host 0.0.0.0 is required so other machines on the LAN can announce.

Auth

Everything except /health requires the shared swarm key: an X-Swarm-Key header matching .swarm-key in the working directory. .swarm-key.previous is also accepted — that is the rotation window: put the new key in .swarm-key, move the old one to .swarm-key.previous, then delete the old file once every navi has picked up the new key.

Endpoints

Endpoint Auth What
GET /health none liveness + machine count + current config version
POST /announce swarm key upsert: {name, instance_id, version, port, meta} — the source IP is recorded as the machine's host (reliable on floating-IP networks), port comes from the payload. The reply may carry config: {fetch, version, hash} — that is how an instance that was offline learns it is behind
GET /peers swarm key {peers: [{name, instance_id, host, port, address, online, last_seen, first_seen, meta}], ttl_sec} — online = announced within the TTL
GET /config swarm key the layers for the asking instance (?name=&instance_id=&tags=a,b); 404 = nothing ever published; layers: [] = "no overlay, use your local config"
POST /config/ack swarm key {instance_id, version, hash, status, restart_required, error} — the instance's report after applying
PUT /config admin publish {layers, note, force}; an identical document is a no-op unless force (which re-notifies without a new version)
GET /config/info admin the current document as published
GET /config/history admin versions, newest first (?limit=)
GET /config/version/{id} admin one version, layers included
POST /config/rollback/{id} admin publish an old document as a new version and ping the fleet
GET /config/status admin per instance: what it reports running, the effective hash for the current layers, whether it is behind, and its last ack

Central config

The hive stores one versioned config document for the fleet (see the "Central config" section of docs/config.md for the document shape, validation rules and the fail scenarios). Operational notes:

  • Publish:

    curl -X PUT http://127.0.0.1:8087/config \
      -H "X-Admin-Key: $(cat .hive-admin-key)" \
      -H 'Content-Type: application/json' \
      -d '{"layers":[{"name":"global","config":{"settings":{"llm_complete_timeout":90}}}],"note":"timeouts"}'

    layers: [] is a real publish: it clears the fleet overlay and every instance returns to its local config.

  • Watch it land: GET /config/status after a publish. behind names the machines still on an older effective hash; each has its last ack and the fields that need a restart. A rejected ack means that instance refused the document — validation errors are in last_error, and the instance is still running its previous config.

  • Retry: the hive pings behind instances every HIVE_NOTIFY_RETRY_SEC and on every announce reply, so a machine that was off catches up by itself. There is nothing to do manually.
  • Rollback is a forward publish of an old document. History is never rewritten, including for a document that turned out to be bad.

Trust model

  • The swarm key can read the config, therefore it can read whatever secrets the document holds. That is the price of a fleet-wide key rotation without per-instance setup; keep port 8087 on the LAN and treat the key as a secret on the same level as the config itself.
  • Publishing needs the admin key (HIVE_ADMIN_KEY_FILE, default .hive-admin-key, also HIVE_ADMIN_KEY env) or a loopback connection (HIVE_ALLOW_LOCAL_ADMIN=0 turns even that off). Once a key is configured it is required from every address, loopback included.
  • hive.db contains the config history with secrets. Back it up like a credential store (0600) and do not copy it around.

Configuration (env vars)

Var Default Meaning
HIVE_KEY_FILE .swarm-key current swarm key file
HIVE_DB hive.db SQLite file
HIVE_TTL_SEC 300 a machine is online if it announced within this window
HIVE_ADMIN_KEY_FILE .hive-admin-key admin key file for publishing/history/rollback/status
HIVE_ADMIN_KEY "" admin key from the environment (takes precedence)
HIVE_ALLOW_LOCAL_ADMIN 1 allow admin calls from loopback without a key (only when no key is configured)
HIVE_NOTIFY_TIMEOUT_SEC 3 per-instance timeout for the config-change ping
HIVE_NOTIFY_RETRY_SEC 300 how often behind instances are re-pinged
HIVE_NOTIFY_SCHEME http scheme used when pinging instances

A Navi announces itself when .env sets HIVE_URL (e.g. HIVE_URL=http://192.168.1.168:8087) and .swarm-key exists next to .env (install.sh generates one; copy the same file to every machine and the hive host to form a swarm). No HIVE_URL — the navi is standalone and keeps working; the registry being down is also non-blocking: the agent just gets a context note that the book is unreachable.

Peer channel (stage 2)

Navi instances talk to each other directly — the hive is only the address book, never on the message path. Each navi exposes on its API port (8099):

Endpoint Auth Purpose
GET /peer/hello none name + uuid prefix + version (discovery ping)
GET /peer/status PSK identity, uptime, machine facts, its hive view
POST /peer/ask PSK ask a peer a question; a one-shot agent run under PEER_ASK_PROFILE answers

Agents use the built-in peer tool (list / status / ask). Loop safety is deterministic: the answering agent is created with peer excluded, so asks cannot recurse; an ask claiming to come from our own uuid is refused (409). Every ask is audited on both sides via structlog (peer.ask_sent / peer.ask_received / peer.ask_answered).