Run Clawdbot on a Proxmox cluster (HA + scaling)

This blueprint targets teams that need reliable AI agent operations under burst traffic. It covers node layout, queue topology, failure domains, and autoscaling tactics for Clawdbot-like workloads.

Reference cluster topology

  • 3 Proxmox nodes minimum for quorum and controlled failover behavior.
  • Control plane VM for orchestration, metrics, and deployment pipelines.
  • Worker pool VMs dedicated to Clawdbot queue consumers.
  • Data services isolated for PostgreSQL, Redis, and vector index storage.

High availability design

Service redundancy

Avoid single points of failure in broker and database paths. Use replication and health-gated routing so failing components are drained before job loss propagates.

Failure domain isolation

Keep worker classes separated by queue and business criticality. If one workflow overloads, critical queues should continue serving user-facing requests.

State consistency

Persist idempotency keys and job checkpoints in durable storage to survive retries and node restarts.

Scaling strategy

  • Scale workers horizontally based on queue depth and job age thresholds.
  • Set per-queue concurrency caps to protect upstream API limits.
  • Apply budget guards for token/API spend anomalies.
  • Separate heavy async tasks from low-latency interactive tasks.

Operational KPIs

Track these metrics per queue and per worker class:

  • p95/p99 execution latency
  • Queue backlog and oldest message age
  • Retry rate and dead-letter volume
  • Database response time and connection saturation
  • Cost per completed workflow

Disaster recovery for agent systems

Maintain recovery runbooks for full cluster loss, broker corruption, and partial database outage. Test restore in staging with representative production load before trusting procedures.

  • Daily encrypted backups with off-cluster copies
  • Weekly restore drills with KPI validation
  • Quarterly failover exercises with runbook updates

Deployment sequence (recommended)

  1. Deploy core data plane and validate replication.
  2. Bring up control plane and observability stack.
  3. Deploy worker pools in staged queue groups.
  4. Enable autoscaling rules and budget guardrails.
  5. Run load test and controlled failover simulation.

Start with hardware planning in dedicated plans, then follow the installation guide before implementing this HA scaling blueprint.