Validator failover

Use a dedicated data directory for the follower. Trustless RPC-to-RPC following is the default, and the upstream must be a validator or RPC node that exposes the consensus RPCs. If following a validator, the upstream validator must also expose WebSockets with --ws.

tempo node --datadir <datadir> \
  --chain <CHAIN> \
  --follow ws://example.url:8546 \
  --http --http.port 8545 \
  --http.api eth,net,web3,txpool,trace,consensus

Do not pass --consensus.signing-key while the standby is running as a follower. The standby is syncing certified consensus state, not participating in consensus.

Check that execution height is advancing and the certified consensus feed is available:

cast block-number --rpc-url http://localhost:8545
cast rpc consensus_getLatest --rpc-url http://localhost:8545

If consensus_getLatest is unavailable, the follower is not syncing consensus state. Confirm that the upstream endpoint is certified and exposes the consensus namespace.

The promoted node must be reachable at the ingress and egress addresses registered onchain. A setup that allows a floating IP or load balancer address that can move from the old validator to the standby will require no onchain updates.

If failover uses a different address, update the validator's onchain IP configuration after fencing the old validator. Follow the update IP addresses procedure.

Stop the old validator or isolate it from the network before starting the replacement. If you use a floating IP, move it to the standby before validator startup.

sudo systemctl stop tempo

Stop the standby follower cleanly so the same data directory can be reopened in validator mode.

sudo systemctl stop tempo-follower

If the follower is not managed by systemd, send SIGINT or SIGTERM and wait for the process to exit.

Restart the same data directory without --follow, and add the validator signing key.

Use the same consensus signing key as the validator you are failing over from. This procedure moves an existing validator identity to the standby; it does not rotate the identity. To use a different signing key, the validator's onchain configuration must be rotated. Follow the rotate the validator identity procedure.

tempo node --datadir <datadir> \
  --chain mainnet \
  --consensus.signing-key <PRIVATE_KEY_PATH> \
  --consensus.listen-address 0.0.0.0:<REGISTERED_INGRESS_PORT> \
  --http --http.port 8545 \
  --http.api eth,net,web3,txpool,trace,consensus \
  --telemetry-url <TELEMETRY_URL>

tempo node --datadir <datadir> \
  --chain testnet \
  --consensus.signing-key <PRIVATE_KEY_PATH> \
  --consensus.listen-address 0.0.0.0:<REGISTERED_INGRESS_PORT> \
  --http --http.port 8545 \
  --http.api eth,net,web3,txpool,trace,consensus \
  --telemetry-url <TELEMETRY_URL>

Do not delete or resync the data directory during failover. The follower's datadir contains the necessary consensus and execution state that allows the node to restart as a validator without a fresh bootstrap.

Check that the node's consensus RPC is live and progressing, then verify the validator status from a public RPC endpoint:

cast rpc consensus_getLatest --rpc-url http://localhost:8545

tempo consensus validator <pubkey> --rpc-url https://rpc.tempo.xyz

tempo consensus validator <pubkey> --rpc-url https://rpc.testnet.tempo.xyz

The validator should remain in_committee: true if it was already active. If you changed IP addresses during failover, wait for the IP update to finalize and for peer discovery to pick up the new address.