Agent updates and signing

Where new versions come from

A new agent version comes with a new panel version: the panel offers the agent of its own version. What's going on with it — “Settings” → “Agent updates”: the version number, whether it's signed, how many agents are already on it and which are older.

Why a signature

The agent runs as root on every server. If it installed whatever the panel offers, a hacked or substituted panel would install its own binary on all servers at once.

So the agent installs an update only if the version's checksum list (SHA256SUMS) is signed with the owner's key:

• the private part of the key is only with you: on your computer and in your password manager. It's neither on the panel nor in CI;
• the public part is built into the agent itself. The agent checks the signature, not the panel;
• the panel only stores and hands out the signature — it can't forge it;
• a signature for one version doesn't fit another.

Version statuses

How to sign

On your computer, in the arcula repository:

make sign-agent PANEL=https://<panel address>

The script downloads the version from the panel, rebuilds the agent from the same commit and signs only if the builds match byte for byte. Upload the finished signature with the “Upload signature” button in “Agent updates”: the panel checks that it fits the version and the key. Step by step — How to update agents.

“Update agent” and “Update everywhere”

• “Update agent” — on the server overview, in the “Agent” row, when the agent's version differs from the panel's: “arcula 0.1.0 · the panel offers 0.2.0”.
• “Update everywhere (N)” — in “Settings” → “Agent updates”: the same command to all servers where the agent is older. Until the version is signed, the button is unavailable.

What the agent does:

1. Picks up the command — usually within a minute.
2. Downloads the version, checks the signature with the built-in key and the file checksums.
3. Waits for running backups to finish.
4. Replaces the binary. The previous one stays next to it — /usr/local/bin/arcula.prev.

While the update runs, the server shows “Updating the agent…”. The result is in the server log: “Agent updated” with the versions, or “Agent didn't update” with the reason.

If the agent is offline, the command waits 30 minutes for it and then expires. Send it again when the agent is back.

Automatic rollback

3 minutes after the replacement, the previous binary checks whether the arcula-agent service responds with the new version. If it doesn't, the previous one is put back, the service restarts, and the log shows “Agent didn't update”. Rollback works when the agent runs as a systemd service; without systemd there is no rollback.

What never happens

• The agent doesn't install an unsigned version.
• On a command from the panel, the agent doesn't move to a version older than its own. An older one can be put back only on the server itself — the command is below.
• The first install isn't protected by a signature: install.sh checks the agent only against the panel's own checksums. The same goes for running install.sh again. The signature protects updates: from the button in the panel and with sudo arcula update.

sudo arcula update --version X.Y.Z