Backups, the comic

The crate that comes back.

The same kitchen as the volumes comic. The pantry has to survive more than a new cook: a spilled pot, a broken disk, a fire. So the crates are packed tight, checked, locked and driven elsewhere, and none of it counts until a crate can go back.

1 A crate you can trust

The kitchen porter packs the whole pantry into one crate, and packs it tight: a .tar.zst. A slip goes with it, saying what is inside, which kitchen it came from, and a check number for the whole crate: the .json with its sha256.

Before it counts, the chef opens the crate once more and checks everything against the slip, down to the last jar. If anything is off, the crate is thrown out: a half crate never ends up on the shelf.

The van comes at three, every night, not “a day after the last one”, so it never drifts. Every kitchen has its own van, and it comes even while you are looking at another kitchen: a schedule belongs to the machine it was made on.

2 While the cooks are busy

A cook still stirring can leave a crate that does not restore. So the van skips and comes back in an hour, or the cooks pause for a few seconds. And if it has to skip three times in a row, the chef phones you.

For a database there is a better way: don’t pack the pot, ask the cook. Ben writes out everything he has in one go: a dump, made by Postgres, MySQL/MariaDB or MongoDB itself, consistent without a pause. It goes back the same way, through Ben.

Some cooks want a moment first. A note at their station says what to do before the van and after it: the labels archive-pre and archive-post, the way offen/docker-volume-backup reads them. If the “before” fails, nothing gets packed.

3 Out of the building

Before a crate leaves the kitchen, it can get a padlock: age encryption, with a password of yours that the app keeps sealed. Without that password nobody opens the crate, not even you. So keep it somewhere else too.

A crate in the same building burns with the building. So the van can take it elsewhere: to an SFTP server, an S3 bucket or WebDAV. Packed and checked here first, with the cooks back at work before it leaves, and no copy left behind here.

At a new storehouse the porter first reads you the sign on the door, and only then hands over the key: the fingerprint of the SFTP server, which you check and trust. Has the sign changed next time? Then he turns round, key still in his pocket.

4 Putting it back

Putting a crate back starts with a check, before any cook stops: the whole crate against its slip, and the padlock with the password you gave. Wrong password? Then nothing was touched, and you simply try again.

Then three ways back: on top of what is there, on emptied shelves, or into a second pantry next to it, to look first. And before anything is overwritten, a crate of what is there now goes to the back.

In Docker Client MX: crates packed tight and counted back, a van at a fixed hour for every kitchen, the cook’s own recipe for a database, a padlock of yours, a storehouse you checked, and a check before anything goes back.

The cheat sheet

Every picture above, in the app’s words.

The kitchen porter: A short-lived helper container that reads the volume
A crate packed tight: A .tar.zst archive
The slip: The .json beside it: volume, machine, size and sha256
Checked against the slip: Read back after writing; if it does not match, it goes away
The van at three: A schedule at a fixed time, daily or weekly
A van for every kitchen: A schedule belongs to its machine
Still stirring: The volume is in use: skip and try again every hour, or pause
“Third skip. Boss?”: A notification after three skips in a row
Everything written out: A dump: pg_dumpall, mysqldump or mariadb-dump, mongodump
Lid on, lid off: The labels docker-volume-backup.archive-pre and archive-post
The padlock: Encryption with age and your backup password
The off-site van: A destination: SFTP, S3 or WebDAV
The sign on the storehouse: The fingerprint of the SFTP server
Wrong key, nothing touched: The whole backup is checked before anything stops or changes
A second pantry, to look first: Restore into a new volume
A crate of what was there: A backup of the volume before it is overwritten

Now pack your first crate.

A schedule at a fixed time, checked after writing, encrypted with your password, off to SFTP, S3 or WebDAV, and for a database its own dump.

What the app does with backups →Volumes, the comic →Download for Windows