Backups, What a Concept

For about three years I ran a single-user Mastodon instance at strangeweb.page in AWS, where the non-IaC bits were mostly manually put together. It was also a big example of "Do as I say, don't do as I do" because I wasn't maintaining any backups.

Recently I moved that instance from AWS EC2 to a Hetzner VM, which meant I had to actually go through the steps of taking the configuration and data from one host to another. As part of that exercise I cobbled together a small shell script that takes those vital bits and copies them to a safe place.

In my case that's object storage at a different provider, avoiding a single point of failure.

  #!/bin/sh

  . "$0.env"

  echo "$(date): Creating a DB snapshot"

  # Create a backup of the mastodon tables:
  sudo -u mastodon \
     pg_dump \
     -Fc mastodon_production \
     -f /home/mastodon/backup.dump

  echo "$(date): Creating a Redis snapshot"

  # Create a snapshot of the redis data:
  redis-cli SAVE

  for src in \
      /home/mastodon/live/.env.production \
      /var/lib/redis/dump.rdb \
      /home/mastodon/backup.dump; do
      echo "$(date): Backing up $src"
      aws s3 \
          --endpoint "https://$ENDPOINT" \
          cp \
          --quiet \
          "$src" \
          "s3://$BUCKET/snapshot/mastodon/"
  done

  echo "$(date): Done"

The .env file contains two settings: ENDPOINT and BUCKET. The ENDPOINT is necessary because the slightly older awscli version on the host was ignoring the endpoint_url directive in ~/.aws/config.

It's a fairly simple file, pointing to an object storage bucket at Scaleway:

[default]
region = nl-ams
output = json
services = scw-nl-ams

The other missing piece is ~/.aws/credentials with credentials for that bucket/account.

Going back to that "Do as I say, don't do as I do" bit, I have not yet performed the ultimate test of restoring a test-server from the backed up data. So if I'm missing something, maybe let me know so I can fix it before it's too late? 😅

I also know that this only maintains a single snapshot unless the object storage has version support and it's enabled.