Using Restic for Backups

Intro

Restic Docs

The doc will mostly be my primary source outside of their official Github Repo as well. Most if the information here will be condensed for my specific use. There are a lot of possibilities and this is a pretty cool little backup tool you can install anywhere and run.

The restic team have explained that they want to focus on a backup tool so they have no real plans to implement a scheduling system. This is fine as we can use any native scheduling tool for their respective platforms. I use scripts and crontab for example.

Restic

Restic is a Golang written backup program that can be used to backup Linux, MacOS and Windows operating systems. Restic attempts to make backing up an easy task with the logical approach of “If we make it easy, you wont find a reason to avoid it”.

Restic supports multiple backends for storing your backup data. A few of these possible backends are

• Local Directories
• SFTP (via SSH)
• HTTP Rest (restic-server)
• S3
• GCP
• Azure Blob Storage

The backend we are mostly going to be using will be a local directory. I map LUNs to my hosts via ISCSI and create a backup directory. At the time of implementation this was easiest for me since I already had an empty lun attached to a host. I will at some point play around with SFTP and I hope to eventually settle on hosting an HTTP rest server. After all, it is a homelab so why not use everything under the sun.

Before we start going through the work flow of things, I just want to start off with this Link. Its the official Restic introduction on their documents. They are not joking that you can get up and running with 4 restic commands. It was the page I used to evaluate if I wanted to use restic and it working straight out of the box hooked me.

Registries

Just think of a Restic registry as any other registry that holds binary data. A good parallel could be Docker’s registry. You build an image, tag it, upload it and you have a versioned list of images you can deploy. Just like the Docker registry model, restic enables tagging and naming when uploading to a registry and you can restore based on tags and versions as needed.

Just a preface before diving into setting up a registry. Restic recommends not using a CIFS(SMB) share is not recommended. So, naturally this is the method I went with before stumbling on the documentation about it. This is another reason I will at some point migrate to HTTP or SFTP.

Setting up a Local Registry

Local Registry

I start with creating the password file that will be used to authenticate to the registry. This password file is imporant for interacting with the registry. Without it you wont be able to restore or even communicate to your registry.

Create a password file and set the permissions on the file to root access. Restic needs to run as root, there are ways to get around this but I just run it as root and move on.

touch .restic
chmod 600 .restic

Then you can edit the file and add your future registry password to it. You will reference this file in commands or you can create an environment variable for this file location.

Setting up your local registry. I will use a brand new directory on the local file system for this purpose.

mkdir /backups

Initialize the restic repository and follow the password prompts.

restic init --repo /srv/restic-repo

At this point you should have a working registry that is capable of hosting your backups.

Backups

We have a location where our backups can be stored. Now we need to actually create our backup and ship it to our registry.

Without getting into the weeds of backup strategies, we will just simply create a backup or what Restic calls a ‘snapshot’. A snapshot is simply the contents of a directory at a specific point in time.

As part of the backup job, restic attempts to deduplicate information in the snapshot at backup time. When the snapshot lands in your registry it might actually look smaller their than on disk. Restic will also attempt file change detection making it so only changes are backed up and the same files are not always created.

You can adjust this file duplication and change detection with feature flags when running restic.

--force
--ignore-ctime
--ignore-inode

One more point I want to make before actually running a backup. I can imagine if you’re using remote storage like in AWS or GCP you might want to learn or troubleshoot without racking up data transfer fees. This is where the --dry-run flag comes in useful.

Backup dry run.

restic -r /backups backup ~/manga_collection --dry-run -vv | grep "added"

The above command should print the filenames of everything that gets added to the snapshot backup. Where the -r flag is the repository, ~/manga_collection is the directory to be backed up. The flag -vv is a higher level of verbosity that outputs filenames and the actions applied to them. Again, this could be useful in testing or learning before shipping to the cloud.

Now, for a backup. Since we already covered some of the basic flags above I wont get into to much detail. I’m also a Neanderthal, so writing out a declaritive sentence that contains my action helps me.

“I want to backup my local managa collection folder to my backup registry”

restic -r /backups backup ~/manga_collection --password-file .restic 

The flag -r /backups is the registry.

The flag backup ~/manga_collection is telling restic to backup this directory.

The flag --password-file .restic is telling restic to use this file to authenticate to our registry.

I am actually pretty happy I dont need to dive into a spiral of caveates and present to you a series of flags to make generic things happen. This simplicity, makes scripting and automating restic backups so easy Its essentially fire and forget.

Restores

We have a registry, we have data, and we have now backed up that data. When we need to test our restores, or if you actually need to restore. Its good to have this be as straight forward and as simple as possible task to execute. So many additional mistakes during an outage are caused by self imposed and even external pressures to resolve the problem quickly.

No one wants to spend 45 minutes reading the restore section of external or internal documentation, figuring out the series of encantations you need to recite (aka flags) to satisfy your action, then wait for the job to complete only to find out you copied the wrong sha or name and need to start over.

I want something akin to a one-liner to restore a directory or single file. Thanks restic!

restic --pasword-file .restic -r /backups restore 79766175 --target /tmp/restored_manga_collection

Yup. Its that easy but what is 79766175 and where did we get it? Well each snapshot in the registry has an ID tied to the backup job, query the registry, find the ID and use that to restore.

Getting a list of all your snapshots.

restic --pasword-file .restic -r /backups snapshots

Checking which files are contained in a snapshot.

restic --pasword-file .restic ls 79766175

These are most likely going to be the two main commands you use to enage with registry snapshots. I do recommend checking out the official docs on this topic as its a nice rabbit hole of use cases and details. Restic working with repositories

There are a lot more possibilites when you want to restore from a snapshot. Do you want a single file? Do you want a single directory or even a specific file type? Well there are flags and regex functionality to make it all possible. You might notice a theme in this article, where I try and get you to engage with the documentation as much as possible as I think its a great document and captures the simplicty of restic.

Restoring from backup

Outro

This hopefully covers how to get started with Restic and should help you get familiar with the Restic CLI. Once you feel you have mastered the basics and understand how to setup, run and restore your backups. A logical evolution for implementing restic at a more managed scale, or just to be used in automation, would be something like Autorestic.

I just know how volitile a learning ecosystem can be. I am also a little chaotic when it comes to my homelab and tend to impact “production” things. Having a simple to use backup tool you actually want to work with is key to removing the dreaded “Oh no, I need to rebuild” feeling. Your backups should also give you the confidence to make changes because restoring a mistake is 1-2 commands away.