Automatically Backup Your Files to a Remote Server with Rsync

Ever worry about losing your data, or get tired of performing manual backups daily or weekly? Use the rsync command and automatically sync your local files to a remote server as often as desired with no manual intervention. Please note, this guide does require access to a remote Linux server (eg. AWS), and is written under Ubuntu 20.04 although any Linux distro should work fine. 

Install rsync

Before anything, check whether or not rsync is installed. On both your local PC and web server run this command:

rsync --version

If you get the current rsync version in return, then you're all set for the next section. Otherwise, if you receive a command not found error, you may install rsync with this command:

sudo apt-get -y install rsync

Generate SSH Key

We will use a SSH key to authenticate the connection between our local PC and the remote server. To generate a new SSH key on your local PC within the terminal run the command:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/rsync.key

When prompted for a password, leave it blank and hit the Enter key twice. This will generate two new files within your ~/.ssh/ directory named rsync.key which is the private key, and rsync.key.pub, the public key.

Setup Remote Server

Although not required, for this guide we will create a new user on the remote server for rsync connections and to store all backup files. Login to the remote server via SSH and run this command:

sudo useradd -m rsync

The above example uses the username rsync, but you may change it to anything you wish. The -m option simply tells Linux to create a home directory for our new user.

To allow your local PC to authenticate, the public SSH key that was generated in the previous section needs to be copied over to the remote server. Open the /~.ssh/rsync.key.pub file in a text editor and you will see one large line that looks something like.

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDNhyYKsjcGGdXmzOM3742+c+TzMLFdZtrMPj1q6JWNWzgY/gTGVy1C72kw6BcTYSG8B8kLQlaBRl16m2Gm8Ra/U1wl0TYSufOnRKjGq2glnBPysWNzR6i9qd4h/byKa4ptNH/ieYkT+BnSJVo8fT0iboYwEaL9D0jPtYxFzZes2ctsGZ/zi78VlX9N224YBtoZcrxK6gzKtcIVrplsXt4MbMCPc0hfr9f2VMt0HignLphTDLQWKwF3sGi4OHDPzNTRkjyHazsIOFIKDLQgdsIJv7b2VMs028YDqPnXHZZl4Ix5vg8ssqE+s/J+rzS0B6gwj2b/f6vJMI9DmTk8SO5LKWtSl4lXjLpQ1eP+xjf3SeMFWWkk2tPpGBo6d+8VJT6htj9Ga927qx3bYJ3FDdqjoE/28yBzMsg3wKI8lobiQGIbF0B0jZmSeq42ds7dh76iU/LOraWJWJhKPIjCYHdaVqj5rgxSulUW6oqr/LOxMNwsj5NLpyKygr5/RVjCUpxQLw5G7AClmW5nOZDFUgtI1CAOzhG8oYQes7jE7ZbQKmMf9IGquNV1BCRGX2mbcYad77UE2IjzPqSG8pFGb7ekZA6ukUk61fqoheL4Zl2jmhhWoXQ09LZE9FNfr1UwIoZ+GwUcip8NPIZPSo+Z4yMB/5VNF7J0o76eTNwh0gZlEw== user@host

This long line is the public SSH key. Copy it to your clipboard, and within your remote server run these commands:

sudo su rsync

mkdir -m 0700 $HOME/.ssh

echo "ssh-rsa AAAAB... user@host" > $HOME/.ssh/authorized_keys

chmod 0644 $HOME/.ssh/authorized_keys

In the second last command, replace the text between the quotation marks with that long public SSH key line. That's it, your local PC will now be able to authenticate with your remote server.

Configure ssh config File

For sake of simplicity, add an entry to the ~/.ssh/config file on your local PC to easily connect to the remote server. Open the file on your local PC with the command.

nano $HOME/.ssh/config

Within the file add an entry for the remote server such as:

host backup_server
    hostname 192.168.0.24
    user rsync
    IdentityFile ~/.ssh/rsync.key

Change the hostname to the IP address of your remote server, and if you used a username other than "rsync" change that as well. You may use anything you wish for the host, but for this example, "backup_server" was used. Save and close the file by pressing Ctrl+X followed by the "Y" and Enter keys.

Test your SSH connection to the remote server with the command.

ssh backup_server

Assuming everything is set up correctly, you should now be logged into your remote server via SSH. Close the connection with the command.

exit

Sync Your Files

Now test the rsync functionality, and for example, to sync your Documents directory on your local PC run the command:

rsync -avz --progress ~/Documents/ backup_server:~/Documents

The first occurrence of ~/Documents/ specifies the local file or directory to sync, backup_server corresponds with the entry added to the ~/.ssh/config file, and the ending :~/Documents simply specifies to upload everything into the /Documents directory of the remote server relative to the home directory.

Log in to the remote server, and you should see a new Documents directory that is in sync with that of your local PC. Each time you run the above command, only files that have been modified since the last time will be uploaded, so you're not constantly uploading the entire contents of the directory.

Automate via Crontab

Now that everything is tested and working properly, we can easily automate the entire process by adding a crontab job to our local PC. To automatically sync your local folder to the remote server every 15 minutes, within the terminal run the command.

(crontab -l; echo "*/15 * * * * rsync -avz --progress ~/Documents/ backup_server:~/Documents > /dev/null 2>&1";) | crontab

You may get a "no crontab for user" message, and you can just ignore it. Change the Documents directory to whatever you wish to backup, but ensure to leave a trailing slash for directories otherwise they will not properly backup.

Check to ensure the crontab job was successfully added with the command.

crontab -l

If you see the crontab job that was just added, then everything is in place. Wait 15 minutes, check your remote server, and all necessary files should be there. Starting from now, all changes made to your files will be automatically uploaded to the remote server every 15 minutes.

Download from Remote Server

You may also use rsync to download files from the remote server and sync them to your local PC. Using the above /Documents directory example, within the terminal run the command:

rsync -chavzP backup_server:~/Documents/ ~/Documents

The ~./Documents directory on your local PC should now be a mirror image of the remote server.

Include and Exclude Patterns

If you ever need to sync only files that match a certain pattern, such as end with .html you can use the --include pattern. Within terminal run the command.

rsync -avz --include "*.html" --progress ~/mysite/ backup_server:~/public_html

Check the remote server, and you will see only files with a .html extension from the local /mysite/ directory have been uploaded into the /public_html/ remote directory. Similarly, you can also sync everything except certain files with the --exclude option. For example, the following command will sync all files except those with a .txt extension.

rsync -avz --exclude "*.txt" --progress ~/mysite/ backup_server:~/public_html

Sync Two Local Directories

If ever needed, you may also sync two local directories with the command.

rsync -zvr ~/source/directory ~/destination/directory

This command works exactly the same as when syncing to a remote server, the only difference being its two local directories.

Rest Easy

You can now breathe a sigh of relief knowing your chances of data loss are now substantially lower. In this article, you have learned what rsync is, how to generate and install an SSH key, define a server within the ~./.ssh/config file, sync a local and remote directory, and automate the entire process via crontab. Going forward, all necessary files will always be synced with your remote server with only a 15-minute delay.