Ever
worry about losing your data, or get tired of performing manual backups daily
or weekly? Use the rsync command and automatically sync your
local files to a remote server as often as desired with no manual intervention.
Please note, this guide does require access to a remote Linux server (eg. AWS),
and is written under Ubuntu 20.04 although any Linux distro should work fine.
Install rsync
Before
anything, check whether or not rsync is installed. On both your local
PC and web server run this command:
If
you get the current rsync version in return, then you're all set for the next
section. Otherwise, if you receive a command not found error, you may
install rsync with this command:
sudo apt-get -y install rsync
Generate SSH Key
We
will use a SSH key to
authenticate the connection between our local PC and the remote server. To
generate a new SSH key on your local PC within the terminal run the command:
ssh-keygen -t rsa -b 4096 -f ~/.ssh/rsync.key
When
prompted for a password, leave it blank and hit the Enter key twice. This will
generate two new files within your ~/.ssh/ directory named rsync.key which
is the private key, and rsync.key.pub, the public key.
Setup Remote Server
Although
not required, for this guide we will create a new user on
the remote server for rsync connections and to store all backup files. Login to
the remote server via SSH and run this command:
The
above example uses the username rsync, but you may change it to anything
you wish. The -m option simply tells Linux to create a home directory
for our new user.
To
allow your local PC to authenticate, the public SSH key that was generated in
the previous section needs to be copied over to the remote server. Open
the /~.ssh/rsync.key.pub file in a text editor and
you will see one large line that looks something like.
ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAACAQDNhyYKsjcGGdXmzOM3742+c+TzMLFdZtrMPj1q6JWNWzgY/gTGVy1C72kw6BcTYSG8B8kLQlaBRl16m2Gm8Ra/U1wl0TYSufOnRKjGq2glnBPysWNzR6i9qd4h/byKa4ptNH/ieYkT+BnSJVo8fT0iboYwEaL9D0jPtYxFzZes2ctsGZ/zi78VlX9N224YBtoZcrxK6gzKtcIVrplsXt4MbMCPc0hfr9f2VMt0HignLphTDLQWKwF3sGi4OHDPzNTRkjyHazsIOFIKDLQgdsIJv7b2VMs028YDqPnXHZZl4Ix5vg8ssqE+s/J+rzS0B6gwj2b/f6vJMI9DmTk8SO5LKWtSl4lXjLpQ1eP+xjf3SeMFWWkk2tPpGBo6d+8VJT6htj9Ga927qx3bYJ3FDdqjoE/28yBzMsg3wKI8lobiQGIbF0B0jZmSeq42ds7dh76iU/LOraWJWJhKPIjCYHdaVqj5rgxSulUW6oqr/LOxMNwsj5NLpyKygr5/RVjCUpxQLw5G7AClmW5nOZDFUgtI1CAOzhG8oYQes7jE7ZbQKmMf9IGquNV1BCRGX2mbcYad77UE2IjzPqSG8pFGb7ekZA6ukUk61fqoheL4Zl2jmhhWoXQ09LZE9FNfr1UwIoZ+GwUcip8NPIZPSo+Z4yMB/5VNF7J0o76eTNwh0gZlEw==
user@host
This
long line is the public SSH key. Copy it to your clipboard, and within your
remote server run these commands:
sudo su rsync
mkdir -m 0700 $HOME/.ssh
echo "ssh-rsa AAAAB... user@host" > $HOME/.ssh/authorized_keys
chmod 0644 $HOME/.ssh/authorized_keys
In
the second last command, replace the text between the quotation marks with that
long public SSH key line. That's it, your local PC will now be able to
authenticate with your remote server.
Configure ssh config File
For
sake of simplicity, add an entry to the ~/.ssh/config file on your
local PC to easily connect to the remote server. Open the file on
your local PC with the command.
Within
the file add an entry for the remote server such as:
host backup_server
hostname 192.168.0.24
user rsync
IdentityFile ~/.ssh/rsync.key
Change
the hostname to the IP address of your remote server, and if you used a
username other than "rsync" change that as well. You may use anything
you wish for the host, but for this example, "backup_server" was
used. Save and close the file by pressing Ctrl+X followed by the "Y"
and Enter keys.
Test
your SSH connection to the remote server with the command.
Assuming
everything is set up correctly, you should now be logged into your remote
server via SSH. Close the connection with the command.
Sync Your Files
Now
test the rsync functionality, and for example, to sync your Documents
directory on your local PC run the command:
rsync -avz --progress ~/Documents/
backup_server:~/Documents
The
first occurrence of ~/Documents/ specifies the local file or
directory to sync, backup_server corresponds with the entry added to
the ~/.ssh/config file, and the ending :~/Documents simply
specifies to upload everything into the /Documents directory of the remote server relative to the home directory.
Log
in to the remote server, and you should see a new Documents directory that is
in sync with that of your local PC. Each time you run the above command, only files that have been modified since the last
time will be uploaded, so you're not constantly uploading the entire contents
of the directory.
Automate via Crontab
Now
that everything is tested and working properly, we can easily automate the
entire process by adding a crontab job to our local PC. To automatically sync
your local folder to the remote server every 15 minutes, within the terminal
run the command.
(crontab -l; echo "*/15 * * * * rsync -avz --progress
~/Documents/ backup_server:~/Documents > /dev/null 2>&1";) | crontab
You
may get a "no crontab for user" message, and you can just ignore it.
Change the Documents directory to whatever you wish to backup, but
ensure to leave a trailing slash for directories otherwise they will not
properly backup.
Check
to ensure the crontab job was successfully added with the command.
If
you see the crontab job that was just added, then everything is in place. Wait
15 minutes, check your remote server, and all necessary files should be there.
Starting from now, all changes made to your files will be automatically
uploaded to the remote server every 15 minutes.
Download from Remote Server
You
may also use rsync to download files from the remote server and sync
them to your local PC. Using the above /Documents directory example, within the
terminal run the command:
rsync -chavzP backup_server:~/Documents/ ~/Documents
The ~./Documents directory
on your local PC should now be a mirror image of the remote server.
Include and Exclude Patterns
If
you ever need to sync only files that match a certain pattern, such as end
with .html you can use the --include pattern. Within
terminal run the command.
rsync -avz --include "*.html" --progress ~/mysite/ backup_server:~/public_html
Check
the remote server, and you will see only files with a .html extension from the
local /mysite/ directory have been uploaded into the /public_html/ remote
directory. Similarly, you can also sync everything except certain files with
the --exclude option. For example, the following command will sync
all files except those with a .txt extension.
rsync -avz --exclude "*.txt" --progress ~/mysite/ backup_server:~/public_html
Sync Two Local Directories
If
ever needed, you may also sync two local directories with the command.
rsync -zvr ~/source/directory ~/destination/directory
This
command works exactly the same as when syncing to a remote server, the only
difference being its two local directories.
Rest Easy
You
can now breathe a sigh of relief knowing your chances of data loss are now
substantially lower. In this article, you have learned what rsync is,
how to generate and install an SSH key, define a server within the ~./.ssh/config file,
sync a local and remote directory, and automate the entire process via crontab.
Going forward, all necessary files will always be synced with
your remote server with only a 15-minute delay.
