Rsync howto in Ubuntu/Debian (Single rsyn-server collect data from other servers automatically)
Reference:
http://troy.jdmz.net/rsync/index.html
Suppose...
You have two linux-box, linux-box1 and linux-box2;
linux-box1 will be used as our rsync-host, and this rsync-host will bring (sync) the files/directory from the remote linux-box2; where linux-box2 is our rsync-client
linux-box1 has hostname rsync-host with ip address 123.49.42.180
linux-box2 has hostname rsync-client with ip address 123.49.42.181
In linux-box2 (rsync-client) there is a directory located as /opt/config_backup , this directory will be rsync by linux-box1 (rsync-host) in its /opt/backup/linux-box2 directory in every hour where ssh port is 22 (you may change your ssh port).
Requirements...
> internet connection ; for packages installation
> rsync
> openssh-server
> cron
|
Login as root to your rsync-client; Here I will take two of my configuration file /etc/squid/squid.conf and /us/bin/nat_firewall.sh backup regularly (in every 30 minutes) to a local directory name /opt/config_backup |
|
# apt-get -q -y install openssh-server rsync # mkdir /opt/config_backup |
| # vim /usr/bin/dsync.sh |
#!/bin/bash cp -r /etc/squid/squid.conf /opt/config_backup/ cp -r /usr/bin/nat_firewall.sh /opt/config_backup/ |
| Save + Exit |
| Change the file permission and run the script... |
|
# chmod 755 /usr/bin/dsync.sh # /usr/bin/dsync.sh |
| Setup up the cron to do this in every 30 minutes automatically... |
| # vim /etc/crontab |
Add the following line ... | */30 * * * * root /usr/bin/dsync.sh |
| Save + Exit |
| Add one system user with a strong passwrod which will help to test rsync-host and will sync the remote-directory... |
| # adduser rsyncuser |
|
Adding user `rsyncuser' ... Adding new group `rsyncuser' (1007) ... Adding new user `rsyncuser' (1007) with group `rsyncuser' ... Creating home directory `/home/rsyncuser' ... Copying files from `/etc/skel' ... Enter new UNIX password: Retype new UNIX password: passwd: password updated successfully Changing the user information for rsyncuser Enter the new value, or press ENTER for the default Full Name []: RSYNC USER Room Number []: Work Phone []: Home Phone []: Other []: Is the information correct? [Y/n] Y ~# |
STEP-02: Prepare your rsync-host (linux-box1)
| Login as root to your rsync-host; |
|
# apt-get -q -y install openssh-server rsync # mkdir -p /opt/backup/linux-box2 |
Checking the rsync from rsync-host to rsync-client by the following manual command... |
| # rsync -avz -e "ssh -p 22" rsyncuser@123.49.42.181:/opt/config_backup/ /opt/backup/linux-box2/ |
|
rsyncuser@123.49.42.181's password: receiving file list ... done ./ squid.conf nat_firewall.sh sent 70 bytes received 7098 bytes 754.53 bytes/sec total size is 50151 speedup is 7.00 ~# |
So, our test is done and successfull! Now we have to automate this procedure using shell script with the help of SSH... leave empty the passphrase |
| # ssh-keygen -t rsa -b 4096 -f /opt/backup/rsync-host-key |
|
Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in /opt/backup/rsync-host-key. Your public key has been saved in /opt/backup/rsync-host-key.pub. The key fingerprint is: 99:2e:5a:8a:2c:92:c9:81:14:12:af:c3:cb:80:f8:69 root@rsync-host ~# |
Copy the rsync-host-key.pub to the remote linux-box2 (rsync-client...) |
| # scp -P 22 /opt/backup/rsync-host-key.pub rsyncuser@123.49.42.181:/home/rsyncuser/ |
|
rsyncuser@123.49.42.181's password: rsync-host-key.pub 100% 731 0.7KB/s 00:00 ~# |
STEP-03: Validate the ssh-key of rsync-host in rsync-client (linux-box2)
| Login as root to your rsync-client; |
|
# cd /home/rsyncuser # vim vkey.sh |
Write the following lines in vkey.sh ... |
|
#!/bin/bash if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi mv rsync-host-key.pub .ssh/ cd .ssh/ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi cat rsync-host-key.pub >> authorized_keys |
| Save + Exit |
| Change the file permission and run vkey.sh |
|
# chmod 755 vkey.sh # ./vkey.sh # chown -R rsyncuser:rsyncuser /home/rsyncuser/ |
STEP-04: Finalizing rsync process in resync-host (linux-box1)
| Login as root to your rsync-host; |
|
# vim /usr/bin/dsyncbox2.sh |
Write the following lines in dsyncbox2.sh ... |
|
#!/bin/sh RSYNC=/usr/bin/rsync SSH=/usr/bin/ssh KEY=/opt/backup/rsync-host-key RUSER=rsyncuser RHOST=123.49.42.181 RPATH=/opt/config_backup/ LPATH=/opt/backup/linux-box2/ $RSYNC -az -e "$SSH -p 22 -i $KEY" --delete $RUSER@$RHOST:$RPATH $LPATH |
| Save + Exit |
| Change the file permission and run dsyncbox2.sh |
|
# chmod 755 /usr/bin/dsyncbox2.sh # /usr/bin/dsyncbox2.sh |
Open crontab file and add the following line to do rsync once in every hour |
| # vim /etc/crontab |
|
01 * * * * root /usr/bin/dsyncbox2.sh
|
| Save + Exit |
| That will sync file in every hour, from linux-box2 (/opt/config_backup) to linux-box1 (/opt/backup/linux-box2) |
Note: Consider Some Security Issue with SSH
|
As, in this howto, I used ssh-tunnel to do the rsync to work automatically, there might have some security issues with ssh login by rsa ssh-host-key. So, for primary level security imposing you can restrict the ssh access to your rsync-client, (a) Changing the default port (22) to any other port, (b) Using the /etc/hosts.deny and /etc/hosts.allow file to permit only from specific ssh host-ip. |
So, I am done for now with rsync, you may use this howto as a guideline and make changes if you require further options in rsync, also you may drop your asking to helpdesk@linux-bd.com |