RSYNC – sync the data between two servers using script

How to sync the data between two servers without using SAN replication ? Do you have better than RSYNC tool for this job ? I don’t think so, you we will not get better than RSYNC. It uses ‘rsync algorithm’ which provides a very fast method for syncing the directories or filesystems. An important feature of rsync is that the mirroring takes place with only one transmission in each direction and which is not available in other similar programs.

1.Verify whether your host will allow to perform RSA key-less authentication.If you didn’t get similar output,then  you need to comment out the lines in sshd_config.

[root@mylinz1 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#"
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys
[root@mylinz1 ~]#

[root@mylinz2 ~]# cat /etc/ssh/sshd_config |egrep "RSA|Pubkey|Authorized" |grep -v "#"
RSAAuthentication yes
PubkeyAuthentication yes
AuthorizedKeysFile      .ssh/authorized_keys

2.Generate the keygen if you didn’t have one already. Here the user is “root”.

[root@mylinz1 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
94:e6:6f:66:bb:cd:a8:30:4d:90:94:31:ae:64:f6:5e root@mylinz1
The key's randomart image is:
+--[ RSA 2048]----+
|      +o         |
|     o.o .       |
|    + + +        |
|   + o =         |
|    . . E        |
|     . + .       |
|      + . =      |
|       o + =     |
|        ..+.o    |
+-----------------+
[root@mylinz1 ~]# cd .ssh/
[root@mylinz1 .ssh]# ls -lrt
total 8
-rw-r--r--. 1 root root  394 Jun 19 00:43 id_rsa.pub
-rw-------. 1 root root 1671 Jun 19 00:43 id_rsa
[root@mylinz1 .ssh]#

[root@mylinz2 ~]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
e7:a8:19:ac:dd:e3:28:b3:42:00:a0:84:4a:10:4e:fe root@mylinz2
The key's randomart image is:
+--[ RSA 2048]----+
|B+               |
|O.               |
|=o               |
|o .              |
| . E    S .      |
|  .  .   +       |
| .    o . .      |
|  . oo *.        |
|   .o+=.o.       |
+-----------------+
[root@mylinz2 ~]# cd .ssh/
[root@mylinz2 .ssh]# ls -lrt
total 12
-rw-r--r--. 1 root root  395 Jun 19 00:17 known_hosts
-rw-------. 1 root root 1675 Jun 19 00:44 id_rsa
-rw-r--r--. 1 root root  394 Jun 19 00:44 id_rsa.pub

3.Share the “id_rsc.pub” file across the servers to enable the ssh key-less authentication.

[root@mylinz1 .ssh]# scp -r id_rsa.pub 192.168.10.25:/root/.ssh/authorized_keys
The authenticity of host '192.168.10.25 (192.168.10.25)' can't be established.
RSA key fingerprint is 5a:56:fd:69:cf:f2:b8:78:b9:67:e8:d0:f2:a4:ef:cb.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.10.25' (RSA) to the list of known hosts.
root@192.168.10.25's password:
id_rsa.pub                               100%  394     0.4KB/s   00:00
[root@mylinz1 .ssh]#

[root@mylinz2 .ssh]# scp -r id_rsa.pub 192.168.10.20:/root/.ssh/authorized_keys
root@192.168.10.20's password:
id_rsa.pub                                  100%  394     0.4KB/s   00:00
[root@mylinz2 .ssh]#

4.Verify your work.

[root@mylinz2 ~]# ssh 192.168.10.20
Last login: Wed Jun 19 00:42:27 2013 from 192.168.10.101
[root@mylinz1 ~]# ssh 192.168.10.25
Last login: Tue Jun 18 23:59:19 2013 from 192.168.10.101
[root@mylinz2 ~]#

 

Let’s move to RSYNC part.

Here is the rsync script which will be used for syncing the data between servers mylinz1 & mylinz2.

Here I am running script from mylinz1 to sync the data.

[root@mylinz1 ~]# cat rsync_oracle.sh
#!/bin/bash
# RSYNC SCRIPT TO SYNC TWO SERVER'S SPECIFIC DIRECTORIES
# Website:www.UnixArena.com
SOURCE_PATH='/db/oracle/'
SOURCE_SERVER='192.168.10.20'  #Added for reference
DESTINATION_PATH='/db/oracle-bck/'
DESTINATION_HOST='192.168.10.25'
DESTINATION_USER='root'
LOGFILE='rsync_oralce.log'
echo $'\n\n' >> $LOGFILE
rsync -av --rsh=ssh $SOURCE_PATH $DESTINATION_USER@$DESTINATION_HOST:$DESTINATION_PATH 2>&1 >> $LOGFILE
echo "Sync Completed at:`/bin/date`" >> $LOGFILE
[root@mylinz1 ~]#pwd
/root
[root@mylinz1 ~]#chmod 700 rsync_oracle.sh

This script creates log as well with newly synchronized  files information. 

1.Run the script manually.

[root@mylinz1 ~]#./rsync_oracle.sh

2.Verify the log file.

[root@mylinz1 ~]# tail -1rsync_oralce.log
kshrc
latrace.conf
ld.so.cache
ld.so.conf
libaudit.conf
libuser.conf
sent 160321 bytes  received 1155 bytes  107650.67 bytes/sec
total size is 156728  speedup is 0.97
Completed at:Wed Jun 19 01:02:25 IST 2013
[root@mylinz1 oracle]#

3.Create a new file to sync the data to mylinz2 server.

[root@mylinz1 oracle]#cd /db/oracle/
[root@mylinz1 oracle]# touch verify_rsync
[root@mylinz1 oracle]# ls -lrt verify_rsync
-rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync
[root@mylinz1 oracle]# cd /root
[root@mylinz1 ~]# ./rsync_oracle.sh
[root@mylinz1 ~]#

[root@mylinz1 ~]# tail -10 rsync_oralce.log
sending incremental file list
./
verify_rsync

sent 1093 bytes  received 34 bytes  2254.00 bytes/sec
total size is 156728  speedup is 139.07
Completed at:Wed Jun 19 01:05:03 IST 2013
[root@mylinz1 ~]#

 

4.Verify the whether the newly created file synced in mylinz2.

[root@mylinz2 ~]# cd /db/oracle-bck/
[root@mylinz2 oracle-bck]# ls -lrt verify_rsync
-rw-r--r--. 1 root root 0 Jun 19 01:04 verify_rsync
[root@mylinz2 oracle-bck]#

 

That’s it. Our rsync script is working fine. 

If you want to sync the data between two servers automatically on preferred time interval,you can add the script in to crontab.

Add the below line in root’s crontab to sync the data for every 5 minutes.  

0,5,10,15,20,25,30,35,40,45,50,55 * * * * /root/rsync.sh 2>&1 > /dev/null

Thank you for reading this article.Please leave a comment if you have any doubt.