Bidirectional file sync in CentOS 7 (csync2 + lsyncd)

If you are a Linux sysadmin, it is vital for you to know how to synchronize directories and files bidirectionally on 2 or more servers. With bidirectional approach I mean that the changes in any server will replicate to the other ones. This approach is different from the master-slave configuration, that only changes on the “master” server will be replicated to the “slave” servers, but never vice versa.

There are multiple ways to achieve this, in this article I’ll show you my way using csync2and lsyncdpackages.

But what is csync2?

It is a powerful cluster synchronization tool that allows you to keep files in multiple hosts in a cluster in sync. It supports complex setups with more than 2 hosts and it can detect and handle conflicts.

And… what about lsyncd?

Lsyncd watches a local directory tree through an event monitor interface (inotify or fsevents). Everytime a change occurs lsyncd combine different processes and commands to achieve the synchronization. By default, it uses rsync but in this tutorial we’ll configure it to use csync2.

The lab

For this tutorial, our testing enviroment is composed of two CentOS 7 VMs with minimal installations. Their hostnames are node1 and node2 and the directory to sync is /opt/testdir.

Before install and configure any package, make sure that the hostnames and IPs of both servers are written in the /etc/hostsof each node. For this scenario, the /etc/hostsmust have the following lines:

192.168.1.100    node01
192.168.1.101    node02

Csync2 installation and setup

1. The first step to install csync2. The easiest way is adding OKay repository to yum.

rpm -ivh http://repo.okay.com.mx/centos/7/x86_64/release/okay-release-1-5.el7.noarch.rpm

2. Then, we can install the required packages with yum.

yum install xinetd sqlite-devel csync2

The csync2 default port is 30865 and should be opened.

firewall-cmd --zone=public --add-port=30865/tcp --permanent
firewall-cmd reload

Is a good practice to have directory for backup purposes, in case we want to recover a file from an undesired update.

mkdir -p /var/csync2/backup

The above steps must be done on each node. Now, we need to create an authentication key and configuration files. These files must be shared with all servers to achieve the synchronization properly.

To create the authentication key, run the following command:

csync2 -k /etc/csync2.key

With csync2 we can have multiple configuration files to sync different directories with different setups. A good practice is to give an alias to each file, such as the file name should be like csync2_<alias>.cfg. In this scenario, because we are syncing /opt/testdir our config file will be named csync2_testdir.cfg.

So, first let’s create our file:

vi /etc/csync2_testdir.cfg

And paste the following lines:

nossl * *;
group testdir
{
        host node01 node02;
        key /etc/csync2.key;
        include /opt/testdir;
        auto younger;
        backup-directory /var/csync2/backup;
        backup-generations 3;
}

About the setted parameters of the config file, here’s a little explanation:

• group: a name we give to identify the cluster of nodes.
• host:s a list of the hostnames of all nodes that belong to the group. The hostnames provided must match with the hosts’s local hostnames.
• key: the path to the authentication key.
• include: the path of the directory to sync. If more directories needed to be sync, you can add more of this lines.
• auto younger: this parameter means that when a synchronization conflict occurs, csync2 will solve it by keeping the newer file. There are other approaches to solve conflicts, for further information please check the csync2 docs.
• backup-directory: the path where the backup files will be stored.
• backup-generations: how many old versions of the files should be kept in the backup directory.

Now, as I mentioned before we need to copy the authentication key and config files to all nodes. You can use any tool that you want, such as scp or rsync.

Once all the hosts have the required files, to test your deployment run the following command in any node:

csync2 -xXrvlB -C <alias>

Where alias is the alias that you give to your config file in the steps before. For this scenario is testdir. So the command should look as follows:

csync2 -xv -C testdir

If everything is okay, no error should raise.

Lsyncd installation and setup

Csync2 by itself cannot detect continuesly file changes in the directories. Lsyncd will in charge of that task. This package is available in the EPEL repository, so you’ll need to install it if you haven’t already.

yum install epel-release
yum update
yum install lsyncd

Once lsycnd is in our system, we need to configure it to work together with csync2. For this purpose, we need to add the following lines to /etc/lsyncd.conf:

To work properly, you need to match in local sources the alias to its correspondent directory.

Finally, we should to start and enable xinetd and lsyncd services

systemctl enable xinetd
systemctl start xinetd
systemctl enable lsyncd
systemctl start lsyncd