(Deprecated) Replicating repositories with Syncthing¶
Hint
WAPT repositories synchronization is now native in WAPT Enterprise.
You can find the new documentation here: Replicating a repository to preserve the bandwidth on remote sites.
Warning
This part of the documentation is no longer maintained. You may use it to collect ideas for doing your replication while using the Community version.
Introducing Syncthing¶
Syncthing is a multi-OS open source peer to peer synchronization utility.
It allows to synchronize folders on several machines while guaranteeing the security, the authenticity and the integrity of the files.
The official Syncthing documentation is available online.
Implementing the replication¶
Hint
The following documentation is applicable to Linux Debian and CentOS/ RedHat based WAPT Servers and remote repositories.
Setting up the remote WAPT repository¶
Debian Linux¶
echo "deb https://wapt.tranquil.it/debian/ ./ " > /etc/apt/sources.list.d/wapt.list
apt update
apt upgrade -y
apt install tis-waptrepo
CentOS/ RedHat Linux¶
The remote WAPT repository is set up, we now must implement the Syncthing replication.
Configuring the Syncthing service¶
a2enmod ssl
a2ensite default-ssl.conf
modify the Apache configuration files to define the correct roots of the VirtualHosts:
/etc/apache2/sites-available/default-ssl.conf
/etc/apache2/sites-available/000-default.conf
change the value of DocumentRoot in each configuration file:
- DocumentRoot /var/www/html
+ DocumentRoot /var/www
finally, restart the Apache web service to apply the new configuration:
/etc/init.d/apache2 restart
Note
It is advisable to specify valid SSL certificates in the Apache configuration of remote repositories.
empty the content of the folders
/var/www/waptand/var/www/wapt-host; Syncthing will fill again these folders with data from the main repository.
rm -rf /var/www/wapt/*
rm -rf /var/www/wapt-host/*
Installing Syncthing on main and remote WAPT repositories¶
Note
This procedure is to be applied on the main repository and on the remote repositories.
## Debian
apt update
apt install sudo curl apt-transport-https
curl -s https://syncthing.net/release-key.txt | apt-key add -
echo "deb https://apt.syncthing.net/ syncthing stable" | tee /etc/apt/sources.list.d/syncthing.list
apt update
apt install syncthing
## CentOS 7
wget https://github.com/mlazarov/syncthing-centos/releases/download/v0.14.7/syncthing-0.14.7-0.el7.centos.x86_64.rpm --no-check-certificate
yum install syncthing-0.14.7-0.el7.centos.x86_64.rpm
Configuring Syncthing¶
Operations to follow:
add the Syncthing service to systemd;
change the listening port to 0.0.0.0;
create an administrator account and enter a strong password;
activate the HTTPS protocole for the web access;
create the definition file for the waptsync service by editing
/etc/systemd/system/waptsync.service:[Unit] Description=WAPT respository sync with syncthing Documentation=http://docs.syncthing.net/ After=network.target ;Wants=syncthing-inotify@.service [Service] User=wapt ExecStart=/usr/bin/syncthing -logflags=0 -home=/opt/wapt/.config/syncthing/ -no-restart Restart=on-failure SuccessExitStatus=3 4 RestartForceExitStatus=3 4 [Install] WantedBy=multi-user.target
create the tree structure required for the waptsync service to start:
mkdir /opt/wapt/.config/
mkdir /opt/wapt/.config/syncthing/
change the owner of the files:
chown -R wapt:www-data /opt/wapt/.config/
activate the waptsync service and start it. The configuration files will appear in the
/opt/wapt/.config/syncthing/folder:
systemctl enable waptsync
systemctl start waptsync
systemctl stop waptsync
change the listening port in the
/opt/wapt/.config/syncthing/config.xmlfile:
<gui enabled="true" tls="true" debugging="false">
<address>0.0.0.0:8384</address>
<apikey>4jvEiL24UbFddsdsAQxqsfixNaLt</apikey>
<theme>default</theme>
</gui>
start the waptsync service:
systemctl start waptsync
Configuring Syncthing’s web service¶
Syncthing’s web interface is now accessible by visiting http://srvwapt.mydomain.lan:8384.
Operations to follow:
change the host name of the remote repository;
add a GUI authorized user;
add the password for the GUI authorized user;
check the box Use HTTPS for the GUI;
click on Save;
connect with SSH on the WAPT Server and restart the Syncthing service:
systemctl restart waptsync
Syncthing’s web interface is now only accessible with HTTPS on https://srvwapt.mydomain.lan:8384.
in the list of shared folders, remove the default folder: ;
configure the replication:
Note
Those actions must be run on WAPT Server.
In the list of shared folders (Shares):
add a shared folder with ;
fill in the path to the directory to be shared, ex:
/var/www/wapt/;in the scroll-down menu ;
in the scroll-down menu ;
redo the same operation for wapt-host:
/var/www/wapt-host/;
add the remote repository to WAPT Server’s Syncthing:
Once Syncthing has been installed on the main and remote repositories, recover the remote repository’s ID ().
This unique identifier appears like
DSINDDC-23ORDNM-PAK6FCL-ZJAKNCH-61GWXAT-77PC3JM-RZ4PPYP-K1QERAV
On the main repository, in the list of remote devices (Other Devices):
add the remote repository with ;
fill in the ID of the remote repository;
tick the checkbox for the shared folder wapt and wapt-host;
On the remote repository, follow these steps:
the remote device receives a Syncthing notification that it has been approved and added to the main device’s replication schedule;
the client receives a popup form asking to accept the synchronized shares
waptandwapt-host;
The replication is now operational.
secure the replication:
By default, the following parameters are active in Syncthing:
Options |
Description |
|---|---|
Activate the NAT |
Use a UPnP port mapping for incoming synchronization connections. |
Local discovery |
Syncthing will then broadcast to announce itself to other Syncthings. |
Global discovery |
Syncthing registers on an external cloud service and can use this cloud based service to search other Syncthing devices. |
Possible relay |
The use of relays allows to use external servers to relay the communications. The relay is activated by default but it will be used only if two devices can not communicate directly between themselves. |
This operating mode simplifies the global setup but it is not the most advisable method in relation to security.
uncheck all boxes in network configuration;
define the listening port (by default port 22000);
replace default by tcp://0.0.0.0:22000;
Then go on the web interface of the remote repositories, click on Change and fill in the IP address of the remote repository:
replace dynamic by tcp://<remote_repo_ip_address>:22000;
This configuration is useful for limiting inbound connections to the Syncthing service.
Configuring the WAPT agents¶
WAPT clients on remote sites must now be configured to look for updates from their closest available repositories.
Two solutions exist:
change manually or via a WAPT package the parameter repo_url in the WAPT agent’s
wapt-get.inifile;
Configuration example of the WAPT agent - filled in local repository:
[global]
waptupdate_task_period=120
waptserver=https://srvwapt.mydomain.lan
repo_url=https://localrepo.mydomain.lan/wapt/
use_hostpackages=1
Example of a WAPT package designed to remotely change the repo_url
in wapt-get.ini:
# -*- coding: utf-8 -*-
from setuphelpers import *
uninstallkey = []
def install():
print('Modifier la configuration agent pour le site de Colmar')
inifile_writestring(WAPT.config_filename,'global','repo_url',
'https://wapt.city02.mydomain.lan/wapt/')