Archives
/
osync
mirror of https://github.com/deajan/osync
2
0
Fork 0
Code Issues Releases Wiki Activity

Osync v0.99 Release candidate 2

Browse Source
pull/2/head
deajan 11 years ago
parent 5f51a1fb2d
commit 86b05d1c65
4 changed files with 135 additions and 82 deletions
Show Stats Download Patch File Download Diff File

22
CHANGELOG.md
Unescape Escape View File

@ -1,19 +1,29 @@
FUTURE IMPROVEMENTS SHORT FUTURE IMPROVEMENTS (post v1.0)
------------------- -------------------------------------
- Sync function merge (master and slave functions are more more or less the same) - Sync function merge (master and slave functions are the same, reduces code size and maintain effort)
- Tree function merge (current and after tree functions are the same except for output filename and logging) - Tree function merge (current and after tree functions are the same except for output filename and logging, reduces code size and maintain effort)
- Tree functions execute piped commands (grep, awk) on master when launched on remote slave which can cause more bandwith usage - Tree functions execute piped commands (grep, awk) on master when launched on remote slave which can cause more bandwith usage
- Fast sync mode (without config file, directly via command line by specifying two directories)
FAR FUTURE IMPROVEMENTS
-----------------------
- Rethink of .osync_workdir/state/* files with PIDs, Host and Task Names to better identify multiple instances on the same fileset
- Improve Master / Slave schema to Multimaster schema
KNOWN ISSUES KNOWN ISSUES
------------ ------------
- If master and remote slave systems don't have rsync in the same path, execution may fail (RSYNC_PATH is always configured on master, even when executed on slave) - None yet, need more testing on MSYS environment
- On MSYS, osync does not propagate deletions
RECENT CHANGES RECENT CHANGES
-------------- --------------
- 02 Nov. 2013: Osync 0.99 RC2
- Minor improvement on operating system detection
- Improved RunLocalCommand execution hook
- Minor improvements on permission checks
- Made more portability improvements (mostly for FreeBSD, must be run with bash shell) - Made more portability improvements (mostly for FreeBSD, must be run with bash shell)
- Added local and remote operating system detection - Added local and remote operating system detection
- Added forced usage of MSYS find on remote MSYS hosts - Added forced usage of MSYS find on remote MSYS hosts

31
README.md
Unescape Escape View File

@ -1,50 +1,50 @@
osync osync
===== =====
A two way sync script based that adds script fault tolerance from obackup project along with multiple usefull options. A two way sync script with fault tolerance, resuming and delete / conflict backups.
## About ## About
Having created obackup script in order to make reliable quick backups, i searched for a nice tool to handle two (or more) way sync scenarios in a reliable way. I searched for a nice tool to handle two (or more) way sync scenarios in a reliable way, easy to use and automate.
While unison handles these scenarios, it's pretty messy to configure, slow, won't handle ACLs and won't automatically resume if something bad happened.
While unison handles these scenarios, it's pretty messy to configure, slow, won't handle ACLs and won't resume if something bad happened.
Then i read about bitpocket, a nice script provided by Marcin Kulik (sickill) at https://github.com/sickill/bitpocket.git Then i read about bitpocket, a nice script provided by Marcin Kulik (sickill) at https://github.com/sickill/bitpocket.git
Bitpocked inspired me to write my own implementation of a two way sync script, implementing features i wanted among: Bitpocked inspired me to write my own implementation of a two way sync script, implementing features i wanted among:
- Fault tolerance with resume scenarios - Fault tolerance with resume scenarios
- Email alerts - Email alerts
- Logging facility - Logging facility
- Soft deletition and multiple backups handling - Soft deletition and multiple backups handling
- Before / after command execution - Before / after command execution
- Time control - Time control
Osync uses a master / slave sync schema. It can sync local or remote directories. By definition, master replica should always be a local directory on the system osync runs on. Osync uses a master / slave sync schema. It can sync local and local or local and remote directories. By definition, master replica should always be a local directory on the system osync runs on.
Also, osync uses pidlocks to prevent multiple concurrent sync processes on/to the same master / slave replica. Be sure a sync process is finished before launching next one. Also, osync uses pidlocks to prevent multiple concurrent sync processes on/to the same master / slave replica. Be sure a sync process is finished before launching next one.
You may launch concurrent sync processes on the same system but only for different master replicas. You may launch concurrent sync processes on the same system but only for different master replicas.
Currently, it has been tested on CentOS 5, CentOS 6, Debian 6.0.7, Linux Mint 14 and Ubuntu 12. Currently, it has been tested on CentOS 5, CentOS 6, Debian 6.0.7, Linux Mint 14, Ubuntu 12.
Osync also runs on FreeBSD and Windows MSYS environment, altough it is not fully tested yet.
## Installation ## Installation
Osync developpment is still not finished. It's currently at beta stage. Please read CHANGELOG.md for a list of known bugs.
Keep in mind that Osync has been designed to not delete any data, but rather make backups or soft deletes. Keep in mind that Osync has been designed to not delete any data, but rather make backups or soft deletes.
Nevertheless, as we're still in beta stage, please make a backup of your data before using Osync. Nevertheless, still consider making backups of your data before trying a sync tool.
First, grab a fresh copy of osync and make it executable: First, grab a fresh copy of osync and make it executable:
$ git clone https://github.com/deajan/osync $ git clone https://github.com/deajan/osync
$ chmod +x ./osync.sh $ chmod +x ./osync.sh
Osync needs to run with bash shell. Using any other shell will most probably result in lots of errors.
There is no need to intialize anything. You can begin sync with two already filled directories. There is no need to intialize anything. You can begin sync with two already filled directories.
You only have to copy the sync.conf file to let's say your.conf and then edit it according to your needs. You only have to customize the sync.conf file according to your needs.
Osync needs a pair of private / public RSA keys to perform remote SSH connections. Osync needs a pair of private / public RSA keys to perform remote SSH connections.
Also, using SUDO_EXEC option requires to configure /etc/sudoers file. Also, running sync as superuser requires to configure /etc/sudoers file.
Documentation is being written, meanwhile you can check Obackup documentation at http://netpower.fr/projects/obackup/documentation.html for the two configurations points above. Please read the documentation on author's site.
## Usage ## Usage
Once you've setup a personalized sync.conf file, you may run osync with the following test run: Once you've customized a sync.conf file, you may run osync with the following test run:
$ ./osync.sh /path/to/your.conf --dry $ ./osync.sh /path/to/your.conf --dry
@ -61,8 +61,7 @@ Once you're confident about your fist runs, you may add osync as cron task with:
$ ./osync.sh /path/to/your.conf --silent $ ./osync.sh /path/to/your.conf --silent
You may then find osync output in /var/log/osync-*.log You may then find osync output in /var/log/osync-*.log (or current directory if /var/log is not writable).
Also, you may always find detailed rsync command results at /tmp/osync_* if verbose switch wasn't specified.
## Author ## Author

84
osync.sh
Unescape Escape View File

@ -2,8 +2,8 @@
###### Osync - Rsync based two way sync engine with fault tolerance ###### Osync - Rsync based two way sync engine with fault tolerance
###### (L) 2013 by Orsiris "Ozy" de Jong (www.netpower.fr) ###### (L) 2013 by Orsiris "Ozy" de Jong (www.netpower.fr)
OSYNC_VERSION=0.99preRC2-MSYS-FreeBSD-compatible OSYNC_VERSION=0.99RC2
OSYNC_BUILD=1110201302 OSYNC_BUILD=0211201301
DEBUG=no DEBUG=no
SCRIPT_PID=$$ SCRIPT_PID=$$
@ -12,8 +12,7 @@ LOCAL_USER=$(whoami)
LOCAL_HOST=$(hostname) LOCAL_HOST=$(hostname)
## Default log file until config file is loaded ## Default log file until config file is loaded
LOG_FILE=/var/log/osync.log if [ -w /var/log ]
if [ -d /var/log ]
then then
LOG_FILE=/var/log/osync.log LOG_FILE=/var/log/osync.log
else else
@ -21,13 +20,13 @@ else
fi fi
## Default directory where to store run files ## Default directory where to store run files
if [ -d /dev/shm ] if [ -w /dev/shm ]
then then
RUN_DIR=/dev/shm RUN_DIR=/dev/shm
elif [ -d /tmp ] elif [ -w /tmp ]
then then
RUN_DIR=/tmp RUN_DIR=/tmp
elif [ -d /var/tmp ] elif [ -w /var/tmp ]
then then
RUN_DIR=/var/tmp RUN_DIR=/var/tmp
else else
@ -100,7 +99,7 @@ function TrapQuit
if type -p pkill > /dev/null 2>&1 if type -p pkill > /dev/null 2>&1
then then
pkill -TERM -P $$ pkill -TERM -P $$
elif [ "$LOCAL_OS" == "msys" ] elif [ "$LOCAL_OS" == "msys" ] || [ "$OSTYPE" == "msys" ]
then then
## This is not really a clean way to get child process pids, especially the tail -n +2 which resolves a strange char apparition in msys bash ## This is not really a clean way to get child process pids, especially the tail -n +2 which resolves a strange char apparition in msys bash
for pid in $(ps -a | awk '{$1=$1}$1' | awk '{print $1" "$2}' | grep " $$$" | awk '{print $1}' | tail -n +2) for pid in $(ps -a | awk '{$1=$1}$1' | awk '{print $1" "$2}' | grep " $$$" | awk '{print $1}' | tail -n +2)
@ -271,8 +270,13 @@ function GetOperatingSystem
LOCAL_OS_VAR=$(uname -spio) LOCAL_OS_VAR=$(uname -spio)
if [ "$REMOTE_SYNC" == "yes" ] if [ "$REMOTE_SYNC" == "yes" ]
then then
eval "$SSH_CMD uname -spio > $RUN_DIR/osync_remote_os_$SCRIPT_PID 2>&1 &" eval "$SSH_CMD \"uname -spio\" > $RUN_DIR/osync_remote_os_$SCRIPT_PID 2>&1"
REMOTE_OS_VAR=$(cat $RUN_DIR/osync_remote_os_$SCRIPT_PID) if [ $? != 0 ]
then
LogError "Cannot Get remote OS type."
else
REMOTE_OS_VAR=$(cat $RUN_DIR/osync_remote_os_$SCRIPT_PID)
fi
fi fi
case $LOCAL_OS_VAR in case $LOCAL_OS_VAR in
@ -307,6 +311,15 @@ function GetOperatingSystem
LogError "Running on remote >> $REMOTE_OS_VAR << not supported. Please report to the author." LogError "Running on remote >> $REMOTE_OS_VAR << not supported. Please report to the author."
exit 1 exit 1
esac esac
if [ "$DEBUG" == "yes" ]
then
Log "Local OS: [$LOCAL_OS_VAR]."
if [ "$REMOTE_BACKUP" == "yes" ]
then
Log "Remote OS: [$REMOTE_OS_VAR]."
fi
fi
} }
# Waits for pid $1 to complete. Will log an alert if $2 seconds passed since current task execution unless $2 equals 0. # Waits for pid $1 to complete. Will log an alert if $2 seconds passed since current task execution unless $2 equals 0.
@ -417,15 +430,16 @@ function RunLocalCommand
Log "Dryrun: Local command [$1] not run." Log "Dryrun: Local command [$1] not run."
return 1 return 1
fi fi
$1 > $RUN_DIR/osync_run_local_$SCRIPT_PID 2>&1 & Log "Running command [$1] on local host."
eval "$1" > $RUN_DIR/osync_run_local_$SCRIPT_PID 2>&1 &
child_pid=$! child_pid=$!
WaitForTaskCompletion $child_pid 0 $2 WaitForTaskCompletion $child_pid 0 $2
retval=$? retval=$?
if [ $retval -eq 0 ] if [ $retval -eq 0 ]
then then
Log "Running command [$1] on local host succeded." Log "Command succeded."
else else
LogError "Running command [$1] on local host failed." LogError "Command failed."
fi fi
if [ $verbose -eq 1 ] if [ $verbose -eq 1 ]
@ -433,7 +447,7 @@ function RunLocalCommand
Log "Command output:\n$(cat $RUN_DIR/osync_run_local_$SCRIPT_PID)" Log "Command output:\n$(cat $RUN_DIR/osync_run_local_$SCRIPT_PID)"
fi fi
if [ "$STOP_ON_CMD_ERROR" == "yes" ] if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]
then then
exit 1 exit 1
fi fi
@ -449,15 +463,16 @@ function RunRemoteCommand
Log "Dryrun: Local command [$1] not run." Log "Dryrun: Local command [$1] not run."
return 1 return 1
fi fi
Log "Running command [$1] on remote host."
eval "$SSH_CMD \"$1\" > $RUN_DIR/osync_run_remote_$SCRIPT_PID 2>&1 &" eval "$SSH_CMD \"$1\" > $RUN_DIR/osync_run_remote_$SCRIPT_PID 2>&1 &"
child_pid=$! child_pid=$!
WaitForTaskCompletion $child_pid 0 $2 WaitForTaskCompletion $child_pid 0 $2
retval=$? retval=$?
if [ $retval -eq 0 ] if [ $retval -eq 0 ]
then then
Log "Running command [$1] succeded." Log "Command succeded."
else else
LogError "Running command [$1] failed." LogError "Command failed."
fi fi
if [ -f $RUN_DIR/osync_run_remote_$SCRIPT_PID ] && [ $verbose -eq 1 ] if [ -f $RUN_DIR/osync_run_remote_$SCRIPT_PID ] && [ $verbose -eq 1 ]
@ -465,7 +480,7 @@ function RunRemoteCommand
Log "Command output:\n$(cat $RUN_DIR/osync_run_remote_$SCRIPT_PID)" Log "Command output:\n$(cat $RUN_DIR/osync_run_remote_$SCRIPT_PID)"
fi fi
if [ "$STOP_ON_CMD_ERROR" == "yes" ] if [ "$STOP_ON_CMD_ERROR" == "yes" ] && [ $retval -ne 0 ]
then then
exit 1 exit 1
fi fi
@ -755,7 +770,7 @@ function LockDirectories
eval "$SSH_CMD \"if [ -f \\\"$SLAVE_STATE_DIR/lock\\\" ]; then cat \\\"$SLAVE_STATE_DIR/lock\\\"; fi\" > $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID" & eval "$SSH_CMD \"if [ -f \\\"$SLAVE_STATE_DIR/lock\\\" ]; then cat \\\"$SLAVE_STATE_DIR/lock\\\"; fi\" > $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID" &
child_pid=$! child_pid=$!
WaitForTaskCompletion $child_pid 0 1800 WaitForTaskCompletion $child_pid 0 1800
if [ -d $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID ] if [ -f $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID ]
then then
slave_lock_pid=$(cat $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID | cut -d'@' -f1) slave_lock_pid=$(cat $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID | cut -d'@' -f1)
slave_lock_id=$(cat $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID | cut -d'@' -f2) slave_lock_id=$(cat $RUN_DIR/osync_remote_slave_lock_$SCRIPT_PID | cut -d'@' -f2)
@ -1319,6 +1334,9 @@ function SoftDelete
else else
Log "Conflict backup cleanup complete on master replica." Log "Conflict backup cleanup complete on master replica."
fi fi
elif [ -d "$MASTER_BACKUP_DIR" ] && ! [ -w "$MASTER_BACKUP_DIR" ]
then
LogError "Warning: Master replica conflict backup dir [$MASTER_BACKUP_DIR] isn't writable. Cannot clean old files."
fi fi
if [ "$REMOTE_SYNC" == "yes" ] if [ "$REMOTE_SYNC" == "yes" ]
@ -1328,9 +1346,9 @@ function SoftDelete
Log "Removing backups older than $CONFLICT_BACKUP_DAYS days on remote slave replica." Log "Removing backups older than $CONFLICT_BACKUP_DAYS days on remote slave replica."
if [ $dryrun -eq 1 ] if [ $dryrun -eq 1 ]
then then
eval "$SSH_CMD \"if [ -d \\\"$SLAVE_BACKUP_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_BACKUP_DIR/\\\" -ctime +$CONFLICT_BACKUP_DAYS; fi\"" eval "$SSH_CMD \"if [ -w \\\"$SLAVE_BACKUP_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_BACKUP_DIR/\\\" -ctime +$CONFLICT_BACKUP_DAYS; fi\""
else else
eval "$SSH_CMD \"if [ -d \\\"$SLAVE_BACKUP_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_BACKUP_DIR/\\\" -ctime +$CONFLICT_BACKUP_DAYS | xargs rm -rf; fi\"" eval "$SSH_CMD \"if [ -w \\\"$SLAVE_BACKUP_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_BACKUP_DIR/\\\" -ctime +$CONFLICT_BACKUP_DAYS | xargs rm -rf; fi\""
fi fi
child_pid=$! child_pid=$!
WaitForCompletion $child_pid $SOFT_MAX_EXEC_TIME 0 WaitForCompletion $child_pid $SOFT_MAX_EXEC_TIME 0
@ -1342,7 +1360,7 @@ function SoftDelete
Log "Conflict backup cleanup complete on slave replica." Log "Conflict backup cleanup complete on slave replica."
fi fi
else else
if [ -d "$SLAVE_BACKUP_DIR" ] if [ -w "$SLAVE_BACKUP_DIR" ]
then then
Log "Removing backups older than $CONFLICT_BACKUP_DAYS days on slave replica." Log "Removing backups older than $CONFLICT_BACKUP_DAYS days on slave replica."
if [ $dryrun -eq 1 ] if [ $dryrun -eq 1 ]
@ -1360,13 +1378,16 @@ function SoftDelete
else else
Log "Conflict backup cleanup complete on slave replica." Log "Conflict backup cleanup complete on slave replica."
fi fi
elif [ -d "$SLAVE_BACKUP_DIR" ] && ! [ -w "$SLAVE_BACKUP_DIR" ]
then
LogError "Warning: Slave replica conflict backup dir [$SLAVE_BACKUP_DIR] isn't writable. Cannot clean old files."
fi fi
fi fi
fi fi
if [ "$SOFT_DELETE" != "no" ] if [ "$SOFT_DELETE" != "no" ]
then then
if [ -d "$MASTER_DELETE_DIR" ] if [ -w "$MASTER_DELETE_DIR" ]
then then
Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on master replica." Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on master replica."
if [ $dryrun -eq 1 ] if [ $dryrun -eq 1 ]
@ -1384,6 +1405,9 @@ function SoftDelete
else else
Log "Soft delete cleanup complete on master replica." Log "Soft delete cleanup complete on master replica."
fi fi
elif [ -d "$MASTER_DELETE_DIR" ] && ! [ -w $MASTER_DELETE_DIR ]
then
LogError "Warning: Master replica deletion backup dir [$MASTER_DELETE_DIR] isn't writable. Cannot clean old files."
fi fi
if [ "$REMOTE_SYNC" == "yes" ] if [ "$REMOTE_SYNC" == "yes" ]
@ -1393,9 +1417,9 @@ function SoftDelete
Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on remote slave replica." Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on remote slave replica."
if [ $dryrun -eq 1 ] if [ $dryrun -eq 1 ]
then then
eval "$SSH_CMD \"if [ -d \\\"$SLAVE_DELETE_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_DELETE_DIR/\\\" -ctime +$SOFT_DELETE_DAYS; fi\"" eval "$SSH_CMD \"if [ -w \\\"$SLAVE_DELETE_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_DELETE_DIR/\\\" -ctime +$SOFT_DELETE_DAYS; fi\""
else else
eval "$SSH_CMD \"if [ -d \\\"$SLAVE_DELETE_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_DELETE_DIR/\\\" -ctime +$SOFT_DELETE_DAYS | xargs rm -rf; fi\"" eval "$SSH_CMD \"if [ -w \\\"$SLAVE_DELETE_DIR\\\" ]; then $COMMAND_SUDO $REMOTE_FIND_CMD \\\"$SLAVE_DELETE_DIR/\\\" -ctime +$SOFT_DELETE_DAYS | xargs rm -rf; fi\""
fi fi
child_pid=$! child_pid=$!
WaitForCompletion $child_pid $SOFT_MAX_EXEC_TIME 0 WaitForCompletion $child_pid $SOFT_MAX_EXEC_TIME 0
@ -1408,7 +1432,7 @@ function SoftDelete
fi fi
else else
if [ -d "$SLAVE_DELETE_DIR" ] if [ -w "$SLAVE_DELETE_DIR" ]
then then
Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on slave replica." Log "Removing soft deleted items older than $SOFT_DELETE_DAYS days on slave replica."
if [ $dryrun -eq 1 ] if [ $dryrun -eq 1 ]
@ -1426,6 +1450,9 @@ function SoftDelete
else else
Log "Soft delete cleanup complete on slave replica." Log "Soft delete cleanup complete on slave replica."
fi fi
elif [ -d "$SLAVE_DELETE_DIR" ] && ! [ -w "$SLAVE_DELETE_DIR" ]
then
LogError "Warning: Slave replica deletion backup dir [$SLAVE_DELETE_DIR] isn't writable. Cannot clean old files."
fi fi
fi fi
fi fi
@ -1433,8 +1460,6 @@ function SoftDelete
function Init function Init
{ {
GetOperatingSystem
# Set error exit code if a piped command fails # Set error exit code if a piped command fails
set -o pipefail set -o pipefail
set -o errtrace set -o errtrace
@ -1448,7 +1473,7 @@ function Init
if [ "$LOGFILE" == "" ] if [ "$LOGFILE" == "" ]
then then
if [ -d /var/log ] if [ -w /var/log ]
then then
LOG_FILE=/var/log/osync_$OSYNC_VERSION-$SYNC_ID.log LOG_FILE=/var/log/osync_$OSYNC_VERSION-$SYNC_ID.log
else else
@ -1556,7 +1581,7 @@ function Init
RSYNC_ARGS="" RSYNC_ARGS=""
fi fi
if [ "$BANDWIDTH" != "0" ] if [ "$BANDWIDTH" != "" ] && [ "$BANDWIDTH" != "0" ]
then then
RSYNC_ARGS=$RSYNC_ARGS" --bwlimit=$BANDWIDTH" RSYNC_ARGS=$RSYNC_ARGS" --bwlimit=$BANDWIDTH"
fi fi
@ -1665,6 +1690,7 @@ then
if [ $? == 0 ] if [ $? == 0 ]
then then
Init Init
GetOperatingSystem
DATE=$(date) DATE=$(date)
Log "-------------------------------------------------------------" Log "-------------------------------------------------------------"
Log "$DRY_WARNING $DATE - Osync v$OSYNC_VERSION script begin." Log "$DRY_WARNING $DATE - Osync v$OSYNC_VERSION script begin."

80
sync.conf
Unescape Escape View File

@ -2,38 +2,44 @@
###### Osync - Rsync based two way sync engine with fault tolerance ###### Osync - Rsync based two way sync engine with fault tolerance
###### (L) 2013 by Orsiris "Ozy" de Jong (www.netpower.fr) ###### (L) 2013 by Orsiris "Ozy" de Jong (www.netpower.fr)
#### Config file rev 1010201302 #### Config file rev 0211201302
## Sync job identification, any string you want, no spaces ## ---------- GENERAL OPTIONS
SYNC_ID="sync_test"
## Leaving this empty will create a logfile at /var/log/osync_version_SYNC_ID.log (or current directory if /var/log doesn't exist) ## Sync job identification
LOGFILE="" SYNC_ID="sync_test"
## Directories to synchronize ## Directories to synchronize. Master must be on the system Osync runs on. Slave can be either on the same system, or on a remote one.
MASTER_SYNC_DIR="/home/git/osync/test/dir1" MASTER_SYNC_DIR="/home/git/osync/test/dir1"
SLAVE_SYNC_DIR="/home/git/osync/test/dir2" SLAVE_SYNC_DIR="/home/git/osync/test/dir2"
## Create sync directories if they do not exist ## Create sync directories if they do not exist
CREATE_DIRS=no CREATE_DIRS=no
## List of directories to exclude in sync on both sides (rsync patterns, wildcards work). Must be relative paths. List is separated by PATH SEPARATOR CHAR defined below (semicolon by default). ## Log file location. Leaving this empty will create a logfile at /var/log/osync_version_SYNC_ID.log (or current directory if /var/log doesn't exist)
LOGFILE=""
## List of directories to exclude from sync on both sides (rsync patterns, wildcards work).
## Paths are relative to sync dirs. List elements are separated by a semicolon.
RSYNC_EXCLUDE_PATTERN="tmp;archives" RSYNC_EXCLUDE_PATTERN="tmp;archives"
## You might change this separator case in the unholy case that your filename may contain semicolons. Change it then to whatever unholy char you want. ## List elements separator char. You may set an alternative seperator char for your directories lists above.
PATH_SEPARATOR_CHAR=";" PATH_SEPARATOR_CHAR=";"
## Generate an alert if master or slave have lass space than given value in KB. ## Generate an alert if master or slave replicas have less free space than given value in KB.
MINIMUM_SPACE=10240 MINIMUM_SPACE=10240
## Bandwidth limit Kbytes / second. Leave 0 to disable limitation ## Bandwidth limit Kbytes / second. Leave 0 to disable limitation
BANDWIDTH=0 BANDWIDTH=0
## If enabled, synchronization will be processed with sudo command. See documentation ## If enabled, synchronization will be processed as superuser. See documentation for /etc/sudoers file configuration.
SUDO_EXEC=no SUDO_EXEC=no
## Paranoia option. Don't change this unless you read the documentation and know what you are doing. ## Paranoia option. Don't change this unless you read the documentation.
RSYNC_EXECUTABLE=rsync RSYNC_EXECUTABLE=rsync
##Remote options (will sync slave through ssh tunnel, needs RSA key. See documentation for remote sync. ## ---------- REMOTE SYNC OPTIONS
## The following options allow Osync to sync a slave replica on a remote system via an SSH tunnel.
## Needs public RSA key need to be put into ~/.ssh/authorized_keys in remote users home directory. See documentation for remote sync.
REMOTE_SYNC=no REMOTE_SYNC=no
SSH_RSA_PRIVATE_KEY=~/.ssh/id_rsa SSH_RSA_PRIVATE_KEY=~/.ssh/id_rsa
REMOTE_USER=syncuser REMOTE_USER=syncuser
@ -41,68 +47,80 @@ REMOTE_HOST=your-remote-host.tld
REMOTE_PORT=22 REMOTE_PORT=22
## ssh compression should be used unless your remote connection is good enough (LAN) ## ssh compression should be used unless your remote connection is good enough (LAN)
SSH_COMPRESSION=yes SSH_COMPRESSION=yes
## Check for connectivity to remote host before launching remote sync task. Be sure the hosts responds to ping. Failing to ping will stop sync. ## Check for connectivity to remote host before launching remote sync task. Be sure the hosts responds to ping. Failing to ping will stop sync.
REMOTE_HOST_PING=no REMOTE_HOST_PING=no
## Check for internet access by pinging one or more 3rd party hosts before remote sync task. Leave empty if you don't want this check to be be performed. Failing to ping will stop sync. ## Check for internet access by pinging one or more 3rd party hosts before remote sync task. Leave empty if you don't want this check to be be performed. Failing to ping will stop sync.
## If you use this function, you should set more than one 3rd party host, and be sure you can ping them. ## If you use this function, you should set more than one 3rd party host, and be sure you can ping them.
## Be aware some DNS like opendns redirect false hostnames. Also, this adds an extra execution time of a bit less than a minute. ## Be aware some DNS like opendns redirect false hostnames. Also, this adds an extra execution time of a bit less than a minute.
REMOTE_3RD_PARTY_HOSTS="www.kernel.org www.google.fr" REMOTE_3RD_PARTY_HOSTS="www.kernel.org www.google.fr"
## Remote rsync executable path. Leave this empty in most cases ## Remote rsync executable path. Leave this empty in most cases
REMOTE_RSYNC_PATH="" REMOTE_RSYNC_PATH=""
## Preserve ACLS. Make sure target FS can hold ACLs or you'll get loads of errors. ## ---------- MISC OPTIONS
## Preserve ACLS. Make sure source and target FS can manage same ACLs or you'll get loads of errors.
PRESERVE_ACL=no PRESERVE_ACL=no
## Preserve Xattr ## Preserve Xattr. Make sure source and target FS can manage same Xattrs or you'll get loads of errors.
PRESERVE_XATTR=no PRESERVE_XATTR=no
## Let RSYNC compress file transfers. Do not use if you already enabled SSH compression.
## Let RSYNC compress file transfers. Do not use this if both master and slave replicas are on local system. Also, do not use this if you already enabled SSH compression.
RSYNC_COMPRESS=yes RSYNC_COMPRESS=yes
## Maximum execution time (in seconds) for sync process. Soft exec time only generates warning. Hard exec time will generate warning and stop sync process. ## Maximum execution time (in seconds) for sync process. Soft exec time only generates a warning. Hard exec time will generate a warning and stop sync process.
SOFT_MAX_EXEC_TIME=7200 SOFT_MAX_EXEC_TIME=7200
HARD_MAX_EXEC_TIME=10600 HARD_MAX_EXEC_TIME=10600
## If the same file exists on both sides, newer version will be used. If both files have the same timestamp but differ, CONFILCT_PREVALANCE sets winner ## ---------- BACKUP AND TRASH OPTIONS
CONFLICT_PREVALANCE=master
## Keep a backup of a file if gets updated from remote side ## Enabling this option will keep a backup of a file on the target replica if it gets updated from the source replica. Backups will be made to .osync_workdir/backups
CONFLICT_BACKUP=yes CONFLICT_BACKUP=yes
## Keep multiple backups of a file if it gets updated from remote side. This can be very space consuming ## Keep multiple backup versions of the same file. Warning, This can be very space consuming.
CONFLICT_BACKUP_MULTIPLE=no CONFLICT_BACKUP_MULTIPLE=no
## Number of days to keep backups ## Osync will clean backup files after a given number of days. Setting this to 0 will disable cleaning and keep backups forever. Warning: This can be very space consuming.
CONFLICT_BACKUP_DAYS=30 CONFLICT_BACKUP_DAYS=30
## If the same file exists on both replicas, newer version will be synced. However, if both files have the same timestamp but differ, CONFILCT_PREVALANCE sets winner replica.
CONFLICT_PREVALANCE=master
## On deletition propagation to sync partner, keep a backup of deleted files on sync partner ## On deletition propagation to the target replica, a backup of the deleted files can be kept. Deletions will be kept in .osync_workdir/deleted
SOFT_DELETE=yes SOFT_DELETE=yes
## Number of days to keep deleted files ## Osync will clean deleted files after a given number of days. Setting this to 0 will disable cleaning and keep deleted files forever. Warning: This can be very space consuming.
SOFT_DELETE_DAYS=30 SOFT_DELETE_DAYS=30
## ---------- RESUME OPTIONS
## Try to resume an aborted sync task ## Try to resume an aborted sync task
RESUME_SYNC=yes RESUME_SYNC=yes
## Number maximum resume tries before initating a new sync ## Number maximum resume tries before initating a fresh sync.
RESUME_TRY=2 RESUME_TRY=2
## When a pidlock exists on slave that does not correspond to master's sync-id, force pidlock removal. Be carefull with this option if you have multiple masters. ## When a pidlock exists on slave replica that does not correspond to master's sync-id, force pidlock removal. Be carefull with this option if you have multiple masters.
FORCE_STRANGER_LOCK_RESUME=no FORCE_STRANGER_LOCK_RESUME=no
## ---------- ALERT OPTIONS
## List of alert mails separated by spaces ## List of alert mails separated by spaces
DESTINATION_MAILS="your@alert.tld" DESTINATION_MAILS="your@alert.tld"
## Windows only mail options (used by sendemail.exe) ## Windows (MSYS environment) only mail options (used by sendemail.exe)
SENDER_MAIL="alert@your.system" SENDER_MAIL="alert@your.system"
SMTP_SERVER=smtp.your.isp.com SMTP_SERVER=smtp.your.isp.com
SMTP_USER= SMTP_USER=
SMTP_PASSWORD= SMTP_PASSWORD=
## Run local commands before and after sync task ## ---------- EXECUTION HOOKS
## Commands can will be run before and / or after sync process (remote execution will only happen if REMOTE_SYNC is set).
LOCAL_RUN_BEFORE_CMD="" LOCAL_RUN_BEFORE_CMD=""
LOCAL_RUN_AFTER_CMD="" LOCAL_RUN_AFTER_CMD=""
## Run commands on remote slave befre and after sync task
REMOTE_RUN_BEFORE_CMD="" REMOTE_RUN_BEFORE_CMD=""
REMOTE_RUN_AFTER_CMD="" REMOTE_RUN_AFTER_CMD=""
## Maximum execution time (in seconds) for commands before sync task. Commands get killed if not finished after MAX_EXC_TIME. Set this to 0 to disable killing. ## Max execution time of commands before they get force killed. Leave 0 if you don't wan't this to happen. Time is specified in seconds.
MAX_EXEC_TIME_PER_CMD_BEFORE=0 MAX_EXEC_TIME_PER_CMD_BEFORE=0
## Maximum execution time (in seconds) for commands after sync task. Commands get killed if not finished after MAX_EXEC_TIME. Set this to 0 to disable killing command.
MAX_EXEC_TIME_PER_CMD_AFTER=0 MAX_EXEC_TIME_PER_CMD_AFTER=0
## Stops osync execution if one of the above commands fail ## Stops Osync execution if one of the above commands fail
STOP_ON_CMD_ERROR=yes STOP_ON_CMD_ERROR=yes

Write Preview
Loading…
Cancel
Save