Archives
/
osync
mirror of https://github.com/deajan/osync
2
0
Fork 0
Code Issues Releases Wiki Activity

Merge branch 'master' of https://github.com/deajan/osync

Browse Source
pull/260/head
Orsiris de Jong 8 months ago
parent 912906ef8c f39fb6613e
commit 19dbdeb28a
14 changed files with 320 additions and 218 deletions
Show Stats Download Patch File Download Diff File

8
.github/workflows/linux.yml vendored
Unescape Escape View File

@ -15,9 +15,11 @@ jobs:
run: | run: |
sudo apt-get install inotify-tools acl sudo apt-get install inotify-tools acl
- name: Execute tests and generate coverage report - name: Execute tests and generate coverage report
env:
RUNNING_ON_GITHUB_ACTIONS: true
run: | run: |
sudo bash ./dev/tests/run_tests.sh export RUNNING_ON_GITHUB_ACTIONS=true
export SSH_PORT=22
echo "Running on github actions: ${RUNNING_ON_GITHUB_ACTIONS}"
echo "Running on ssh port ${SSH_PORT}"
sudo -E bash ./dev/tests/run_tests.sh
- name: Upload Coverage to Codecov - name: Upload Coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v1

6
.github/workflows/macos.yml vendored
Unescape Escape View File

@ -20,9 +20,9 @@ jobs:
brew install fswatch brew install fswatch
echo "/usr/local/bin" >> $GITHUB_PATH echo "/usr/local/bin" >> $GITHUB_PATH
- name: Execute tests and generate coverage report - name: Execute tests and generate coverage report
env:
RUNNING_ON_GITHUB_ACTIONS: true
run: | run: |
sudo bash ./dev/tests/run_tests.sh export RUNNING_ON_GITHUB_ACTIONS=true
export SSH_PORT=22
sudo -E bash ./dev/tests/run_tests.sh
- name: Upload Coverage to Codecov - name: Upload Coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v1

8
.github/workflows/windows.yml vendored
Unescape Escape View File

@ -16,12 +16,14 @@ jobs:
additional-packages: additional-packages:
dos2unix dos2unix
rsync rsync
openssh-server
- name: Execute tests and generate coverage report - name: Execute tests and generate coverage report
shell: wsl-bash {0} shell: wsl-bash {0}
env:
RUNNING_ON_GITHUB_ACTIONS: true
run: | run: |
find . -type f -print0 | xargs -0 -n 1 -P 4 dos2unix export RUNNING_ON_GITHUB_ACTIONS=true
export SSH_PORT=22
find ./ -type f ! -path "./.git/*" -print0 | xargs -0 -n 1 -P 4 dos2unix
service ssh start
./dev/tests/run_tests.sh ./dev/tests/run_tests.sh
- name: Upload Coverage to Codecov - name: Upload Coverage to Codecov
uses: codecov/codecov-action@v1 uses: codecov/codecov-action@v1

5
CHANGELOG.md
Unescape Escape View File

@ -1,13 +1,14 @@
## RECENT CHANGES ## RECENT CHANGES
### dd Mmm YYYY: To be done ### Current master
- Make --log-conflicts non experimental (randomly fails) - Make --log-conflicts non experimental (randomly fails)
- ! new option FORCE_CONFLICT_PREVALANCE which will always use Initiator or Target, regardless of best time - ! new option FORCE_CONFLICT_PREVALANCE which will always use Initiator or Target, regardless of best time
- ! target-helper: destination mails etc on target, also, no cmd after on configs - ! target-helper: destination mails etc on target, also, no cmd after on configs
### dd Mmm YYYY: osync v1.3 release (for full changelog since v1.2 branch see all v1.3-beta/RC entries) ### 16 June 2023: osync v1.3 release (for full changelog since v1.2 branch see all v1.3-beta/RC entries)
- Fix for new RSYNC protocol
- New options ALWAYS_SEND_MAILS to allow sending logs regardless of execution states - New options ALWAYS_SEND_MAILS to allow sending logs regardless of execution states
### 29 June 2020: osync v1.3-RC1 release ### 29 June 2020: osync v1.3-RC1 release

26
README.md
Unescape Escape View File

@ -1,4 +1,12 @@
# osync [![Build Status](https://travis-ci.org/deajan/osync.svg?branch=master)](https://travis-ci.org/deajan/osync) [![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause) [![GitHub Release](https://img.shields.io/github/release/deajan/osync.svg?label=Latest)](https://github.com/deajan/osync/releases/latest) [![Percentage of issues still open](http://isitmaintained.com/badge/open/deajan/osync.svg)](http://isitmaintained.com/project/deajan/osync "Percentage of issues still open") [![Codacy Badge](https://api.codacy.com/project/badge/Grade/651acb2fd64642eb91078ba523b7f887)](https://www.codacy.com/app/ozy/osync?utm_source=github.com&utm_medium=referral&utm_content=deajan/osync&utm_campaign=Badge_Grade) # osync
[![License](https://img.shields.io/badge/License-BSD%203--Clause-blue.svg)](https://opensource.org/licenses/BSD-3-Clause)
[![GitHub Release](https://img.shields.io/github/release/deajan/osync.svg?label=Latest)](https://github.com/deajan/osync/releases/latest)
[![Percentage of issues still open](http://isitmaintained.com/badge/open/deajan/osync.svg)](http://isitmaintained.com/project/deajan/osync "Percentage of issues still open")
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/651acb2fd64642eb91078ba523b7f887)](https://www.codacy.com/app/ozy/osync?utm_source=github.com&utm_medium=referral&utm_content=deajan/osync&utm_campaign=Badge_Grade)
[![linux tests](https://github.com/deajan/osync/actions/workflows/linux.yml/badge.svg)](https://github.com/deajan/osync/actions/workflows/linux.yml/badge.svg)
[![windows tests](https://github.com/deajan/osync/actions/workflows/windows.yml/badge.svg)](https://github.com/deajan/osync/actions/workflows/windows.yml/badge.svg)
[![macos tests](https://github.com/deajan/osync/actions/workflows/macos.yml/badge.svg)](https://github.com/deajan/osync/actions/workflows/macos.yml/badge.svg)
A two way filesync script running on bash Linux, BSD, Android, MacOSX, Cygwin, MSYS2, Win10 bash and virtually any system supporting bash). A two way filesync script running on bash Linux, BSD, Android, MacOSX, Cygwin, MSYS2, Win10 bash and virtually any system supporting bash).
File synchronization is bidirectional, and can be run manually, as scheduled task, or triggered on file changes in daemon mode. File synchronization is bidirectional, and can be run manually, as scheduled task, or triggered on file changes in daemon mode.
@ -30,8 +38,8 @@ osync uses pidlocks to prevent multiple concurrent sync processes on/to the same
You may launch concurrent sync processes on the same system but as long as the replicas to synchronize are different. You may launch concurrent sync processes on the same system but as long as the replicas to synchronize are different.
Multiple osync tasks may be launched sequentially by osync osync-batch tool. Multiple osync tasks may be launched sequentially by osync osync-batch tool.
Currently, it has been tested on CentOS 5.x, 6.x, 7.x, Fedora 22-25, Debian 6-8, Linux Mint 14-18, Ubuntu 12.04-12.10, FreeBSD 8.3-11, Mac OS X, QTS 4.5.1(x86) and pfSense 2.3.x. Currently, it has been tested on CentOS 5.x, 6.x, 7.x, AlmaLinux 9, Fedora 22-25, Debian 6-11, Linux Mint 14-18, Ubuntu 12.04-22.04, FreeBSD 8.3-11, Mac OS X, QTS 4.5.1(x86) and pfSense 2.3.x.
Microsoft Windows is supported via MSYS or Cygwin and now via Windows 10 bash. Microsoft Windows is supported via MSYS, Cygwin and or via WSL.
Android support works via Termux. Android support works via Termux.
Some users also have successfully used osync on Gentoo and created an openRC init scriptt for it. Some users also have successfully used osync on Gentoo and created an openRC init scriptt for it.
@ -40,12 +48,6 @@ Some users also have successfully used osync on Gentoo and created an openRC ini
osync has been designed to not delete any data, but rather make backups of conflictual files or soft deletes. osync has been designed to not delete any data, but rather make backups of conflictual files or soft deletes.
Nevertheless, you should always have a neat backup of your data before trying a new sync tool. Nevertheless, you should always have a neat backup of your data before trying a new sync tool.
You may get osync on github (stable or latest dev snapshot) or on the author's site (stable version)
Getting osync via author's site on <https://www.netpower.fr/osync>
$ wget http://www.netpower.fr/projects/osync/osync.v1.2.tar.gz
$ tar xvf osync.v1.2.tar.gz
Getting osync via github (remove the -b "stable" if you want latest dev snapshot) Getting osync via github (remove the -b "stable" if you want latest dev snapshot)
$ git clone -b "stable" https://github.com/deajan/osync $ git clone -b "stable" https://github.com/deajan/osync
@ -74,11 +76,11 @@ Archlinux packages are available at <https://aur.archlinux.org/packages/osync/>
Since osync v1.1 the config file format has changed in semantics and adds new config options. Since osync v1.1 the config file format has changed in semantics and adds new config options.
Also, master is now called initiator and slave is now called target. Also, master is now called initiator and slave is now called target.
osync v1.2 also added multiple new configuration options. osync v1.3 also added multiple new configuration options.
You can upgrade all v1.0x-v1.2-dev config files by running the upgrade script You can upgrade all v1.0x-v1.3-dev config files by running the upgrade script
$ ./upgrade-v1.0x-v1.2x.sh /etc/osync/your-config-file.conf $ ./upgrade-v1.0x-v1.3x.sh /etc/osync/your-config-file.conf
The script will backup your config file, update it's content and try to connect to initiator and target replicas to update the state dir. The script will backup your config file, update it's content and try to connect to initiator and target replicas to update the state dir.

19
dev/common_install.sh
Unescape Escape View File

@ -10,7 +10,7 @@ PROGRAM_BINARY=$PROGRAM".sh"
PROGRAM_BATCH=$PROGRAM"-batch.sh" PROGRAM_BATCH=$PROGRAM"-batch.sh"
SSH_FILTER="ssh_filter.sh" SSH_FILTER="ssh_filter.sh"
SCRIPT_BUILD=2020112901 SCRIPT_BUILD=2023061101
INSTANCE_ID="installer-$SCRIPT_BUILD" INSTANCE_ID="installer-$SCRIPT_BUILD"
## osync / obackup / pmocr / zsnap install script ## osync / obackup / pmocr / zsnap install script
@ -80,15 +80,26 @@ function SetLocalOSSettings {
} }
function GetInit { function GetInit {
init="none"
if [ -f /sbin/openrc-run ]; then if [ -f /sbin/openrc-run ]; then
init="openrc" init="openrc"
Logger "Detected openrc." "NOTICE" Logger "Detected openrc." "NOTICE"
elif [ -f /usr/lib/systemd/systemd ]; then
init="systemd"
Logger "Detected systemd." "NOTICE"
elif [ -f /sbin/init ]; then elif [ -f /sbin/init ]; then
if file /sbin/init | grep systemd > /dev/null; then if type -p file > /dev/null 2>&1; then
init="systemd" if file /sbin/init | grep systemd > /dev/null; then
Logger "Detected systemd." "NOTICE" init="systemd"
Logger "Detected systemd." "NOTICE"
else
init="initV"
fi
else else
init="initV" init="initV"
fi
if [ $init == "initV" ]; then
Logger "Detected initV." "NOTICE" Logger "Detected initV." "NOTICE"
fi fi
else else

89
dev/debug_osync.sh
Unescape Escape View File

@ -4,10 +4,10 @@
#Check dryruns with nosuffix mode for timestampList #Check dryruns with nosuffix mode for timestampList
PROGRAM="osync" # Rsync based two way sync engine with fault tolerance PROGRAM="osync" # Rsync based two way sync engine with fault tolerance
AUTHOR="(C) 2013-2022 by Orsiris de Jong" AUTHOR="(C) 2013-2023 by Orsiris de Jong"
CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr" CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr"
PROGRAM_VERSION=1.3.0-rc3 PROGRAM_VERSION=1.3.0
PROGRAM_BUILD=2021062901 PROGRAM_BUILD=2023061401
IS_STABLE=true IS_STABLE=true
CONFIG_FILE_REVISION_REQUIRED=1.3.0 CONFIG_FILE_REVISION_REQUIRED=1.3.0
@ -42,8 +42,8 @@ CONFIG_FILE_REVISION_REQUIRED=1.3.0
# UnlockReplicas yes #__WITH_PARANOIA_DEBUG # UnlockReplicas yes #__WITH_PARANOIA_DEBUG
# CleanUp no #__WITH_PARANOIA_DEBUG # CleanUp no #__WITH_PARANOIA_DEBUG
_OFUNCTIONS_VERSION=2.4.3 _OFUNCTIONS_VERSION=2.5.1
_OFUNCTIONS_BUILD=2022050801 _OFUNCTIONS_BUILD=2023061401
_OFUNCTIONS_BOOTSTRAP=true _OFUNCTIONS_BOOTSTRAP=true
if ! type "$BASH" > /dev/null; then if ! type "$BASH" > /dev/null; then
@ -66,6 +66,8 @@ _LOGGER_SILENT=false
_LOGGER_VERBOSE=false _LOGGER_VERBOSE=false
_LOGGER_ERR_ONLY=false _LOGGER_ERR_ONLY=false
_LOGGER_PREFIX="date" _LOGGER_PREFIX="date"
_LOGGER_WRITE_PARTIAL_LOGS=false # Writes partial log files to /tmp so sending logs via alerts can feed on them
_OFUNCTIONS_SHOW_SPINNER=true # Show spinner in ExecTasks function
if [ "$KEEP_LOGGING" == "" ]; then if [ "$KEEP_LOGGING" == "" ]; then
KEEP_LOGGING=1801 KEEP_LOGGING=1801
fi fi
@ -174,7 +176,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -290,18 +292,18 @@ function Logger {
if [ "$level" == "CRITICAL" ]; then if [ "$level" == "CRITICAL" ]; then
_Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
# ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. Need to keep this flag. # ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. We need to create these flag files for ERROR_ALERT / WARN_ALERT to be picked up by Alert
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "ERROR" ]; then elif [ "$level" == "ERROR" ]; then
_Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "WARN" ]; then elif [ "$level" == "WARN" ]; then
_Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true
WARN_ALERT=true WARN_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.warn.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "NOTICE" ]; then elif [ "$level" == "NOTICE" ]; then
if [ "$_LOGGER_ERR_ONLY" != true ]; then if [ "$_LOGGER_ERR_ONLY" != true ]; then
@ -423,11 +425,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi
@ -487,7 +489,11 @@ function SendAlert {
fi fi
fi fi
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")" if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ]; then
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")"
else
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$LOG_FILE")"
fi
if [ $ERROR_ALERT == true ]; then if [ $ERROR_ALERT == true ]; then
subject="Error alert for $INSTANCE_ID" subject="Error alert for $INSTANCE_ID"
@ -498,7 +504,7 @@ function SendAlert {
fi fi
if [ $runAlert == true ]; then if [ $runAlert == true ]; then
subject="Currently runing - $subject" subject="Currently running - $subject"
else else
subject="Finished run - $subject" subject="Finished run - $subject"
fi fi
@ -980,7 +986,7 @@ function ExecTasks {
# soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop # soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop
function _ExecTasksTimeCheck { function _ExecTasksTimeCheck {
if [ $spinner == true ]; then if [ $spinner == true ] && [ "$_OFUNCTIONS_SHOW_SPINNER" != false ]; then
Spinner Spinner
fi fi
if [ $counting == true ]; then if [ $counting == true ]; then
@ -2248,6 +2254,16 @@ function InitRemoteOSDependingSettings {
## Set rsync default arguments (complete with -r or -d depending on recursivity later) ## Set rsync default arguments (complete with -r or -d depending on recursivity later)
RSYNC_DEFAULT_ARGS="-ltD -8" RSYNC_DEFAULT_ARGS="-ltD -8"
## NPF-MOD: Regarding #242, we need to add --old-args if rsync > 3.2.3
#rsync_version=$("${RSYNC_EXECUTABLE}" --version 2>/dev/null| head -1 | awk '{print $3}')
#if [ $(VerComp $rsync_version 3.2.3) -eq 1 ]; then
# RSYNC_DEFAULT_ARGS="$RSYNC_DEFAULT_ARGS --old-args"
#fi
# NPF-MOD: Strangely enough, also happens on RHEL7 rsync 3.1.1
# Let's resolve this easier
export RSYNC_OLD_ARGS=1
if [ "$_DRYRUN" == true ]; then if [ "$_DRYRUN" == true ]; then
RSYNC_DRY_ARG="-n" RSYNC_DRY_ARG="-n"
DRY_WARNING="/!\ DRY RUN " DRY_WARNING="/!\ DRY RUN "
@ -2255,6 +2271,7 @@ function InitRemoteOSDependingSettings {
RSYNC_DRY_ARG="" RSYNC_DRY_ARG=""
fi fi
RSYNC_ATTR_ARGS="" RSYNC_ATTR_ARGS=""
if [ "$PRESERVE_PERMISSIONS" != false ]; then if [ "$PRESERVE_PERMISSIONS" != false ]; then
RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p" RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p"
@ -2477,16 +2494,8 @@ function FileMove () {
mv -f "$source" "$dest" mv -f "$source" "$dest"
return $? return $?
elif [ -w "$source" ]; then elif [ -w "$source" ]; then
if [ -f "$dest" ]; then # for files we don't need recursive delete [ -f "$dest" ] && rm -f "$dest"
rm -f "$dest" cp -p "$source" "$dest" && rm -f "$source"
elif [ -d "$dest" ]; then # for directories we need recursive delete
rm -rf "$dest"
fi
if [ -f "$source" ]; then
cp -p "$source" "$dest" && rm -f "$source" # for files we don't need recursive copy & delete
elif [ -d "$source" ]; then
cp -rp "$source" "$dest" && rm -rf "$source" # for directories we need recursive copy & delete
fi
return $? return $?
else else
return -1 return -1
@ -2641,6 +2650,7 @@ _OFUNCTIONS_BOOTSTRAP=true
[ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1 [ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1
_LOGGER_PREFIX="time" _LOGGER_PREFIX="time"
_LOGGER_WRITE_PARTIAL_LOGS=true
## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc ## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc
OSYNC_DIR=".osync_workdir" OSYNC_DIR=".osync_workdir"
@ -3056,7 +3066,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -3485,7 +3495,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -4118,7 +4128,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -4768,7 +4778,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -4873,16 +4883,8 @@ function FileMove () {
mv -f "$source" "$dest" mv -f "$source" "$dest"
return $? return $?
elif [ -w "$source" ]; then elif [ -w "$source" ]; then
if [ -f "$dest" ]; then # for files we don't need recursive delete [ -f "$dest" ] && rm -f "$dest"
rm -f "$dest" cp -p "$source" "$dest" && rm -f "$source"
elif [ -d "$dest" ]; then # for directories we need recursive delete
rm -rf "$dest"
fi
if [ -f "$source" ]; then
cp -p "$source" "$dest" && rm -f "$source" # for files we don't need recursive copy & delete
elif [ -d "$source" ]; then
cp -rp "$source" "$dest" && rm -rf "$source" # for directories we need recursive copy & delete
fi
return $? return $?
else else
return -1 return -1
@ -5783,7 +5785,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -6071,7 +6073,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -6596,6 +6598,7 @@ function Usage {
echo "--no-prefix Will suppress time / date suffix from output" echo "--no-prefix Will suppress time / date suffix from output"
echo "--silent Will run osync without any output to stdout, used for cron jobs" echo "--silent Will run osync without any output to stdout, used for cron jobs"
echo "--errors-only Output only errors (can be combined with silent or verbose)" echo "--errors-only Output only errors (can be combined with silent or verbose)"
echo "--non-interactive Don't show running animation in cron / service mode"
echo "--summary Outputs a list of transferred / deleted files at the end of the run" echo "--summary Outputs a list of transferred / deleted files at the end of the run"
echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files" echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files"
echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)" echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)"
@ -6872,6 +6875,10 @@ function GetCommandlineArguments {
opts=$opts" --errors-only" opts=$opts" --errors-only"
_LOGGER_ERR_ONLY=true _LOGGER_ERR_ONLY=true
;; ;;
--non-interactive)
opts=$opts" --non-interactive"
_OFUNCTIONS_SHOW_SPINNER=false
;;
--summary) --summary)
opts=$opts" --summary" opts=$opts" --summary"
_SUMMARY=true _SUMMARY=true

12
dev/n_osync.sh
Unescape Escape View File

@ -4,10 +4,10 @@
#Check dryruns with nosuffix mode for timestampList #Check dryruns with nosuffix mode for timestampList
PROGRAM="osync" # Rsync based two way sync engine with fault tolerance PROGRAM="osync" # Rsync based two way sync engine with fault tolerance
AUTHOR="(C) 2013-2022 by Orsiris de Jong" AUTHOR="(C) 2013-2023 by Orsiris de Jong"
CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr" CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr"
PROGRAM_VERSION=1.3.0-rc3 PROGRAM_VERSION=1.3.0
PROGRAM_BUILD=2021062901 PROGRAM_BUILD=2023061401
IS_STABLE=true IS_STABLE=true
CONFIG_FILE_REVISION_REQUIRED=1.3.0 CONFIG_FILE_REVISION_REQUIRED=1.3.0
@ -48,6 +48,7 @@ include #### _OFUNCTIONS_BOOTSTRAP SUBSET ####
[ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1 [ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1
_LOGGER_PREFIX="time" _LOGGER_PREFIX="time"
_LOGGER_WRITE_PARTIAL_LOGS=true
## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc ## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc
OSYNC_DIR=".osync_workdir" OSYNC_DIR=".osync_workdir"
@ -2898,6 +2899,7 @@ function Usage {
echo "--no-prefix Will suppress time / date suffix from output" echo "--no-prefix Will suppress time / date suffix from output"
echo "--silent Will run osync without any output to stdout, used for cron jobs" echo "--silent Will run osync without any output to stdout, used for cron jobs"
echo "--errors-only Output only errors (can be combined with silent or verbose)" echo "--errors-only Output only errors (can be combined with silent or verbose)"
echo "--non-interactive Don't show running animation in cron / service mode"
echo "--summary Outputs a list of transferred / deleted files at the end of the run" echo "--summary Outputs a list of transferred / deleted files at the end of the run"
echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files" echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files"
echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)" echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)"
@ -3174,6 +3176,10 @@ function GetCommandlineArguments {
opts=$opts" --errors-only" opts=$opts" --errors-only"
_LOGGER_ERR_ONLY=true _LOGGER_ERR_ONLY=true
;; ;;
--non-interactive)
opts=$opts" --non-interactive"
_OFUNCTIONS_SHOW_SPINNER=false
;;
--summary) --summary)
opts=$opts" --summary" opts=$opts" --summary"
_SUMMARY=true _SUMMARY=true

54
dev/ofunctions.sh
Unescape Escape View File

@ -14,6 +14,7 @@
## _LOGGER_VERBOSE=true/false ## _LOGGER_VERBOSE=true/false
## _LOGGER_ERR_ONLY=true/false ## _LOGGER_ERR_ONLY=true/false
## _LOGGER_PREFIX="date"/"time"/"" ## _LOGGER_PREFIX="date"/"time"/""
## _LOGGER_WRITE_PARTIAL_LOGS=true/false
## Also, set the following trap in order to clean temporary files ## Also, set the following trap in order to clean temporary files
## trap GenericTrapQuit TERM EXIT HUP QUIT ## trap GenericTrapQuit TERM EXIT HUP QUIT
@ -30,8 +31,8 @@
#### OFUNCTIONS FULL SUBSET #### #### OFUNCTIONS FULL SUBSET ####
#### OFUNCTIONS MINI SUBSET #### #### OFUNCTIONS MINI SUBSET ####
#### OFUNCTIONS MICRO SUBSET #### #### OFUNCTIONS MICRO SUBSET ####
_OFUNCTIONS_VERSION=2.4.3 _OFUNCTIONS_VERSION=2.5.1
_OFUNCTIONS_BUILD=2022050801 _OFUNCTIONS_BUILD=2023061401
#### _OFUNCTIONS_BOOTSTRAP SUBSET #### #### _OFUNCTIONS_BOOTSTRAP SUBSET ####
_OFUNCTIONS_BOOTSTRAP=true _OFUNCTIONS_BOOTSTRAP=true
#### _OFUNCTIONS_BOOTSTRAP SUBSET END #### #### _OFUNCTIONS_BOOTSTRAP SUBSET END ####
@ -56,6 +57,8 @@ _LOGGER_SILENT=false
_LOGGER_VERBOSE=false _LOGGER_VERBOSE=false
_LOGGER_ERR_ONLY=false _LOGGER_ERR_ONLY=false
_LOGGER_PREFIX="date" _LOGGER_PREFIX="date"
_LOGGER_WRITE_PARTIAL_LOGS=false # Writes partial log files to /tmp so sending logs via alerts can feed on them
_OFUNCTIONS_SHOW_SPINNER=true # Show spinner in ExecTasks function
if [ "$KEEP_LOGGING" == "" ]; then if [ "$KEEP_LOGGING" == "" ]; then
KEEP_LOGGING=1801 KEEP_LOGGING=1801
fi fi
@ -172,7 +175,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -289,18 +292,18 @@ function Logger {
if [ "$level" == "CRITICAL" ]; then if [ "$level" == "CRITICAL" ]; then
_Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
# ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. Need to keep this flag. # ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. We need to create these flag files for ERROR_ALERT / WARN_ALERT to be picked up by Alert
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "ERROR" ]; then elif [ "$level" == "ERROR" ]; then
_Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "WARN" ]; then elif [ "$level" == "WARN" ]; then
_Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true
WARN_ALERT=true WARN_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.warn.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "NOTICE" ]; then elif [ "$level" == "NOTICE" ]; then
if [ "$_LOGGER_ERR_ONLY" != true ]; then if [ "$_LOGGER_ERR_ONLY" != true ]; then
@ -426,11 +429,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi
@ -494,7 +497,11 @@ function SendAlert {
fi fi
fi fi
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")" if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ]; then
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")"
else
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$LOG_FILE")"
fi
if [ $ERROR_ALERT == true ]; then if [ $ERROR_ALERT == true ]; then
subject="Error alert for $INSTANCE_ID" subject="Error alert for $INSTANCE_ID"
@ -505,7 +512,7 @@ function SendAlert {
fi fi
if [ $runAlert == true ]; then if [ $runAlert == true ]; then
subject="Currently runing - $subject" subject="Currently running - $subject"
else else
subject="Finished run - $subject" subject="Finished run - $subject"
fi fi
@ -989,7 +996,7 @@ function ExecTasks {
# soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop # soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop
function _ExecTasksTimeCheck { function _ExecTasksTimeCheck {
if [ $spinner == true ]; then if [ $spinner == true ] && [ "$_OFUNCTIONS_SHOW_SPINNER" != false ]; then
Spinner Spinner
fi fi
if [ $counting == true ]; then if [ $counting == true ]; then
@ -2268,6 +2275,16 @@ function InitRemoteOSDependingSettings {
## Set rsync default arguments (complete with -r or -d depending on recursivity later) ## Set rsync default arguments (complete with -r or -d depending on recursivity later)
RSYNC_DEFAULT_ARGS="-ltD -8" RSYNC_DEFAULT_ARGS="-ltD -8"
## NPF-MOD: Regarding #242, we need to add --old-args if rsync > 3.2.3
#rsync_version=$("${RSYNC_EXECUTABLE}" --version 2>/dev/null| head -1 | awk '{print $3}')
#if [ $(VerComp $rsync_version 3.2.3) -eq 1 ]; then
# RSYNC_DEFAULT_ARGS="$RSYNC_DEFAULT_ARGS --old-args"
#fi
# NPF-MOD: Strangely enough, also happens on RHEL7 rsync 3.1.1
# Let's resolve this easier
export RSYNC_OLD_ARGS=1
if [ "$_DRYRUN" == true ]; then if [ "$_DRYRUN" == true ]; then
RSYNC_DRY_ARG="-n" RSYNC_DRY_ARG="-n"
DRY_WARNING="/!\ DRY RUN " DRY_WARNING="/!\ DRY RUN "
@ -2275,6 +2292,7 @@ function InitRemoteOSDependingSettings {
RSYNC_DRY_ARG="" RSYNC_DRY_ARG=""
fi fi
RSYNC_ATTR_ARGS="" RSYNC_ATTR_ARGS=""
if [ "$PRESERVE_PERMISSIONS" != false ]; then if [ "$PRESERVE_PERMISSIONS" != false ]; then
RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p" RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p"
@ -2504,16 +2522,8 @@ function FileMove () {
mv -f "$source" "$dest" mv -f "$source" "$dest"
return $? return $?
elif [ -w "$source" ]; then elif [ -w "$source" ]; then
if [ -f "$dest" ]; then # for files we don't need recursive delete [ -f "$dest" ] && rm -f "$dest"
rm -f "$dest" cp -p "$source" "$dest" && rm -f "$source"
elif [ -d "$dest" ]; then # for directories we need recursive delete
rm -rf "$dest"
fi
if [ -f "$source" ]; then
cp -p "$source" "$dest" && rm -f "$source" # for files we don't need recursive copy & delete
elif [ -d "$source" ]; then
cp -rp "$source" "$dest" && rm -rf "$source" # for directories we need recursive copy & delete
fi
return $? return $?
else else
return -1 return -1

165
dev/tests/run_tests.sh
Unescape Escape View File

@ -1,6 +1,6 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# osync test suite 2022070702 # osync test suite 2023061401
# Allows the following environment variables # Allows the following environment variables
@ -50,20 +50,23 @@
if [ "$SKIP_REMOTE" = "" ]; then if [ "$SKIP_REMOTE" = "" ]; then
SKIP_REMOTE=false SKIP_REMOTE=false
REMOTE_USER=root
fi fi
homedir=$(eval echo ~${REMOTE_USER})
# drupal servers are often unreachable for whetever reason or give 0 bytes files # drupal servers are often unreachable for whetever reason or give 0 bytes files
#LARGE_FILESET_URL="http://ftp.drupal.org/files/projects/drupal-8.2.2.tar.gz" #LARGE_FILESET_URL="http://ftp.drupal.org/files/projects/drupal-8.2.2.tar.gz"
LARGE_FILESET_URL="http://www.netpower.fr/sites/default/files/osync-test-files.tar.gz" LARGE_FILESET_URL="http://www.netpower.fr/sites/default/files/osync-test-files.tar.gz"
# Fakeroot for install / uninstall and test of executables
FAKEROOT="${HOME}/osync_test_install"
OSYNC_DIR="$(pwd)" OSYNC_DIR="$(pwd)"
OSYNC_DIR=${OSYNC_DIR%%/dev*} OSYNC_DIR=${OSYNC_DIR%%/dev*}
DEV_DIR="$OSYNC_DIR/dev" DEV_DIR="$OSYNC_DIR/dev"
TESTS_DIR="$DEV_DIR/tests" TESTS_DIR="$DEV_DIR/tests"
# Fakeroot for install / uninstall and test of executables
FAKEROOT="${homedir}/osync_test_install"
CONF_DIR="$TESTS_DIR/conf" CONF_DIR="$TESTS_DIR/conf"
LOCAL_CONF="local.conf" LOCAL_CONF="local.conf"
REMOTE_CONF="remote.conf" REMOTE_CONF="remote.conf"
@ -76,7 +79,7 @@ OSYNC_UPGRADE="upgrade-v1.0x-v1.3x.sh"
TMP_FILE="$DEV_DIR/tmp" TMP_FILE="$DEV_DIR/tmp"
OSYNC_TESTS_DIR="${HOME}/osync-tests" OSYNC_TESTS_DIR="${homedir}/osync-tests"
INITIATOR_DIR="$OSYNC_TESTS_DIR/initiator" INITIATOR_DIR="$OSYNC_TESTS_DIR/initiator"
TARGET_DIR="$OSYNC_TESTS_DIR/target" TARGET_DIR="$OSYNC_TESTS_DIR/target"
OSYNC_WORKDIR=".osync_workdir" OSYNC_WORKDIR=".osync_workdir"
@ -93,46 +96,52 @@ PRIVKEY_NAME="id_rsa_local_osync_tests"
PUBKEY_NAME="${PRIVKEY_NAME}.pub" PUBKEY_NAME="${PRIVKEY_NAME}.pub"
function SetupSSH { function SetupSSH {
echo "Setting up an ssh key to ${HOME}/.ssh/${PRIVKEY_NAME}" echo "Setting up an ssh key to ${homedir}/.ssh/${PRIVKEY_NAME}"
echo -e 'y\n'| ssh-keygen -t rsa -b 2048 -N "" -f "${HOME}/.ssh/${PRIVKEY_NAME}" echo -e 'y\n'| ssh-keygen -t rsa -b 2048 -N "" -f "${homedir}/.ssh/${PRIVKEY_NAME}"
SSH_AUTH_LINE="from=\"*\",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=\"$FAKEROOT/usr/local/bin/ssh_filter.sh SomeAlphaNumericToken9\" $(cat ${HOME}/.ssh/${PUBKEY_NAME})"
echo "ls -alh ${HOME}"
ls -alh "${HOME}" SSH_AUTH_LINE="from=\"*\",no-port-forwarding,no-X11-forwarding,no-agent-forwarding,no-pty,command=\"$FAKEROOT/usr/local/bin/ssh_filter.sh SomeAlphaNumericToken9\" $(cat ${homedir}/.ssh/${PUBKEY_NAME})"
echo "ls -alh ${HOME}/.ssh" echo "ls -alh ${homedir}"
ls -alh "${HOME}/.ssh" ls -alh "${homedir}"
echo "ls -alh ${homedir}/.ssh"
if [ -f "${HOME}/.ssh/authorized_keys" ]; then ls -alh "${homedir}/.ssh"
if ! grep "$(cat ${HOME}/.ssh/${PUBKEY_NAME})" "${HOME}/.ssh/authorized_keys"; then
echo "$SSH_AUTH_LINE" >> "${HOME}/.ssh/authorized_keys" if [ -f "${homedir}/.ssh/authorized_keys" ]; then
if ! grep "$(cat ${homedir}/.ssh/${PUBKEY_NAME})" "${homedir}/.ssh/authorized_keys"; then
echo "Adding auth line in authorized_keys file ${homedir}/.ssh/authorized_keys"
echo "$SSH_AUTH_LINE" >> "${homedir}/.ssh/authorized_keys"
fi fi
else else
echo "$SSH_AUTH_LINE" >> "${HOME}/.ssh/authorized_keys" echo "Creating authorized_keys file ${homedir}/.ssh/authorized_keys"
echo "$SSH_AUTH_LINE" >> "${homedir}/.ssh/authorized_keys"
fi fi
chmod 600 "${HOME}/.ssh/authorized_keys" chmod 600 "${homedir}/.ssh/authorized_keys"
# Add localhost to known hosts so self connect works # Add localhost to known hosts so self connect works
if [ -z "$(ssh-keygen -F localhost)" ]; then if [ -z "$(ssh-keygen -F localhost)" ]; then
ssh-keyscan -H localhost >> "${HOME}/.ssh/known_hosts" ssh-keyscan -H localhost >> "${homedir}/.ssh/known_hosts"
fi fi
# Update remote conf files with SSH port # Update remote conf files with SSH port and file id location
sed -i.tmp 's#ssh://.*@localhost:[0-9]*/${HOME}/osync-tests/target#ssh://'$REMOTE_USER'@localhost:'$SSH_PORT'/${HOME}/osync-tests/target#' "$CONF_DIR/$REMOTE_CONF" sed -i.tmp 's#ssh://.*@localhost:[0-9]*/${HOME}/osync-tests/target#ssh://'$REMOTE_USER'@localhost:'$SSH_PORT'/'${homedir}'/osync-tests/target#' "$CONF_DIR/$REMOTE_CONF"
sed -i.tmp2 's#SSH_RSA_PRIVATE_KEY="${HOME}/.ssh/id_rsa_local_osync_tests"#SSH_RSA_PRIVATE_KEY="'${homedir}'/.ssh/id_rsa_local_osync_tests"#' "$CONF_DIR/$REMOTE_CONF"
echp "ls -alh ${HOME}/.ssh" echo "ls -alh ${homedir}/.ssh"
ls -alh "${HOME}/.ssh" ls -alh "${homedir}/.ssh"
echo "cat ${HOME}/.ssh.authorized_keys" echo "cat ${homedir}/.ssh/authorized_keys"
cat "${HOME}/.ssh/authorized_keys" cat "${homedir}/.ssh/authorized_keys"
echo "###" echo "###"
echo "END SETUP SSH" echo "END SETUP SSH"
} }
function RemoveSSH { function RemoveSSH {
if [ -f "${HOME}/.ssh/id_rsa_local_osync_tests" ]; then echo "Now removing SSH keys"
if [ -f "${homedir}/.ssh/id_rsa_local_osync_tests" ]; then
echo "Restoring SSH authorized_keys file" echo "Restoring SSH authorized_keys file"
sed -i.bak "s|.*$(cat "${HOME}/.ssh/id_rsa_local_osync_tests.pub")||g" "${HOME}/.ssh/authorized_keys" sed -i.bak "s|.*$(cat "${homedir}/.ssh/id_rsa_local_osync_tests.pub")||g" "${homedir}/.ssh/authorized_keys"
rm -f "${HOME}/.ssh/{id_rsa_local_osync_tests.pub,id_rsa_local_osync_tests}" rm -f "${homedir}/.ssh/{id_rsa_local_osync_tests.pub,id_rsa_local_osync_tests}"
fi fi
} }
@ -192,8 +201,16 @@ function PrepareLocalDirs () {
function oneTimeSetUp () { function oneTimeSetUp () {
START_TIME=$SECONDS START_TIME=$SECONDS
mkdir -p "$FAKEROOT" #echo "Running forced merge"
#cd "${DEV_DIR}"
#$SUDO_CMD ./merge.sh osync
echo "Setting security for files"
$SUDO_CMD find ${OSYNC_DIR} -exec chmod 755 {} \+
echo "Show content of osync dir"
ls -alh ${OSYNC_DIR}
echo "Running install.sh from ${OSYNC_DIR}"
$SUDO_CMD ${OSYNC_DIR}/install.sh --no-stats --prefix="${FAKEROOT}"
source "$DEV_DIR/ofunctions.sh" source "$DEV_DIR/ofunctions.sh"
# Fix default umask because of ACL test that expects 0022 when creating test files # Fix default umask because of ACL test that expects 0022 when creating test files
@ -206,16 +223,20 @@ function oneTimeSetUp () {
# Set some travis related changes # Set some travis related changes
if [ "$RUNNING_ON_GITHUB_ACTIONS" == true ]; then if [ "$RUNNING_ON_GITHUB_ACTIONS" == true ]; then
echo "Running with GITHUB ACTIONS settings" echo "Running with GITHUB ACTIONS settings"
REMOTE_USER="runner" #REMOTE_USER="runner"
REMOTE_USER="root" # WIP
homedir=$(eval echo ~${REMOTE_USER})
RHOST_PING=false RHOST_PING=false
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_3RD_PARTY_HOSTS" "" SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_3RD_PARTY_HOSTS" ""
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_HOST_PING" false SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_HOST_PING" false
SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_3RD_PARTY_HOSTS" "" SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_3RD_PARTY_HOSTS" ""
SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_HOST_PING" false SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_HOST_PING" false
else else
echo "Running with local settings" echo "Running with local settings"
REMOTE_USER="root" REMOTE_USER="root"
homedir=$(eval echo ~${REMOTE_USER})
RHOST_PING=true RHOST_PING=true
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_3RD_PARTY_HOSTS" "\"www.kernel.org www.google.com\"" SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_3RD_PARTY_HOSTS" "\"www.kernel.org www.google.com\""
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_HOST_PING" true SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "REMOTE_HOST_PING" true
@ -224,9 +245,21 @@ function oneTimeSetUp () {
SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_HOST_PING" true SetConfFileValue "$CONF_DIR/$OLD_CONF" "REMOTE_HOST_PING" true
fi fi
# Fix test directories for Github actions
SetConfFileValue "$CONF_DIR/$LOCAL_CONF" INITIATOR_SYNC_DIR "\"${homedir}/osync-tests/initiator\""
SetConfFileValue "$CONF_DIR/$LOCAL_CONF" TARGET_SYNC_DIR "\"${homedir}/osync-tests/target\""
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" INITIATOR_SYNC_DIR "\"${homedir}/osync-tests/initiator\""
SetConfFileValue "$CONF_DIR/$OLD_CONF" MASTER_SYNC_DIR "\"${homedir}/osync-tests/initiator\""
SetConfFileValue "$CONF_DIR/$OLD_CONF" SLAVE_SYNC_DIR "\"${homedir}/osync-tests/target\""
# Get default ssh port from env # Get default ssh port from env
if [ "$SSH_PORT" == "" ]; then if [ "$SSH_PORT" == "" ]; then
SSH_PORT=22 SSH_PORT=22
echo "Running with SSH_PORT=${SSH_PORT}"
fi fi
# Setup modes per test # Setup modes per test
@ -236,8 +269,8 @@ function oneTimeSetUp () {
readonly __confRemote=3 readonly __confRemote=3
osyncParameters=() osyncParameters=()
osyncParameters[$__quickLocal]="--initiator=$INITIATOR_DIR --target=$TARGET_DIR --instance-id=quicklocal" osyncParameters[$__quickLocal]="--initiator=$INITIATOR_DIR --target=$TARGET_DIR --instance-id=quicklocal --non-interactive"
osyncParameters[$__confLocal]="$CONF_DIR/$LOCAL_CONF" osyncParameters[$__confLocal]="$CONF_DIR/$LOCAL_CONF --non-interactive"
osyncDaemonParameters=() osyncDaemonParameters=()
@ -248,8 +281,8 @@ function oneTimeSetUp () {
# Do not check remote config on msys or cygwin since we don't have a local SSH server # Do not check remote config on msys or cygwin since we don't have a local SSH server
if [ "$LOCAL_OS" != "msys" ] && [ "$LOCAL_OS" != "Cygwin" ] && [ $SKIP_REMOTE != true ]; then if [ "$LOCAL_OS" != "msys" ] && [ "$LOCAL_OS" != "Cygwin" ] && [ $SKIP_REMOTE != true ]; then
osyncParameters[$__quickRemote]="--initiator=$INITIATOR_DIR --target=ssh://localhost:$SSH_PORT/$TARGET_DIR --rsakey=${HOME}/.ssh/id_rsa_local_osync_tests --instance-id=quickremote --remote-token=SomeAlphaNumericToken9" osyncParameters[$__quickRemote]="--initiator=$INITIATOR_DIR --target=ssh://localhost:$SSH_PORT/$TARGET_DIR --rsakey=${homedir}/.ssh/id_rsa_local_osync_tests --instance-id=quickremote --remote-token=SomeAlphaNumericToken9 --non-interactive"
osyncParameters[$__confRemote]="$CONF_DIR/$REMOTE_CONF" osyncParameters[$__confRemote]="$CONF_DIR/$REMOTE_CONF --non-interactive"
osyncDaemonParameters[$__remote]="$CONF_DIR/$REMOTE_CONF --on-changes" osyncDaemonParameters[$__remote]="$CONF_DIR/$REMOTE_CONF --on-changes"
@ -331,20 +364,28 @@ function test_SSH {
failure=false failure=false
# Testing as "remote user"
echo "ls -alh ${homedir}/.ssh"
ls -alh "${homedir}/.ssh"
echo "Running SSH test as ${REMOTE_USER}" echo "Running SSH test as ${REMOTE_USER}"
# SSH_PORT and SSH_USER are set by oneTimeSetup # SSH_PORT and SSH_USER are set by oneTimeSetup
ssh -i "${REMOTE_USER}/.ssh/${PUBKEY_NAME}" -p $SSH_PORT ${REMOTE_USER}@localhost "echo \"Remotely:\"; whoami; echo \"TEST OK\"" $SUDO_CMD ssh -i "${homedir}/.ssh/${PRIVKEY_NAME}" -p $SSH_PORT ${REMOTE_USER}@localhost "env _REMOTE_TOKEN=SomeAlphaNumericToken9 echo \"Remotely:\"; whoami; echo \"TEST OK\""
if [ $? -ne 0 ]; then if [ $? -ne 0 ]; then
echo "SSH test failed" echo "SSH test failed"
failure=true failure=true
fi fi
# Testing as current user
#echo "ls -alh ${homedir}/.ssh"
#ls -alh "${homedir}/.ssh"
echo "Running SSH test as $(whoami)" #echo "Running SSH test as $(whoami)"
ssh -i "$(whoami)/.ssh/${PUBKEY_NAME}" -p $SSH_PORT $(whoami)@localhost "echo \"Remotely:\"; whoami; echo \"TEST OK\"" #$SUDO_CMD ssh -i "${homedir}/.ssh/${PRIVKEY_NAME}" -p $SSH_PORT $(whoami)@localhost "env _REMOTE_TOKEN=SomeAlphaNumericToken9 echo \"Remotely:\"; whoami; echo \"TEST OK\""
if [ $? -ne 0 ]; then #if [ $? -ne 0 ]; then
echo "SSH test failed" # echo "SSH test failed"
failure=true # failure=true
fi #fi
if [ $failure == true ]; then if [ $failure == true ]; then
exit 1 # Try to see if we can abort all tests exit 1 # Try to see if we can abort all tests
@ -382,7 +423,7 @@ function test_Merge () {
assertEquals "Install failed" "0" $? assertEquals "Install failed" "0" $?
} }
function xtest_LargeFileSet () { function test_LargeFileSet () {
for i in "${osyncParameters[@]}"; do for i in "${osyncParameters[@]}"; do
cd "$OSYNC_DIR" cd "$OSYNC_DIR"
@ -400,7 +441,7 @@ function xtest_LargeFileSet () {
done done
} }
function xtest_controlMaster () { function test_controlMaster () {
cd "$OSYNC_DIR" cd "$OSYNC_DIR"
PrepareLocalDirs PrepareLocalDirs
@ -409,7 +450,7 @@ function xtest_controlMaster () {
assertEquals "Running quick remote test with controlmaster enabled." "0" $? assertEquals "Running quick remote test with controlmaster enabled." "0" $?
} }
function xtest_Exclusions () { function test_Exclusions () {
# Will sync except php files # Will sync except php files
# RSYNC_EXCLUDE_PATTERN="*.php" is set at runtime for quicksync and in config files for other runs # RSYNC_EXCLUDE_PATTERN="*.php" is set at runtime for quicksync and in config files for other runs
@ -437,7 +478,7 @@ function xtest_Exclusions () {
done done
} }
function xtest_Deletetion () { function test_Deletetion () {
local iFile1="$INITIATOR_DIR/i fic" local iFile1="$INITIATOR_DIR/i fic"
local iFile2="$INITIATOR_DIR/i foc (something)" local iFile2="$INITIATOR_DIR/i foc (something)"
local tFile1="$TARGET_DIR/t fic" local tFile1="$TARGET_DIR/t fic"
@ -481,7 +522,7 @@ function xtest_Deletetion () {
done done
} }
function xtest_deletion_failure () { function test_deletion_failure () {
if [ "$LOCAL_OS" == "WinNT10" ] || [ "$LOCAL_OS" == "msys" ] || [ "$LOCAL_OS" == "Cygwin" ]; then if [ "$LOCAL_OS" == "WinNT10" ] || [ "$LOCAL_OS" == "msys" ] || [ "$LOCAL_OS" == "Cygwin" ]; then
echo "Skipping deletion failure test as Win10 does not have chattr support." echo "Skipping deletion failure test as Win10 does not have chattr support."
return 0 return 0
@ -548,7 +589,7 @@ function xtest_deletion_failure () {
done done
} }
function xtest_skip_deletion () { function test_skip_deletion () {
local modes local modes
if [ "$OSYNC_MIN_VERSION" == "1" ]; then if [ "$OSYNC_MIN_VERSION" == "1" ]; then
@ -624,7 +665,7 @@ function xtest_skip_deletion () {
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "SKIP_DELETION" "" SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "SKIP_DELETION" ""
} }
function xtest_handle_symlinks () { function test_handle_symlinks () {
if [ "$OSYNC_MIN_VERSION" == "1" ]; then if [ "$OSYNC_MIN_VERSION" == "1" ]; then
echo "Skipping symlink tests as osync v1.1x didn't handle this." echo "Skipping symlink tests as osync v1.1x didn't handle this."
return 0 return 0
@ -805,7 +846,7 @@ function xtest_handle_symlinks () {
done done
} }
function xtest_softdeletion_cleanup () { function test_softdeletion_cleanup () {
#declare -A files #declare -A files
files=() files=()
@ -882,7 +923,7 @@ function xtest_softdeletion_cleanup () {
} }
function xtest_FileAttributePropagation () { function test_FileAttributePropagation () {
if [ "$RUNNING_ON_GITHUB_ACTIONS" == true ]; then if [ "$RUNNING_ON_GITHUB_ACTIONS" == true ]; then
echo "Skipping FileAttributePropagation tests as travis does not support getfacl / setfacl." echo "Skipping FileAttributePropagation tests as travis does not support getfacl / setfacl."
@ -970,7 +1011,7 @@ function xtest_FileAttributePropagation () {
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "PRESERVE_XATTR" false SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "PRESERVE_XATTR" false
} }
function xtest_ConflictBackups () { function test_ConflictBackups () {
for i in "${osyncParameters[@]}"; do for i in "${osyncParameters[@]}"; do
cd "$OSYNC_DIR" cd "$OSYNC_DIR"
PrepareLocalDirs PrepareLocalDirs
@ -1006,7 +1047,7 @@ function xtest_ConflictBackups () {
done done
} }
function xtest_MultipleConflictBackups () { function test_MultipleConflictBackups () {
local additionalParameters local additionalParameters
@ -1068,7 +1109,7 @@ function xtest_MultipleConflictBackups () {
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "CONFLICT_BACKUP_MULTIPLE" false SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "CONFLICT_BACKUP_MULTIPLE" false
} }
function xtest_Locking () { function test_Locking () {
# local not running = resume # local not running = resume
# remote same instance_id = resume # remote same instance_id = resume
# remote different instance_id = stop # remote different instance_id = stop
@ -1175,7 +1216,7 @@ function xtest_Locking () {
SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "FORCE_STRANGER_LOCK_RESUME" false SetConfFileValue "$CONF_DIR/$REMOTE_CONF" "FORCE_STRANGER_LOCK_RESUME" false
} }
function xtest_ConflictDetetion () { function test_ConflictDetetion () {
# Tests compatible with v1.4+ # Tests compatible with v1.4+
if [ $OSYNC_MIN_VERSION -lt 4 ]; then if [ $OSYNC_MIN_VERSION -lt 4 ]; then
@ -1226,7 +1267,7 @@ function xtest_ConflictDetetion () {
return 0 return 0
} }
function xtest_WaitForTaskCompletion () { function test_WaitForTaskCompletion () {
local pids local pids
# Tests compatible with v1.1 syntax # Tests compatible with v1.1 syntax
@ -1320,7 +1361,7 @@ function xtest_WaitForTaskCompletion () {
assertEquals "WaitForTaskCompletion test 5" "2" $? assertEquals "WaitForTaskCompletion test 5" "2" $?
} }
function xtest_ParallelExec () { function test_ParallelExec () {
if [ "$OSYNC_MIN_VERSION" == "1" ]; then if [ "$OSYNC_MIN_VERSION" == "1" ]; then
echo "Skipping ParallelExec test because osync v1.1 ofunctions don't have this function." echo "Skipping ParallelExec test because osync v1.1 ofunctions don't have this function."
return 0 return 0
@ -1381,7 +1422,7 @@ function xtest_ParallelExec () {
assertNotEquals "ParallelExec full test 3" "0" $? assertNotEquals "ParallelExec full test 3" "0" $?
} }
function xtest_timedExecution () { function test_timedExecution () {
local arguments local arguments
# Clever usage of indexes and exit codes # Clever usage of indexes and exit codes
@ -1427,7 +1468,7 @@ function xtest_timedExecution () {
done done
} }
function xtest_UpgradeConfRun () { function test_UpgradeConfRun () {
if [ "$OSYNC_MIN_VERSION" == "1" ]; then if [ "$OSYNC_MIN_VERSION" == "1" ]; then
echo "Skipping Upgrade script test because no further dev will happen on this for v1.1" echo "Skipping Upgrade script test because no further dev will happen on this for v1.1"
return 0 return 0
@ -1444,7 +1485,7 @@ function xtest_UpgradeConfRun () {
assertEquals "Conf file upgrade" "0" $? assertEquals "Conf file upgrade" "0" $?
# Update remote conf files with SSH port # Update remote conf files with SSH port
sed -i.tmp 's#ssh://.*@localhost:[0-9]*/${HOME}/osync-tests/target#ssh://'$REMOTE_USER'@localhost:'$SSH_PORT'/${HOME}/osync-tests/target#' "$CONF_DIR/$TMP_OLD_CONF" sed -i.tmp 's#ssh://.*@localhost:[0-9]*/${homedir}/osync-tests/target#ssh://'$REMOTE_USER'@localhost:'$SSH_PORT'/${homedir}/osync-tests/target#' "$CONF_DIR/$TMP_OLD_CONF"
$OSYNC_EXECUTABLE "$CONF_DIR/$TMP_OLD_CONF" $OSYNC_EXECUTABLE "$CONF_DIR/$TMP_OLD_CONF"
assertEquals "Upgraded conf file execution test" "0" $? assertEquals "Upgraded conf file execution test" "0" $?
@ -1453,7 +1494,7 @@ function xtest_UpgradeConfRun () {
rm -f "$CONF_DIR/$TMP_OLD_CONF.save" rm -f "$CONF_DIR/$TMP_OLD_CONF.save"
} }
function xtest_DaemonMode () { function test_DaemonMode () {
if [ "$LOCAL_OS" == "WinNT10" ] || [ "$LOCAL_OS" == "msys" ] || [ "$LOCAL_OS" == "Cygwin" ]; then if [ "$LOCAL_OS" == "WinNT10" ] || [ "$LOCAL_OS" == "msys" ] || [ "$LOCAL_OS" == "Cygwin" ]; then
echo "Skipping daemon mode test as [$LOCAL_OS] does not have inotifywait support." echo "Skipping daemon mode test as [$LOCAL_OS] does not have inotifywait support."
return 0 return 0
@ -1509,7 +1550,7 @@ function xtest_DaemonMode () {
} }
function xtest_NoRemoteAccessTest () { function test_NoRemoteAccessTest () {
RemoveSSH RemoveSSH
cd "$OSYNC_DIR" cd "$OSYNC_DIR"

43
install.sh
Unescape Escape View File

@ -10,15 +10,15 @@ PROGRAM_BINARY=$PROGRAM".sh"
PROGRAM_BATCH=$PROGRAM"-batch.sh" PROGRAM_BATCH=$PROGRAM"-batch.sh"
SSH_FILTER="ssh_filter.sh" SSH_FILTER="ssh_filter.sh"
SCRIPT_BUILD=2020112901 SCRIPT_BUILD=2023061101
INSTANCE_ID="installer-$SCRIPT_BUILD" INSTANCE_ID="installer-$SCRIPT_BUILD"
## osync / obackup / pmocr / zsnap install script ## osync / obackup / pmocr / zsnap install script
## Tested on RHEL / CentOS 6 & 7, Fedora 23, Debian 7 & 8, Mint 17 and FreeBSD 8, 10 and 11 ## Tested on RHEL / CentOS 6 & 7, Fedora 23, Debian 7 & 8, Mint 17 and FreeBSD 8, 10 and 11
## Please adapt this to fit your distro needs ## Please adapt this to fit your distro needs
_OFUNCTIONS_VERSION=2.4.3 _OFUNCTIONS_VERSION=2.5.1
_OFUNCTIONS_BUILD=2022050801 _OFUNCTIONS_BUILD=2023061401
_OFUNCTIONS_BOOTSTRAP=true _OFUNCTIONS_BOOTSTRAP=true
if ! type "$BASH" > /dev/null; then if ! type "$BASH" > /dev/null; then
@ -41,6 +41,8 @@ _LOGGER_SILENT=false
_LOGGER_VERBOSE=false _LOGGER_VERBOSE=false
_LOGGER_ERR_ONLY=false _LOGGER_ERR_ONLY=false
_LOGGER_PREFIX="date" _LOGGER_PREFIX="date"
_LOGGER_WRITE_PARTIAL_LOGS=false # Writes partial log files to /tmp so sending logs via alerts can feed on them
_OFUNCTIONS_SHOW_SPINNER=true # Show spinner in ExecTasks function
if [ "$KEEP_LOGGING" == "" ]; then if [ "$KEEP_LOGGING" == "" ]; then
KEEP_LOGGING=1801 KEEP_LOGGING=1801
fi fi
@ -145,7 +147,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -256,18 +258,18 @@ function Logger {
if [ "$level" == "CRITICAL" ]; then if [ "$level" == "CRITICAL" ]; then
_Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
# ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. Need to keep this flag. # ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. We need to create these flag files for ERROR_ALERT / WARN_ALERT to be picked up by Alert
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "ERROR" ]; then elif [ "$level" == "ERROR" ]; then
_Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "WARN" ]; then elif [ "$level" == "WARN" ]; then
_Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true
WARN_ALERT=true WARN_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.warn.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "NOTICE" ]; then elif [ "$level" == "NOTICE" ]; then
if [ "$_LOGGER_ERR_ONLY" != true ]; then if [ "$_LOGGER_ERR_ONLY" != true ]; then
@ -382,11 +384,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi
@ -601,11 +603,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi
@ -653,15 +655,26 @@ function SetLocalOSSettings {
} }
function GetInit { function GetInit {
init="none"
if [ -f /sbin/openrc-run ]; then if [ -f /sbin/openrc-run ]; then
init="openrc" init="openrc"
Logger "Detected openrc." "NOTICE" Logger "Detected openrc." "NOTICE"
elif [ -f /usr/lib/systemd/systemd ]; then
init="systemd"
Logger "Detected systemd." "NOTICE"
elif [ -f /sbin/init ]; then elif [ -f /sbin/init ]; then
if file /sbin/init | grep systemd > /dev/null; then if type -p file > /dev/null 2>&1; then
init="systemd" if file /sbin/init | grep systemd > /dev/null; then
Logger "Detected systemd." "NOTICE" init="systemd"
Logger "Detected systemd." "NOTICE"
else
init="initV"
fi
else else
init="initV" init="initV"
fi
if [ $init == "initV" ]; then
Logger "Detected initV." "NOTICE" Logger "Detected initV." "NOTICE"
fi fi
else else

14
osync-batch.sh
Unescape Escape View File

@ -54,7 +54,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -166,18 +166,18 @@ function Logger {
if [ "$level" == "CRITICAL" ]; then if [ "$level" == "CRITICAL" ]; then
_Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
# ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. Need to keep this flag. # ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. We need to create these flag files for ERROR_ALERT / WARN_ALERT to be picked up by Alert
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "ERROR" ]; then elif [ "$level" == "ERROR" ]; then
_Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "WARN" ]; then elif [ "$level" == "WARN" ]; then
_Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true
WARN_ALERT=true WARN_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.warn.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "NOTICE" ]; then elif [ "$level" == "NOTICE" ]; then
if [ "$_LOGGER_ERR_ONLY" != true ]; then if [ "$_LOGGER_ERR_ONLY" != true ]; then
@ -224,11 +224,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi

87
osync.sh
Unescape Escape View File

@ -4,17 +4,17 @@
#Check dryruns with nosuffix mode for timestampList #Check dryruns with nosuffix mode for timestampList
PROGRAM="osync" # Rsync based two way sync engine with fault tolerance PROGRAM="osync" # Rsync based two way sync engine with fault tolerance
AUTHOR="(C) 2013-2022 by Orsiris de Jong" AUTHOR="(C) 2013-2023 by Orsiris de Jong"
CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr" CONTACT="http://www.netpower.fr/osync - ozy@netpower.fr"
PROGRAM_VERSION=1.3.0-rc3 PROGRAM_VERSION=1.3.0
PROGRAM_BUILD=2021062901 PROGRAM_BUILD=2023061401
IS_STABLE=true IS_STABLE=true
CONFIG_FILE_REVISION_REQUIRED=1.3.0 CONFIG_FILE_REVISION_REQUIRED=1.3.0
_OFUNCTIONS_VERSION=2.4.3 _OFUNCTIONS_VERSION=2.5.1
_OFUNCTIONS_BUILD=2022050801 _OFUNCTIONS_BUILD=2023061401
_OFUNCTIONS_BOOTSTRAP=true _OFUNCTIONS_BOOTSTRAP=true
if ! type "$BASH" > /dev/null; then if ! type "$BASH" > /dev/null; then
@ -37,6 +37,8 @@ _LOGGER_SILENT=false
_LOGGER_VERBOSE=false _LOGGER_VERBOSE=false
_LOGGER_ERR_ONLY=false _LOGGER_ERR_ONLY=false
_LOGGER_PREFIX="date" _LOGGER_PREFIX="date"
_LOGGER_WRITE_PARTIAL_LOGS=false # Writes partial log files to /tmp so sending logs via alerts can feed on them
_OFUNCTIONS_SHOW_SPINNER=true # Show spinner in ExecTasks function
if [ "$KEEP_LOGGING" == "" ]; then if [ "$KEEP_LOGGING" == "" ]; then
KEEP_LOGGING=1801 KEEP_LOGGING=1801
fi fi
@ -141,7 +143,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -252,18 +254,18 @@ function Logger {
if [ "$level" == "CRITICAL" ]; then if [ "$level" == "CRITICAL" ]; then
_Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[1;33;41m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
# ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. Need to keep this flag. # ERROR_ALERT / WARN_ALERT is not set in main when Logger is called from a subprocess. We need to create these flag files for ERROR_ALERT / WARN_ALERT to be picked up by Alert
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "ERROR" ]; then elif [ "$level" == "ERROR" ]; then
_Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[91m$value\e[0m" true
ERROR_ALERT=true ERROR_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.error.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "WARN" ]; then elif [ "$level" == "WARN" ]; then
_Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true _Logger "$prefix($level):$value" "$prefix\e[33m$value\e[0m" true
WARN_ALERT=true WARN_ALERT=true
echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.${FUNCNAME[0]}.warn.$SCRIPT_PID.$TSTAMP" echo -e "[$retval] in [$(joinString , ${FUNCNAME[@]})] SP=$SCRIPT_PID P=$$\n$prefix($level):$value" >> "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP"
return return
elif [ "$level" == "NOTICE" ]; then elif [ "$level" == "NOTICE" ]; then
if [ "$_LOGGER_ERR_ONLY" != true ]; then if [ "$_LOGGER_ERR_ONLY" != true ]; then
@ -378,11 +380,11 @@ function GenericTrapQuit {
local exitcode=0 local exitcode=0
# Get ERROR / WARN alert flags from subprocesses that call Logger # Get ERROR / WARN alert flags from subprocesses that call Logger
if [ -f "$RUN_DIR/$PROGRAM.Logger.warn.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.WARN_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
WARN_ALERT=true WARN_ALERT=true
exitcode=2 exitcode=2
fi fi
if [ -f "$RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID.$TSTAMP" ]; then if [ -f "$RUN_DIR/$PROGRAM.ERROR_ALERT.$SCRIPT_PID.$TSTAMP" ]; then
ERROR_ALERT=true ERROR_ALERT=true
exitcode=1 exitcode=1
fi fi
@ -441,7 +443,11 @@ function SendAlert {
fi fi
fi fi
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")" if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ]; then
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP")"
else
body="$MAIL_ALERT_MSG"$'\n\n'"Last 1000 lines of current log"$'\n\n'"$(tail -n 1000 "$LOG_FILE")"
fi
if [ $ERROR_ALERT == true ]; then if [ $ERROR_ALERT == true ]; then
subject="Error alert for $INSTANCE_ID" subject="Error alert for $INSTANCE_ID"
@ -912,7 +918,7 @@ function ExecTasks {
# soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop # soft / hard execution time checks that needs to be a subfunction since it is called both from main loop and from parallelExec sub loop
function _ExecTasksTimeCheck { function _ExecTasksTimeCheck {
if [ $spinner == true ]; then if [ $spinner == true ] && [ "$_OFUNCTIONS_SHOW_SPINNER" != false ]; then
Spinner Spinner
fi fi
if [ $counting == true ]; then if [ $counting == true ]; then
@ -2098,6 +2104,16 @@ function InitRemoteOSDependingSettings {
## Set rsync default arguments (complete with -r or -d depending on recursivity later) ## Set rsync default arguments (complete with -r or -d depending on recursivity later)
RSYNC_DEFAULT_ARGS="-ltD -8" RSYNC_DEFAULT_ARGS="-ltD -8"
## NPF-MOD: Regarding #242, we need to add --old-args if rsync > 3.2.3
#rsync_version=$("${RSYNC_EXECUTABLE}" --version 2>/dev/null| head -1 | awk '{print $3}')
#if [ $(VerComp $rsync_version 3.2.3) -eq 1 ]; then
# RSYNC_DEFAULT_ARGS="$RSYNC_DEFAULT_ARGS --old-args"
#fi
# NPF-MOD: Strangely enough, also happens on RHEL7 rsync 3.1.1
# Let's resolve this easier
export RSYNC_OLD_ARGS=1
if [ "$_DRYRUN" == true ]; then if [ "$_DRYRUN" == true ]; then
RSYNC_DRY_ARG="-n" RSYNC_DRY_ARG="-n"
DRY_WARNING="/!\ DRY RUN " DRY_WARNING="/!\ DRY RUN "
@ -2105,6 +2121,7 @@ function InitRemoteOSDependingSettings {
RSYNC_DRY_ARG="" RSYNC_DRY_ARG=""
fi fi
RSYNC_ATTR_ARGS="" RSYNC_ATTR_ARGS=""
if [ "$PRESERVE_PERMISSIONS" != false ]; then if [ "$PRESERVE_PERMISSIONS" != false ]; then
RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p" RSYNC_ATTR_ARGS=$RSYNC_ATTR_ARGS" -p"
@ -2327,16 +2344,8 @@ function FileMove () {
mv -f "$source" "$dest" mv -f "$source" "$dest"
return $? return $?
elif [ -w "$source" ]; then elif [ -w "$source" ]; then
if [ -f "$dest" ]; then # for files we don't need recursive delete [ -f "$dest" ] && rm -f "$dest"
rm -f "$dest" cp -p "$source" "$dest" && rm -f "$source"
elif [ -d "$dest" ]; then # for directories we need recursive delete
rm -rf "$dest"
fi
if [ -f "$source" ]; then
cp -p "$source" "$dest" && rm -f "$source" # for files we don't need recursive copy & delete
elif [ -d "$source" ]; then
cp -rp "$source" "$dest" && rm -rf "$source" # for directories we need recursive copy & delete
fi
return $? return $?
else else
return -1 return -1
@ -2491,6 +2500,7 @@ _OFUNCTIONS_BOOTSTRAP=true
[ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1 [ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0) or build it with merge.sh" && exit 1
_LOGGER_PREFIX="time" _LOGGER_PREFIX="time"
_LOGGER_WRITE_PARTIAL_LOGS=true
## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc ## Working directory. This directory exists in any replica and contains state files, backups, soft deleted files etc
OSYNC_DIR=".osync_workdir" OSYNC_DIR=".osync_workdir"
@ -2897,7 +2907,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -3314,7 +3324,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -3930,7 +3940,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -4565,7 +4575,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -4665,16 +4675,8 @@ function FileMove () {
mv -f "$source" "$dest" mv -f "$source" "$dest"
return $? return $?
elif [ -w "$source" ]; then elif [ -w "$source" ]; then
if [ -f "$dest" ]; then # for files we don't need recursive delete [ -f "$dest" ] && rm -f "$dest"
rm -f "$dest" cp -p "$source" "$dest" && rm -f "$source"
elif [ -d "$dest" ]; then # for directories we need recursive delete
rm -rf "$dest"
fi
if [ -f "$source" ]; then
cp -p "$source" "$dest" && rm -f "$source" # for files we don't need recursive copy & delete
elif [ -d "$source" ]; then
cp -rp "$source" "$dest" && rm -rf "$source" # for directories we need recursive copy & delete
fi
return $? return $?
else else
return -1 return -1
@ -5566,7 +5568,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -5842,7 +5844,7 @@ function _Logger {
echo -e "$logValue" >> "$LOG_FILE" echo -e "$logValue" >> "$LOG_FILE"
# Build current log file for alerts if we have a sufficient environment # Build current log file for alerts if we have a sufficient environment
if [ "$RUN_DIR/$PROGRAM" != "/" ]; then if [ "$_LOGGER_WRITE_PARTIAL_LOGS" == true ] && [ "$RUN_DIR/$PROGRAM" != "/" ]; then
echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP" echo -e "$logValue" >> "$RUN_DIR/$PROGRAM._Logger.$SCRIPT_PID.$TSTAMP"
fi fi
fi fi
@ -6354,6 +6356,7 @@ function Usage {
echo "--no-prefix Will suppress time / date suffix from output" echo "--no-prefix Will suppress time / date suffix from output"
echo "--silent Will run osync without any output to stdout, used for cron jobs" echo "--silent Will run osync without any output to stdout, used for cron jobs"
echo "--errors-only Output only errors (can be combined with silent or verbose)" echo "--errors-only Output only errors (can be combined with silent or verbose)"
echo "--non-interactive Don't show running animation in cron / service mode"
echo "--summary Outputs a list of transferred / deleted files at the end of the run" echo "--summary Outputs a list of transferred / deleted files at the end of the run"
echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files" echo "--log-conflicts [EXPERIMENTAL] Outputs a list of conflicted files"
echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)" echo "--alert-conflicts Send an email if conflictual files found (implies --log-conflicts)"
@ -6629,6 +6632,10 @@ function GetCommandlineArguments {
opts=$opts" --errors-only" opts=$opts" --errors-only"
_LOGGER_ERR_ONLY=true _LOGGER_ERR_ONLY=true
;; ;;
--non-interactive)
opts=$opts" --non-interactive"
_OFUNCTIONS_SHOW_SPINNER=false
;;
--summary) --summary)
opts=$opts" --summary" opts=$opts" --summary"
_SUMMARY=true _SUMMARY=true

2
sync.conf.example
Unescape Escape View File

@ -100,7 +100,7 @@ REMOTE_3RD_PARTY_HOSTS="www.kernel.org www.google.com"
[MISC_OPTIONS] [MISC_OPTIONS]
## Optional arguments passed to rsync executable. The following are already managed by the program and shoul never be passed here ## Optional arguments passed to rsync executable. The following are already managed by the program and shoul never be passed here
## -r -l -p -t -g -o -D -E - u- i- n --executability -A -X -L -K -H -8 -zz –skip-compress –checksum –bwlimit –partial –partial-dir –no-whole-file –whole-file –backup –backup-dir –suffix ## -r -l -p -t -g -o -D -E - u- i- n --executability -A -X -L -K -H -8 --zz -–skip-compress -–checksum –-bwlimit –-partial –-partial-dir –-no-whole-file –-whole-file –-backup –-backup-dir –-suffix
## --exclude --exclude-from --include --include-from --list-only --stats ## --exclude --exclude-from --include --include-from --list-only --stats
## When dealing with different filesystems for sync, or using SMB mountpoints, try adding --modify-window=2 --omit-dir-times as optional arguments. ## When dealing with different filesystems for sync, or using SMB mountpoints, try adding --modify-window=2 --omit-dir-times as optional arguments.
RSYNC_OPTIONAL_ARGS="" RSYNC_OPTIONAL_ARGS=""

Write Preview
Loading…
Cancel
Save