mirror of https://github.com/deajan/osync
You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
277 lines
9.0 KiB
Plaintext
277 lines
9.0 KiB
Plaintext
Coding style used for my bash projects (v3.2 Oct 2018)
|
|
As bash is clearly an error prone script language, we'll use as much standard coding as possible, including some quick and dirty debug techniques described here.
|
|
|
|
++++++ Header
|
|
|
|
Always use the following header
|
|
|
|
----BEGIN HEADER
|
|
#!/usr/bin/env bash
|
|
|
|
PROGRAM="program-name" # Long description
|
|
AUTHOR="(C) 20XX-20YY by Orsiris de Jong"
|
|
CONTACT="http://www.example.com me@example.com"
|
|
PROGRAM_BUILD=YYYYMMDDVV
|
|
|
|
## Optional instructions
|
|
----END HEADER
|
|
|
|
Using bind style versionning:
|
|
YYYYMMDDVV (Year, Month, Day, Revision): Example: 2015012402 = 2nd revision of 24 Jan 2015
|
|
|
|
#!/usr/bin/env bash instead of #!/bin/bash
|
|
|
|
Change old scripts with
|
|
for i in $(grep -r '#!/bin/bash' * |cut -f1 -d':'); do sed -i 's&#!/bin/bash&#!/usr/bin/env bash&g' $i; done
|
|
|
|
type instead of type -p for bash test (other shells don't know -p)
|
|
|
|
++++++ Indentation
|
|
|
|
Using tabs
|
|
Transform old shell scripts using unexpand command
|
|
|
|
++++++ Comments
|
|
|
|
Some command # comment
|
|
## Some comment on a new line
|
|
################################################# Some separation
|
|
|
|
++++++ Work comments
|
|
|
|
Whenever there is some idea to postpone, use #TODO(priority):[dev-name:] some remark
|
|
Priority can be critical, high, medium, low, verylow. No release can happen if there are TODOs other than low or verylow.
|
|
Example: #TODO(high):deajan: need to do something
|
|
|
|
A "work in progress" marker must be left on the line a dev is working when it's work isn't finished). Marker is #WIP:dev-name: some remark
|
|
dev-name is mandatory if more than one person is coding
|
|
Example: #WIP:deajan: missing function something
|
|
|
|
++++++ Variables
|
|
|
|
All local variables names have each first letter of the words uppercase and all others lowercase, except for the first word where all letters are lowercase
|
|
Example: someLongVariable
|
|
All global variables are full upercase, separated by _
|
|
Example: EXEC_TIME
|
|
All environment variables (verbose, silent, debug, etc) have prefix _ and are full upercase, separated by _
|
|
Example: _PARANOIA_DEBUG
|
|
|
|
Exec time variables that can take boolean values should use true and false instead of 1 and 0.
|
|
|
|
++++++ Functions
|
|
|
|
All function names should begin with an uppercase letter for every word, the other letters should be lowercase
|
|
Example: SomeFunctionThatRocks
|
|
|
|
Bash does not provide any checks against missing function arguments. Also, missing quotes can lead to an inconsistent number of arguments.
|
|
Most functions should have a first line that calls the special function __CheckArguments, which checks the number of given arguments for a function in order
|
|
to find possible problems. Number of arguments are given as first argument to __CheckArguments. May be a number or a range, eg 0-2 if the function takes optional arguments.
|
|
__CheckArguments will only trigger when the script is launched with _PARANOIA_DEBUG=yes. Also, it will only exist in the debug version.
|
|
Use the following convention for function definition:
|
|
|
|
function SomeFunction {
|
|
__CheckArguments 0 $# "$@" #__WITH_PARANOIA_DEBUG
|
|
...
|
|
}
|
|
|
|
Use sed ':a;N;$!ba;s/\n{\n/ {\n/g' to convert functions that have opening brackets on a new line.
|
|
|
|
If the function has arguments, use local variable names that are more readable than $1...$n. Explain via comments what those variables contain if needed.
|
|
Declare arguments before launching __CheckArguments:
|
|
|
|
function AnotherFunction {
|
|
local varName="${1}"
|
|
local otherVarName="${2}" # This variable contains stuff
|
|
__CheckArguments 2 $# "$@" #__WITH_PARANOIA_DEBUG
|
|
...
|
|
}
|
|
|
|
Functions should always have return status
|
|
function RandomFunction {
|
|
...
|
|
return $?
|
|
}
|
|
|
|
++++++ Sub functions
|
|
|
|
When a function is a subroutine of another function, it is called _SomethingAsSubFunction:
|
|
Example:
|
|
|
|
function _ApplyLocally
|
|
function _ApplyRemotely
|
|
function Apply
|
|
|
|
++++++ For and While statements
|
|
|
|
For and while statements will have the "do" part on the first line.
|
|
Example:
|
|
|
|
for i in "${var[@]}"; do
|
|
...
|
|
done
|
|
|
|
while [ $i -eq 1 ]; do
|
|
...
|
|
done
|
|
|
|
++++++ If statements
|
|
|
|
If statements will be fully written (word "if" must be used). then is written on the same line.
|
|
(Use sed ':a;N;$!ba;s/]\n\t*then/]; then/g' to convert files to this format... Replace "],new line, zero or more tabs, then" by "; then")
|
|
if [ something ]; then
|
|
stuff
|
|
else
|
|
other stuff
|
|
fi
|
|
|
|
++++++ Logging
|
|
|
|
A logging function is available that writes both to log file and stdout/stderr.
|
|
It has the following global variable modifiers:
|
|
|
|
_LOGGER_SILENT=true/false: disables any output to stdout/stderr
|
|
_LOGGER_VERBOSE=true/false: logs messages with log level VERBOSE
|
|
_LOGGER_ERR_ONLY=true/false: disables logging to log file and stdout/stderr except for CRITICAL, ERROR, WARN and ALWAYS log levels.
|
|
|
|
The following log levels exist:
|
|
|
|
- PARANOIA_DEBUG: Only used by debugging functions themselves
|
|
- DEBUG: Only log this when _DEBUG flag is set in program. Any command forged for eval instruction should be logged by this.
|
|
- NOTICE: Standard messages
|
|
- ALWAYS: Standard messages, regardless of _LOGGER_ERR_ONLY
|
|
- WARN: Requires attention
|
|
- ERROR: Program produced an error but continues execution
|
|
- CRITICAL: Program execution is halted
|
|
|
|
Can be called with:
|
|
Logger "My message" "LOGLEVEL" $retval
|
|
|
|
$retval is an optional parameter that passes the exit code of the command that triggered the logging message
|
|
$retval, along with function stack, script pid and current pid can be found in the ERROR /WARN alert files ($RUN_DIR/$PROGRAM.Logger.error.$SCRIPT_PID)
|
|
|
|
++++++ Eval
|
|
|
|
Most commands should be logged to a tmp file.
|
|
The basic way of doing is:
|
|
|
|
cmd='"something '$somevar'" > some_file 2>&1'
|
|
eval $cmd &
|
|
WaitForTaskCompletion $! 0 0 1 0 true false true false
|
|
retval=$?
|
|
if [ $retval -ne 0 ]; then
|
|
Logger "Some error message" "ERROR" $retval
|
|
fi
|
|
|
|
++++++ includes
|
|
|
|
Using merge.sh, the program may have includes like
|
|
include #### RemoteLogger SUBSET ####
|
|
All possible includes are listed in ofunctions.sh
|
|
Mostly, includes are needed to port functions to a remote shell without writing them again.
|
|
|
|
++++++ Remote execution
|
|
|
|
Remote commands should always invoke bash (using '"'"' to escape single quotes of 'bash -c "command"'). It is preferable to use ssh heredoc in order to use plain code.
|
|
If local and remote code is identical, wrap remote code in a function so only minor modifications are needed.
|
|
Remote code return code is transmitted via exit.
|
|
|
|
cmd=$SSH_CMD' '"'"'bash -c "some; commands \"'$VARIABLE'\" some; other; commands" > some_file'"'"' 2>&1'
|
|
|
|
Better formule
|
|
|
|
$SSH_CMD remoteVar="'$localVar'" 'bash -s' << 'ENDSSH' > 2>&1
|
|
function remoteSub {
|
|
some code
|
|
return 0
|
|
}
|
|
|
|
remoteSub
|
|
exit $?
|
|
ENDSSH
|
|
retval=$?
|
|
if [ $retval -ne 0 ]; then
|
|
Logger "Some error message" "ERROR" $retval
|
|
fi
|
|
|
|
We also need to transmit a couple of environment variables (RUN_DIR; PROGRAM; _LOGGER_VERBOSE... see current setups) in order to make standard code.
|
|
Include works here too.
|
|
|
|
++++++ File variables
|
|
|
|
All eval cmd should exit their content to a file called "$RUNDIR/$PROGRAM.${FUNCNAME[0]}.$SCRIPT_PID"
|
|
Dots are used instead of '_' so variables can be separated with a forbidden char in variable names, so the separtors apply as wished.
|
|
|
|
|
|
++++++ String function calls
|
|
|
|
String returning functions should only be called this way in order to deal with spaces:
|
|
Quoting happens outside the function call.
|
|
|
|
echo "$(myStringFunction $myStringVar)"
|
|
|
|
++++++ ofunctions
|
|
|
|
As obackup and osync share alot of common functions, ofunctions.sh will host all shared code.
|
|
Dev programs n_osync.sh and n_obackup.sh will source ofunctions.sh
|
|
Release programs will still include ofunctions.sh in order to enhance ease of use.
|
|
|
|
Ofunctions are defined like:
|
|
|
|
#__FUNC:FunctionName
|
|
function FunctionName {
|
|
|
|
}
|
|
#__ENDFUNC
|
|
|
|
These functions are inserted into code that has placeholders like #__FUNC:FuncName
|
|
|
|
+++++++ includes
|
|
|
|
ofunctions parts can be directly included in shell scripts.
|
|
The parts that needs to be included must be contained within specific comments:
|
|
|
|
#### MyFunction SUBSET ####
|
|
function MyFunction {
|
|
...
|
|
}
|
|
#### MyFunction SUBSET END ####
|
|
|
|
These can later be included in shell scripts with:
|
|
|
|
|
|
include #### MyFunction SUBSET ####
|
|
|
|
In order to have those includes parsed, we use bootstrap.sh to launch the original shell script.
|
|
Original shell script will not work because include is not a bash statement.
|
|
Include the following code into original shell script
|
|
|
|
include #### _OFUNCTIONS_BOOTSTRAP SUBSET ####
|
|
[ "$_OFUNCTIONS_BOOTSTRAP" != true ] && echo "Please use bootstrap.sh to load this dev version of $(basename $0)" && exit 1
|
|
+++++++ Exit codes
|
|
|
|
Normal exit code = 0
|
|
Run with errors exit code = 1
|
|
Run with warnings exit code = 2
|
|
Wrong shell exit code = 127
|
|
Usage function exit code = 128
|
|
|
|
+++++++ Detailled debugging
|
|
|
|
When launching the program with 'bash -x', add SLEEP_TIME=1 so wait functions won't spam output
|
|
Ex:
|
|
|
|
SLEEP_TIME=1 bash -x ./program.sh
|
|
|
|
++++++ Finding code errors
|
|
|
|
Before every release, shellcheck must be run
|
|
Also a grep -Eri "TODO|WIP" osync/* must be run in order to find potential release blockers
|
|
|
|
Use shellcheck.net now and then (ignore SC2086 in our case)
|
|
|
|
Use a low tech approach to find uneven number of quotes per line
|
|
|
|
tr -cd "'\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'
|
|
tr -cd "\"\n" < my_bash_file.sh | awk 'length%2==1 {print NR, $0}'
|
|
|