Library of bash scripts.
See http://aks.github.io/bash-lib/ for a prettier rendition.
Author: Alan K. Stebbens [email protected]
export PATH=$PATH:$HOME/lib
source bash-lib.sh
To create a command-line interface (CLI) script, you can start with
cli-template.sh:
cp $HOME/lib/cli-template.sh my-new-script
Then, edit my-new-script as appropriate, adding options, adding functions and doing
whatever the script is intended to do.
If the files are installed somewhere else, then change $HOME/lib accordingly.
This bash library is modular, and the individual utilities can be independently sourced, as needed.
Each library has a corresponding test script to ensure proper operation before installation. These test scripts are also the basis for regression tests, after new features are added (or bugs are fixed).
For example, the text-utils.sh library has a test script called
test-text-utils.sh. The test-utils.sh library is used to operate all the
tests and makes a very good example of how to implement TDD in bash scripts.
The installation is managed with make, using Makefile which, in turn,
sources Makefile.inc. If any changes are needed to support your
installation, the changes should be made within the Makefile.
make
Show the various make targets.
make tests
Run all the tests to confirm proper operation. Some of the tests can take a few minutes. Progress will be shown, so there is no guessing.
make install
Install the bash library into $HOME/lib (the default).
make install libdirs=/usr/local/lib
Install into /usr/local/lib.
make clean
This will remove any temporary output files (from testing). It will also
remove prompt-colors.sh because it is a generated file.
If this bash library is installed into an alternative path, e.g., /opt/lib,
then any scripts that wish to make use of them will need to modify the PATH
environment variable, in order to source the library files without explicit
paths.
This library is available at [https://github.com/aks/bash-lib.git].
There is a script called maybe-install-bash-lib that can be incorporated into
the installation process of other bash libraries that depend on bash-lib. It
will check to see if the named utilities are installed, and if not, perform an
installation from the repository.
If you wish to make improvements, feel free to fork this repo, make and test your changes, and the issue a pull request.
As part of your testing, you'll probably need to source reset-util-vars.sh,
which defines reset_util_vars, which you can then invoke to reset the shell
variables that prevent redundant sourcings. Alternatively, you can increment
the version number in the utility libraries that you are modifying.
Many of these utility functions have helpful argument checking. In order to
avoid unnecessary overhead, each function name that provides argument checking
also has a more efficient, non-argument checking name, prefixed with __.
Follow the links below for detailed descriptions of each module.
- arg-utils.sh
- calendar-utils.sh
- date-utils.sh
- hash-utils.sh
- help-util.sh
- list-utils.sh
- prompt-colors.sh
- real-utils.sh
- run-utils.sh
- sh-utils.sh
- talk-utils.sh
- text-utils.sh
- test-utils.sh
- time-utils.sh
The arg-utils.sh library is a collection of bash functions that enable
flexible argument handling on functions that need to be able to accept
arguments on the command-line or on STDIN.
When writing a bash function that can accept input on the command line or from
STDIN, the function should begin with an invocation of one of the following
functions.
For example, if we had a function that needed a numeric argument, the following invocation would be used:
local f=`numarg_or_input "$1"`
If a text argument is needed:
local txtarg=`arg_or_input "$1"`
For those cases where two or more arguments can be accepted, either on the
command-line or from STDIN:
local args=( `args_or_input "$@"` )
The following are the arg-util functions:
numarg_or_input "$1"
Return a numeric argument or read it from STDIN
arg_or_input "$1"
Return the argument or read it from STDIN
args_or_input "$@"
Return arguments or read them from STDIN
args_or_stdin "$@"
Return the arguments or read all of STDIN
append_args "$@"
Append the arguments to the next line from STDIN
append_arg "$1"
Append the argument to the next line from STDIN
Let's say we have two bash functions to convert Celsius to Fahrenheit and
vice-versa. Let's call them c2f and f2c. With these functions, they can
be used in two ways:
Typical functions with arguments:
c2f 69 # convert 69C to F
f2c 10 # convert 10F to C
Or, accepting their input on STDIN, as in a pipe:
echo 69 | c2f # convert 69C to F
echo 10 | f2c # convert 10F to C
The advantage of the latter approach is that the functions can be fitted into
a pipe where the data can come from another process directly on its STDOUT.
The definition of these two functions would be:
# f2c -- convert F to C via: (°F - 32) x 5/9 = °C
function f2c() {
local f=`numarg_or_input "$1"`
echo "scale=1; ($f - 32)*5/9' | bc
}
# c2f -- convert C to F via °C x 9/5 + 32 = °F
function c2f() {
local c=`numarg_or_input "$1"`
echo "scale=1; $c * 9/5 + 32" | bc
}
calendar-utils provide some basic calendaring functions.
Usage:
source calendar-utils.sh
Functions:
date_to_jdn YYYY MM DD -- return Julian Day Number for given DATE
day_of_week YEAR MM DD [STYLE] -- return the day of the week for the given date
days_in_month YEAR MM -- return # of days in the month MM for YEAR
days_in_month[MM] -- array indexed by month (1..12) to return # days
easter YEAR -- return date of Easter for YEAR
gregorian_easter YEAR -- return date of Easter for YEAR in Gregorian calendar
is_leap_year YEAR -- return whether or not YEAR is a leap year
jdn_to_date JDN -- return date for given Julian Day Number (JDN)
julian_period YEAR -- return Julian year for given Gregorian YEAR
style_for_year YEAR -- compute the calendar style for the given YEAR
week_number YYYY MM DD -- return the week number for the given date
Astronomical (obscure) Functions:
epact YEAR -- return the epact for the given YEAR
golden_number YEAR -- return the golden number for YEAR
indiction YEAR -- return indication for YEAR
paschal_full_moon YEAR -- return the Paschal full moon date for YEAR
solar_number YEAR -- return the solar number for YEAR
The date-utils library enables easy management of dates and its year, month,
and day components. A variety of date formats are supported both on input
parsing, and on output formatting.
The envar EUROPEAN_DATES controls how the format NN/NN/NNNN is interpreted:
if set to 1, that format is taken to mean DD/MM/YYYY, where DD is the day of
the month; otherwise, it is parsed as MM/DD/YYYY.
date_parse [DATESTRING] # can be any modern date format
date_arg [DATESTRING] # an alias for old scripts
Parse DATESTRING in one of several recognized formats: YYYY-MM-DD,
YYYY.MM.DD, YYYY/MM/DD, YYYY MM DD, MM DD YYYYY, DD.MM.YYYY, and DD MM YYYYY (if EUROPEAN_DATES is set). If the DATESTRING is successfully
parsed, the variables year, month, and day are set to their respective
numbers. date_arg is another name for the same function.
If DATESTRING is empty, a line of input from STDIN is read and used
instead. This makes the script handy in a pipe. For example:
extract_first_date_from_log /var/log/messages | date_parse
The function date_parse, as well as all of the functions below will set
year, month, and day from the date extracted.
date_parse_str "DATESTRING"
date_parse_ymd YYYY MM DD
date_parse_mdy MM DD YYYY
date_parse_dmy DD MM YYYY
date_parse_mmmdy MMM DD YYYY
datetime_parse_ymdhm YYYY MM DD HH MM
Each of the above functions are used by date_parse once it has matched the
corresponding syntax. So, if your script/command knows, in advance, the
precise date (or datetime) format, it can use one of the above,
format-specific functions.
month_number MONTHNAME
month_num MONTHNAME
Given a month name, output it's index.
days_in_month MONTH
The days_in_month function converts a month number or name (spelled out or
abbreviated) into a number of days corresponding to that month (not including
leap-year effects). Example: days_in_month Feb ==> 28
days_in_month[M]
Array of integers, indexed by month number, corresponding to the number of
days in the given month M.
days_before_month[M]
Array of integers representing the number of days from the beginning of the
year up to the month M.
is_leap_year YEAR
Return 0 (true) if YEAR is a leap year; return 1 (false) otherwise.
last_day_of_month YYYY MM
Return the last day of the month corresponding to year YYYY and month MM.
All dates can be converted to a given number of days since the beginning of the Julian calendar, jdays, or the beginning of the Common Era calendar, adays, which as been defined as 12/31/0000. Just like Celsius and Kelvin are similar measures of temperature with dissimilar origins, jdays and adays are both measures of days, but from different origins.
date_to_jdays YYYY MM DD
date_to_adays YYYY-MM-DD
Returns the number of Julian or absolute days from the beginning of the
Gregorian calendar for the given date, which can be specified with three
numeric arguments, or a single string argument, which must be parsable by
date_parse.
jdays_to_date JDAYS
Converts JDAYS (a Julian day number) into the corresponding date. If the
date is greater than October 10, 1584, then the Gregorian calendar is used,
otherwise the Julian calendar is used for earlier dates.
adays_to_date ABSDAYS
Converts ABSDAYS into a date formatted by print_date.
adays_to_jdays ADAYS
jdays_to_adays JDAYS
These functions convert from absolute days to Julian day number, and vice-versa.
week_number [DATESTRING | YYYY MM DD] - the week number
date_to_weekday_name [DATESTRING | YYYY MM DD] - the name of the week day
date_to_weekday_num [DATESTRING | YYYY MM DD] - the day of the week (0..6)
date_day_num [DATESTRING | YYYY MM DD] - the day number of the date
days_at_epoch # The number of absolute days at 1970-01-01.
date_to_days_since_epoch DATESTRING - the number of days since the Epoch
date_to_seconds DATESTRING - the number of seconds since the Epoch
today -- today's date
yesterday -- yesterday's date
tomorrow -- tomorrow's date
date_format [FORMAT] YYYY MM DD
date_format [FORMAT] YYYY-MM-DD
The format_date function accepts an optional format string, followed by
three numeric arguments, for the year, month, and day, or a single string
argument in any of the parsable date formats, and reformats into the default
date format, given by DATE_FORMAT. If DATE_FORMAT is not defined, the format
%F is used. (See man strftime for details).
Most of the date format codes are performed by a function, which are listed
here. Typically, they would be invoked via date_format CODE DATESTRING. All
of these df_... functions accept no arguments, taking the date component
values they need from the variables set by date_parse: year, month, and
day.
| Function | Code | Result |
|-------------------+------+---------------------------------------|
| df_weekday_name | %A | full weekday name |
| df_weekday_abbrev | %a | abbreviated weekday name |
| df_month_name | %B | full month name |
| df_month_abbrev | %b | abbreviated month name |
| df_century | %C | The century digits of the year |
| df_date_time | %c | %a %b %D %Y |
| df_mmddyy | %D | mm/dd/yyyy |
| df_day | %d | day of month (01..31) |
| df_month | %e | m or mm: month number, space leader |
| df_fin_date | %F | financial date: %Y-%m-%d (YYYY-MM-DD) |
| df_year4 | %G | Year as 4 digits, using space leader |
| df_year2 | %g | The last two digits of the year |
| df_day_of_year | %j | The day of the year: (000 .. 366) |
| df_month0 | %m | mm - month number, zero-filled |
| \n | %n | _newline_ |
| df_seconds | %s | seconds |
| \t | %t | _tab_ |
| df_week_num0 | %U | week number (0..51) |
| df_weekday_num1 | %u | weekday number (1..7) for (Mon..Sun) |
| df_week_num_ISO | %V | week number by ISO standards |
| df_eby4 | %v | %e %b %Y |
| df_week_num1 | %W | week number (1..52) |
| df_weekday_num0 | %w | weekday number (0..6) for (Sun..Sat) |
| df_date | %x | mm/dd/YYYY (%m/%d/%Y) |
| df_year04 | %Y | Year as 4 digits, with zero leader |
| % | %% | percent-sign |
month_names=( - 'January' 'February' ... 'December' )
The month_names array is indexed with origin-1 as 'January'. It is used by
the month name formatting functions.
print_date [FORMAT] YYYY MM DD
print_date [FORMAT] DATESTRING
printd [FORMAT] DATESTRING
The function print_date and printd will print the DATESTRING value, or
the date components, as given by YYYY, MM, DD.
date_adjust DATESTRING [+-] NUM [KIND] ... - adjust the DATE by +- NUM KIND [d,w,m,y]
date_add DATESTRING NUM [dwmy] - add NUM days, weeks, months, or years
date_sub DATESTRING NUM [dwmy] - subtract NUM days, weeks, months, or years
days_between DATESTRING1 DATESTRING2 - compute difference (in days) between two dates
get_date_x_years_after X DATESTRING - get the date X years after a date
get_date_x_years_since X DATESTRING - alias to .._after
get_date_x_years_before X DATESTRING - get the date X years before a date
get_date_last_quarter_end DATESTRING - get the date of the last quarter end
get_date_x_days_after X DATESTRING - get the date X days after a date
get_date_x_days_since X DATESTRING - alias
get_date_x_days_before X DATESTRING - get the date X days before a date
In all the above cases, DATESTRING defaults to the value of today.
Hashes are associative arrays. Hashes have keys and associated values. Use this library to simplify and ease your use of associated arrays.
These are the hash utilities:
hash_init VAR [DEFAULT]
Initialize VAR as an empty hash.
hash_default VAR
Return the default value for HASH.
hash_set_default VAR DEFAULT
Set the default value for HASH.
hash_put VAR KEY VAL ...
Insert [KEY]=VAL into the hash.
hash_set VAR KEY VAL
Alias to hash_put.
hash_get VAR KEY
Output the item associated with KEY in VAR to STDOUT.
hash_delete VAR KEY
Delete VAR[KEY].
hash_delete_if VAR KEY CONDITION
Delete VAR[KEY] if CONDITION is true.
hash_keys VAR
Return all the keys in hash VAR.
hash_values VAR
Return all the values in hash VAR.
hash_each VAR KEYVAR EXPR
Evaluate EXPR, setting KEYVAR to each key in VAR.
hash_copy HASH NEWHASH KEY1 ...
Copy items at KEY1, .. from HASH1 to NEWHASH.
in_hash VAR KEY
has_key VAR KEY
hash_member VAR KEY
hash_include VAR KEY
Test if KEY is in the hash VAR.
hash_size VAR
Returns the number of [key]=value pairs
hash_merge VAR1 VAR2
Merge key/value pairs from VAR2 with VAR1.
hash_print HASHVAR [indent=INDENT] [width=WIDTH] [gutter=GUTTER] [cols=COLS] [sep=SEP]
Print the items in HASHVAR in vertically-sorted columns. The
number of columns is determined by COLS, if given, or by WIDTH
(defaults to 80) divided by the maximum width of all items in
HASHVAR.
Use GUTTER blanks (default 2) to separate columns.
If SEP is given, it is used to delimit columns instead of blanks.
Each option may be abbreviated to its leading character (e.g., "g" for "gutter").
hash_help # describe the list functions
This utility makes it easy to provide helpful responses for shell functions that are missing arguments.
Each collection of related shell functions can share a common help_FUNC
function, which is then filtered for the specific function name for which
help is being sought.
Each function that can be used by a user should start with a call to
help_args_func, passing the HELPFUNC, $#, and the minimum number of
arguments.
If the using function is called with less than the required arguments the
HELPFUNC is invoked and the output filtered through a simple filter that does
not print until the calling function name is found and then prints only until
the next empty line of test.
Each collection of functions that wish to make use of this utility should
have a HELPFUNC that prints a brief description of each command (function),
where each function name begins an unindented comment line, with exactly one
blank after the comment character. A description may follow -- as a bash
comment, indented or not. Finally, the doc entry for the given function is
an empty comment line.
For reference examples, please see either list-utils.sh or hash-utils.sh.
help_pager <<END_OF_MESSAGE
some message
...
END_OF_MESSAGE
bash script utilities for managing lists of things.
In the descriptions below, VAR is an array variable and VAL*, are
values.
The functions that modify a list variable (e.g., list_push, list_pop)
cannot be used within a sub-shell (e.g., a command substitution).
Commands executed within a sub-shell are incapable of affecting variables in
the parent shell. In other words, the expression var=$(list_pop list) looks
nice but won't work. Instead, do:
list_pop some_list # sets "$item" with the popped value
These are the list utilities:
list_init VAR
Initialize VAR as an empty list.
list_add VAR VAL1 [VAL2 ...]
Add one or more values (VAL1..) to the end of VAR. There is no check for
duplicates.
list_add_once VAR VAL1 [VAL2 ..]
Add one or more values (VAL1..) uniquely to the end of VAR.
list_push VAR VAL ...
Alias to list_add.
list_insert VAR VAL ...
Insert VAL.. at the front of VAR.
list_insert_once VAR VAL ...
Insert one or more values (VAL..) at the front of VAR.
list_pop VAR
Removes VAL from the list VAR and returns it in the variable item.
list_remove VAR VAL ...
Remove one or more given values (VAL..) from the list VAR.
list_get VAR N
Get the Nth item of VAR to STDOUT.
list_item VAR N
Set the variable item to the Nth item of VAR.
list_set VAR N VAL
Set the Nth item of VAR to VAL.
list_items VAR [START [END]]
Return list items from START to END (or all).
list_copy LIST NEWLIST [START [END]]
Copy list LIST to NEWLIST, from START to END.
in_list VAR [-any|-all] VAL ...
Return true if one or more values are in list VAR.
list_size VAR
Returns the number of items in VAR.
list_dump VAR
Output the contents of VAR, with indexes.
sort_str VAL ...
Sort the space-separated words of VAL ...
list_sort VAR
Sort the contents of VAR (a list) in place.
list_sorted VAR
Output the items of VAR sorted.
sort_str2lines STRING
Sort STRING with each item in a separate line.
sort_list2lines VAR
Sort list VAR with each item in a separate line.
split_into VAR "STRING" [SEP]
Split STRING by SEP into VAR.
split_str "STRING" [SEP]
Split STRING by SEP.
list_join VAR [SEP] ..
Join the items in VAR into a list, separated by SEP.
SEP can be:
-
AND- separate with" and " -
OR- separate with" or " -
KEYS- enclose each item withX'and', followed by, -
TAB- use tabs to separate items -
NL- separate each item with newline (and some spaces) -
NOWRAP- do not auto-wrap long lines (default isWRAP) -
','- separate items with a comma (default) -
str- separate each item with an given string.join_lines
Read STDIN and catenate lines; remove trailing NL.
list_lookup LISTVAR KEY
Lookup and return matching KEY in LISTVAR. KEY can be an abbreviation.
list_grep LISTVAR PAT
Perform a grep using pattern PAT across the contents of LISTVAR.
list_map LISTVAR EXPR [JOINSTR]
Create a list of EXPR applied to each item in LISTVAR.
list_reduce LISTVAR EXPR [INIT]
Reduce LISTVAR using EXPR, with initial value INIT.
list_sum LISTVAR
Sum the items in LISTVAR.
list_max LISTVAR
Return the maximum item in LISTVAR.
list_min LISTVAR
Return the minimum item in LISTVAR.
list_avg LISTVAR
Return the average of the items in LISTVAR.
list_print LISTVAR [indent=INDENT] [width=WIDTH] [sep=SEP] [cols=COLS]
list_print LISTVAR [i=INDENT] [w=WIDTH] [s=SEP] [c=COLS]
Print the items in LIST in vertically-sorted columns. Use COLS if given,
otherwise the number of columns is computed from WIDTH (defaults to 80) and
the maximum width of all the items in LISTVAR.
list_help
Describe the list functions.
There are convenient aliases for most list_XXX functions as XXX_list. For
example, join_list => list_join, list_pop => pop_list, map_list =>
list_map, etc. This allows people who think VERB-NOUN to use the functions
like grep_list, while other people who think NOUN-VERB can use list_grep.
The canonical function name begins with list_.
Programmers wanting to make use of the list functions can use any of the list
names prefixed with __ to avoid the argument checking that is more helpful
for interactive usage. For example, within a script, the size of a list FOO
is obtained with __list_size FOO.
split_into VAR "STRING" SEP
Splits a STRING into parts using separator (SEP) (default is ',')
and assigns the resulting separated, quoted strings to the VAR.
split_str "STRING" [SEP]
Outputs the split of STRING into parts using a separator SEP
(defaulting to space/tab).
split_input [SEP]
Splits the input text into parts using separator (SEP) (default is tab).
For the split functions:
If SEP does not include a space (" "), care is taken to avoid removing
whitespace from the split values.
SEP can be multiple characters; for example ' ,' (a space, comma) will
split using both space and comma. By default, splitting is done by tabs.
list_lookup LIST "WORD"
Looks up WORD in the array LIST for the uniquely matching item, using
disambiguating case-insensitive matching. If no match, return empty string and
code 1; if 2 or more matches, return empty string, and error code 2.
list_grep LIST PATTERN
Look up items matching PATTERN in the array LIST. Return all matching items,
and return code 0. If no matching items, return empty string and return code 1.
lookup_error CODE WORD [NOTFOUNDMSG [AMBIGMSG]]
A utility function to be used in conjunction with a lookup_list or
grep_list invocation. CODE is an error code returned from lookup_list or
grep_list. WORD is the word used on the search, and is used as the "%s"
argument in either error message. NOTFOUNDMSG is the error message used in
the case of error code 1. AMBIGMSG is the error message used in the case of
error code 2.
lookup_error is used like this:
read -p "What word do you want to use?" word
words=( a list of words to search from )
found=`lookup_list words $word` || lookup_error $? $word \
"'%s' is not a valid word" \
"'%s" is an ambiguous word"
The option-util.sh library is a small set of functions to manage
building options and arguments, which is often needed in the
development of command-line utilities.
These functions use two global variables: option_pairs and
options. The option_pairs variable is used to accumulate pairs
of options and arguments, e.g.: -F FILE, while options is used to
accumulate single character options that can be clustered behind a
single dash "-".
All of the accumulated options and arguments can be output with
all_opts.
init_opts # empty "option_pairs" and "options"
reset_opts # same as init_opts
add_optarg OPTION ARG .. # add OPTION and ARG to the option_pairs list
add_option OPTION .. # add OPTION to the single options list
add_opt OPTION .. # eg: add_arg -c -d .. or add_arg c d ..
all_opts # outputs both option_pairs and options
prompt-colors.sh is a bash script that creates two functions:
define_color_names and reset_color_names, and then invokes the former. The
define_color_names function creates a bunch of color variable names, setting
them to the corresponding bash prompt escape sequences. This allows the
bash PS1 and related prompts to be easily colored using color names, like
${Red} and ${BoldCyan}. The function reset_color_names removes all the
color names from the current bash session.
Usage:
source prompt-colors.sh
The file prompt-colors.sh is actually dynamically generated from the script
generate-color-names, and is not even part of this repository. To create it, you
must run make, or invoke generate-prompt-colors manually.
The real-utils.sh bash library provides real number arithmetic in bash
scripts. Real numbers are managed as floating point strings in the format
"X.Y", where X is the integer portion, and Y is the fractional part.
Usage:
source real-utils.sh
real_compute "EXPRESSIN" [SCALE]
real_eval "EXPRESSION" [SCALE]
real_cond EXPRESSION [SCALE]
real_int REAL
real_frac REAL
Descriptions:
real_compute "EXPRESSION" [SCALE]
The real_compute bash function evaluates EXPRESSION using syntax, operators
and functions as described in the bc manual. All numbers and variables
within EXPRESSION are interpreted by bc. The result of the computation is
output to STDOUT.
If an error occurs, there is no indication. This function does not set a
return code, nor does it set the shell status variable $?. Use real_eval
for those effects.
In addition to the operators and functions defined by bc, the following
additional functions are also made available within the EXPRESSION:
abs(x) deg(x) log10(x) rad(x)
acos(x) exp(x) logn(x) round(x,s)
asin(x) frac(x) ndeg(x) sin(x)
atan(x) int(x) pi() tan(x)
cos(x) log(x) pow(x,y)
To see the bc definitions of these functions, use the real_functions
function.
real_eval "EXPRESSION" [SCALE]
The real_eval bash function invokes real_compute on the arguments, prints
the result on STDOUT, and returns with the bc return code $? (0 or 1, for
success or error, respectively).
real_cond "EXPRESSION" [SCALE]
EXPRESSION is a real number conditional which should evaluate to 1 or 0. The
return status is 0 for true, 1 for false. Example usage:
if real_cond "$num < $max" 2 ; then
...
fi
real_scale=NUM
Set the precision of subsequent real number arithmetic results. The default is 2.
real_int REAL -- outputs the integer portion of a REAL number
real_frac REAL -- outputs the fractional portion of a REAL number
sin R, cos R, tan R -- trig functions on radians R
asin X, acos X, atan X -- inverse trig functions
cotan X, sec X, cosec X -- cotangent, secant, cosecant
arccot X -- arc-cotangent
hypot X Y -- hypotenuse X, Y [sqrt(X^2 + Y^2)]
sqrt X -- square-root of X
logn X, log X -- natural log, log base 10
exp X -- exponent X of E (e.g., e^X)
pow X Y -- power function [X^Y]
rad D -- convert degrees D to radians
deg R -- convert radians R to degrees
ndeg R -- convert radians R to natural degrees (0..360)
round X S -- Round X to S decimals. When S=0, rounds to the nearest integer.
real_int X -- outputs integer portion of X
real_frac X -- outputs fractional portion of X
abs X -- Return the absolute value of X.
PI = 3.141592653589793
TAU = 6.283185307179586 # 2*PI
E = 2.718281828459045
Shell utility functions for running system commands:
run COMMAND ARGS .. Show `COMMAND` `ARGS` if `$norun` or `$verbose`;
run `COMMAND` unless `$norun`.
safe_run COMMAND ARGS ... Same as "run", but always executes.
rm_file_later FILE Cause `FILE` to be removed upon program exit.
add_trap "CMD" SIGNAL .. Add `CMD` to the trap list for `SIGNAL`
Handy functions for writing bash-based scripts
The shell command utility functions consist of several groups of functions which collectively are quite useful in development command-line utilities and other system scripts.
The following are separate modules that are included with sh-utils:
arg-utils- help with arguments orSTDINhelp-util- help with help on selected functionsoption-utils- manage option and argument listsrun-utils- run system commands, with$norunand$verbosetalk-utils- conditional output toSTDERR
There are also some miscellaneous functions:
rm_file_later FILE Cause `FILE` to be removed upon program exit.
add_trap "CMD" SIGNAL .. Add `CMD` to the trap list for `SIGNAL`
rm_trap "CMD" SIGNAL .. Remove `CMD` from the trap list for `SIGNAL` ..
get_traps SIGNAL Get the trap list for `SIGNAL`
filter_traps 'CMD' Filter `CMD` from the trap list on `STDIN`
reset_traps SIGNAL Reset (remove) the trap list for `SIGNAL`
fn_exists FUNCTION Return 0 (true) if `FUNCTION` exists; 1 (false) otherwise
The talk, error, and die functions print their arguments on STDERR.
The talk and talkf functions print unconditionally to STDERR, and return
success (0). The related functions with prefixes of 'v', 'vo', 'nr', 'nv',
'nq' and 'nrv' print conditionally and return success (0) if they printed, and
failure (1) otherwise. This allows them to be used on conditionals.
The warn function is just another name for talk: it prints its output on
STDERR. The error function does the same, accepting an optional error
CODE, and then exits. The die function sends a SIGABRT signal to the
parent process id, forcing an abort.
talk MSG .. Print all args on `STDERR`
vtalk MSG .. If `$norun` or `$verbose` is set, print all args.
votalk MSG .. If `$verbose` only (no `$norun`) is set, print all args.
nrtalk MSG .. if `$norun` set, print all args
nvtalk MSG .. Unless `$verbose` is set, print all args
nqtalk MSG .. Unless `$quiet` is set, print all args
nrvtalk MSG .. If `$norun` or `$verbose` is set, print all args.
talkf FMT ARGS .. Printf the arguments on `STDERR`
vtalkf FMT ARGS .. If `$norun` or `$verbose` is set, `talkf` the args
votalkf FMT ARGS .. If `$verbose` only (no `$norun`) is set, `talkf` the args
nrtalkf FMT ARGS .. If `$norun` set, `talkf` the args
nvtalkf FMT ARGS .. Unless `$verbose` is set, `talkf` the args
nqtalkf FMT ARGS .. Unless `$quiet` is set, `talkf` the args
nrvtalkf FMT ARGS .. If `$norun` or `$verbose` is set, `talkf` the args
warn MSG Print all args on `STDERR`
error [CODE] "MSG" Print `MSG` on `STDERR`, then exit with code `CODE` (or 2)
die "MSG" Print `MSG` on `STDERR`, then die (with `kill -ABRT`)
warnf FMT ARGS .. Printf `FMT` `ARGS` on `STDERR`
errorf [CODE] FMT ARGS .. Printf `FMT` `ARGS` on `STDERR`, then exit `$CODE` [2]
dief FMT ARGS .. Printf `FMT` `ARGS` on `STDERR`, then die (with `kill -ABRT`)
Text processing utilities for bash scripts.
usage:
export PATH=.:$HOME/lib:$PATH
source text-utils.sh
The following functions are provided by this library:
lowercase STRING # return the lowercase string
uppercase STRING # return the uppercase string
trim STRING # trim blanks surrounding string
ltrim STRING # trim left-side blanks from STRING
rtrim STRING # trim right-side blanks from STRING
squeeze STRING # squeeze multiple blanks in string
split_str STRING [SEP] # split STRING using SEP [default: ' \t']
split_input [SEP] # split STDIN using SEP [default: ' \t']
args2lines [ARG ..] # echo each ARG (or STDIN) on a separate line
sort_str2lines "STRING .." # output the sorted words in STRING on separate lines
join_lines # join lines on STDIN together with newlines
sort_str [WORDS TO BE SORTED] # return the sorted list of WORDS
str_sort [WORDS TO BE SORTED] # an alias for 'sort_str'
html_encode [STRING] # encode STRING (or STDIN) for html
url_encode [STRING] # encode STRING (or STDIN) for url
html_decode [STRING] # decode STRING (or STDIN) from HTML encoding
url_decode [STRING] # decode STRING (or STDIN) from URL encoding
Most functions, except split_str and sort_str, can be used in a pipe
without an argument. For example:
echo "This is a string" | uppercase => "THIS IS A STRING"
html_encode <input-file >html-file
The test-utils.sh library provides an infrastructure for test-driven
development (TDD) of bash scripts.
Usage:
source test-utils.sh
test_NAME1() {
start_test
... # perform operations and test the results
end_test
}
test_NAME2() {
start_test
... # perform operations and test the results
end_test
}
init_tests [ARGUMENTS]
run_tests
summarize_tests
A run is a collection of tests (within a single file); each test has a name.
A test is a set of related operations with checks on the results.
A check` tests or compares values, which quietly succeeds, or results in an error. The error message can be provided, or a default one is used.
At the end of each test, the number of checks and errors is remembered for later summarization.
At the end of the run, all checks and error counts are summarized.
While the tests and checks are being performed, output is occuring to show the progress. There are three modes of output: terse, errors-only, and detailed.
Terse mode shows each test name followed by the number of checks, and how many of those checks had errors. Terse mode is the default.
In errors-only mode, successful tests still show the same as terse mode, but
tests with error checks show the error message followed by a stack dump
indicating the location of the error. Errors-mode is indicated by the -e
option when invoking the test script.
In details mode, the tests and checks are run in verbose mode, showing both
successful checks and errors. Details mode is indicated by the -d option.
When invoking the test script, the command line argument can be used to pass
a PATTERN that is used to match a subset of the test names. By default, all
tests with the pattern "test_" are run. For example, if the pattern "basic" is
used, all tests with the string "basic"` will be run, and no others.
In order to be discovered for automatic test runs, the tests functions must
have the function name begin with test_.
A common technique for test naming is: test_NN_some_descriptive_name, where
NN is a number. This allows easy reference by the NN to selectively run a
test or tests.
Below are the tests that are currently supported:
check_value VAL ERROR
check_empty VAL ERROR
Expression tests
check_true "EXPR" ERROR
check_false "EXPR" ERROR
Array item tests
check_size LIST SIZE ERROR # same as check_size_eq
check_size_XX LIST SIZE ERROR
check_item LIST INDEX VAL ERROR
check_item_equal LIST INDEX VAL ERROR
check_item_unequal LIST INDEX NONVAL ERROR
Hash tests
check_key HASH KEY ERROR
check_no_key HASH KEY ERROR
check_key_value HASH KEY VALUE ERROR
String tests
check_equal VAL1 VAL2 ERROR
check_unequal VAL1 VAL2 ERROR
check_match VAL1 REGEXP ERROR
check_nomatch VAL1 REGEXP ERROR
Numeric tests
check_eq N1 N2 ERROR
check_ne N1 N2 ERROR
check_lt N1 N2 ERROR
check_le N1 N2 ERROR
check_gt N1 N2 ERROR
check_ge N1 N2 ERROR
Output tests
check_output [NAME] EXPRESSION [ERROR]
Evaluate EXPRESSION and compare its output against a previously collected
reference output. If the output matches, the test succeeds. If the output
does not match, print ERROR or a default error message.
Use NAME as the unique identifier for files in which the stdout, stderr,
and reference output is identified.
Reference output can be created by the -k ($keep) option when the test is
run.
The first time a new check_output test is evaluated, there will not be a
collected reference output to compare against, and the test will fail.
Please carefully review the various test files in this repository:
test-date-utils.sh -- test the functions in date-utils.sh
test-hash-utils.sh -- test the functions in hash-utils.sh
test-list-utils.sh -- test the functions in list-utils.sh
test-real-utils.sh -- test the functions in real-utils.sh
test-sh-utils.sh -- test the functions in sh-utils.sh
test-text-utils.sh -- test the functions in text-utils.sh
test-template.sh -- a template for future tests
test-utils.sh -- the functions described here
NOTE: The following functions are not yet implemented.
check_out [NAME] EXPRESSION [ERROR]
check_out_none [NAME] EXPRESSION [ERROR]
check_err [NAME] EXPRESSION [ERROR]
check_err_none [NAME] EXPRESSION [ERROR]
Check that STDOUT or STDERR is or is not empty when evaluating
EXPRESSION, or show the ERROR (or default) message.
check_out_eq [NAME] EXPRESSION VALUE [ERROR]
check_err_eq [NAME] EXPRESSION VALUE [ERROR]
Check that the STDOUT, or STDERR of the evaluated EXPRESSION matches
VALUE, or show the ERROR (or a default error message).
check_out_ne [NAME] EXPRESSION VALUE [ERROR]
check_err_ne [NAME] EXPRESSION VALUE [ERROR]
Check that the STDOUT or STDERR of the evaluated EXPRESSION does not
contain VALUE, or show the ERROR.
In all cases, the ERROR message is optional.
The time-utils library enables easy management of timestamps, with hour,
minute, seconds, and timezone components. A variety of time formats are
supported both on input parsing, and on output formatting.
time_parse [TIMESTRING]
time_arg [TIMESTRING]
Parse TIMESTRING in one of several recognized formats: HH:MM:SS,
HH:MM:SS.ssss, If the TIMESTRING is successfully parsed, the variables
hours, mins, and secs are set the corresponding numbers. time_arg is
another name for the same function.
If TIMESTRING is empty, a line of input from STDIN is read and used instead.
time2secs [TIMESTRING]
Parse TIMESTRING (or STDIN) and convert to seconds.
time_format [FORMAT] HOURS MINS SECS
time_format [FORMAT] TIMESTRING
The time_format function accepts an optional format string, followed by three
numeric arguments, for the hour, minutes, and seconds, or a single string
argument in any of the parsable date formats, and reformats into the default
time format, given by TIME_FORMAT. If TIME_FORMAT is not defined, the
format %T is used. (See man strftime for details).
time_add TIME1 TIME2
Add TIME1 to TIME2 and produce a time_format result.
time_sub TIME1 TIME2
Subtract TIME2 from TIME1 and produce a time_format result.
vim: set ai sw=2