Skip to content

Latest commit

 

History

History
1204 lines (974 loc) · 22.3 KB

bash.md

File metadata and controls

1204 lines (974 loc) · 22.3 KB
Raw

Bash

Comments

  • Single line comments
# This is single line comment
  • Multi lines comments
<<COMMENTS
Line #1 of Multilines comments
Line #2 of Multilines comments
COMMENTS

Echo

  • Simply print message
echo "Hello world!"

Hello world!

  • Print message without interpreting special sequence like \t \n etc
echo "Hello\tworld!"

Hello\tworld!

  • Print message as well as interpret special sequence like \t \n etc
echo -e "Hello\tworld!"

Hello world!

Special variables

  • $0 - Name of the bash script
  • $1 - $9 - Argument passed to the script
  • $# - Number of passed arguments
  • $@ - List of the passed arguments
  • $$ - The process ID (pid) of the current script
  • $? - Exit status of the most recently executed process
  • $! - The process ID (pid) of the most recently executed process
  • $USER - Retrieve username
  • $HOSTNAME - Retrieve machine hostname
  • $SECONDS - The number of seconds since the script was started
  • $RANDOM - Returns a different random number each time is it referred to
  • $LINENO - Current line number of the script

Sleep

sleep 1  # Sleep for 1 second
sleep 1s # Sleep for 1 second
sleep 1m # Sleep for 1 minute
sleep 1h # Sleep for 1 hour
sleep 1d # Sleep for 1 day

Variables

  • Declare variable having integer value
ivar=10
  • Declare variable having character value
cvar='a'
  • Declare variable having string
svar="bash"

Arrays

There are two types of arrays

Indexed

  • Integers can be used to refernece an element

  • Zero based

  • Syntax: declare -a array

  • Declare an empty list

declare -a arr=()

echo "Size of array: ${#arr[@]}"

Size of array: 0

  • Declare an array with two elements
declare -a arr
arr[0]="entry0"
arr[1]="entry1"

echo "Size of array: ${#arr[@]}"

Size of array: 2

  • Declare an array with two elements
declare -a arr=("entry0" "entry1")

echo "Size of array: ${#arr[@]}"

Size of array: 2

  • Declare an array with two elements
arr=("entry0" "entry1")

Size of array: 2

  • Append a new element to the array
declare -a arr=("entry0" "entry1")
arr+=("entry2")

echo "Size of array: ${#arr[@]}"

Size of array: 3

  • Declare an array: zero index based array
iarray=( 10 20 30 40 50 )
  • Array manipulations
iarray=( 10 20 30 40 50 )

echo "Count: ${#iarray[@]}" # Show number of elements in the array
echo "Elems: ${iarray[@]}" # Show all elements of the array
echo "Sub elems: ${iarray[@]:1:3}" # Show all elements of the array b/w the range

Count: 5
Elems: 10 20 30 40 50
Sub elems: 20 30 40

  • Iterate over the array
for ((i=0; i < ${#iarray[@]}; ++i)); do
  echo "Index # $i: ${iarray[i]}"
done

Index # 0: 10
Index # 1: 20
Index # 2: 30
Index # 3: 40
Index # 4: 50

  • Remove a specific element from the array
iarray=( 10 20 30 40 50 )

unset iarray[2] # Remove element # 2
echo ${iarray[@]} # Show all elements of the array
  • Assign sub-elements to a new array
iarray=( 10 20 30 40 50 )

iarr=("${iarray[@]:1:3}") # Assign elements # 1,2,3 of iarray to iarr
echo ${iarr[@]} # Show all elements of the array

20 30 40

Associative

  • Use arbitrary strings
  • Syntax: declare -A array
declare -A aarr
aarr[animal]=lion
aarr[location]=African

Add new key

aarr+=([group]=Pride)
echo "Count: ${#aarr[@]}" # Show number of elements in the array
echo "Elems: ${aarr[@]}" # Show all elements of the array

Count: 3
Elems: lion african pride

  • Iterate keys of the array
for key in "${!aarr[@]}"; do
  echo "Key # $key";
done

Key # animal
Key # location
Key # group

  • Show all keys
echo "${!aarr[@]}"

animal location group

  • Iterate values of the array
for val in "${aarr[@]}"; do
  echo "Value # $val";
done

Value # lion
Value # african
Value # pride

  • Show all values
echo "${assArray1[@]}"

lion african pride

  • Iterate array in key-value pair
for key in "${!aarr[@]}"; do
 echo "$key: ${aarr[$key]}";
done

animal: lion
location: african
group: pride

Array 2D

declare -A matrix
matrix[0,0]='00'
matrix[0,1]='01'
matrix[1,0]='10'
matrix[1,1]='11'

num_rows=2
num_columns=2

for ((x=0;x<num_rows;x++)) do
  for ((y=0;y<num_columns;y++)) do
    echo "$x,$y = ${matrix[$x,$y]}"
  done
done

0,0 = 00
0,1 = 01
1,0 = 10
1,1 = 11

Arithmetics

Simple

  • Different ways to perform calculation
x=2
y=10
echo $(( x + y )) # spaces does not matter
echo $((x+y)) # spaces does not matter
echo $(($x+$y)) # having $ for variable does not matter

12
12
12

  • Perform additional calculation
x=2
((x += 3))
echo x

5

The same can be done for:

  • Addition (+, += )
  • Subtraction (-, -=)
  • Division (/, /=)
  • Multiplication (*, *=)
  • Exponent (**)
  • Modulus (%, %=)

Using let

x=2
# x++ # Error: x++ command not found
let x++ # Using let will allow you to perform it
echo $x

3

  • Different ways to perform calculation
x=2
y=3

z=$((x+y))
echo $z

let z=$((x+y))
echo $z

z="$((x+y))"
echo $z

let z="$((x+y))"
echo $z

5
5
5
5

Using backtick & expr

a=2
b=3
c=`expr $a + $b`
echo $c

5

If statement

  • Need a space after if, [ and before ]
  • Only handful of operators can be used in [ ]
  • You can use < inside [ ] however it must be escaped by \ to stop it acting as redirection operator.
  • Recommendations:
    • Use bash keywords style operator (lt, le, ge, gt, eq, ne) inside [ ]
    • or use (( ))
x=2
y=3

if [ $x == 2 ] && [ $y == 3 ]; then
  echo "Value of x is 2 and y is 3"
elif [ $x == 2 ]; then
  echo "Value of x is 2 and y is 3"
else
  echo "Values of x and y are not valid"
fi

Value of x is 2 and y is 3

  • Same if condition in different style
if [ $x \< $y ]; then
  echo "Value of x is less than y"
fi

if [ "$x" \< "$y" ]; then
  echo "Value of x is less than y"
fi

if [ $x -le $y ]; then
  echo "Value of x is less than y"
fi

if [ "$x" -le "$y" ]; then
  echo "Value of x is less than y"
fi

if (( $x < $y )); then
  echo "Value of x is less than y"
fi
  • Using builtin operators for number comparisons
[[ 2 -eq 2 ]] && echo "Both numbers are equal"
[[ 2 -ne 5 ]] && echo "Both numbers are different"
[[ 2 -lt 6 ]] && echo "First number is less than second number"
[[ 2 -le 7 ]] && echo "First number is less than/equal second number"
[[ 9 -gt 8 ]] && echo "First number is greater than second number"
[[ 9 -ge 9 ]] && echo "First number is greater than/equal second number"

Both numbers are equal
Both numbers are different
First number is less than second number
First number is less than/equal second number
First number is greater than second number
First number is greater than/equal second number

Switch statement

x=7
case $x in
3)
    echo "x is having value 3"
    ;;
4 | 5 | 6 | 7)  # Specifying multiple options
    echo "x is having value b/w 4..7"
    ;;
8)
    echo "x is having value 8"
    ;;
*)
    echo "x is having unknown value"
    ;;
esac

x is having value b/w 4..7

File validation

# -e can be used to check existence of a file or directory
[[ -e "/path/to/folder" ]]      && echo "Directory is there" || echo "Directory is not there"
[[ -e "/path/to/folder/file" ]] && echo "File is there"      || echo "File is not there"

# -f can be only be used to check existence of a file
[[ -f "/path/to/folder" ]]      && echo "Directory is there" || echo "Directory is not there"
[[ -f "/path/to/folder/file" ]] && echo "File is there"      || echo "File is not there"
[[ -f "/path/to/folder/file" ]] || echo "File is not there"
[[ -f "/path/to/folder/file" ]] || { echo "File is not there"; exit 1; }

# Use -d for directory
[[ -d "/path/to/folder" ]]      && echo "Directory is there" || echo "Directory is not there"
[[ -d "/path/to/folder/file" ]] && echo "File is there"      || echo "File is not there"

# Validation of file path
[[ -r "/path/to/folder/file" ]] && echo "File is readable"   || echo "File is not readable"
[[ -w "/path/to/folder/file" ]] && echo "File is writable"   || echo "File is not writable"
[[ -x "/path/to/folder/file" ]] && echo "File is executable" || echo "File is not executable"
[[ -s "/path/to/folder/file" ]] && echo "File size >0 bytes" || echo "File size is of 0 bytes"
[[ -h "/path/to/folder/file" ]] && echo "File is symblink"   || echo "File is not symlink"

# File comparison
[[ "/path/to/f1" -nt "/path/to/f2" ]] && echo "f1 is more recent than f2"
[[ "/path/to/f1" -ot "/path/to/f2" ]] && echo "f2 is more recent than 1"
[[ "/path/to/f1" -ef "/path/to/f2" ]] && echo "f1 & f2 are the same files"

Compound statement

  • second part of the command after ; will not be executed
[ -f test.txt ] && echo "first command"; cat "second command";
  • use compound statement to execute multiple commands
[ -f sample.txt ] && { echo "first command"; cat "second command"; }

Loops

  • while loop (for ever)
while true; do echo "hello"; sleep 2; done
  • while loop
a=1
while [[ $a -le 5 ]]; do
    echo "While loop idx # $a"
    let ++a
done

While loop idx # 1
While loop idx # 2
While loop idx # 3
While loop idx # 4
While loop idx # 5

  • For loop with defined range and default increment
for i in {1..10}; do
    echo "loop # $i with step @ 1"
done

loop # 1 with step @ 1
loop # 2 with step @ 1
loop # 3 with step @ 1
loop # 4 with step @ 1
loop # 5 with step @ 1
loop # 6 with step @ 1
loop # 7 with step @ 1
loop # 8 with step @ 1
loop # 9 with step @ 1
loop # 10 with step @ 1

  • For loop with defined range and custom increment
for i in {1..10..2}; do
    echo "loop # $i with step @ 2"
done

loop # 1 with step @ 2
loop # 3 with step @ 2
loop # 5 with step @ 2
loop # 7 with step @ 2
loop # 9 with step @ 2

  • For loop c-style
for ((i = 1 ; i <= 10 ; i++)); do
  echo "c-style loop # $i with step @ 1"
done

c-style loop # 1 with step @ 1
c-style loop # 2 with step @ 1
c-style loop # 3 with step @ 1
c-style loop # 4 with step @ 1
c-style loop # 5 with step @ 1
c-style loop # 6 with step @ 1
c-style loop # 7 with step @ 1
c-style loop # 8 with step @ 1
c-style loop # 9 with step @ 1
c-style loop # 10 with step @ 1

  • For loop in c-style
for ((i = 1 ; i <= 10 ; i += 2)); do
  echo "c-style loop # $i with step @ 2"
done

c-style loop # 1 with step @ 2
c-style loop # 3 with step @ 2
c-style loop # 5 with step @ 2
c-style loop # 7 with step @ 2
c-style loop # 9 with step @ 2

  • For loop with seq expression
for i in `seq 1 5`; do
  echo "loop with seq expr # $i"
done

loop with seq expr # 1
loop with seq expr # 2
loop with seq expr # 3
loop with seq expr # 4
loop with seq expr # 5

  • for loop to iterate over word list
words="word1 word2 word3"
for word in $words; do
  echo "$word"
done

word1
word2
word3

for word in word1 word2 word3; do echo "$word"; done

word1
word2
word3

  • for loop to iterate over word list
words="
word1
word2  word3
word4           word5
"
for word in $words; do
  echo "$word"
done

word1
word2
word3
word4
word5

String manipulation

Length

  • Length of a string
str="string"
echo "${#str}"

6

Slicing

  • String can be sliced down from front as well from backward. Format will be ${string:position:length}.

  • Extract string starting from the specified character position to the end of the string.

str="1234567890"
echo "${str:6}"

7890

  • Extract string starting from the specified character position to the start of the string.
str="1234567890"
echo "${str:(-6)}"

567890

  • Extract string starting from the specified character position to the specified number of characters.
str="1234567890"
echo "${str:6:4}"

7890

  • Extract string starting from the specified character position to the specified number of characters.
str="1234567890"
echo "${str:(-6):3}"

567

  • Extract specified number characters starting from the string.
str="1234567890"
echo "${str::3}"

123

  • Extract string starting from the specified character from the backwards to the start of the string.
str="1234567890"
echo "${str::-3}"

1234567

Concatenate

str="string1"
str+=" string2"
str="${str} string3"
echo "${str}"

string1 string2 string3

Changing case

  • Convert whole string into uppercase
str=bash
echo "${str^^}"

BASH

  • Convert first character to uppercase
str=bash
echo "${str^}"

Bash

  • Convert whole string into lowercase
str=BASH
echo "${str,,}"

bash

str=BASH
echo "${str,}"

bASH

  • Reverse the each character of the string
str=baSH
echo "${str~~}"

BAsh

  • Reverse the first character of the string
str=baSH
echo "${str~}"

BaSH

Comparison

  • Bash if condition with ==, != can be used for case sensitive comparison.
str="string"
if [ "${str}" == "string" ]; then
    echo "Both strings are equal"
fi

if [ "${str}" != "String" ]; then
    echo "Both strings are different"
fi

Both strings are equal
Both strings are different

  • String comparison using single line if statements
[[ "bash" == "bash" ]] && echo "String are equal" || echo "String are not equal"
[[ "bash" != "bash" ]] && echo "String not equal" || echo "String are equal"

String are not equal
String are equal

See Compound statement

# You can skip any part
[[ "bash" == "bash" ]] && echo "String are equal"
[[ "bash" != "bash" ]] || echo "String are equal"

String are not equal
String are equal

Substitution

For pattern, look for https://wiki.bash-hackers.org/syntax/pattern

  • ${PARAMETER/PATTERN/STRING} - replace first occurrence
  • ${PARAMETER//PATTERN/STRING} - replace all occurrences
  • ${PARAMETER/PATTERN} - remove first occurrence of pattern
  • ${PARAMETER//PATTERN} - remove all occurrences of pattern
str="Bash, hello world! Bash, hello world!"
echo ${str//l/L} # Replace l with L
echo ${str//world} # Remove world

Bash, heLLo worLd! Bash, heLLo worLd!
Bash, hello ! Bash, hello !

  • If $substring matches front end of $string, substitute $replacement for $substring.
str="test hello world test"
echo ${str/#test/L}

L hello world test

  • If $substring matches back end of $string, substitute $replacement for $substring.
str="test hello world test"
echo ${str/%test/L}

test hello world L

  • Remove prefix
str="test hello world test"
echo ${str/#test}

hello world test

  • Remove suffix
str="test hello world test"
echo ${str/%test}

test hello world

  • Remove long suffix
${VAR%%suffix}
  • Remove long prefix
${VAR##prefix}

Using builtin -z and -n operator

For string

str="bash"
[[ -z $str ]] && echo "string is empty" || echo "string is not empty"

string is not empty

str=""
[[ -z $str ]] && echo "string is empty" || echo "string is not empty"

string is empty

str="bash"
[[ -n $str ]] && echo "string is not empty" || echo "string is empty"

string is not empty

str="bash"
[[ -n $str ]] && echo "string is not empty" || echo "string is empty"

string is empty

For variables

a=0
if [ -n "$a" ]; then
  echo "a is set (Using -n)"
else
  echo "a is not set (Using -n)"
fi

# b=0
if [ -n "$b" ]; then
  echo "b is set (Using -n)"
else
  echo "b is not set (Using -n)"
fi

a is set (Using -n)
b is not set (Using -n)

x=0
if [ -z "$x" ]; then
  echo "x is empty (Using -z)"
else
  echo "x is not empty (Using -z)"
fi

# y=0
if [ -z "$y" ]; then
  echo "y is empty (Using -z)"
else
  echo "y is not empty (Using -z)"
fi

x is not empty (Using -z)
y is empty (Using -z)

File path manipulation

path="/path/to/file.ext"
echo "                  ${path}"
echo "Path + File Name: ${path%.ext}"
echo "Path:             ${path%/*.*}"
echo "File Name + ext:  ${path##*/}"
echo "Extension:        ${path##*.}"

/path/to/file.ext
Path + File Name: /path/to/file
Path: /path/to
File Name + ext: file.ext
Extension: ext

Variable substitution

https://www.gnu.org/software/bash/manual/bash.html#Shell-Parameter-Expansion https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02

${parameter:-word}

var=10
echo ${var:-20} # Use 20; if var is not set or null

10

echo ${var:-20} # Use 20 as var is null

20

var=
echo ${var:-20} # Use 20 as var is null

20

${parameter-word}

echo ${var-20} # Use 20 as var is not declared

20

var=
echo ${var-20} # 20 will not be used as var is declared

NOTE: ${var:-20} vs ${var:20} - Funtionality wise both are same. : is forcing to set the value by 20 even if variable is declared but no value is being assigned.

Invoking script

  • Invoking another script (as a separate process) from within script
  • As new script is a separate process so envrionment modifications done for the current script won't be visible to the new script
#!/bin/bash
./script.sh
# - or -
/path/to/script.sh
# - or -
/bin/bash script.sh
  • Invoking another script with in the process
  • Now envrionment modification will be visisble to the new script
#!/bin/bash
. ./script.sh
# - or -
. /path/to/script.sh
# - or -
source script.sh

Random number

  • Generate 32 characters having a-z, A-Z, 0-9
cat /dev/urandom | tr -dc 'a-zA-Z0-9' | head -c 32
  • Generate a number between 0-999
cat /dev/urandom | tr -dc '0-9' | fold -w3 | head -n1

Use -w3 of fold command to control of number of digits in a number.

  • Generate numbers between ranges
shuf -i 100-250 -n 10
shuf -i 100-250 -n 10 --random-source /dev/urandom

Generate numbers b/w 100-250.

Function

function foo() {
  ...
}
  • foo a simple function which grouped a number of statements and to call it simply type foo.
  • function keyboard prior to foo is optional.

Passing arugments to funtions

  • funa is a function which takes any number of parameters and print those.
function funa() {
  echo "funa is passed $# params"
  for i in "$@"; do
    echo "param value is $i";
  done
}
funa

funa is passed 0 params

funa 1

funa is passed 1 params
param value is 1

funa 1 2

funa is passed 2 params
param value is 1
param value is 2

funa 1 2 3

funa is passed 3 params
param value is 1
param value is 2
param value is 3

Return value from function

funb() {
  local param1=$1
  local param2=$2
  local result=$param1+$param2
  echo $result
}
  • echo command of funb will print message on conole
funb "a" "b"

a+b

  • Capture it to a variable
output=$(funb "c" "d")
echo "Output: $output"

Output: c+d

return statement in function

If function is not having any return statement, exit status of the last command of the function will be returned. You can use return statement to provided specific return value. It can be upto 0 to 255.

funa() {
	return 1
}
funa
echo $?

1

funb() {
	return 110
}
funb
echo $?

110

fund() {
	echo "Hello"
}
fund
echo $?

Hello
0

Different style of functions

Global and local variables

Variables are global by nature.

Usage # 1

var=10
function test() {
  var=20 # Global var has been modified
  echo "var in test = $var"
}

test
echo "var in main = $var"

Output:

var in test = 20
var in main = 20

Usage # 2

var=10
function test() {
  local var=20 # Global var will retain its value
               # A var has been created which is local to this function
  echo "var in test = $var"
}

test
echo "var in main = $var"

Output:

var in test = 20
var in main = 10

Usage # 3

func() {
  VAR1='A'
  local VAR2='B'

  echo "func: Variable VAR1: $VAR1"
  echo "func: Variable VAR2: $VAR2"
}

fund() {
  VAR3='C'
  echo "fund: Variable VAR3: $VAR3"

  # Although VAR1 is declared in func but still fund can see
  # because variable in bash are global in nature not matter
  # where you are defining it
  echo "fund: Variable VAR1: $VAR1"

  # As VAR2 is defined using local so fund can't access it
  echo "fund: Variable VAR2: $VAR2"
}

# Call functions
func
fund

func: Variable VAR1: A
func: Variable VAR2: B
fund: Variable VAR3: C
fund: Variable VAR1: A
fund: Variable VAR2:

Argument handling

  • getopts is a built-in Unix command to parse arguments. getopts usage

    • If : is not found at the start of the argument list, getopts does not perform any error checking. Otherwise error will be reported on having unkown option or paramter.
    • getopts does not handle long options (--long). getopt can do the job however it is not built-in functionality of the Bourne shell.

Read a file

FILE='test.txt'
while read LINE; do
  echo $LINE
done < $FILE