Bash
Jump to navigation
Jump to search
Which files are touched by bash
echo exit | strace bash -li |& grep '^open'
getting history of running process
echo -e "attach 23450\ncall write_history(\"/tmp/blah.txt\")\n" > cmd.txt ; gdb -batch -x cmd.txt
useful links
http://mywiki.wooledge.org/BashFAQ/
moving around on bash prompt
C-a Go to beginning C-e Go to end C-f Forward one character C-b Backward one character M-f Forward one word M-b Backward one word C-k Kill to end of line. C-d Kill word forward C-w Kill word backward C-y Yank word
bash Shell
For using multiple filenames in one command in Unix commandline use the following command:
for i in <equal part of all filenames>*;do <command> $i;done
$i is the variable set by the loop.
Example:
for i in sync_m4606_danube_sg_org_*;do /usr/atria/etc/mkorder -data $i -fship blrsxa16.blr.infineon.com;done
More examples:
for i in `ct lsvob |grep systemq | cut -d" " -f4`;do /usr/atria/etc/utils/albd_list -s $i;done
All Files starting with sync_m4606_danube_sg_org_ are used to perform a mkorder -data to forward these packets to Bangalore server.
for i in `cut -f2 -d" " viewlist.del`;do rm -r /home/memhub.view/$i;done
Use viewtags of column 2 of the file viewlist.del to remove the viewstorage.
Alternative
Use xargs for simply passing over results to a simple command:
ps- elf | grep <string> |tr -s " " |cut f<row nr.> -d" "|xargs <command>
tr - text replace with -s for substitute
e.g.:
ps -elf | grep CCMS | tr -s " " | cut -f5 -d" " |xargs kill
loop number
i=0; while [ $i -lt 10 ]; do echo $i; i=`expr $i + 1`; done
interactive shell
echo $- If shell is interactive variable will contain i.
explanation
operand 3, 3 + 6 = 9 '+' is the operator and '3' and '6' are the operands. operator +, 3 + 6 = 9 '+' is the operator and '3' and '6' are the operands. primary That which stands first in order, rank, or importance Arithmetic most elementary branch of mathematics, Command control Testing exit status of a command in order to determine whether a portion of the program should be executed. Conditional branch Logical point in the program when a condition determines what happens next. Logic flow The overall design of the program. Determines logical sequence of tasks so that the result is successful and controlled. Loop Part of the program that is performed zero or more times. User input Information provided by an external source while the program is running, can be stored and recalled when needed.
debug
set -x # activate debugging from here w set +x # stop debugging from here
debug login
Find out what is blocking login.
bash --login --verbose
variables
Character Definition $* Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with the value of each parameter separated by the first character of the IFS special variable. $@ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a separate word. $# Expands to the number of positional parameters in decimal. $? Expands to the exit status of the most recently executed foreground pipeline. $- A hyphen expands to the current option flags as specified upon invocation, by the set built-in command, or those set by the shell itself (such as the -i). $$ Expands to the process ID of the shell. $! Expands to the process ID of the most recently executed background (asynchronous) command. $0 Expands to the name of the shell or shell script. $_ The underscore variable is set at shell startup and contains the absolute file name of the shell or script being executed as passed in the argument list. Subsequently, it expands to the last argument to the previous command, after expansion. It is also set to the full pathname of each command executed and placed in the environment exported to that command. When checking mail, this parameter holds the name of the mail file.
quoting
Single quotes
Single quotes () are used to preserve the literal value of each character enclosed within the quotes. A single quote may not occur between single quotes, even when preceded by a backslash.
Double quotes
Using double quotes the literal value of all characters enclosed is preserved, except for the dollar sign, the backticks (backward single quotes, ``) and the backslash. The dollar sign and the backticks retain their special meaning within the double quotes. The backslash retains its meaning only when followed by dollar, backtick, double quote, backslash or newline. Within double quotes, the backslashes are removed from the input stream when followed by one of these characters. Backslashes preceding characters that don't have a special meaning are left unmodified for processing by the shell interpreter. A double quote may be quoted within double quotes by preceding it with a backslash.
eight kinds of expansion performed
1 Brace expansion
echo {01..3}{a..c}
1a 1b 1c 2a 2b 2c 3a 3b 3c
2 Tilde expansion
If a word begins with an unquoted tilde character ("~"), all of the characters up to the first unquoted slash (or all characters, if there is no unquoted slash) are considered a tilde-prefix. If none of the characters in the tilde-prefix are quoted, the characters in the tilde-prefix following the tilde are treated as a possible login name. If this login name is the null string, the tilde is replaced with the value of the HOME shell variable.
If HOME is unset, the home directory of the user executing the shell is substituted instead. Otherwise, the tilde-prefix is replaced with the home directory associated with the specified login name.
If the tilde-prefix is "~+", the value of the shell variable PWD replaces the tilde-prefix. If the tilde-prefix is "~-", the value of the shell variable OLDPWD, if it is set, is substituted.
If the characters following the tilde in the tilde-prefix consist of a number N, optionally prefixed by a "+" or a "-", the tilde-prefix is replaced with the corresponding element from the directory stack, as it would be displayed by the dirs built-in invoked with the characters following tilde in the tilde-prefix as an argument. If the tilde-prefix, without the tilde, consists of a number without a leading "+" or "-", "+" is assumed.
If the login name is invalid, or the tilde expansion fails, the word is left unchanged.
Ech variable assignment is checked for unquoted tilde-prefixes immediately following a ":" or "=". In these cases, tilde expansion is also performed. Consequently, one may use file names with tildes in assignments to PATH, MAILPATH, and CDPATH, and the shell assigns the expanded value.
3 Shell parameter and variable expansion
echo ${!N*}
NNTPPORT NNTPSERVER NPX_PLUGIN_PATH
The following construct allows for creation of the named variable if it does not yet exist:
${VAR:=value}
4. Command substitution
Command substitution allows the output of a command to replace the command itself. Command substitution occurs when a command is enclosed like this:
$(command) or like this using backticks:`command`
Bash performs the expansion by executing COMMAND and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.
When the old-style backquoted form of substitution is used, backslash retains its literal meaning except when followed by "$", "`", or "\". The first backticks not preceded by a backslash terminates the command substitution. When using the "$(COMMAND)" form, all characters between the parentheses make up the command; none are treated specially.
Command substitutions may be nested. To nest when using the backquoted form, escape the inner backticks with backslashes.
If the substitution appears within double quotes, word splitting and file name expansion are not performed on the results.
5. Arithmetic expansion
$(( EXPRESSION ))
$[ EXPRESSION ]
6. Process substitution
Process substitution is supported on systems that support named pipes (FIFOs) or the /dev/fd method of naming open files. It takes the form of
<(LIST)
or
>(LIST)
The process LIST is run with its input or output connected to a FIFO or some file in /dev/fd. The name of this file is passed as an argument to the current command as the result of the expansion. If the ">(LIST)" form is used, writing to the file will provide input for LIST. If the "<(LIST)" form is used, the file passed as an argument should be read to obtain the output of LIST. Note that no space may appear between the < or > signs and the left parenthesis, otherwise the construct would be interpreted as a redirection.
7. Word splitting
The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.
The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly "'<space><tab><newline>'",
8. File name expansion
File name expansion Bash scans each word for the characters "*", "?", and "[". If one of these characters appears, then the word is regarded as a PATTERN, and replaced with an alphabetically sorted list of file names matching the pattern.
regular expression metacharacters
Operator Effect
. Matches any single character.
? The preceding item is optional and will be matched, at most, once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{N} The preceding item is matched exactly N times.
{N,} The preceding item is matched N or more times.
{N,M} The preceding item is matched at least N times, but not more than M times.
- represents the range if it's not first or last in a list or the ending point of a range in a list.
^ Matches the empty string at the beginning of a line; also represents the characters not in the range of a list.
$ Matches the empty string at the end of a line.
\b Matches the empty string at the edge of a word.
\B Matches the empty string provided it's not at the edge of a word.
\< Match the empty string at the beginning of word.
\> Match the empty string at the end of word.
Two regular expressions may be concatenated; the resulting regular expression matches any string formed by concatenating two substrings that respectively match the concatenated subexpressions.
Two regular expressions may be joined by the infix operator "|"; the resulting regular expression matches any string matching either subexpression.
Repetition takes precedence over concatenation, which in turn takes precedence over alternation. A whole subexpression may be enclosed in parentheses to override these precedence rules.
Parentheses -- ( ) -- enclose groups of REs. They are useful with the following "|" operator and in substring extraction using expr.
The postfix question mark (?) operator makes the preceding regexp optional (i.e, it can occur zero or one times). This shorthand is equivalent to a common use of summing. These two regexps are equivalent:
a(foo)?
afoo|a
Basic versus extended regular expressions
In basic regular expressions the metacharacters "?", "+", "{", "|", "(", and ")" lose their special meaning; instead use the backslashed versions "\?", "\+", "\{", "\|", "\(", and "\)".
POSIX Character Classes. [:class:]
This is an alternate method of specifying a range of characters to match. # [:alnum:] matches alphabetic or numeric characters. This is equivalent to [A-Za-z0-9]. [:alpha:] matches alphabetic characters. This is equivalent to [A-Za-z]. [:blank:] matches a space or a tab. [:cntrl:] matches control characters. [:digit:] matches (decimal) digits. This is equivalent to [0-9]. [:graph:] (graphic printable characters). Matches characters in the range of ASCII 33 - 126. This is the same as [:print:], below, but excluding the space character. [:lower:] matches lowercase alphabetic characters. This is equivalent to [a-z]. [:print:] (printable characters). Matches characters in the range of ASCII 32 - 126. This is the same as [:graph:], above, but adding the space character. [:space:] matches whitespace characters (space and horizontal tab). [:upper:] matches uppercase alphabetic characters. This is equivalent to [A-Z]. [:xdigit:] matches hexadecimal digits. This is equivalent to [0-9A-Fa-f].
Grep multiple words. 21 and 23.
grep --color=always -vE '10\.[0-9]{1,3}\.(21|23)\.[0-9]{1,3}'
Character classes
A bracket expression is a list of characters enclosed by "[" and "]". It matches any single character in that list; if the first character of the list is the caret, "^", then it matches any character NOT in the list. For example, the regular expression "[0123456789]" matches any single digit. Within a bracket expression, a range expression consists of two characters separated by a hyphen. It matches any single character that sorts between the two characters, inclusive, using the locale's collating sequence and character set. For example, in the default C locale, "[a-d]" is equivalent to "[abcd]". Many locales sort characters in dictionary order, and in these locales "[a-d]" is typically not equivalent to "[abcd]"; it might be equivalent to "[aBbCcDd]", for example. To obtain the traditional interpretation of bracket expressions, you can use the C locale by setting the LC_ALL environment variable to the value "C".
command examples
Does you account exist locally?
if grep $USER /etc/passwd;then echo "your user account is managed locally"; fi
Matching lines starting with a given pattern and ending in a second pattern are showed like this:
sandy ~> sed -n '/^This.*errors.$/p' example This is a line not containing any errors.
Are you root?
if [ "$(whoami)" != 'root' ]; then
echo "You have no permission to run $0 as non-root user."
exit 1;
fi
With Bash, you can shorten this type of construct. The compact equivalent of the above test is as follows:
[ "$(whoami)" != 'root' ] && ( echo you are using a non-privileged account; exit 1 ) Similar to the "&&" expression which indicates what to do if the test proves true, "||" specifies what to do if the test is false.
real programmer...
test "$(whoami)" != 'root' && (echo you are using a non-privileged account; exit 1)
Scriptname.
#!/bin/bash
basename=${0##*/}
echo $basename
type of input.
#!/bin/bash case $1 in *[!a-zA-Z]*) echo -e "\nContains only numbers.\n";; *) echo -e "\nIs not only numbers.\n";; esac
- repeat a number of time.
for (( i = 1 ; i <= 10 ; i++ ))
do
echo "i is ${i}"
done
Character classes
Character classes can be specified within the square braces, using the syntax [:CLASS:], where CLASS is defined in the POSIX standard and has one of the values "alnum", "alpha", "ascii", "blank", "cntrl", "digit", "graph", "lower", "print", "punct", "space", "upper", "word" or "xdigit". ls -ld [[:digit:]]*
sed commands
The sed program can perform text pattern substitutions and deletions using regular expressions, like the ones used with the grep command; see Section 4.2. The editing commands are similar to the ones used in the vi editor: Command Result a\ Append text below current line. c\ Change text in the current line with new text. d Delete text. i\ Insert text above current line. p Print text. r Read a file. s Search and replace text. w Write to a file.
awk
The variables $1, $2, $3, ..., $N hold the values of the first, second, third until the last field of an input line. The variable $0 (zero) holds the value of the entire line. This is depicted in the image below,
\a Bell character \n Newline character \t Tab
Quotes, dollar signs and other meta-characters should be escaped with a backslash.
2008-01-27 23:38$ ls -l | awk 'BEGIN { print "You are a loser" }{ print $8 "\t is \t" $5 " bytes" } END { print "But what do you care?" }'
You are a loser
12341 is 0 bytes
12342 is 0 bytes
12343 is 0 bytes
But what do you care?
BEGIN { FS=":" }
{ print $1 "\t" $5 }
$ awk -f printnames.awk /etc/passwd
awk 'BEGIN { OFS=";" ; ORS="\n-->\n" } { print $1,$2}' test
record1;data1
-->
record2;data2
cat processed.awk
BEGIN { OFS="-" ; ORS="\n--> done\n" }
{ print "Record number " NR ":\t" $1,$2 }
END { print "Number of records processed: " NR }
awk -f processed.awk test
Record number 1: record1-data1
--> done
Record number 2: record2-data2
--> done
Number of records processed: 2
--> done
Count the size of file in current directory.
find . -maxdepth 1 -type f -ls | awk 'BEGIN { print "Hello\n" } { total=total + $7 } { print "counting " total } END { print "---------------------------------\nTotal revenue: " total }'
Print last word of sentance.
awk '{ print $NF }'
printf
define output format.
if
if TEST-COMMANDS; then CONSEQUENT-COMMANDS; fi The TEST-COMMAND list is executed, and if its return status is zero, the CONSEQUENT-COMMANDS list is executed. The return status is the exit status of the last command executed, or zero if no condition tested true.
Primary expressions used with if.
The table below contains an overview of the so-called "primaries" that make up the TEST-COMMAND command or list of commands. These primaries are put between square brackets to indicate the test of a conditional expression.
[ -a FILE ] True if FILE exists. [ -b FILE ] True if FILE exists and is a block-special file. [ -c FILE ] True if FILE exists and is a character-special file. [ -d FILE ] True if FILE exists and is a directory. [ -e FILE ] True if FILE exists. [ -f FILE ] True if FILE exists and is a regular file. [ -g FILE ] True if FILE exists and its SGID bit is set. [ -h FILE ] True if FILE exists and is a symbolic link. [ -k FILE ] True if FILE exists and its sticky bit is set. [ -o optname ] True if shell option optname is enabled. [ -p FILE ] True if FILE exists and is a named pipe (FIFO). [ -r FILE ] True if FILE exists and is readable. [ -s FILE ] True if FILE exists and has a size greater than zero. [ -t FD ] True if file descriptor FD is open and refers to a terminal. [ -u FILE ] True if FILE exists and its SUID (set user ID) bit is set. [ -w FILE ] True if FILE True if FILE exists and is writable. [ -x FILE ] True if FILE exists and is executable. [ -O FILE ] True if FILE exists and is owned by the effective user ID. [ -G FILE ] True if FILE exists and is owned by the effective group ID. [ -L FILE ] True if FILE exists and is a symbolic link. [ -N FILE ] True if FILE exists and has been modified since it was last read. [ -S FILE ] True if FILE exists and is a socket. [ FILE1 -nt FILE2 ] True if FILE1 has been changed more recently than FILE2, or if FILE1 exists and FILE2 does not. [ FILE1 -ot FILE2 ] True if FILE1 is older than FILE2, or is FILE2 exists and FILE1 does not. [ FILE1 -ef FILE2 ] True if FILE1 and FILE2 refer to the same device and inode numbers. [ -o OPTIONNAME ] True if shell option "OPTIONNAME" is enabled. [ -z STRING ] True of the length of "STRING" is zero. [ -n STRING ] or [ STRING ] True of the length of "STRING" is non-zero. [ STRING1 == STRING2 ] True if the strings are equal. "=" may be used instead of "==" for strict POSIX compliance. [ STRING1 != STRING2 ] True if the strings are not equal. [ STRING1 < STRING2 ] True if "STRING1" sorts before "STRING2" lexicographically in the current locale. [ STRING1 > STRING2 ] True if "STRING1" sorts after "STRING2" lexicographically in the current locale. [ ARG1 OP ARG2 ] "OP" is one of -eq, -ne, -lt, -le, -gt or -ge. These arithmetic binary operators return true if "ARG1" is equal to, not equal to, less than, less than or equal to, greater than, or greater than or equal to "ARG2", respectively. "ARG1" and "ARG2" are integers. -eq equal -ne not equal -lt lesser than -gt greater than. -ge greater than or equal
Expressions may be combined using the following operators, listed in decreasing order of precedence:
Operation Effect [ ! EXPR ] True if EXPR is false. [ ( EXPR ) ] Returns the value of EXPR. This may be used to override the normal precedence of operators. [ EXPR1 -a EXPR2 ] True if both EXPR1 and EXPR2 are true. [ EXPR1 -o EXPR2 ] True if either EXPR1 or EXPR2 is true.
check both success and error
diff -q /tmp/1 /tmp/2 2>/dev/null && echo same || echo broken
elif
Elif statement.
#!/bin/bash count=99 if [ $count -eq 100 ] then echo "Count is 100" elif [ $count -gt 100 ] then echo "Count is greater than 100" else echo "Count is less than 100" fi
case statement
case EXPRESSION in CASE1) COMMAND-LIST;; CASE2) COMMAND-LIST;; CASEN) COMMAND-LIST;; esac
Each case is an expression matching a pattern. The commands in the COMMAND-LIST for the first match are executed. The "|" symbol is used for separating multiple patterns, and the ")" operator terminates a pattern list. Each case plus its according commands are called a clause. Each clause must be terminated with ";;". Each case statement is ended with the esac statement.
read
read [options] NAME1 NAME2 ... NAMEN
One line is read from the standard input, or from the file descriptor supplied as an argument to the -u option. The first word of the line is assigned to the first name, NAME1, the second word to the second name, and so on, with leftover words and their intervening separators assigned to the last name, NAMEN. If there are fewer words read from the input stream than there are names, the remaining names are assigned empty values.
Option Meaning -a ANAME The words are assigned to sequential indexes of the array variable ANAME, starting at 0. All elements are removed from ANAME before the assignment. Other NAME arguments are ignored. -d DELIM The first character of DELIM is used to terminate the input line, rather than newline. -e readline is used to obtain the line. -n NCHARS read returns after reading NCHARS characters rather than waiting for a complete line of input. -p PROMPT Display PROMPT, without a trailing newline, before attempting to read any input. The prompt is displayed only if input is coming from a terminal. -r If this option is given, backslash does not act as an escape character. The backslash is considered to be part of the line. In particular, a backslash-newline pair may not be used as a line continuation. -s Silent mode. If input is coming from a terminal, characters are not echoed. -t TIMEOUT Cause read to time out and return failure if a complete line of input is not read within TIMEOUT seconds. This option has no effect if read is not reading input from the terminal or from a pipe. -u FD Read input from file descriptor FD.
Do something on each line of output
#!/bin/bash line do echo mv \'$line\' backupold done < <(ls | grep -v backupold)
Do something on all lines of output.
#!/bin/bash
domassive()
{
local item
while read item
do
echo "I have been feed with the following lines" \"${item}\" &
done
wait
}
ls | domassive
Do something on each line of variable.
while read -r line do echo "$line" done <<< "$var"
file descriptors
File input and output are accomplished by integer handles that track all open files for a given process. These numeric values are known as file descriptors. The best known file descriptors are stdin, stdout and stderr, with file descriptor numbers 0, 1 and 2, respectively. These numbers and respective devices are reserved. Bash can take TCP or UDP ports on networked hosts as file descriptors as well.
The output below shows how the reserved file descriptors point to actual devices:
michel ~> ls -l /dev/std* lrwxrwxrwx 1 root root 17 Oct 2 07:46 /dev/stderr -> ../proc/self/fd/2 lrwxrwxrwx 1 root root 17 Oct 2 07:46 /dev/stdin -> ../proc/self/fd/0 lrwxrwxrwx 1 root root 17 Oct 2 07:46 /dev/stdout -> ../proc/self/fd/1 michel ~> ls -l /proc/self/fd/[0-2] lrwx------ 1 michel michel 64 Jan 23 12:11 /proc/self/fd/0 -> /dev/pts/6 lrwx------ 1 michel michel 64 Jan 23 12:11 /proc/self/fd/1 -> /dev/pts/6 lrwx------ 1 michel michel 64 Jan 23 12:11 /proc/self/fd/2 -> /dev/pts/6
Bash allows for both standard output and standard error to be redirected
&> FILE
Echo test after header and before footer.
echo -e '\nTest\n' | cat header.txt /dev/fd/0 footer.txt
Since child processes inherit open file descriptors, it is good practice to close a file descriptor when it is no longer needed. This is done using the
exec fd<&-
Here documents
Frequently, your script might call on another program or script that requires input. The here document provides a way of instructing the shell to read input from the current source until a line containing only the search string is found (no trailing blanks). All of the lines read up to that point are then used as the standard input for a command.
# Start here document cat << BROWSERS mozilla konqueror firefox BROWSERS
# Remove prepended tabs. <<-TAG ... TAG
# Do not expand variables. cat <<"EOF" This won't be expanded: $PATH EOF
here strings
<<< WORD
# The here-strings are a variation of the here-documents.
cat <<< "Hello world... $NAME is here..."
Modifications on variable
ANSWER=$(sed 's/abjorklund/king/g' <<< $USERNAME );echo $ANSWER
for loop
for NAME [in LIST ]; do COMMANDS; done
If [in LIST] is not present, it is replaced with in $@ and for executes the COMMANDS once for each positional parameter
SAVEIFS=$IFS ; IFS=$(echo -en "\n\b"); for FILM in $(cd /filmdir;ls -1);do echo ${FILM}; ln /temp/${FILM} /films/child/Pingu_${FILM}";done ;IFS=$SAVEIFS
for loop with space names in it.
while loop
The while construct allows for repetitive execution of a list of commands, as long as the command controlling the while loop executes successfully (exit status of zero). The syntax is:
while CONTROL-COMMAND; do CONSEQUENT-COMMANDS; done
#!/bin/bash # echo 1 to 5. i="1" while [ $i -le 5 ] do echo $i i=$[$i+1] done
In this example, the break statement will skip the while loop when user enters -1, otherwise it will keep adding two numbers:
#!/bin/bash
while :
do
read -p "Enter two numnbers ( - 1 to quit ) : " a b
if [ $a -eq -1 ]
then
break
fi
ans=$(( a + b ))
echo $ans
done
Early continuation with the continue statement
- To resume the next iteration of the enclosing WHILE loop use the continue statement as follows:
while [ condition ] do statements1 #Executed as long as condition is true and/or, up to a disaster-condition if any. statements2 if (condition) then continue #Go to next iteration of I in the loop and skip statements3 fi statements3 done
until loop
As long as this command fails, the loop continues. The syntax is the same as for the while loop:
until TEST-COMMAND; do CONSEQUENT-COMMANDS; done
redirection
In the example below, output of the find command is used as input for the read command controlling a while loop:
#!/bin/bash # This script creates a subdirectory in the current directory, to which old # files are moved. # Might be something for cron (if slightly adapted) to execute weekly or # monthly. ARCHIVENR=`date +%Y%m%d` DESTDIR="$PWD/archive-$ARCHIVENR" mkdir $DESTDIR find $PWD -type f -a -mtime +5 | while read file do gzip "$file"; mv "$file".gz "$DESTDIR" echo "$file archived" done
break
Mind that break exits the loop, not the script.
continue
The continue statement resumes iteration of an enclosing for, while, until or select loop.
for name in "$LIST"; do if [[ "$name" != *upper:* ]]; then continue fi mv ... done
select
The select construct allows easy menu generation. The syntax is quite similar to that of the for loop: select WORD [in LIST]; do RESPECTIVE-COMMANDS; done LIST is expanded, generating a list of items. The expansion is printed to standard error; each item is preceded by a number. If in LIST is not present, the positional parameters are printed, as if in $@ would have been specified. LIST is only printed once. Upon printing all the items, the PS3 prompt is printed and one line from standard input is read. If this line consists of a number corresponding to one of the items, the value of WORD is set to the name of that item. If the line is empty, the items and the PS3 prompt are displayed again. If an EOF (End Of File) character is read, the loop exits. Since most users don't have a clue which key combination is used for the EOF sequence, it is more user-friendly to have a break command as one of the items. Any other value of the read line will set WORD to be a null string. The read line is saved in the REPLY variable.
#!/bin/bash
echo "This script can make any of the files in this directory private."
echo "Enter the number of the file you want to protect:"
PS3="Your choice: "
QUIT="QUIT THIS PROGRAM - I feel safe now."
touch "$QUIT"
select FILENAME in *;
do
case $FILENAME in
"$QUIT")
echo "Exiting."
break
;;
*)
echo "You picked $FILENAME ($REPLY)"
chmod go-rwx "$FILENAME"
;;
esac
done
rm "$QUIT"
shift
The shift command is one of the Bourne shell built-ins that comes with Bash. This command takes one argument, a number. The positional parameters are shifted to the left by this number, N. The positional parameters from N+1 to $# are renamed to variable names from $1 to $# - N+1.
Say you have a command that takes 10 arguments, and N is 4, then $4 becomes $1, $5 becomes $2 and so on. $10 becomes $7 and the original $1, $2 and $3 are thrown away.
Types of variables
General assignment of values
VARIABLE=12
Using a declare statement, we can limit the value assignment to variables.
declare OPTION(s) VARIABLE=value
Option Meaning -a Variable is an array. -f Use function names only. -i The variable is to be treated as an integer; arithmetic evaluation is performed when the variable is assigned a value (see Section 3.4.6). -p Display the attributes and values of each variable. When -p is used, additional options are ignored. -r Make variables read-only. These variables cannot then be assigned values by subsequent assignment statements, nor can they be unset. -t Give each variable the trace attribute. -x Mark each variable for export to subsequent commands via the environment. Using + instead of - turns off the attribute instead. When used in a function, declare creates local variables.
Constants.
readonly OPTION VARIABLE(s)
arrays
Create multi word array and loop over it.
while IFS= read -r line; do ARRAY+=( "$line" ); done < <( ls -al ) ; for i in "${ARRAY[@]}"; do echo '***' "$i"; done
Create multi word array and loop over it.
mapfile -t ARRAY < <( ls -la ) ; for i in "${ARRAY[@]}"; do echo '***' "$i"; done
declare -a ARRAYNAME ARRAY=(value1 value2 ... valueN) ARRAYNAME[indexnumber]=value
ARRAY=(one two three)
echo ${ARRAY[*]}
one two three
[bob in ~] echo $ARRAY[*]
one[*]
[bob in ~] echo ${ARRAY[2]}
three
[bob in ~] ARRAY[3]=four
[bob in ~] echo ${ARRAY[*]}
one two three four
Deleting array variables
The unset built-in is used to destroy arrays
Code example of Arrays.
#!/bin/bash
# Bash arrays have numbered indexes only, but they are sparse, ie you don't have to define all the indexes. An entire array can be assigned by enclosing thearray items in parenthesis:
arr=(Hello World)
# Individual items can be assigned with the familiar array syntax (unless you're used to Basic or Fortran):
arr2[0]=Hello2
arr2[1]=World2
# But it gets a bit ugly when you want to refer to an array item:
echo ${arr[0]} ${arr[1]}
echo ${arr2[0]} ${arr2[1]}
# In addition the following funky constructs are available:
echo ${arr[*]} # All of the items in the array
echo ${!arr[*]} # All of the indexes in the array
echo ${#arr[*]} # Number of items in the array
echo ${#arr[0]} # Length if item zero
echo ${arr2[*]} # All of the items in the array
echo ${!arr2[*]} # All of the indexes in the array
echo ${#arr2[*]} # Number of items in the array
echo ${#arr2[0]} # Length if item zero
add value to array
ping_reachable+=("${VAR}")
add to variable
Add content to variable
s=test1 s+=test2 echo $s test1test2
Operations on variables
Using the ${#VAR} syntax will calculate the number of characters in a variable
Substitution
If VAR is not defined or null, the expansion of WORD is substituted; otherwise the value of VAR is substituted:
${VAR:-WORD}
This form is often used in conditional tests, for instance in this one:
[ -z "${COLUMNS:-}" ] && COLUMNS=80
It is a shorter notation for
if [ -z "${COLUMNS:-}" ]; then
COLUMNS=80
fi
the value is assigned to the parameter if it does not exist:
echo $TEST2
echo ${TEST2:=$TEST}
a_string
echo $TEST2
a_string
If variable does not exist set it to the one after :=
GOD="andreas"
echo ${GOOD:=${GOD}}
andreas
The following syntax tests the existence of a variable. If it is not set, the expansion of WORD is printed to standard out and non-interactive shells quit. A demonstration:
echo ${TESTVAR:?"There's so much I still wanted to do..."}
echo "TESTVAR is set, we can proceed."
Change variable if it exists. ${VARIABLE:+hej}
Removing substrings
To strip a number of characters, equal to OFFSET, from a variable, use this syntax:
${VAR:OFFSET:LENGTH}
The LENGTH parameter defines how many characters to keep, starting from the first character after the offset point. If LENGTH is omitted, the remainder of the variable content is taken:
export STRING="thisisaverylongname"
echo ${STRING:4}
isaverylongname
echo ${STRING:6:5}
avery
${VAR#WORD} and ${VAR##WORD}
These constructs are used for deleting the pattern matching the expansion of WORD in VAR. WORD is expanded to produce a pattern just as in file name expansion. If the pattern matches the beginning of the expanded value of VAR, then the result of the expansion is the expanded value of VAR with the shortest matching pattern ("#") or the longest matching pattern (indicated with "##"). If VAR is * or @, the pattern removal operation is applied to each positional parameter in turn, and the expansion is the resultant list. If VAR is an array variable subscribed with "*" or "@", the pattern removal operation is applied to each member of the array in turn, and the expansion is the resultant list. This is shown in the examples below:
The opposite effect is obtained using "%" and "%%", as in this example below. WORD should match a trailing portion of string:
Modify variables content.
export TEST="beginning.andreas.ending" && export TEST="${TEST##beginning.}" && export TEST="${TEST%.ending}" && echo "${TEST}"
Replacing parts of variable names
${VAR/PATTERN/STRING} or ${VAR//PATTERN/STRING}
syntax. The first form replaces only the first match, the second replaces all matches of PATTERN with STRING:
variable expansion
parameter result
----------- ------------------------------
${NAME} polish.ostrich.racing.champion
${NAME#*.} ostrich.racing.champion
${NAME##*.} champion
${NAME%%.*} polish
${NAME%.*} polish.ostrich.racing
${LINK/pending.htm/update.htm} string pending.htm is replaced with update.htm in variable.
echo ${TEST} ; /temp/temp/test.txt => echo ${TEST//\//_} ; _temp_temp_test.txt = replace all occurrences of / with _.
convert minor to capital letters
${NAME^^}
Function syntax
Functions either use the syntax
function FUNCTION { COMMANDS; }
FUNCTION () { COMMANDS; }
Both define a shell function FUNCTION. The use of the built-in command function is optional; however, if it is not used, parentheses are needed. The commands listed between curly braces make up the body of the function. These commands are executed whenever FUNCTION is specified as the name of a command. The exit status is the exit status of the last command executed in the body.
Positional parameters in functions
Functions are like mini-scripts: they can accept parameters, they can use variables only known within the function (using the local shell built-in) and they can return values to the calling shell. A function also has a system for interpreting positional parameters. However, the positional parameters passed to a function are not the same as the ones passed to a command or script. When a function is executed, the arguments to the function become the positional parameters during its execution. The special parameter # that expands to the number of positional parameters is updated to reflect the change. Positional parameter 0 is unchanged. The Bash variable FUNCNAME is set to the name of the function, while it is executing. If the return built-in is executed in a function, the function completes and execution resumes with the next command after the function call. When a function completes, the values of the positional parameters and the special parameter # are restored to the values they had prior to the function's execution. If a numeric argument is given to return, that status is returned. A simple example:
trap
trap [COMMANDS] [SIGNALS] This instructs the trap command to catch the listed SIGNALS, which may be signal names with or without the SIG prefix, or signal numbers. If a signal is 0 or EXIT, the COMMANDS are executed when the shell exits. If one of the signals is DEBUG, the list of COMMANDS is executed after every simple command. A signal may also be specified as ERR; in that case COMMANDS are executed each time a simple command exits with a non-zero status. Note that these commands will not be executed when the non-zero exit status comes from part of an if statement, or from a while or until loop. Neither will they be executed if a logical AND (&&) or OR (||) result in a non-zero exit code, or when a command's return status is inverted using the ! operator.
(())
The (( )) construct expands and evaluates an arithmetic expression. If the expression evaluates as zero, it returns an exit status of 1, or "false". A non-zero expression returns an exit status of 0, or "true". This is in marked contrast to using the test and [ ] constructs previously discussed.
arithmetic operators
+ plus - minus * multiplication / division ** exponentiation % modulo, or mod (returns the remainder of an integer division operation) += plus-equal (increment variable by a constant) -= minus-equal (decrement variable by a constant) *= times-equal (multiply variable by a constant) /= slash-equal (divide variable by a constant) %= mod-equal (remainder of dividing variable by a constant)
id++ id-- variable post-increment and post-decrement ++id --id variable pre-increment and pre-decrement - + unary minus and plus ! ~ logical and bitwise negation ** exponentiation * / % multiplication, division, remainder + - addition, subtraction << >> left and right bitwise shifts <= >= < > comparison == != equality and inequality & bitwise AND ^ bitwise exclusive OR | bitwise OR && logical AND || logical OR expr ? expr : expr conditional evaluation = *= /= %= += -= <<= >>= &= ^= |= assignment expr1 , expr2 comma
[[]]
Using the [[ ... ]] test construct, rather than [ ... ] can prevent many logic errors in scripts. For example, the &&, ||, <, and > operators work within a [[ ]] test, despite giving an error within a [ ] construct.
{}
Block of code [curly brackets]. Also referred to as an inline group, this construct, in effect, creates an anonymous function (a function without a name). However, unlike in a "standard" function, the variables inside a code block remain visible to the remainder of the script.
Bash options
Abbreviation Name Effect -C noclobber Prevent overwriting of files by redirection (may be overridden by >|) -D (none) List double-quoted strings prefixed by $, but do not execute commands in script -a allexport Export all defined variables -b notify Notify when jobs running in background terminate (not of much use in a script) -c ... (none) Read commands from ... -e errexit Abort script at first error, when a command exits with non-zero status (except in until or while loops, if-tests, list constructs) -f noglob Filename expansion (globbing) disabled -i interactive Script runs in interactive mode -n noexec Read commands in script, but do not execute them (syntax check) -o Option-Name (none) Invoke the Option-Name option -o posix POSIX Change the behavior of Bash, or invoked script, to conform to POSIX standard. -o pipefail pipe failure Causes a pipeline to return the exit status of the last command in the pipe that returned a non-zero return value. -p privileged Script runs as "suid" (caution!) -r restricted Script runs in restricted mode (see Chapter 21). -s stdin Read commands from stdin -t (none) Exit after first command -u nounset Attempt to use undefined variable outputs error message, and forces an exit -v verbose Print each command to stdout before executing it -x xtrace Similar to -v, but expands commands - (none) End of options flag. All other arguments are positional parameters. -- (none) Unset positional parameters. If arguments given (-- arg1 arg2), positional parameters set to arguments.
bash prompt
^r Search history. !$ expands to the end word of the previous command. !* all args from last line in history !$ last arg from last line in history !^ first arg from last line in history :p !xyz:p will print the command that would be executed rather than executing it. :p is also clever enough to add the printed command to your history list as the last command executed !-2 Run command two commands away. ^a^b Change all a to be in the former command. h: Remove a trailing file name component, leaving only the head. t: Remove all leading file name components, leaving the tail. r: Remove a trailing suffix of the form .xxx, leaving the basename. e: Remove all but the trailing suffix. :s/a/b/ Substitute first a with b in command line. :gs/a/b/ Substitute first a with b in command line. cd !$:h cd to directory mentioned last in your command.
Control Characters
Ctl-A Beginning of line Ctl-B Backspace (nondestructive). Ctl-C Break. Terminate a foreground job. Ctl-D Log out from a shell (similar to exit). Ctl-E End of line. Ctl-F Move forward (nondestructive. Ctl-G BEL. On some old-time teletype terminals, this would actually ring a bell. In an xterm it might beep. Ctl-H Rubout (destructive backspace). Erases characters the cursor backs over while backspacing. Ctl-I Horizontal tab. Ctl-J Newline (line feed). In a script, may also be expressed in octal notation -- '\012' or in hexadecimal -- '\x0a'. Ctl-K Vertical tab. Ctl-L Formfeed (clear the terminal screen). In a terminal, this has the same effect as the clear command. When sent to a printer, a Ctl-L causes an advance to end of the paper sheet. Ctl-M Carriage return. Ctl-Q Resume (XON). This resumes stdin in a terminal. Ctl-S Suspend (XOFF). This freezes stdin in a terminal. (Use Ctl-Q to restore input.) Ctl-U Erase a line of input, from the cursor backward to beginning of line. In some settings, Ctl-U erases the entire line of input, regardless of cursor position. Ctl-V When inputting text, Ctl-V permits inserting control characters. For example, the following two are equivalent: Ctl-V is primarily useful from within a text editor. Ctl-W When typing text on the console or in an xterm window, Ctl-W erases from the character under the cursor backwards to the first instance of whitespace. Ctl-Y Pastes back text previously erased (with Ctl-K or Ctl-U). Ctl-Z Pauses a foreground job.
The default shell on most Linux operating systems is called Bash. There are a couple of important hotkeys that you should get familiar with if you plan to spend a lot of time at the command line. These shortcuts will save you a ton of time if you learn them. Here is the List of Bash Shell Keyboard Shortcuts For Linux Users Ctrl + A - Go to the beginning of the line you are currently typing on Ctrl + E - Go to the end of the line you are currently typing on Ctrl + L - Clears the Screen, similar to the clear command Ctrl + U - Clears the line before the cursor position. If you are at the end of the line, clears the entire line. Ctrl + H - Same as backspace Ctrl + R - Let’s you search through previously used commands Ctrl + C - Kill whatever you are running Ctrl + D - Exit the current shell Ctrl + Z - Puts whatever you are running into a suspended background process. fg restores it. Ctrl + W - Delete the word before the cursor Ctrl + K - Clear the line after the cursor Ctrl + T - Swap the last two characters before the cursor Esc + T - Swap the last two words before the cursor Alt + F - Move cursor forward one word on the current line Alt + B - Move cursor backward one word on the current line Tab - Auto-complete files and folder names Ctrl + W - erase word before cursor Ctrl + Y - to paste it (as in delete and copy) all text in front of the cursor Esc + . (or Esc + Underscore) - Insert Last Argument Ctrl + b - Move back a character Ctrl + f - Move forward a character Ctrl + r - Search the history backwards Ctrl + xx - Move between EOL and current cursor position Ctrl + x @ - Show possible hostname completions Alt + < - Move to the first line in the history Alt + > - Move to the last line in the history Alt + ? - Show current completion list Alt + * - Insert all possible completions Alt + / - Attempt to complete filename Alt + . - Yank last argument to previous command Alt + c - Capitalize the word Alt + d - Delete word Alt + l - Make word lowercase Alt + n - Search the history forwards non-incremental Alt + p - Search the history backwards non-incremental Alt + r - Recall command Alt + t - Move words around Alt + u - Make word uppercase Alt + back-space - Delete backward from cursor Here "2T" means Press TAB twice $ 2T - All available commands(common) $ (string) 2T - All available commands starting with (string) $ /2T - Entire directory structure including Hidden one $ 2T - Only Sub Dirs inside including Hidden one $ *2T - Only Sub Dirs inside without Hidden one $ ~2T - All Present Users on system from "/etc/passwd" $ $2T - All Sys variables $ @2T - Entries from "/etc/hosts" $ =2T - Output like ls or dir
Color text.
echo -e "\033[1mThis is bold text.\033[0m" echo -e '\E[1;33;44m'"BOLD yellow text on blue background"; tput sgr0 Color Foreground Background black 30 40 red 31 41 green 32 42 yellow 33 43 blue 34 44 magenta 35 45 cyan 36 46 white 37 47
graphical input output.
zenity
pipe
Character Action > Redirect standard output 2> Redirect standard error 2>&1 Redirect standard error to standard output < Redirect standard input | Pipe standard output to another command >> Append to standard output 2>&1| Pipe standard output and stan >/dev/null 2>&1 Send both standard output and error to /dev/null some_cmd > some_file 2>&1 & Run command send output to file with standard error and put in background.
Redirecting standard output temporary
#!/bin/bash LOGFILE=~/script.log exec 3>&1 #save FD1 exec 1>$LOGFILE #stdout going to $LOGFILE ls -alh /usr/share #do a command pwd # and another command who am i # at least now I know ;-) echo "Finished" >&3 # This now goes to stdout echo "Now I'm writing to the log file again" exec 1>&3 #Reset FD1 exec 3>&- #Close FD3
Add logging to script
DATE_FILE=$(date '+%Y-%m-%d_%H-%M-%S')
exec >>/tmp/${0##*/}.${DATE_FILE}.$$.log
exec 2>>/tmp/${0##*/}.${DATE_FILE}.$$.log
set -x
echo to standard error
{ echo >&2 "This is an error" ; } >/dev/null
This is an error
- Stack overflow gave this
>&2 echo "error"
where document
cat << EOF > /tmp/file Hello my friend. whats up? EOF
get random number
echo $(( ( RANDOM % 20 ) + 1 ))
spawn processes
#!/bin/bash
JOB_LIMIT=20
for NUMBER in {1..40};do
until [ $(wc -l <<< "${JOBS}") -lt 20 ] ;do
echo "Working with ${NUMBER}. Job limit ${JOB_LIMIT}."
JOBS=$(jobs)
sleep 1
done
(sleep $(( ( RANDOM % 20 ) + 1 )); echo $NUMBER) &
JOBS=$(jobs)
sleep 0.2
done
wait
echo "All jobs are done."
Standard error in pipe
mvn clean install $@ | tee $logfile
echo ${PIPESTATUS[0]}
disable bell
bind 'set bell-style none'