grep is a workhorse command line utility whose basic job is to search files for patterns and print out matches according to specified options.
Its name stands for global regular expression print, which points out that it can do more than just match simple strings; it can work with more complicated regular expressions which can contain wildcards and other special attributes.
The simplest example of using grep would be:
$ grep pig file
pig
dirty pig
pig food
which finds three instances of the string "pig" in file.
As an example:
$ grep -i -e pig -e dog -r .
will search all files in the current directory and those below it for the strings "pig" or "dog", ignoring case.
If we try to explore the use of regular expressions in detail, it would be a large topic, but here are some examples:
# print all lines that start with "dog"
$ grep "^dog" file
# print all lines that end with "dog"
$ grep "dog$" file
# print all lines that end with "dog"
$ grep d[a-p] file
grep has many options; some of the most important are:
| Option | Meaning |
|---|---|
| -i | Ignore case |
| -v | Invert match |
| -n | Print line number |
| -H | Print filename |
| -a | Treat binary files as text |
| -I | Ignore binary files |
| -r | Recurse through subdirectories |
| -l | Print out names of all files that contain matches |
| -L | Print out names of all files that do not contain matches |
| -c | Print out number of matching lines only |
| -e | Use the following pattern; useful for multiple strings and special characters |
sed stands for stream editor. Its job is to make substitutions and other modifications in files and in streamed output.
Any of the following methods will change all first instances of the string "pig" with "cow" for each line of file, and put the results in newfile:
$ sed s/pig/cow/ file > newfile
$ sed s/pig/cow/ < file > newfile
$ cat file | sed s/pig/cow/ > newfile
where the s stands for substitute. If you want to change all instances, you have to add the g (global) qualifier, as in:
$ sed s/pig/cow/g file > newfile
The / characters are used to delimit the new and old strings. You can choose to use another character, as in:
$ sed s:pig:cow:g file > newfile
Some of the complications come in when you want to use special characters in the strings to be searched for or inserted. For example, suppose you want to replace all back slashes with forward slashes:
$ sed s/’\\’/’\/’/g file > newfile
It is never a bad idea to put the strings in either single or double quotes, the main difference being that environment variables and escaped special characters in double quotes are expanded out while they are not in single quotes, as in:
$ echo "$HOME"
/home/coop
$ echo ’$HOME’
$HOME
If you want to make multiple simultaneous substitutions, you need to use the -e option, as in:
$ sed -e s/"pig"/"cow"/g -e s/"dog"/"cat"/g < file > newfile
and you can work directly on streams generated from commands, as in:
$ echo hello | sed s/"hello"/"goodbye"/g
goodbye
If you have a lot of commands, you can put them in a file and apply the -f option, as in:
$ cat scriptfile
s/pig/cow/g
s/dog/cat/g
s/frog/toad/g
$ sed -f scriptfile < file > newfile
Find all files under the /tmp directory that are newer than an already existing file and give a detailed listing:
$ find /tmp -newer /tmp/tstfile -ls
where it is assumed you will substitute the name of an existing file for /tmp/tstfile.
Find all files under the /etc directory that have a suffix of .conf:
$ find /etc -name "*.conf"
Find all subdirectories under the /etc directory:
$ find /etc -type d
Find all backup files on the system (ending in .bak) and delete them. Do not do this if you need any such files!
$ find / -name "*.bak" -exec rm {} ’;’
Find all entries in /etc/services that include the string ftp:
$ grep ftp /etc/services
Restrict to those that use the tcp protocol:
$ grep ftp /etc/services | grep tcp
or to those that do not use the tcp protocol, while printing out the line number:
$ grep -n ftp /etc/services | grep -v tcp
Get all strings that start with ts or end with st:
$ grep ’^ts’ /etc/services
$ grep ’st$’ /etc/services