Grep and RegEx One-Liners

GREP (Global Regular Expression Print) is a command-line tool for searching and manipulating text files. It allows you to search for specific text patterns, called regular expressions, within a file or set of files and perform various operations on the matched text.

Regular expressions, or regex for short, are patterns used to match and manipulate text. They are a set of rules that define a specific search pattern, allowing you to search for and match text strings based on certain criteria.

Regular expressions are used by various programming languages, text editors, and command-line tools, including GREP. They are powerful tools that allow you to search for complex text patterns, such as email addresses, phone numbers, and URLs, and perform text manipulation tasks such as find-and-replace operations.

In summary, GREP is a command-line tool that uses regular expressions to search for and manipulate text in files. Regular expressions, or regex, are patterns used to match and manipulate text based on certain criteria.

Linux Grep and Regex

The grep command is a command-line utility for searching plain-text data sets for lines that match a regular expression.

To use the command, simply type grep. grep can be piped from and to other commands/operators to fine-tune the search.

Command Format: cat file.txt | grep word

grep regex example:

cat ansible.cfg | grep plugins

Grep common flags

GREP has several command-line options or flags that can modify its behaviour. Some of the most commonly used flags are:

-i: Ignores case sensitivity when searching for patterns.

-r: Searches files recursively in directories and subdirectories.

-n: Displays line numbers of the matched patterns.

-v: Inverts the search results, displaying lines that do not match the pattern.

-w: Matches only whole words that exactly match the search pattern.

-c: Displays the count of the matched patterns.

-l: Displays only the names of the files that contain the matched patterns.

-e: Allows you to specify the pattern to match. This is useful when searching for patterns that start with a hyphen (-).

-f: Allows you to specify a file containing a list of patterns to match.

These are just a few of the most commonly used flags, but many more can be used with GREP. You can type “man grep” in your terminal or command prompt to see a full list of available options.

Regular expression

A “regular expression” is a text string that describes a particular search pattern. (Examples further down)

Regular expressions (regex) consist of various operators or metacharacters that help to match and manipulate text patterns. Some of the most common regex operators are:

. (dot): Matches any character except a new line.

* (asterisk): Matches zero or more occurrences of the preceding character or group.

(plus): Matches one or more occurrences of the preceding character or group.

? (question mark): Matches zero or one occurrence of the preceding character or group.

| (pipe or vertical bar): Matches either the expression before or after the operator.

[]{(square brackets): Matches any character within the brackets.

^ (caret): Matches the start of a line or string.

$ (dollar sign): Matches the end of a line or string.

() (parentheses): Groups multiple characters or expressions together to act as a single unit.

These are just a few of the most commonly used regex operators, but many more can be used to match and manipulate text patterns. You can refer to online resources or documentation for specific programming languages or text editors to learn more about regular expressions.

Grep regular expression examples

Search files for lines with the word Linux:

Command Format: grep linux files

With Linux at the start of a line:

Command Format: grep '^linux' files

sudo grep '^May' /var/log/yum.log

With Linux at the end of a line:

Command Format:  grep 'linux$' files

sudo grep 'root$' /var/log/secure

Show lines containing only linux:

Command Format:  grep '^linux$' files
To demonstrate this RegEx I created a testfile containing:
turbogeek
turbogeek.co.uk
www.turbogeek.co.uk
https://www.turbogeek.co.uk

grep '^turbogeek$' testfile

Lines starting with ‘^s’, \ escapes the ^:

Command Format:  grep '\^s' files

grep '\turbogeek^s' testfile

Note: Output = null as grep ignores

Search for either case sensitive eg. Linux or linux:

Command Format:  grep '[Ll]inux' files
To demonstrate this I updated the testfile
turbogeek
Turbogeek
turbogeek.co.uk
Turbogeek.co.uk
www.turbogeek.co.uk
https://www.turbogeek.co.uk

grep '[T]urbogeek' testfile

grep '[t]urbogeek' testfile

Search for BOB, Bob, BOb or BoB:

grep 'B[oO][bB]' files

Search for blank lines:

grep '^$' files

Search for pairs of numeric digits:

grep '[0-9][0-9]' file

search for uat, replace with sit, globally

:%s/uat/sit/g

What is the correct way to learn Red Hat/Linux/RHCSA? I am asked about this frequently, so I have put together this miniseries as a starter for RHCSA Certification. The skills required for the RHCSA are quite demanding, it is a very challenging exam, but the RHCSA salary is why so many choose to train as a Red Hat Certified Engineer.

Type of grep

grep = grep -G # Basic Regular Expression (BRE)
fgrep = grep -F # fixed text, ignoring meta-characters
egrep = grep -E # Extended Regular Expression (ERE)
pgrep = grep -P # Perl Compatible Regular Expressions (PCRE)
rgrep = grep -r # recursive

Grep and count the number of empty lines

grep -c "^$"

Here, the ^ character matches the beginning of a line, and the $ character matches the end of a line. So, ^$ matches an empty line.

The -c the option counts the number of matching lines rather than displaying the lines themselves.

Replace filename With the file’s name, you want to search for empty lines.

Note that this command counts only completely empty lines. If a line contains any whitespace characters (such as spaces or tabs), it will not be considered empty by this command.

Grep and return only integer

grep -o '[0-9]*'
#or
grep -oP '\d*'

Both grep -o '[0-9]*' and grep -oP '\d*' commands search for zero or more occurrences of digits in the input, and output each occurrence on a separate line using the -o option.

Here’s a breakdown of the commands:

• grep is the command for searching text files for a specified pattern.
• -o option tells grep to only output the matched parts of the line, rather than the entire line.
• [0-9]* is a regular expression pattern that matches zero or more occurrences of digits (0-9).
• \d* is a Perl-compatible regular expression pattern that matches zero or more occurrences of digits.

For example, if you have a file called example.txt with the following contents:

There are 5 apples and 3 oranges.
The number 42 is the answer to everything.

Running the command grep -o '[0-9]*' example.txt or grep -oP '\d*' example.txt would output:

5
3
42

This is because the regular expression pattern [0-9]* or \d* matches all occurrences of digits in the input, and the -o option tells grep to output each match on a separate line.

Grep integer with a certain number of digits (e.g. 3)

grep '[0-9]\{3\}'
# or
grep -E '[0-9]{3}'
# or
grep -P '\d{3}'

Grep only IP address

grep -Eo '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
# or
grep -Po '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}'

Grep whole word (e.g. ‘target’)

grep -w 'target'

#or using RE
grep '\btarget\b'

Grep returning lines before and after match (e.g. ‘bridge0’)

# return also 3 lines after match
grep -A 3 'bridge0'

# return also 3 lines before match
grep -B 3 'bridge0'

# return also 3 lines before and after match
grep -C 3 'bridge0'

Grep string starting with (e.g. ‘S’)

grep -o 'S.*'

Extract text between words (e.g. w1,w2)

grep -o -P '(?<=w1).*(?=w2)'

Grep lines without word (e.g. ‘foo’)

grep -v foo filename

Grep lines not begin with string (e.g. #)

grep -v '^#' file.txt

Grep variables with space within it (e.g. myvar=”some strings”)

grep "$myvar" filename
#remember to quote the variable!

Grep only one/first match (e.g. ‘foo’)

grep -m 1 foo filename

Grep and return number of matching line(e.g. ‘foo’)

grep -c foo filename

Count occurrence (e.g. three times a line count three times)

grep -o foo filename |wc -l

The command grep -o foo filename | wc -l searches for the string foo in the file filename, displays each occurrence of foo on a separate line using the -o option, and then counts the number of lines using the wc command with the -l option. This gives you the total number of occurrences of foo in the file.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• -o option tells grep to only output the matched parts of the line, rather than the entire line.
• foo is the pattern to search for.
• filename is the name of the file to search in.
• | is the pipe symbol, which sends the output of the grep command to the wc command.
• wc is the command for counting words, lines, and characters in a file or stream.
• -l option tells wc to count the number of lines in the input.

For example, if you have a file called example.txt with the following contents:

This line contains foo, foo and foo
Another line containing foo foo
A line that does not match

Running the command grep -o foo example.txt | wc -l would output:

5

This is because there are 5 occurrences of the string foo in the file example.txt. The grep command outputs each occurrence of foo on a separate line, and the wc -l command counts the number of lines in the output, which is 5.

Case insensitive grep (e.g. ‘foo’/’fOO’/’Foo’)

grep -i "foo" filename

The command grep -i "foo" filename searches for the string foo in the file filename and displays the matching lines, ignoring the case of the text.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• -i option tells grep to perform a case-insensitive search.
• foo is the pattern to search for.
• filename is the name of the file to search in.

For example, if you have a file called example.txt with the following contents:

This line contains Foo
Another line containing fOO
A line that does not match

Running the command grep -i "foo" example.txt would output:

This line contains Foo
Another line containing fOO

This is because both the first and second lines contain the string foo, even though the case of the text is different from the search term. The -i option tells grep to ignore the case of the text when searching for the pattern.

COLOR the match (e.g. ‘foo’)!

grep --color foo filename

The command grep --color foo filename searches for the string foo in the file filename and displays the matching lines with the matching text highlighted in color.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• --color option tells grep to highlight the matching text in color.
• foo is the pattern to search for.
• filename is the name of the file to search in.

For example, if you have a file called example.txt with the following contents:

This line contains foo
Another line containing foo
A line that does not match

Running the command grep --color foo example.txt would output:

This line contains \e[01;31mfoo\e[0m
Another line containing \e[01;31mfoo\e[0m

This is because the first two lines of the file contain the string foo somewhere in their contents. The matching text is highlighted in red, which is the default color used by grep. The \e[01;31m and \e[0m are escape sequences used by grep to specify the color of the highlighted text.

Grep searches all files in a directory(e.g. ‘foo’)

grep -R foo /path/to/directory
# or
grep -r foo /path/to/directory

The command grep -R foo /path/to/directory or grep -r foo /path/to/directory (the lowercase “r” option is equivalent to the uppercase “R” option) searches for all files in the directory /path/to/directory and its subdirectories, that contain the string foo and displays the matching lines along with the name of the file where the match was found.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• -R or -r option tells grep to perform a recursive search through directories.
• foo is the pattern to search for.
• /path/to/directory is the path to the directory to search in.

For example, if you have a directory called myfiles with the following files:

myfiles/
├── file1.txt
├── subdir/
│   ├── file2.txt
│   └── file3.txt
└── subdir2/
    ├── file4.txt
    └── file5.txt

Running the command grep -R foo myfiles or grep -r foo myfiles would output:

myfiles/file1.txt:This line contains foo
myfiles/file1.txt:Another line containing foo
myfiles/subdir/file2.txt:Line containing foo in file2.txt
myfiles/subdir/file3.txt:Line containing foo in file3.txt
myfiles/subdir2/file4.txt:Line containing foo in file4.txt
myfiles/subdir2/file5.txt:Line containing foo in file5.txt

This is because all of the files contain the string foo somewhere in their contents. Note that the search is case-sensitive, so it will only match lines containing foo, not Foo or FOO. The output also shows the name of the file where the matching line was found, followed by the matching line itself.

Search all files in a directory, do not ouput the filenames (e.g. ‘foo’)

grep -rh foo /path/to/directory

The command grep -rh foo /path/to/directory searches for all files in the directory /path/to/directory and its subdirectories, that contain the string foo and displays the matching lines without showing the name of the file where the match was found.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• -r option tells grep to perform a recursive search through directories.
• -h option tells grep not to display the filename where the matching line was found.
• foo is the pattern to search for.
• /path/to/directory is the path to the directory to search in.

For example, if you have a directory called myfiles with the following files:

myfiles/
├── file1.txt
├── subdir/
│   ├── file2.txt
│   └── file3.txt
└── subdir2/
    ├── file4.txt
    └── file5.txt

Running the command grep -rh foo myfiles would output:

This line contains foo
Another line containing foo
Line containing foo in file2.txt
Line containing foo in file3.txt
Line containing foo in file4.txt
Line containing foo in file5.txt

This is because all of the files contain the string foo somewhere in their contents. Note that the search is case-sensitive, so it will only match lines containing foo, not Foo or FOO.

Search all files in a directory, and output ONLY the filenames with matches(e.g., ‘foo’)

grep -rl foo /path/to/directory

the command grep -rl foo /path/to/directory would search for all files in the directory /path/to/directory and its subdirectories that contain the string foo, and display the names of the matching files.

Here’s a breakdown of the command:

• grep is the command for searching text files for a specified pattern.
• -r option tells grep to perform a recursive search through directories.
• -l option tells grep to display only the names of the files that match the pattern.
• foo is the pattern to search for.
• /path/to/directory is the path to the directory to search in.

For example, if you have a directory called myfiles with the following files:

myfiles/
├── file1.txt
├── subdir/
│   ├── file2.txt
│   └── file3.txt
└── subdir2/
    ├── file4.txt
    └── file5.txt

Running the command grep -rl foo myfiles would output:

myfiles/file1.txt
myfiles/subdir/file2.txt
myfiles/subdir/file3.txt
myfiles/subdir2/file4.txt
myfiles/subdir2/file5.txt

This is because all of the files contain the string foo somewhere in their contents. Note that the search is case-sensitive, so it will only match files containing foo, not Foo or FOO.

Grep OR (e.g. A or B or C or D)

grep 'A\|B\|C\|D'

The command grep 'A\|B\|C\|D' Search for lines in a file or input containing any characters ‘A,’ ‘B,’ ‘C,’ or ‘D’. The vertical bar | is used to separate the different characters to search for.

For example, if you have a file called example.txt with the following content:

Apple
Banana
Cherry
Date

Running the command grep 'A\|B\|C\|D' example.txt would output:

Apple
Banana
Cherry
Date

This is because all of the lines contain at least one of the specified characters. Note that the search is case-sensitive and will only match uppercase or lowercase letters.

Grep AND (e.g. A and B)

grep 'A.*B'

Regex any single character (e.g. ACB or AEB)

grep 'A.B'

Consider a file named example.txt with the following contents:

This is an example file.
It contains some AxB patterns, such as A1B, A#B, and A_B.

If we run the grep 'A.B' example.txt command, it will search for lines in the file that contain the letters “A” and “B” with any single character in between them. In this case, the pattern 'A.B' will match the following string in the file:

It contains some AxB patterns, such as A1B, A#B, and A_B.

This is because the pattern matches all occurrences of “A” followed by any single character followed by “B” in the line. Therefore, grep will output the entire line as a match.

Note that the period (.) in the pattern matches any single character, including spaces and special characters. So, the pattern will also match strings like “A B”, “A_B”, “A#B”, etc. The . is a wildcard character in regular expressions and can be very useful when searching for patterns that have some variability.

Regex with or without a certain character (e.g. color or colour)

grep 'colou\?r'

Here’s what the command you provided does:

• grep: The command itself.
• 'colou\?r': This is the pattern that grep will search for. It consists of two parts:
  • colou: This matches the characters “colou” exactly.
  • \?: This is a special character that matches the preceding character (in this case, the letter “u”) either zero or one times. So, the \? makes the “u” optional, allowing the pattern to match either “color” or “colour”.
  • r: This matches the letter “r”.

So, in summary, the command searches for lines in a file that contain the words “color” or “colour”, with the “u” being optional. This can be useful when searching for variations in spelling or regional differences.

Grep all content of a fileA from fileB

grep -f fileA fileB

Here’s what the command you provided does:

• grep: The command itself.
• -f fileA: This option tells grep to read the search patterns from fileA instead of the command line. Each line in fileA is treated as a separate pattern to search for.
• fileB: This is the file that grep will search for the patterns in.

So, in summary, the command searches for each line in fileA in fileB. If a line in fileB contains any of the patterns in fileA, grep will output that line. This can be useful for finding lines that match a set of specific patterns, such as a list of keywords or regular expressions.

Grep a tab

grep $'\t'

Here’s what the command you provided does:

• grep: The command itself.
• $'\t': This is a bash-specific syntax that represents a tab character. The $ before the single quotes indicates that this is a special shell parameter, and the \t inside the single quotes is replaced with a tab character before the command is executed.

So, in summary, the command searches for lines in a file that contain a tab character. The exact usage may depend on what file or input is being searched, as the command does not specify a file or input. For example, you might use this command to search for lines in a text file that have tab-separated values.

Grep variable from a variable

$echo "$long_str"|grep -q "$short_str"
if [ $? -eq 0 ]; then echo 'found'; fi
#grep -q will output 0 if match found
#remember to add space between []!

This is a shell script that tests whether the variable $short_str is a substring of the variable $long_str. Here’s how it works:

• echo "$long_str": This command outputs the value of the variable $long_str.
• grep -q "$short_str": This command searches for the value of the variable $short_str in the output of the echo command. The -q option tells grep to be quiet and not output anything; instead, it simply sets the exit code to 0 if a match is found, or 1 otherwise.
• if [ $? -eq 0 ]; then echo 'found'; fi: This is an if statement that tests the exit code of the grep command. If the exit code is 0, then a match was found and the script outputs the message “found”. Otherwise, the script does nothing.

So, in summary, this shell script tests whether $short_str is a substring of $long_str, and outputs the message “found” if it is. The grep -q command is used to perform the substring search, and the if statement tests the exit code of the grep command to determine whether a match was found.

Grep strings between a bracket()

grep -oP '\(\K[^\)]+'

Here’s what the command you provided does:

• grep: The command itself.
• -o: This option tells grep to only output the matching part of the line, rather than the entire line.
• -P: This option enables Perl-compatible regular expressions, which allows for more powerful pattern matching.
• '\(\K[^\)]+': This is the pattern that grep will search for. It consists of two parts:
  • \(: This matches an opening parenthesis.
  • \K: This is a special construct that tells the regular expression engine to “keep” everything that was matched before this point, but not include it in the final match. In other words, \K resets the starting point of the match, effectively “forgetting” what came before it.
  • [^\)]+: This matches one or more characters that are not a closing parenthesis.

So, in summary, the command searches for a pattern consisting of an opening parenthesis, followed by any number of non-parenthesis characters. However, the \K construct causes grep to “forget” about the opening parenthesis, so only the non-parenthesis characters are output. The -o option causes grep to only output this matched pattern, rather than the entire line on which the pattern occurs. The -P option enables the use of Perl-compatible regular expressions, which allows for more powerful pattern matching.

Grep number of characters with known strings in between(e.g. AAEL000001-RA)

grep -o -w "\w\{10\}\-R\w\{1\}"
# \w word character [0-9a-zA-Z_] \W not word character

Here’s what the command you provided does:

• grep: The command itself.
• -o: This option tells grep to only output the matching part of the line rather than the entire line.
• -w: This option tells grep to only match whole words. For example, if the pattern is cat, grep will match lines containing cat but not lines containing catapult.
• "\w\{10\}\-R\w\{1\}": This is the pattern that grep will search for. It consists of two parts:
  • \w\{10\}: This matches any sequence of 10 word characters. Word characters include letters, digits, and underscores. The \{10\} specifies that the preceding character or character class (in this case, \w) must be matched exactly 10 times.
  • \-R: This matches a hyphen (-) followed by an uppercase R.
  • \w\{1\}: This matches any single-word character.

So, in summary, the command searches for a pattern consisting of a sequence of 10-word characters, followed by a hyphen and an uppercase R, and then one more word character. The -o option causes grep to only output this matched pattern rather than the entire line on which the pattern occurs.

Skip directory (e.g. ‘foo’)

grep -d skip 'foo' /path/to/files/*

Here’s what the command you provided does:

• grep: The command itself.
• -d skip: This option tells grep to skip directories while searching for files. If grep encounters a directory, it will not search its contents. Instead, it will move on to the next file.
• 'foo': This is the pattern that grep will search for. In this case, foo is enclosed in single quotes to prevent the shell from interpreting it as a special character.
• /path/to/files/*: This is the path to the directory where grep will start its search. The asterisk * is a wildcard character that matches any file or directory name in that directory.

So, in summary, the command searches for the pattern ‘foo’ in all files in the directory /path/to/files/, but skips any directories it encounters.

RHCSA MiniSeries 

use Grep and Regular Expressions (RegEx)

How to manipulate files in RedHat

Red Hat Permissions

Change the root password on Red Hat

How to use Red Hat as a virtual machine host

configure local storage

manage users and groups in Red Hat Linux

how to configure NTP Network Time Protocol in Red Hat