Introduction
One of the most useful tools in the linux user’s toolbox is grep. In this article, you’ll learn what grep is, and what it can be used for in the Linux command line and in bash scripting. At the end of this article, you should be comfortable using grep in simple use cases, and understand how it can be used in more complex pipelines
Prerequisites
Grep was one of the first Unix tools developed, and is available on all Linux distributions and on all versions Mac OS X. While Windows does not come with grep built in, you can download a version for Windows or you can use the Powershell command select-string, which is similar.
Step 1 – Using grep to Search a File
The most basic use for grep is looking inside a file to see if it contains a certain string. We’ll start by looking at the “bashrc” file, which is present on both Mac & Linux machines. We’ll check to see if it contains the word “overwrite” (which the default version does). To check the file for the string overwrite, you can run
grep overwrite ~/.bashrc
And the command line will show you the line containing the word overwrite.
For more information, you can use the -n flag to include the line number that contains the match. This can be useful for larger files, or if you want to edit the file later. By default, grep will show you partial matches for yout string. For example, if your search string is spa, grep will show you results for span or sparkas well. To limit results full word matches, use the -w flag.
Step 2 – Using grep to Search Multiple Files
Searching one file is the most basic use for grep, but often we want to look in multiple files to see where some string is. For example, suppose you have a directory called meeting-notes where you keep all your notes for different meetings, in files called meeting-january-2019.txt, meeting february-2018.txtetc. You know which one contains the meeting where you talked about purchasing a VPN subscription. You could open each file and re-read all your notes, or you could just use grep to scan all the files to look for the word VPN:
grep -i vpn meeting-notes/*
And grep will print a list of all the filenames that contain the word VPN, along with the line where it was used. Notice how we use the -i flag to make the search case-insensitive. This will return results for VPNvpnor , or any capitalization.
What if you are storing all your notes in different subdirectories of the meeting-notes directory? For example, you have a meeting-notes/2018 directory, a meeting-notes/2019 directory, etc. grep can handle that as well! Simply use the -R flag (or -r, or –recursive, all work) and grep will show results for all subdirectories as well.
Sometimes you’re only interested in the names of the files that have a match. You can tell grep that you don’t want to see the contents of the files, just the filenames themselves, by using the -l flag (or the full –files-with-matches flag). Then the output will be just the filenames, not the contents.
Step 3 – Using grep in Pipelines
One of the most common and most useful use cases for grep is as part of a command pipeline. Many Unix commands put out much more output than you need, and it can be hard to read fast enough to get the information you want. Grep offers an easy way to filter that down to just the output you’re interested in.
For example, suppose you’re hosting a webpage with nginx on a dedicated server. You want to watch the real-time server logs to see if anyone is visiting the webpage for a contest you created. One way to do this is to repeatedly run grep on the server log file the way we covered in step 1, but that’s inefficient. You have to keep running the same command over and over.
The best way to do this is with a pipeline, a string of commands that run together. You can use the unix command tailto follow the logs in real time, and then pipe that into grep to filter down the real-time logs into just the output you want:
tail -f /var/log/nginx/access.log | grep contest
This simple pipeline shows the power of grep when working with other programs. Often you can make a complicated pipeline and stick grep at the end to cut it down to just the output you need. Here, we use tail with the -f flag to follow the access log in real time, and then use grep to just show the output we care about.
One common trick with grep is to count the number of matches in a file using a pipeline with the wc (wordcount) program using -lthe flag to count the lines of output in grep. For example, to count the number of times a times the word pythonappears in your requirements.txtfile, you can do
grep python requirements.txt | wc -l
Step 4 – Using RegExs with grep
One final way to level up your grep usage is to include regexes (regular expressions). In fact, by default grep interprets all your input as a regex, it’s just that if you’re looking for a string, the regex and the string are identical.
This can be really useful if you’re trying to search for multiple terms. A regex to combine the terms “inside” and “outside” would be inside|outside. It would be great if we could just put that into grep as our search term, however the pipe symbol (|) is a special character in bash, and would break our command. The solution? First, we have to add in quotation marks around our search term. Then we escape it using the backslash character.
grep "inside\|outside" myfile.txt
You can expand this to many more terms using the pipe to orindicate as it does in a typical regular expression. Quotation marks are also useful. Remember, you’re on a command line, so if your search term contains spaces or other special characters (like ! or *) you’ll need to put quotes around it in order for grep to work.
Conclusion
In this article, we looked at 3 increasingly useful and complicated examples of grep in action. You can use grep for everything from looking at single files to filtering down long pipelines. Grep is extremely powerful and extremely fast, and is available on every Unix-based server. It’s a great tool to get to know.
To learn more about grep, you can run man grepand read the documents. They include helpful examples to run as well.
Leave a Reply