Text Processing & I/O Redirection
Module 2 · Files, Text & Permissions

Text Processing & I/O Redirection

grep, sed, awk, and the pipe-and-redirect toolkit that turns raw command output into exactly the answer you need.

Viewing File Contents

TaskCommand
Print an entire file to the terminalcat file
Print with line numbers, tabs marked, etc.cat -A file
Concatenate multiple files togethercat file1 file2 > combined
Show the first 10 lineshead file
Show the last 10 linestail file
Follow a growing file (log tailing) livetail -f file
Page through a file, scroll up and downless file
Page through a file, scroll forward onlymore file

Searching Text: grep

grep searches file contents for a pattern:

FlagEffect
grep "text" filePrint lines matching text
grep -i "text" fileMatch case-insensitively
grep -r "text" dir/Search recursively through a directory
grep -v "text" fileInvert the match — print lines that don’t match
grep -c "text" fileCount matching lines instead of printing them
grep -E "regex"Enable extended regular expressions (or use egrep)

Transforming Text: cut, sort, uniq, tr, sed, awk

TaskCommand
Extract a column by delimitercut -d: -f1 /etc/passwd
Arrange lines alphabetically/numericallysort file (add -n for numeric, -r for reverse)
Collapse adjacent duplicate linesuniq file (usually piped after sort)
Translate/delete characterstr 'a-z' 'A-Z' < file
Extract specified lines by numbersed -n '5,10p' file
Find-and-replace across a filesed 's/old/new/g' file
Pattern-driven field processing (a full mini-language)awk '{print $1}' file
Merge lines from multiple files side by sidepaste file1 file2
Show line-by-line differences between two filesdiff file1 file2

sed and awk are both “programs to modify text according to parameters” — the difference is scope: sed is a stream editor built around find/replace and line selection; awk is a full field-processing language built around columns and conditions. If a question needs “print the 3rd column where the 2nd column equals X,” that’s awk. If it needs “replace every instance of X with Y,” that’s sed.


Standard Streams & Redirection

Every process has three open streams:

StreamNumberPurpose
stdin (standard input)0Where a command reads its input from — the keyboard, by default
stdout (standard output)1Where normal output goes — the terminal, by default
stderr (standard error)2Where error messages go — also the terminal, by default, but a separate stream

Redirection operators point a stream somewhere other than the terminal:

OperatorEffect
command > fileRedirect stdout to a file, overwriting it
command >> fileRedirect stdout to a file, appending to it
command < fileRedirect stdin to read from a file instead of the keyboard
command 2> fileRedirect stderr only
command &> fileRedirect both stdout and stderr to the same file
command1 | command2Pipe — send command1’s stdout directly into command2’s stdin
⚠️ Common Pitfall: A Single > Will Destroy Data Silently

command > file truncates file to zero length before writing — there’s no confirmation prompt. Appending with >> is the safe default whenever you’re not certain a file should be replaced outright.

Combining and discarding output:

TaskCommand
Run a command for each line of inputxargs, e.g. find . -name "*.log" | xargs rm
Send output to the screen and copy it to a filecommand | tee file
Discard all output written to itRedirect to /dev/null — command > /dev/null 2>&1

🧠 Knowledge Check

1. You want to see every line in access.log containing 'error', regardless of capitalization. Which command does that?

2. What's the risk of running `long_command > output.txt` on a file that already contains important data?

3. Which command should you reach for to extract and print only the 3rd column of a space-delimited file where the 1st column equals "active"?


What’s Next

Now that you can read and reshape file contents, Lesson 3 covers who’s allowed to do that in the first place — permissions, ownership, and access control lists.