Bash Text Utilities

Small text tools such as sort, uniq, cut, tr, and paste each do one job well. Pipes connect them into powerful chains. A chain turns raw data into a clean report in a single line.

The Assembly Line Picture

  raw data --> [ cut ] --> [ sort ] --> [ uniq -c ] --> report
               pick        order        count
               column                   repeats

Sample File

Create sales.csv with these lines:

Asha,Pune,300
Ravi,Delhi,150
Meena,Pune,200
Karan,Mumbai,150
Asha,Pune,100

cut: Pick Columns

cut -d',' -f1 sales.csv
cut -d',' -f1,3 sales.csv
cut -c1-4 sales.csv

The -d option sets the separator and -f chooses the fields. The -c option picks character positions. The first command prints the names column.

sort: Order Lines

sort names.txt                 # alphabetical
sort -r names.txt              # reverse order
sort -n numbers.txt            # numeric order
sort -u names.txt              # remove duplicates
sort -t',' -k3 -n sales.csv    # sort by the third column

Numeric sorting matters. Plain sorting puts 100 before 20 because it compares text character by character.

uniq: Handle Repeated Lines

The uniq command compares neighboring lines, so sort the data first.

cut -d',' -f1 sales.csv | sort | uniq
cut -d',' -f1 sales.csv | sort | uniq -c

Output of the second command:

      2 Asha
      1 Karan
      1 Meena
      1 Ravi

The option -d shows only repeated lines, and -u shows only lines that appear once.

tr: Translate or Delete Characters

echo "hello" | tr 'a-z' 'A-Z'          # HELLO
echo "a,b,c" | tr ',' ' '              # a b c
echo "hello   world" | tr -s ' '       # squeeze repeated spaces
echo "phone: 98-76" | tr -d '0-9'      # delete digits

paste and join: Combine Data

paste names.txt scores.txt
paste -d',' names.txt scores.txt

The paste command places lines from several files side by side. The join command merges two sorted files on a shared first column.

wc: Count Things

wc -l sales.csv     # lines
wc -w notes.txt     # words
wc -c notes.txt     # bytes

head, tail, and tee

sort -t',' -k3 -nr sales.csv | head -n 2
sort sales.csv | tee sorted.csv | wc -l

The first line shows the two largest sales. The second line saves the sorted data into a file and counts the lines in one step. The tee command splits the stream in two.

Practical Example: Top Cities

cut -d',' -f2 sales.csv | sort | uniq -c | sort -nr

Output:

      3 Pune
      1 Mumbai
      1 Delhi

The chain cuts the city column, groups the names, counts each group, and ranks the counts from high to low.

Practical Example: Word Frequency

cat article.txt | tr 'A-Z' 'a-z' | tr -cs 'a-z' '\n' | sort | uniq -c | sort -nr | head -5

The command prints the five most common words in a text file. The first tr lowercases every letter. The second tr turns every non-letter into a line break, so each word lands on its own line.

Key Takeaways

• cut picks columns and sort orders lines.
• uniq works on sorted data, and uniq -c counts repeats.
• tr swaps or removes single characters.
• paste merges files side by side.
• Pipes connect small tools into one powerful chain.