Bash Awk

The awk command reads text line by line and splits each line into columns. It filters rows, picks columns, and calculates totals. Think of awk as a small spreadsheet tool that lives in the terminal.

How Awk Sees a Line

Awk cuts every line into fields at each space. Field names are $1, $2, and so on. The name $0 holds the full line.

  Line:   Asha   Tester   45000   Pune
            |       |       |       |
            v       v       v       v
           $1      $2      $3      $4
           \_______________________/
                      $0

Sample File

Create staff.txt with these lines:

Asha Tester 45000 Pune
Ravi Developer 60000 Delhi
Meena Designer 52000 Pune
Karan Developer 65000 Mumbai

Print Columns

awk '{print $1}' staff.txt
awk '{print $1, $3}' staff.txt

Output of the second command:

Asha 45000
Ravi 60000
Meena 52000
Karan 65000

Awk Program Structure

  awk 'BEGIN { before }   { each line }   END { after }' file
        |                    |                 |
        |                    |                 +--> runs once at the end
        |                    +--------------------> runs for every line
        +-----------------------------------------> runs once at the start

Filter Rows With a Condition

awk '$3 > 50000' staff.txt
awk '$4 == "Pune"' staff.txt
awk '$2 == "Developer" {print $1}' staff.txt

The first line prints every row where the salary is above 50000. The third line prints only the names of developers.

Add a Header and Footer

awk 'BEGIN {print "Name   Salary"} {print $1, $3} END {print "Done"}' staff.txt

Output:

Name   Salary
Asha 45000
Ravi 60000
Meena 52000
Karan 65000
Done

Calculate Totals and Averages

awk '{sum += $3} END {print "Total:", sum}' staff.txt
awk '{sum += $3} END {print "Average:", sum/NR}' staff.txt

Output:

Total: 222000
Average: 55500

The built-in variable NR counts the number of lines read so far. At the end it holds the total line count.

Built-In Variables

Variable	Meaning
NR	Current line number
NF	Number of fields in the current line
FS	Input field separator
OFS	Output field separator
$NF	Last field of the line

Change the Separator

The -F option sets a different separator. CSV files use commas:

echo "Asha,Tester,Pune" | awk -F',' '{print $1 " works in " $3}'

Output:

Asha works in Pune

Read the user names from the system file that separates fields with colons:

awk -F':' '{print $1}' /etc/passwd

Format Output With printf

awk '{printf "%-8s %-10s %6d\n", $1, $2, $3}' staff.txt

Output:

Asha     Tester      45000
Ravi     Developer   60000
Meena    Designer    52000
Karan    Developer   65000

The symbol %-8s pads text to eight characters on the left side. The symbol %6d right-aligns a number in six spaces.

Group Totals With an Array

Awk arrays work with names as keys. This command adds salaries for each city:

awk '{city[$4] += $3} END {for (c in city) print c, city[c]}' staff.txt

Output (order may differ):

Pune 97000
Delhi 60000
Mumbai 65000

Combine Awk With Other Commands

ls -l | awk '{print $9, $5}'
df -h | awk 'NR>1 {print $1, $5}'

The first line prints file names and sizes. The second line prints disk names and usage while skipping the header row.

Key Takeaways

• Awk splits each line into fields named $1, $2, and so on.
• A condition before the braces filters rows.
• BEGIN runs once at the start and END runs once at the finish.
• NR counts lines and NF counts fields.
• -F changes the field separator.
• Arrays calculate totals for each group.