Day 32intermediateMar 4, 2026

Extract and Transform Text with awk

awk lets you slice, filter, and transform text data in one line — no scripts or spreadsheets needed.

linuxtext-processingproductivity

What

awk is a powerful text processing tool that works line-by-line, splitting each line into fields. It's perfect for extracting columns from command output, log files, or CSV data. By default, awk splits on whitespace, but you can set any delimiter with the -F flag.

Why It Matters

DevOps engineers constantly deal with text output — logs, command results, config files, CSVs. Tools like cut are limited, and writing Python scripts for simple extraction is overkill. awk gives you column extraction, filtering, and transformation in a single command.

Example

# Extract disk name and usage percentage from df output
df -h | awk '{print $1, $5}'

# Extract the second column from a CSV file
awk -F',' '{print $2}' data.csv

# Print lines where the third field is greater than 80
awk '$3 > 80 {print $1, $3}' report.txt

# Sum values in the second column
awk '{sum += $2} END {print "Total:", sum}' numbers.txt

# Print specific fields with custom formatting
ps aux | awk '{printf "%-10s %s\n", $1, $11}'

bash

Common Mistake

Forgetting to set the field separator with -F when processing non-space-delimited data. If your CSV uses commas, plain awk treats each entire line as field $1 because it defaults to splitting on whitespace.

Quick Fix

Always specify -F for non-whitespace delimiters: awk -F',' for CSV, awk -F':' for /etc/passwd, awk -F'\t' for TSV files. Check your delimiter first with head -1 yourfile.