awk command: columns, patterns, sums and real log examples

Short answer: the awk command splits every line into fields, so awk '{print $1}' file prints the first field of every line, where a field is anything between runs of spaces or tabs. $NF is the last field, -F: changes the separator, and a pattern in front of the braces picks which lines the action runs on: awk '$9 == 404 {print $7}' access.log prints the path of every request that got a 404. Add a BEGIN or END block and the same one-liner can sum a column or count values.

awk '{print $1}' access.log                       # first column
awk -F: '{print $1, $NF}' /etc/passwd             # user name and login shell
awk '{sum += $10} END {print sum}' access.log     # add up a column

Every awk example below was run on Debian 12 with both GNU Awk 5.2.1 (gawk) and mawk 1.3.4, which is the awk Debian and Ubuntu install by default; the log examples ran against a 16-line sample nginx access log. Where the two awks behave differently, the section says so.

awk cheat sheet

How awk reads input: records, fields and $0

awk reads its input one line (a record) at a time, splits it into fields and runs your program against it. $0 is the whole line, $1 to $NF are the fields, NF is how many fields this line has and NR is the line number so far:

printf 'one two three\nfour five\nsix\n' | awk '{print NR": "NF" fields, last is "$NF}'

1: 3 fields, last is three
2: 2 fields, last is five
3: 1 fields, last is six

A program is a list of pattern { action } pairs. Leave out the pattern and the action runs on every line; leave out the action and awk prints every line the pattern matches. That is why awk '/error/' behaves like a basic grep, and why awk 'NF' drops blank lines: a blank line has zero fields, and zero counts as false.

Printing columns: $1, $NF and the comma

By default a field separator is any run of spaces and tabs, and leading blanks are ignored. That makes awk the right tool for aligned command output, where the columns are padded with a varying number of spaces. Two details catch people out:

echo 'alpha beta' | awk '{print $1 $2; print $1, $2}'

alphabeta
alpha beta

• The comma matters. print $1, $2 puts the output field separator (a space by default) between the values; print $1 $2 glues them together with nothing in between.
• $NF is not NF. NF is the number of fields; $NF is the field at that position, so the last one. $(NF-1) needs the parentheses: $NF-1 takes the last field and subtracts one from it.

-F: changing the field separator

awk -F: '{print $1, $NF}' /etc/passwd | head -3

root /bin/bash
daemon /usr/sbin/nologin
bin /usr/sbin/nologin

-F takes a single character or a regular expression. A few forms worth knowing:

awk -F'\t' '{print $2}' data.tsv          # tab-separated
awk -F'[;,]' '{print $2}' mixed.txt        # either ; or ,
awk -F' *= *' '{print $2}' app.conf        # key = value, spaces optional
awk -F: '$3 >= 1000 && $3 < 65534 {print $1}' /etc/passwd   # human accounts

The last line is the classic way to list the real users on a Linux system. To change what goes between fields on output, set OFS. awk only rebuilds a line with the new separator once a field has been assigned, which is what the odd-looking $1=$1 is for:

echo 'alpha beta gamma' | awk 'BEGIN {OFS=","} {print $1, $2; $1=$1; print}'

alpha,beta
alpha,beta,gamma

Patterns: choosing which lines to act on

Comparisons are numeric when both sides look like numbers and string comparisons otherwise, which is why $9 >= 500 finds server errors and $1 == "198.51.100.23" finds one address.

BEGIN and END: sums, counts and averages

BEGIN runs once before the first line is read (set a separator, print a header), END runs once after the last (print the totals). Variables start at zero, so a sum needs no setup. access.log here is the sample log, in nginx’s default format (fields in the nginx section below); $10 is the response size:

awk '{sum += $10} END {printf "%d requests, %.0f bytes, %.1f KB average\n", NR, sum, sum/NR/1024}' access.log

16 requests, 52463953 bytes, 3202.1 KB average

Inside END, NR still holds the number of the last line, so it doubles as the line count. On empty input NR is zero and the division fails: gawk stops with fatal: division by zero attempted, mawk quietly prints inf. Guard it with END {if (NR) print sum/NR} in a script.

Counting with arrays

awk arrays take strings as keys, so c[$1]++ counts how often each value of the first field appears, and an END loop prints the result:

awk '{c[$9]++} END {for (s in c) print s, c[s]}' access.log | sort

200 7
401 1
404 6
502 2

The order of a for (k in c) loop is not defined, so pipe the result into sort when the order matters. The same idea gives the shortest way to remove duplicate lines without sorting first: awk '!seen[$0]++' prints a line only the first time it appears.

printf: aligned, formatted output

awk '{kb[$1] += $10} END {for (ip in kb) printf "%-15s %10.1f KB\n", ip, kb[ip]/1024}' access.log | sort -k2 -rn

203.0.113.7        51230.5 KB
203.0.113.88           2.5 KB
198.51.100.23          0.9 KB
192.0.2.41             0.5 KB

printf works like the C function: %s for a string, %d for an integer, %.1f for one decimal place, %-15s to left-align in 15 columns, %% for a literal percent sign. Unlike print it adds no newline, so end the format with \n.

awk on an nginx access log

nginx’s default combined log format splits on spaces into fields that stay put on a normal request:

# top client IPs
sudo awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# requests per status code
sudo awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn

# which paths return server errors
sudo awk '$9 >= 500 {print $9, $7}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# one minute of traffic (string comparison on the timestamp, same day only)
sudo awk '$4 >= "[21/Oct/2026:09:15" && $4 < "[21/Oct/2026:09:16"' /var/log/nginx/access.log

# share of failed requests
sudo awk '$9 ~ /^5/ {e++} END {printf "%d of %d requests failed (%.1f%%)\n", e, NR, 100*e/NR}' /var/log/nginx/access.log

      6 198.51.100.23
      4 203.0.113.7
      3 203.0.113.88
      3 192.0.2.41

      7 200
      6 404
      2 502
      1 401

These are the same counts the 404 and high CPU guides use to find a scanner or a broken link. One trap: a malformed request, such as a TLS handshake sent to port 80 or an empty "-" request line, has fewer fields before the status, so $9 reads "-" for those lines. When the counts must be exact, split on the quote character instead; the status is then always the first word of $3, and the user agent is $6:

sudo awk -F'"' '{split($3, s, " "); print s[1]}' /var/log/nginx/access.log | sort | uniq -c | sort -rn
sudo awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head    # user agents

Quoting and shell variables: -v

Always wrap the awk program in single quotes. Inside double quotes the shell replaces $1 with its own first argument before awk ever sees it, usually an empty string, so the program becomes {print } and prints whole lines without any error:

echo 'alpha beta gamma' | awk "{print $1}"     # prints: alpha beta gamma
echo 'alpha beta gamma' | awk '{print $1}'     # prints: alpha

To get a shell value into the program, pass it with -v rather than splicing it into the quotes:

code=404
awk -v code="$code" '$9 == code {print $7}' access.log

Which awk do you have? mawk vs gawk

On Debian and Ubuntu, awk is mawk unless someone installed gawk: on Debian 12, dpkg-query lists mawk as a required package and gawk as optional. Rocky Linux, AlmaLinux and CentOS Stream use gawk, and macOS ships its own BSD awk. awk -W version answers on both mawk and gawk (awk --version fails on mawk with not an option: --version). The differences that matter in practice:

The large-number row is the one that bites on a server: with mawk 1.3.4 (the Debian 12 version), summing the bytes of a busy log with print gives scientific notation, and printf "%d" is worse: it caps the value at 2147483647. Use printf "%.0f\n", which prints the full number on both, or install gawk with sudo apt install gawk; it then takes over the awk name through the alternatives system. For a case-insensitive match that works everywhere, write tolower($0) ~ /error/.

Common awk errors and what they mean

• awk: line 2: missing } near end of file (mawk) or unexpected newline or end of string (gawk): a brace or a quote is not closed. Count them; a single quote inside the program ends the shell string early.
• awk: line 1: extra '}' followed by syntax error at or near end of line (mawk), or cmd. line:1: ... syntax error with a caret under the spot (gawk): one closing brace too many. A syntax error on its own can also be gawk-only syntax run under mawk.
• function gensub never defined: a gawk extension under mawk. Install gawk or rewrite with sub() or gsub().
• Whole lines instead of one column. The program is in double quotes and the shell ate the $1. Switch to single quotes.
• Columns shift on some lines. A field contains spaces (a user agent, a quoted name) or a request line is malformed. Pick a separator that matches the format, such as -F'"' for nginx logs.

awk vs sed vs cut vs grep

The cut limit shows up as soon as the input is aligned for humans. Given ps-style output where the columns are padded with runs of spaces, cut -d' ' -f2 prints empty lines, because every single space starts a new, empty field; awk '{print $2}' prints the PID column. For editing a config file from a script, sed is still the better fit, and the Linux commands cheat sheet lists where the other everyday text tools fit in.

Frequently asked