Leeladharan Achar - alias - leelu ~ blogging...hola mi amigos..'s Blog

Posted Oct. 9, 2017   66 views

Processing Nginx Logs

Below are list of few commands that can be used in linux to process nginx logs

Processing log file group by HTTP Status Code

awk '{print $11}' access.log | sort | uniq -c | sort -rn

The $11 is the field in log line, where each field is delimited by a delimiter like (space). In my log files $11 holds the HTTP Status Code

Getting All URL's in log file of specific Status Code, below example 502

awk '($11 ~ /502/)' access.log | awk '{print $4, $9}' | sort | uniq -c | sort -rn

Here $4 is time and $9 is request_uri, this prints unique urls count with 502 status including query strings.

To group by request_uri's excluding query string params below is the command

awk '($11 ~ /502/)' access.log | awk '{print $9}' | sed '/^$/d' | sed 's/\?.*//g' | sort | uniq -c | sort -rn 

Most Requested URL

awk -F\" '{print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -r

Most Requested URL containing xyz

awk -F\" '($2 ~ "xyz"){print $2}' access.log | awk '{print $2}' | sort | uniq -c | sort -r