Nginx Log Analysis: How I Read Access and Error Logs Without Drowning in Noise

By /

When a site feels slow or unreliable, I still start with logs more often than dashboards. Metrics help, but logs tell me what actually happened to specific requests. They also expose the ugly details dashboards smooth over: repeated client aborts, a sudden flood of 404s from one broken deploy, one upstream timing out while another is fine, or a bot hammering a login path from rotating addresses.

This article is a practical guide to reading and analyzing Nginx access and error logs. I am not trying to turn Nginx into a full observability platform. I am showing the commands and logging patterns I actually use when something is wrong or when I want to understand traffic better. If you are investigating upstream issues, keep my 502 guide, 503 guide, and 504 timeout guide nearby. For caching and balancing context, my proxy cache article and upstream load balancing guide pair well with log work.

The default access log format and what the fields mean

Many distributions ship a combined or main style format that looks roughly like this:

log_format combined '$remote_addr - $remote_user [$time_local] '
                    '"$request" $status $body_bytes_sent '
                    '"$http_referer" "$http_user_agent"';

A typical line looks like this:

203.0.113.10 - - [22/Jul/2026:12:15:30 +0000] "GET /login HTTP/1.1" 200 4821 "https://example.com/" "Mozilla/5.0"

The important fields are:

  • $remote_addr — client IP address as seen by Nginx.
  • $remote_user — authenticated username if HTTP auth is used.
  • $time_local — local timestamp.
  • $request — method, URI, and protocol version.
  • $status — final HTTP response code.
  • $body_bytes_sent — response body size.
  • $http_referer — referrer header.
  • $http_user_agent — client user agent.

That is enough for basic traffic analysis, but not enough for serious upstream troubleshooting.

Why I almost always add timings and upstream fields

When Nginx fronts an application, I want to know how long Nginx spent waiting on the backend and which backend handled the request.

I use a format like this:

log_format upstreamlog '$remote_addr - $host [$time_local] '
                       '"$request" $status $body_bytes_sent '
                       'rt=$request_time '
                       'uct=$upstream_connect_time '
                       'uht=$upstream_header_time '
                       'urt=$upstream_response_time '
                       'ua="$upstream_addr" us="$upstream_status" '
                       'ref="$http_referer" ua_client="$http_user_agent"';

access_log /var/log/nginx/access.log upstreamlog;

The added variables matter a lot:

  • $request_time — total request processing time seen by Nginx.
  • $upstream_connect_time — time to connect to the upstream.
  • $upstream_header_time — time until response headers arrive.
  • $upstream_response_time — total upstream response time.
  • $upstream_addr — upstream server address.
  • $upstream_status — upstream response code.

When users say, “the site is slow,” this format lets me separate slow clients from slow upstreams. That alone saves a lot of guesswork.

Combined vs main format

People often ask about combined versus main. There is nothing magical there. They are just names for log formats defined in config. On one server main may include more fields than combined; on another they may be identical.

Do not assume based on the name. Check the actual log_format definition:

sudo nginx -T | grep -n 'log_format\|access_log'

That command dumps the full running config and shows where the format is defined and used. I use it constantly on inherited systems.

Reading error logs by level

Access logs describe requests. Error logs describe problems and internal notices. Nginx error log levels are, from lowest severity upward:

  • debug
  • info
  • notice
  • warn
  • error
  • crit
  • alert
  • emerg

In normal production use, I keep the error log at warn or error unless I am actively tracing something.

error_log /var/log/nginx/error.log warn;

What I generally expect:

  • warn — suspicious but not fatal conditions.
  • error — request-level failures or backend communication problems.
  • crit and above — serious failures that need attention quickly.

I avoid leaving debug enabled on busy production boxes because it can create a lot of noise fast.

Real-time tailing without losing context

I still use tail -f, but with filters.

sudo tail -f /var/log/nginx/access.log
sudo tail -f /var/log/nginx/error.log

For active incidents, I narrow it down.

sudo tail -f /var/log/nginx/error.log | grep --line-buffered -E 'upstream|timed out|connect\(\) failed|no live upstreams'

If I know the client IP or path involved:

sudo tail -f /var/log/nginx/access.log | grep --line-buffered '203.0.113.10'
sudo tail -f /var/log/nginx/access.log | grep --line-buffered ' /api/'

The --line-buffered flag matters when you want live filtered output instead of seeing lines in bursts.

Common error log messages and how I read them

A few patterns show up over and over.

Upstream timeout

upstream timed out (110: Connection timed out) while reading response header from upstream

This usually means Nginx connected to the backend, but the backend did not return headers before proxy_read_timeout expired. That is often application slowness, backend worker exhaustion, or a database problem behind the app.

Connection refused

connect() failed (111: Connection refused) while connecting to upstream

The upstream process is not listening, crashed, or listening on a different port or socket than the config expects.

No live upstreams

no live upstreams while connecting to upstream

Every server in the upstream group is considered unavailable. That points to a broader outage or overly aggressive failure settings.

These messages become much more actionable if your access log also records $upstream_addr, $upstream_status, and request timings.

Using awk, sort, and uniq for quick answers

I still lean on basic shell pipelines because they work everywhere.

Top client IPs

awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -20

That quickly shows whether a small number of clients are dominating traffic.

Top requested URLs

For a standard format where the request is in double quotes:

awk -F'"' '{print $2}' /var/log/nginx/access.log | awk '{print $2}' | sort | uniq -c | sort -nr | head -20

That extracts the path portion from the request field.

Top 4xx responses

awk '$9 ~ /^4/ {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -20

This assumes a traditional combined-style log where field 9 is the status and field 7 is the URI. On custom formats, adapt the parsing to match reality.

Count by status code

awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -nr

If 499, 502, or 504 suddenly jump, I dig deeper immediately.

Watching for repeated 499 client-closed requests

499 is an Nginx-specific status meaning the client closed the connection before Nginx finished sending the response.

A few 499s are normal. Lots of them are a signal.

awk '$9 == 499 {print $7}' /var/log/nginx/access.log | sort | uniq -c | sort -nr | head -20

High 499 rates often mean one of these:

  • clients are timing out or navigating away because the app is slow
  • a load balancer or CDN in front of Nginx is closing idle requests
  • a health-checking system is impatient
  • an API client has unrealistic timeouts

I do not treat 499 as “the client’s problem” and stop there. A spike in 499s often points to user-visible slowness upstream.

Logging conditionally with map

Some traffic is just noise. Health checks, internal status endpoints, or obvious probes can bloat logs.

I use map for conditional logging.

map $request_uri $loggable {
    default 1;
    /healthz 0;
    /metrics 0;
}

access_log /var/log/nginx/access.log upstreamlog if=$loggable;

That keeps logs focused on useful requests without losing visibility everywhere else.

You can also map by status, user agent, or source address, but I try not to get too clever. Over-filtering makes incident response harder later.

Structured JSON logging

If logs are shipped to Elasticsearch, Loki, or another aggregator, JSON logging makes parsing much less fragile.

log_format json_combined escape=json '{'
    '"time":"$time_iso8601",'
    '"remote_addr":"$remote_addr",'
    '"host":"$host",'
    '"request":"$request",'
    '"status":$status,'
    '"bytes_sent":$body_bytes_sent,'
    '"request_time":$request_time,'
    '"upstream_addr":"$upstream_addr",'
    '"upstream_status":"$upstream_status",'
    '"upstream_response_time":"$upstream_response_time",'
    '"referer":"$http_referer",'
    '"user_agent":"$http_user_agent"'
    '}';

access_log /var/log/nginx/access.json json_combined;

I like JSON when there is a real parser on the other end. I do not switch to JSON just for fashion. For local shell work, plain text remains easier to eyeball.

GoAccess for quick terminal analytics

For ad hoc interactive analysis, GoAccess is still useful.

sudo goaccess /var/log/nginx/access.log --log-format=COMBINED

If you use a custom format, define it explicitly. GoAccess can show top visitors, status codes, requests over time, and bandwidth without writing your own scripts.

I do not treat it as a replacement for understanding the raw log. I use it when I want a fast overview and then confirm specifics manually.

Looking at request and upstream timing together

Once you log request_time and upstream_response_time, you can identify patterns quickly.

If request_time is high and upstream_response_time is also high, the backend is slow.

If request_time is high but upstream time is low, the delay may be client-side transfer speed, large response bodies, or proxy buffering behavior.

If upstream_connect_time is high, you may have network issues, overloaded backends, or connection pool problems.

That distinction matters. It stops you from blaming the application for a slow client download or blaming the network for a slow SQL query.

Reading upstream failures in context

During application incidents I usually combine access and error log views.

sudo grep -E 'upstream timed out|connect\(\) failed|no live upstreams' /var/log/nginx/error.log | tail -n 50
awk '/ ua="/ && / 502 | 504 / {print}' /var/log/nginx/access.log | tail -n 20

If the error log shows timeouts and the access log shows one specific upstream address repeatedly associated with failures, I check that backend first. Poor upstream visibility and poor balancing often make log analysis harder than necessary.

Finding slow requests without fancy tooling

Slow requests are where logs start paying for themselves. If I already have request_time in the log, I do not wait for an APM tool before checking whether the problem is broad or tied to a few endpoints.

On a custom log format where rt= appears as a key, a quick pass looks like this:

awk -F'rt=' 'NF > 1 {split($2,a," "); print a[1], $0}' /var/log/nginx/access.log | sort -nr | head -20

That shows the slowest requests first. Then I usually follow up by grouping just the path field for the slow lines so I can see whether one endpoint dominates. For example, if every slow request is /api/reports/export, that tells me more than a vague complaint about “the website being slow.”

I also look at size and timing together. Large downloads naturally inflate request_time, so a five-second PDF transfer is not the same as a five-second HTML page generation delay. This is where upstream timing variables help again: if urt is tiny and rt is large, the bottleneck is probably not the app itself.

Splitting logs by site and environment

On multi-site systems, one shared access log is better than nothing and worse than it sounds. A noisy site can hide a real issue on a quiet site, and a global top-URLs list may tell you nothing useful about the domain the user actually complained about.

I prefer per-vhost logs when I can manage them cleanly:

server {
    server_name app.example.com;
    access_log /var/log/nginx/app.example.com-access.log upstreamlog;
    error_log  /var/log/nginx/app.example.com-error.log warn;
}

server {
    server_name api.example.com;
    access_log /var/log/nginx/api.example.com-access.log upstreamlog;
    error_log  /var/log/nginx/api.example.com-error.log warn;
}

That makes focused analysis much easier. If I only care about API failures, I do not want to parse image requests from the marketing site at the same time. Per-site logs also make retention choices easier because API traffic often needs different analysis windows from brochureware traffic.

If you cannot split the files, at least log $host and filter on it. That is one reason I nearly always include $host in custom formats.

Real client IPs behind proxies

Log analysis is only as good as the IP address you trust. If Nginx sits behind Cloudflare, a load balancer, or another proxy and you have not configured the real IP module correctly, every request may appear to come from the proxy tier instead of the client. Then your top-IP analysis is basically fiction.

A typical real IP setup looks like this:

set_real_ip_from 10.0.0.0/8;
real_ip_header X-Forwarded-For;
real_ip_recursive on;

The exact trusted addresses depend on your environment, and with public CDNs you should use their published ranges rather than inventing them. The important operational point is this: do not trust X-Forwarded-For from arbitrary sources. Only accept it from proxies you control or explicitly trust.

Once this is set correctly, your log pipelines for top IPs, rate-limit candidates, and abuse sources become much more useful. Pair that with my rate limiting guide when you move from observation to mitigation.

Log rotation and the USR1 signal

Logs grow quickly, so rotation matters.

With logrotate, the important detail is that Nginx must reopen its log files after rotation. The usual safe pattern is to send USR1.

sudo kill -USR1 $(cat /run/nginx.pid)

That tells Nginx to reopen logs without a full restart. Most distro logrotate packages already handle this, but I still verify inherited systems.

If rotation is broken, Nginx can keep writing to deleted file handles and you will wonder why disk usage and file visibility do not match.

I also review my general log rotation policy whenever log growth becomes part of the incident.

Common patterns worth watching

Certain patterns are worth routine attention even when nothing is obviously broken.

A sudden spike in 4xx

This often means a bad deploy, broken links, bot scanning, or changed authentication behavior.

Repeated 301 or 302 loops

Usually a redirect mistake, proxy header issue, or conflicting HTTP-to-HTTPS rule.

499 growth during traffic spikes

Often users giving up on slow responses.

502 and 504 clustered around one path

Usually an expensive endpoint or broken app component.

Large user-agent concentration on one endpoint

Potential scraper or abuse case.

One upstream carrying all the traffic

Potential balancing misconfiguration or failed peers.

A practical troubleshooting workflow

When somebody says, “Nginx is broken,” I use a short sequence.

  1. Validate config and reload history.
  2. Tail the error log.
  3. Check status code distribution in the access log.
  4. Look for timing fields and slow endpoints.
  5. Identify whether one upstream is worse than others.
  6. Correlate with recent deploys, restarts, or certificate changes.

Basic validation commands:

sudo nginx -t
sudo systemctl status nginx --no-pager
sudo tail -n 50 /var/log/nginx/error.log
sudo tail -n 50 /var/log/nginx/access.log

That is still one of the fastest ways to orient yourself.

Security and privacy considerations

Logs are useful, but they can become a liability.

I avoid logging secrets, raw authorization tokens, request bodies with credentials, or anything that creates unnecessary privacy risk. If you move to JSON logs, it becomes even easier to over-collect. Keep only what you actually use.

Also make sure log permissions and rotation policies are correct. A world-readable access log full of client IPs, paths, and referrers is sloppy operational hygiene.

Final thoughts

Good Nginx log analysis is less about fancy tooling and more about logging the right fields, knowing what common errors look like, and having a few reliable shell pipelines ready. I keep access logs rich enough to show timings and upstream behavior, error logs focused enough to matter, and rotation clean enough that the logs remain available when I need them.

When logs are set up well, they shorten incidents. When they are vague, missing, or full of noise, they become one more problem to debug while everything else is already on fire.

Related Posts

Scroll to Top