DEV Community

Cover image for Five log analyzers, one nginx log, five visitor counts
Marko Savran
Marko Savran

Posted on Originally published at opensource-analytics.com

Five log analyzers, one nginx log, five visitor counts

I generated a month of nginx access log myself, so I knew the right answer before I started. The log has 43,231 people in it. They made 60,706 visits and 109,351 pageviews. Out of 502,103 lines, 118,216 come from bots.

Then I ran the same file through GoAccess, AWStats, Webalizer (the Stone Steps branch), Matomo's log importer and lnav. I expected small differences. The "unique visitors" numbers came back like this:

Tool Unique visitors
AWStats 49,616
Webalizer 50,707
Matomo 66,674
GoAccess with --ignore-crawlers 67,863
GoAccess, default 105,885

The true figure is 43,231, and none of the five is close to it. GoAccess in its default mode says 105,885, which is more than twice the real number.

Five visitor counts for the same 43,231 people

I blamed the bots first

My first guess was that bots inflated the counts. So I filtered them out and counted again. The numbers still ranged from 49,576 to 60,209. Bots explain part of it at most.

Each tool answers a different question

Once I put the outputs side by side, the cause was plain. The tools use three different definitions of a visitor:

  • AWStats and Webalizer count one visitor per address per month.
  • GoAccess counts a visitor as an address plus a date plus a user agent.
  • Matomo, run on logs without cookies, treats every visit as a new visitor. It reported 66,675 visits and 66,674 visitors.

One reader on a phone at breakfast and a laptop in the evening is one person. Under these rules that reader can be one visitor, two, or a new one on every visit.

One reader, three counting rules

Visits and pageviews held up

This part surprised me in a good way. Every tool landed within 1% of the true visit count. Pageviews were either exact or 1.4% high. If you compare tools, or compare one month with the next, those two columns are safe to use.

My conclusion is that "unique visitors" from a log file is a setting you chose, and nobody measured anything. Compare visits and pageviews instead.

The scraper nobody caught

I put one scraper in the log that claims to be Chrome 141. It sent 7,285 requests. No bot list in any of the five tools flagged it, and it pushed pageviews up by 6.7%. A list of known user agents only knows what someone already reported, so a fresh disguise passes.

Speed on this log

Timings on the same file:

  • Webalizer: 3.4 s
  • lnav: 4.5 s
  • GoAccess: 6.1 s
  • AWStats: 19.5 s
  • Matomo: 265 s

Matomo took 265 s, far behind the rest. If all you need is a quick answer from a log, the other four finish within 20 seconds.

What I'd do with this

I'd pick the tool by what I want to see and ignore its visitor number. If someone asks me for a headcount from nginx logs, I now say it can't be read from the log alone, and I show visits instead.

Have you ever compared your own analyzer's visitor count with something you could verify? I'd like to hear how far off yours was.

The full comparison, with commands and versions for all five tools, is on opensource-analytics.com.

Top comments (0)