Use GoAccess 1.5.7 to Generate Report with Multiple Nginx Log Files

About three months ago, I posted a blog about using GoAccess 1.5.5 to generate the report. There is some update, and I decided to make it a new, clean how-to here.

Very first, my setup is a VPS at Vultr.com. (Get your own VPS with $100 credit now.)

  • 1 vCPU
  • 1 GB memory
  • 32 GB NVMe SSD
  • Ubuntu 22.04 Lts
  • Nginx 1.21.6

The configuration of the Nginx is in the HTTP block of /etc/nginx/nginx.conf.

log_format goaccessmain '$remote_addr [$time_local] "$host" "$request" $status $body_bytes_sent "$http_referer" "$http_user_agent" "$http_x_forwarded_for"';

access_log /var/log/nginx/access.log goaccessmain;

Normally, the server will do the log rotation every day, and keep about 14 days log file.

Install GoAccess

The GoAccess in the Ubuntu repo is very old. So I decided to use the goaccess’s own repository as below to install the latest version 1.5.7.

$ wget -O - https://deb.goaccess.io/gnugpg.key | gpg --dearmor | sudo tee /usr/share/keyrings/goaccess.gpg >/dev/null
$ echo "deb [signed-by=/usr/share/keyrings/goaccess.gpg arch=$(dpkg --print-architecture)] https://deb.goaccess.io/ $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/goaccess.list
$ sudo apt-get update
$ sudo apt-get install goaccess

Modify the Config file of GoAccess

GoAccess config file is located at /etc/goaccess/goaccess.conf

sudo nano /etc/goaccess/goaccess.conf

Add the following code or modify the code inside of the goaccess.conf file.

time-format %H:%M:%S

date-format %d/%b/%Y

log-format %h [%d:%t %z] %v "%r" %s %b "%R" "%u" "~h"

geoip-database /var/GeoIP/dbip-city-lite-2022-05.mmdb

db-path /var/goaccess/cache

persist true

restore true

 

At the line of  log-format, %v = the virtual Host. Because I have multiple websites hosted on this server. I would like to know the hits per website.

db-path, is the location to keep the history data of the report. It will be used when add new data.

geoip-database, is the file location. The GeoIP database file. There is the free download on db-ip.com

I downloaded the mmdb format GEOIP2 IP database from https://db-ip.com/db/download/ip-to-city-lite

Decompress it and put it at /var/GeoIP/

Then I created a folder at /var/goaccess/cache

Run GoAccess to generate the report

sudo goaccess /var/log/nginx/access.log -o /home/davidyin/mywebsite.com/report.html[/bashe]

Now, the report included a Virtual Host panel. It shows the server names on the server. The report looks like the below:

GoAccess 1.5.7 Report
GoAccess 1.5.7 Report

 

To generate the report every hour, I add the command below to the Cron job list.

sudo goaccess /var/log/nginx/access.log /var/log/nginx/access.log.1 -o /home/davidyin/mywebsite.com/report.html

Note:
If you have a lot of log files already in the /var/log/nginx/ folder, you can use the following command to make a report for the first run.

sudo zcat /var/log/nginx/access.log.*.gz | sudo goaccess /var/log/nginx/access.log /var/log/nginx/access.log.1 - -o /home/davidyin/mywebsite.com/report.htm

After that, still use my Cron job to do the hourly update.

Leave a Reply

Your email address will not be published. Required fields are marked *