David Yin's Blog

Currently Viewing Posts Tagged web report

Use GoAccess to Generate Report with Multiple Nginx Log Files

March 12, 2019
0 Comments

I have a VPS with Nginx as a web server and also a lot of web sites on it. It is on Digital Ocean.

Here is the VPS basic information.

  • SFO2
  • 1GB Nanode
  • Ubuntu Linux 18.04.2
  • Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz, 1 cores
  • Nginx 1.15.9

All sites use one access.log file. The following code is in the http block of /etc/nginx/nginx.conf file.

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';

access_log /var/log/nginx/access.log main;

All access logs look like below:

Now it is time to show you how to use GoAccess to Generate Report with Multiple Nginx Log files.

Install GoAccess

sudo install goaccess

 

Modify the Config file of GoAccess

GoAccess config file is located at /etc/

sudo nano /etc/goaccess.conf

Add the following code or modify the code inside of the goaccess.conf.

time-format %H:%M:%S

date-format %d/%b/%Y

log-format %h %^[%d:%t %^] "%r" %s %b "%R" "%u"

Run GoAccess to generate the report

I would like to get a static report.

sudo goaccess /var/log/nginx/access.log -o /home/davidyin/mywebsite.com/report.html --log-format=COMBINED

The report looks like below:

Due to the log is not just one file. The log files are generated by date and also compressed.

Two log files are not compressed.

  • access.log
  • access.log.1

The older log files are compressed.

  • access.log.2.gz
  • access.log.3.gz
  • access.log.4.gz
  • access.log.5.gz

I want to generate all the access log files. So the command is changed to the way which can phrase multiple log files.

zcat /var/log/nginx/access.log.*.gz | goaccess /var/log/nginx/access.log /var/log/nginx/access.log.1 - -o /home/davidyin/mywebsite.com/report.html --log-format=COMBINED

The sample command above using mywebsite.com. Please replace it with your own sites.

At the last, I add this command into Cron jobs. Schedule it every hour.

  • Archives