Using GoAccess with Nginx Proxy Manager (NPM) to visuallize logs.

I mainly wanted to make sure and get this docker-compose.yml in here for future use.

Note the last couple of lines. You can set to the log file to digest all logs. However, GoAccess doesn't support any filtering per hostname, so if you want to see analytics for only a particular host name, you want would to just mount the proxy-host-1_access.log for example. npm creates one of these for each proxy host.

services:
  nginx_proxy_manager:
    image: "jc21/nginx-proxy-manager:latest"
    container_name: nginx_proxy_manager
    restart: unless-stopped
    network_mode: "host"
    ports:
      - "80:80"    # HTTP
      - "443:443"  # HTTPS
      - "81:81"    # Admin Panel
    volumes:
      - ./data/npm_data:/data
      - ./data/npm_letsencrypt:/etc/letsencrypt
      - ./data/npm_logs:/var/log/nginx  
    environment:
      DB_SQLITE_FILE: "/data/database.sqlite" # Using SQLite instead of MySQL for simplicity
      INITIAL_ADMIN_EMAIL: admin@example.com
      INITIAL_ADMIN_PASSWORD: changeme


  goaccess:
    image: justsky/goaccess-for-nginxproxymanager:latest
    container_name: goaccess
    restart: unless-stopped
    environment:
        - TZ=America/Denver
        #- SKIP_ARCHIVED_LOGS=False #optional   
        #- BASIC_AUTH=False #optional
        #- BASIC_AUTH_USERNAME=user #optional
        #- BASIC_AUTH_PASSWORD=pass #optional                
    ports:
        - '7880:7880'
    volumes:
        #- ./data/npm_data/logs:/opt/log
        - ./data/npm_data/logs/proxy-host-1_access.log:/opt/log/proxy-host-1_access.log

Note when using a cloud flare tunnel, make sure you add this in the advanced tab. This will ensure you're getting the real ip from users from the cloudflare tunnel.

real_ip_header CF-Connecting-IP;

Running goaccess within the container.

Start a shell inside goaccess

sudo docker exec -it goaccess sh

Then to run goaccess

/goaccess/goaccess /opt/log/proxy-host-1_access.log --log-format=COMBINED

Note that this expects regular nginx formatted logs and does not invoke our .conf configuration. I'm still working on this, if I specify the config file it also tries to start the web server which is already running.

/goaccess/goaccess -f /opt/log/proxy-host-1_access.log --config-file=/goaccess-config/goaccess.conf

/goaccess/goaccess -f /opt/log/proxy-host-1_access.log --config-file=/goaccess-config/goaccess-no-html.conf

That config looks just like the default goaccess.conf file that is created by the container. We can modify it not include the html report. Also can ignore statuses

# goaccess_no_html.conf
time-format %T
date-format %d/%b/%Y
log_format [%d:%t %^] %^ %^ %s - %m %^ %v "%U" [Client %h] [Length %b] [Gzip %^] [Sent-to %^] "%u" "%R"
port 7890
real-time-html false

ignore-status 200
ignore-status 201
ignore-status 202
ignore-status 204
ignore-status 301
ignore-status 302
ignore-status 304
ignore-status 400
ignore-status 403
ignore-status 404
ignore-status 405
ignore-status 408
ignore-status 500
ignore-status 502
ignore-status 503
ignore-status 504

log-file /goaccess-config/archive.log
log-file /goaccess-config/active.log

# Proxy logs
log-file /opt/log/proxy-host-2_access.log
log-file /opt/log/proxy-host-4_access.log
log-file /opt/log/proxy-host-3_access.log
log-file /opt/log/proxy-host-1_access.log
log-file /opt/log/proxy-host-5_access.log
log-file /goaccess-logs/archives/proxy-host-3_access.log.1
log-file /goaccess-logs/archives/proxy-host-1_access.log.1
log-file /goaccess-logs/archives/proxy-host-4_access.log.1
log-file /goaccess-logs/archives/proxy-host-2_access.log.1

geoip-database /goaccess-config/GeoLite2-City.mmdb
geoip-database /goaccess-config/GeoLite2-ASN.mmdb
geoip-database /goaccess-config/GeoLite2-Country.mmdb

You'll just want to include the file as a volume that you can reference when you're in a shell within the container

  goaccess:
    image: justsky/goaccess-for-nginxproxymanager:latest
    container_name: goaccess
    restart: unless-stopped
    environment:
        - TZ=America/New_York
        #- SKIP_ARCHIVED_LOGS=False #optional   
        #- BASIC_AUTH=False #optional
        #- BASIC_AUTH_USERNAME=user #optional
        #- BASIC_AUTH_PASSWORD=pass #optional                
    ports:
        - '7880:7880'
    volumes:
        #- ./data/npm_data/logs:/opt/log
        - ./data/npm_data/logs:/opt/log
        - ./goaccess-config/goaccess-no-html.conf:/goaccess-config/goaccess-no-html.conf

You can also just run a command and not worry about referencing the custom config

/goaccess/goaccess /goaccess-config/archive.log \
/goaccess-config/active.log \
/opt/log/proxy-host-2_access.log \
/opt/log/proxy-host-4_access.log \
/opt/log/proxy-host-3_access.log \
/opt/log/proxy-host-1_access.log \
/opt/log/proxy-host-5_access.log \
/goaccess-logs/archives/proxy-host-3_access.log.1 \
/goaccess-logs/archives/proxy-host-1_access.log.1 \
/goaccess-logs/archives/proxy-host-4_access.log.1 \
/goaccess-logs/archives/proxy-host-2_access.log.1 \
--log-format='[%d:%t %^] %^ %^ %s - %m %^ %v "%U" [Client %h] [Length %b] [Gzip %^] [Sent-to %^] "%u" "%R"' \
--time-format=%T \
--date-format=%d/%b/%Y \
--port=7890 \
# --real-time-html=false \
--ignore-status=200 \
--ignore-status=201 \
--ignore-status=202 \
--ignore-status=204 \
--ignore-status=301 \
--ignore-status=302 \
--ignore-status=304 \
--ignore-status=400 \
--ignore-status=403 \
--ignore-status=404 \
--ignore-status=405 \
--ignore-status=408 \
--ignore-status=500 \
--ignore-status=502 \
--ignore-status=503 \
--ignore-status=504 \
--geoip-database=/goaccess-config/GeoLite2-City.mmdb \
--geoip-database=/goaccess-config/GeoLite2-ASN.mmdb \
--geoip-database=/goaccess-config/GeoLite2-Country.mmdb

Comments