mirror of
https://github.com/praktimarc/kst4contest.git
synced 2026-09-12 04:05:34 +02:00
Added user counter and analytics to the kst4contest page
This commit is contained in:
Marc Froehlich
@@ -0,0 +1,457 @@
|
||||
# Server-side website statistics
|
||||
|
||||
This directory contains installation examples for the confirmed Ubuntu 24.04
|
||||
server baseline and privacy-conscious traffic statistics. Nothing here
|
||||
installs or activates the production service automatically.
|
||||
|
||||
The design has two separate outputs:
|
||||
|
||||
- private static GoAccess HTML and JSON reports for each registered project
|
||||
subdomain and for all registered project subdomains combined;
|
||||
- a small public `visitor-count.json` file for sites which explicitly enable
|
||||
the counter.
|
||||
|
||||
The public number is the sum of daily approximate unique visits since the
|
||||
configured activation date. GoAccess treats requests with the same IP address,
|
||||
date and user agent as one visit. That is useful for a rough trend. It is not a
|
||||
count of people.
|
||||
|
||||
## Data flow
|
||||
|
||||
```text
|
||||
eligible page request
|
||||
|
|
||||
v
|
||||
dedicated Nginx analytics log (14 days)
|
||||
|
|
||||
v
|
||||
GoAccess with IP anonymisation
|
||||
|
|
||||
+--> private per-site and combined HTML/JSON reports (395 days)
|
||||
|
|
||||
v
|
||||
durable daily counter state --> public visitor-count.json
|
||||
```
|
||||
|
||||
Nginx writes a dedicated, reduced log. For each site, the generator gives the
|
||||
existing, uncompressed `.1` rotation and then the current log directly to
|
||||
GoAccess. GoAccess uses its persistent state to skip entries already
|
||||
processed. The generator does not use an incremental shell pipeline or
|
||||
decompress older rotations. Reports, database updates and public counter data
|
||||
are first prepared in a staging directory. GoAccess output is validated
|
||||
before any published report changes. The last valid report therefore survives
|
||||
a failed GoAccess run.
|
||||
|
||||
The counter state is deliberately separate from the 395-day report database.
|
||||
Each day is replaced with the latest value reported by GoAccess instead of
|
||||
being added again. This makes repeated runs idempotent. Values older than 395
|
||||
days remain in the counter state and continue to contribute to the public
|
||||
total.
|
||||
|
||||
## Files
|
||||
|
||||
- `generate-reports.js` validates configuration and state, runs GoAccess and
|
||||
publishes outputs atomically.
|
||||
- `sites.example.json` is the registry template.
|
||||
- `goaccess.conf.template` is rendered per report with a private database path
|
||||
and the configured GeoIP2 Country database.
|
||||
- `nginx/` contains the reduced log format, request filters, public endpoint
|
||||
and protected report-vhost examples.
|
||||
- `systemd/` contains a hardened oneshot service and hourly timer.
|
||||
- `logrotate/` retains 14 daily analytics-log rotations.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Node.js 18.19.1 or newer;
|
||||
- GoAccess with GeoIP2/MMDB support;
|
||||
- one GeoIP2 **Country** database, not a City database;
|
||||
- Nginx;
|
||||
- an unprivileged service account, shown as `hamradio-analytics` in the
|
||||
examples.
|
||||
|
||||
No npm package is required by the generator. GoAccess is the only external
|
||||
program it starts.
|
||||
|
||||
The production compatibility baseline is GoAccess 1.8.1 built with
|
||||
`--enable-geoip=mmdb` and `--with-openssl`, but without `--with-zlib`. Zlib is
|
||||
not required for the regular operating mode because it reads only
|
||||
uncompressed files. The configuration check reports the detected build
|
||||
features and explicitly accepts this combination.
|
||||
|
||||
The confirmed production baseline is Node.js 18.19.1. The generator and tests
|
||||
must remain compatible with it; upgrading Node.js is not part of this setup.
|
||||
|
||||
The service account needs read access to the dedicated analytics logs and
|
||||
`/var/lib/GeoIP/GeoLite2-Country.mmdb`. It needs write access only to its state,
|
||||
report and public-output directories. Access is group-based. The setup does
|
||||
not depend on ACLs or `setfacl`. Nginx receives read access to reports and the
|
||||
public counter through the `www-data` group, but no write access.
|
||||
|
||||
## Installation and permissions
|
||||
|
||||
Do not trust Unix modes stored in a ZIP created on Windows. Install every
|
||||
script, configuration and unit with an explicit owner, group and mode. The
|
||||
following commands assume that the package has been unpacked into the current
|
||||
directory. Create the dedicated, unprivileged service account once:
|
||||
|
||||
```sh
|
||||
if ! getent passwd hamradio-analytics >/dev/null; then
|
||||
sudo useradd --system --user-group --home-dir /nonexistent --no-create-home \
|
||||
--shell /usr/sbin/nologin hamradio-analytics
|
||||
fi
|
||||
```
|
||||
|
||||
Then install the generator, configuration and units:
|
||||
|
||||
```sh
|
||||
sudo install -d -o root -g hamradio-analytics -m 0750 \
|
||||
/opt/hamradioonline-analytics /etc/hamradioonline-analytics
|
||||
sudo install -o root -g hamradio-analytics -m 0750 \
|
||||
website/ops/analytics/generate-reports.js \
|
||||
/opt/hamradioonline-analytics/generate-reports.js
|
||||
sudo install -o root -g hamradio-analytics -m 0640 \
|
||||
website/ops/analytics/goaccess.conf.template \
|
||||
/etc/hamradioonline-analytics/goaccess.conf.template
|
||||
sudo install -o root -g hamradio-analytics -m 0640 \
|
||||
website/ops/analytics/sites.example.json \
|
||||
/etc/hamradioonline-analytics/sites.json
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/systemd/hamradioonline-analytics.service.example \
|
||||
/etc/systemd/system/hamradioonline-analytics.service
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/systemd/hamradioonline-analytics.timer.example \
|
||||
/etc/systemd/system/hamradioonline-analytics.timer
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/logrotate/hamradioonline-analytics.example \
|
||||
/etc/logrotate.d/hamradioonline-analytics
|
||||
```
|
||||
|
||||
Create the writable tree deliberately. The state root is traversable but not
|
||||
readable by Nginx. Only the report and public branches use the `www-data`
|
||||
group and the set-group-ID bit:
|
||||
|
||||
```sh
|
||||
sudo install -d -o hamradio-analytics -g hamradio-analytics -m 0711 \
|
||||
/var/lib/hamradioonline-analytics
|
||||
sudo install -d -o hamradio-analytics -g hamradio-analytics -m 0750 \
|
||||
/var/lib/hamradioonline-analytics/db
|
||||
sudo install -d -o hamradio-analytics -g www-data -m 2750 \
|
||||
/var/lib/hamradioonline-analytics/reports \
|
||||
/var/lib/hamradioonline-analytics/reports/combined \
|
||||
/var/lib/hamradioonline-analytics/reports/kst4contest \
|
||||
/var/lib/hamradioonline-analytics/public \
|
||||
/var/lib/hamradioonline-analytics/public/kst4contest
|
||||
```
|
||||
|
||||
Generated HTML and JSON reports use mode `0640`. The public
|
||||
`visitor-count.json` uses `0644`. Private GoAccess databases and
|
||||
`public-counter-state.json` remain owned by `hamradio-analytics` and unreadable
|
||||
by Nginx. The service unit uses `StateDirectoryMode=0711` to retain this
|
||||
boundary after systemd has prepared the state directory.
|
||||
|
||||
Create the analytics log only when it does not already exist. Running
|
||||
`install /dev/null` unconditionally would empty an existing log:
|
||||
|
||||
```sh
|
||||
if [ ! -e /var/log/nginx/kst4contest-analytics.log ]; then
|
||||
sudo install -o www-data -g hamradio-analytics -m 0640 /dev/null \
|
||||
/var/log/nginx/kst4contest-analytics.log
|
||||
fi
|
||||
sudo stat -c '%U:%G %a %n' /var/log/nginx/kst4contest-analytics.log
|
||||
```
|
||||
|
||||
The resulting log owner and mode must be
|
||||
`www-data:hamradio-analytics 640`. Logrotate preserves that ownership. The
|
||||
generator's configuration check fails clearly if required output directories
|
||||
are missing or if the executing user cannot read an analytics log or the
|
||||
Country database.
|
||||
|
||||
## Registry
|
||||
|
||||
Copy `sites.example.json` outside the checkout and adjust it to the private
|
||||
server layout. Each site entry contains:
|
||||
|
||||
- a stable `id` used for state and the GoAccess database;
|
||||
- its exact `hostname`;
|
||||
- the current, uncompressed `analyticsLog` path;
|
||||
- the `activatedOn` date used by the public counter;
|
||||
- a `publicCounter` switch;
|
||||
- the private `reportOutputDirectory`;
|
||||
- a `publicJsonPath` when the public counter is enabled.
|
||||
|
||||
The top-level `combined.reportOutputDirectory` receives the combined report.
|
||||
Only registered sites are included. The generator rejects
|
||||
`stats.hamradioonline.de`, so the report host cannot accidentally become part
|
||||
of the project statistics.
|
||||
|
||||
To add another project subdomain later, add one registry entry and one matching
|
||||
dedicated `access_log` line to its Nginx server block. Do not enable a public
|
||||
counter unless that site should publish one.
|
||||
|
||||
Treat `activatedOn` as persistent data. Once counting has started, changing it
|
||||
would change the meaning of the total. The generator refuses to combine a new
|
||||
activation date with existing counter state.
|
||||
|
||||
The generator derives the optional `.1` path from `analyticsLog`. It is valid
|
||||
for `.1` not to exist before the first rotation. Do not enter a rotation or a
|
||||
compressed `.gz` file in the registry.
|
||||
|
||||
## Nginx logging
|
||||
|
||||
Install the log-format and filter maps from `nginx/` in the `http` context.
|
||||
Then add a dedicated analytics `access_log` to every registered project server
|
||||
block. Keep the existing operational access log unless its replacement has
|
||||
been reviewed separately. If the operational log is inherited from the
|
||||
`http` context, repeat its directive in the server block before adding the
|
||||
analytics log; an `access_log` at a lower level changes inheritance.
|
||||
|
||||
Install Nginx snippets explicitly as `root:root` with mode `0644`; do not copy
|
||||
the modes from the ZIP. Nginx `map` exact-string keys use the path itself, for
|
||||
example `/visitor-count.json`, without the location-modifier prefix `=`.
|
||||
|
||||
```sh
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/nginx/analytics-filters.conf.example \
|
||||
/etc/nginx/snippets/hamradioonline-analytics-filters.conf
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/nginx/analytics-log.conf.example \
|
||||
/etc/nginx/conf.d/hamradioonline-analytics-log.conf
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/nginx/public-counter.conf.example \
|
||||
/etc/nginx/snippets/kst4contest-public-counter.conf
|
||||
```
|
||||
|
||||
Prepare the public-counter include without enabling it in the active site yet.
|
||||
Likewise, keep the statistics vhost disabled until the certificate bootstrap
|
||||
step below.
|
||||
|
||||
The analytics format contains only:
|
||||
|
||||
- server name;
|
||||
- client IP address;
|
||||
- timestamp;
|
||||
- method;
|
||||
- normalized path without query string;
|
||||
- protocol;
|
||||
- status;
|
||||
- transferred body size;
|
||||
- user agent.
|
||||
|
||||
It does not contain a referrer, query string or authenticated user name. The
|
||||
filter accepts only eligible page `GET` requests. It excludes the update feed,
|
||||
public counter, sitemap, robots file, favicons, CSS, JavaScript, images, fonts,
|
||||
source maps, manual assets and the listed monitoring paths. Known crawler user
|
||||
agents are rejected before logging. GoAccess applies its own crawler list as a
|
||||
second layer and treats unknown browsers or operating systems as crawlers.
|
||||
|
||||
Review the monitoring-path list against the real server before activation.
|
||||
When a new health endpoint or asset family is added, update the filter first.
|
||||
|
||||
Test the complete Nginx configuration before reloading it:
|
||||
|
||||
```sh
|
||||
sudo nginx -t
|
||||
```
|
||||
|
||||
## GoAccess reports
|
||||
|
||||
The template enables IP anonymisation before persistent aggregation, ignores
|
||||
crawlers, keeps 395 days, and uses a separate persistent database for every
|
||||
site and the combined report. It leaves only the panels needed here: visits by
|
||||
day, requested pages, countries, HTTP status codes and virtual hosts. Host,
|
||||
remote-user, referrer, keyphrase, operating-system, browser and other detailed
|
||||
panels are disabled.
|
||||
|
||||
The Country database is provided through the registry at
|
||||
`/var/lib/GeoIP/GeoLite2-Country.mmdb`. A file whose name contains `City` is
|
||||
rejected. Do not replace it with a City database merely because one happens to
|
||||
be available.
|
||||
|
||||
The generator supplies `--persist`, conditionally supplies `--restore`, and
|
||||
uses an isolated `--db-path` through the rendered template. It passes the
|
||||
uncompressed `.1` rotation, when present, and then the current log as direct
|
||||
GoAccess arguments. This chronological order also covers entries appended
|
||||
shortly before rotation. GoAccess tracks the processed files in its persistent
|
||||
state and processes only new entries on later runs. The first successful run
|
||||
creates each database. Later runs copy the last valid database into staging,
|
||||
restore it and persist the updated result only after all reports have
|
||||
succeeded.
|
||||
|
||||
Logrotate must use `delaycompress`, as shown in the example. This leaves `.1`
|
||||
uncompressed for one rotation cycle. Older `.gz` files are not part of the
|
||||
regular hourly run, and importing them is a separate maintenance task outside
|
||||
this repository workflow. Do not add an unstable decompression pipeline to
|
||||
the timer service.
|
||||
|
||||
If the generator is unavailable for longer than the uncompressed rotation
|
||||
window, the regular run cannot recover entries found only in older `.gz`
|
||||
files. Preserve those files under the raw-log retention policy and plan any
|
||||
necessary historical import separately before resuming normal processing.
|
||||
|
||||
## Checking and running
|
||||
|
||||
Validate paths, registry values, the template contract and GoAccess
|
||||
availability without producing reports:
|
||||
|
||||
```sh
|
||||
sudo -u hamradio-analytics /usr/bin/node \
|
||||
/opt/hamradioonline-analytics/generate-reports.js \
|
||||
--registry /etc/hamradioonline-analytics/sites.json \
|
||||
--config-template /etc/hamradioonline-analytics/goaccess.conf.template \
|
||||
--check
|
||||
```
|
||||
|
||||
The check prints the detected GoAccess version and whether GeoIP2/MMDB,
|
||||
OpenSSL and Zlib build options are present. Missing GeoIP2/MMDB support is a
|
||||
configuration error with exit code 2. OpenSSL remains informational. Missing
|
||||
Zlib is supported for this operating mode and does not make the check fail.
|
||||
The same message explains that only the current log and optional uncompressed
|
||||
`.1` are processed and that older `.gz` files are not imported.
|
||||
|
||||
Exercise the complete GoAccess and output-validation path without changing
|
||||
published reports, databases or counter state:
|
||||
|
||||
```sh
|
||||
sudo -u hamradio-analytics /usr/bin/node \
|
||||
/opt/hamradioonline-analytics/generate-reports.js \
|
||||
--registry /etc/hamradioonline-analytics/sites.json \
|
||||
--config-template /etc/hamradioonline-analytics/goaccess.conf.template \
|
||||
--dry-run
|
||||
```
|
||||
|
||||
Run without either flag to publish. A lock prevents concurrent production
|
||||
runs. A dry-run uses a temporary working directory and deliberately neither
|
||||
needs nor creates the production lock below `/run`. Configuration errors use
|
||||
exit code 2, an active production lock uses exit code 3, and generation or
|
||||
publication errors use exit code 1.
|
||||
|
||||
Before the first production run, seed any earlier daily values which must be
|
||||
preserved into `public-counter-state.json`. There is no honest way to recreate
|
||||
history which is no longer present in the raw logs. Back up this state file: it
|
||||
is the durable source for public totals older than the detailed retention
|
||||
window.
|
||||
|
||||
## Scheduling and report access
|
||||
|
||||
Install the systemd files as local units after adapting paths and permissions.
|
||||
The timer runs hourly, catches up after downtime and adds a small random delay.
|
||||
The service has no network access and only the documented read/write paths.
|
||||
If the installed GoAccess build unexpectedly requires network access, find the
|
||||
reason before weakening that restriction; local log processing and a local
|
||||
Country database do not require it.
|
||||
|
||||
The statistics vhost serves static files over HTTPS and protects the complete
|
||||
host with HTTP Basic Authentication. This includes `/`, its redirect to
|
||||
`/combined/`, and every individual report. Store the password file outside
|
||||
this repository. Prepare it so only root and the Nginx group can access it:
|
||||
|
||||
```sh
|
||||
sudo install -d -o root -g www-data -m 0750 /etc/nginx/htpasswd
|
||||
if [ -e /etc/nginx/htpasswd/hamradioonline-analytics ]; then
|
||||
sudo htpasswd /etc/nginx/htpasswd/hamradioonline-analytics stats-reader
|
||||
else
|
||||
sudo htpasswd -c /etc/nginx/htpasswd/hamradioonline-analytics stats-reader
|
||||
fi
|
||||
sudo chown root:www-data /etc/nginx/htpasswd/hamradioonline-analytics
|
||||
sudo chmod 0640 /etc/nginx/htpasswd/hamradioonline-analytics
|
||||
```
|
||||
|
||||
Choose the account name locally and enter the password interactively. Never
|
||||
store the resulting password hash in this repository or the installation ZIP.
|
||||
The example opens no GoAccess WebSocket and no additional GoAccess port. Its
|
||||
own access log is disabled and responses use a private, no-store cache policy.
|
||||
|
||||
Do not activate the final HTTPS vhost before its certificate files exist.
|
||||
First install the temporary HTTP bootstrap without changing the parallel apt
|
||||
Certbot installation or either renewal timer:
|
||||
|
||||
```sh
|
||||
sudo install -d -o root -g root -m 0755 /var/lib/letsencrypt
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/nginx/stats-vhost-http-bootstrap.conf.example \
|
||||
/etc/nginx/sites-available/stats.hamradioonline.de
|
||||
if [ ! -e /etc/nginx/sites-enabled/stats.hamradioonline.de ] && \
|
||||
[ ! -L /etc/nginx/sites-enabled/stats.hamradioonline.de ]; then
|
||||
sudo ln -s /etc/nginx/sites-available/stats.hamradioonline.de \
|
||||
/etc/nginx/sites-enabled/stats.hamradioonline.de
|
||||
fi
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
sudo /snap/bin/certbot certonly --webroot \
|
||||
--webroot-path /var/lib/letsencrypt \
|
||||
-d stats.hamradioonline.de
|
||||
```
|
||||
|
||||
Only after Certbot has created the certificate, replace the bootstrap with the
|
||||
final vhost. This does not remove HTTP completely. The final file keeps an
|
||||
IPv4 port 80 block for `/.well-known/acme-challenge/` so the certificate issued
|
||||
with `--webroot` can be renewed automatically. Every other HTTP request is
|
||||
redirected permanently to the same URI on HTTPS. The HTTPS block uses the
|
||||
existing Ubuntu/Certbot TLS files
|
||||
`/etc/letsencrypt/options-ssl-nginx.conf` and
|
||||
`/etc/letsencrypt/ssl-dhparams.pem`:
|
||||
|
||||
```sh
|
||||
sudo install -o root -g root -m 0644 \
|
||||
website/ops/analytics/nginx/stats-vhost.conf.example \
|
||||
/etc/nginx/sites-available/stats.hamradioonline.de
|
||||
sudo nginx -t
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
After activation, verify both authentication and renewal from the server:
|
||||
|
||||
```sh
|
||||
curl -I https://stats.hamradioonline.de/
|
||||
curl -I -u stats-reader https://stats.hamradioonline.de/
|
||||
sudo /snap/bin/certbot renew --dry-run \
|
||||
--cert-name stats.hamradioonline.de
|
||||
```
|
||||
|
||||
The first HTTPS request must return `401`. The authenticated request must
|
||||
return the redirect to `/combined/`. Enter the Basic Auth password
|
||||
interactively; do not put it on the command line. The Certbot dry-run must
|
||||
complete while the final vhost is active.
|
||||
|
||||
The templates listen on IPv4 only. Add an IPv6 listener later, after the AAAA
|
||||
record for `stats.hamradioonline.de` has been confirmed and tested.
|
||||
|
||||
The public counter location serves only `visitor-count.json` through an exact
|
||||
Nginx `alias`; it does not modify the deployed Eleventy release directory. Its
|
||||
own access log is disabled. Responses use
|
||||
`Cache-Control: public, max-age=3600` and
|
||||
`X-Content-Type-Options: nosniff`. The file contains the schema version, total,
|
||||
activation date and update time. It contains no IP address, user agent,
|
||||
hostname or per-day detail.
|
||||
|
||||
Use this rollout order:
|
||||
|
||||
1. Install the server files with explicit modes. Prepare directories, log
|
||||
ownership, filters and still-inactive Nginx and systemd configuration.
|
||||
2. Push the website changes, including the Privacy Policy, and let the existing
|
||||
deployment cron job publish them. Until the JSON endpoint exists, the
|
||||
visitor count remains hidden automatically.
|
||||
3. Verify the published Privacy Policy. Only then activate analytics logging,
|
||||
the report generator and timer, the public counter location and the
|
||||
protected statistics vhost.
|
||||
4. Run `nginx -t` before every Nginx reload and perform the real `--check` and
|
||||
`--dry-run` on the server before the first production generation.
|
||||
|
||||
A short period in which the Privacy Policy is already visible but logging is
|
||||
not yet active is acceptable. Starting analytics logging before publishing the
|
||||
updated policy is not.
|
||||
|
||||
## Retention and recovery
|
||||
|
||||
- Dedicated analytics raw logs: 14 days through the Logrotate example.
|
||||
- Anonymised detailed GoAccess aggregates: rolling 395 days.
|
||||
- Public daily counter values: retained from activation onward.
|
||||
|
||||
Back up the counter state and, if fast report recovery matters, the GoAccess
|
||||
database directories. Reports themselves are derived output. To recover, stop
|
||||
the timer, restore the state and database directories with their ownership,
|
||||
run `--check`, then run `--dry-run` before publishing again.
|
||||
|
||||
The repository contains no password, password hash, MaxMind download key,
|
||||
server IP address, TLS private key or private backup destination. Keep it that
|
||||
way.
|
||||
@@ -0,0 +1,874 @@
|
||||
#!/usr/bin/env node
|
||||
"use strict";
|
||||
|
||||
const fs = require("node:fs");
|
||||
const os = require("node:os");
|
||||
const path = require("node:path");
|
||||
const { spawnSync } = require("node:child_process");
|
||||
|
||||
const STATE_SCHEMA_VERSION = 1;
|
||||
const PUBLIC_SCHEMA_VERSION = 1;
|
||||
const MAX_SAFE_INTEGER = Number.MAX_SAFE_INTEGER;
|
||||
const RESERVED_STATS_HOST = "stats.hamradioonline.de";
|
||||
|
||||
class ConfigurationError extends Error {
|
||||
constructor(message) {
|
||||
super(message);
|
||||
this.name = "ConfigurationError";
|
||||
this.exitCode = 2;
|
||||
}
|
||||
}
|
||||
|
||||
class LockError extends Error {
|
||||
constructor(message) {
|
||||
super(message);
|
||||
this.name = "LockError";
|
||||
this.exitCode = 3;
|
||||
}
|
||||
}
|
||||
|
||||
function readJson(filePath, label) {
|
||||
let parsed;
|
||||
|
||||
try {
|
||||
parsed = JSON.parse(fs.readFileSync(filePath, "utf8"));
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(`${label} is not valid JSON: ${error.message}`);
|
||||
}
|
||||
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseIsoDate(value) {
|
||||
const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(value);
|
||||
|
||||
if (!match) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const year = Number(match[1]);
|
||||
const month = Number(match[2]);
|
||||
const day = Number(match[3]);
|
||||
const date = new Date(Date.UTC(year, month - 1, day));
|
||||
|
||||
if (date.getUTCFullYear() !== year
|
||||
|| date.getUTCMonth() !== month - 1
|
||||
|| date.getUTCDate() !== day) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return `${match[1]}-${match[2]}-${match[3]}`;
|
||||
}
|
||||
|
||||
function requireAbsolutePath(value, label) {
|
||||
if (typeof value !== "string" || /[\r\n\0]/.test(value) || !path.isAbsolute(value)) {
|
||||
throw new ConfigurationError(`${label} must be an absolute path`);
|
||||
}
|
||||
|
||||
return path.normalize(value);
|
||||
}
|
||||
|
||||
function isWithin(parent, candidate) {
|
||||
const relative = path.relative(parent, candidate);
|
||||
return relative !== ""
|
||||
&& relative !== ".."
|
||||
&& !relative.startsWith(`..${path.sep}`)
|
||||
&& !path.isAbsolute(relative);
|
||||
}
|
||||
|
||||
function validateRegistry(registry) {
|
||||
if (!registry || typeof registry !== "object" || Array.isArray(registry)) {
|
||||
throw new ConfigurationError("registry must be an object");
|
||||
}
|
||||
if (registry.schemaVersion !== 1) {
|
||||
throw new ConfigurationError("registry schemaVersion must be 1");
|
||||
}
|
||||
if (!Array.isArray(registry.sites) || registry.sites.length === 0) {
|
||||
throw new ConfigurationError("registry.sites must contain at least one site");
|
||||
}
|
||||
|
||||
const stateDirectory = requireAbsolutePath(
|
||||
registry.stateDirectory,
|
||||
"registry.stateDirectory"
|
||||
);
|
||||
const counterStatePath = requireAbsolutePath(
|
||||
registry.counterStatePath,
|
||||
"registry.counterStatePath"
|
||||
);
|
||||
if (!isWithin(stateDirectory, counterStatePath)) {
|
||||
throw new ConfigurationError("counterStatePath must be below stateDirectory");
|
||||
}
|
||||
|
||||
const ids = new Set();
|
||||
const hostnames = new Set();
|
||||
const analyticsLogPaths = new Set();
|
||||
const outputDirectories = new Set();
|
||||
const publicJsonPaths = new Set();
|
||||
const sites = registry.sites.map((site, index) => {
|
||||
const label = `registry.sites[${index}]`;
|
||||
|
||||
if (!site || typeof site !== "object" || Array.isArray(site)) {
|
||||
throw new ConfigurationError(`${label} must be an object`);
|
||||
}
|
||||
if (typeof site.id !== "string" || !/^[a-z0-9][a-z0-9-]{0,62}$/.test(site.id)) {
|
||||
throw new ConfigurationError(`${label}.id is invalid`);
|
||||
}
|
||||
if (site.id === "combined") {
|
||||
throw new ConfigurationError(`${label}.id is reserved`);
|
||||
}
|
||||
if (ids.has(site.id)) {
|
||||
throw new ConfigurationError(`duplicate site id: ${site.id}`);
|
||||
}
|
||||
ids.add(site.id);
|
||||
|
||||
if (typeof site.hostname !== "string"
|
||||
|| !/^[a-z0-9.-]+$/.test(site.hostname)
|
||||
|| site.hostname.includes("..")) {
|
||||
throw new ConfigurationError(`${label}.hostname is invalid`);
|
||||
}
|
||||
const hostname = site.hostname.toLowerCase();
|
||||
if (hostname === RESERVED_STATS_HOST) {
|
||||
throw new ConfigurationError(`${RESERVED_STATS_HOST} must not be registered`);
|
||||
}
|
||||
if (hostnames.has(hostname)) {
|
||||
throw new ConfigurationError(`duplicate hostname: ${hostname}`);
|
||||
}
|
||||
hostnames.add(hostname);
|
||||
|
||||
const activatedOn = parseIsoDate(site.activatedOn);
|
||||
if (!activatedOn) {
|
||||
throw new ConfigurationError(`${label}.activatedOn must be a valid ISO date`);
|
||||
}
|
||||
if (typeof site.publicCounter !== "boolean") {
|
||||
throw new ConfigurationError(`${label}.publicCounter must be boolean`);
|
||||
}
|
||||
const analyticsLog = requireAbsolutePath(
|
||||
site.analyticsLog,
|
||||
`${label}.analyticsLog`
|
||||
);
|
||||
if (/\.(?:\d+|gz)$/i.test(path.basename(analyticsLog))) {
|
||||
throw new ConfigurationError(
|
||||
`${label}.analyticsLog must identify the current uncompressed log`
|
||||
);
|
||||
}
|
||||
if (analyticsLogPaths.has(analyticsLog)) {
|
||||
throw new ConfigurationError(`analytics log is registered more than once: ${analyticsLog}`);
|
||||
}
|
||||
analyticsLogPaths.add(analyticsLog);
|
||||
|
||||
const reportOutputDirectory = requireAbsolutePath(
|
||||
site.reportOutputDirectory,
|
||||
`${label}.reportOutputDirectory`
|
||||
);
|
||||
if (!isWithin(stateDirectory, reportOutputDirectory)) {
|
||||
throw new ConfigurationError(`${label}.reportOutputDirectory must be below stateDirectory`);
|
||||
}
|
||||
if (outputDirectories.has(reportOutputDirectory)) {
|
||||
throw new ConfigurationError(`duplicate report output directory: ${reportOutputDirectory}`);
|
||||
}
|
||||
outputDirectories.add(reportOutputDirectory);
|
||||
|
||||
const publicJsonPath = site.publicCounter
|
||||
? requireAbsolutePath(site.publicJsonPath, `${label}.publicJsonPath`)
|
||||
: null;
|
||||
if (publicJsonPath && !isWithin(stateDirectory, publicJsonPath)) {
|
||||
throw new ConfigurationError(`${label}.publicJsonPath must be below stateDirectory`);
|
||||
}
|
||||
if (publicJsonPath === counterStatePath) {
|
||||
throw new ConfigurationError(`${label}.publicJsonPath conflicts with counterStatePath`);
|
||||
}
|
||||
if (publicJsonPath && publicJsonPaths.has(publicJsonPath)) {
|
||||
throw new ConfigurationError(`duplicate public JSON path: ${publicJsonPath}`);
|
||||
}
|
||||
if (publicJsonPath) {
|
||||
publicJsonPaths.add(publicJsonPath);
|
||||
}
|
||||
|
||||
return {
|
||||
id: site.id,
|
||||
hostname,
|
||||
activatedOn,
|
||||
publicCounter: site.publicCounter,
|
||||
analyticsLog,
|
||||
reportOutputDirectory,
|
||||
publicJsonPath
|
||||
};
|
||||
});
|
||||
|
||||
if (!registry.combined || typeof registry.combined !== "object") {
|
||||
throw new ConfigurationError("registry.combined must be an object");
|
||||
}
|
||||
|
||||
const combinedReportOutputDirectory = requireAbsolutePath(
|
||||
registry.combined.reportOutputDirectory,
|
||||
"registry.combined.reportOutputDirectory"
|
||||
);
|
||||
if (!isWithin(stateDirectory, combinedReportOutputDirectory)) {
|
||||
throw new ConfigurationError(
|
||||
"registry.combined.reportOutputDirectory must be below stateDirectory"
|
||||
);
|
||||
}
|
||||
if (outputDirectories.has(combinedReportOutputDirectory)) {
|
||||
throw new ConfigurationError(
|
||||
`duplicate report output directory: ${combinedReportOutputDirectory}`
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
schemaVersion: 1,
|
||||
stateDirectory,
|
||||
counterStatePath,
|
||||
lockFile: requireAbsolutePath(registry.lockFile, "registry.lockFile"),
|
||||
geoIpCountryDatabase: requireAbsolutePath(
|
||||
registry.geoIpCountryDatabase,
|
||||
"registry.geoIpCountryDatabase"
|
||||
),
|
||||
combined: {
|
||||
reportOutputDirectory: combinedReportOutputDirectory
|
||||
},
|
||||
sites
|
||||
};
|
||||
}
|
||||
|
||||
function resolveAnalyticsLogs(site) {
|
||||
const logs = [];
|
||||
const candidates = [
|
||||
{ path: `${site.analyticsLog}.1`, required: false },
|
||||
{ path: site.analyticsLog, required: true }
|
||||
];
|
||||
|
||||
for (const candidate of candidates) {
|
||||
let stats;
|
||||
try {
|
||||
stats = fs.statSync(candidate.path);
|
||||
} catch (error) {
|
||||
if (!candidate.required && error.code === "ENOENT") {
|
||||
continue;
|
||||
}
|
||||
throw new ConfigurationError(`analytics log is not readable: ${candidate.path}`);
|
||||
}
|
||||
if (!stats.isFile()) {
|
||||
throw new ConfigurationError(`analytics log is not a file: ${candidate.path}`);
|
||||
}
|
||||
try {
|
||||
fs.accessSync(candidate.path, fs.constants.R_OK);
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(
|
||||
`analytics log is not readable by the current user: ${candidate.path}`
|
||||
);
|
||||
}
|
||||
logs.push(candidate.path);
|
||||
}
|
||||
|
||||
return logs;
|
||||
}
|
||||
|
||||
function requireWritableDirectory(directory, label) {
|
||||
let stats;
|
||||
try {
|
||||
stats = fs.statSync(directory);
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(`${label} does not exist: ${directory}`);
|
||||
}
|
||||
if (!stats.isDirectory()) {
|
||||
throw new ConfigurationError(`${label} is not a directory: ${directory}`);
|
||||
}
|
||||
try {
|
||||
fs.accessSync(directory, fs.constants.W_OK | fs.constants.X_OK);
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(
|
||||
`${label} is not writable by the current user: ${directory}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function validateInputs(registry, configTemplate) {
|
||||
const requiredConfigParts = [
|
||||
"{{DB_PATH}}",
|
||||
"{{RESTORE_DIRECTIVE}}",
|
||||
"{{GEOIP_COUNTRY_DATABASE}}",
|
||||
"persist true",
|
||||
"anonymize-ip true",
|
||||
"ignore-crawlers true",
|
||||
"unknowns-as-crawlers true",
|
||||
"keep-last 395"
|
||||
];
|
||||
|
||||
for (const required of requiredConfigParts) {
|
||||
if (!configTemplate.includes(required)) {
|
||||
throw new ConfigurationError(`GoAccess template is missing: ${required}`);
|
||||
}
|
||||
}
|
||||
|
||||
const analyticsLogsBySite = new Map(registry.sites.map(site => [
|
||||
site.id,
|
||||
resolveAnalyticsLogs(site)
|
||||
]));
|
||||
|
||||
let geoStats;
|
||||
try {
|
||||
geoStats = fs.statSync(registry.geoIpCountryDatabase);
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(
|
||||
`GeoIP Country database is not readable: ${registry.geoIpCountryDatabase}`
|
||||
);
|
||||
}
|
||||
if (!geoStats.isFile() || /city/i.test(path.basename(registry.geoIpCountryDatabase))) {
|
||||
throw new ConfigurationError("geoIpCountryDatabase must be a Country database file");
|
||||
}
|
||||
try {
|
||||
fs.accessSync(registry.geoIpCountryDatabase, fs.constants.R_OK);
|
||||
} catch (error) {
|
||||
throw new ConfigurationError(
|
||||
`GeoIP Country database is not readable by the current user: `
|
||||
+ registry.geoIpCountryDatabase
|
||||
);
|
||||
}
|
||||
|
||||
requireWritableDirectory(registry.stateDirectory, "stateDirectory");
|
||||
requireWritableDirectory(
|
||||
registry.combined.reportOutputDirectory,
|
||||
"combined report output directory"
|
||||
);
|
||||
for (const site of registry.sites) {
|
||||
requireWritableDirectory(
|
||||
site.reportOutputDirectory,
|
||||
`report output directory for ${site.id}`
|
||||
);
|
||||
if (site.publicCounter) {
|
||||
requireWritableDirectory(
|
||||
path.dirname(site.publicJsonPath),
|
||||
`public output directory for ${site.id}`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return analyticsLogsBySite;
|
||||
}
|
||||
|
||||
function renderGoAccessConfig(template, dbPath, restore, geoIpCountryDatabase) {
|
||||
const rendered = template
|
||||
.replaceAll("{{DB_PATH}}", dbPath)
|
||||
.replaceAll(
|
||||
"{{RESTORE_DIRECTIVE}}",
|
||||
restore ? "restore true" : "# restore is disabled until a database exists"
|
||||
)
|
||||
.replaceAll("{{GEOIP_COUNTRY_DATABASE}}", geoIpCountryDatabase);
|
||||
|
||||
if (/{{[A-Z_]+}}/.test(rendered)) {
|
||||
throw new ConfigurationError("GoAccess template contains an unknown placeholder");
|
||||
}
|
||||
|
||||
return rendered;
|
||||
}
|
||||
|
||||
function defaultRunGoAccess({ binary, args }) {
|
||||
const result = spawnSync(binary, args, {
|
||||
encoding: "utf8",
|
||||
maxBuffer: 1024 * 1024,
|
||||
timeout: 30 * 60 * 1000,
|
||||
windowsHide: true
|
||||
});
|
||||
|
||||
if (result.error) {
|
||||
throw new Error(`could not start GoAccess: ${result.error.message}`);
|
||||
}
|
||||
if (result.status !== 0) {
|
||||
const detail = (result.stderr || result.stdout || "no diagnostic output").trim();
|
||||
throw new Error(`GoAccess failed with exit code ${result.status}: ${detail}`);
|
||||
}
|
||||
}
|
||||
|
||||
function defaultCheckGoAccess(binary) {
|
||||
const result = spawnSync(binary, ["--version"], {
|
||||
encoding: "utf8",
|
||||
timeout: 10000,
|
||||
windowsHide: true
|
||||
});
|
||||
|
||||
if (result.error || result.status !== 0) {
|
||||
const detail = result.error
|
||||
? result.error.message
|
||||
: (result.stderr || result.stdout || "no diagnostic output").trim();
|
||||
throw new ConfigurationError(`GoAccess is not available: ${detail}`);
|
||||
}
|
||||
|
||||
return parseGoAccessVersion(`${result.stdout || ""}\n${result.stderr || ""}`);
|
||||
}
|
||||
|
||||
function parseGoAccessVersion(output) {
|
||||
const versionMatch = /GoAccess\s+-\s+([0-9]+(?:\.[0-9]+)+)/i.exec(output);
|
||||
|
||||
return {
|
||||
version: versionMatch ? versionMatch[1] : "unknown",
|
||||
geoIpMmdb: /--enable-geoip=mmdb\b/i.test(output),
|
||||
openSsl: /--with-openssl\b/i.test(output),
|
||||
zlib: /--with-zlib\b/i.test(output)
|
||||
};
|
||||
}
|
||||
|
||||
function formatGoAccessCheck(capabilities) {
|
||||
const detected = [
|
||||
`GeoIP2/MMDB ${capabilities.geoIpMmdb ? "enabled" : "not detected"}`,
|
||||
`OpenSSL ${capabilities.openSsl ? "enabled" : "not detected"}`,
|
||||
`Zlib ${capabilities.zlib ? "enabled" : "not detected"}`
|
||||
].join(", ");
|
||||
const summary = `GoAccess ${capabilities.version}: ${detected}.`;
|
||||
|
||||
if (capabilities.zlib) {
|
||||
return summary;
|
||||
}
|
||||
|
||||
return `${summary} This build has no Zlib support. That is valid for regular mode: `
|
||||
+ "the current analytics log and its optional uncompressed .1 rotation are read "
|
||||
+ "directly; older .gz logs are not imported.";
|
||||
}
|
||||
|
||||
function parseGoAccessDate(value) {
|
||||
if (typeof value !== "string") {
|
||||
return null;
|
||||
}
|
||||
if (parseIsoDate(value)) {
|
||||
return value;
|
||||
}
|
||||
if (/^\d{8}$/.test(value)) {
|
||||
return parseIsoDate(`${value.slice(0, 4)}-${value.slice(4, 6)}-${value.slice(6, 8)}`);
|
||||
}
|
||||
|
||||
const match = /^(\d{2})\/([A-Za-z]{3})\/(\d{4})$/.exec(value);
|
||||
const months = {
|
||||
Jan: "01", Feb: "02", Mar: "03", Apr: "04", May: "05", Jun: "06",
|
||||
Jul: "07", Aug: "08", Sep: "09", Oct: "10", Nov: "11", Dec: "12"
|
||||
};
|
||||
|
||||
return match && months[match[2]]
|
||||
? parseIsoDate(`${match[3]}-${months[match[2]]}-${match[1]}`)
|
||||
: null;
|
||||
}
|
||||
|
||||
function validateReport(report, combined) {
|
||||
if (!report || typeof report !== "object" || Array.isArray(report)) {
|
||||
throw new Error("GoAccess JSON report must be an object");
|
||||
}
|
||||
const requiredPanels = ["visitors", "requests", "status_codes", "geo_location"];
|
||||
if (combined) {
|
||||
requiredPanels.push("virtual_hosts");
|
||||
}
|
||||
if (!report.general || typeof report.general !== "object") {
|
||||
throw new Error("GoAccess JSON report has no general summary");
|
||||
}
|
||||
for (const panel of requiredPanels) {
|
||||
if (!report[panel] || !Array.isArray(report[panel].data)) {
|
||||
throw new Error(`GoAccess JSON report has no ${panel} panel`);
|
||||
}
|
||||
}
|
||||
|
||||
extractDailyVisits(report);
|
||||
return report;
|
||||
}
|
||||
|
||||
function readAndValidateReport(jsonPath, htmlPath, combined) {
|
||||
let report;
|
||||
try {
|
||||
report = JSON.parse(fs.readFileSync(jsonPath, "utf8"));
|
||||
} catch (error) {
|
||||
throw new Error(`GoAccess JSON output is invalid: ${error.message}`);
|
||||
}
|
||||
validateReport(report, combined);
|
||||
|
||||
const html = fs.readFileSync(htmlPath, "utf8");
|
||||
if (html.length < 100 || !/<html(?:\s|>)/i.test(html)) {
|
||||
throw new Error("GoAccess HTML output is missing or implausibly small");
|
||||
}
|
||||
|
||||
return report;
|
||||
}
|
||||
|
||||
function extractDailyVisits(report) {
|
||||
const daily = {};
|
||||
|
||||
for (const row of report.visitors.data) {
|
||||
const date = parseGoAccessDate(row && row.data);
|
||||
const visits = row && row.visitors && row.visitors.count;
|
||||
|
||||
if (!date || !Number.isSafeInteger(visits) || visits < 0) {
|
||||
throw new Error("GoAccess visitors panel contains invalid daily data");
|
||||
}
|
||||
if (Object.hasOwn(daily, date)) {
|
||||
throw new Error(`GoAccess visitors panel contains duplicate date ${date}`);
|
||||
}
|
||||
daily[date] = visits;
|
||||
}
|
||||
|
||||
return daily;
|
||||
}
|
||||
|
||||
function loadCounterState(statePath) {
|
||||
if (!fs.existsSync(statePath)) {
|
||||
return { schemaVersion: STATE_SCHEMA_VERSION, sites: {} };
|
||||
}
|
||||
|
||||
const state = readJson(statePath, "counter state");
|
||||
if (state.schemaVersion !== STATE_SCHEMA_VERSION
|
||||
|| !state.sites || typeof state.sites !== "object" || Array.isArray(state.sites)) {
|
||||
throw new ConfigurationError("counter state has an unsupported structure");
|
||||
}
|
||||
|
||||
for (const [siteId, site] of Object.entries(state.sites)) {
|
||||
if (!site || typeof site !== "object" || !parseIsoDate(site.since)
|
||||
|| !site.dailyVisits || typeof site.dailyVisits !== "object"
|
||||
|| Array.isArray(site.dailyVisits)) {
|
||||
throw new ConfigurationError(`counter state for ${siteId} is invalid`);
|
||||
}
|
||||
for (const [date, visits] of Object.entries(site.dailyVisits)) {
|
||||
if (!parseIsoDate(date) || !Number.isSafeInteger(visits) || visits < 0) {
|
||||
throw new ConfigurationError(`counter state value for ${siteId}/${date} is invalid`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return state;
|
||||
}
|
||||
|
||||
function updateCounterState(state, site, report) {
|
||||
const current = state.sites[site.id];
|
||||
if (current && current.since !== site.activatedOn) {
|
||||
throw new ConfigurationError(
|
||||
`activation date for ${site.id} differs from the existing counter state`
|
||||
);
|
||||
}
|
||||
if (current && current.hostname !== site.hostname) {
|
||||
throw new ConfigurationError(
|
||||
`hostname for ${site.id} differs from the existing counter state`
|
||||
);
|
||||
}
|
||||
|
||||
const dailyVisits = current ? { ...current.dailyVisits } : {};
|
||||
for (const [date, visits] of Object.entries(extractDailyVisits(report))) {
|
||||
if (date >= site.activatedOn) {
|
||||
dailyVisits[date] = visits;
|
||||
}
|
||||
}
|
||||
state.sites[site.id] = {
|
||||
hostname: site.hostname,
|
||||
since: site.activatedOn,
|
||||
dailyVisits
|
||||
};
|
||||
}
|
||||
|
||||
function publicPayload(state, site, now) {
|
||||
const siteState = state.sites[site.id];
|
||||
let visits = 0;
|
||||
|
||||
for (const [date, value] of Object.entries(siteState.dailyVisits)) {
|
||||
if (date >= site.activatedOn) {
|
||||
visits += value;
|
||||
if (!Number.isSafeInteger(visits) || visits > MAX_SAFE_INTEGER) {
|
||||
throw new Error(`public visit total for ${site.id} exceeds the safe integer range`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
schemaVersion: PUBLIC_SCHEMA_VERSION,
|
||||
visits,
|
||||
since: site.activatedOn,
|
||||
updatedAt: now.toISOString().replace(/\.\d{3}Z$/, "Z")
|
||||
};
|
||||
}
|
||||
|
||||
function atomicWriteFile(destination, content, mode = 0o640) {
|
||||
const directory = path.dirname(destination);
|
||||
const temporary = path.join(
|
||||
directory,
|
||||
`.${path.basename(destination)}.${process.pid}.${Date.now()}.tmp`
|
||||
);
|
||||
const handle = fs.openSync(temporary, "wx", mode);
|
||||
|
||||
try {
|
||||
fs.writeFileSync(handle, content);
|
||||
fs.fchmodSync(handle, mode);
|
||||
fs.fsyncSync(handle);
|
||||
} finally {
|
||||
fs.closeSync(handle);
|
||||
}
|
||||
try {
|
||||
fs.renameSync(temporary, destination);
|
||||
} catch (error) {
|
||||
fs.rmSync(temporary, { force: true });
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function atomicCopyFile(source, destination) {
|
||||
atomicWriteFile(destination, fs.readFileSync(source));
|
||||
}
|
||||
|
||||
function replaceDirectory(source, destination) {
|
||||
fs.mkdirSync(path.dirname(destination), { recursive: true });
|
||||
const backup = `${destination}.previous-${process.pid}`;
|
||||
const hadDestination = fs.existsSync(destination);
|
||||
|
||||
if (fs.existsSync(backup)) {
|
||||
throw new Error(`stale database backup blocks replacement: ${backup}`);
|
||||
}
|
||||
|
||||
try {
|
||||
if (hadDestination) {
|
||||
fs.renameSync(destination, backup);
|
||||
}
|
||||
fs.renameSync(source, destination);
|
||||
if (hadDestination) {
|
||||
fs.rmSync(backup, { recursive: true, force: true });
|
||||
}
|
||||
} catch (error) {
|
||||
if (!fs.existsSync(destination) && fs.existsSync(backup)) {
|
||||
fs.renameSync(backup, destination);
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
function acquireLock(lockPath) {
|
||||
fs.mkdirSync(path.dirname(lockPath), { recursive: true });
|
||||
let descriptor;
|
||||
|
||||
try {
|
||||
descriptor = fs.openSync(lockPath, "wx", 0o640);
|
||||
fs.writeFileSync(descriptor, `${process.pid}\n`);
|
||||
} catch (error) {
|
||||
throw new LockError(`another analytics run is active (${lockPath})`);
|
||||
}
|
||||
|
||||
return () => {
|
||||
fs.closeSync(descriptor);
|
||||
fs.rmSync(lockPath, { force: true });
|
||||
};
|
||||
}
|
||||
|
||||
function createReportJob(id, logs, outputDirectory, combined) {
|
||||
return { id, logs, outputDirectory, combined };
|
||||
}
|
||||
|
||||
function prepareReport(job, context) {
|
||||
const jobDirectory = path.join(context.runDirectory, job.id);
|
||||
const stagedDb = path.join(jobDirectory, "db");
|
||||
const currentDb = path.join(context.registry.stateDirectory, "db", job.id);
|
||||
const outputJson = path.join(jobDirectory, "report.json");
|
||||
const outputHtml = path.join(jobDirectory, "report.html");
|
||||
const runConfig = path.join(jobDirectory, "goaccess.conf");
|
||||
let hasDatabase = false;
|
||||
|
||||
if (fs.existsSync(currentDb)) {
|
||||
if (!fs.statSync(currentDb).isDirectory()) {
|
||||
throw new Error(`GoAccess database path is not a directory: ${currentDb}`);
|
||||
}
|
||||
hasDatabase = fs.readdirSync(currentDb).length > 0;
|
||||
}
|
||||
|
||||
fs.mkdirSync(jobDirectory, { recursive: true });
|
||||
if (hasDatabase) {
|
||||
fs.cpSync(currentDb, stagedDb, { recursive: true, errorOnExist: true });
|
||||
} else {
|
||||
fs.mkdirSync(stagedDb);
|
||||
}
|
||||
fs.writeFileSync(runConfig, renderGoAccessConfig(
|
||||
context.configTemplate,
|
||||
stagedDb,
|
||||
hasDatabase,
|
||||
context.registry.geoIpCountryDatabase
|
||||
));
|
||||
|
||||
const args = [
|
||||
...job.logs,
|
||||
"--no-global-config",
|
||||
"--config-file", runConfig,
|
||||
"--output", outputJson,
|
||||
"--output", outputHtml
|
||||
];
|
||||
context.runGoAccess({
|
||||
binary: context.goaccessBinary,
|
||||
args,
|
||||
id: job.id,
|
||||
combined: job.combined,
|
||||
outputJson,
|
||||
outputHtml,
|
||||
dbPath: stagedDb
|
||||
});
|
||||
|
||||
return {
|
||||
...job,
|
||||
report: readAndValidateReport(outputJson, outputHtml, job.combined),
|
||||
outputJson,
|
||||
outputHtml,
|
||||
stagedDb,
|
||||
currentDb
|
||||
};
|
||||
}
|
||||
|
||||
function generateReports(options, dependencies = {}) {
|
||||
const registryPath = path.resolve(options.registryPath);
|
||||
const configTemplatePath = path.resolve(options.configTemplatePath);
|
||||
const registry = validateRegistry(readJson(registryPath, "site registry"));
|
||||
const configTemplate = fs.readFileSync(configTemplatePath, "utf8");
|
||||
const runGoAccess = dependencies.runGoAccess || defaultRunGoAccess;
|
||||
const checkGoAccess = dependencies.checkGoAccess || defaultCheckGoAccess;
|
||||
const now = dependencies.now ? dependencies.now() : new Date();
|
||||
|
||||
const analyticsLogsBySite = validateInputs(registry, configTemplate);
|
||||
const goAccess = checkGoAccess(options.goaccessBinary || "goaccess");
|
||||
if (!goAccess || !goAccess.geoIpMmdb) {
|
||||
throw new ConfigurationError(
|
||||
"GoAccess must be built with GeoIP2/MMDB support (--enable-geoip=mmdb)"
|
||||
);
|
||||
}
|
||||
if (options.check) {
|
||||
return { checked: true, sites: registry.sites.length, goAccess };
|
||||
}
|
||||
|
||||
const releaseLock = options.dryRun || dependencies.skipLock
|
||||
? () => {}
|
||||
: acquireLock(registry.lockFile);
|
||||
let runDirectory;
|
||||
|
||||
try {
|
||||
runDirectory = fs.mkdtempSync(path.join(
|
||||
options.dryRun ? os.tmpdir() : registry.stateDirectory,
|
||||
".analytics-run-"
|
||||
));
|
||||
|
||||
const jobs = registry.sites.map(site => createReportJob(
|
||||
site.id,
|
||||
analyticsLogsBySite.get(site.id),
|
||||
site.reportOutputDirectory,
|
||||
false
|
||||
));
|
||||
jobs.push(createReportJob(
|
||||
"combined",
|
||||
registry.sites.flatMap(site => analyticsLogsBySite.get(site.id)),
|
||||
registry.combined.reportOutputDirectory,
|
||||
true
|
||||
));
|
||||
|
||||
const context = {
|
||||
registry,
|
||||
configTemplate,
|
||||
goaccessBinary: options.goaccessBinary || "goaccess",
|
||||
runGoAccess,
|
||||
runDirectory
|
||||
};
|
||||
const prepared = jobs.map(job => prepareReport(job, context));
|
||||
const state = loadCounterState(registry.counterStatePath);
|
||||
|
||||
for (const site of registry.sites.filter(entry => entry.publicCounter)) {
|
||||
const generated = prepared.find(entry => entry.id === site.id);
|
||||
updateCounterState(state, site, generated.report);
|
||||
}
|
||||
|
||||
const publicFiles = registry.sites
|
||||
.filter(site => site.publicCounter)
|
||||
.map(site => ({
|
||||
destination: site.publicJsonPath,
|
||||
content: `${JSON.stringify(publicPayload(state, site, now), null, 2)}\n`
|
||||
}));
|
||||
|
||||
if (!options.dryRun) {
|
||||
for (const report of prepared) {
|
||||
atomicCopyFile(
|
||||
report.outputJson,
|
||||
path.join(report.outputDirectory, "report.json")
|
||||
);
|
||||
atomicCopyFile(
|
||||
report.outputHtml,
|
||||
path.join(report.outputDirectory, "report.html")
|
||||
);
|
||||
}
|
||||
for (const report of prepared) {
|
||||
replaceDirectory(report.stagedDb, report.currentDb);
|
||||
}
|
||||
atomicWriteFile(
|
||||
registry.counterStatePath,
|
||||
`${JSON.stringify(state, null, 2)}\n`
|
||||
);
|
||||
for (const publicFile of publicFiles) {
|
||||
atomicWriteFile(publicFile.destination, publicFile.content, 0o644);
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
checked: false,
|
||||
dryRun: Boolean(options.dryRun),
|
||||
sites: registry.sites.length,
|
||||
reports: prepared.length,
|
||||
publicCounters: publicFiles.length
|
||||
};
|
||||
} finally {
|
||||
if (runDirectory && fs.existsSync(runDirectory)) {
|
||||
fs.rmSync(runDirectory, { recursive: true, force: true });
|
||||
}
|
||||
releaseLock();
|
||||
}
|
||||
}
|
||||
|
||||
function parseArguments(argv) {
|
||||
const options = { check: false, dryRun: false, goaccessBinary: "goaccess" };
|
||||
|
||||
for (let index = 0; index < argv.length; index += 1) {
|
||||
const argument = argv[index];
|
||||
if (argument === "--check") {
|
||||
options.check = true;
|
||||
} else if (argument === "--dry-run") {
|
||||
options.dryRun = true;
|
||||
} else if (["--registry", "--config-template", "--goaccess"].includes(argument)) {
|
||||
const value = argv[index + 1];
|
||||
if (!value) {
|
||||
throw new ConfigurationError(`${argument} requires a value`);
|
||||
}
|
||||
index += 1;
|
||||
if (argument === "--registry") options.registryPath = value;
|
||||
if (argument === "--config-template") options.configTemplatePath = value;
|
||||
if (argument === "--goaccess") options.goaccessBinary = value;
|
||||
} else {
|
||||
throw new ConfigurationError(`unknown argument: ${argument}`);
|
||||
}
|
||||
}
|
||||
|
||||
if (!options.registryPath || !options.configTemplatePath) {
|
||||
throw new ConfigurationError(
|
||||
"usage: generate-reports.js --registry FILE --config-template FILE "
|
||||
+ "[--goaccess FILE] [--check|--dry-run]"
|
||||
);
|
||||
}
|
||||
if (options.check && options.dryRun) {
|
||||
throw new ConfigurationError("--check and --dry-run are mutually exclusive");
|
||||
}
|
||||
|
||||
return options;
|
||||
}
|
||||
|
||||
if (require.main === module) {
|
||||
try {
|
||||
const result = generateReports(parseArguments(process.argv.slice(2)));
|
||||
const action = result.checked ? "Configuration check" : "Analytics generation";
|
||||
if (result.checked) {
|
||||
process.stdout.write(`${formatGoAccessCheck(result.goAccess)}\n`);
|
||||
}
|
||||
process.stdout.write(`${action} completed: ${JSON.stringify(result)}\n`);
|
||||
} catch (error) {
|
||||
process.stderr.write(`analytics: ${error.message}\n`);
|
||||
process.exitCode = error.exitCode || 1;
|
||||
}
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ConfigurationError,
|
||||
extractDailyVisits,
|
||||
formatGoAccessCheck,
|
||||
generateReports,
|
||||
parseGoAccessVersion,
|
||||
parseArguments,
|
||||
publicPayload,
|
||||
updateCounterState,
|
||||
validateRegistry,
|
||||
validateReport
|
||||
};
|
||||
@@ -0,0 +1,43 @@
|
||||
# Rendered by generate-reports.js. Do not use this file without replacing
|
||||
# all {{...}} placeholders.
|
||||
|
||||
datetime-format %Y-%m-%dT%H:%M:%S%z
|
||||
log-format %v\t%h\t%x\t%m\t%U\t%H\t%s\t%b\t"%u"
|
||||
|
||||
anonymize-ip true
|
||||
anonymize-level 2
|
||||
ignore-crawlers true
|
||||
unknowns-as-crawlers true
|
||||
keep-last 395
|
||||
|
||||
persist true
|
||||
{{RESTORE_DIRECTIVE}}
|
||||
db-path {{DB_PATH}}
|
||||
|
||||
geoip-database {{GEOIP_COUNTRY_DATABASE}}
|
||||
|
||||
json-pretty-print true
|
||||
no-progress true
|
||||
no-parsing-spinner true
|
||||
no-color true
|
||||
agent-list false
|
||||
http-method true
|
||||
http-protocol true
|
||||
max-items 500
|
||||
|
||||
# The Nginx analytics log contains $uri rather than $request_uri, so query
|
||||
# strings never reach GoAccess. These panels are deliberately unavailable.
|
||||
ignore-panel HOSTS
|
||||
ignore-panel OS
|
||||
ignore-panel BROWSERS
|
||||
ignore-panel REFERRERS
|
||||
ignore-panel REFERRING_SITES
|
||||
ignore-panel KEYPHRASES
|
||||
ignore-panel REMOTE_USER
|
||||
ignore-panel REQUESTS_STATIC
|
||||
ignore-panel VISIT_TIMES
|
||||
ignore-panel NOT_FOUND
|
||||
ignore-panel ASN
|
||||
ignore-panel MIME_TYPE
|
||||
ignore-panel TLS_TYPE
|
||||
ignore-panel CACHE_STATUS
|
||||
@@ -0,0 +1,14 @@
|
||||
/var/log/nginx/*-analytics.log {
|
||||
daily
|
||||
rotate 14
|
||||
missingok
|
||||
notifempty
|
||||
compress
|
||||
# Keep .1 uncompressed because the regular generator reads it directly.
|
||||
delaycompress
|
||||
create 0640 www-data hamradio-analytics
|
||||
sharedscripts
|
||||
postrotate
|
||||
invoke-rc.d nginx rotate >/dev/null 2>&1
|
||||
endscript
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
# Only page GET requests which are candidates for the reach statistics enter
|
||||
# the dedicated analytics log. GoAccess performs the second bot-classification
|
||||
# layer, including unknown browsers and operating systems.
|
||||
map $request_method $hamradioonline_analytics_method {
|
||||
default 0;
|
||||
GET 1;
|
||||
}
|
||||
|
||||
map $uri $hamradioonline_analytics_path {
|
||||
default 1;
|
||||
|
||||
/visitor-count.json 0;
|
||||
/kst4ContestVersionInfo.xml 0;
|
||||
/sitemap.xml 0;
|
||||
/robots.txt 0;
|
||||
/favicon.ico 0;
|
||||
/assets/favicon.svg 0;
|
||||
/health 0;
|
||||
/healthz 0;
|
||||
/ping 0;
|
||||
/status 0;
|
||||
|
||||
~*^/(?:assets|manual/assets)/ 0;
|
||||
~*\.(?:css|js|mjs|map|json|png|jpe?g|gif|svg|webp|avif|ico|woff2?|ttf|otf|eot|xml|txt|pdf|zip|gz|wasm|mp4|webm)$ 0;
|
||||
}
|
||||
|
||||
map $http_user_agent $hamradioonline_analytics_known_bot {
|
||||
default 0;
|
||||
~*(?:bot|crawler|spider|slurp|headless|monitor|healthcheck|uptime|wget|curl) 1;
|
||||
}
|
||||
|
||||
map "$hamradioonline_analytics_method:$hamradioonline_analytics_path:$hamradioonline_analytics_known_bot"
|
||||
$hamradioonline_analytics_loggable {
|
||||
default 0;
|
||||
"1:1:0" 1;
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
# Include in the Nginx http context. $uri is the normalized path and excludes
|
||||
# the query string. The format intentionally omits referrer and remote user.
|
||||
log_format hamradioonline_analytics
|
||||
'$server_name\t$remote_addr\t$time_iso8601\t$request_method\t$uri\t'
|
||||
'$server_protocol\t$status\t$body_bytes_sent\t"$http_user_agent"';
|
||||
|
||||
# Include the maps below in the Nginx http context as well.
|
||||
include /etc/nginx/snippets/hamradioonline-analytics-filters.conf;
|
||||
|
||||
# Add this extra log to each registered project server block. Keep the
|
||||
# existing operational access_log directive; do not replace it implicitly.
|
||||
#
|
||||
# access_log /var/log/nginx/kst4contest-analytics.log
|
||||
# hamradioonline_analytics if=$hamradioonline_analytics_loggable;
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
# Include inside the kst4contest.hamradioonline.de HTTPS server block. Grant
|
||||
# the Nginx worker read access to the file and directory, but no write access.
|
||||
location = /visitor-count.json {
|
||||
alias /var/lib/hamradioonline-analytics/public/kst4contest/visitor-count.json;
|
||||
default_type application/json;
|
||||
access_log off;
|
||||
add_header Cache-Control "public, max-age=3600" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
}
|
||||
@@ -0,0 +1,17 @@
|
||||
# Temporary HTTP-only virtual host for initial certificate provisioning.
|
||||
# Replace it with stats-vhost.conf.example after Certbot has succeeded.
|
||||
server {
|
||||
listen 80;
|
||||
server_name stats.hamradioonline.de;
|
||||
|
||||
access_log off;
|
||||
|
||||
location ^~ /.well-known/acme-challenge/ {
|
||||
root /var/lib/letsencrypt;
|
||||
default_type text/plain;
|
||||
}
|
||||
|
||||
location / {
|
||||
return 404;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
# Keep this HTTP server active so Certbot can renew the webroot certificate.
|
||||
server {
|
||||
listen 80;
|
||||
server_name stats.hamradioonline.de;
|
||||
|
||||
access_log off;
|
||||
|
||||
location ^~ /.well-known/acme-challenge/ {
|
||||
root /var/lib/letsencrypt;
|
||||
default_type text/plain;
|
||||
}
|
||||
|
||||
location / {
|
||||
return 301 https://$host$request_uri;
|
||||
}
|
||||
}
|
||||
|
||||
# This server exposes static reports only. Provision the certificate and the
|
||||
# htpasswd file outside the repository. Do not add this host to sites.json.
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name stats.hamradioonline.de;
|
||||
|
||||
ssl_certificate /etc/letsencrypt/live/stats.hamradioonline.de/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/stats.hamradioonline.de/privkey.pem;
|
||||
include /etc/letsencrypt/options-ssl-nginx.conf;
|
||||
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
|
||||
|
||||
root /var/lib/hamradioonline-analytics/reports;
|
||||
index report.html;
|
||||
|
||||
auth_basic "Private project statistics";
|
||||
auth_basic_user_file /etc/nginx/htpasswd/hamradioonline-analytics;
|
||||
|
||||
access_log off;
|
||||
add_header Cache-Control "private, no-store" always;
|
||||
add_header X-Content-Type-Options "nosniff" always;
|
||||
add_header X-Frame-Options "DENY" always;
|
||||
|
||||
location = / {
|
||||
try_files /__no_report_at_root__ @combined_reports;
|
||||
}
|
||||
|
||||
location @combined_reports {
|
||||
return 302 /combined/;
|
||||
}
|
||||
|
||||
location / {
|
||||
try_files $uri $uri/ =404;
|
||||
}
|
||||
|
||||
location ~ /\. {
|
||||
deny all;
|
||||
}
|
||||
}
|
||||
|
||||
# IPv6 is deliberately omitted until the DNS AAAA record has been confirmed.
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"schemaVersion": 1,
|
||||
"stateDirectory": "/var/lib/hamradioonline-analytics",
|
||||
"counterStatePath": "/var/lib/hamradioonline-analytics/public-counter-state.json",
|
||||
"lockFile": "/run/hamradioonline-analytics/generator.lock",
|
||||
"geoIpCountryDatabase": "/var/lib/GeoIP/GeoLite2-Country.mmdb",
|
||||
"combined": {
|
||||
"reportOutputDirectory": "/var/lib/hamradioonline-analytics/reports/combined"
|
||||
},
|
||||
"sites": [
|
||||
{
|
||||
"id": "kst4contest",
|
||||
"hostname": "kst4contest.hamradioonline.de",
|
||||
"analyticsLog": "/var/log/nginx/kst4contest-analytics.log",
|
||||
"activatedOn": "2026-09-11",
|
||||
"publicCounter": true,
|
||||
"reportOutputDirectory": "/var/lib/hamradioonline-analytics/reports/kst4contest",
|
||||
"publicJsonPath": "/var/lib/hamradioonline-analytics/public/kst4contest/visitor-count.json"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
[Unit]
|
||||
Description=Generate private GoAccess reports and public project counters
|
||||
After=nginx.service
|
||||
|
||||
[Service]
|
||||
Type=oneshot
|
||||
User=hamradio-analytics
|
||||
Group=hamradio-analytics
|
||||
UMask=0027
|
||||
Environment=LC_TIME=C
|
||||
ExecStart=/usr/bin/node /opt/hamradioonline-analytics/generate-reports.js --registry /etc/hamradioonline-analytics/sites.json --config-template /etc/hamradioonline-analytics/goaccess.conf.template
|
||||
|
||||
NoNewPrivileges=true
|
||||
PrivateDevices=true
|
||||
PrivateNetwork=true
|
||||
ProtectClock=true
|
||||
ProtectControlGroups=true
|
||||
ProtectHome=true
|
||||
ProtectKernelLogs=true
|
||||
ProtectKernelModules=true
|
||||
ProtectKernelTunables=true
|
||||
ProtectSystem=strict
|
||||
PrivateTmp=true
|
||||
ProtectProc=invisible
|
||||
ProcSubset=pid
|
||||
RestrictAddressFamilies=AF_UNIX
|
||||
ReadOnlyPaths=/etc/hamradioonline-analytics /opt/hamradioonline-analytics /var/log/nginx /var/lib/GeoIP/GeoLite2-Country.mmdb
|
||||
ReadWritePaths=/var/lib/hamradioonline-analytics
|
||||
StateDirectory=hamradioonline-analytics
|
||||
StateDirectoryMode=0711
|
||||
RuntimeDirectory=hamradioonline-analytics
|
||||
RuntimeDirectoryMode=0750
|
||||
RestrictSUIDSGID=true
|
||||
LockPersonality=true
|
||||
@@ -0,0 +1,11 @@
|
||||
[Unit]
|
||||
Description=Run hamradioonline analytics once per hour
|
||||
|
||||
[Timer]
|
||||
OnCalendar=hourly
|
||||
Persistent=true
|
||||
RandomizedDelaySec=4m
|
||||
AccuracySec=1m
|
||||
|
||||
[Install]
|
||||
WantedBy=timers.target
|
||||
Generated
+3
@@ -7,6 +7,9 @@
|
||||
"": {
|
||||
"name": "kst4contest-website",
|
||||
"version": "0.1.0",
|
||||
"engines": {
|
||||
"node": ">=18.19.1"
|
||||
},
|
||||
"dependencies": {
|
||||
"markdown-it": "^14.3.0",
|
||||
"markdown-it-anchor": "^9.2.0"
|
||||
|
||||
@@ -2,6 +2,9 @@
|
||||
"name": "kst4contest-website",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"engines": {
|
||||
"node": ">=18.19.1"
|
||||
},
|
||||
"scripts": {
|
||||
"start": "eleventy --serve",
|
||||
"build": "eleventy && npm run validate:version-info",
|
||||
|
||||
@@ -81,5 +81,8 @@
|
||||
{% if heroFx %}
|
||||
<script src="/assets/js/hero-radio-fx.js?v={{ build.version }}" defer></script>
|
||||
{% endif %}
|
||||
{% if visitorCount %}
|
||||
<script src="/assets/js/visitor-count.js?v={{ build.version }}" defer></script>
|
||||
{% endif %}
|
||||
</body>
|
||||
</html>
|
||||
|
||||
@@ -659,6 +659,12 @@ a:hover {
|
||||
padding: clamp(28px, 5vw, 60px);
|
||||
}
|
||||
|
||||
.visitor-count {
|
||||
margin: 24px 0 0;
|
||||
color: var(--soft);
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.manual-content {
|
||||
max-width: 980px;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
"use strict";
|
||||
|
||||
const VISITOR_COUNT_ENDPOINT = "/visitor-count.json";
|
||||
const VISITOR_COUNT_TIMEOUT_MS = 2500;
|
||||
|
||||
function parseIsoDate(value) {
|
||||
const match = /^(\d{4})-(\d{2})-(\d{2})$/.exec(value);
|
||||
|
||||
if (!match) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const year = Number(match[1]);
|
||||
const month = Number(match[2]);
|
||||
const day = Number(match[3]);
|
||||
const date = new Date(Date.UTC(year, month - 1, day));
|
||||
|
||||
if (date.getUTCFullYear() !== year
|
||||
|| date.getUTCMonth() !== month - 1
|
||||
|| date.getUTCDate() !== day) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return date;
|
||||
}
|
||||
|
||||
function isIsoTimestamp(value) {
|
||||
if (typeof value !== "string") {
|
||||
return false;
|
||||
}
|
||||
|
||||
const match = /^(\d{4}-\d{2}-\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.\d+)?(Z|[+-](\d{2}):(\d{2}))$/.exec(value);
|
||||
|
||||
if (!match || !parseIsoDate(match[1])) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const hour = Number(match[2]);
|
||||
const minute = Number(match[3]);
|
||||
const second = Number(match[4]);
|
||||
const offsetHour = match[6] === undefined ? 0 : Number(match[6]);
|
||||
const offsetMinute = match[7] === undefined ? 0 : Number(match[7]);
|
||||
|
||||
return hour <= 23
|
||||
&& minute <= 59
|
||||
&& second <= 59
|
||||
&& offsetHour <= 23
|
||||
&& offsetMinute <= 59
|
||||
&& Number.isFinite(Date.parse(value));
|
||||
}
|
||||
|
||||
function validateVisitorCount(data) {
|
||||
if (!data || typeof data !== "object" || Array.isArray(data)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
if (data.schemaVersion !== 1
|
||||
|| !Number.isSafeInteger(data.visits)
|
||||
|| data.visits < 0
|
||||
|| !isIsoTimestamp(data.updatedAt)) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const since = typeof data.since === "string"
|
||||
? parseIsoDate(data.since)
|
||||
: null;
|
||||
|
||||
return since ? { visits: data.visits, since } : null;
|
||||
}
|
||||
|
||||
function formatVisitorCount(data) {
|
||||
const valid = validateVisitorCount(data);
|
||||
|
||||
if (!valid) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const date = new Intl.DateTimeFormat("en-GB", {
|
||||
day: "numeric",
|
||||
month: "long",
|
||||
year: "numeric",
|
||||
timeZone: "UTC"
|
||||
}).format(valid.since);
|
||||
const visits = new Intl.NumberFormat("en-GB").format(valid.visits);
|
||||
|
||||
return `Visits since ${date}: ${visits}`;
|
||||
}
|
||||
|
||||
async function loadVisitorCount({
|
||||
element,
|
||||
fetchImpl = globalThis.fetch,
|
||||
AbortControllerImpl = globalThis.AbortController,
|
||||
timeoutMs = VISITOR_COUNT_TIMEOUT_MS
|
||||
}) {
|
||||
if (!element || typeof fetchImpl !== "function"
|
||||
|| typeof AbortControllerImpl !== "function") {
|
||||
return false;
|
||||
}
|
||||
|
||||
element.hidden = true;
|
||||
element.textContent = "";
|
||||
|
||||
const controller = new AbortControllerImpl();
|
||||
const timeout = setTimeout(() => controller.abort(), timeoutMs);
|
||||
|
||||
try {
|
||||
const response = await fetchImpl(VISITOR_COUNT_ENDPOINT, {
|
||||
credentials: "omit",
|
||||
signal: controller.signal
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
return false;
|
||||
}
|
||||
|
||||
const text = formatVisitorCount(await response.json());
|
||||
|
||||
if (!text) {
|
||||
return false;
|
||||
}
|
||||
|
||||
element.textContent = text;
|
||||
element.hidden = false;
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
} finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof document !== "undefined") {
|
||||
const element = document.querySelector("[data-visitor-count]");
|
||||
|
||||
if (element) {
|
||||
loadVisitorCount({ element });
|
||||
}
|
||||
}
|
||||
|
||||
if (typeof module !== "undefined" && module.exports) {
|
||||
module.exports = {
|
||||
formatVisitorCount,
|
||||
loadVisitorCount,
|
||||
parseIsoDate,
|
||||
validateVisitorCount
|
||||
};
|
||||
}
|
||||
@@ -3,6 +3,7 @@ layout: base.njk
|
||||
title: KST4Contest – ON4KST Contest Client for VHF, UHF and SHF
|
||||
description: KST4Contest combines ON4KST chat, candidate prioritisation, sked planning, AirScout data and logger integration in one desktop client.
|
||||
heroFx: true
|
||||
visitorCount: true
|
||||
---
|
||||
|
||||
<section class="hero hero-split">
|
||||
@@ -150,6 +151,8 @@ heroFx: true
|
||||
<a class="button secondary" href="/roadmap/">Development status</a>
|
||||
<a class="button ghost" href="/support/">Support development</a>
|
||||
</div>
|
||||
|
||||
<p class="visitor-count" data-visitor-count hidden aria-live="polite"></p>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@@ -165,4 +168,4 @@ heroFx: true
|
||||
"description": "KST4Contest combines ON4KST chat, candidate prioritisation, sked planning, AirScout data and logger integration for VHF, UHF and SHF contest operation.",
|
||||
"license": "https://github.com/praktimarc/kst4contest/blob/main/LICENSE"
|
||||
}
|
||||
</script>
|
||||
</script>
|
||||
|
||||
@@ -7,7 +7,7 @@ description: Privacy policy for the KST4Contest website.
|
||||
<section class="hero">
|
||||
<p class="badge">Privacy</p>
|
||||
<h1>Privacy Policy</h1>
|
||||
<p class="lead">Information about data processing on this static website.</p>
|
||||
<p class="lead">Information about data processing on this website.</p>
|
||||
</section>
|
||||
|
||||
<section class="section narrow">
|
||||
@@ -30,12 +30,46 @@ description: Privacy policy for the KST4Contest website.
|
||||
<h2>Legal basis</h2>
|
||||
<p>
|
||||
The legal basis is Art. 6(1)(f) GDPR: legitimate interest in secure and reliable operation
|
||||
of the website.
|
||||
of the website and in understanding the approximate use of its project pages.
|
||||
</p>
|
||||
|
||||
<h2>Cookies and tracking</h2>
|
||||
<h2>Server-side reach statistics</h2>
|
||||
<p>
|
||||
This website currently does not use cookies, analytics tracking, advertising pixels or user profiling.
|
||||
The website uses server-side reach statistics to understand approximate demand and which
|
||||
project pages are used. Measurement is based on a dedicated, reduced server access log. It
|
||||
does not use cookies, an analytics or tracking script, advertising pixels, local storage,
|
||||
third-party requests or user profiles.
|
||||
</p>
|
||||
<p>
|
||||
The analytics log contains the server name, IP address, time, HTTP method, normalised path
|
||||
without a query string, protocol, response status, transferred size and user agent. It does
|
||||
not contain a referrer or authenticated user name. Known crawlers are filtered, and unknown
|
||||
browsers or operating systems are treated as crawlers.
|
||||
</p>
|
||||
<p>
|
||||
IP addresses are anonymised before data is stored in the statistics aggregates. The dedicated
|
||||
analytics raw logs are retained for 14 days. Anonymised detailed aggregates are retained on a
|
||||
rolling basis for 395 days. They include countries, but no city or host statistics. Countries
|
||||
are derived with a Country database only.
|
||||
</p>
|
||||
<p>
|
||||
A visit is an approximate daily value based on the combination of IP address, date and user
|
||||
agent used by GoAccess. It must not be read as a number of unique people. Page views and visits
|
||||
remain separate measures. Bots, monitoring requests, software update checks and static assets
|
||||
are excluded as far as they can be identified.
|
||||
</p>
|
||||
|
||||
<h2>Public visitor count</h2>
|
||||
<p>
|
||||
The home page displays an element containing the total number of these approximate daily visits
|
||||
since the stated activation date. The display loads a same-origin JSON file and remains hidden
|
||||
if that file is unavailable or invalid. This request is not included in the statistic.
|
||||
</p>
|
||||
<p>
|
||||
The public file contains only the counter value, activation date, schema version and update time.
|
||||
It contains no personal data or daily breakdown. The non-personal daily counter values are kept
|
||||
permanently so that the total does not decrease when detailed aggregates reach the end of their
|
||||
395-day retention period.
|
||||
</p>
|
||||
|
||||
<h2>External links</h2>
|
||||
@@ -71,4 +105,4 @@ description: Privacy policy for the KST4Contest website.
|
||||
You have the right to lodge a complaint with a competent data protection supervisory authority.
|
||||
</p>
|
||||
</article>
|
||||
</section>
|
||||
</section>
|
||||
|
||||
@@ -0,0 +1,519 @@
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs");
|
||||
const os = require("node:os");
|
||||
const path = require("node:path");
|
||||
const test = require("node:test");
|
||||
|
||||
const {
|
||||
formatGoAccessCheck,
|
||||
generateReports,
|
||||
parseGoAccessVersion
|
||||
} = require("../ops/analytics/generate-reports");
|
||||
|
||||
const GOACCESS_WITHOUT_ZLIB = {
|
||||
version: "1.8.1",
|
||||
geoIpMmdb: true,
|
||||
openSsl: true,
|
||||
zlib: false
|
||||
};
|
||||
|
||||
function goAccessReport(dailyVisits, combined = false) {
|
||||
const report = {
|
||||
general: { total_requests: 1 },
|
||||
visitors: {
|
||||
data: Object.entries(dailyVisits).map(([date, count]) => ({
|
||||
data: date.replaceAll("-", ""),
|
||||
visitors: { count }
|
||||
}))
|
||||
},
|
||||
requests: { data: [] },
|
||||
status_codes: { data: [] },
|
||||
geo_location: { data: [] }
|
||||
};
|
||||
if (combined) {
|
||||
report.virtual_hosts = { data: [] };
|
||||
}
|
||||
return report;
|
||||
}
|
||||
|
||||
function fixture(siteDefinitions) {
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), "kst4-analytics-test-"));
|
||||
const stateDirectory = path.join(root, "state");
|
||||
const geoIpCountryDatabase = path.join(root, "GeoLite2-Country.mmdb");
|
||||
const configTemplatePath = path.join(root, "goaccess.conf.template");
|
||||
const registryPath = path.join(root, "sites.json");
|
||||
fs.writeFileSync(geoIpCountryDatabase, "test database placeholder");
|
||||
fs.copyFileSync(
|
||||
path.join(__dirname, "../ops/analytics/goaccess.conf.template"),
|
||||
configTemplatePath
|
||||
);
|
||||
|
||||
const sites = siteDefinitions.map((definition, index) => {
|
||||
const id = definition.id || `site-${index}`;
|
||||
const log = path.join(root, `${id}.log`);
|
||||
fs.writeFileSync(log, "example log line\n");
|
||||
if (definition.rotated !== false) {
|
||||
fs.writeFileSync(`${log}.1`, "rotated example log line\n");
|
||||
}
|
||||
if (definition.compressed) {
|
||||
fs.writeFileSync(`${log}.2.gz`, "compressed placeholder\n");
|
||||
}
|
||||
return {
|
||||
id,
|
||||
hostname: definition.hostname || `${id}.example.test`,
|
||||
analyticsLog: log,
|
||||
activatedOn: definition.activatedOn || "2026-01-01",
|
||||
publicCounter: definition.publicCounter,
|
||||
reportOutputDirectory: path.join(stateDirectory, "reports", id),
|
||||
...(definition.publicCounter
|
||||
? { publicJsonPath: path.join(stateDirectory, "public", `${id}.json`) }
|
||||
: {})
|
||||
};
|
||||
});
|
||||
const registry = {
|
||||
schemaVersion: 1,
|
||||
stateDirectory,
|
||||
counterStatePath: path.join(stateDirectory, "counter-state.json"),
|
||||
lockFile: path.join(root, "run", "generator.lock"),
|
||||
geoIpCountryDatabase,
|
||||
combined: {
|
||||
reportOutputDirectory: path.join(stateDirectory, "reports", "combined")
|
||||
},
|
||||
sites
|
||||
};
|
||||
fs.mkdirSync(stateDirectory, { recursive: true });
|
||||
fs.mkdirSync(registry.combined.reportOutputDirectory, { recursive: true });
|
||||
for (const site of sites) {
|
||||
fs.mkdirSync(site.reportOutputDirectory, { recursive: true });
|
||||
if (site.publicCounter) {
|
||||
fs.mkdirSync(path.dirname(site.publicJsonPath), { recursive: true });
|
||||
}
|
||||
}
|
||||
fs.writeFileSync(registryPath, JSON.stringify(registry));
|
||||
|
||||
return {
|
||||
root,
|
||||
registry,
|
||||
registryPath,
|
||||
configTemplatePath,
|
||||
cleanup: () => fs.rmSync(root, { recursive: true, force: true })
|
||||
};
|
||||
}
|
||||
|
||||
function fakeGoAccess(reports, calls, failureId) {
|
||||
return invocation => {
|
||||
calls.push(invocation);
|
||||
if (invocation.id === failureId) {
|
||||
throw new Error("simulated GoAccess failure");
|
||||
}
|
||||
fs.writeFileSync(
|
||||
invocation.outputJson,
|
||||
JSON.stringify(reports[invocation.id])
|
||||
);
|
||||
fs.writeFileSync(
|
||||
invocation.outputHtml,
|
||||
`<!doctype html><html><body>${"report".repeat(30)}</body></html>`
|
||||
);
|
||||
fs.writeFileSync(path.join(invocation.dbPath, "persisted.db"), invocation.id);
|
||||
};
|
||||
}
|
||||
|
||||
function run(testFixture, reports, calls = [], failureId) {
|
||||
return generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
goaccessBinary: "fake-goaccess"
|
||||
}, {
|
||||
checkGoAccess: () => GOACCESS_WITHOUT_ZLIB,
|
||||
runGoAccess: fakeGoAccess(reports, calls, failureId),
|
||||
now: () => new Date("2026-09-11T07:00:00Z"),
|
||||
skipLock: true
|
||||
});
|
||||
}
|
||||
|
||||
test("replaces repeated daily values and retains older public days", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
fs.mkdirSync(path.dirname(testFixture.registry.counterStatePath), { recursive: true });
|
||||
fs.writeFileSync(testFixture.registry.counterStatePath, JSON.stringify({
|
||||
schemaVersion: 1,
|
||||
sites: {
|
||||
alpha: {
|
||||
hostname: "alpha.example.test",
|
||||
since: "2026-01-01",
|
||||
dailyVisits: { "2026-01-01": 7 }
|
||||
}
|
||||
}
|
||||
}));
|
||||
|
||||
run(testFixture, {
|
||||
alpha: goAccessReport({ "2026-09-11": 3 }),
|
||||
combined: goAccessReport({ "2026-09-11": 3 }, true)
|
||||
});
|
||||
run(testFixture, {
|
||||
alpha: goAccessReport({ "2026-09-11": 5 }),
|
||||
combined: goAccessReport({ "2026-09-11": 5 }, true)
|
||||
});
|
||||
|
||||
const state = JSON.parse(fs.readFileSync(
|
||||
testFixture.registry.counterStatePath,
|
||||
"utf8"
|
||||
));
|
||||
const published = JSON.parse(fs.readFileSync(
|
||||
testFixture.registry.sites[0].publicJsonPath,
|
||||
"utf8"
|
||||
));
|
||||
assert.deepEqual(state.sites.alpha.dailyVisits, {
|
||||
"2026-01-01": 7,
|
||||
"2026-09-11": 5
|
||||
});
|
||||
assert.deepEqual(published, {
|
||||
schemaVersion: 1,
|
||||
visits: 12,
|
||||
since: "2026-01-01",
|
||||
updatedAt: "2026-09-11T07:00:00Z"
|
||||
});
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("keeps valid outputs unchanged when a GoAccess job fails", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
const reportDirectory = testFixture.registry.sites[0].reportOutputDirectory;
|
||||
fs.mkdirSync(reportDirectory, { recursive: true });
|
||||
fs.mkdirSync(path.dirname(testFixture.registry.counterStatePath), { recursive: true });
|
||||
fs.mkdirSync(path.dirname(testFixture.registry.sites[0].publicJsonPath), { recursive: true });
|
||||
fs.writeFileSync(path.join(reportDirectory, "report.json"), "old-json");
|
||||
fs.writeFileSync(path.join(reportDirectory, "report.html"), "old-html");
|
||||
fs.writeFileSync(testFixture.registry.counterStatePath, "old-state");
|
||||
fs.writeFileSync(testFixture.registry.sites[0].publicJsonPath, "old-public");
|
||||
|
||||
assert.throws(() => run(testFixture, {
|
||||
alpha: goAccessReport({ "2026-09-11": 3 }),
|
||||
combined: goAccessReport({ "2026-09-11": 3 }, true)
|
||||
}, [], "combined"), /simulated GoAccess failure/);
|
||||
|
||||
assert.equal(fs.readFileSync(path.join(reportDirectory, "report.json"), "utf8"), "old-json");
|
||||
assert.equal(fs.readFileSync(path.join(reportDirectory, "report.html"), "utf8"), "old-html");
|
||||
assert.equal(fs.readFileSync(testFixture.registry.counterStatePath, "utf8"), "old-state");
|
||||
assert.equal(fs.readFileSync(testFixture.registry.sites[0].publicJsonPath, "utf8"), "old-public");
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("processes subdomains separately and together without publishing disabled counters", () => {
|
||||
const testFixture = fixture([
|
||||
{ id: "alpha", publicCounter: true, compressed: true },
|
||||
{ id: "bravo", publicCounter: false, rotated: false, compressed: true }
|
||||
]);
|
||||
const calls = [];
|
||||
try {
|
||||
const result = run(testFixture, {
|
||||
alpha: goAccessReport({ "2026-09-11": 3 }),
|
||||
bravo: goAccessReport({ "2026-09-11": 4 }),
|
||||
combined: goAccessReport({ "2026-09-11": 6 }, true)
|
||||
}, calls);
|
||||
|
||||
assert.equal(result.reports, 3);
|
||||
assert.deepEqual(calls.map(call => call.id), ["alpha", "bravo", "combined"]);
|
||||
const alphaLogs = [
|
||||
`${testFixture.registry.sites[0].analyticsLog}.1`,
|
||||
testFixture.registry.sites[0].analyticsLog
|
||||
];
|
||||
const bravoLogs = [testFixture.registry.sites[1].analyticsLog];
|
||||
assert.deepEqual(calls[0].args.slice(0, 2), alphaLogs);
|
||||
assert.deepEqual(calls[1].args.slice(0, 1), bravoLogs);
|
||||
assert.deepEqual(
|
||||
calls[2].args.slice(0, 3),
|
||||
[...alphaLogs, ...bravoLogs]
|
||||
);
|
||||
assert.equal(calls.some(call => call.args.some(argument => argument.endsWith(".gz"))), false);
|
||||
assert.equal(fs.existsSync(path.join(
|
||||
testFixture.registry.stateDirectory,
|
||||
"public",
|
||||
"bravo.json"
|
||||
)), false);
|
||||
assert.equal(fs.existsSync(path.join(
|
||||
testFixture.registry.combined.reportOutputDirectory,
|
||||
"report.html"
|
||||
)), true);
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("dry-run validates generated data without changing production paths", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
const blockedLockParent = path.join(testFixture.root, "blocked-lock-parent");
|
||||
fs.writeFileSync(blockedLockParent, "not a directory");
|
||||
testFixture.registry.lockFile = path.join(blockedLockParent, "generator.lock");
|
||||
fs.writeFileSync(testFixture.registryPath, JSON.stringify(testFixture.registry));
|
||||
|
||||
const result = generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
goaccessBinary: "fake-goaccess",
|
||||
dryRun: true
|
||||
}, {
|
||||
checkGoAccess: () => GOACCESS_WITHOUT_ZLIB,
|
||||
runGoAccess: fakeGoAccess({
|
||||
alpha: goAccessReport({ "2026-09-11": 3 }),
|
||||
combined: goAccessReport({ "2026-09-11": 3 }, true)
|
||||
}, []),
|
||||
now: () => new Date("2026-09-11T07:00:00Z")
|
||||
});
|
||||
|
||||
assert.equal(result.dryRun, true);
|
||||
assert.equal(fs.statSync(blockedLockParent).isFile(), true);
|
||||
assert.equal(fs.readdirSync(
|
||||
testFixture.registry.sites[0].reportOutputDirectory
|
||||
).length, 0);
|
||||
assert.equal(fs.existsSync(testFixture.registry.sites[0].publicJsonPath), false);
|
||||
assert.equal(fs.existsSync(testFixture.registry.counterStatePath), false);
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("configuration check accepts and explains a GoAccess build without Zlib", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true, rotated: false }]);
|
||||
try {
|
||||
const versionOutput = [
|
||||
"GoAccess - 1.8.1.",
|
||||
"Build configure arguments:",
|
||||
" --enable-geoip=mmdb",
|
||||
" --with-openssl"
|
||||
].join("\n");
|
||||
const capabilities = parseGoAccessVersion(versionOutput);
|
||||
const result = generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
goaccessBinary: "fake-goaccess",
|
||||
check: true
|
||||
}, {
|
||||
checkGoAccess: () => capabilities,
|
||||
runGoAccess: () => {
|
||||
throw new Error("configuration check must not generate reports");
|
||||
}
|
||||
});
|
||||
const message = formatGoAccessCheck(result.goAccess);
|
||||
|
||||
assert.deepEqual(capabilities, GOACCESS_WITHOUT_ZLIB);
|
||||
assert.equal(result.checked, true);
|
||||
assert.match(message, /Zlib not detected/);
|
||||
assert.match(message, /no Zlib support/);
|
||||
assert.match(message, /valid for regular mode/);
|
||||
assert.match(message, /optional uncompressed \.1 rotation/);
|
||||
assert.match(message, /older \.gz logs are not imported/);
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("detects Zlib support when it is present", () => {
|
||||
const capabilities = parseGoAccessVersion([
|
||||
"GoAccess - 1.8.1.",
|
||||
"Build configure arguments:",
|
||||
" --enable-geoip=mmdb --with-openssl --with-zlib"
|
||||
].join("\n"));
|
||||
|
||||
assert.equal(capabilities.zlib, true);
|
||||
assert.match(formatGoAccessCheck(capabilities), /Zlib enabled/);
|
||||
assert.doesNotMatch(formatGoAccessCheck(capabilities), /older \.gz logs/);
|
||||
});
|
||||
|
||||
test("configuration check rejects GoAccess without GeoIP2 MMDB support", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
assert.throws(() => generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
check: true
|
||||
}, {
|
||||
checkGoAccess: () => ({
|
||||
version: "1.8.1",
|
||||
geoIpMmdb: false,
|
||||
openSsl: true,
|
||||
zlib: false
|
||||
})
|
||||
}), error => error.exitCode === 2
|
||||
&& /must be built with GeoIP2\/MMDB support/.test(error.message));
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("configuration check verifies analytics-log readability", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true, rotated: false }]);
|
||||
const originalAccessSync = fs.accessSync;
|
||||
try {
|
||||
fs.accessSync = (filePath, mode) => {
|
||||
if (filePath === testFixture.registry.sites[0].analyticsLog
|
||||
&& mode === fs.constants.R_OK) {
|
||||
const error = new Error("permission denied");
|
||||
error.code = "EACCES";
|
||||
throw error;
|
||||
}
|
||||
return originalAccessSync(filePath, mode);
|
||||
};
|
||||
|
||||
assert.throws(() => generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
check: true
|
||||
}, {
|
||||
checkGoAccess: () => GOACCESS_WITHOUT_ZLIB
|
||||
}), /analytics log is not readable by the current user/);
|
||||
} finally {
|
||||
fs.accessSync = originalAccessSync;
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("configuration check reports missing output directories", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
fs.rmSync(testFixture.registry.sites[0].reportOutputDirectory, {
|
||||
recursive: true,
|
||||
force: true
|
||||
});
|
||||
|
||||
assert.throws(() => generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
check: true
|
||||
}, {
|
||||
checkGoAccess: () => GOACCESS_WITHOUT_ZLIB
|
||||
}), /report output directory for alpha does not exist/);
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("rejects a compressed log as the regular analytics input", () => {
|
||||
const testFixture = fixture([{ id: "alpha", publicCounter: true }]);
|
||||
try {
|
||||
testFixture.registry.sites[0].analyticsLog += ".2.gz";
|
||||
fs.writeFileSync(testFixture.registryPath, JSON.stringify(testFixture.registry));
|
||||
|
||||
assert.throws(() => generateReports({
|
||||
registryPath: testFixture.registryPath,
|
||||
configTemplatePath: testFixture.configTemplatePath,
|
||||
check: true
|
||||
}, {
|
||||
checkGoAccess: () => GOACCESS_WITHOUT_ZLIB
|
||||
}), /must identify the current uncompressed log/);
|
||||
} finally {
|
||||
testFixture.cleanup();
|
||||
}
|
||||
});
|
||||
|
||||
test("Nginx filters exclude non-page traffic before analytics logging", () => {
|
||||
const source = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/nginx/analytics-filters.conf.example"
|
||||
), "utf8");
|
||||
|
||||
for (const excluded of [
|
||||
"/visitor-count.json",
|
||||
"/kst4ContestVersionInfo.xml",
|
||||
"/sitemap.xml",
|
||||
"/robots.txt",
|
||||
"manual/assets",
|
||||
"healthz"
|
||||
]) {
|
||||
assert.match(source, new RegExp(excluded.replaceAll("/", "\\/"), "i"));
|
||||
}
|
||||
assert.doesNotMatch(source, /^\s*=\//m);
|
||||
assert.match(source, /^\s*\/visitor-count\.json\s+0;/m);
|
||||
assert.match(source, /^\s*\/health\s+0;/m);
|
||||
assert.match(source, /\|map\|/);
|
||||
assert.match(source, /\$uri/);
|
||||
assert.match(source, /known_bot/);
|
||||
});
|
||||
|
||||
test("server templates use the production GeoIP path and permission model", () => {
|
||||
const registry = JSON.parse(fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/sites.example.json"
|
||||
), "utf8"));
|
||||
const service = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/systemd/hamradioonline-analytics.service.example"
|
||||
), "utf8");
|
||||
const generator = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/generate-reports.js"
|
||||
), "utf8");
|
||||
|
||||
assert.equal(registry.geoIpCountryDatabase, "/var/lib/GeoIP/GeoLite2-Country.mmdb");
|
||||
assert.match(service, /\/var\/lib\/GeoIP\/GeoLite2-Country\.mmdb/);
|
||||
assert.match(service, /^StateDirectoryMode=0711$/m);
|
||||
assert.match(generator, /fs\.fchmodSync\(handle, mode\)/);
|
||||
assert.match(generator, /publicFile\.destination, publicFile\.content, 0o644/);
|
||||
});
|
||||
|
||||
test("public counter template disables logging and sets cache and type headers", () => {
|
||||
const source = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/nginx/public-counter.conf.example"
|
||||
), "utf8");
|
||||
|
||||
assert.match(source, /^\s*access_log off;$/m);
|
||||
assert.match(source, /Cache-Control "public, max-age=3600" always/);
|
||||
assert.match(source, /X-Content-Type-Options "nosniff" always/);
|
||||
});
|
||||
|
||||
test("statistics vhost preserves ACME renewal and protects HTTPS reports", () => {
|
||||
const source = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/nginx/stats-vhost.conf.example"
|
||||
), "utf8");
|
||||
const bootstrap = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/nginx/stats-vhost-http-bootstrap.conf.example"
|
||||
), "utf8");
|
||||
const httpsOffset = source.search(/server\s*\{\s*listen 443 ssl http2;/);
|
||||
assert.notEqual(httpsOffset, -1);
|
||||
const httpSource = source.slice(0, httpsOffset);
|
||||
const httpsSource = source.slice(httpsOffset);
|
||||
|
||||
assert.match(httpSource, /^\s*listen 80;$/m);
|
||||
assert.match(httpSource, /^\s*access_log off;$/m);
|
||||
assert.match(httpSource, /location \^~ \/\.well-known\/acme-challenge\//);
|
||||
assert.match(httpSource, /root \/var\/lib\/letsencrypt;/);
|
||||
assert.match(httpSource, /return 301 https:\/\/\$host\$request_uri;/);
|
||||
assert.match(httpsSource, /^\s*listen 443 ssl http2;$/m);
|
||||
assert.match(httpsSource, /^\s*auth_basic "Private project statistics";$/m);
|
||||
assert.match(httpsSource, /^\s*access_log off;$/m);
|
||||
assert.match(httpsSource, /location = \/[\s\S]*return 302 \/combined\/;/);
|
||||
assert.match(httpsSource, /include \/etc\/letsencrypt\/options-ssl-nginx\.conf;/);
|
||||
assert.match(httpsSource, /ssl_dhparam \/etc\/letsencrypt\/ssl-dhparams\.pem;/);
|
||||
assert.doesNotMatch(source, /listen \[::\]/);
|
||||
assert.match(bootstrap, /^\s*listen 80;$/m);
|
||||
assert.doesNotMatch(bootstrap, /ssl_certificate/);
|
||||
});
|
||||
|
||||
test("logrotate uses the Ubuntu Nginx rotation action", () => {
|
||||
const source = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../ops/analytics/logrotate/hamradioonline-analytics.example"
|
||||
), "utf8");
|
||||
|
||||
assert.match(source, /invoke-rc\.d nginx rotate >\/dev\/null 2>&1/);
|
||||
assert.match(source, /^\s*delaycompress$/m);
|
||||
assert.match(source, /^\s*rotate 14$/m);
|
||||
assert.match(source, /^\s*create 0640 www-data hamradio-analytics$/m);
|
||||
});
|
||||
|
||||
test("website package records the Node.js 18.19.1 baseline", () => {
|
||||
const packageJson = JSON.parse(fs.readFileSync(path.join(__dirname, "../package.json")));
|
||||
const packageLock = JSON.parse(fs.readFileSync(path.join(__dirname, "../package-lock.json")));
|
||||
|
||||
assert.equal(packageJson.engines.node, ">=18.19.1");
|
||||
assert.equal(packageLock.packages[""].engines.node, ">=18.19.1");
|
||||
});
|
||||
@@ -0,0 +1,119 @@
|
||||
const assert = require("node:assert/strict");
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const test = require("node:test");
|
||||
|
||||
const {
|
||||
formatVisitorCount,
|
||||
loadVisitorCount,
|
||||
validateVisitorCount
|
||||
} = require("../src/assets/js/visitor-count");
|
||||
|
||||
const VALID_DATA = {
|
||||
schemaVersion: 1,
|
||||
visits: 1234,
|
||||
since: "2026-09-11",
|
||||
updatedAt: "2026-09-11T07:00:00Z"
|
||||
};
|
||||
|
||||
function element() {
|
||||
return { hidden: true, textContent: "" };
|
||||
}
|
||||
|
||||
test("formats valid data for the English website", async () => {
|
||||
const target = element();
|
||||
let request;
|
||||
const shown = await loadVisitorCount({
|
||||
element: target,
|
||||
fetchImpl: async (url, options) => {
|
||||
request = { url, options };
|
||||
return { ok: true, json: async () => VALID_DATA };
|
||||
}
|
||||
});
|
||||
|
||||
assert.equal(shown, true);
|
||||
assert.equal(target.hidden, false);
|
||||
assert.equal(target.textContent, "Visits since 11 September 2026: 1,234");
|
||||
assert.equal(request.url, "/visitor-count.json");
|
||||
assert.equal(request.options.credentials, "omit");
|
||||
});
|
||||
|
||||
test("rejects unsupported schemas and invalid visit counts", () => {
|
||||
for (const changes of [
|
||||
{ schemaVersion: 2 },
|
||||
{ visits: -1 },
|
||||
{ visits: 1.5 },
|
||||
{ visits: Number.MAX_SAFE_INTEGER + 1 }
|
||||
]) {
|
||||
assert.equal(validateVisitorCount({ ...VALID_DATA, ...changes }), null);
|
||||
}
|
||||
});
|
||||
|
||||
test("rejects invalid dates and timestamps without timezone shifts", () => {
|
||||
for (const changes of [
|
||||
{ since: "2026-02-30" },
|
||||
{ since: "11-09-2026" },
|
||||
{ updatedAt: "2026-09-11" },
|
||||
{ updatedAt: "2026-02-30T07:00:00Z" },
|
||||
{ updatedAt: "2026-09-11T25:00:00Z" },
|
||||
{ updatedAt: "not-a-timestamp" }
|
||||
]) {
|
||||
assert.equal(formatVisitorCount({ ...VALID_DATA, ...changes }), null);
|
||||
}
|
||||
});
|
||||
|
||||
test("keeps the element hidden for HTTP and fetch failures", async () => {
|
||||
for (const fetchImpl of [
|
||||
async () => ({ ok: false }),
|
||||
async () => { throw new Error("network failure"); }
|
||||
]) {
|
||||
const target = element();
|
||||
assert.equal(await loadVisitorCount({ element: target, fetchImpl }), false);
|
||||
assert.equal(target.hidden, true);
|
||||
assert.equal(target.textContent, "");
|
||||
}
|
||||
});
|
||||
|
||||
test("aborts a slow request and keeps the element hidden", async () => {
|
||||
const target = element();
|
||||
let aborted = false;
|
||||
const fetchImpl = (url, options) => new Promise((resolve, reject) => {
|
||||
options.signal.addEventListener("abort", () => {
|
||||
aborted = true;
|
||||
reject(new Error("aborted"));
|
||||
});
|
||||
});
|
||||
|
||||
assert.equal(await loadVisitorCount({
|
||||
element: target,
|
||||
fetchImpl,
|
||||
timeoutMs: 5
|
||||
}), false);
|
||||
assert.equal(aborted, true);
|
||||
assert.equal(target.hidden, true);
|
||||
});
|
||||
|
||||
test("uses textContent and loads the script only for the home page", () => {
|
||||
const script = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../src/assets/js/visitor-count.js"
|
||||
), "utf8");
|
||||
const home = fs.readFileSync(path.join(__dirname, "../src/index.njk"), "utf8");
|
||||
const layout = fs.readFileSync(path.join(
|
||||
__dirname,
|
||||
"../src/_layouts/base.njk"
|
||||
), "utf8");
|
||||
const sourceRoot = path.join(__dirname, "../src");
|
||||
const otherPages = fs.readdirSync(sourceRoot, { recursive: true })
|
||||
.filter(entry => entry.endsWith(".njk") && entry !== "index.njk");
|
||||
|
||||
assert.doesNotMatch(script, /innerHTML/);
|
||||
assert.match(script, /textContent/);
|
||||
assert.match(home, /^visitorCount: true$/m);
|
||||
assert.match(layout, /\{% if visitorCount %\}/);
|
||||
assert.equal(
|
||||
otherPages.some(entry => fs.readFileSync(path.join(sourceRoot, entry), "utf8")
|
||||
.includes("visitorCount: true")),
|
||||
false
|
||||
);
|
||||
});
|
||||
Reference in New Issue
Block a user