Add post on GoAccess log format for OpenBSD httpd

1 files changed, 57 insertions, 0 deletions

diff --git a/posts/goaccess-log-format-for-openbsd-httpd.md b/posts/goaccess-log-format-for-openbsd-httpd.md
new file mode 100644
index 0000000..e5c79cf
--- /dev/null
+++ b/posts/goaccess-log-format-for-openbsd-httpd.md
@@ -0,0 +1,57 @@
+---
+title: "GoAccess log format for OpenBSD httpd"
+description: "How to import OpenBSD httpd access logs into GoAccess."
+published: 2021-05-14
+---
+
+[GoAccess](https://goaccess.io/) is a neat HTTP access log reporting tool for the command-line.
+In this post I'll briefly describe how to import access logs generated by OpenBSD's built-in HTTP server.
+
+OpenBSD's [`httpd(8)`](https://man.openbsd.org/OpenBSD-6.9/httpd) supports four different log styles:
+The *common* and *combined* log styles are similar to the de facto standard access log formats of the [Apache HTTP Server](https://httpd.apache.org/docs/2.4/logs.html#accesslog).
+The *forwarded* log style extends the combined log style by appending the widespread `X-Forwarded-For` and `X-Forwarded-Port` headers.
+In contrast to this, the *connection* log style writes a summary of all requests per connection.
+I won't go into the details of the connection log style here.
+You can find the relevant source on the [CVS](https://cvsweb.openbsd.org/src/usr.sbin/httpd/server_http.c?rev=1.143&content-type=text/x-cvsweb-markup) and on the [GitHub mirror](https://github.com/openbsd/src/blob/4e551392332139eb053d51857a69c1d83e2ede2c/usr.sbin/httpd/server_http.c).
+
+A typical forwarded style log message looks like this:
+
+    www.example.com 127.0.0.1 - - [14/May/2021:13:26:56 +0000] "GET /posts/ HTTP/1.1" 200 2066 "https://www.example.com/" "Mozilla/5.0 (X11; OpenBSD amd64; rv:88.0) Gecko/20100101 Firefox/88.0" 10.146.199.139 -
+
+It consists of twelve elments:
+
+1. Server name, alias virtual host
+1. Client IP address, alias remote host
+1. [RFC 1413](https://www.rfc-editor.org/rfc/rfc1413.html) user identity, if any
+1. HTTP authentication user identity, if any
+1. Date and time
+1. Request method, path, and protocol
+1. Response status code
+1. Response body size in bytes
+1. Referer header, if any
+1. User-Agent header, if any
+1. X-Forwarded-For header, if any
+1. X-Forwarded-Port header, if any
+
+The corresponding format strings for GoAccess are:
+
+* `%v %h %^ %e [%d:%t] "%r" %s %b` for the common log style
+* `%v %h %^ %e [%d:%t] "%r" %s %b %R %u` for the combined log style
+* `%v %^ %^ %e [%d:%t] "%r" %s %b %R %u ~h{," } %^` for the forwarded log style
+* `%d/%b/%Y` for the date
+* `%H:%M:%S %z` for the time
+
+For example, you can import forwarded style access logs as follows:
+
+    $ zcat -f /var/www/logs/access.log* \
+        | grep -v 'logfile turned over$' \
+        | awk '$8=$1$8' \
+        | goaccess \
+            --no-global-config \
+            --log-format='%v %^ %^ %e [%d:%t] "%r" %s %b %R %u ~h{," } %^' \
+            --date-format='%d/%b/%Y' \
+            --time-format='%H:%M:%S %z' \
+            --no-color
+
+This pipeline uncompresses and concatenates all access logs in the default location, filters log file rotation messages, prepends the server name to the request path, and imports the result into GoAccess to show an interactive command-line report.
+Append `--output=access.html` to generate a pretty HTML report instead.