Post OpenBSD on QEMU

1 files changed, 242 insertions, 0 deletions

diff --git a/posts/autoinstall-openbsd-on-qemu.md b/posts/autoinstall-openbsd-on-qemu.md
new file mode 100644
index 0000000..ca80cfa
--- /dev/null
+++ b/posts/autoinstall-openbsd-on-qemu.md
@@ -0,0 +1,242 @@
+---
+title: Auto-install OpenBSD on QEMU
+description: How to perform an unattended installation of OpenBSD on the QEMU virtual machine monitor.
+published: 2020-07-22
+---
+
+I happen to develop a pet project of mine on a Linux desktop, while actually targeting an [OpenBSD](https://www.openbsd.org/) server.
+Thus I searched for a scriptable way to install OpenBSD on the [QEMU](https://www.qemu.org/) virtual machine manager, such that I could automate a local integration test against OpenBSD running on Linux.
+
+As expected, OpenBSD has a remarkably straightforward unattended installation solution.
+During a normal, interactive installation, you answer a series of questions like what timezone you are in or which file sets to install.
+At the end of the installation, the installer sends a recorded list of those questions along with your answers to the root user's mail box.
+You can then adapt this so-called _response file_ to your needs and feed it to the [autoinstall(8)](https://man.openbsd.org/OpenBSD-6.7/autoinstall) command to perform an unattended installation.
+
+In the remainder of this post, I will show how to auto-install OpenBSD/amd64 6.7 on the QEMU PC system emulator 5.0.0.
+In the end, I will present a simple yet [complete POSIX shell script](/files/autoinstall-openbsd-on-qemu.sh) to get the job done.
+The script is intended to run on Linux, though.
+If you already have a running OpenBSD installation, you should consider to use OpenBSD's own hypervisor [vmm(4)](https://man.openbsd.org/OpenBSD-6.7/vmm) instead of QEMU as described in the [OpenBSD FAQ](https://www.openbsd.org/faq/faq16.html) and in this [blog post](https://eradman.com/posts/autoinstall-openbsd.html) by Eric Radman.
+
+We will perform the following steps:
+
+1. Install several prerequisites
+1. Setup a local OpenBSD mirror
+1. Configure the installation
+1. Setup a network boot environment
+1. Install the virtual machine
+1. Log in to the virtual machine
+
+## Prerequisites
+
+We will use the following tools:
+
+* [curl](https://curl.haxx.se/), a data transfer tool (and library)
+* [OpenSSH](https://www.openssh.com/), a remote login tool
+* [QEMU](https://www.qemu.org/), a virtual machine monitor (or hypervisor)
+* [rsync](https://rsync.samba.org/), an incremental file transfer tool
+* [signify](https://github.com/aperezdc/signify), a cryptographic signature tool[^signify-portable]
+* [socat](http://www.dest-unreach.org/socat/), a successor of the infamous TCP/IP Swiss Army knife [netcat](https://nc110.sourceforge.io/)
+
+Chances are that your Linux distribution of choice packages these tools.
+For example, the following command installs them on Arch Linux:
+
+    $ sudo pacman -S curl openssh qemu rsync signify socat
+
+## Local OpenBSD mirror
+
+To begin with, we setup a partial, local [OpenBSD mirror](https://www.openbsd.org/ftp.html).
+
+First, we create the relevant part of the [directory layout](https://www.openbsd.org/ftp.html#layout):
+
+    $ mkdir -p mirror/pub/OpenBSD/6.7/amd64
+
+Second, we fetch the base public key from the official HTTPS mirror using [curl(1)](https://curl.haxx.se/docs/manpage.html):
+
+    $ curl \
+        --output mirror/pub/OpenBSD/6.7/openbsd-67-base.pub \
+        https://ftp.openbsd.org/pub/OpenBSD/6.7/openbsd-67-base.pub
+
+Third, we fetch the kernel, the [PXE](https://en.wikipedia.org/w/index.php?title=Preboot_Execution_Environment&oldid=955913424) bootstrap program, and the file sets from an untrusted [rsync mirror](https://www.openbsd.org/ftp.html#rsync) using [rsync(1)](https://download.samba.org/pub/rsync/rsync.1):
+
+    $ rsync --archive --files-from=- --verbose \
+        rsync://ftp.halifax.rwth-aachen.de/openbsd/6.7/amd64/ \
+        mirror/pub/OpenBSD/6.7/amd64 \
+        << EOF
+    SHA256.sig
+    base67.tgz
+    bsd
+    bsd.mp
+    bsd.rd
+    comp67.tgz
+    game67.tgz
+    man67.tgz
+    pxeboot
+    xbase67.tgz
+    xfont67.tgz
+    xserv67.tgz
+    xshare67.tgz
+    EOF
+
+Fourth, we verify the fetched files using [signify(1)](https://man.openbsd.org/OpenBSD-6.7/signify) and the previously fetched base public key:[^sha256]
+
+    $ ( cd mirror/pub/OpenBSD/6.7/amd64 && signify -C \
+        -p ../openbsd-67-base.pub \
+        -x SHA256.sig \
+        -- bsd bsd.* pxeboot *67.tgz )
+
+Finally, we serve the local mirror at <http://127.0.0.1:8080/>.
+Feel free to use your web server of choice.
+Chances are, that your Linux distribution comes with Python's [http.server](https://docs.python.org/3/library/http.server.html) module:
+
+    $ python3 \
+        -m http.server \
+        --directory mirror \
+        --bind 127.0.0.1 8080
+
+## Configuration
+
+First, we create a response file for [autoinstall(8)](https://man.openbsd.org/OpenBSD-6.7/autoinstall) at `mirror/install.conf`:[^response-file]
+
+    Change the default console to com0 = yes
+    Which speed should com0 use = 115200
+    System hostname = openbsd
+    Password for root = *************
+    Allow root ssh login = no
+    Setup a user = puffy
+    Password for user = *************
+    Public ssh key for user = ssh-rsa AAAAB3N... alex@example
+    What timezone are you in = UTC
+    Location of sets = http
+    HTTP Server = 10.0.2.1
+    Unable to connect using https. Use http instead = yes
+    URL to autopartitioning template for disklabel = http://10.0.2.1/disklabel
+    Set name(s) = site67.tgz
+    Checksum test for site67.tgz failed. Continue anyway = yes
+    Unverified sets: site67.tgz. Continue without verification = yes
+
+Take care to insert your own public SSH key here, for example, the contents of `~/.ssh/id_rsa.pub`.
+
+Note that we effectively disable password-based authentication here by assigning the conventional 13 asterisks as encrypted passwords for both users, `root` and `puffy`, see [passwd(5)](https://man.openbsd.org/OpenBSD-6.7/passwd.5).
+Instead, we enable the user `puffy` to login using the given SSH key.
+Besides, we will later permit the user `puffy` to run any command as root without entering his password using [doas(1)](https://man.openbsd.org/OpenBSD-6.7/doas).
+
+Note also that we will later instruct QEMU to redirect port 80 on the virtual network address 10.0.2.1 to port 8080 on the local host.
+
+Next, we create a [disklabel(8)](https://man.openbsd.org/OpenBSD-6.7/disklabel) template at `mirror/disklabel`:
+
+    /            2G
+    swap         8G
+    /tmp         1G
+    /var         1G
+    /usr         2G
+    /usr/X11R6 500M
+    /usr/local   4G
+    /usr/src     1M
+    /usr/obj     1M
+    /home        4G
+
+Finally, we create an optional site-specific file set.
+This way, we can run some commands at the end of the installation.
+Here, we reset the OpenBSD mirror server used by [pkg_add(1)](https://man.openbsd.org/OpenBSD-6.7/pkg_add) and other commands.[^restrict-network]
+Moreover, we permit the user group `wheel` --- and thus the user `puffy` --- to run any command as the user `root` without entering their password using [doas(1)](https://man.openbsd.org/OpenBSD-6.7/doas).
+
+Create the file `site/install.site`:
+
+```
+#! /bin/ksh
+set -o errexit
+echo "https://cdn.openbsd.org/pub/OpenBSD" > /etc/installurl
+echo "permit nopass keepenv :wheel" >> /etc/doas.conf
+```
+
+Then, make the file executable, package the file set, and add it to the local OpenBSD mirror:
+
+    $ chmod +x site/install.site
+    $ ( cd site && tar -czf ../mirror/pub/OpenBSD/6.7/amd64/site67.tgz . )
+    $ ( cd mirror/pub/OpenBSD/6.7/amd64 && ls -l > index.txt )
+
+## Network boot environment
+
+We create a dedicated directory to serve the OpenBSD kernel and PXE bootstrap program over [TFTP](https://en.wikipedia.org/w/index.php?title=Trivial_File_Transfer_Protocol&oldid=959587822):
+
+    $ mkdir tftp
+    $ ln -s ../mirror/pub/OpenBSD/6.7/amd64/pxeboot tftp/auto_install
+    $ ln -s ../mirror/pub/OpenBSD/6.7/amd64/bsd.rd tftp/bsd.rd
+
+Furthermore, we create a [boot(8)](https://man.openbsd.org/OpenBSD-6.7/man8/amd64/boot.8) configuration file at `tftp/etc/boot.conf`:
+
+    stty com0 115200
+    set tty com0
+    boot tftp:/bsd.rd
+
+## Installation
+
+First, we create a copy-on-write disk image using [qemu-img(1)](https://www.qemu.org/docs/master/tools/qemu-img.html):
+
+    $ qemu-img create -f qcow2 disk.qcow2 24G
+
+Then, we start a virtual machine --- and thus the unattended installation of OpenBSD --- off this disk using [qemu-system-x86_64(1)](https://www.qemu.org/docs/master/system/qemu-manpage.html) and [socat(1)](http://www.dest-unreach.org/socat/doc/socat.html):
+
+    $ qemu-system-x86_64 \
+        -enable-kvm \
+        -smp "cpus=4" \
+        -m 4G \
+        -drive "file=disk.qcow2,media=disk,if=virtio" \
+        -device e1000,netdev=n1 \
+        -netdev "user,id=n1,hostname=openbsd-vm,tftp-server-name=10.0.2.1,tftp=tftp,bootfile=auto_install,hostfwd=tcp::2222-:22,guestfwd=tcp:10.0.2.1:80-cmd:socat STDIO TCP4:127.0.0.1:8080" \
+        -nographic
+
+Let's break this last command down.
+The `-enable-kvm` option enables the Linux [Kernel-based Virtual Machine (KVM)](https://www.linux-kvm.org/) support.
+The `-smp` option instructs QEMU to simulate a [symmetric multiprocessing (SMP)](https://en.wikipedia.org/w/index.php?title=Symmetric_multiprocessing&oldid=951686602) system.
+The `-m` option sets the amount of virtual memory.
+The `-drive` option attaches the previously created copy-on-write disk image as a [Virtio](https://wiki.libvirt.org/page/Virtio) disk drive.
+The `-device` option attaches a standard network adapter.
+The `-netdev` option configures a virtual network `10.0.2.0/24` where `10.0.2.2` and `10.0.2.15` point to the QEMU host and guest respectively.
+Moreover, we instruct QEMU to (a) run an embedded TFTP server at `10.0.2.1`, (b) redirect port 2222 on the host to port 22 on the guest, and (c) redirect port 80 on the virtual host address `10.0.2.1` to port 8080 on the host.
+The former port redirection enables us to log in to the guest using [ssh(1)](https://man.openbsd.org/OpenBSD-6.7/ssh), and the latter port redirection frees us from binding to the privileged port 80 on the host.[^socat]
+Finally, the `-nographic` option turns QEMU into a command-line application that redirects the emulated serial port to the console.
+Press `C-a x` to stop the virtual machine, or `C-a h` to show other options.
+
+## Login
+
+Once the virtual machine has booted, you can log in as the user `puffy` using [ssh(1)](https://man.openbsd.org/OpenBSD-6.7/ssh):
+
+    ssh \
+      -o "StrictHostKeyChecking no" \
+      -o "UserKnownHostsFile /dev/null" \
+      -o "Port 2222" \
+      puffy@127.0.0.1
+
+Here, we use the `StrictHostKeyChecking` and `UserKnownHostsFile` options to keep the presumably temporary virtual machine's host key out of our known hosts file.
+
+## Conclusion
+
+We auto-installed OpenBSD/amd64 6.7 on the QEMU PC system emulator 5.0.0 by means of several command-line tools.
+First, we setup a partial, local OpenBSD mirror using [rsync(1)](https://download.samba.org/pub/rsync/rsync.1) and [signify(1)](https://man.openbsd.org/OpenBSD-6.7/signify).
+Second, we prepared a response file for [autoinstall(8)](https://man.openbsd.org/OpenBSD-6.7/autoinstall), a [disklabel(8)](https://man.openbsd.org/OpenBSD-6.7/disklabel) template, and a site-specific file set.
+Third, we setup a standard network boot environment.
+Fourth, we actually installed OpenBSD on a QEMU guest machine.
+Finally, we logged in to the virtual machine using [ssh(1)](https://man.openbsd.org/OpenBSD-6.7/ssh).
+
+Of course, you can automate the whole process.
+For example, I use a simple, yet [complete POSIX shell script](/files/autoinstall-openbsd-on-qemu.sh) to auto-install OpenBSD on QEMU.
+In fact, I have written another script to install and test the said pet project of mine as well.
+But this is out of scope here.
+
+[^signify-portable]:
+  We use Adrian Perez' [portable version](https://github.com/aperezdc/signify) of OpenBSD's [signify(1)](https://man.openbsd.org/OpenBSD-6.7/signify) here.
+
+[^sha256]:
+  You can also verify the [SHA256 checksums](https://ftp.openbsd.org/pub/OpenBSD/6.7/amd64/SHA256) of the fetched files if you cannot use [signify(1)](https://man.openbsd.org/OpenBSD-6.7/signify).
+
+[^response-file]:
+  You can serve per-host response files for [autoinstall(8)](https://www.tumfatig.net/20190426/openbsd-automatic-upgrade/) by prefixing the MAC address or the hostname.
+  Besides, you can add the response file to the RAM disk kernel `bsd.rd` using [rdsetroot(8)](https://man.openbsd.org/OpenBSD-6.7/rdsetroot).
+
+[^restrict-network]:
+  You can create a fully isolated virtual machine by (a) including patches, packages, and ports in your local OpenBSD mirror, and (b) restricting the virtual network created by QEMU.
+
+[^socat]:
+  I failed to forward port 80 on the virtual host address to port 8080 on the local host using [qemu-system-x86_64(1)](https://www.qemu.org/docs/master/system/qemu-manpage.html)'s `guestfwd` alone.
+  That's why I resorted to the invaluable [socat(1)](http://www.dest-unreach.org/socat/doc/socat.html).