diff options
Diffstat (limited to '_drafts')
| -rw-r--r-- | _drafts/autoinstall-openbsd-on-qemu.md | 230 | ||||
| -rw-r--r-- | _drafts/autoinstall-openbsd-on-qemu.sh | 225 |
2 files changed, 455 insertions, 0 deletions
diff --git a/_drafts/autoinstall-openbsd-on-qemu.md b/_drafts/autoinstall-openbsd-on-qemu.md new file mode 100644 index 0000000..b99f4be --- /dev/null +++ b/_drafts/autoinstall-openbsd-on-qemu.md @@ -0,0 +1,230 @@ +--- +title: Auto-install OpenBSD on QEMU +description: How to perform an unattended installation of OpenBSD on the QEMU virtual machine monitor. +published: 2020-07-15 +--- + +<!-- TODO: Update published date in post and script. --> +<!-- TODO: Configure shell script mime type. --> +<!-- TODO: https://www.tumfatig.net/20190625/automated-openbsd-deployment-on-vmd8/ --> +<!-- TODO: https://www.tumfatig.net/20190426/openbsd-automatic-upgrade/ --> + +I happen to develop a pet project of mine on a Linux desktop, while actually targeting an [OpenBSD](https://www.openbsd.org/) server. +Thus I searched for a scriptable way to install OpenBSD on the [QEMU](https://www.qemu.org/) virtual machine manager, such that I could automate a local integration test against OpenBSD running on Linux. + +As expected, OpenBSD has a remarkably straightforward unattended installation solution. +During a normal, interactive installation, you answer a series of questions like what timezone you are in or which file sets to install. +At the end of the installation, the installer sends a recorded list of those questions along with your answers to the root user's mail box. +You can then adapt this so-called _response file_ to your needs and feed it to the [autoinstall(8)](https://man.openbsd.org/OpenBSD-6.7/autoinstall) command to perform an unattended installation. + +In the remainder of this post, I show how to auto-install OpenBSD/amd64 6.7 on the QEMU PC system emulator 5.0.0. +In the end, I will present a simple yet [complete POSIX shell script](/files/autoinstall-openbsd-on-qemu.sh) to get the job done. +The script is intended to run on Linux, though. +If you already have a running OpenBSD installation, you should consider to use OpenBSD's own hypervisor [vmm(4)](https://man.openbsd.org/OpenBSD-6.7/vmm) instead of QEMU as described in the [OpenBSD FAQ](https://www.openbsd.org/faq/faq16.html) and in this [blog post](https://eradman.com/posts/autoinstall-openbsd.html) by Eric Radman. + +## Outline + +We will perform the following steps: + +1. Create a local OpenBSD mirror. +1. Configure the unattended installation. +1. Create a network boot environment. +1. Create a virtual machine. +1. Log in to the virtual machine. + +## Prerequisites + +We will use the following tools: + +* [curl](https://curl.haxx.se/), a data transfer tool (and library) +* [OpenSSH](https://www.openssh.com/portable.html), a remote login tool +* [QEMU](https://www.qemu.org/), a virtual machine monitor +* [rsync](https://rsync.samba.org/), an incremental file transfer tool +* [signify](https://github.com/aperezdc/signify), a cryptographic signature tool +* [socat](http://www.dest-unreach.org/socat/), a successor of the infamous +TCP/IP swiss army knife, [netcat](https://nc110.sourceforge.io/) + +Chances are that your Linux distribution of choice packages these tools. +For example, the following command installs them on Arch Linux: + + $ sudo pacman -S curl openssh qemu rsync signify socat + +## Create a local OpenBSD mirror + +To begin with, we setup a partial, local [OpenBSD mirror](https://www.openbsd.org/ftp.html). + +First, we create the [directory layout](https://www.openbsd.org/ftp.html#layout): + + $ mkdir -p mirror/pub/OpenBSD/6.7/amd64 + +Second, we fetch the base public key from the official HTTPS mirror using [curl(1)](https://curl.haxx.se/docs/manpage.html): + + $ curl \ + --output mirror/pub/OpenBSD/6.7/openbsd-67-base.pub \ + https://ftp.openbsd.org/pub/OpenBSD/6.7/openbsd-67-base.pub + +Third, we fetch the kernel, [PXE](https://en.wikipedia.org/w/index.php?title=Preboot_Execution_Environment&oldid=955913424) bootstrap program, and file sets from an untrusted [rsync mirror](https://www.openbsd.org/ftp.html#rsync) using [rsync(1)](https://download.samba.org/pub/rsync/rsync.1): + + $ rsync --archive --files-from=- --verbose \ + rsync://ftp.halifax.rwth-aachen.de/openbsd/6.7/amd64/ \ + mirror/pub/OpenBSD/6.7/amd64 \ + << EOF + SHA256.sig + base67.tgz + bsd + bsd.mp + bsd.rd + comp67.tgz + game67.tgz + man67.tgz + pxeboot + xbase67.tgz + xfont67.tgz + xserv67.tgz + xshare67.tgz + EOF + +Fourth, we verify the fetched files using [signify(1)](https://man.openbsd.org/OpenBSD-6.7/signify) and the previously fetched base public key: + + $ ( cd mirror/pub/OpenBSD/6.7/amd64 && signify -C \ + -p ../openbsd-67-base.pub \ + -x SHA256.sig \ + -- bsd bsd.* pxeboot *67.tgz ) + +Finally, we serve the local mirror at <http://127.0.0.1:8080/>. +Feel free to use your webserver of choice. +Chances are, that your Linux distribution comes with Python's [http.server module](https://docs.python.org/3/library/http.server.html): + + $ python3 \ + -m http.server \ + --directory mirror \ + --bind 127.0.0.1 8080 + +## Configure the installation + +First, we create a response file for [autoinstall(8)](https://man.openbsd.org/OpenBSD-6.7/autoinstall) at `mirror/install.conf`: + + Change the default console to com0 = yes + Which speed should com0 use = 115200 + System hostname = openbsd + Password for root = ************* + Allow root ssh login = no + Setup a user = puffy + Password for user = ************* + Public ssh key for user = ssh-rsa AAAAB3N... alex@example + What timezone are you in = UTC + Location of sets = http + HTTP Server = 10.0.2.1 + Unable to connect using https. Use http instead = yes + URL to autopartitioning template for disklabel = http://10.0.2.1/disklabel + Set name(s) = site67.tgz + Checksum test for site67.tgz failed. Continue anyway = yes + Unverified sets: site67.tgz. Continue without verification = yes + +Take care to insert your own public SSH key here, for example, the contents of `~/.ssh/id_rsa.pub`. + +Note that we effectively disable password-based authentication here by assigning the conventional 13 asterisks as encrypted passwords for both users, `root` and `puffy`, see [passwd(5)](https://man.openbsd.org/OpenBSD-6.7/passwd.5). +Instead, we enable the user `puffy` to login using the given SSH key. +Besides, we will later permit the user `puffy` to run any command as root without entering his password using [doas(1)](https://man.openbsd.org/OpenBSD-6.7/doas). + +Note also that we will later instruct QEMU to redirect port 80 on the virtual network address 10.0.2.1 to port 8080 on the local host. + +Next, we create a [disklabel(8)](https://man.openbsd.org/OpenBSD-6.7/disklabel) template at `mirror/disklabel`: + + / 2G + swap 8G + /tmp 1G + /var 1G + /usr 2G + /usr/X11R6 500M + /usr/local 4G + /usr/src 1M + /usr/obj 1M + /home 4G + +Finally, we create an optional site-specific file set. +This way, we can run some commands at the end of the installation. +Here, we reset the OpenBSD mirror server used by pkg_add(1) and other commands. +Otherwise, we would need to include the binary packages in the local OpenBSD mirror. +Moreover, we permit the wheel user group --- and thus the user `puffy` --- to run any command as root without entering their password using [doas(1)](https://man.openbsd.org/OpenBSD-6.7/doas). + +Create the file `site/install.site`: + +``` +#! /bin/ksh +set -o errexit +echo "https://cdn.openbsd.org/pub/OpenBSD" > /etc/installurl +echo "permit nopass keepenv :wheel" >> /etc/doas.conf +``` + +Then, make the file executable, package the file set, and add it to the local OpenBSD mirror: + + $ chmod +x site/install.site + $ ( cd site && tar -czf ../mirror/pub/OpenBSD/6.7/amd64/site67.tgz . ) + $ ( cd mirror/pub/OpenBSD/6.7/amd64 && ls -l > index.txt ) + +## Create a network boot environment + +We create a dedicated directory to serve the OpenBSD kernel and PXE bootstrap program over [TFTP](https://en.wikipedia.org/w/index.php?title=Trivial_File_Transfer_Protocol&oldid=959587822): + + $ mkdir tftp + $ ln -s ../mirror/pub/OpenBSD/6.7/amd64/pxeboot tftp/auto_install + $ ln -s ../mirror/pub/OpenBSD/6.7/amd64/bsd.rd tftp/bsd.rd + +Furthermore, we create a [boot(8)](https://man.openbsd.org/OpenBSD-6.7/man8/amd64/boot.8) configuration file at `tftp/etc/boot.conf`: + + stty com0 115200 + set tty com0 + boot tftp:/bsd.rd + +## Create a virtual machine + +First, we create a copy-on-write disk image using [qemu-img(1)](https://www.qemu.org/docs/master/tools/qemu-img.html): + + $ qemu-img create -f qcow2 disk.qcow2 24G + +Then, we start a virtual machine --- and thus the unattended installation of OpenBSD --- off this disk using [qemu-system-x86_64(1)](https://www.qemu.org/docs/master/system/qemu-manpage.html) and [socat(1)](http://www.dest-unreach.org/socat/doc/socat.html): + + $ qemu-system-x86_64 \ + -enable-kvm \ + -smp "cpus=4" \ + -m 4G \ + -drive "file=disk.qcow2,media=disk,if=virtio" \ + -device e1000,netdev=n1 \ + -netdev "user,id=n1,hostname=openbsd-vm,tftp-server-name=10.0.2.1,tftp=tftp,bootfile=auto_install,hostfwd=tcp::2222-:22,guestfwd=tcp:10.0.2.1:80-cmd:socat STDIO TCP4:127.0.0.1:8080" \ + -nographic + +Let's break this last command down. +The `-enable-kvm` option enables the Linux [Kernel-based Virtual Machine (KVM)](https://www.linux-kvm.org/) support. +The `-smp` option instructs QEMU to simulate a [symmetric multiprocessing (SMP)](https://en.wikipedia.org/w/index.php?title=Symmetric_multiprocessing&oldid=951686602) system. +The `-m` option sets the amount of virtual memory. +The `-drive` option attaches the previously created copy-on-write disk image as a [virtio](https://wiki.libvirt.org/page/Virtio) disk drive. +The `-device` option attaches a standard network adapter. +The `-netdev` option configures a virtual network `10.0.2.0/24` where `10.0.2.2` and `10.0.2.15` point to the QEMU host and guest respectively. +Moreover, we instruct QEMU to redirect (a) port 2222 on the host to port 22 on the guest, and (b) port 80 on the virtual host address `10.0.2.1` to port 8080 on the host. +The former enables us to `ssh` into the guest, and the latter frees us from binding to the privileged port 80 on the host. +Finally, the `-nographic` option turns QEMU into a command-line application that redirects the emulated serial port to the console. +Press `C-a x` to stop the virtual machine, or `C-a h` to show other options. + +<!-- TODO: Why socat? --> + +## Log in to the virtual machine + +Once the virtual machine has booted, you can login as the user `puffy` using [ssh(1)](https://man.openbsd.org/OpenBSD-6.7/ssh): + + ssh \ + -o "StrictHostKeyChecking no" \ + -o "UserKnownHostsFile /dev/null" \ + -o "Port 2222" \ + puffy@127.0.0.1 + +Here, we use the `StrictHostKeyChecking` and `UserKnownHostsFile` options to keep the presumably temporary virtual machine's host key out of our known hosts file. + +## Conclusion + +<!-- TODO: Summary --> +<!-- TODO: Advertise script again --> +<!-- TODO: Special URLs used by autoinstall --> +<!-- TODO: Offline VM using full local OpenBSD mirror or caching proxy --> +<!-- TODO: Verify using SHA256 instead of signify --> +<!-- TODO: StrictHostKeyChecking yes, UserKnownHostsFile ./tmp_known_hosts --> diff --git a/_drafts/autoinstall-openbsd-on-qemu.sh b/_drafts/autoinstall-openbsd-on-qemu.sh new file mode 100644 index 0000000..104456a --- /dev/null +++ b/_drafts/autoinstall-openbsd-on-qemu.sh @@ -0,0 +1,225 @@ +#! /bin/sh + +# Auto-install OpenBSD/amd64 6.7 on QEMU. +# +# First published at https://www.skreutz.com/posts/autoinstall-openbsd-on-qemu/ +# on 15 June 2020. +# +# Copyright (c) 2020 Stefan Kreutz <mail@skreutz.com> +# +# Permission to use, copy, modify, and distribute this software for any purpose +# with or without fee is hereby granted, provided that the above copyright +# notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH +# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY +# AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, +# INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM +# LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR +# OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR +# PERFORMANCE OF THIS SOFTWARE. + +set -o errexit +set -o nounset + +# Trusted HTTPS OpenBSD mirror to fetch the base public key from. +HTTPS_MIRROR="${HTTPS_MIRROR-https://ftp.openbsd.org/pub/OpenBSD/}" + +# Untrusted rsync OpenBSD mirror. +RSYNC_MIRROR="${RSYNC_MIRROR-rsync://ftp.halifax.rwth-aachen.de/openbsd/}" + +# File name of the disk image. +DISK_FILE="${DISK_FILE-disk.qcow2}" + +# Size of the disk image. +DISK_SIZE="${DISK_SIZE-24G}" + +# Number of virtual CPUs. +CPU_COUNT="${CPU_COUNT-4}" + +# Size of virtual memory. +MEMORY_SIZE="${MEMORY_SIZE-4G}" + +# File name of the public SSH key to authorize. +SSH_KEY="${SSH_KEY-${HOME}/.ssh/id_rsa.pub}" + +# Check required commands. +for cmd in curl qemu-img qemu-system-x86_64 rsync signify socat ssh +do + if ! command -v "${cmd}" >/dev/null + then + ( >&2 printf "command not found: %s\\n" "${cmd}" ) + exit 1 + fi +done + +# Fetch base public key from trusted HTTPS mirror. +mkdir -p mirror/pub/OpenBSD/6.7 +if [ ! -e mirror/pub/OpenBSD/6.7/openbsd-67-base.pub ] +then + curl \ + --silent \ + --output mirror/pub/OpenBSD/6.7/openbsd-67-base.pub \ + "${HTTPS_MIRROR}6.7/openbsd-67-base.pub" + printf "Fetched base public key from %s\\n" "${HTTPS_MIRROR}" +fi + +# Fetch kernel, PXE bootstrap program, and file sets from untrusted rsync +# mirror. +if [ ! -d mirror/pub/OpenBSD/6.7/amd64 ] +then + mkdir -p tmp + printf "Fetching installation files ...\\n" + rsync --archive --files-from=- --quiet \ + "${RSYNC_MIRROR}6.7/amd64/" \ + tmp/ \ + << EOF +SHA256.sig +base67.tgz +bsd +bsd.mp +bsd.rd +comp67.tgz +game67.tgz +man67.tgz +pxeboot +xbase67.tgz +xfont67.tgz +xserv67.tgz +xshare67.tgz +EOF + ( cd tmp && signify -C -q \ + -p ../mirror/pub/OpenBSD/6.7/openbsd-67-base.pub \ + -x SHA256.sig \ + -- bsd bsd.* pxeboot *67.tgz ) + mv tmp mirror/pub/OpenBSD/6.7/amd64 + printf "Fetched kernel, PXE bootstrap program, and file sets from %s\\n" "${RSYNC_MIRROR}" +fi + +# Create autoinstall configuration if not exists. +if [ ! -e mirror/install.conf ] +then + cat << EOF > mirror/install.conf +Change the default console to com0 = yes +Which speed should com0 use = 115200 +System hostname = openbsd +Password for root = ************* +Allow root ssh login = no +Setup a user = puffy +Password for user = ************* +Public ssh key for user = $( cat "${SSH_KEY}" ) +What timezone are you in = UTC +Location of sets = http +HTTP Server = 10.0.2.1 +Unable to connect using https. Use http instead = yes +URL to autopartitioning template for disklabel = http://10.0.2.1/disklabel +Set name(s) = site67.tgz +Checksum test for site67.tgz failed. Continue anyway = yes +Unverified sets: site67.tgz. Continue without verification = yes +EOF + printf "Created example response file for autoinstall(8) at ./mirror/install.conf\\n" +fi + +# Create disklabel configuration if not exists. +if [ ! -e mirror/disklabel ] +then + cat << EOF > mirror/disklabel +/ 2G +swap 8G +/tmp 1G +/var 1G +/usr 2G +/usr/X11R6 500M +/usr/local 4G +/usr/src 1M +/usr/obj 1M +/home 4G +EOF + printf "Created example disklabel(8) template at ./mirror/disklabel.conf\\n" +fi + +# Create site-specific file set if not exists. +if [ ! -d site ] +then + mkdir site + cat << EOF > site/install.site +#! /bin/ksh + +set -o errexit + +# Reset OpenBSD mirror server used by pkg_add(1) and other commands. +echo "https://cdn.openbsd.org/pub/OpenBSD" > /etc/installurl + +# Permit user group wheel to run any command as root without entering their +# password using doas(1). +echo "permit nopass keepenv :wheel" > /etc/doas.conf + +# Patch the base system on the first boot. +#echo "syspatch && shutdown -r now" >> /etc/rc.firsttime +EOF + chmod +x site/install.site + printf "Created example site-specific file set at ./site\\n" +fi + +# Package site-specific file set if not exists or changed. +site_dir_changed="$( find site -exec stat -c %Y {} \; | sort -r | head -n 1 )" +if [ ! -e mirror/pub/OpenBSD/6.7/amd64/site67.tgz ] || [ "$( stat -c %Y mirror/pub/OpenBSD/6.7/amd64/site67.tgz )" -lt "${site_dir_changed}" ] +then + rm -f mirror/pub/OpenBSD/6.7/amd64/site67.tgz + ( cd site && tar -czf ../mirror/pub/OpenBSD/6.7/amd64/site67.tgz . ) + ( cd mirror/pub/OpenBSD/6.7/amd64 && ls -l > index.txt ) +fi + +# Create TFTP directory if not exists. +if [ ! -d tftp ] +then + mkdir tftp + ln -s ../mirror/pub/OpenBSD/6.7/amd64/pxeboot tftp/auto_install + ln -s ../mirror/pub/OpenBSD/6.7/amd64/bsd.rd tftp/bsd.rd + mkdir tftp/etc + cat << EOF > tftp/etc/boot.conf +stty com0 115200 +set tty com0 +boot tftp:/bsd.rd +EOF + printf "Created example boot(8) configuration at ./tftp/etc/boot.conf\\n" +fi + +# Remove existing disk image if configuration changed. +if [ -e "${DISK_FILE}" ] +then + vm_created="$( stat -c %W "${DISK_FILE}" )" + for f in mirror/install.conf mirror/disklabel mirror/pub/OpenBSD/6.7/amd64/site67.tgz tftp/etc/boot.conf + do + if [ "${vm_created}" -lt "$( stat -c %Y "$f" )" ] + then + printf "Re-creating virtual machine due to changed configuration: %s\\n" "$f" + rm "${DISK_FILE}" + fi + done +fi + +# Create disk image if not exists. +if [ ! -e "${DISK_FILE}" ] +then + qemu-img create -q -f qcow2 "${DISK_FILE}" "${DISK_SIZE}" + printf "Created %s copy-on-write disk image at %s\\n" "${DISK_SIZE}" "${DISK_FILE}" +fi + +# Wait until ./mirror is served at http://127.0.0.1:8080/. +while [ ! "$( curl --silent --location --write-out '%{http_code}\n' --output /dev/null http://127.0.0.1:8080/install.conf )" = 200 ] +do + ( >&2 printf "Please serve the directory ./mirror at http://127.0.0.1:8080/\n" ) + sleep 5 +done + +# Auto-install OpenBSD. +printf "Starting virtual machine ...\\n" +qemu-system-x86_64 \ + -enable-kvm \ + -smp "cpus=${CPU_COUNT}" \ + -m "${MEMORY_SIZE}" \ + -drive "file=${DISK_FILE},media=disk,if=virtio" \ + -device e1000,netdev=n1 \ + -netdev "user,id=n1,hostname=openbsd-vm,tftp-server-name=10.0.2.1,tftp=tftp,bootfile=auto_install,hostfwd=tcp::2222-:22,guestfwd=tcp:10.0.2.1:80-cmd:socat STDIO TCP4:127.0.0.1:8080" \ + -nographic |