From 8f78d7ac1e4e187fcfe973d1e84308cf870ba9a2 Mon Sep 17 00:00:00 2001 From: Stefan Kreutz Date: Sat, 24 Jul 2021 20:12:02 +0200 Subject: Release installiso 0.1.0 --- BACKLOG.md | 1 - _drafts/first-release-of-installiso.md | 193 --------------------------------- deploy | 2 +- files/installiso-0.1.0.tar.gz | Bin 0 -> 4733 bytes files/installiso.8-0.1.0.html | 180 ++++++++++++++++++++++++++++++ posts/first-release-of-installiso.md | 183 +++++++++++++++++++++++++++++++ 6 files changed, 364 insertions(+), 195 deletions(-) delete mode 100644 _drafts/first-release-of-installiso.md create mode 100644 files/installiso-0.1.0.tar.gz create mode 100644 files/installiso.8-0.1.0.html create mode 100644 posts/first-release-of-installiso.md diff --git a/BACKLOG.md b/BACKLOG.md index 5b7b7f4..5b8bed1 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -59,7 +59,6 @@ * SQLite hexastore vs. Neo4j * Setup a FreeBSD sftp server with basic email notifications from the command-line using bash, ssh, and the DigitalOcean command-line application * Cloud/VPS provider comparison -* goaccess log format for OpenBSD httpd * Practical, cross-platform, append-only backups using restic, rclone, and ssh: ready to use scripts/shortcuts for OpenBSD, Linux, macOS, Windows 10 * nix vs. guix * Find man(1) pages interactively using fzf(1), see ~/.kshrc diff --git a/_drafts/first-release-of-installiso.md b/_drafts/first-release-of-installiso.md deleted file mode 100644 index 7c6537d..0000000 --- a/_drafts/first-release-of-installiso.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -title: "First release of installiso" -description: "A utility to customize OpenBSD installation images for unattended installation." -published: 2021-07-24 ---- - - - - - - -One year ago I [posted](/posts/autoinstall-openbsd-on-qemu/) how to script an unattended installation of OpenBSD on the QEMU virtual machine monitor on Linux. -The script involved setting up a complicated network boot environment because I treated the installation image as a black box. -Of course, I could have mounted the ISO 9660 image and created a modified image using [`mkisofs(8)`](http://cdrtools.sourceforge.net/private/man/cdrecord/mkisofs.8.html). -But I didn't know how to insert the [`autoinstall(8)`](https://man.openbsd.org/OpenBSD-6.9/autoinstall) response file into the RAMDISK kernel in the ISO 9660 image. -That was no surprise -- why would anyone need to change an OpenBSD kernel on Linux. - -OpenBSD, on the other hand, includes adequate utilities. -Thanks to [`vmctl(8)`](https://man.openbsd.org/OpenBSD-6.9/vmctl), [`rdsetroot(8)`](https://man.openbsd.org/OpenBSD-6.9/rdsetroot), and [`mkhybrid(8)`](https://man.openbsd.org/OpenBSD-6.9/mkhybrid), we can modify the ISO 9660 image _and_ the contained RAMDISK kernel. -The exact process is a bit tedious so I decided to automate it. -The resulting script is more hacky than pretty but it gets the job done and I found it useful enough to give it a name, `installiso`, and release it today. -Currently, it's limited to the _amd64_ architecture, though it should work on other architectures alike. -You can download the very first release [here](/files/installiso-0.1.0.tar.gz). -The man page is also viewable [here](/files/installiso.8-0.1.0.html). -Feedback appreciated! - -In the remainder of this post I'll show how to use `installiso` to create custom OpenBSD installation images for unattended -- and possibly offline -- installation. -As an example, I'll show how to create virtual machines on OpenBSD's own virtual machine monitor, [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm). -Finally, I'll list the concrete commands necessary to patch an installation image without `installiso` because I would want to know how it works. - -## Installation - -You can install the `installiso` utility as follows. - - $ ftp https://www.skreutz.com/files/installiso-0.1.0.tar.gz - $ tar -xzf installiso-0.1.0.tar.gz - $ cd installiso-0.1.0/ - $ doas make install - $ man 8 installiso - -Alternatively, you can run `./installiso.ksh` without installation. - -## Usage - -Given a response file `install.conf`, and a file-specific file set `site/`, you can download, verify, and customize an official installation image as follows. - - $ installiso -v fetch -r 6.9 - $ doas installiso -v \ - patch -i install.conf -s site/ install69.iso custom.iso - -You can also specify a mirror, another release or the latest development snapshot, and a [`signify(1)`](https://man.openbsd.org/OpenBSD-6.9/signify) public key if you like. -See the [man page](/files/installiso.8-0.1.0.html). - -## Example - -You can create a virtual machine on OpenBSD's own virtual machine monitor, [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm), as follows. - -First, we create an [`autoinstall(8)`](https://man.openbsd.org/OpenBSD-6.9/autoinstall) response file. -Here, we'll setup a regular user and authorize an [`ssh(1)`](https://man.openbsd.org/OpenBSD-6.9/ssh) public key. -Of course, you can skip this step and have the installer mail you the responses recorded during an interactive installation instead. - - $ cat >install.conf <site/install.site < /etc/installurl - - # Permit user group wheel to run any command as root - # without entering their password using doas(1). - echo "permit nopass keepenv :wheel" > /etc/doas.conf - - # Install packages on the first boot. - echo "pkg_add sqlite3" >> /etc/rc.firsttime - EOF - -Third, fetch and patch an installation image as above. - - $ installiso -v fetch -r 6.9 - $ doas installiso -v \ - patch -i install.conf -s site/ install69.iso custom.iso - -Forth, we start a virtual machine off a new disk image and the custom installation image. -The OpenBSD FAQ contains a good [introduction](https://www.openbsd.org/faq/faq16.html) to [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm). -I assume you've setup the network option 2. - - $ vmctl create -s 10G disk.qcow2 - $ doas vmctl start -c -i 1 -L -m 512M -d disk.qcow2 \ - -r custom.iso tmp - -Finally, we can log into the new virtual machine once the unattended installation has completed. - - $ ssh \ - -o "StrictHostKeyChecking no" \ - -o "UserKnownHostsFile /dev/null" \ - 100.64.1.3 - -## Disklabel template - -In the above example, the OpenBSD installer allocates all disk space automatically, see [`disklabel(8)`](https://man.openbsd.org/OpenBSD-6.9/disklabel). -You can supply a custom disklabel template instead if you need more control. - -First, create a disklabel template. - - $ cat >disklabel_template <> install.conf - -Finally, serve the file during the unattended installation. - - $ printf 'HTTP/1.0 200 OK\n\n' | - cat - disklabel | - nc -lN 8080 - -## Inner workings - -The `installiso patch` command used above - - $ doas installiso -v \ - patch -i install.conf -s site/ install69.iso custom.iso - -boils down to the following commands, excluding error handling, temporary files, options, etc. - -First, we extract the ISO 9660 image. - - # vnconfig vnd0 install69.iso - # mount -t cd9660 /dev/vnd0c /mnt - # mkdir cd - # tar -C /mnt -c -f - . | tar -C cd -x -p -f - - # umount /mnt - # vnconfig -u vnd0 - -Second, we patch the RAMDISK kernel. - - # gzip -d -o bsd.rd cd/6.9/amd64/bsd.rd - # rdsetroot -x bsd.rd disk.fs - # vnconfig vnd0 disk.fs - # mount /dev/vnd0a" /mnt - # install -o root -g wheel -m 0644 -C \ - install.conf /mnt/auto_install.conf - # umount /mnt - # vnconfig -u vnd0 - # rdsetroot bsd.rd disk.fs - # gzip -9fnq bsd.rd - -Third, we insert the patched kernel and the site-specific file set into the extracted ISO 9660 image. - - # install -o root -g 2000 -m 0755 -C \ - bsd.rd.gz cd/6.9/amd64/bsd.rd - # ( cd site/ && tar -c -z -f ../cd/6.9/amd64/site69.tgz . ) - # ( cd cd/6.9/amd64/ && ls -l > index.txt ) - -Finally, we create the bootable ISO 9660 image. - - # mkhybrid -a -R -T -L -l -d -D -N -o custom.iso \ - -A "Custom OpenBSD 6.9 amd64 Install CD" \ - -b 6.9/amd64/cdbr -c 6.9/amd64/boot.catalog \ - cd - -[^tmpdir]: -The `installiso` utility may fail due to not enough space in `/tmp`. -If so, you can set the `TMPDIR` environment variable of [`mktemp(1)`](https://man.openbsd.org/OpenBSD-6.9/mktemp). -Remember that [`doas(1)`](https://man.openbsd.org/OpenBSD-6.9/doas) creates a new environment by default, though. -You can either configure `doas(1)` to keep the `TMPDIR`, or you execute a shell: -`doas sh -c 'TMPDIR=/path/to/tmp installiso patch ...'`. diff --git a/deploy b/deploy index f748a4e..bd260ac 100755 --- a/deploy +++ b/deploy @@ -5,7 +5,7 @@ # Re-generates the static website from source and asserts a clean working tree # before uploading the website to the server. # -# Submits the live sitemap to Google and Bing. +# Submits the sitemap URL to Google and Bing. set -o errexit set -o nounset diff --git a/files/installiso-0.1.0.tar.gz b/files/installiso-0.1.0.tar.gz new file mode 100644 index 0000000..e063852 Binary files /dev/null and b/files/installiso-0.1.0.tar.gz differ diff --git a/files/installiso.8-0.1.0.html b/files/installiso.8-0.1.0.html new file mode 100644 index 0000000..31c06af --- /dev/null +++ b/files/installiso.8-0.1.0.html @@ -0,0 +1,180 @@ + + + + + + + + INSTALLISO(8) + + + + + + + + +
INSTALLISO(8)System Manager's ManualINSTALLISO(8)
+
+
+

+

installiso — + Customize OpenBSD installation images

+
+
+

+ + + + + +
installiso[-v] command + [arg ...]
+
+
+

+

The installiso utility facilitates + creating custom ISO 9660 installation images for the unattended installation + of OpenBSD.

+

The common options are as follows:

+
+
+
Verbose mode. Causes installiso to print + informative messages. Multiple -v options increase + the verbosity. The maximum is 2. By default, + installiso is quiet.
+
+

The commands are as follows:

+
+
+ [-f] [-m + mirror] [-r + release] [-p + key] [-o + output]
+
Download and verify an official ISO 9660 installation image. +
+
+
Force overwriting existing files. By default, + fetch will not overwrite existing files.
+
+ mirror
+
The HTTP(S) OpenBSD + mirror to use. Defaults to the mirror specified + by installurl(5) or else + https://cdn.openbsd.org/pub/OpenBSD/.
+
+ release
+
The OpenBSD release. + Defaults to the latest development snapshot of -current.
+
+ key
+
The public signify(1) key used + to verify the downloaded installation image. Defaults to the key + /etc/signify/openbsd-*-base.pub corresponding + to the downloaded release.
+
+
+
+ [-f] [-i + install_conf] [-u + upgrade_conf] [-s + site_dir] input + output
+
Patch an ISO 9660 installation image. +
+
+
Force overwriting existing files. By default, + patch will not overwrite existing files.
+
+ install_conf
+
Insert an autoinstall(8) response file for + unattended installation.
+
+ upgrade_conf
+
Insert an autoinstall(8) response file for + unattended upgrade.
+
+ site
+
Package and insert the directory site as a + site-specific file set.
+
input
+
The input installation image file.
+
output
+
The output installation image file.
+
+
+
+
+
+

+

The installiso utility exits 0 on + success, and >0 if an error occurs.

+
+
+

+

Fetch the latest development snapshot:

+
+
$ installiso -v fetch
+
+

Fetch a specific release:

+
+
$ installiso -v fetch -r 6.9
+
+

Create a custom image for unattended installation:

+
+
$ doas installiso -v patch -i install.conf install69.iso custom.iso
+
+
+
+

+

autoinstall(8), mkhybrid(8), + rdsetroot(8), sysupgrade(8), + vnconfig(8)

+
+
+

+

Stefan Kreutz + <mail@skreutz.com>

+
+
+

+

The installiso utility is currently + limited to the amd64 architecture because it wasn't tested on any other + architecture.

+
+
+ + + + + +
July 24, 2021OpenBSD 6.9
+ + diff --git a/posts/first-release-of-installiso.md b/posts/first-release-of-installiso.md new file mode 100644 index 0000000..c0d5502 --- /dev/null +++ b/posts/first-release-of-installiso.md @@ -0,0 +1,183 @@ +--- +title: "First release of installiso" +description: "A utility to customize OpenBSD installation images for unattended installation." +published: 2021-07-24 +--- + +One year ago I [posted](/posts/autoinstall-openbsd-on-qemu/) how to script an unattended installation of OpenBSD on the QEMU virtual machine monitor on Linux. +The script involved setting up a complicated network boot environment because I treated the installation image as a black box. +Of course, I could have mounted the ISO 9660 image and created a modified image using [`mkisofs(8)`](http://cdrtools.sourceforge.net/private/man/cdrecord/mkisofs.8.html). +But I didn't know how to insert the [`autoinstall(8)`](https://man.openbsd.org/OpenBSD-6.9/autoinstall) response file into the RAMDISK kernel in the ISO 9660 image. +That was no surprise -- why would anyone need to change an OpenBSD kernel on Linux. + +OpenBSD, on the other hand, includes adequate utilities. +Thanks to [`vmctl(8)`](https://man.openbsd.org/OpenBSD-6.9/vmctl), [`rdsetroot(8)`](https://man.openbsd.org/OpenBSD-6.9/rdsetroot), and [`mkhybrid(8)`](https://man.openbsd.org/OpenBSD-6.9/mkhybrid), we can modify the ISO 9660 image _and_ the contained RAMDISK kernel. +The exact process is a bit tedious so I decided to automate it. +The resulting script is more hacky than pretty but it gets the job done and I found it useful enough to give it a name, `installiso`, and release it today. +Currently, it's limited to the _amd64_ architecture, though it should work on other architectures alike. +You can download the very first release [here](/files/installiso-0.1.0.tar.gz). +The man page is also viewable [here](/files/installiso.8-0.1.0.html). +Feedback appreciated! + +In the remainder of this post I'll show how to use `installiso` to create custom OpenBSD installation images for unattended -- and possibly offline -- installation. +As an example, I'll show how to create virtual machines on OpenBSD's own virtual machine monitor, [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm). +Finally, I'll list the concrete commands necessary to patch an installation image without `installiso` because I would want to know how it works. + +## Installation + +You can install the `installiso` utility as follows. + + $ ftp https://www.skreutz.com/files/installiso-0.1.0.tar.gz + $ tar -xzf installiso-0.1.0.tar.gz + $ cd installiso-0.1.0/ + $ doas make install + $ man 8 installiso + +Alternatively, you can run `./installiso.ksh` without installation. + +## Usage + +Given a response file `install.conf`, and a file-specific file set `site/`, you can download, verify, and customize an official installation image as follows. + + $ installiso -v fetch -r 6.9 + $ doas installiso -v \ + patch -i install.conf -s site/ install69.iso custom.iso + +You can also specify a mirror, another release or the latest development snapshot, and a [`signify(1)`](https://man.openbsd.org/OpenBSD-6.9/signify) public key if you like. +See the [man page](/files/installiso.8-0.1.0.html). + +## Example + +You can create a virtual machine on OpenBSD's own virtual machine monitor, [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm), as follows. + +First, we create an [`autoinstall(8)`](https://man.openbsd.org/OpenBSD-6.9/autoinstall) response file. +Here, we'll setup a regular user and authorize an [`ssh(1)`](https://man.openbsd.org/OpenBSD-6.9/ssh) public key. +Of course, you can skip this step and have the installer mail you the responses recorded during an interactive installation instead. + + $ cat >install.conf <site/install.site < /etc/installurl + + # Permit user group wheel to run any command as root + # without entering their password using doas(1). + echo "permit nopass keepenv :wheel" > /etc/doas.conf + + # Install packages on the first boot. + echo "pkg_add sqlite3" >> /etc/rc.firsttime + EOF + $ chmod +x site/install.site + +Third, fetch and patch an installation image as above. + + $ installiso -v fetch -r 6.9 + $ doas installiso -v \ + patch -i install.conf -s site/ install69.iso custom.iso + +Forth, we start a virtual machine off a new disk image and the custom installation image. +The OpenBSD FAQ contains a good [introduction](https://www.openbsd.org/faq/faq16.html) to [`vmm(4)`](https://man.openbsd.org/OpenBSD-6.9/vmm). +I assume you've setup the network option 2. + + $ vmctl create -s 10G disk.qcow2 + $ doas vmctl start -c -i 1 -L -m 512M -d disk.qcow2 \ + -r custom.iso tmp + +Finally, we can log into the new virtual machine once the unattended installation has completed. + + $ ssh \ + -o "StrictHostKeyChecking no" \ + -o "UserKnownHostsFile /dev/null" \ + 100.64.1.3 + +## Disklabel template + +In the above example, the OpenBSD installer allocates all disk space automatically, see [`disklabel(8)`](https://man.openbsd.org/OpenBSD-6.9/disklabel). +You can supply a custom disklabel template instead if you need more control. + +First, create a disklabel template. + + $ cat >disklabel_template <> install.conf + +Finally, serve the file during the unattended installation. + + $ printf 'HTTP/1.0 200 OK\n\n' | + cat - disklabel_template | + nc -lN 8080 + +## Inner workings + +The `installiso patch` command used above + + $ doas installiso -v \ + patch -i install.conf -s site/ install69.iso custom.iso + +boils down to the following commands -- excluding error handling, temporary files, command-line options, etc. + +First, we extract the ISO 9660 image. + + # vnconfig vnd0 install69.iso + # mount -t cd9660 /dev/vnd0c /mnt + # mkdir cd + # tar -C /mnt -c -f - . | tar -C cd -x -p -f - + # umount /mnt + # vnconfig -u vnd0 + +Second, we patch the RAMDISK kernel. + + # gzip -d -o bsd.rd cd/6.9/amd64/bsd.rd + # rdsetroot -x bsd.rd disk.fs + # vnconfig vnd0 disk.fs + # mount /dev/vnd0a /mnt + # install -o root -g wheel -m 0644 -C \ + install.conf /mnt/auto_install.conf + # umount /mnt + # vnconfig -u vnd0 + # rdsetroot bsd.rd disk.fs + # gzip -9fnq bsd.rd + +Third, we insert the patched kernel and the site-specific file set into the extracted ISO 9660 image. + + # install -o root -g 2000 -m 0755 -C \ + bsd.rd.gz cd/6.9/amd64/bsd.rd + # ( cd site/ && tar -c -z -f ../cd/6.9/amd64/site69.tgz . ) + # ( cd cd/6.9/amd64/ && ls -l > index.txt ) + +Finally, we create the bootable ISO 9660 image. + + # mkhybrid -a -R -T -L -l -d -D -N -o custom.iso \ + -A "Custom OpenBSD 6.9 amd64 Install CD" \ + -b 6.9/amd64/cdbr -c 6.9/amd64/boot.catalog \ + cd + -- cgit v1.2.3