diff options
Diffstat (limited to '_drafts')
| -rw-r--r-- | _drafts/dotfiles-under-revision-control.md | 83 | ||||
| -rw-r--r-- | _drafts/my-preferred-tools.md | 34 | ||||
| -rw-r--r-- | _drafts/ssh-proxy.md | 296 | ||||
| -rw-r--r-- | _drafts/temporary-postgresql-server.md | 100 |
4 files changed, 513 insertions, 0 deletions
diff --git a/_drafts/dotfiles-under-revision-control.md b/_drafts/dotfiles-under-revision-control.md new file mode 100644 index 0000000..4cccea3 --- /dev/null +++ b/_drafts/dotfiles-under-revision-control.md @@ -0,0 +1,83 @@ +--- +title: "Dotfiles under revision control" +description: "How I manage my configuration files with Git." +published: 2019-09-02 +--- + +<!-- TODO: Test commands --> + +I regularly spend some time to fit my preferred tools to my personal need and taste. +Luckily, most command-line tools and a growing number of graphical tools, accept configuration files -- commonly called _dotfiles_ because of the typical dot at the beginnig of the file name, e.g., `.tmux.conf`. +This post describes my not-so-special way to put these dotfiles under revision control using Git and Bash. +In fact, you'll find a myriad of public dotfiles repositories on the web, for example on [GitHub](https://github.com/search?q=dotfiles). + +Add the following snippet to your `~/.bashrc`. +The first line defines a `dotfiles` alias for `git` to distinguish your dotfiles repository from any other Git repository in and below your home directory. +The remaining lines reuse -- you might say hack -- Git's Bash completion for the alias. + +```sh +alias dotfiles="git --git-dir=\${HOME}/.dotfiles/ --work-tree=\${HOME}" +if [ -f /usr/share/git/completion/git-completion.bash ]; then + source /usr/share/git/completion/git-completion.bash + __git_complete dotfiles __git_main +fi +``` + +Initialize a bare Git repository for your dotfiles, and tell Git to ignore untracked files. + +```sh +mkdir ~/.dotfiles +git -C ~/.dotfiles init --bare +dotfiles config status.showUntrackedFiles no +``` + +Now you can `add`, `commit`, and `push` your dotfiles as usual. + +You can even add other repositories as submodules. +The following snippet, for example, adds Vim and Tmux plug-ins for the acclaimed [Solarized](https://ethanschoonover.com/solarized/) color scheme. + +```sh +mkdir -p ~/.tmux/plugins +cd ~/.tmux/plugins +dotfiles submodule add https://github.com/seebi/tmux-colors-solarized.git + +mkdir -p ~/.vim/pack/stefan/{start,opt} +cd ~/.vim/pack/stefan/start +dotfiles submodule add https://github.com/altercation/vim-colors-solarized.git + +dotfiles add ~/gitmodules +``` + +Update the submodules as always. + +```sh +dotfiles submodule update --remote --merge +``` + +Generate help tags. + +```sh +for d in ~/.vim/pack/stefan/*/*/doc; do + vim -u NONE -c "helptags $d" -c q +done +``` + +Finally, clone your dotfiles to another machine. +Be careful to clone into a temporary directory, though. +Otherwise you might screw up your home directory. + +```sh +git clone \ + --recurse-submodules \ + --separate-git-dir=$HOME/.dotfiles \ + example.com:~/git/dotfiles ~/dotfiles-tmp +rm ~/dotfiles-tmp/.git +cp -ai ~/dotfiles-tmp/.* ~ +rm -r ~/dotfiles-tmp +dotfiles config status.showUntrackedFiles no +``` + +That's it. Happy tracking! + +P.S. Did you know that Unix' hidden files were a mistake? +See [this archived post](https://web.archive.org/web/20190318012059/https://plus.google.com/101960720994009339267/posts/R58WgWwN9jp) by Rob Pike. diff --git a/_drafts/my-preferred-tools.md b/_drafts/my-preferred-tools.md new file mode 100644 index 0000000..b0a1d65 --- /dev/null +++ b/_drafts/my-preferred-tools.md @@ -0,0 +1,34 @@ +--- +title: "My preferred tools" +description: "A whirlwind tour of my preferred command-line tools." +date: 2019-06-29 +--- + +Tools shape our perception of problems. +That is, one's proficiency with a certain set of tools, and personal preference for some tools over others, suggests and sometimes limits, not only how we tackle a given problem, but also _what_ exactly we attempt to solve. +Hence, let me introduce you to my preferred tools so you may get a feeling for my view on software. + +_Disclaimer:_ +Follow the hyperlinks at your own risk. +You might end up in a rabbit hole. + +I prefer to use and _compose_ small, sharp tools as coined by Eric Steve Raymond in his book [The Art of Unix Programming](http://www.catb.org/esr/writings/taoup/). +Think [sed](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/sed.html), [rsync](https://rsync.samba.org/), [cURL](https://curl.haxx.se/), and [friends](https://en.wikipedia.org/w/index.php?title=List_of_Unix_commands&oldid=892119460). +As such, I feel most comfortable on the command-line of Unix-like, do-it-yourself operating systems like [Arch Linux](https://www.archlinux.org/), [FreeBSD](https://www.freebsd.org/), and [OpenBSD](https://www.openbsd.org/). +Most of the time, you'll see my terminal [Solarized](https://ethanschoonover.com/solarized/) and tiled by [tmux](https://github.com/tmux/tmux). + +When it comes to editing source code and markup, I rely on _the ubiquitous text editor_ [Vim](https://www.vim.org/) with a few, handpicked plug-ins like [Fugitive](https://github.com/tpope/vim-fugitive). +Needless to say, I prefer the keyboard over a mouse or touch interface --- a split keyboard with a Dvorak layout and a compose key to draw German umlauts. + +I love to search my files at the speed of light using [fzf](https://github.com/junegunn/fzf) and [ripgrep](https://github.com/BurntSushi/ripgrep), both inside and outside of Vim. +I enjoy to version control my files --- not just code but also configuration, notes, and this very blog --- with [Git](https://git-scm.com/). +As a matter of fact, even my favorite password manager, [pass](https://www.passwordstore.org/), uses Git under the hood to track changes. + +Writing of source code, I favor two, arguably polar opposite programming languages. +On the one hand, I love [Go](https://golang.org/) for its straightforwardness, exhaustive standard library and excellent tooling. +On the other hand, I am deeply attracted by the expressiveness and safety of [Haskell](https://www.haskell.org/). +Considering documentation, I generally forechoose [Asciidoctor](https://asciidoctor.org/)'s extension of [Asciidoc](http://asciidoc.org/) over anybody's flavor of [Markdown](https://daringfireball.net/projects/markdown/). +Finally, I prefer to store and evaluate large amounts of structured information using [SQLite](https://sqlite.org/index.html) and [PostgreSQL](https://www.postgresql.org/). + +When you read this far without loosing yourself in the numerous hyperlinks above, you deserve yourself a cookie. +Otherwise, remember that a tool without a purpose is a toy --- and that's fine. diff --git a/_drafts/ssh-proxy.md b/_drafts/ssh-proxy.md new file mode 100644 index 0000000..26a68c0 --- /dev/null +++ b/_drafts/ssh-proxy.md @@ -0,0 +1,296 @@ +--- +title: "SSH Proxy" +description: "How to establish a secure tunnel between two firewalled machines." +published: 2020-06-22 +--- + +Consumer grade internet connections typically prevent inbound connections by means of network address translation (NAT) or firewalls. +While this protects consumers to some degree from the evil outside, it also hinders them from providing network services. +Luckily, people came up with several [NAT traversal techniques](https://en.wikipedia.org/w/index.php?title=NAT_traversal&oldid=950406393#Techniques) including the widespread [SOCKS](https://en.wikipedia.org/w/index.php?title=SOCKS&oldid=963014782) protocol and the martial [UDP hole punching](https://en.wikipedia.org/w/index.php?title=UDP_hole_punching&oldid=957144154). +However, I found that [OpenSSH](https://www.openssh.com/)'s little-known remote port forwarding feature enables a simple and yet secure alternative that I would like to share in this post. + +We'll use the [OpenSSH](https://www.openssh.com/) client and server here because its a free, battle-tested, and portable implementation of the SSH protocol. +Chances are, that your operating system ships with OpenSSH built-in or packaged. +In fact, even [macOS](https://support.apple.com/guide/remote-desktop/about-systemsetup-apd95406b8d/mac) and [Windows 10](https://docs.microsoft.com/en-us/windows-server/administration/openssh/openssh_overview) include OpenSSH, activatable from the command-line. + +Contents: + +* TCP port forwarding. Anyone on the proxy may access the exposed service. +* End-to-end encrypted tunnel. Servers needs to run local SSH server. +* Unix domain socket forwarding. Protected socket instead of TCP port. + +## TL;DR + +Basically, you can ... + +You've got three different options to share a local network service by means of a publically reachable OpenSSH proxy: + +1. Create a remote port forwarding from the proxy to the server and + +Authentication and authorization aside, you can share a local network service in three steps: + +1. Create a remote port forwarding from a publically reachable proxy to the firewalled server: + + $ ssh -nNTR 8080:localhost:80 proxy.example.com + +1. Create a local port forwarding from the client to the proxy: + + $ ssh -nNTL 8080:localhost:8080 proxy.example.com + +1. Access the exposed network service from the client: + + $ curl http://localhost:8080/ + +So far so good, but the real work is of course to setup the public key authentication and to restrict the keys' permissions. +Thus, the remainder of this blog post describes a typical use case in detail. + +## Screen sharing example + +Let's say, Alex wants to share his screen with Tyler. +Sure enough, Alex can run a local [RFB](https://www.iana.org/assignments/rfb/rfb.xml) server such as [TigerVNC](https://tigervnc.org/) or [Apple Remote Desktop](https://support.apple.com/remote-desktop) on port 5900. +But thanks to their internet provider's NAT, neither of the two may connect to the other directly. +Fortunately, Tyler has access to a publicly reachable OpenSSH server that they can use as a forward proxy as follows. + +Note: +OpenSSH is developed as a part of [OpenBSD](https://www.openbsd.org/). +Thus, it comes with excellent [man pages](https://www.openssh.com/manual.html). +So please _read the fine manual_ to understand the various options that we'll use. +I cannot describe them better. + +First, Alex starts an OpenSSH server on his machine. +By default, the SSH server should listen on port 22 of all local addresses and permit both, password and public key authentication, see [`sshd_config(5)`](https://man.openbsd.org/sshd_config). +This is sufficient for our setup. +However, the truly paranoid may safely bind the SSH server to the loopback interface only and forbid password authentication: + + ListenAddress 127.0.0.1 + ListenAddress ::1 + Port 22 + PasswordAuthentication no + PubkeyAuthentication yes + +Second, Alex prepares a [`known_hosts`](https://man.openbsd.org/sshd#SSH_KNOWN_HOSTS_FILE_FORMAT) file for Tyler: + + $ cat /etc/ssh/ssh_host_*_key.pub \ + | sed -e 's/^/alex_workstation /' \ + | tee -a alex_public_host_keys + alex_workstation ssh-dss AAAAB3N... root@alex.localdomain + alex_workstation ssh-ed25519 AAAAC3N... root@alex.localdomain + alex_workstation ssh-rsa AAAAB3N... root@alex.localdomain + +Third, Alex generates an SSH key to authenticate at the proxy. +We'll generate a dedicated key here, so you can follow this guide without using your normal SSH key: + + $ ssh-keygen -f ~/.ssh/id_proxy + +Fourth, Tyler imports Alex' public host keys on his machine: + + $ cat alex_public_host_keys >> ~/.ssh/known_hosts + +Fifth, Tyler authorizes Alex to connect to the proxy. +That is, Tyler adds Alex' public user key to his personal [`authorized_keys`](https://man.openbsd.org/sshd#AUTHORIZED_KEYS_FILE_FORMAT) file on the proxy, applying the following restrictions to prevent Alex from misusing Tyler's account: + + restrict,command="echo 'ssh command restricted by authorized_keys'",port-forwarding,permitlisten="2222" ssh-rsa AAAAB3N... + +The truly paranoid may wish to create a dedicated [`nologin(8)`](https://man.openbsd.org/nologin) user account on the proxy instead and apply the above restrictions by means of the [`sshd_config(5)`](https://man.openbsd.org/sshd_config): + + Match User port-forward-only + DisableForwarding yes + ForceCommand echo 'ssh command forced by sshd_config' + PermitTTY no + AllowTcpForwarding remote + PermitListen 2222 + +Sixth, Tyler prepares a [`known_hosts`](https://man.openbsd.org/sshd#SSH_KNOWN_HOSTS_FILE_FORMAT) file for Alex: + + $ cat /etc/ssh/ssh_host_*_key.pub \ + | sed -e 's/^/[proxy.example.com]:22 /' \ + | tee -a proxy_public_host_keys + [proxy.example.com]:22 ssh-dss AAAAB3N... root@proxy.example.com + [proxy.example.com]:22 ssh-ed25519 AAAAC3N... root@proxy.example.com + [proxy.example.com]:22 ssh-rsa AAAAB3N... root@proxy.example.com + +Seventh, Alex imports the public host keys of the proxy on his machine: + + $ cat proxy_public_host_keys >> ~/.ssh/known_hosts + +Eighth, Alex authorizes Tyler to connect to his machine. +That is, Alex adds Tyler's public user key to his personal [`authorized_keys`](https://man.openbsd.org/sshd#AUTHORIZED_KEYS_FILE_FORMAT) file on his machine, applying the following restrictions to prevent Tyler from misusing Alex' account: + + restrict,command="echo 'ssh command restricted by authorized_keys'",port-forwarding,permitopen="5900" ssh-rsa AAAAB3N... + +Again, the truly paranoid may wish to create a dedicated user account as described above. + +Ninth, Alex connects to the proxy and forwards port 2222 to port 22 on his machine: + + $ ssh \ + -nNT \ + -R 2222:localhost:22 \ + -i ~/.ssh/id_proxy \ + -o "IdentitiesOnly yes" \ + -o "StrictHostKeyChecking yes" \ + -o "ExitOnForwardFailure yes" \ + -l tyler \ + proxy.example.com + +Alternatively, Alex may add the following settings to his personal [`ssh_config(5)`](https://man.openbsd.org/ssh_config) and run `ssh -nN proxy`: + + Host proxy + Hostname proxy.example.com + Port 22 + User alex + StrictHostKeyChecking yes + IdentitiesFile ~/.ssh/id_proxy + IdentitiesOnly yes + RemoteForward 2222 localhost:22 + ExitOnForwardFailure yes + RequestTTY no + + Host * + Protocol 2 + AddKeysToAgent yes + IgnoreUnknown UseKeychain + UseKeychain yes + + +Tenth, Tyler connects to Alex' machine and forwards port 5900 from his machine to Alex' machine: + + $ ssh \ + -J tyler@proxy.example.com \ + -L 5900:localhost:5900 \ + -o "ExitOnForwardFailure yes" \ + -o "StrictHostKeyChecking yes" \ + -l alex \ + -p 2222 \ + -o "HostKeyAlias alex_workstation" \ + localhost + +Alternatively, Tyler may add the following settings to his personal [`ssh_config(5)`](https://man.openbsd.org/ssh_config) and run `ssh alex_workstation`: + + Host alex_workstation + ProxyJump tyler@proxy.example.com + User alex + Hostname localhost + Port 2222 + HostKeyAlias alex_workstation + StrictHostKeyChecking yes + ExitOnForwardFailure yes + LocalForward 5900 localhost:5900 + + Host * + AddKeysToAgent yes + IgnoreUnknown UseKeychain + UseKeychain yes + +Finally, Tyler accesses the tunneled RFD server from his machine. +For example, using TigerVNC's [`vncviewer(1)`](https://tigervnc.org/doc/vncviewer.html)[^vncviewer] + + $ vncviewer localhost + +[^vncviewer]: +Apparently, the [TigerVNC](https://tigervnc.org/)'s client, [`vncviewer(1)`](https://tigervnc.org/doc/vncviewer.html), may not connect to an [Apple Remote Desktop](https://support.apple.com/remote-desktop) agent, even if you enable the legacy VNC option of the latter. +However, you can use [FreeRDP](https://www.freerdp.com/) instead, or one of its graphical front-ends like [Gnome Boxes](https://wiki.gnome.org/Apps/Boxes). + +TODO: permitlisten="none",permitopen="none" + +## Conclusion + +We've established a secure, end-to-end encrypted tunnel between two otherwise disconnected machines through an OpenSSH forward proxy. +Moreover, the two parties need minimal trust in each other because we've restricted the keys' permissions. + +OpenSSH enables even more fine grained control if need be. +For example: + +* Apply an `expiry-time` to your `authorized_keys` to grant temporary access. + +* Add a trusted `cert-authority` to your `authorized_keys` instead of individual keys. + +* Enable `VerifyHostKeyDNS` to automatically trust host keys with a corresponding SSHFP resource record in the DNS. + +* Use [Unix domain sockets](https://en.wikipedia.org/wiki/Unix_domain_socket) instead of network ports to further restrict access to the tunnel from the proxy to the actual server.[^socket] + +[^socket]: +Simply replace the port number with an absolute path like `/home/tyler/proxy.sock`. +Do not use the `~` to represent the user's home directory. + +# Rewrite + +## Goal + +Forward a (protected) Unix domain socket on the proxy to a local port. +Such that nobody else on the proxy may use the forwarding, as with a forwarded TCP port. + +Serve the current directory at http://127.0.0.1:8080/: + + python3 -m http.server --bind 127.0.0.1 8080 + +Using explicit loopback address 127.0.0.1 (or ::1) instead of localhost lest socat or ssh shoud bind to a non-loopback address. + +Want either end-to-end encryption (using SSH server on the server) or protected address (Unix domain socket instead of TCP port) on the proxy. + +## OpenSSH RemoteForward + +Works as advertised. +Missing `PermitListen` (and `permitlisten`) equivalent to restrict the name of the socket. + +## OpenBSD netcat + +Idea: manually bind to remote socket using netcat. +Problem: Doesn't even work locally. Neither on Arch Linux nor on OpenBSD. + +Forward Unix domain socket test.sock to TCP port 8080 using OpenBSD's [`nc`](https://man.openbsd.org/nc): + + #! /bin/sh + rm -f backpipe + mkfifo backpipe + nc -lkU test.sock 0<backpipe \ + | nc 127.0.0.1 8080 1>backpipe + +Retrieve the home page http://127.0.0.1:8080/ via the socket: + + printf "GET / HTTP/1.0\r\n\r\n" \ + | nc -UN test.sock + +Unfortunately, this doesn't work reliably. +The retrieval command prints the response at most once, ofter on the second invocation and NOT on the first invocation. + +I found neither the cause of this behavior nor a workaround. +I suspect, the problem is a combination of the following: + +* Netcat closes connection to early because of an early EOF + +* Shell pipe errors + +* Shell buffers standard input/output + +Moreover, each Linux appears to implement a slightly different version of OpenBSD's netcat. + +## socat + +Forward remote socket test.sock to local port 8080: + + socat EXEC:'ssh -T engine.skreutz.com socat "UNIX-LISTEN:test.sock,fork,unlink-early" STDIO' TCP4:127.0.0.1:8080,fork + +TODO: Test local nc and remote socat, because it might be easier to find a nc implementation for Windows than socat. + +Forward local port 8081 to remote socket test.sock: + + ssh -vnNT -L 127.0.0.1:8081:/home/stefan/test.sock -o "ExitOnForwardFailure yes" engine.skreutz.com + +Note: Specify the local bind address 127.0.0.1 and option `ExitOnForwardFailure` to make `ssh` fail if `GatewayPorts` is set to `yes`. + +Noto: Specify the absolute path of the socket. Do not rely on the ~. + +Retrieve home page: + + printf "GET / HTTP/1.0\r\n\r\n" \ + | nc -N 127.0.0.1 8081 + +Alternatively: + + curl http://127.0.0.1:8081/ + +TODO: Restrict authorized_keys +TODO: Inspect ports with netstat. +TODO: permitlisten="none",permitopen="none". +TODO: Request OpenSSH feature PermitStreamLocalListen/Open analoguous to PermitListen/Open. +TODO: Test tcpserver instead of socat: https://cr.yp.to/ucspi-tcp.html diff --git a/_drafts/temporary-postgresql-server.md b/_drafts/temporary-postgresql-server.md new file mode 100644 index 0000000..a46cd86 --- /dev/null +++ b/_drafts/temporary-postgresql-server.md @@ -0,0 +1,100 @@ +--- +title: "Temporary PostgreSQL server" +description: "A simple shell script to run the PostgreSQL server off a temporary directory." +published: 2020-06-02 +--- + +Sometimes I need to spin up a local PostgreSQL server for one-off purposes. +In these cases, I don't particularly like to either configure PostgreSQL manually or use a pre-configured Docker image because I experience this as overkill and inaccessible respectively. +Instead, I use the simple shell script below to run the PostgreSQL server off a temporary directory until I decide to `kill` it. +The script is inspired by a [blog post](https://www.johbo.com/2017/on-demand-postgresql-for-your-development-environment.html) by Johannes Bornhold that reminded me of Unix' simplicity. + +The script essentially performs seven steps: + +* Create a temporary directory using `mktemp` +* Initialize the directory using `initdb` +* Serve the directory using `postgres` +* Ensure the server is up using `pg_isready` +* Create a database using `createdb` +* Wait for a `SIGINT` +* Remove the temporary directory + +Obviously, you still need to install PostgreSQL to use the script. +However, you may use the [Nix package manager](https://nixos.org/nix/) to install PostgreSQL _on-the-fly_ and have it removed too, if you are into this. +Simply put the following shebang in front of the script. + +```sh +#! /usr/bin/env nix-shell +#! nix-shell --pure --packages postgresql -i bash +``` + +Here is the full script with minimal error handling. + +```sh +#! /bin/sh + +# temp_postgres runs a PostgreSQL server with a temporary data directory until +# it receives a SIGINT. + +set -o nounset + +# Remove the temporary directory before exiting +trap 'quit' INT +quit() { + code="${1:-0}" + trap '' INT TERM + kill -TERM 0 + wait + rm -rf "${tmpdir-}" || { + >&2 printf "temp_postgres: failed to remove temporary directory \"%s\"\\n" "${tmpdir}" + [ "${code}" -ne 0 ] || code=1 + } + exit "${code}" +} + +# Parse arguments +[ $# -eq 2 ] || { + >&2 printf "temp_postgres: invalid arguments\\n" + printf "Usage: temp_postgres <dbname> <username>\\n" + quit 1 +} +dbname="$1" +username="$2" + +# Create a temporary directory +tmpdir="$( mktemp --directory )" || { + >&2 printf "temp_postgres: failed to create temporary directory\\n" + quit 1 +} + +# Initialize the directory +initdb --pgdata="${tmpdir}" --username="${username}" || { + >&2 printf "temp_postgres: failed to initialize database\\n" + quit 1 +} + +# Serve the directory +( postgres -k "${tmpdir}" -D "${tmpdir}" </dev/null ) & + +# Test the connection +sleep 1 +pg_isready --host="${tmpdir}" --dbname="postgres" --username="${username}" --timeout=10 || { + >&2 printf "temp_postgres: failed to connect to server\\n" + quit 1 +} + +# Create a database +createdb --host="${tmpdir}" --username="${username}" --no-password "${dbname}" || { + >&2 printf "temp_postgres: failed to create database\\n" + quit 1 +} + +printf ' +Connect with the following command: + +\tpsql --host=localhost "%s" "%s" + +' "${dbname}" "${username}" + +wait +``` |