From 5be569b9c3ef9bbf905167a07537825346d412c9 Mon Sep 17 00:00:00 2001 From: Patrick Spek Date: Sat, 7 May 2022 01:51:35 +0200 Subject: Publish bashtard post --- .../posts/2022/2022-04-25-bashtard-introduction.md | 265 --------------------- .../posts/2022/2022-05-07-bashtard-introduction.md | 264 ++++++++++++++++++++ 2 files changed, 264 insertions(+), 265 deletions(-) delete mode 100644 content/posts/2022/2022-04-25-bashtard-introduction.md create mode 100644 content/posts/2022/2022-05-07-bashtard-introduction.md diff --git a/content/posts/2022/2022-04-25-bashtard-introduction.md b/content/posts/2022/2022-04-25-bashtard-introduction.md deleted file mode 100644 index c416433..0000000 --- a/content/posts/2022/2022-04-25-bashtard-introduction.md +++ /dev/null @@ -1,265 +0,0 @@ ---- -draft: true -title: Configuring my Machines with Bashtard -date: 2022-04-25 -tags: -- Bash -- FreeBSD -- GNU+Linux -- Programming ---- - -Over the past couple weeks I've been spending some time here and there to work -on my own application for configuring my machines. Before this I've tried -Ansible, but found it to be very convoluted to use, and requires a lot of -conditionals if your machines aren't all running the same base system. - -So I made something in Bash, with a few abstractions to make certain -interactions less annoying to do manually every time. This used to be called -`tyilnet`, but I've discussed the setup with a few people on IRC, and decided it -would be a fun project to make it a bit more agnostic, so other people could -also easily start using it. This resulted in the creation of -[Bashtard](https://git.tyil.nl/bashtard/), pronounced as "bash", followed by -"tard". - -It works by simply writing Bash scripts to do the configuration, and provides -abstractions for using the system's package manager, service manager, and some -utilities such as logging and dealing with configured values. Configuration -values can be set on a per-host or per-OS basis. Since I run a varied base of -OSs, including Gentoo, Debian, and FreeBSD, the per-OS configuration comes in -very handy to me. - -As for the reason to use Bash, I chose it because most of the systems I run -already have this installed, so it doesn't add a dependency _most of the time_. -I would've liked to do it in POSIX sh, but I feel that when you're reaching a -certain level of complexity, Bash offers some very nice features which can make -your code cleaner, or less likely to contain bugs. Features such as `[[ ]]`, -`local`, and arrays come to mind. - -I've been kindly asked to guide potential new users to writing their first -Bashtard script, known as a _playbook_, so if you want to know about how it -works in practice, keep on reading. If you're satisfied with your current -configuration management system, this might not be quite as interesting to you, -so be warned. - -The first steps for a new user would obviously to install Bashtard, as it's not -in any OSs package repositories yet. A `Makefile` is supplied in the repository, -which should make this easy enough. - -```txt -git clone https://git.tyil.nl/bashtard -cd bashtard -sudo make install -hash -r -``` - -Once installed, it needs some initialization. - -```txt -bashtard init -``` - -This will create the basic structure in `/etc/bashtard`, including a -`playbooks.d`. Inside this `playbooks.d` directory, any directory is considered -to be a playbook, which requires a `description.txt` and a `playbook.bash`. - -```txt -cd /etc/bashtard/playbooks.d -mkdir ssh -cd ssh -echo "OpenSSH configuration" > description.txt -$EDITOR playbook.bash -``` - -The `playbook.bash` needs to contain 3 functions which are used by `bashtard`, a -`playbook_add()`, `playbook_sync()`, and `playbook_del()`. These will be called -by the `bashtard` subcommand `add`, `sync`, and `del` respectively. - -I generally start with the `playbook_sync()` function first, since this is the -function that'll ensure all the configurations are kept in sync with my desires. -I want to have my own `sshd_config`, which needs some templating for the -`Subsystem sftp` line. There's a `file_template` function provided by bashtard, -which does some very simple templating. I'll pass it the `sftp` variable to use. - -```bash -playbook_sync() { - file_template sshd_config \ - "sftp=$(config "ssh.sftp")" \ - > /etc/ssh/sshd_config -} -``` - -Now to create the actual template. The `file_template` function looks for -templates in the `share` directory inside the playbook directory. - -```txt -mkdir share -$EDITOR share/sshd_config -``` - -Since I already know what I want my `sshd_config` to look like from previous -installed systems, I'll just use that, but with a variable for the `Subsystem -sftp` value. - -```cfg -# Connectivity -Port 22 -AddressFamily any -ListenAddress 0.0.0.0 -ListenAddress :: - -# Fluff -PrintMotd yes - -# SFTP -Subsystem sftp ${sftp} - -# Authentication -AuthorizedKeysFile /etc/ssh/authorized_keys .ssh/authorized_keys -PermitRootLogin no -PasswordAuthentication no -ChallengeResponseAuthentication no -PubkeyAuthentication no - -# Allow tyil -Match User tyil - PubkeyAuthentication yes - -# Allow public key authentication over VPN -Match Address 10.57.0.0/16 - PubkeyAuthentication yes - PermitRootLogin prohibit-password -``` - -The `${sftp}` placeholder will be filled with whatever value is returned by -`config "ssh.sftp"`. And for this to work properly, we will need to define the -variable somewhere. These are written to the `etc` directory inside a playbook. -You can specify defaults in a file called `defaults`, and this can be -overwritten by OS-specific values, which in turn can be overwritten by -host-specific values. - -```txt -mkdir etc -$EDITOR etc/defaults -``` - -The format for these files is a very simple `key=value`. It splits on the first -`=` to determine what the key and value are. This means you can use `=` in your -values, but not your keys. - -```txt -ssh.sftp=/usr/lib/openssh/sftp-server -``` - -This value is correct for Debian and derivatives, but not for my Gentoo or -FreeBSD systems, so I've created OS-specific configuration files for these. - -```txt -mkdir etc/os.d -cat etc/os.d/linux-gentoo -ssh.sftp=/usr/lib64/misc/sftp-server -``` - -```txt -cat etc/os.d/freebsd -ssh.sftp=/usr/lib64/misc/sftp-server -``` - -My `sshd_config` template also specifies the use of a `Motd`, so that needs to -be created as well. This can again be done using the `template` function. - -```bash -file_template "motd" \ - "fqdn=${BASHTARD_PLATFORM[fqdn]}" \ - "time=$(date -u "+%FT%T")" \ - > /etc/motd -``` - -The `motd` template gets saved at `share/motd`. - -```txt - ████████╗██╗ ██╗██╗██╗ ███╗ ██╗███████╗████████╗ - ╚══██╔══╝╚██╗ ██╔╝██║██║ ████╗ ██║██╔════╝╚══██╔══╝ - ██║ ╚████╔╝ ██║██║ ██╔██╗ ██║█████╗ ██║ - ██║ ╚██╔╝ ██║██║ ██║╚██╗██║██╔══╝ ██║ - ██║ ██║ ██║███████╗██╗██║ ╚████║███████╗ ██║ - ╚═╝ ╚═╝ ╚═╝╚══════╝╚═╝╚═╝ ╚═══╝╚══════╝ ╚═╝ - -Welcome to ${fqdn}, last updated on ${time}. -``` - -Lastly, we want to ensure the SSH daemon gets reloaded after every sync, so -let's add that to the `playbook_sync()` function as well. - -```bash -svc reload "sshd" -``` - -The `svc` utility looks for a configuration value that starts with `svc.`, -followed by the service you're trying to act upon, so in this case that would be -`svc.sshd`. We can add this to our configuration files in `etc`. Across all my -machines, `sshd` seems to work as the value, so I only need to add one line to -`etc/defaults`. - -```txt -svc.sshd=sshd -``` - -This should take care of all the things I want automatically synced. The -`playbook_add()` function is intended for all one-time setup required for any -playbook. In this case that means the SSH daemon's service needs to be -activated, since it is not active by default on all my setups. - -```bash -playbook_add() { - svc enable "sshd" - svc start "sshd" -} -``` - -However, `add` does not call a `sync`, and I don't want my SSH service to run -with default configuration until a `sync` is initialized. So before enabling and -starting the service, I will call `sync` manually, by running a `playbook_sync` -first. This in turn, however, poses another problem, as `playbook_sync()` wants -to reload the service, which it can't do unless it is already running. To fix -this, I'll add an `if` statement to skip reloading if `bashtard` is running the -`add` command. - -```bash -playbook_add() { - playbook_sync - - svc enable "sshd" - svc start "sshd" -} - -playbook_sync() { - ... - - [[ $BASHTARD_COMMAND == "add" ]] && return - - svc reload "sshd" -} -``` - -Now, `bashtard add sshd` will run the `playbook_add()` function, which calls the -`playbook_sync()` function before enabling and starting the `sshd` service. All -that is left is the `playbook_del()` function, which only really needs to stop -and disable the service. The templated files can be removed here as well if -desired, of course. - -```bash -playbook_del() { - svc stop "sshd" - svc disable "sshd" -} -``` - -Lastly, I configured my `crond` to run `bashtard sync` every 20 minutes, so -whenever I update my configurations, it can take up to 20 minutes to propagate -to all my machines. Having an abstraction to deal with `cron` (or SystemD timers -where applicable) in Bashtard is something I'd like to add, but I have no -concrete plans on how to do this, yet. - -You can find the full `playbook.bash` source on -[git.tyil.nl](https://git.tyil.nl/tyilnet/tree/playbooks.d/ssh/playbook.bash?id=319ab064370cb1e65be115ffddf5c0cd519af2dd). diff --git a/content/posts/2022/2022-05-07-bashtard-introduction.md b/content/posts/2022/2022-05-07-bashtard-introduction.md new file mode 100644 index 0000000..f928ac8 --- /dev/null +++ b/content/posts/2022/2022-05-07-bashtard-introduction.md @@ -0,0 +1,264 @@ +--- +title: Configuring my Machines with Bashtard +date: 2022-05-07 +tags: +- Bash +- FreeBSD +- GNU+Linux +- Programming +--- + +Over the past couple weeks I've been spending some time here and there to work +on my own application for configuring my machines. Before this I've tried +Ansible, but found it to be very convoluted to use, and requires a lot of +conditionals if your machines aren't all running the same base system. + +So I made something in Bash, with a few abstractions to make certain +interactions less annoying to do manually every time. This used to be called +`tyilnet`, but I've discussed the setup with a few people on IRC, and decided it +would be a fun project to make it a bit more agnostic, so other people could +also easily start using it. This resulted in the creation of +[Bashtard](https://git.tyil.nl/bashtard/), pronounced as "bash", followed by +"tard". + +It works by simply writing Bash scripts to do the configuration, and provides +abstractions for using the system's package manager, service manager, and some +utilities such as logging and dealing with configured values. Configuration +values can be set on a per-host or per-OS basis. Since I run a varied base of +OSs, including Gentoo, Debian, and FreeBSD, the per-OS configuration comes in +very handy to me. + +As for the reason to use Bash, I chose it because most of the systems I run +already have this installed, so it doesn't add a dependency _most of the time_. +I would've liked to do it in POSIX sh, but I feel that when you're reaching a +certain level of complexity, Bash offers some very nice features which can make +your code cleaner, or less likely to contain bugs. Features such as `[[ ]]`, +`local`, and arrays come to mind. + +I've been kindly asked to guide potential new users to writing their first +Bashtard script, known as a _playbook_, so if you want to know about how it +works in practice, keep on reading. If you're satisfied with your current +configuration management system, this might not be quite as interesting to you, +so be warned. + +The first steps for a new user would obviously to install Bashtard, as it's not +in any OSs package repositories yet. A `Makefile` is supplied in the repository, +which should make this easy enough. + +```txt +git clone https://git.tyil.nl/bashtard +cd bashtard +sudo make install +hash -r +``` + +Once installed, it needs some initialization. + +```txt +bashtard init +``` + +This will create the basic structure in `/etc/bashtard`, including a +`playbooks.d`. Inside this `playbooks.d` directory, any directory is considered +to be a playbook, which requires a `description.txt` and a `playbook.bash`. + +```txt +cd /etc/bashtard/playbooks.d +mkdir ssh +cd ssh +echo "OpenSSH configuration" > description.txt +$EDITOR playbook.bash +``` + +The `playbook.bash` needs to contain 3 functions which are used by `bashtard`, a +`playbook_add()`, `playbook_sync()`, and `playbook_del()`. These will be called +by the `bashtard` subcommand `add`, `sync`, and `del` respectively. + +I generally start with the `playbook_sync()` function first, since this is the +function that'll ensure all the configurations are kept in sync with my desires. +I want to have my own `sshd_config`, which needs some templating for the +`Subsystem sftp` line. There's a `file_template` function provided by bashtard, +which does some very simple templating. I'll pass it the `sftp` variable to use. + +```bash +playbook_sync() { + file_template sshd_config \ + "sftp=$(config "ssh.sftp")" \ + > /etc/ssh/sshd_config +} +``` + +Now to create the actual template. The `file_template` function looks for +templates in the `share` directory inside the playbook directory. + +```txt +mkdir share +$EDITOR share/sshd_config +``` + +Since I already know what I want my `sshd_config` to look like from previous +installed systems, I'll just use that, but with a variable for the `Subsystem +sftp` value. + +```cfg +# Connectivity +Port 22 +AddressFamily any +ListenAddress 0.0.0.0 +ListenAddress :: + +# Fluff +PrintMotd yes + +# SFTP +Subsystem sftp ${sftp} + +# Authentication +AuthorizedKeysFile /etc/ssh/authorized_keys .ssh/authorized_keys +PermitRootLogin no +PasswordAuthentication no +ChallengeResponseAuthentication no +PubkeyAuthentication no + +# Allow tyil +Match User tyil + PubkeyAuthentication yes + +# Allow public key authentication over VPN +Match Address 10.57.0.0/16 + PubkeyAuthentication yes + PermitRootLogin prohibit-password +``` + +The `${sftp}` placeholder will be filled with whatever value is returned by +`config "ssh.sftp"`. And for this to work properly, we will need to define the +variable somewhere. These are written to the `etc` directory inside a playbook. +You can specify defaults in a file called `defaults`, and this can be +overwritten by OS-specific values, which in turn can be overwritten by +host-specific values. + +```txt +mkdir etc +$EDITOR etc/defaults +``` + +The format for these files is a very simple `key=value`. It splits on the first +`=` to determine what the key and value are. This means you can use `=` in your +values, but not your keys. + +```txt +ssh.sftp=/usr/lib/openssh/sftp-server +``` + +This value is correct for Debian and derivatives, but not for my Gentoo or +FreeBSD systems, so I've created OS-specific configuration files for these. + +```txt +mkdir etc/os.d +cat etc/os.d/linux-gentoo +ssh.sftp=/usr/lib64/misc/sftp-server +``` + +```txt +cat etc/os.d/freebsd +ssh.sftp=/usr/lib64/misc/sftp-server +``` + +My `sshd_config` template also specifies the use of a `Motd`, so that needs to +be created as well. This can again be done using the `template` function. + +```bash +file_template "motd" \ + "fqdn=${BASHTARD_PLATFORM[fqdn]}" \ + "time=$(date -u "+%FT%T")" \ + > /etc/motd +``` + +The `motd` template gets saved at `share/motd`. + +```txt + ████████╗██╗ ██╗██╗██╗ ███╗ ██╗███████╗████████╗ + ╚══██╔══╝╚██╗ ██╔╝██║██║ ████╗ ██║██╔════╝╚══██╔══╝ + ██║ ╚████╔╝ ██║██║ ██╔██╗ ██║█████╗ ██║ + ██║ ╚██╔╝ ██║██║ ██║╚██╗██║██╔══╝ ██║ + ██║ ██║ ██║███████╗██╗██║ ╚████║███████╗ ██║ + ╚═╝ ╚═╝ ╚═╝╚══════╝╚═╝╚═╝ ╚═══╝╚══════╝ ╚═╝ + +Welcome to ${fqdn}, last updated on ${time}. +``` + +Lastly, we want to ensure the SSH daemon gets reloaded after every sync, so +let's add that to the `playbook_sync()` function as well. + +```bash +svc reload "sshd" +``` + +The `svc` utility looks for a configuration value that starts with `svc.`, +followed by the service you're trying to act upon, so in this case that would be +`svc.sshd`. We can add this to our configuration files in `etc`. Across all my +machines, `sshd` seems to work as the value, so I only need to add one line to +`etc/defaults`. + +```txt +svc.sshd=sshd +``` + +This should take care of all the things I want automatically synced. The +`playbook_add()` function is intended for all one-time setup required for any +playbook. In this case that means the SSH daemon's service needs to be +activated, since it is not active by default on all my setups. + +```bash +playbook_add() { + svc enable "sshd" + svc start "sshd" +} +``` + +However, `add` does not call a `sync`, and I don't want my SSH service to run +with default configuration until a `sync` is initialized. So before enabling and +starting the service, I will call `sync` manually, by running a `playbook_sync` +first. This in turn, however, poses another problem, as `playbook_sync()` wants +to reload the service, which it can't do unless it is already running. To fix +this, I'll add an `if` statement to skip reloading if `bashtard` is running the +`add` command. + +```bash +playbook_add() { + playbook_sync + + svc enable "sshd" + svc start "sshd" +} + +playbook_sync() { + ... + + [[ $BASHTARD_COMMAND == "add" ]] && return + + svc reload "sshd" +} +``` + +Now, `bashtard add sshd` will run the `playbook_add()` function, which calls the +`playbook_sync()` function before enabling and starting the `sshd` service. All +that is left is the `playbook_del()` function, which only really needs to stop +and disable the service. The templated files can be removed here as well if +desired, of course. + +```bash +playbook_del() { + svc stop "sshd" + svc disable "sshd" +} +``` + +Lastly, I configured my `crond` to run `bashtard sync` every 20 minutes, so +whenever I update my configurations, it can take up to 20 minutes to propagate +to all my machines. Having an abstraction to deal with `cron` (or SystemD timers +where applicable) in Bashtard is something I'd like to add, but I have no +concrete plans on how to do this, yet. + +You can find the full `playbook.bash` source on +[git.tyil.nl](https://git.tyil.nl/tyilnet/tree/playbooks.d/ssh/playbook.bash?id=319ab064370cb1e65be115ffddf5c0cd519af2dd). -- cgit v1.1