diff options
44 files changed, 793 insertions, 1044 deletions
@@ -1,5 +1,6 @@ -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later +dist tmp diff --git a/CHANGELOG.md b/CHANGELOG.md index e31930f..1680fac 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,5 @@ <!-- -SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +SPDX-FileCopyrightText: 2024 Patrick Spek <p.spek@tyil.nl> SPDX-License-Identifier: AGPL-3.0-or-later --> @@ -11,7 +11,57 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [UNRELEASED] + +### Fixed + +- `config_subkeys` and `config_subkeys_for` should now leave off the `&` at the + end of a variable names which are references. + +## [2.1.0] - 2024-04-02 + +### Added + +- Configuration variables can be assigned values of other variables with the + `&=` assignment. This allows a single value to be re-used dynamically, rather + than having to explicitly set the same value several times. +- A `zap` command has been added to remove a playbook from the registry without + running the playbook's `playbook_del()` function. This is intended to easily + remove registry entries when a playbook itself has been deleted or is + otherwise broken in a way that the regular `del` subcommand cannot fix. + +### Changed + +- The `description.txt` is now allowed to be used without the `.txt` suffix. + Usage with the `.txt` suffix continues to be supported as well. + +### Fixed + +- Passing an empty string as default value to `config` should now properly + return an empty string without a warning about the configuration key not + existing. + +## [2.0.2] - 2024-02-28 + +### Fixed + +- Configuration values with `=` in their value part should now work properly + with `file_template`. Keys with `=` in them are still *not supported*. + +## [2.0.1] - 2023-09-25 + +### Added + +- A new `make` target has been added to build a .tar.gz distributable. + +### Changed + +- The `svc_` utils should now check which init service you're using when using a + linux system. The supported options are still only openrc and systemd. +- The `pull` subcommand should now properly return with exit-code 0 if no + problem were encountered. + +## [2.0.0] - 2023-05-22 ### Added @@ -28,6 +78,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 GNU+Linux distributions. - The `$BASHTARD_PLATFORM` variable now contains an additional entry, `init`, to allow for handling different init systems on GNU+Linux in a cleaner fashion. +- A `file_hash` utility function has been added. It currently uses `md5`, but is + written in such a fashion that this can easily be updated for the future. Its + intent is to encapsulate differences between naming and usage of hashing + utilities found on different systems. +- A `dir_hash` utility function has been added, which will give you a hash based + on the file contents of a directory. This function will find files + recursively, calculate a hash for each of them, and then calculate a hash + based on the total result. The intended goal is to allow running before and + after templating some files, to deduce whether something actually changed. +- A `diff` subcommand has been added to show all non-committed changes. It is a + convenience wrapper to avoid having to change directory and run `git diff` to + get an overview of all pending changes. +- A `pull` subcommand has been added to only pull the latest changes into the + `$BASHTARD_ETCDIR`, without running `sync` on all the playbooks. +- A new global variable, `$BASHTARD_PLAYBOOK_VARS` has been added. Currently, + its only purpose is to check for "required" variables to be used in the + playbook. Before an `add` or `sync`, any variables declared to be `required` + in the `$BASHTARD_PLAYBOOK_VARS` array will be checked to be non-empty. If any + are empty, an error will be thrown and the playbook will not be ran. +- A new directory has been added, `data.d`, for data that should be shared + between playbook runs. This new directory is intended to create a clearer + seperation between a playbook and a user's specific data used by the playbook, + which in turn should make re-using playbooks easier. +- A convenience function has been introduced, `playbook_path()`, which can give + you the absolute path to the playbook's base or data directory. +- A `top` subcommand has been added to give some generic information of all + nodes known to Bashtard. It uses information from the `sysinfo` subcommand, + which it will pull in through an `ssh` invocation. ### Changed @@ -41,6 +119,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 passed to the `pkg_*()` function will be used by the host's package manager. - `datetime()` now always passes `-u` on to `date`. - All manpages now include a `NAME` section. +- The `sync` subcomman will now `stash` any changes before it attempts to + `pull`. Afterwards, `stash pop` will be ran to apply the last `stash`ed + changes again. ## [1.0.0] - 2022-05-06 @@ -1,6 +1,6 @@ #!/usr/bin/env make -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -9,9 +9,12 @@ ETCDIR=/etc # Variables for building (binary) packages PREFIX= -PKG_VERSION=0.0.0 +PKG_VERSION=2.1.0 PKG_WORKDIR:=$(or $(PKG_WORKDIR),$(shell mktemp -d)) PKG_ROOT:=$(PKG_WORKDIR)/bashtard-$(PKG_VERSION) +SOURCE_DATE_EPOCH:=$(shell git show -s --format=%ct) + +rwildcard=$(foreach d,$(wildcard $(1:=/*)),$(call rwildcard,$d,$2) $(filter $(subst *,%,$2),$d)) all: : # No build step required, use `make install` to install this program @@ -49,28 +52,53 @@ man: pkg-debian: # Create the directories to work with - mkdir -pv "$(PKG_ROOT)" - - # Create an original, and use it for building - tar czf "$(PKG_WORKDIR)/bashtard_$(PKG_VERSION).orig.tar.gz" . - cd "$(PKG_WORKDIR)" && tar xzf "bashtard_$(PKG_VERSION).orig.tar.gz" -C "$(PKG_ROOT)" - - # Move Debian packaging info - mv -v "$(PKG_ROOT)/etc/debian" "$(PKG_ROOT)/debian" - - # Make all Bash files +x to please Lintian - find "$(PKG_ROOT)" -iname "*.bash" -exec chmod +x {} \; - - # Build the Debian package - cd "$(PKG_ROOT)" && dpkg-source --build . - cd "$(PKG_ROOT)" && dpkg-buildpackage - - # TODO: Remove lintian, that's a manual step - lintian --no-tag-display-limit "$(PKG_WORKDIR)/bashtard_$(PKG_VERSION)-1_amd64.deb" + mkdir -pv -- "$(PKG_ROOT)/usr" + + # Install into the workdir + $(MAKE) PREFIX="$(PKG_ROOT)" install + $(MAKE) PREFIX="$(PKG_ROOT)" man + + # Copy Debian's control file + mkdir -pv -- "$(PKG_ROOT)/DEBIAN" + printf "Version: %s\n" "$(PKG_VERSION)" > "$(PKG_ROOT)/DEBIAN/control" + grep -v '^\#' ./etc/debian/control >> "$(PKG_ROOT)/DEBIAN/control" + + # Include the changelog + mkdir -pv -- "$(PKG_ROOT)/usr/share/doc/bashtard" + gzip -9n < CHANGELOG.md > "$(PKG_ROOT)/usr/share/doc/bashtard/changelog.gz" + + # Include license + cp -v -- LICENSES/AGPL-3.0-or-later.txt "$(PKG_ROOT)/usr/share/doc/bashtard/copyright" + + # Debian wants all .bash files to be +x + find "$(PKG_ROOT)" -type f -iname "*.bash" | xargs chmod +x + + # Generate the .deb file + mkdir -pv -- "dist" + dpkg-deb -b "$(PKG_ROOT)" "dist/bashtard-$(PKG_VERSION).deb" + +pkg-targz: + mkdir -pv -- "dist" + tar \ + --exclude dist \ + --group=0 \ + --mtime="@$(SOURCE_DATE_EPOCH)" \ + --numeric-owner \ + --owner=0 \ + --pax-option="exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime" \ + --sort=name \ + --transform "s@^./@bashtard-$(PKG_VERSION)/@" \ + --exclude-vcs \ + --exclude-vcs-ignores \ + -cvzf "dist/bashtard-$(PKG_VERSION).tar.gz" . qa: + # Check licenses reuse lint - shellcheck -x bin/bashtard **/*.bash + + # Check code + shellcheck -s sh bin/bashtard + shellcheck -s bash $(call rwildcard,.,*.bash) share: mkdir -pv -- "$(PREFIX)$(DESTDIR)/share/bashtard" @@ -1,5 +1,5 @@ <!-- -SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> SPDX-License-Identifier: AGPL-3.0-or-later --> @@ -65,19 +65,6 @@ called for the `add`, `sync`, end `del` subactions respectively. Using these functions, one can automate the setup, synchronization, or disabling of certain components of a system. -#### Local - -A local playbook is simply a directory made in `/etc/bashtard/playbooks.d` with -the two files specified earlier. - -#### Remote - -A remote playbook is recorded in `/etc/bashtard/playbooks.d/remotes`. This is a -space-seperated file, containing 3 columns per line, `name`, `url`, and -`branch`. When the `sync` subcommand is called, the latest commits on the -specified `branch` are pulled before calling `playbook_sync()` on them. The -structure of the actual playbook is the same as a local one. - ### SSH Bashtard comes with an `ssh` subcommand to allow the user to run any command on diff --git a/etc/debian/README.Debian b/etc/debian/README.Debian deleted file mode 100644 index b912eed..0000000 --- a/etc/debian/README.Debian +++ /dev/null @@ -1,6 +0,0 @@ -bashtard for Debian ------------------- - -<possible notes regarding this package - if none, delete this file> - - -- root <root@unknown> Wed, 24 Aug 2022 21:09:03 +0000 diff --git a/etc/debian/README.source b/etc/debian/README.source deleted file mode 100644 index 505676b..0000000 --- a/etc/debian/README.source +++ /dev/null @@ -1,10 +0,0 @@ -bashtard for Debian ------------------- - -<this file describes information about the source package, see Debian policy -manual section 4.14. You WILL either need to modify or delete this file> - - - - -- root <root@unknown> Wed, 24 Aug 2022 21:09:03 +0000 - diff --git a/etc/debian/bashtard-docs.docs b/etc/debian/bashtard-docs.docs deleted file mode 100644 index 7319041..0000000 --- a/etc/debian/bashtard-docs.docs +++ /dev/null @@ -1,2 +0,0 @@ -README.source -README.Debian diff --git a/etc/debian/bashtard.cron.d.ex b/etc/debian/bashtard.cron.d.ex deleted file mode 100644 index f6ec714..0000000 --- a/etc/debian/bashtard.cron.d.ex +++ /dev/null @@ -1,4 +0,0 @@ -# -# Regular cron jobs for the bashtard package -# -0 4 * * * root [ -x /usr/bin/bashtard_maintenance ] && /usr/bin/bashtard_maintenance diff --git a/etc/debian/bashtard.doc-base.EX b/etc/debian/bashtard.doc-base.EX deleted file mode 100644 index 5855377..0000000 --- a/etc/debian/bashtard.doc-base.EX +++ /dev/null @@ -1,20 +0,0 @@ -Document: bashtard -Title: Debian bashtard Manual -Author: <insert document author here> -Abstract: This manual describes what bashtard is - and how it can be used to - manage online manuals on Debian systems. -Section: unknown - -Format: debiandoc-sgml -Files: /usr/share/doc/bashtard/bashtard.sgml.gz - -Format: postscript -Files: /usr/share/doc/bashtard/bashtard.ps.gz - -Format: text -Files: /usr/share/doc/bashtard/bashtard.text.gz - -Format: HTML -Index: /usr/share/doc/bashtard/html/index.html -Files: /usr/share/doc/bashtard/html/*.html diff --git a/etc/debian/changelog b/etc/debian/changelog deleted file mode 100644 index adf9d49..0000000 --- a/etc/debian/changelog +++ /dev/null @@ -1,5 +0,0 @@ -bashtard (2.0.0-1) unstable; urgency=medium - - * Initial release to Debian (Closes: #1010851) - - -- Patrick Spek <p.spek+debian@tyil.work> Wed, 24 Aug 2022 21:09:03 +0000 diff --git a/etc/debian/control b/etc/debian/control index 1358461..91cab85 100644 --- a/etc/debian/control +++ b/etc/debian/control @@ -1,17 +1,12 @@ -Source: bashtard -Section: admin -Priority: optional -Maintainer: Patrick Spek <p.spek+debian@tyil.nl> -Build-Depends: debhelper-compat (= 13) -Build-Depends-Indep: scdoc -Standards-Version: 4.5.1 -Homepage: https://git.tyil.nl/bashtard -Rules-Requires-Root: no - +#SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> +# +#SPDX-License-Identifier: AGPL-3.0-or-later Package: bashtard -Architecture: any -Depends: ${shlibs:Depends}, ${misc:Depends} -Description: Simple configuration management in Bash +Architecture: all +Maintainer: Patrick Spek <p.spek@tyil.nl> +Priority: optional +Section: admin +Description: Configuration Management System in Bash Bashtard is a configuration management system built on the idea of simplicity for the user. It lets you write reasonably simple Bash scripts to configure your systems, while providing just enough abstractions to make it easy to diff --git a/etc/debian/copyright b/etc/debian/copyright deleted file mode 100644 index 6efc10a..0000000 --- a/etc/debian/copyright +++ /dev/null @@ -1,43 +0,0 @@ -Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ -Upstream-Name: bashtard -Upstream-Contact: <preferred name and address to reach the upstream project> -Source: <url://example.com> - -Files: * -Copyright: <years> <put author's name and email here> - <years> <likewise for another author> -License: <special license> - <Put the license of the package here indented by 1 space> - <This follows the format of Description: lines in control file> - . - <Including paragraphs> - -# If you want to use GPL v2 or later for the /debian/* files use -# the following clauses, or change it to suit. Delete these two lines -Files: debian/* -Copyright: 2022 root <root@unknown> -License: GPL-2+ - This package is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 2 of the License, or - (at your option) any later version. - . - This package is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - . - You should have received a copy of the GNU General Public License - along with this program. If not, see <https://www.gnu.org/licenses/> - . - On Debian systems, the complete text of the GNU General - Public License version 2 can be found in "/usr/share/common-licenses/GPL-2". - -# Please also look if there are files or directories which have a -# different copyright/license attached and list them here. -# Please avoid picking licenses with terms that are more restrictive than the -# packaged work, as it may make Debian's contributions unacceptable upstream. -# -# If you need, there are some extra license texts available in two places: -# /usr/share/debhelper/dh_make/licenses/ -# /usr/share/common-licenses/ diff --git a/etc/debian/manpage.1.ex b/etc/debian/manpage.1.ex deleted file mode 100644 index e4d9f0b..0000000 --- a/etc/debian/manpage.1.ex +++ /dev/null @@ -1,56 +0,0 @@ -.\" Hey, EMACS: -*- nroff -*- -.\" (C) Copyright 2022 root <root@unknown>, -.\" -.\" First parameter, NAME, should be all caps -.\" Second parameter, SECTION, should be 1-8, maybe w/ subsection -.\" other parameters are allowed: see man(7), man(1) -.TH Bashtard SECTION "August 24 2022" -.\" Please adjust this date whenever revising the manpage. -.\" -.\" Some roff macros, for reference: -.\" .nh disable hyphenation -.\" .hy enable hyphenation -.\" .ad l left justify -.\" .ad b justify to both left and right margins -.\" .nf disable filling -.\" .fi enable filling -.\" .br insert line break -.\" .sp <n> insert n+1 empty lines -.\" for manpage-specific macros, see man(7) -.SH NAME -bashtard \- program to do something -.SH SYNOPSIS -.B bashtard -.RI [ options ] " files" ... -.br -.B bar -.RI [ options ] " files" ... -.SH DESCRIPTION -This manual page documents briefly the -.B bashtard -and -.B bar -commands. -.PP -.\" TeX users may be more comfortable with the \fB<whatever>\fP and -.\" \fI<whatever>\fP escape sequences to invode bold face and italics, -.\" respectively. -\fBbashtard\fP is a program that... -.SH OPTIONS -These programs follow the usual GNU command line syntax, with long -options starting with two dashes (`-'). -A summary of options is included below. -For a complete description, see the Info files. -.TP -.B \-h, \-\-help -Show summary of options. -.TP -.B \-v, \-\-version -Show version of program. -.SH SEE ALSO -.BR bar (1), -.BR baz (1). -.br -The programs are documented fully by -.IR "The Rise and Fall of a Fooish Bar" , -available via the Info system. diff --git a/etc/debian/manpage.sgml.ex b/etc/debian/manpage.sgml.ex deleted file mode 100644 index cc74e42..0000000 --- a/etc/debian/manpage.sgml.ex +++ /dev/null @@ -1,154 +0,0 @@ -<!doctype refentry PUBLIC "-//OASIS//DTD DocBook V4.1//EN" [ - -<!-- Process this file with docbook-to-man to generate an nroff manual - page: `docbook-to-man manpage.sgml > manpage.1'. You may view - the manual page with: `docbook-to-man manpage.sgml | nroff -man | - less'. A typical entry in a Makefile or Makefile.am is: - -manpage.1: manpage.sgml - docbook-to-man $< > $@ - - - The docbook-to-man binary is found in the docbook-to-man package. - Please remember that if you create the nroff version in one of the - debian/rules file targets (such as build), you will need to include - docbook-to-man in your Build-Depends control field. - - --> - - <!-- Fill in your name for FIRSTNAME and SURNAME. --> - <!ENTITY dhfirstname "<firstname>FIRSTNAME</firstname>"> - <!ENTITY dhsurname "<surname>SURNAME</surname>"> - <!-- Please adjust the date whenever revising the manpage. --> - <!ENTITY dhdate "<date>August 24 2022</date>"> - <!-- SECTION should be 1-8, maybe w/ subsection other parameters are - allowed: see man(7), man(1). --> - <!ENTITY dhsection "<manvolnum>SECTION</manvolnum>"> - <!ENTITY dhemail "<email>root@unknown</email>"> - <!ENTITY dhusername "root"> - <!ENTITY dhucpackage "<refentrytitle>Bashtard</refentrytitle>"> - <!ENTITY dhpackage "bashtard"> - - <!ENTITY debian "<productname>Debian</productname>"> - <!ENTITY gnu "<acronym>GNU</acronym>"> - <!ENTITY gpl "&gnu; <acronym>GPL</acronym>"> -]> - -<refentry> - <refentryinfo> - <address> - &dhemail; - </address> - <author> - &dhfirstname; - &dhsurname; - </author> - <copyright> - <year>2003</year> - <holder>&dhusername;</holder> - </copyright> - &dhdate; - </refentryinfo> - <refmeta> - &dhucpackage; - - &dhsection; - </refmeta> - <refnamediv> - <refname>&dhpackage;</refname> - - <refpurpose>program to do something</refpurpose> - </refnamediv> - <refsynopsisdiv> - <cmdsynopsis> - <command>&dhpackage;</command> - - <arg><option>-e <replaceable>this</replaceable></option></arg> - - <arg><option>--example <replaceable>that</replaceable></option></arg> - </cmdsynopsis> - </refsynopsisdiv> - <refsect1> - <title>DESCRIPTION</title> - - <para>This manual page documents briefly the - <command>&dhpackage;</command> and <command>bar</command> - commands.</para> - - <para>This manual page was written for the &debian; distribution - because the original program does not have a manual page. - Instead, it has documentation in the &gnu; - <application>Info</application> format; see below.</para> - - <para><command>&dhpackage;</command> is a program that...</para> - - </refsect1> - <refsect1> - <title>OPTIONS</title> - - <para>These programs follow the usual &gnu; command line syntax, - with long options starting with two dashes (`-'). A summary of - options is included below. For a complete description, see the - <application>Info</application> files.</para> - - <variablelist> - <varlistentry> - <term><option>-h</option> - <option>--help</option> - </term> - <listitem> - <para>Show summary of options.</para> - </listitem> - </varlistentry> - <varlistentry> - <term><option>-v</option> - <option>--version</option> - </term> - <listitem> - <para>Show version of program.</para> - </listitem> - </varlistentry> - </variablelist> - </refsect1> - <refsect1> - <title>SEE ALSO</title> - - <para>bar (1), baz (1).</para> - - <para>The programs are documented fully by <citetitle>The Rise and - Fall of a Fooish Bar</citetitle> available via the - <application>Info</application> system.</para> - </refsect1> - <refsect1> - <title>AUTHOR</title> - - <para>This manual page was written by &dhusername; &dhemail; for - the &debian; system (and may be used by others). Permission is - granted to copy, distribute and/or modify this document under - the terms of the &gnu; General Public License, Version 2 any - later version published by the Free Software Foundation. - </para> - <para> - On Debian systems, the complete text of the GNU General Public - License can be found in /usr/share/common-licenses/GPL. - </para> - - </refsect1> -</refentry> - -<!-- Keep this comment at the end of the file -Local variables: -mode: sgml -sgml-omittag:t -sgml-shorttag:t -sgml-minimize-attributes:nil -sgml-always-quote-attributes:t -sgml-indent-step:2 -sgml-indent-data:t -sgml-parent-document:nil -sgml-default-dtd-file:nil -sgml-exposed-tags:nil -sgml-local-catalogs:nil -sgml-local-ecat-files:nil -End: ---> diff --git a/etc/debian/manpage.xml.ex b/etc/debian/manpage.xml.ex deleted file mode 100644 index 367972f..0000000 --- a/etc/debian/manpage.xml.ex +++ /dev/null @@ -1,291 +0,0 @@ -<?xml version='1.0' encoding='UTF-8'?> -<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN" -"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [ - -<!-- - -`xsltproc -''-nonet \ - -''-param man.charmap.use.subset "0" \ - -''-param make.year.ranges "1" \ - -''-param make.single.year.ranges "1" \ - /usr/share/xml/docbook/stylesheet/docbook-xsl/manpages/docbook.xsl \ - manpage.xml' - -A manual page <package>.<section> will be generated. You may view the -manual page with: nroff -man <package>.<section> | less'. A typical entry -in a Makefile or Makefile.am is: - -DB2MAN = /usr/share/sgml/docbook/stylesheet/xsl/docbook-xsl/manpages/docbook.xsl -XP = xsltproc -''-nonet -''-param man.charmap.use.subset "0" - -manpage.1: manpage.xml - $(XP) $(DB2MAN) $< - -The xsltproc binary is found in the xsltproc package. The XSL files are in -docbook-xsl. A description of the parameters you can use can be found in the -docbook-xsl-doc-* packages. Please remember that if you create the nroff -version in one of the debian/rules file targets (such as build), you will need -to include xsltproc and docbook-xsl in your Build-Depends control field. -Alternatively use the xmlto command/package. That will also automatically -pull in xsltproc and docbook-xsl. - -Notes for using docbook2x: docbook2x-man does not automatically create the -AUTHOR(S) and COPYRIGHT sections. In this case, please add them manually as -<refsect1> ... </refsect1>. - -To disable the automatic creation of the AUTHOR(S) and COPYRIGHT sections -read /usr/share/doc/docbook-xsl/doc/manpages/authors.html. This file can be -found in the docbook-xsl-doc-html package. - -Validation can be done using: `xmllint -''-noout -''-valid manpage.xml` - -General documentation about man-pages and man-page-formatting: -man(1), man(7), http://www.tldp.org/HOWTO/Man-Page/ - ---> - - <!-- Fill in your name for FIRSTNAME and SURNAME. --> - <!ENTITY dhfirstname "FIRSTNAME"> - <!ENTITY dhsurname "SURNAME"> - <!-- dhusername could also be set to "&dhfirstname; &dhsurname;". --> - <!ENTITY dhusername "root"> - <!ENTITY dhemail "root@unknown"> - <!-- SECTION should be 1-8, maybe w/ subsection other parameters are - allowed: see man(7), man(1) and - http://www.tldp.org/HOWTO/Man-Page/q2.html. --> - <!ENTITY dhsection "SECTION"> - <!-- TITLE should be something like "User commands" or similar (see - http://www.tldp.org/HOWTO/Man-Page/q2.html). --> - <!ENTITY dhtitle "bashtard User Manual"> - <!ENTITY dhucpackage "Bashtard"> - <!ENTITY dhpackage "bashtard"> -]> - -<refentry> - <refentryinfo> - <title>&dhtitle;</title> - <productname>&dhpackage;</productname> - <authorgroup> - <author> - <firstname>&dhfirstname;</firstname> - <surname>&dhsurname;</surname> - <contrib>Wrote this manpage for the Debian system.</contrib> - <address> - <email>&dhemail;</email> - </address> - </author> - </authorgroup> - <copyright> - <year>2007</year> - <holder>&dhusername;</holder> - </copyright> - <legalnotice> - <para>This manual page was written for the Debian system - (and may be used by others).</para> - <para>Permission is granted to copy, distribute and/or modify this - document under the terms of the GNU General Public License, - Version 2 or (at your option) any later version published by - the Free Software Foundation.</para> - <para>On Debian systems, the complete text of the GNU General Public - License can be found in - <filename>/usr/share/common-licenses/GPL</filename>.</para> - </legalnotice> - </refentryinfo> - <refmeta> - <refentrytitle>&dhucpackage;</refentrytitle> - <manvolnum>&dhsection;</manvolnum> - </refmeta> - <refnamediv> - <refname>&dhpackage;</refname> - <refpurpose>program to do something</refpurpose> - </refnamediv> - <refsynopsisdiv> - <cmdsynopsis> - <command>&dhpackage;</command> - <!-- These are several examples, how syntaxes could look --> - <arg choice="plain"><option>-e <replaceable>this</replaceable></option></arg> - <arg choice="opt"><option>--example=<parameter>that</parameter></option></arg> - <arg choice="opt"> - <group choice="req"> - <arg choice="plain"><option>-e</option></arg> - <arg choice="plain"><option>--example</option></arg> - </group> - <replaceable class="option">this</replaceable> - </arg> - <arg choice="opt"> - <group choice="req"> - <arg choice="plain"><option>-e</option></arg> - <arg choice="plain"><option>--example</option></arg> - </group> - <group choice="req"> - <arg choice="plain"><replaceable>this</replaceable></arg> - <arg choice="plain"><replaceable>that</replaceable></arg> - </group> - </arg> - </cmdsynopsis> - <cmdsynopsis> - <command>&dhpackage;</command> - <!-- Normally the help and version options make the programs stop - right after outputting the requested information. --> - <group choice="opt"> - <arg choice="plain"> - <group choice="req"> - <arg choice="plain"><option>-h</option></arg> - <arg choice="plain"><option>--help</option></arg> - </group> - </arg> - <arg choice="plain"> - <group choice="req"> - <arg choice="plain"><option>-v</option></arg> - <arg choice="plain"><option>--version</option></arg> - </group> - </arg> - </group> - </cmdsynopsis> - </refsynopsisdiv> - <refsect1 id="description"> - <title>DESCRIPTION</title> - <para>This manual page documents briefly the - <command>&dhpackage;</command> and <command>bar</command> - commands.</para> - <para>This manual page was written for the Debian distribution - because the original program does not have a manual page. - Instead, it has documentation in the GNU <citerefentry> - <refentrytitle>info</refentrytitle> - <manvolnum>1</manvolnum> - </citerefentry> format; see below.</para> - <para><command>&dhpackage;</command> is a program that...</para> - </refsect1> - <refsect1 id="options"> - <title>OPTIONS</title> - <para>The program follows the usual GNU command line syntax, - with long options starting with two dashes (`-'). A summary of - options is included below. For a complete description, see the - <citerefentry> - <refentrytitle>info</refentrytitle> - <manvolnum>1</manvolnum> - </citerefentry> files.</para> - <variablelist> - <!-- Use the variablelist.term.separator and the - variablelist.term.break.after parameters to - control the term elements. --> - <varlistentry> - <term><option>-e <replaceable>this</replaceable></option></term> - <term><option>--example=<replaceable>that</replaceable></option></term> - <listitem> - <para>Does this and that.</para> - </listitem> - </varlistentry> - <varlistentry> - <term><option>-h</option></term> - <term><option>--help</option></term> - <listitem> - <para>Show summary of options.</para> - </listitem> - </varlistentry> - <varlistentry> - <term><option>-v</option></term> - <term><option>--version</option></term> - <listitem> - <para>Show version of program.</para> - </listitem> - </varlistentry> - </variablelist> - </refsect1> - <refsect1 id="files"> - <title>FILES</title> - <variablelist> - <varlistentry> - <term><filename>/etc/foo.conf</filename></term> - <listitem> - <para>The system-wide configuration file to control the - behaviour of <application>&dhpackage;</application>. See - <citerefentry> - <refentrytitle>foo.conf</refentrytitle> - <manvolnum>5</manvolnum> - </citerefentry> for further details.</para> - </listitem> - </varlistentry> - <varlistentry> - <term><filename>${HOME}/.foo.conf</filename></term> - <listitem> - <para>The per-user configuration file to control the - behaviour of <application>&dhpackage;</application>. See - <citerefentry> - <refentrytitle>foo.conf</refentrytitle> - <manvolnum>5</manvolnum> - </citerefentry> for further details.</para> - </listitem> - </varlistentry> - </variablelist> - </refsect1> - <refsect1 id="environment"> - <title>ENVIRONMENT</title> - <variablelist> - <varlistentry> - <term><envar>FOO_CONF</envar></term> - <listitem> - <para>If used, the defined file is used as configuration - file (see also <xref linkend="files"/>).</para> - </listitem> - </varlistentry> - </variablelist> - </refsect1> - <refsect1 id="diagnostics"> - <title>DIAGNOSTICS</title> - <para>The following diagnostics may be issued - on <filename class="devicefile">stderr</filename>:</para> - <variablelist> - <varlistentry> - <term><errortext>Bad configuration file. Exiting.</errortext></term> - <listitem> - <para>The configuration file seems to contain a broken configuration - line. Use the <option>--verbose</option> option, to get more info. - </para> - </listitem> - </varlistentry> - </variablelist> - <para><command>&dhpackage;</command> provides some return codes, that can - be used in scripts:</para> - <segmentedlist> - <segtitle>Code</segtitle> - <segtitle>Diagnostic</segtitle> - <seglistitem> - <seg><errorcode>0</errorcode></seg> - <seg>Program exited successfully.</seg> - </seglistitem> - <seglistitem> - <seg><errorcode>1</errorcode></seg> - <seg>The configuration file seems to be broken.</seg> - </seglistitem> - </segmentedlist> - </refsect1> - <refsect1 id="bugs"> - <!-- Or use this section to tell about upstream BTS. --> - <title>BUGS</title> - <para>The program is currently limited to only work - with the <package>foobar</package> library.</para> - <para>The upstreams <acronym>BTS</acronym> can be found - at <ulink url="http://bugzilla.foo.tld"/>.</para> - </refsect1> - <refsect1 id="see_also"> - <title>SEE ALSO</title> - <!-- In alpabetical order. --> - <para><citerefentry> - <refentrytitle>bar</refentrytitle> - <manvolnum>1</manvolnum> - </citerefentry>, <citerefentry> - <refentrytitle>baz</refentrytitle> - <manvolnum>1</manvolnum> - </citerefentry>, <citerefentry> - <refentrytitle>foo.conf</refentrytitle> - <manvolnum>5</manvolnum> - </citerefentry></para> - <para>The programs are documented fully by <citetitle>The Rise and - Fall of a Fooish Bar</citetitle> available via the <citerefentry> - <refentrytitle>info</refentrytitle> - <manvolnum>1</manvolnum> - </citerefentry> system.</para> - </refsect1> -</refentry> - diff --git a/etc/debian/postinst.ex b/etc/debian/postinst.ex deleted file mode 100644 index 650b2ee..0000000 --- a/etc/debian/postinst.ex +++ /dev/null @@ -1,39 +0,0 @@ -#!/bin/sh -# postinst script for bashtard -# -# see: dh_installdeb(1) - -set -e - -# summary of how this script can be called: -# * <postinst> `configure' <most-recently-configured-version> -# * <old-postinst> `abort-upgrade' <new version> -# * <conflictor's-postinst> `abort-remove' `in-favour' <package> -# <new-version> -# * <postinst> `abort-remove' -# * <deconfigured's-postinst> `abort-deconfigure' `in-favour' -# <failed-install-package> <version> `removing' -# <conflicting-package> <version> -# for details, see https://www.debian.org/doc/debian-policy/ or -# the debian-policy package - - -case "$1" in - configure) - ;; - - abort-upgrade|abort-remove|abort-deconfigure) - ;; - - *) - echo "postinst called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -# dh_installdeb will replace this with shell code automatically -# generated by other debhelper scripts. - -#DEBHELPER# - -exit 0 diff --git a/etc/debian/postrm.ex b/etc/debian/postrm.ex deleted file mode 100644 index 8a98cd0..0000000 --- a/etc/debian/postrm.ex +++ /dev/null @@ -1,37 +0,0 @@ -#!/bin/sh -# postrm script for bashtard -# -# see: dh_installdeb(1) - -set -e - -# summary of how this script can be called: -# * <postrm> `remove' -# * <postrm> `purge' -# * <old-postrm> `upgrade' <new-version> -# * <new-postrm> `failed-upgrade' <old-version> -# * <new-postrm> `abort-install' -# * <new-postrm> `abort-install' <old-version> -# * <new-postrm> `abort-upgrade' <old-version> -# * <disappearer's-postrm> `disappear' <overwriter> -# <overwriter-version> -# for details, see https://www.debian.org/doc/debian-policy/ or -# the debian-policy package - - -case "$1" in - purge|remove|upgrade|failed-upgrade|abort-install|abort-upgrade|disappear) - ;; - - *) - echo "postrm called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -# dh_installdeb will replace this with shell code automatically -# generated by other debhelper scripts. - -#DEBHELPER# - -exit 0 diff --git a/etc/debian/preinst.ex b/etc/debian/preinst.ex deleted file mode 100644 index cfb9d1f..0000000 --- a/etc/debian/preinst.ex +++ /dev/null @@ -1,35 +0,0 @@ -#!/bin/sh -# preinst script for bashtard -# -# see: dh_installdeb(1) - -set -e - -# summary of how this script can be called: -# * <new-preinst> `install' -# * <new-preinst> `install' <old-version> -# * <new-preinst> `upgrade' <old-version> -# * <old-preinst> `abort-upgrade' <new-version> -# for details, see https://www.debian.org/doc/debian-policy/ or -# the debian-policy package - - -case "$1" in - install|upgrade) - ;; - - abort-upgrade) - ;; - - *) - echo "preinst called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -# dh_installdeb will replace this with shell code automatically -# generated by other debhelper scripts. - -#DEBHELPER# - -exit 0 diff --git a/etc/debian/prerm.ex b/etc/debian/prerm.ex deleted file mode 100644 index 2e0bee4..0000000 --- a/etc/debian/prerm.ex +++ /dev/null @@ -1,38 +0,0 @@ -#!/bin/sh -# prerm script for bashtard -# -# see: dh_installdeb(1) - -set -e - -# summary of how this script can be called: -# * <prerm> `remove' -# * <old-prerm> `upgrade' <new-version> -# * <new-prerm> `failed-upgrade' <old-version> -# * <conflictor's-prerm> `remove' `in-favour' <package> <new-version> -# * <deconfigured's-prerm> `deconfigure' `in-favour' -# <package-being-installed> <version> `removing' -# <conflicting-package> <version> -# for details, see https://www.debian.org/doc/debian-policy/ or -# the debian-policy package - - -case "$1" in - remove|upgrade|deconfigure) - ;; - - failed-upgrade) - ;; - - *) - echo "prerm called with unknown argument \`$1'" >&2 - exit 1 - ;; -esac - -# dh_installdeb will replace this with shell code automatically -# generated by other debhelper scripts. - -#DEBHELPER# - -exit 0 diff --git a/etc/debian/rules b/etc/debian/rules deleted file mode 100755 index 8750992..0000000 --- a/etc/debian/rules +++ /dev/null @@ -1,29 +0,0 @@ -#!/usr/bin/make -f -# See debhelper(7) (uncomment to enable) -# output every command that modifies files on the build system. -#export DH_VERBOSE = 1 - - -# see FEATURE AREAS in dpkg-buildflags(1) -#export DEB_BUILD_MAINT_OPTIONS = hardening=+all - -# see ENVIRONMENT in dpkg-buildflags(1) -# package maintainers to append CFLAGS -#export DEB_CFLAGS_MAINT_APPEND = -Wall -pedantic -# package maintainers to append LDFLAGS -#export DEB_LDFLAGS_MAINT_APPEND = -Wl,--as-needed - - -%: - dh $@ - -install: - make PREFIX="debian/bashtard" DESTDIR=/usr install - make PREFIX="debian/bashtard" man - - -# dh_make generated override targets -# This is example for Cmake (See https://bugs.debian.org/641051 ) -#override_dh_auto_configure: -# dh_auto_configure -- \ -# -DCMAKE_LIBRARY_PATH=$(DEB_HOST_MULTIARCH) diff --git a/etc/debian/salsa-ci.yml.ex b/etc/debian/salsa-ci.yml.ex deleted file mode 100644 index 260ebbe..0000000 --- a/etc/debian/salsa-ci.yml.ex +++ /dev/null @@ -1,11 +0,0 @@ -# For more information on what jobs are run see: -# https://salsa.debian.org/salsa-ci-team/pipeline -# -# To enable the jobs, go to your repository (at salsa.debian.org) -# and click over Settings > CI/CD > Expand (in General pipelines). -# In "Custom CI config path" write debian/salsa-ci.yml and click -# in "Save Changes". The CI tests will run after the next commit. ---- -include: - - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/salsa-ci.yml - - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/pipeline-jobs.yml diff --git a/etc/debian/source/format b/etc/debian/source/format deleted file mode 100644 index 163aaf8..0000000 --- a/etc/debian/source/format +++ /dev/null @@ -1 +0,0 @@ -3.0 (quilt) diff --git a/etc/debian/watch.ex b/etc/debian/watch.ex deleted file mode 100644 index 6b2116b..0000000 --- a/etc/debian/watch.ex +++ /dev/null @@ -1,38 +0,0 @@ -# Example watch control file for uscan -# Rename this file to "watch" and then you can run the "uscan" command -# to check for upstream updates and more. -# See uscan(1) for format - -# Compulsory line, this is a version 4 file -version=4 - -# PGP signature mangle, so foo.tar.gz has foo.tar.gz.sig -#opts="pgpsigurlmangle=s%$%.sig%" - -# HTTP site (basic) -#http://example.com/downloads.html \ -# files/bashtard-([\d\.]+)\.tar\.gz debian uupdate - -# Uncomment to examine an FTP server -#ftp://ftp.example.com/pub/bashtard-(.*)\.tar\.gz debian uupdate - -# SourceForge hosted projects -# http://sf.net/bashtard/ bashtard-(.*)\.tar\.gz debian uupdate - -# GitHub hosted projects -#opts="filenamemangle=s%(?:.*?)?v?(\d[\d.]*)\.tar\.gz%<project>-$1.tar.gz%" \ -# https://github.com/<user>/bashtard/tags \ -# (?:.*?/)?v?(\d[\d.]*)\.tar\.gz debian uupdate - -# PyPI -# https://pypi.debian.net/bashtard/bashtard-(.+)\.(?:zip|tgz|tbz|txz|(?:tar\.(?:gz|bz2|xz))) - -# Direct Git -# opts="mode=git" http://git.example.com/bashtard.git \ -# refs/tags/v([\d\.]+) debian uupdate - - - - -# Uncomment to find new files on GooglePages -# http://example.googlepages.com/foo.html bashtard-(.*)\.tar\.gz diff --git a/lib/main.bash b/lib/main.bash index bbfd602..821c2d2 100644 --- a/lib/main.bash +++ b/lib/main.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -34,6 +34,7 @@ main() { # Declare some global variables declare -A BASHTARD_PLATFORM + declare -A BASHTARD_PLAYBOOK_VARS # Figure out system details debug "$BASHTARD_NAME/main" "Discovering system information" @@ -41,6 +42,7 @@ main() { # Export BASHTARD_ variables export BASHTARD_PLATFORM + export BASHTARD_PLAYBOOK_VARS # Source the file defining the subcommand. debug "$BASHTARD_NAME/main" "Sourcing $subcommand_src" @@ -77,30 +79,44 @@ Usage: $BASHTARD_NAME -h $BASHTARD_NAME add <playbook> $BASHTARD_NAME del <playbook> + $BASHTARD_NAME diff $BASHTARD_NAME init [repository] $BASHTARD_NAME pkg <install|uninstall> <name> + $BASHTARD_NAME pull $BASHTARD_NAME ssh <command> $BASHTARD_NAME sync [playbook] $BASHTARD_NAME sysinfo + $BASHTARD_NAME top $BASHTARD_NAME var [-p <playbook>] <key> $BASHTARD_NAME var [-s] <key> <value> + $BASHTARD_NAME zap <playbook> Perform maintenance on your infra. Commands: add Add a configuration playbook to this machine. + del Remove a configuration playbook from this machine. + diff Show a diff of all uncommitted changes. init Initialize the $BASHTARD_NAME configuration system. pkg Interface into the pkg abstraction used by $BASHTARD_NAME. - del Remove a configuration playbook from this machine. + pull Pull the latest changes through git. ssh Run a given command on all known hosts. sync Pull latest changes through git, and synchronize all added playbooks. sysinfo Show gathered information about this system. + top Show resource information about all known hosts. var Show or set the value of a given configuration key. - -Playbooks: + zap Remove a playbook from the registry without attempting to run + the delete step from the playbook. EOF + if [[ ! -d "$BASHTARD_ETCDIR/playbooks.d" ]] + then + return 0 + fi + + printf "\nPlaybooks:\n" + # Find all playbooks mapfile -t playbooks < <(find \ "$BASHTARD_ETCDIR/playbooks.d"/* \ @@ -122,9 +138,19 @@ EOF # Render playbook descriptions for playbook in "${playbooks[@]}" do + local description="" + + if [[ -f "$playbook/description.txt" ]] + then + description="$(cat "$playbook/description.txt")" + elif [[ -f "$playbook/description" ]] + then + description="$(cat "$playbook/description")" + fi + printf "\t%-${playbook_longest}s %s\n" \ "$(basename "$playbook")" \ - "$(cat "$playbook/description.txt")" + "$description" done } @@ -210,6 +236,7 @@ discover_system_init() { discover_system_key() { local key + # shellcheck disable=SC2031 key+="${BASHTARD_PLATFORM[os]}" if [[ ${BASHTARD_PLATFORM[distro]} ]] diff --git a/lib/subcommands/add.bash b/lib/subcommands/add.bash index c72582b..db3ec40 100644 --- a/lib/subcommands/add.bash +++ b/lib/subcommands/add.bash @@ -1,12 +1,15 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later subcommand() { local buffer + local missing_vars=0 + local playbook_base + local playbook_registry export BASHTARD_PLAYBOOK="$1" ; shift @@ -16,8 +19,8 @@ subcommand() return 2 fi - local playbook_base="$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK" - local playbook_registry="$BASHTARD_ETCDIR/registry.d/${BASHTARD_PLATFORM[fqdn]}" + playbook_base="$(playbook_path "base")" + playbook_registry="$BASHTARD_ETCDIR/registry.d/${BASHTARD_PLATFORM[fqdn]}" # Make sure we only run add if the playbook is not in the registry yet if grep -Fqx "$BASHTARD_PLAYBOOK" "$playbook_registry" @@ -40,9 +43,32 @@ subcommand() return 1 fi - # shellcheck disable=SC1090 + # shellcheck disable=SC1090,SC1091 . "$playbook_base/playbook.bash" + # Ensure all required vars are non-empty + debug "bashtard/add" "Checking for \$BASHTARD_PLAYBOOK_VARS" + + for key in "${!BASHTARD_PLAYBOOK_VARS[@]}" + do + # shellcheck disable=SC2086 + in_args "required" ${BASHTARD_PLAYBOOK_VARS[$key]} || continue + + debug "bashtard/add" "Checking \$BASHTARD_PLAYBOOK_VARS[$key]" + + if [[ "$(config "$key")" == "" ]] + then + missing_vars=$(( missing_vars + 1)) + fi + done + + if (( 0 < missing_vars )) + then + emerg "bashtard/add" "One or more required variables are unset" + return 3 + fi + + # Run the playbook if ! playbook_add then crit "bashtard/add" "$BASHTARD_PLAYBOOK reported an error" diff --git a/lib/subcommands/backup.bash b/lib/subcommands/backup.bash index 9a23ede..3d56539 100644 --- a/lib/subcommands/backup.bash +++ b/lib/subcommands/backup.bash @@ -22,9 +22,9 @@ subcommand() { then debug "$BASHTARD_NAME/backup" "Retrieving elements from config" - while read -r element + while read -r key do - set -- "$@" "$(config "$element")" + set -- "$@" "$(config "bashtard.backup.elements.$key")" done < <(config_subkeys "bashtard.backup.elements") fi @@ -39,14 +39,12 @@ subcommand() { # Loop over all repositories, and run backup functions for each of them while read -r key do - index="$(awk -F. '{ print $NF }' <<< "$key")" - for element in "$@" do - "backup_$element" "$index" + "backup_$element" "$key" done - info "$BASHTARD_NAME/backup/$index" "Backups completed" + info "$BASHTARD_NAME/backup/$key" "Backups completed" done < <(config_subkeys "bashtard.backup.repositories") } @@ -55,7 +53,6 @@ backup_filesystem() { local borg local cmd_create local cmd_prune - local indexes borg="$(config "bashtard.backup.borg.command" "borg")" remote="$(config "bashtard.backup.borg.remote_paths.$index" "borg")" @@ -75,12 +72,13 @@ backup_filesystem() { "--keep-weekly" "$(config "bashtard.backup.keep.weekly")" "--keep-monthly" "$(config "bashtard.backup.keep.monthly")" "--keep-yearly" "$(config "bashtard.backup.keep.yearly")" - "--prefix" "{fqdn}-" + "--glob-archives" "${BASHTARD_PLATFORM[fqdn]}-" "--remote-path" "$remote" "$repo/hostfs" ) notice "$BASHTARD_NAME/backup/$index" "> ${cmd_prune[*]}" + # shellcheck disable=SC2068 ${cmd_prune[@]} # Create new backups @@ -93,12 +91,13 @@ backup_filesystem() { ) # Add all paths to the command - while read -r path + while read -r key do - cmd_create+=("$(config "$path")") + cmd_create+=("$(config "bashtard.backup.fs.paths.$key")") done < <(config_subkeys "bashtard.backup.fs.paths") notice "$BASHTARD_NAME/backup/$index" "> ${cmd_create[*]}" + # shellcheck disable=SC2068 ${cmd_create[@]} } @@ -145,12 +144,13 @@ backup_database_postgresql() { "--keep-weekly" "$(config "bashtard.backup.keep.weekly")" "--keep-monthly" "$(config "bashtard.backup.keep.monthly")" "--keep-yearly" "$(config "bashtard.backup.keep.yearly")" - "--prefix" "$database-" + "--glob-archives" "$database-" "--remote-path" "$remote" "$repo/postgresql-$database" ) notice "$BASHTARD_NAME/backup/$index" "> ${cmd_prune[*]}" + # shellcheck disable=SC2068 ${cmd_prune[@]} # Create new backups @@ -162,7 +162,8 @@ backup_database_postgresql() { "-" ) - notice "$BASHTARD_NAME/backup/$index" "> pg_dump "$database" | ${cmd_create[*]}" + notice "$BASHTARD_NAME/backup/$index" "> pg_dump $database | ${cmd_create[*]}" + # shellcheck disable=SC2068 pg_dump "$database" | ${cmd_create[@]} done < <(psql -AXt -d template1 -c "SELECT datname FROM pg_database") } diff --git a/lib/subcommands/del.bash b/lib/subcommands/del.bash index f5204d8..293e401 100644 --- a/lib/subcommands/del.bash +++ b/lib/subcommands/del.bash @@ -1,12 +1,14 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later subcommand() { local buffer + local playbook_base + local playbook_registry export BASHTARD_PLAYBOOK="$1" ; shift @@ -16,13 +18,13 @@ subcommand() return fi - local playbook_base="$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK" - local playbook_registry="$BASHTARD_ETCDIR/registry.d/${BASHTARD_PLATFORM[fqdn]}" + playbook_base="$(playbook_path "base")" + playbook_registry="$BASHTARD_ETCDIR/registry.d/${BASHTARD_PLATFORM[fqdn]}" # Make sure we only run add if the playbook is not in the registry yet if ! grep -Fqx "$BASHTARD_PLAYBOOK" "$playbook_registry" then - crit "bashtard/add" "'$BASHTARD_PLAYBOOK' is not registered for ${BASHTARD_PLATFORM[fqdn]}" + crit "bashtard/del" "'$BASHTARD_PLAYBOOK' is not registered for ${BASHTARD_PLATFORM[fqdn]}" return 3 fi @@ -30,17 +32,17 @@ subcommand() if [[ ! -d "$playbook_base" ]] then - emerg "bashtard/sync" "No such directory: $playbook_base" + emerg "bashtard/del" "No such directory: $playbook_base" return 1 fi if [[ ! -f "$playbook_base/playbook.bash" ]] then - emerg "bashtard/sync" "No such file: $playbook_base/playbook.bash" + emerg "bashtard/del" "No such file: $playbook_base/playbook.bash" return 1 fi - # shellcheck disable=SC1090 + # shellcheck disable=SC1090,SC1091 . "$playbook_base/playbook.bash" if ! playbook_del @@ -53,6 +55,6 @@ subcommand() # Remove the playbook from the registry cp -- "$playbook_registry" "$buffer" - grep -Fqvx "$BASHTARD_PLAYBOOK" "$buffer" \ + grep -Fvx "$BASHTARD_PLAYBOOK" "$buffer" \ | sort > "$playbook_registry" } diff --git a/lib/subcommands/diff.bash b/lib/subcommands/diff.bash new file mode 100644 index 0000000..714897f --- /dev/null +++ b/lib/subcommands/diff.bash @@ -0,0 +1,10 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> +# +# SPDX-License-Identifier: AGPL-3.0-or-later + +subcommand() +{ + git -C "$BASHTARD_ETCDIR" diff +} diff --git a/lib/subcommands/init.bash b/lib/subcommands/init.bash index 46497f5..824956e 100644 --- a/lib/subcommands/init.bash +++ b/lib/subcommands/init.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -23,6 +23,7 @@ init_local() local dirs=( "$BASHTARD_ETCDIR" "$BASHTARD_ETCDIR/conf.d" + "$BASHTARD_ETCDIR/data.d" "$BASHTARD_ETCDIR/hosts.d" "$BASHTARD_ETCDIR/os.d" "$BASHTARD_ETCDIR/playbooks.d" @@ -35,7 +36,6 @@ init_local() "$BASHTARD_ETCDIR/secrets" "$BASHTARD_ETCDIR/hosts.d/${BASHTARD_PLATFORM[fqdn]}" "$BASHTARD_ETCDIR/os.d/${BASHTARD_PLATFORM[key]}" - "$BASHTARD_ETCDIR/playbooks.d/remotes" ) for dir in "${dirs[@]}" @@ -62,7 +62,7 @@ init_remote() local remote="$1" ; shift notice "init" "Cloning $remote to $BASHTARD_ETCDIR" - git clone "$remote" "$BASHTARD_ETCDIR" + git clone --recurse-submodules "$remote" "$BASHTARD_ETCDIR" local files=( "$BASHTARD_ETCDIR/hosts.d/${BASHTARD_PLATFORM[fqdn]}" @@ -77,12 +77,4 @@ init_remote() notice "bashtard/init" "Creating $file" touch -- "$file" done - - while read -r playbook url _ - do - notice "bashtard/init" "Cloning $playbook from $url" - git clone "$url" "$BASHTARD_ETCDIR/playbooks.d/$playbook" - - # TODO: Clone appropriate branch - done < "$BASHTARD_ETCDIR/playbooks.d/remotes" } diff --git a/lib/subcommands/patch.bash b/lib/subcommands/patch.bash deleted file mode 100644 index 72350ef..0000000 --- a/lib/subcommands/patch.bash +++ /dev/null @@ -1,15 +0,0 @@ -#!/usr/bin/env bash - -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> -# -# SPDX-License-Identifier: AGPL-3.0-or-later - -subcommand() -{ - chgdir "$BASHTARD_ETCDIR" - git add . \ - && git diff --cached \ - && git reset \ - && git restore . \ - && git clean -fd -} diff --git a/lib/subcommands/pull.bash b/lib/subcommands/pull.bash new file mode 100644 index 0000000..8dfb48b --- /dev/null +++ b/lib/subcommands/pull.bash @@ -0,0 +1,24 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> +# +# SPDX-License-Identifier: AGPL-3.0-or-later + +subcommand() +{ + local dirty + + notice "bashtard/pull" "Pulling latest changes in $BASHTARD_ETCDIR" + + if ! git -C "$BASHTARD_ETCDIR" diff-index --quiet HEAD -- + then + dirty=1 + fi + + [[ -n $dirty ]] && git -C "$BASHTARD_ETCDIR" stash + git -C "$BASHTARD_ETCDIR" pull origin master || return 4 + git -C "$BASHTARD_ETCDIR" submodule update --init --recursive || return 4 + [[ -n $dirty ]] && git -C "$BASHTARD_ETCDIR" stash pop + + return 0 +} diff --git a/lib/subcommands/ssh.bash b/lib/subcommands/ssh.bash index 563aa73..3a781f3 100644 --- a/lib/subcommands/ssh.bash +++ b/lib/subcommands/ssh.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -20,7 +20,7 @@ subcommand() local host user="$(config_for "$node" "bashtard.ssh.user" "$USER")" - host="$(config_for "$node" "bashtard.ssh.host" "$(config_for "$node" "vpn.ipv4")")" + host="$(config_for "$node" "bashtard.ssh.host")" if [[ -z "$host" ]] then @@ -30,6 +30,7 @@ subcommand() notice "$BASHTARD_NAME/ssh" "$user@$node ($host) > $*" + # shellcheck disable=SC2029 ssh "$user@$host" "$@" unset user diff --git a/lib/subcommands/sync.bash b/lib/subcommands/sync.bash index 9537922..8d437f6 100644 --- a/lib/subcommands/sync.bash +++ b/lib/subcommands/sync.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -15,22 +15,8 @@ subcommand() return fi - notice "bashtard/sync" "Syncing $BASHTARD_ETCDIR" - pushd -- "$BASHTARD_ETCDIR" > /dev/null - git pull origin master || return 4 - popd > /dev/null - - # Otherwise, do a full sync - notice "bashtard/sync" "Syncing remote playbooks" - - # Update all playbook sources - while read -r playbook _ branch - do - pushd -- "$BASHTARD_ETCDIR/playbooks.d/$playbook" > /dev/null \ - || return 2 - git pull origin "$branch" - popd > /dev/null - done < "$BASHTARD_ETCDIR/playbooks.d/remotes" + # Pull latest changes + "$BASHTARD_BIN" pull # Run a sync for each registered playbook for this host while read -r playbook @@ -42,7 +28,10 @@ subcommand() sync_playbook() { - local playbook_base="$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK" + local playbook_base + local missing_vars=0 + + playbook_base="$(playbook_path "base")" notice "bashtard/sync" "Running sync for $BASHTARD_PLAYBOOK" @@ -58,8 +47,31 @@ sync_playbook() return 1 fi - # shellcheck disable=SC1090 + # shellcheck disable=SC1090,SC1091 . "$playbook_base/playbook.bash" + debug "bashtard/sync" "Checking for \$BASHTARD_PLAYBOOK_VARS" + + # Ensure all required vars are non-empty + for key in "${!BASHTARD_PLAYBOOK_VARS[@]}" + do + # shellcheck disable=SC2086 + in_args "required" ${BASHTARD_PLAYBOOK_VARS[$key]} || continue + + debug "bashtard/sync" "Checking \$BASHTARD_PLAYBOOK_VARS[$key]" + + if [[ "$(config "$key")" == "" ]] + then + missing_vars=$(( missing_vars + 1)) + fi + done + + if (( 0 < missing_vars )) + then + emerg "bashtard/sync" "One or more required variables are unset" + return 3 + fi + + # Run the playbook playbook_sync } diff --git a/lib/subcommands/sysinfo.bash b/lib/subcommands/sysinfo.bash index 8aa237e..4bceb42 100644 --- a/lib/subcommands/sysinfo.bash +++ b/lib/subcommands/sysinfo.bash @@ -5,12 +5,81 @@ # SPDX-License-Identifier: AGPL-3.0-or-later subcommand() { + local load_1 + local load_5 + local load_15 + local memory_free + local memory_total + local memory_used + local storage_free + local storage_total + local storage_used + local uptime + + # Set the variables which are compatible on every POSIX-compliant system + storage_used="$(df | awk '/[0-9]+/ { sum += $3 } END { print sum }')" + storage_free="$(df | awk '/[0-9]+/ { sum += $4 } END { print sum }')" + storage_total=$(( storage_used + storage_free )) + + # And do platform-specific magic + case "${BASHTARD_PLATFORM[os]}" in + freebsd) + load_1="$(uptime | awk -F' *,? *' '{ print $(NF-2) }')" + load_5="$(uptime | awk -F' *,? *' '{ print $(NF-1) }')" + load_15="$(uptime | awk -F' *,? *' '{ print $NF }')" + memory_total=$(( $(sysctl -n hw.physmem) / 1024 )) + memory_used=$(( ( $(sysctl -n vm.stats.vm.v_active_count) * $(sysctl -n hw.pagesize) ) / 1024 )) + memory_free=$(( memory_total - memory_used)) + uptime=$(( "$(date +%s)" - "$(sysctl -a | awk '/^kern.boottime/ { print substr($5, 0, length($5)-1) }')" )) + ;; + *) + load_1="$(awk '{ print $1 }' < /proc/loadavg)" + load_5="$(awk '{ print $2 }' < /proc/loadavg)" + load_15="$(awk '{ print $3 }' < /proc/loadavg)" + memory_total="$(awk '/MemTotal/ { print $2 }' < /proc/meminfo)" + memory_free="$(awk '/MemAvailable/ { print $2 }' < /proc/meminfo)" + memory_used=$(( memory_total - memory_free )) + uptime="$(awk -F. '{ print $1 }' < /proc/uptime)" + ;; + esac + + # For any value that isn't set, just default to 0 to avoid all sorts of errors + [[ -z "$load_1" ]] && load_1=0 + [[ -z "$load_5" ]] && load_5=0 + [[ -z "$load_15" ]] && load_15=0 + [[ -z "$memory_free" ]] && memory_free=0 + [[ -z "$memory_total" ]] && memory_total=0 + [[ -z "$memory_used" ]] && memory_used=0 + [[ -z "$storage_free" ]] && storage_free=0 + [[ -z "$storage_total" ]] && storage_total=0 + [[ -z "$storage_used" ]] && storage_used=0 + [[ -z "$uptime" ]] && uptime=0 + + # Print the values that can be set by package maintainers printf "%-15s %s\n" "etcdir" "$BASHTARD_ETCDIR" printf "%-15s %s\n" "libdir" "$BASHTARD_LIBDIR" printf "%-15s %s\n" "sharedir" "$BASHTARD_SHAREDIR" + # Print all the discovered platform information for key in "${!BASHTARD_PLATFORM[@]}" do printf "%-15s %s\n" "$key" "${BASHTARD_PLATFORM[$key]}" done + + # Print fun little extras + printf "%-15s %0.1fGi / %0.1fGi\n" "memory" \ + "$(awk '{ print($1 / 1024 / 1024) }' <<< "$memory_used")" \ + "$(awk '{ print($1 / 1024 / 1024) }' <<< "$memory_total")" + printf "%-15s %0.1fGb / %0.1fGb\n" "storage" \ + "$(awk '{ print($1 / 1024 / 1024) }' <<< "$storage_used")" \ + "$(awk '{ print($1 / 1024 / 1024) }' <<< "$storage_total")" + printf "%-15s %0.2f %0.2f %0.2f\n" "load" \ + "$load_1" \ + "$load_5" \ + "$load_15" + printf "%-15s %dd %02dh %02dm %02ds\n" "uptime" \ + "$(( uptime / 60 / 60 / 24 ))" \ + "$(( uptime / 60 / 60 % 24 ))" \ + "$(( uptime / 60 % 60 ))" \ + "$(( uptime % 60 ))" } diff --git a/lib/subcommands/top.bash b/lib/subcommands/top.bash new file mode 100644 index 0000000..d4a9b3f --- /dev/null +++ b/lib/subcommands/top.bash @@ -0,0 +1,113 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> +# +# SPDX-License-Identifier: AGPL-3.0-or-later + +subcommand() +{ + local results + local widths + + declare -A results + declare -A widths=( + [load]=0 + [memory]=0 + [node]=0 + [storage]=0 + [uptime]=0 + ) + + if [[ ! -d "$BASHTARD_ETCDIR/hosts.d" ]] + then + crit "$BASHTARD_NAME/ssh" "Could not find hosts file at $BASHTARD_ETCDIR/hosts.d" + return 3 + fi + + chgdir "$BASHTARD_ETCDIR/hosts.d" + + for node in * + do + local user + local host + + user="$(config_for "$node" "bashtard.ssh.user" "$USER")" + host="$(config_for "$node" "bashtard.ssh.host")" + + if [[ -z "$host" ]] + then + crit "$BASHTARD_NAME/ssh" "bashtard.ssh.host is not configured for $node" + continue + fi + + if [[ "$node" == "${BASHTARD_PLATFORM[fqdn]}" ]] + then + results+=(["$node"]="$("$BASHTARD_BIN" sysinfo)") + else + debug "$BASHTARD_NAME/ssh" "$user@$node ($host) > bashtard sysinfo" + # shellcheck disable=SC2029 + results+=(["$node"]="$(ssh "$user@$host" "bashtard sysinfo" 2>/dev/null)") + fi + + unset user + unset host + done + + # Check widths + for node in "${!results[@]}" + do + node_load="$(grep '^load' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" + node_memory="$(grep '^memory' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" + node_storage="$(grep '^storage' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" + node_uptime="$(grep '^uptime' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" + + width_load="$(wc -c <<< "$node_load")" + width_memory="$(wc -c <<< "$node_memory")" + width_node="$(wc -c <<< "$node")" + width_storage="$(wc -c <<< "$node_storage")" + width_uptime="$(wc -c <<< "$node_uptime")" + + if (( widths[load] < width_load )) + then + widths[load]=$width_load + fi + + if (( widths[memory] < width_memory )) + then + widths[memory]=$width_memory + fi + + if (( widths[node] < width_node )) + then + widths[node]=$width_node + fi + + if (( widths[storage] < width_storage )) + then + widths[storage]=$width_storage + fi + + if (( widths[uptime] < width_uptime )) + then + widths[uptime]=$width_uptime + fi + done + + # Print results + printf "$(tput bold)%-${widths[node]}s %-${widths[load]}s %-${widths[memory]}s %-${widths[storage]}s %-${widths[uptime]}s$(tput sgr0)\n" \ + "Node" \ + "Load" \ + "Memory" \ + "Storage" \ + "Uptime" + + for node in "${!results[@]}" + do + printf "%-${widths[node]}s %${widths[load]}s %${widths[memory]}s %${widths[storage]}s %${widths[uptime]}s\n" \ + "$node" \ + "$(grep '^load' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" \ + "$(grep '^memory' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" \ + "$(grep '^storage' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" \ + "$(grep '^uptime' <<< "${results["$node"]}" | sed 's/[^ ]* *//')" + done | sort +} diff --git a/lib/subcommands/zap.bash b/lib/subcommands/zap.bash new file mode 100644 index 0000000..dcefffd --- /dev/null +++ b/lib/subcommands/zap.bash @@ -0,0 +1,37 @@ +#!/usr/bin/env bash + +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> +# +# SPDX-License-Identifier: AGPL-3.0-or-later + +subcommand() +{ + local buffer + local playbook_registry + + export BASHTARD_PLAYBOOK="$1" ; shift + + if [[ -z "$BASHTARD_PLAYBOOK" ]] + then + crit "bashtard/zap" "No playbook name specified" + return + fi + + playbook_registry="$BASHTARD_ETCDIR/registry.d/${BASHTARD_PLATFORM[fqdn]}" + + # Make sure we only run add if the playbook is not in the registry yet + if ! grep -Fqx "$BASHTARD_PLAYBOOK" "$playbook_registry" + then + crit "bashtard/zap" "'$BASHTARD_PLAYBOOK' is not registered for ${BASHTARD_PLATFORM[fqdn]}" + return 3 + fi + + notice "bashtard/zap" "Removing playbook '$BASHTARD_PLAYBOOK' from '${BASHTARD_PLATFORM[fqdn]}'" + + buffer="$(tmpfile)" + + # Remove the playbook from the registry + cp -- "$playbook_registry" "$buffer" + grep -Fvx "$BASHTARD_PLAYBOOK" "$buffer" \ + | sort > "$playbook_registry" +} diff --git a/lib/util.bash b/lib/util.bash index 6ad69e1..ad84247 100644 --- a/lib/util.bash +++ b/lib/util.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -18,6 +18,11 @@ chgdir() { cd -- "$1" || die "Failed to change directory to $1" } +# Removes whitespace surrounding a given text. +chomp() { + awk '{$1=$1};1' <<< "$@" +} + # Create a datetime stamp. This is a wrapper around the date utility, ensuring # that the date being formatted is always in UTC and respect SOURCE_DATE_EPOCH, # if it is set. @@ -56,6 +61,26 @@ die() { exit "${code:-1}" } +# Recursively hash files in a directory, and hashing the output of all those +# hashes again. This results in a single hash representing the state of files +# in a directory. It can be used to check whether contents changed after +# templating files in a given directory. +dir_hash() { + local path + + path="$1" ; shift + + for entry in "$path"/* + do + if [[ -d "$entry" ]] + then + dir_hash "$entry" + fi + + file_hash "$entry" + done | file_hash - +} + # Fetch a file from an URL. Using this function introduces a dependency on curl. fetch_http() { local OPTIND @@ -96,45 +121,35 @@ fetch_http_wget() { wget --quiet --output-document "$2" "$1" } -# Check if the first argument given appears in the list of all following -# arguments. -in_args() { - local needle="$1" - shift - - for arg in "$@" - do - [[ $needle == "$arg" ]] && return 0 - done - - return 1 +# Hash a given file. This is a convenience function to work around different +# systems calling their file hashing programs differently, and generating +# different output. This function only expects 1 file as argument, and only +# outputs the hash of this particular file. +file_hash() { + file_hash_md5 "$@" } -# Join a list of arguments into a single string. By default, this will join -# using a ",", but you can set a different character using -c. Note that this -# only joins with a single character, not a string of characters. -join_args() { - local OPTIND - local IFS="," - - while getopts ":c:" opt - do - case "$opt" in - c) IFS="$OPTARG" ;; - *) warn "bashtard/join_args" "Unused opt specified: $opt" ;; - esac - done +file_hash_md5() { + local file - shift $(( OPTIND - 1)) + file="$1" ; shift - printf "%s" "$*" + case "${BASHTARD_PLATFORM[key]}" in + freebsd) md5 "$file" | awk '{ print $NF }' ;; + linux-*) md5sum "$file" | awk '{ print $1 }' ;; + esac } +# A very simple means of templating a file, using sed and awk. The template +# file is assumed to exist within the share directory of the current playbook. +# Variables are passed as key=value pairs to this function. Inside the +# template, they are expected to be written as ${key}. file_template() { - local file="$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK/share/$1" ; shift + local file local sedfile + file="$(playbook_path "base")/share/$1" ; shift sedfile="$(tmpfile)" if [[ ! -f $file ]] @@ -147,18 +162,18 @@ file_template() do debug "bashtard/template" "Adding $kv to sedfile at $sedfile" - key="$(awk -F= '{ print $1 }' <<< "$kv")" + key="$(cut -d'=' -f -1 <<< "$kv")" if [[ -z "$key" ]] then - crit "bashtard/template" "Empty key in '$kv' while rendering $file?" + crit "bashtard/template" "Empty key in '$kv' while rendering $file" fi - value="$(awk -F= '{ print $NF }' <<< "$kv")" + value="$(cut -d'=' -f 2- <<< "$kv")" if [[ -z "$value" ]] then - crit "bashtard/template" "Empty key in '$kv' while rendering $file?" + crit "bashtard/template" "Empty key in '$kv' while rendering $file" fi # shellcheck disable=SC2016 @@ -168,6 +183,58 @@ file_template() sed -f "$sedfile" "$file" } +# Check if the first argument given appears in the list of all following +# arguments. +in_args() { + local needle="$1" + shift + + for arg in "$@" + do + [[ $needle == "$arg" ]] && return 0 + done + + return 1 +} + +# Join a list of arguments into a single string. By default, this will join +# using a ",", but you can set a different character using -c. Note that this +# only joins with a single character, not a string of characters. +join_args() { + local OPTIND + local IFS="," + + while getopts ":c:" opt + do + case "$opt" in + c) IFS="$OPTARG" ;; + *) warn "bashtard/join_args" "Unused opt specified: $opt" ;; + esac + done + + shift $(( OPTIND - 1)) + + printf "%s" "$*" +} + +# Convenience function to easily get paths used by the playbook, or to use in +# your playbook. +playbook_path() { + if [[ -z "$BASHTARD_PLAYBOOK" ]] + then + crit "bashtard/playbook_path" "Called outside of a playbook" + return 1 + fi + + case "$1" in + base) printf "%s/playbooks.d/%s" "$BASHTARD_ETCDIR" "$BASHTARD_PLAYBOOK" ;; + data) printf "%s/data.d/%s" "$BASHTARD_ETCDIR" "$BASHTARD_PLAYBOOK" ;; + *) + crit "bashtard/playbook_path" "Invalid path '$1'" + return 1 + esac +} + # Create a temporary directory. Similar to tempfile, but you'll get a directory # instead. tmpdir() { diff --git a/lib/util/config.bash b/lib/util/config.bash index 78841e1..ff20bd0 100644 --- a/lib/util/config.bash +++ b/lib/util/config.bash @@ -1,6 +1,6 @@ #!/usr/bin/env bash -# SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +# SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> # # SPDX-License-Identifier: AGPL-3.0-or-later @@ -17,7 +17,11 @@ config_subkeys() { config_for() { local host=$1 ; shift local key=$1 ; shift - local default=$1 ; shift + + # Use a variable definition test to define default, in order to ensure + # it is _not_ defined if no argument for it was passed, but _is_ + # defined even if an empty string was passed. + test -v 1 && { local default=$1 ; shift ; } local default local file @@ -34,8 +38,8 @@ config_for() { then debug "bashtard/config_for" "BASHTARD_PLAYBOOK=$BASHTARD_PLAYBOOK, adding etc entries" files+=( - "$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK/etc/os.d/${BASHTARD_PLATFORM[key]}" - "$BASHTARD_ETCDIR/playbooks.d/$BASHTARD_PLAYBOOK/etc/defaults" + "$(playbook_path "base")/etc/os.d/${BASHTARD_PLATFORM[key]}" + "$(playbook_path "base")/etc/defaults" ) fi @@ -51,7 +55,26 @@ config_for() { [[ ! -f $file ]] && continue - value="$(awk -F= '$1 == "'"$key"'" { print $NF }' "$file")" + # Check if the lookup is a reference variable, defined by using + # &= instead of just a single = as seperator. If this exists, + # do a new config_for lookup, this time using the value as the + # key for the new lookup. + value="$(awk -F= '$1 == "'"$key"'&" { print $0 }' "$file" | cut -d'=' -f 2-)" + + if [[ -n $value ]] + then + debug "bashtard/config_for" "Found reference for $key to $value in $file" + + printf "%s" "$(config_for "$host" "$value")" + return + fi + + # Use awk to find the right line, then use cut to get the + # actual value. Cutting it out with awk _is_ possible, but + # comes with whitespace issues or having to deal with values + # containing the seperator (=), so using cut is much simpler + # and easier to understand. + value="$(awk -F= '$1 == "'"$key"'" { print $0 }' "$file" | cut -d'=' -f 2-)" if [[ -n $value ]] then @@ -62,8 +85,8 @@ config_for() { fi done - # Return default value - if [[ -n $default ]] + # Return default value, if one has been defined + if test -v default then printf "%s" "$default" return @@ -111,7 +134,13 @@ config_subkeys_for() { while read -r result do - local subkey="$(awk -F. '{ print $1 }' <<< "${result#"$key."}")" + local subkey + + subkey="$(awk -F. '{ print $1 }' <<< "${result#"$key."}")" + + # Remove trailing ampersand if it exists + [[ "${subkey: -1}" == "&" ]] && subkey="${subkey:0:-1}" + debug "bashtard/config_subkeys" "Found '$subkey' as subkey of '$key' through '$result'" results+=("$subkey") done < <(grep "^$key\." "$file" | awk -F= '{ print $1 }') diff --git a/lib/util/pkg.bash b/lib/util/pkg.bash index 1538c3e..1433815 100644 --- a/lib/util/pkg.bash +++ b/lib/util/pkg.bash @@ -33,10 +33,11 @@ pkg_install() { local app=$1 ; shift case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- /usr/sbin/pkg install -y "$app" ;; - linux-debian*) set -- apt install -y "$app" ;; - linux-gentoo) set -- emerge --ask=n --update "$app" ;; - linux-ubuntu) set -- apt install -y "$app" ;; + freebsd) set -- /usr/sbin/pkg install -y "$app" ;; + linux-alpine_linux) set -- apk add --no-cache "$app" ;; + linux-debian*) set -- apt install -y "$app" ;; + linux-gentoo) set -- emerge --ask=n --update "$app" ;; + linux-ubuntu) set -- apt install -y "$app" ;; *) crit "$system" "No package manager configured for ${BASHTARD_PLATFORM[key]}" return 1 @@ -54,10 +55,11 @@ pkg_uninstall() { local app=$1 ; shift case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- /usr/sbin/pkg uninstall -y "$app" ;; - linux-debian*) set -- apt remove -y "$app" ;; - linux-gentoo) set -- emerge --ask=n --unmerge "$app" ;; - linux-ubuntu) set -- apt remove -y "$app" ;; + freebsd) set -- /usr/sbin/pkg uninstall -y "$app" ;; + linux-alpine_linux) set -- apk del "$app" ;; + linux-debian*) set -- apt remove -y "$app" ;; + linux-gentoo) set -- emerge --ask=n --unmerge "$app" ;; + linux-ubuntu) set -- apt remove -y "$app" ;; *) crit "$system" "No package manager configured for ${BASHTARD_PLATFORM[key]}" return 1 diff --git a/lib/util/svc.bash b/lib/util/svc.bash index fd65e4c..1b1ecbc 100644 --- a/lib/util/svc.bash +++ b/lib/util/svc.bash @@ -34,11 +34,18 @@ svc_disable() { local service=$1 - case "${BASHTARD_PLATFORM[key]}" in - linux-gentoo) set -- /sbin/rc-update del "$service" ;; - linux-*) set -- systemctl disable "$service" ;; + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-update del "$service" ;; + systemd) set -- systemctl disable "$service" ;; + *) + crit "$system" "No service manager support to disable with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to enable for ${BASHTARD_PLATFORM[key]}" return 1 esac @@ -53,7 +60,16 @@ svc_enable() { local service=$1 local rc - case "${BASHTARD_PLATFORM[key]}" in + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-update add "$service" ;; + systemd) set -- systemctl enable "$service" ;; + *) + crit "$system" "No service manager support to enable with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; freebsd) rc="$(printf '%s_enable="YES"' "$service")" @@ -63,12 +79,9 @@ svc_enable() { fi return 0 - ;; - linux-gentoo) set -- /sbin/rc-update add "$service" ;; - linux-*) set -- systemctl enable "$service" ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to enable for ${BASHTARD_PLATFORM[key]}" return 1 esac @@ -82,12 +95,21 @@ svc_reload() { local service=$1 - case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- service "$service" reload ;; - linux-gentoo) set -- /sbin/rc-service "$service" reload ;; - linux-*) set -- systemctl reload "$service" ;; + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-service "$service" reload ;; + systemd) set -- systemctl reload "$service" ;; + *) + crit "$system" "No service manager support to reload with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; + freebsd) + set -- service "$service" reload + ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to reload for ${BASHTARD_PLATFORM[key]}" return 1 esac @@ -101,12 +123,21 @@ svc_restart() { local service=$1 - case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- service "$service" restart ;; - linux-gentoo) set -- /sbin/rc-service "$service" restart ;; - linux-*) set -- systemctl restart "$service" ;; + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-service "$service" restart ;; + systemd) set -- systemctl restart "$service" ;; + *) + crit "$system" "No service manager support to restart with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; + freebsd) + set -- service "$service" restart + ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to restart for ${BASHTARD_PLATFORM[key]}" return 1 esac @@ -120,12 +151,21 @@ svc_start() { local service=$1 - case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- service "$service" start ;; - linux-gentoo) set -- /sbin/rc-service "$service" start ;; - linux-*) set -- systemctl start "$service" ;; + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-service "$service" start ;; + systemd) set -- systemctl start "$service" ;; + *) + crit "$system" "No service manager support to start with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; + freebsd) + set -- service "$service" start + ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to start for ${BASHTARD_PLATFORM[key]}" return 1 esac @@ -139,12 +179,21 @@ svc_stop() { local service=$1 - case "${BASHTARD_PLATFORM[key]}" in - freebsd) set -- service "$service" stop ;; - linux-gentoo) set -- /sbin/rc-service "$service" stop ;; - linux-*) set -- systemctl stop "$service" ;; + case "${BASHTARD_PLATFORM[os]}" in + linux) + case "${BASHTARD_PLATFORM[init]}" in + openrc) set -- /sbin/rc-service "$service" stop ;; + systemd) set -- systemctl stop "$service" ;; + *) + crit "$system" "No service manager support to stop with ${BASHTARD_PLATFORM[init]}" + return 1 + esac + ;; + freebsd) + set -- service "$service" stop + ;; *) - crit "$system" "No service manager configured for ${BASHTARD_PLATFORM[key]}" + crit "$system" "No service manager support to stop for ${BASHTARD_PLATFORM[key]}" return 1 esac diff --git a/share/doc/bashtard.1.scd b/share/doc/bashtard.1.scd index aabc2d7..1a681e0 100644 --- a/share/doc/bashtard.1.scd +++ b/share/doc/bashtard.1.scd @@ -1,6 +1,6 @@ bashtard(1) -; SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +; SPDX-FileCopyrightText: 2024 Patrick Spek <p.spek@tyil.nl> ; SPDX-License-Identifier: AGPL-3.0-or-later # NAME @@ -12,13 +12,16 @@ bashtard - A Bash-based configuration management utility *bashtard* -h++ *bashtard* add <_playbook_>++ *bashtard* del <_playbook_>++ +*bashtard* diff++ *bashtard* init [_repository_]++ *bashtard* pkg <_install_|_uninstall_> <_name_>++ +*bashtard* pull++ *bashtard* ssh <_command_>++ *bashtard* sync [_playbook_]++ *bashtard* var [-p _playbook_] <_key_>++ *bashtard* var [-s] <_key_> <_value_>++ *bashtard* sysinfo +*bashtard* zap <playbook> # DESCRIPTION @@ -40,6 +43,10 @@ function of the *playbook.bash* file in the root of the playbook directory. It also removes the playbook name from the host's registry, no longer including it during a *sync*. +## diff + +Show current uncommitted changes present in the etcdir. + ## init Initializes *bashtard*, creating the required directory structure in @@ -56,6 +63,11 @@ Allows direct interaction with the *pkg_\*()* functions provided by the *bashtard* utility library. This in turn makes allows a user to use a single package management interface accross all their machines. +## pull + +Perform a git pull on the etcdir, but without syncing all the playbooks +afterwards. + ## ssh Run a given command over SSH to all configured nodes. Every node needs to have @@ -80,6 +92,13 @@ configuration file. This variant accepts the *-s* option, to instead save it to the *secrets* file, which should only be readable by the user maintaining the system through *bashtard*. +## zap + +Removes a playbook from the registry. Unlike *del*, this will not attempt to run +the *del* function defined within the playbook. This functionality can be used +to remove the entry of a misbehaving playbook, or to be able to re-run the *add* +function on it. + # SEE ALSO - _bashtard(3)_ -- Functions and variables exposed by *bashtard* for use in diff --git a/share/doc/bashtard.3.scd b/share/doc/bashtard.3.scd index 341d8fc..0f582fa 100644 --- a/share/doc/bashtard.3.scd +++ b/share/doc/bashtard.3.scd @@ -1,6 +1,6 @@ bashtard(3) -; SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +; SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> ; SPDX-License-Identifier: AGPL-3.0-or-later # NAME @@ -46,6 +46,13 @@ more information on this. The name of the currently running playbook. +## BASHTARD_PLAYBOOK_VARS + +An associative array, in which the keys are the config keys. The value of each +element is a space-seperated list of attributes to which the config value must +adhere. Currently, only _required_ is available, forcing a variable's value to +be non-empty when the playbook is ran. + # FUNCTIONS ## config $key @@ -75,6 +82,15 @@ resolve the *pkg.nginx*, and install whichever value it came up with. These are generally defined in the *os.d* of a *playbook*, to handle situations where certain systems name their packages differently. +## playbook_path $name + +The *playbook_path()* function is a convenience function to allow easy access to +various (absolute) paths that are useful during playbook exection. Currently, +the following _name_ arguments are accepted: + +- *base* +- *data* + ## svc $action $svc The *svc()* function abstracts away the running system's service manager, diff --git a/share/doc/bashtard.7.scd b/share/doc/bashtard.7.scd index fc9333e..6413f52 100644 --- a/share/doc/bashtard.7.scd +++ b/share/doc/bashtard.7.scd @@ -1,6 +1,6 @@ bashtard(7) "/etc/bashtard" -; SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +; SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> ; SPDX-License-Identifier: AGPL-3.0-or-later # NAME @@ -14,6 +14,13 @@ bashtard - A Bash-based configuration management utility The defaults file is a _bashtard(5)_ key-value file. It specifies the values of variables in case there are no host or os specific values. +# data.d + +The *data.d* directory contains a directory for each playbook that actively uses +it. It stores data generated or otherwise used by the playbook between +consecutive runs. This allows a playbook writer to keep a clear seperation +between a user's data and the playbook. + # hosts.d The *hosts.d* directory contains one file for each specific host you're managing @@ -29,18 +36,8 @@ certain GNU+Linux distributions. # playbooks.d -The *playbooks.d* directory contains a single plain-text file, *remotes*, -describing all the remote playbooks tracked by *bashtard*. The layout of -playbooks themselves can be found in _playbooks.d(7)_. - -## remotes - -The *remotes* file is a space-seperated file, specifying a _name_, _url_, and -_branch_ per line, to use for keeping remote playbooks synchronized. - -``` -webserver https://git.tyil.nl/bashtard-webserver.git master -``` +The *playbooks.d* directory contains all the playbook directories. For more +information, check _playbooks.d(7)_. # registry.d diff --git a/share/doc/playbooks.d.7.scd b/share/doc/playbooks.d.7.scd index b4624a2..20fa094 100644 --- a/share/doc/playbooks.d.7.scd +++ b/share/doc/playbooks.d.7.scd @@ -1,6 +1,6 @@ playbooks.d(7) "/etc/bashtard/playbooks.d" -; SPDX-FileCopyrightText: 2022 Patrick Spek <p.spek@tyil.nl> +; SPDX-FileCopyrightText: 2023 Patrick Spek <p.spek@tyil.nl> ; SPDX-License-Identifier: AGPL-3.0-or-later # NAME @@ -10,9 +10,7 @@ bashtard - A Bash-based configuration management utility # INTRODUCTION *bashtard* playbooks are, in essence, no more than directories with a small -number of required files. Even remote playbooks just get cloned into regular -directories inside _playbooks.d_. The files you must include in any playbook -are: +number of required files. The files you must include in any playbook are: - description.txt - playbook.bash |