From 68b323fe53da1d9884e14a8ca5d5119d5631b4f2 Mon Sep 17 00:00:00 2001 From: Patrick Spek Date: Mon, 24 Jul 2023 11:02:28 +0200 Subject: Add blog post introducing Mieshu --- .../2023/2023-07-24-new-server-rack-mieshu.md | 89 ++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 content/posts/2023/2023-07-24-new-server-rack-mieshu.md diff --git a/content/posts/2023/2023-07-24-new-server-rack-mieshu.md b/content/posts/2023/2023-07-24-new-server-rack-mieshu.md new file mode 100644 index 0000000..f024784 --- /dev/null +++ b/content/posts/2023/2023-07-24-new-server-rack-mieshu.md @@ -0,0 +1,89 @@ +--- +date: 2023-07-23 +title: "My New Server Rack: Mieshu" +tags: +- GNU+Linux +- Gentoo +- Systemd +- Garage +--- + +After saving up for a long while and thinking about what I want in my new home, +I have finally taken the leap and gotten myself a server rack for home use. Its +has a 15U capacity, which should be plenty to get started, but this same brand +has larger racks too, in case I do want to upgrade it and keep the same style. + +That said, for now there's only two 4U units in them, one for (file) storage, +and one for database purposes. I sadly don't have anything dedicated for +workloads yet, so for now, both of these servers are intended to also run some +light workloads. I haven't made my mind up yet on how to solve the workload +issues. Now that I have a rack, I obviously want something rack-mountable, and +I probably want it to run a Kubernetes cluster too. + +In this regard, I _could_ go for a set of [Raspberry Pi](https://www.raspberrypi.com/) +units, there's [3U mounts that can hold up to 12 Raspberry Pi machines](https://www.uctronics.com/uctronics-19-inch-3u-rack-mount-for-raspberry-pi-4-with-8-mounting-plates.html), +which would be a nice amount. However, I am not yet completely sold on full-ARM +workloads, and I'm not entirely convinced of the power of Raspberry Pi units in +general. I'd much rather standardize on another brand, [Odroid](https://www.hardkernel.com/), +as they have more types of units available, and are not limited to just ARM. But +since they're not the popular kid in class, there's very few off-the-shelf +rack mounting equipment for it. I'll be thinking about this for just a bit more +before making a decision. + +For now, though, I wanted to talk about the setup of the first server, Mieshu, +who will be used as a storage server. Mieshu currently runs 8 HDDs, and 2 NVMe +drives. One of the NVMe drives is used for the rootfs, and the other is used for +caching in certain applications. The HDDs themselves offer the data storage +capacity. + +The HDDs are currently comprised of four 16TB drives, and four 8TB drives. The +smaller disks come from my desktop, Edephas, which used to serve as data storage +until Mieshu took over. All disks are configured into pairs, which themselves +make mirrors. This means I have four sets of mirror pools, two times 16TB, and +two times 8TB, for a total of 48TB of storage. I'm currently using about half of +this, and it should give me plenty of time before needing to increase the size +again. + +I chose to use mirrors since it has a good chance of your data being recoverable +on disk failure, and it allows me to buy disks per two, rather than in larger +numbers. This hopefully keeps the cost of expansion within reasonable limits. +The mirrors themselves are currently [ZFS](https://openzfs.org/wiki/Main_Page) +pools, but I hope to be able to use [bcachefs](https://bcachefs.org/) very soon +as well. + +Just a bunch of mirrors is rather inconvenient, however, so I'm also leveraging +[MergerFS](https://github.com/trapexit/mergerfs) to combine all the mirrors into +a single usable pool. This slightly odd setup was chosen over RAID-0 or RAID-Z* +to lower the impact of disk failure. Even if two disks in the same mirror were +the die at the same time, I wouldn't lose _all_ data, just the bits on that +particular mirror. It would be very annoying, but it wouldn't be disastrous. + +Apart from generic mass storage, I also host S3 buckets for personal use. This +is where I upload CI artifacts to, and [my MissKey instance](https://fedi.tyil.nl/@tyil) +uses it for storing objects as well. Future services such as [Mimir](https://grafana.com/oss/mimir/) +will probably leverage S3 for storage as well. This is achieved through +[Garage](https://garagehq.deuxfleurs.fr/). I've also tried [SeaweedFS](https://seaweedfs.github.io/), +which is a very neat project on its own, but Garage is just simpler to +configure, and allows a replicated setup with only two servers, whereas SeaweedFS +demands an odd number of master servers. + +And lastly, Mieshu runs [K3s](https://k3s.io/) for its Kubernetes component. It +is currently not serving anything yet, as the other server is supposed to become +the database server, which is needed for most workloads. Once that is up and +running, Mieshu will start hosting things such as +[Grafana](https://grafana.com/oss/grafana) and [Loki](https://grafana.com/oss/loki/), +monitoring stuff basically. Perhaps I'll move [Laminar](https://laminar.ohwg.net/) +to this server as well, but I'm unsure if I will run that as a Kubernetes service. + +The server itself runs on Gentoo, as it still is the most stable experience I +can get out of any GNU+Linux distribution. I am, however, not using the default +of OpenRC as the init system and service manager. For the first time, I'm +running Gentoo with systemd. After several years, it appears to have become +stable enough to trust with serious workloads. With its increased use, however, +some things have become simpler by just using systemd. I hope to get a better +understanding of it, and learn to bend it to my will as needed, by simply using +it on my own systems. + +I hope to have time to work on the other server sooner rather than later, so +I can finish up the base of my new setup. Be on the lookout for the next post, +where I'll go into detail on Nouki, the database server. -- cgit v1.1