So, a new year, and a lot of time to continue some adventures over the Xmas holiday break. After the usual visit to my family's place, I came home and was surprised with a broken k8s cluster. So I thought. But after some investigation, I found out, that both of my Raspberry Zeros had problems. So the k8s cluster was not broken, only the supporting Zeros.
My Raspi Zero WH, which runs as my primary Pi Hole/DNS and collects data from my weather sensors, had Storage card problems. Somehow during the last few weeks, the SD card got a filesystem error which resulted in memory errors and system crashes. So first system reconstruction.
Then my second Raspi Zero 2W, which runs a desktop raspberry os, ran out of memory, cause the Chromium browser used up all the memory and since this Raspi was also configured as my secondary Pi Hole/DNS, via keepalived and gravity sync, it had to jump in since the primary Pi Hole service was no longer stable, it started crashing, too. So the second system needs a new setup.
My brother got me a used Raspi Pi 4B 8GB from a friend of his, as Xmas present, but I don't care these days where a Raspi Pi falls in my hands, so I wanted to set up my k8s cluster Raspi's, too. Another 5 systems waiting for a setup, configuration and installation.
Repair of Weather Pi
Originally I wanted to use DietPi OS whenever I had time to install this Raspi. But unfortunately, I couldn't get it to work. I tried DietPi OS for armv6, armv7 and armv8, but I always got a boot error, with a kernel panic. Don't know why and how to fix it, but after 5 or 6 tries I gave up and went back to the 32-bit Raspberry OS lite.
This installation was painless. I installed only nginx as only additional software. With ansible, I installed my python program for serving a simple web page with the current weather data via uwsgi (as a service) and nginx.
The second program, I installed, also via ansible, was the service, also a python program, that pushes the sensor data every ten minutes to a CouchDB. I described this software in a blog last year.
Since I'm not sure how stable the SD card is, I decided not to install Pi-Hole/DNS anymore. I will move my primary Pi-Hole/DNS to one of the Raspi Pi 4B's.
The Monitor Raspi
So one down, 6 to go. The Monitor Raspi Pi was easy to fix. I stopped the Chromium browser for now. So now Monitor at the moment. And it currently is not needed, since it shows Uptime-Kuma and the Weather Dashboard (via Node-Red) and those services only run, when the k8s run. And since I'm going to set up this cluster, no need for a Monitor. For the moment this Raspi will be the primary Pi-Hole/DNS service. "keepalived" is disabled for now, too.
As soon as my new k8s cluster is running again, I configure this Raspi as a secondary (fallback) Pi-Hole/DNS, again. I use "keepalived" and "Gravity Sync". Have a lock at this video and the documentation from Techno Tim.
And of course, I enable the Chromium browser for displaying the Dashboards again, too. But this will take some time.
Plans for the new k8s cluster
Ok, after collecting Raspberry Pi's for 1,5 years, now, I have 5 of them. 2 x 4B's with 8 GB and 3 x 4B with 4 GB. Have a look at the Infrastructure Architecture.
Hardware I want to use:
5 Raspi Pi's 4B (2x8GB, 3x4GB)
1 used 250 GB Crucial SSD
4 Inteso 32 GB USB Sticks
2 new 1TB M2 SSDs for storage as a replacement for my disk share on my old Mac Mini from 2007, which soon will go into retirement
1 Raspi Pi Power Adapters
1 USB 60W 4-Port (2 USB-C, 2 USB-A) UGREEN Power Brick
1 Noctua 120mm Premium Fans, 5V versions, with USB Adapter
1 6-time stackable Pi Mounting rack
1 Netgear 8 Port 1GB Switch
5 0,25m Cat 6 Network cables
1 USB Fan Speed Regulator
1 Ikea Alu Box
Software I want to use:
k3sup for k8s setup
Docker for Load Balancing (nginx), Prometheus and Portainer
NFS (DietPi) as NFS Server and Backup Server
Keepalived for Pi-Hole/DNS
nginx as Load Balancer for the k8s cluster
Prometheus
Grafana
Uptime-Kuma
So I built the cluster over the last few weeks and now it is up and running.
Next steps are to fill the cluster with life... Grafana... Uptime... Heimdall... Databases... Node-Red... Portainer... Gitea... Drone...
And of course when all the services are moved from my old 2007 Mac Mini (Ha... 12 Years in Service, since 2011, when I bought it used, beat that M2 Mac Mini ;-) ), I can eventually retire him. I pack him in his original package, yes still have it, and put the box in a safe place. Who knows, if I don't need this old fellow in the future, again.
So my k8s cluster adventures continue...
As always apply this rule: "Questions, feel free to ask. If you have ideas or find errors, mistakes, problems or other things which bother or enjoy you, use your common sense and be a self-reliant human being."
Have a good one. Alex