If you are #selfhosting / #homelabbing: you have my utmost respect.
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d I have suffered from the consequences of minor parts of my setup failing and it was already pretty bad.
And even now I am still in dire need to improve things, but I'm building the foundation of something which should allow me to more rapidly fallback to some other system, if stuff breaks.
Excited to finally set those plans into motion. -
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d All in the practice of independence in your technology.
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d Yes, learned this lesson the hard way as well. my DNS stays on my Unifi controller and HomeAssistant stays on it's own hardware. Then, downtime on the proxmox host is less impactful for me.
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
The number that makes this concrete: 99.9% uptime is 8.7 hours of outage a year and 99.99% is 52 minutes. A single host setup sits somewhere well below both, and you find out which during the outage.
Cheapest useful fallback is usually DNS and lighting on separate hardware, so you can still see and reach the rack while you fix the rest.
-
@h3artbl33d Yes, learned this lesson the hard way as well. my DNS stays on my Unifi controller and HomeAssistant stays on it's own hardware. Then, downtime on the proxmox host is less impactful for me.
Totally! Main DNS server in the homelab. Secondary on a Raspberry Pi elsewhere in the home.
For Home Assistant I do not have HA. I do however have one spare Zigbee2MQTT box with another Zigbee stick (coordinator) just in case. If everything goes down, I do not need Home Assistant per se, as long as I can make sure that there is light in the darkness haha.
-
@h3artbl33d I have suffered from the consequences of minor parts of my setup failing and it was already pretty bad.
And even now I am still in dire need to improve things, but I'm building the foundation of something which should allow me to more rapidly fallback to some other system, if stuff breaks.
Excited to finally set those plans into motion.Totally. Makes much sense. I do have the core documented nowadays, including a hardcopy of the relevant steps (without getting too technical), should anything happen and I am not around.
If you are posting your progress: looking forward to it!
-
The number that makes this concrete: 99.9% uptime is 8.7 hours of outage a year and 99.99% is 52 minutes. A single host setup sits somewhere well below both, and you find out which during the outage.
Cheapest useful fallback is usually DNS and lighting on separate hardware, so you can still see and reach the rack while you fix the rest.
Totally. Main DNS in the homelab, secondary DNS in a SBC in the complete opposite of the homelab.
And for the lighting I do have a fallback plus documentation for any scenario where I am not there.
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d
But so far, my 6 year old desktop I turned into a server has 100% uptime!/s
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d I have, more than once, had to turn the hotspot on my phone on so I could download an ISO of #pfSense to reflash my #router.
-
If you are #selfhosting / #homelabbing: you have my utmost respect. Do not trust other parties to have your back - because they don't.
However, few words to the wise: if you operate a minimalist setup - eg everything running in VMs on a single host: please do consider the consequences of downtime.
You probably don't want to run your critical services on one machine without a fallback. If those services include your smarthome foundation, DNS and firewalling: you don't want to troubleshoot in the dark because everything is down and you can't even turn on the light anymore.
You aren't the first nor won't be the last to learn this lesson when disaster strikes though
@h3artbl33d That’s why I’m still hesitating to go all in.
-
Totally. Makes much sense. I do have the core documented nowadays, including a hardcopy of the relevant steps (without getting too technical), should anything happen and I am not around.
If you are posting your progress: looking forward to it!
@h3artbl33d I do have enough documentation that someone could roughly trace what I am doing, but not easy so far.
I've been thinking about doing something physical, but I don't see who I could help with that.Currently I'm thinking of making it possible and easy to restore a service on a different machine, if something breaks on the primary such that I could initiate a move in little time and get things back up easily.
Standardizing the majority of services on FreeBSD jails should make that easy, though I'll definitely at least have to overhaul networking for migrations to be easily possible.
Plus I'm thinking about giving someone else backup access such that if I were to be unavailable for an extended period of time, they could fix minor things or run important security updates etc.And yeah, ofc I'm going to be posting about it, as it happens. Primary limitation currently is time, as is so often
-
S strit@mastodon.social shared this topic
