My homelab died today

My homelab died today. It was just a small but mightly N150 mini pc from Beelink. It wouldn’t boot at all. I looked at the DHCP reservations and see it was last active at 8:26 AM. So I know the time of death. RIP my friend.

I needed to understand why it failed. So, I hooked it up to a monitor and started observing. The power LED comes on, the fan spins, and it reaches BIOS setup every time. Video output is fine. However, Proxmox decides to hang while trying to boot and after a while emits this message:

  ata2: link is slow to respond, please be patient (ready=0)
  ata2: limiting SATA link speed to 3.0 Gbps
  ata2: hardreset failed
  ata2: reset failed, giving up

Something’s up with SSD and it probably died. Cause of death = Brain Damage.

Even though life continued, not having my homelab did cause some inconveniences. Notably, I couldn’t control my bedroom lights anymore which I could do with voice commands over a Google Home Hub. I had achieved this quite nicely with a Home Assistant instance + Zigbee + Matter Bridge that would let any smart home device control my Home Assistant entities.

I also was living YOLO. No backups whatsoever. Apart from Home Assistant, I was running a few VMS and LXC containers. Some Ghost CMS instances which I can hopefully revive from internet archives. You know they say, “the best backup is the one you put publicly on the internet”.

But what’s a story without a comeback. We’re going to build back stronger. I have reached out to Beelink support and sent them detailed debugging steps I performed with some pictures they requested. I just hope they send me back a new SSD and I can revive this server.

In addition, I have own a cheap Optiplex through Goodwill which will probably act as my primary server. And this time, we will have offsite backups like a sane and respectable homelab.

Stay tuned!