In case anyone was wondering, yes we were down for ~2 hours or so. I apologize for the inconvenience.
We had a botched upgrade path from 0.17.4
-> 0.18.0
. I spent some time debugging but eventually gave up and restored a snapshot (taken on Saturday Jun 24, 2023 @ 11:00 UTC).
We’ll likely stick to 0.17.4
till I can figure out a safe path to upgrade to a bigger (and up-to-date) instance and carry over all the user data. Any help/advice welcome. Hopefully this doesn’t occur again!
It sounds like lemmy.world also had issues upgrading to 0.18.1 rc1. I believe they skipped 0.18.0 because the captcha feature was removed in that version, but readded for 0.18.1. They’re solutions may be helpful, assuming it was similar issues.
thank you for maintaining this server, and being very transparent about it ❤️
I suspect almost everyone here understands how this happens and feels more sympathy than anything. We’ve been there, and with higher stakes. Thanks for taking the time to administer this instance.
Sigh, I tried it one more time. Same problem. Downtime this time was probably < 5 mins though so hopefully not noticeable. I am wondering whether the database is in some bad shape (no idea why/how). I guess we wait and retry with
0.18.1
.