About db141

 We have very important news regarding db141.

Yesterday, we were able to access and recover the data on the corrupted drives, including the drives which contained the original db141. We intend to begin restoring affected wikis as soon as possible and we will be releasing more information about it once we get details finalised.

Now, during our scheduled maintenance yesterday, we encountered an issue attempting to get new storage drives detected by our cloud13server. We asked our hosting provider to reseat the disks but we then deemed it unnecessary and cancelled the request. Unfortunately, the request was still being processed and our hosting provider mistakenly reseated cloud14's drives while the server was on. Due to this, the server locked up and we had to run some file system repair tools (fsck specifcially) to get it back online.

Once cloud14 and all its virtual servers came back up, we discovered that because the new db141 was running and writing data when that happened, the database had become corrupted. Thankfully, we ran backups for most wikis yesterday so they should be safe. This has made the task of restoring wikis affected by the original db141 outage a bit easier. What we plan to do is restore all original db141 wikis from the recovered disks and then, using our backups, merge new edits made on the recreated wikis back into these original wikis. We do not have an ETA for when we will do this but we are thrilled to have recovered the data.

We apologise for yet another downtime on these wikis but this incident has helped foment stronger backup procedures to prevent catastrophic disasters from occurring. We are now working to restore these wikis and we will provide more information once we have it. Thank you for your understanding.

TL;DR: We recovered the data from the broken, original db141 disks but the new db141 was corrupted due to a hosting provider error. We have backups from yesterday so we will now restore the original db141 and merge the edits from recreated wikis back into the old wikis. Miraheze Site Reliability Engineering 00:00, 19 December 2022 (UTC)

Comments

Post a Comment

Popular posts from this blog

The Bad User’s rant on FoHSA

 Fandom of Halloween Specials Archive is a toxic site that I no longer want to take a part of. If I were you, I would get out while I still can too. It's just a place where toxicity spreads because of its conditions. Everyone is an admin for no good reason and have complete control over their pages, and they like to everyone a ripoff just because they have "Hannah" in their name, and that's their excuse to try to move and edit every page on the Miraculous Wiki. (also once Mimi changed my Cure name from Cure Spicy to Cure Chilly because she didn't like it so they vandalize for other reasons than just ripoffs) Fandom of Halloween Specials is an official affiliate for a weird group called "Anti-BanG Dream Against Vandals", a group made specifically just for editing supposed "ripoffs" (hypocritically, this is the same wiki that copies everything, why do these so-called "bootlegs" not get the boot?), just to make them even worse (ex. chang...
Read more

ERROR

  Sorry! This site is experiencing technical difficulties. Please try again in a few minutes, these errors usually resolve by then. If you are receiving this error persistently, please check our Twitter (https://twitter.com/MirahezeStatus) for more information or contact Miraheze System administrators by joining #miraheze-sre on irc.libera.chat or by joining our Discord server (https://discord.gg/TVAJTE4CUn). (Cannot access the database:  Cannot access the database: Connection timed out (db141) (db141) )
Read more

NEW LEAK + UPDATE!

Image
  20:00 (UTC), Monday, 26 December 2022 - We have uploaded the databases to our new database server. We are now working to get the wiki back online. Due to the holidays, this may take a bit longer than usual. Thank you for your understanding.
Read more