503 errors earlier today

[classicdrogn]

Any idea what was causing them? I don't remember the times, but it was about three hours I think.

[Bob Schroeck]

Have no idea, this is the first I'm aware of them.

[Star Ranger4]

The 503 Service Unavailable error is an HTTP status code that means the website's server is simply not available right now. Most of the time, it occurs because the server is too busy or because there's maintenance being performed on it.

(looked it up)

 Therefore, what had to happen was on the ISP side, either a router burped or something similar.  Bob probably would never have known if you hadn't hit the berf and mentioned it.

[classicdrogn]

Well, I'd have expected at least a couple other users to have tried to check the site during that time, since it was early evening Prime Time, but if no one else noticed I suppose it must have been something betwixt hither and yon. I didn't have any other connection problems, but I pretty much only look at SB/SV/QQ, FFn, and here most days, with Calibre poking AO3 and Wuxiaworld/Fastnovel/Scribblehub/RR for a few things. Not exactly a wide net to cast.

[robkelk]

Oh, I noticed - it interfered with my ability to post actual content to the "There's Nothing Better" forum. I alerted Bob by email... and, knowing Bob's usual schedule for being online, included start and stop times.

[classicdrogn]

... and now both SB and SV are just showing Cloiudflare error 523 pages. Not a good weekend for fora, it seems.

[Norgarth]

Yeah, I couldn't access the board for a few hours as well.

[Bob Schroeck]

Whatever it was, it didn't register big enough to get a post about it on my ISP's Twitter feed. If my workload tomorrow permits, I'll give their support line a call and query about it.

[Aleh]

... and now both SB and SV are just showing Cloiudflare error 523 pages. Not a good weekend for fora, it seems.

In their cases, it was a literal fire at the datacenter. Some of the servers even suffered water damage.

Given that this site suffered a lot less, I rather doubt it was impacted by that issue.

[Bob Schroeck]

Nah, not even close. I believe they're hosted somewhere in the Midwest? Netcarrier, my host, is in Pennsylvania.

And besides, we already saw what happens when Netcarrier's data center is sunk, back in October.

[classicdrogn]

The fact that the incidents are not related just makes it cumulatively a worse weekend for fora. I mean, what are the odds?

[Black Aeronaut]

SB and SV are hosted at the same datacenter in Ogden, Utah.

Apparently what had happened was that they were load-testing a backup generator that powers some of the core routers and the fire suppression system.

And then the generator caught on fire.

For whatever reason, the fire department would not move in until they shut down power to the ENTIRE data center, even though the circuits in question were completely isolated from the server farms.

I have no idea how the hell water got into the server farms, but it didn't hurt the server rack that SV is in.  SB, on the other hand, seems like it got wet and will be undergoing recovery to get what little was lost.  (SV gets backed-up every 15 minutes, so I assume SB does the same.)

[nocarename]

I saw in passing on the SV Discord that the 15 minute backups had been revised to 20 minutes due to load issues at some point, but yeah. We'll see what happens I suppose.

[robkelk]

The Utah outage has made the international news. Well, it made The Register.

Imagine your data center backup generator kicks in during power outage ... and catches fire. Well, it happened