# Arc Flash Incident in Iowa



## derekleffew (Aug 9, 2022)

3 critically injured after Google facility 'electrical incident'

Three electricians sustained serious burns at a Google data center in Council Bluffs,...



www.sfgate.com





> Three electricians were working on a substation close to the data center buildings when an arc flash (an electric explosion) occurred, causing significant burns to all three electricians.
> 
> One person was taken by helicopter to Nebraska Medical Center, while two more were transported by ambulance. (Nebraska Medical Center is a short distance from Council Bluffs, which is right on the Iowa-Nebraska border.) All three people were conscious and breathing when they were transported for medical care, according to the Council Bluffs Police Department.


----------



## Van (Aug 9, 2022)

A guy I was working with a couple few years ago had an arc-flash incident. Horrible.


----------



## Jay Ashworth (Aug 9, 2022)

What isn't yet clear is why a DC failure in Iowa percolated up into visibility across Google's entire footprint.


----------



## MRW Lights (Aug 9, 2022)

I trained with a journeyman who made it a point to make you watch arcflash videos as an example of what not to do. I remember a particular "training" video where you saw the arc flash and then like a very bad magic trick the guy was gone.... You can't unlearn that.


----------



## TimMc (Aug 9, 2022)

Jay Ashworth said:


> What isn't yet clear is why a DC failure in Iowa percolated up into visibility across Google's entire footprint.


Unrelated, according to Google.








Google: Monday outage unrelated to arc flash at Iowa data center

Three people are now in stable condition after sustaining burn injuries at a Google data...



www.sfgate.com


----------



## Van (Aug 9, 2022)

MRW Lights said:


> I trained with a journeyman who made it a point to make you watch arcflash videos as an example of what not to do. I remember a particular "training" video where you saw the arc flash and then like a very bad magic trick the guy was gone.... You can't unlearn that.


I received an Arc flash video in a CD that I received when I purchased a Fluke Meter several years ago. it was to teach you about transients. Scared the hell out of me.


----------



## soundman (Aug 9, 2022)

During Covid I got a job doing controls work for Amazon. My site was a new build out and we had to wait for things to be signed off on before we could get to work so I had a lot of free time. Amazon has some pretty neat software for seeing what issue other sites are having with the goal it would be possible to learn from each other and have better trained maintenance and controls techs. The amount of sites that had major electrical issues was mind blowing. It seemed like a site a week was going down for power related issues. There was a site on the East Coast that had solar panels on the roof and something between the cells and the inverter caused a major electrical fire. The photos were insane. To go along with the monitoring software there is a conference call for SEV 1 and SEV 2 events. I really believe the site rep for that site has a live event background, as the longer the call goes more and higher managers join the call. Every time a new manager would joining the call they would ask about if the next shift would be effected and the site rep calmly would respond, the fire department is working hard to extinguish the flames but they are currently 15' tall across most of the roof. I don't think we will be able to start the night shift as scheduled..." The site we did our training at was shut for three days because of a massive fault in one of the electrical rooms. Eaton had to come out and replace several damaged panels. I can't find my notes but like 1200 amps 480v or higher distribution. 

All this to say, massive buildings that are built quickly seem to have electrical issues. For their part Amazon was strict on the size and capacity of breakers the in house techs were allowed to activate. Anything over a certain cal rating and an outsider contractor was brought in.


----------



## Jay Ashworth (Aug 9, 2022)

TimMc said:


> Unrelated, according to Google.
> 
> 
> 
> ...


The writethru postdates my reading the piece; thanks.


----------



## jonliles (Aug 10, 2022)

Jay Ashworth said:


> What isn't yet clear is why a DC failure in Iowa percolated up into visibility across Google's entire footprint.


Depends on what the DC system does. It could cause cascading failures in backup services. DC systems at data centers do everything from back up power supplies to the servers, switchgear operation, and emer diesel starting motors. If that DC is going through a converter, potential big amperages and voltages.


----------



## Jay Ashworth (Aug 10, 2022)

jonliles said:


> Depends on what the DC system does. It could cause cascading failures in backup services. DC systems at data centers do everything from back up power supplies to the servers, switchgear operation, and emer diesel starting motors. If that DC is going through a converter, potential big amperages and voltages.


Sorry, acronym collision. I was talking about data center crashes, not DC power crashes. My point was that if the failure in Iowa was the root cause of the user visible errors that Google was experiencing right around that same time, that would not speak well of their isolation practices.


----------



## jonliles (Aug 10, 2022)

Jay Ashworth said:


> Sorry, acronym collision. I was talking about data center crashes, not DC power crashes. My point was that if the failure in Iowa was the root cause of the user visible errors that Google was experiencing right around that same time, that would not speak well of their isolation practices.


It happens!


----------



## Jay Ashworth (Aug 10, 2022)

jonliles said:


> It happens!


Indeed.

But I'm a system administrator, and like detectives, I'm not fond of coincidences.

When's the last previous time you remember Google Search throwing 500's?


----------



## jonliles (Aug 10, 2022)

Jay Ashworth said:


> Indeed.
> 
> But I'm a system administrator, and like detectives, I'm not fond of coincidences.
> 
> When's the last previous time you remember Google Search throwing 500's?


At least 2 decades


----------



## Jay Ashworth (Aug 10, 2022)

jonliles said:


> At least 2 decades


They're surprisingly good at after-action reports on things that make the papers.

We'll see what happens.


----------



## Van (Aug 10, 2022)

Jay Ashworth said:


> Indeed.
> 
> But I'm a system administrator, and like detectives, I'm not fond of coincidences.
> 
> When's the last previous time you remember Google Search throwing 500's?


Maybe, and just hear me out here, Maybe their AI got really depressed about all the news coverage saying it was sentient and decided to take some action. Huh, Maybe. That or Aliens.


----------



## Jay Ashworth (Aug 10, 2022)

Van said:


> Maybe, and just hear me out here, Maybe their AI got really depressed about all the news coverage saying it was sentient and decided to take some action. Huh, Maybe. That or Aliens.


All good possibilities. And maybe that National Cat Day thing was just its apologia for taking that night off?


----------



## Van (Aug 11, 2022)

Oh, hey guys I wanted to make sure it's clear. I am not making fun of what happened just injecting levity into a terrible story. I'm beginning to realize that isn't always appreciated. No offense intended.


----------



## Jay Ashworth (Aug 11, 2022)

Van said:


> Oh, hey guys I wanted to make sure it's clear. I am not making fun of what happened just injecting levity into a terrible story. I'm beginning to realize that isn't always appreciated. No offense intended.


I see a heart and a haha; didn't seem like anyone took it wrong to me.

We're all wearing black here...


----------



## Van (Aug 11, 2022)

Jay Ashworth said:


> I see a heart and a haha; didn't seem like anyone took it wrong to me.
> 
> We're all wearing black here...


It's just a general rule I'm learning. (at 58) I'm too irreverent for my own good sometimes.


----------



## TimMc (Aug 11, 2022)

Van said:


> It's just a general rule I'm learning. (at 58) I'm too irreverent for my own good sometimes.


I'm voting for aliens. Marvin the Martian wasn't exactly friendly.


----------

