# resetting sensor rack errors



## TimMiller (Jan 8, 2009)

I was at a school and they have two sensor racks and both were throwing big hissy fits. They both were in desperate needs of cleaning. I did try to clear the errors with the doors open but i cannot remember how to reset the errors. I though you just pushed clear or reset to clear the errors, but it has been a while since i have played with the CEM's. I pushed clear, nothing happened, so i pushed reset, the CEM reset but gave me the same errors, they were air flow, rack over temp, dimmers xx, xx, xx.... over temp, and backup mismatch. Most errors i have seen on a rack. My button pushing finger was getting sore scrolling through all of the errors. I am going to go in next week or the week after and do a full maintenance cleaning on both of their racks. I will have to schedule some time for their theater to be dark since the dust is so caked up i will have to shut down the racks and suck as much dust out as possible then blow the rest of it out. I figured the rack wouldn't throw a over temp error if there wasn't a load on it and the room temp was rather cool.


----------



## renegadeblack (Jan 8, 2009)

I had to do that recently, do you have to power down the racks or does pulling out all of the dimmers and the control module do enough? When I did it last, I did kill the power to the rack, but I was just wondering whether or not that's neccesary.


----------



## DCATTechie (Jan 8, 2009)

To the best of my knowledge there is no way to clear the errors UNTIL the errors are fixed. Each time the CEM module resets I think it does a system scan and searches for problems. If a problem appears it will continue to flash the error code until the sensors see that everything is back to normal. My guess is that it does this to prevent the user from ignoring the problems and causing severe damage to the rack and the surrounding area. Once you clean the rack and fix the errors, flip on the main breaker, then reset the CEM and the messages should go away. If they don't I reccomend call ETC because it might be a faulty sensor or a problem in the CEM module.


----------



## DCATTechie (Jan 8, 2009)

renegadeblack said:


> I had to do that recently, do you have to power down the racks or does pulling out all of the dimmers and the control module do enough? When I did it last, I did kill the power to the rack, but I was just wondering whether or not that's neccesary.



I always kill power to the rack just to avoid risk or electrocution, wandering fingers tend to turn on dimmers when I least want them to be on. I also kill power because I clean the fans, and they're kind of difficult to clean when they're spinning around. My advice, don't be throwing your fingers anywhere where there is 120v power flowing around.


----------



## Esoteric (Jan 8, 2009)

KILL THE PANEL!!!! 100% absolutely without a doubt, KILL THE PANEL!!!

Mike


----------



## derekleffew (Jan 8, 2009)

If, after throughly cleaning the rack *with all sources of energy removed*, after a reset the CEM errors still persist, while at the rack, call Technical Support at 1.800.688.4116. There are configuration settings that ETC does not like to disclose because in the wrong hands they can create real havoc. I call them every time I have to move a CEM from one type of rack to another, for example.


----------



## Esoteric (Jan 8, 2009)

Yeah, I got a wierd error on a Unison rack once and they had to run me through some menu levels that I didn't know existed to fix the problem. I have to say though I have noticed a disturbing trend among schools, churches, and community theaters of not doing routine maintenance on their gear. Believe me it is cheaper to pay a professional to maintain your gear than it is to get it fixed when it breaks down. My dad has 30+ years in maintenance and you can not underestimate its importance. I know this is not practical for most venues but at the PAC we had 1/3 of our inventory down for maintenance at all times. It simply was not available and would be switched out with another 1/3 every 4 months. Kept everything running very smoothly.

Mike


----------



## Chris Chapman (Jan 8, 2009)

Double check which hardware version of the CEM you have as well. Original CEM modules didn't have a "snorkel" airflor attachment that drew air into the unit and cooled the CPU. Our first rounds of CEM's (Circa 97-98) did not have these and would give us flaky over temp and Dimmer Fail messages.

The snorkel is a little metal tube on the left hand side of the Mother Board, by the face plate. (I think.)


----------



## icewolf08 (Jan 8, 2009)

Is is a Sensor rack or a Sensor+ Rack? It makes a little difference. In a Sensor rack, the airflow sensor was notorious for getting clogged, you may have to vacuum or blow that clean, it is on the CEM. The Overtemp error won't clear until the rack come back to the right temp. Overtemp is probably due to the fact that the filters are clogged and air circulation is poor.


----------



## Sony (Jan 8, 2009)

renegadeblack said:


> I had to do that recently, do you have to power down the racks or does pulling out all of the dimmers and the control module do enough? When I did it last, I did kill the power to the rack, but I was just wondering whether or not that's neccesary.



*ALWAYS DE-ENERGIZE YOUR RACK WHEN YOU ARE PULLING MODULES FOR CLEANING.* The modules are in theory hot swappable in a Sensor Rack BUT you should only do so in emergency situations such as if one catastrophically fails during a show and you REALLY need it and then you should only do it when there is NO LOAD on the module i.e. it's turned off. More importantly is when you are removing a large amount of modules for cleaning you could very easily brush one of the live Feed Plates and kill yourself. It's just plain common sense that when you are doing anything inside a rack that you need to de-energize the rack. 

I've been told by a couple people that the CEM's are hot swappable too but I don't think I would ever be comfortable doing that.


----------



## renegadeblack (Jan 8, 2009)

Sony said:


> *ALWAYS DE-ENERGIZE YOUR RACK WHEN YOU ARE PULLING MODULES FOR CLEANING.*



I don't have an ETC, nor did I plan on cleaning without de-energizing, but I wasn't sure as being as on an innovator rack when you pull out the control module, the fan and everything turn off, I wasn't sure if it also killed power.


----------



## Sony (Jan 8, 2009)

renegadeblack said:


> I don't have an ETC, nor did I plan on cleaning without de-energizing, but I wasn't sure as being as on an innovator rack when you pull out the control module, the fan and everything turn off, I wasn't sure if it also killed power.



If I removed your brain would you still function? No...

The rack doesn't turn off it just stops functioning because you removed the control module. The fan and all the modules only run when they are getting signal from the control module. The same thing happens on ETC Rack and every other rack that has ever been made. I can guarantee that your feed plates are still very live and VERY deadly. 

You're very brave unplugging your Control Module when the rack is still energized. I don't know if the Innovator racks are mean to do that...you could very easily fry your Control Module and then you'd be out a CM, a whole Rack and easily $2,000 to get it fixed. I'm sure your administration would not be happy about that. Rule of thumb is ALWAYS de-energize your rack whenever you are doing any servicing. Hot swapping is meant to be a last resort in dire emergencies...i.e. fixing a failure during a live performance when de-energizing an entire rack is not an option.


----------



## renegadeblack (Jan 8, 2009)

Sony said:


> If I removed your brain would you still function? No...
> 
> The rack doesn't turn off it just stops functioning because you removed the control module. The fan and all the modules only run when they are getting signal from the control module. The same thing happens on ETC Rack and every other rack that has ever been made. I can guarantee that your feed plates are still very live and VERY deadly.
> 
> You're very brave unplugging your Control Module when the rack is still energized. I don't know if the Innovator racks are mean to do that...you could very easily fry your Control Module and then you'd be out a CM, a whole Rack and easily $2,000 to get it fixed. I'm sure your administration would not be happy about that. Rule of thumb is ALWAYS de-energize your rack whenever you are doing any servicing. Hot swapping is meant to be a last resort in dire emergencies...i.e. fixing a failure during a live performance when de-energizing an entire rack is not an option.



I figured as much, however, the guy before me who quit the crew had told me that the control module was hot-swappable. 

I also figured that they were still energized, but wasn't sure. I'd never dare do anything before de-energizing, but was just wondering.


----------



## cdub260 (Jan 8, 2009)

I've seen a lot of mentions in this thread about the need to de-energize the rack prior to servicing, but not one about using lockout/tagout procedures when doing so. For your own protection, when you de-energize the rack, you need to lock out the switch or breaker for the main power feed, so that no random passers by can re-energize the rack while you're working on it. This should be standard procedure when powering down any electrical system that you intend to work on, even if it's as simple as changing out a receptacle.


----------



## Esoteric (Jan 8, 2009)

cdub260 said:


> I've seen a lot of mentions in this thread about the need to de-energize the rack prior to servicing, but not one about using lockout/tagout procedures when doing so. For your own protection, when you de-energize the rack, you need to lock out the switch or breaker for the main power feed, so that no random passers by can re-energize the rack while you're working on it. This should be standard procedure when powering down any electrical system that you intend to work on, even if it's as simple as changing out a receptacle.



Certainly. However in every install I do, the breaker for the dimmers is very close to the dimmer rack, so that someone going to "mess with" the breaker would have to walk past/across the person working on the rack. But if that is not the case, then you MUST make sure that no one can accidentally or otherwise repower the rack while you are working on it. Then when you are done, get the panel moved!!!

Mike


----------



## TimMiller (Jan 8, 2009)

i am going to pull all of the modules out of this rack and give it a very good cleaning. But I am then going to teach them how to do simple cleaning procedures such as cleaning the air filter and vacuuming off the front of the dimmers. This school is several years old, and they just got a new theater teacher who actually cares about theater and understands that everything in the theater is lacking in maintance.


----------



## Sony (Jan 8, 2009)

cdub260 said:


> I've seen a lot of mentions in this thread about the need to de-energize the rack prior to servicing, but not one about using lockout/tagout procedures when doing so. For your own protection, when you de-energize the rack, you need to lock out the switch or breaker for the main power feed, so that no random passers by can re-energize the rack while you're working on it. This should be standard procedure when powering down any electrical system that you intend to work on, even if it's as simple as changing out a receptacle.



Good catch, like Esoteric all the racks I have worked on have had the service disconnect right next to me while working on the rack. I however definitely agree that you should use proper lockout/tagout procedures, ESPECIALLY if the disconnect is in another room or somewhere out of sight from where you are working.


----------



## renegadeblack (Jan 9, 2009)

Yeah, the disconnect is in the basement which is a fair distance from the actual dimmers. When I cleaned it out, I put red tape over the breaker saying don't turn me back on, maintenance underway or something of that sort, fortunately, it's fairly rare that people even go down there so the chance of someone even going down there and flipping it is fairly slim.


----------



## Esoteric (Jan 9, 2009)

I would still want to lock the panel if I were you. Then I would go out and bid an electrician to have the panel moved (or a disconnect added) in the dimmer room.

Mike


----------



## TimMiller (Jan 9, 2009)

rather than moving a panel just use proper lock out tag out procedures. they sell locks at Grainger Industrial Supply for breakers and disconnects of all sizes. Then someone would have to cut the lock in order to turn it on.


----------



## icewolf08 (Jan 9, 2009)

First of all, if ETC says that it is OK to hot swap modules and CEMs then it is OK to do it. You would never say that you shouldn't hot swap a USB or FireWire hard drive right? I know the voltage is 10x higher for dimmers, but that 12V on USB is certainly enough to fry the gear!

Sure, if you are planning on removing more than one module at a time you should power down the rack as one could easily find the copper conductor plates with one's fingers. However for day to day use and maintenance you should be able to leave the rack up.

Don't get me wrong I am all for safety, but if a manufacturer tells me that something is safe and OK to do then it should be.

Thats all, I return you all to your debates and whatnot.


----------



## Esoteric (Jan 9, 2009)

Actually I don't hotswap USB devices. Ruined an iPOD that way.

But that is just me.

Mike


----------



## Sony (Jan 9, 2009)

icewolf08 said:


> but that 12V on USB is certainly enough to fry the gear!



USB runs off 5v...just FYI.


----------



## lieperjp (Jan 9, 2009)

derekleffew said:


> call Technical Support at 1.800.688.4116.



Do you have this number memorized?

I agree with Alex, if the manufacturer says it's ok to do something, it should be ok. But it's not a bad idea to play it safe, either...


----------



## cdub260 (Jan 9, 2009)

renegadeblack said:


> Yeah, the disconnect is in the basement which is a fair distance from the actual dimmers. When I cleaned it out, I put red tape over the breaker saying don't turn me back on, maintenance underway or something of that sort, fortunately, it's fairly rare that people even go down there so the chance of someone even going down there and flipping it is fairly slim.



All it takes is once.


lieperjp said:


> Do you have this number memorized?



More likely, he's got it on speed dial.


----------



## derekleffew (Jan 9, 2009)

cdub260 said:


> ...More likely, he's got it on speed dial.


As I've often preached before, EVERY theatre should have the toll-free 24/7/365 phone number of their manufacturers posted next to the phone in the light booth and in the dimmer room.

Sorry, I can't help if you have no light booth or dimmer room, or if your manufacturer doesn't have an emergency number. But you DO have the office and after-hours phone numbers to your local friendly lighting supplier, right?

Yes, it's programmed into my cellphone, along with many other contacts for support. In Chicago, my Colortran rep. used to keep Prestige and D192 parts in his car.


----------



## Esoteric (Jan 9, 2009)

derekleffew said:


> As I've often preached before, EVERY theatre should have the toll-free 24/7/365 phone number of their manufacturers posted next to the phone in the light booth and in the dimmer room.
> 
> Sorry, I can't help if you have no light booth or dimmer room, or if your manufacturer doesn't have an emergency number. But you DO have the office and after-hours phone numbers to your local friendly lighting supplier, right?
> 
> Yes, it's programmed into my cellphone, along with many other contacts for support. In Chicago, my Colortran rep. used to keep Prestige and D192 parts in his car.



I have all my suppliers emergency/cell phone numbers! When I do an install I post them all over the place.

Mike


----------



## soundman (Jan 9, 2009)

It is possible to do a deep reset of the rack the keys escape me but you hold reset and another key until it reboots and flashes password. Then you can clear out all the errors and other information. If you are not sure what you are doing it is no place to poke around as you can reset to blank slate.


----------



## jmabray (Jan 9, 2009)

Actually - regardless of what anyone tells you, you should know all the rules and regulations your self before you go hot swapping dimmer modules. Can anyone here tell me what NFPA 701 says about this?


----------



## derekleffew (Jan 9, 2009)

jmabray said:


> ... Can anyone here tell me what NFPA 701 says about this?


I can tell you that

NFPA 701: _Standard Methods of Fire Tests for Flame Propagation of Textiles and Films_ is probably not applicable, and that you meant NFPA 70E. And agree wholeheartedly, anyone going near an electrical source or distribution should be familiar with its contents.


----------



## Sony (Jan 10, 2009)

I just wish you didn't have to pay to read the 70E...I can't really afford all those books and subscriptions on my measily college student income...


----------



## LightStud (Jan 10, 2009)

> Seems she forgot all about the liberry,
> Like she told her old man, now.


Wilson/Love, 1964.


----------



## jmabray (Jan 10, 2009)

Yep. That was the one. Sorry about that. It's what I get trying to do things from memory....


----------



## TimMiller (Jan 10, 2009)

USB devices are made to be hot swapped. You have to eject the device before removing it, especially if it has a hard drive, because then it properly shuts it down. Also when you eject a USB device it disconnects power to that device. So you cannot hurt anything if it is made to be hot swapped. Although, ejecting is not the same as truely hot swapping, in which electronics really do not like.


----------



## SteveB (Jan 13, 2009)

renegadeblack said:


> I had to do that recently, do you have to power down the racks or does pulling out all of the dimmers and the control module do enough? When I did it last, I did kill the power to the rack, but I was just wondering whether or not that's neccesary.



From the ETC Users forum, and as posted by an ETC Tech Services Rep.

"Sorry about the junk on your CEM+ Display. When you get back to the Rack, pull the CEM+ Processor Module and push it back in - which causes it to restart. You can pull it out by pushing the white Ejection symbol on the front side in the middle bottom and then pulling on the lever until the whole CEM+ pulls out."

So - Yes, you can hot seat CEM's and CEM+'s.

And FWIW, you can do the same with the dimmer and relay modules, PROVIDED you kill the load breaker on the module before pulling the module, then re-set it once it's re-seated.

Steve B.


----------



## jmabray (Jan 13, 2009)

Not according to NFPA 70E. At least not without a lot of protective gear being worn.....


----------



## beachcombah15 (Jan 13, 2009)

> "Sorry about the junk on your CEM+ Display. When you get back to the Rack, pull the CEM+ Processor Module and push it back in - which causes it to restart. You can pull it out by pushing the white Ejection symbol on the front side in the middle bottom and then pulling on the lever until the whole CEM+ pulls out."



That was the answer to MY question...haha


----------



## SteveB (Jan 13, 2009)

jmabray said:


> Not according to NFPA 70E. At least not without a lot of protective gear being worn.....



Correct and I'll amend my statement to say that it while according to ETC, it doesn't hurt either the dimmer or CEM to have a module inserted with the rack energized, it's possibly bad practice.

I'm un-clear as to the risk of Arc Flash in a Sensor, due to the design with the lugs behind the module, especially as the risks associated with arc flash is generally a problem in circuit and fuse panels. But good practice dictates that the rack be de-energized if possible.

Steve B.


----------



## derekleffew (Nov 13, 2011)

SteveB said:


> Correct and I'll amend my statement to say that it while according to ETC, it doesn't hurt either the dimmer or CEM to have a module inserted with the rack energized, it's possibly bad practice.
> 
> I'm un-clear as to the risk of Arc Flash in a Sensor, due to the design with the lugs behind the module, especially as the risks associated with arc flash is generally a problem in circuit and fuse panels. But good practice dictates that the rack be de-energized if possible.


Note that ETC may have revised its opinion regarding the hot-swapping of control and/or dimmer modules since the above posts were written in 2009.

It's in the wiki entry Lock-Out/Tag-Out should anyone need to find the text in the future:

> Even something as mundane and seemingly harmless as swapping a dimmer module (a very common troubleshooting procedure) now requires completely powering down the rack.
> 
> starksk said:
> 
> ...


----------

