# DANTE Network with X32, 2 M32's and a D400 Aviom Switch- Audio Popping



## StradivariusBone (Dec 29, 2020)

So I posted this in the Music Tribe forums as well, but I'm not sure how much hope I have in getting a solution over there. Here's the deal:

We have two venues linked to a single broadcast room via an isolated network that supports both DANTE and NDI traffic. Despite DANTE Controller not showing any clock errors, we get popping artifacts in the audio stream on the output side of the DANTE network. The diagram shows the layout of our network:



Originally we started using DANTE with a single M32 networked to the MacBook Pro running DVS. That setup by itself would cause pops to be heard, but again no clock issues showed up in Dante. All that was on the network at that point was the M32 Dante card, the D400 and the MBP running DVS. I need to do some more homework/troubleshooting, but I can't rule out the Aviom D400 as a problem child. 

Anytime we try to set either M32 to sync to external, we start having loud pops in the PA as the clock sync with the DL32's gets messed up. Enabling sync to external in Dante Controller sorta works for a bit, but will ultimately cause clock failure as well. THe most stable config seems to be M32's sync internally, X32 producer set to sync to external and letting Dante Controller sort out the clock master. I should also point out that we are using shielded cable from M32 to DL32. We even took the DL32 out of the link in venue #2 to see if it made a difference and it did not. 

I can't seem to find anyone else with similar issues. Any google-fu with M32/X32 and popping only leads to the STP issue which is not our problem. Also, the NDI equipment was added, but the popping problems existed prior to the addition of that hardware to our network so I don't think it's an issue with NDI multicast causing problems.

And I know about the Ubiquiti switches and their questionable support of multicast traffic. The initial setup had an old 10/100 switch and had the same symptoms.


----------



## StradivariusBone (Dec 29, 2020)

Here's a video with the symptoms. Recording from Venue #2 during this, it's hard to hear over computer speakers, but headphones on and you can hear the little click/pops. 








New video by Ed Dunne





photos.app.goo.gl


----------



## FMEng (Dec 29, 2020)

Have all of the X-Dante cards been upgraded to the latest firmware? The cards have their own firmware, apart from the console firmware.


----------



## StradivariusBone (Dec 29, 2020)

As far as I can tell via Dante Controller yes. The only two devices that aren't are the MBP DVS, because there was a bug with the latest firmware on that a few months back (might've been patched by now) and I guess the firmware on the DL32's themselves have been patched recently, though not directly related to the Dante network.


----------



## FMEng (Dec 29, 2020)

Keep in mind that I haven't done any Dante with my X32s in a few years, but I will be venturing into it again in the next few weeks. (I will be putting NDI and Dante together with Tricaster. Wish me luck.) 

The latest Dante firmware is 4.08. Check it with Dante Firmware Update Manager. It would be a good idea to do the update, with the file directly from Behringer/Midas, to be sure. Use the latest version of Controller, too. The consoles must all be set to sync from X-Dante, and then one of them has to be chosen as sync master in Controller. I would set the consoles as preferred masters, and turn off others. Never make any sync changes with any speakers running.

I've heard about enough issues with Macs and Dante that I would suspect an issue there. Eliminate the MBP from the system and see if things improve, especially if it isn't running the current version of DVS. I know nothing about Macs, but in the PC world, I would experiment with not using any other sound devices while running DVS, making DVS the default sound device, and even going so far as disabling internal sound hardware. Conflicts there could ripple across the network, and multiple sound devices do conflict.


----------



## StradivariusBone (Dec 29, 2020)

The mac in or out unfortunately doesn't make much different in the audio out to the X32. The other problem we have is that setting sync to the card on each M32 will result in loud pops through the PA as the DL32's start losing sync. I think that's our problem honestly, but I cannot find any documentation about it.


----------



## MNicolai (Dec 29, 2020)

StradivariusBone said:


> Originally we started using DANTE with a single M32 networked to the MacBook Pro running DVS.



I would start troubleshooting here. Break the network down into smaller pieces and work through one problem at a time and take notes as you connect up other parts of the system where the problems start.

If I recall correctly, the network switches you're using are not Layer 2 managed switches. Before you dig too far, I would swap out for Cisco SG350's. On a Layer 2 unmanaged switch, multicast becomes broadcast and QoS can fail to work (which is important for clocking, also if you have other forms of streaming media like the NDI's on the network).

Checking Dante firmware as described above is a good idea, though I doubt your issue is with DVS. Most of the Macbook issues I've heard about relate to the recent OS migration to exclusively 64-bit and it's been bumpy for Audinate and others to work out the kinks in their drivers.

Doesn't sound like a lot of equipment all things considered. You could also move everything into one room, eliminate the Ubiquiti and the fiber/media converter bits. Then get the network stable without the interconnections where you can troubleshoot everything at once in the same room before you start diagnosing issues with the interconnections between the systems.


----------



## StradivariusBone (Dec 29, 2020)

Yeah, the first switch we had was an old LinkSys 10/100 "dumb" switch. These new ones are Netgears that are at least somewhat managed Layer 2 switches and I've pulled their model numbers and google fu says they should do alright with Dante. They can handle VLAN's but we haven't messed with that yet because I'm not sure it'd play nice between the two switch manufacturers. I haven't done a deep dive on setting it up through the Unifi.

We do have the option to add another fiber link to the mix between broadcast and venue #2, an option we were thinking of exploring if we added more cameras from that side. However with just 3 NDI sources on that pipe, it's not really filling it up.

I agree though, back to basics with maybe just an M32 and the X32 together and see if I can isolate it. I really feel it's related to the clock sync on the boards and how it handles clocking on that AES50 line. That's the only thing that's the same between our first foray into this world and where we are now. One thing I have not tried is removing the DL32 and setting clock to external on the M32. Like I said, it can't handle clocking on the dante card with a DL32 for some reason.

Edit- Oh! And that Aviom switch. I don't trust that thing. It sometimes has little panic moments when we bring another Dante device online.


----------



## MNicolai (Dec 29, 2020)

Have you logged into the Netgears and configured the QoS priorities, disable IEEE, usual Dante preferences, etc?


----------



## StradivariusBone (Dec 29, 2020)

Yeah, I don't think these ones even support IEEE. The QoS is pretty basic unfortunately and I don't think the Unifi even supports QoS, which is odd. That whole switch is odd. Turning on IGMP snooping on that switch disables 100% of the Dante multicast traffic and kills it dead. It's known issue for them, at least on their forums.


----------



## FMEng (Dec 29, 2020)

The whole system has to be clocked together, otherwise there will always be issues. The DL32 has to get its clock from the console. The console has to get its clock from Dante. Everything seems to point to a network issue causing the consoles not to get reliable sync.

Awhile back, I used a media converter, and found it to be a bottleneck, so I'm suspicious of that. I'm also leery of running ethernet between buildings (assuming Ubiquiti and Venue 1 are not co-located). Buildings are at different ground potentials, and the ethernet transformers may not provide enough isolation and noise immunity under those conditions. Non-audio data is more robust because packet re-sends cover up a multitude of sins. AoIP doesn't tolerate missing or corrupted packets at all.

The bottom line is you need serious switches with SFP modules for fiber, and runs between buildings to be fiber. I like Netgear switches for more typical uses, but for AoIP, I only use proven models, such as SG-350 series. The cost isn't that bad. In fact, I have two sitting next to some NDI cameras, on my dining table. When I started my project, I told myself I wouldn't do it without top quality network switches. I'm volunteering to do the installation, so the least the church can do is pay for components that won't waste large amounts of my time and make my hair turn gray.

To prove the point, buy one SG-350. Configure it properly. Put it and all of the audio devices together in the same room. Fire up the tone generator in one of the consoles and ship the audio around via Dante. I will bet it all works perfectly.


----------



## CrazyTechie (Dec 29, 2020)

StradivariusBone said:


> They can handle VLAN's but we haven't messed with that yet because I'm not sure it'd play nice between the two switch manufacturers.



This should be fine as long as both switches support VLANs, which it sounds like they do, and you use the same VLAN numbers on all the switches. VLAN tagging is built into the ethernet frame structure so it's universal. You'll just have to get the hang of tagged vs untagged ports. 

So, if you setup VLAN 10 on both your Ubiquiti and Netgear switches, traffic will flow between them along your trunk ports. 

The Cisco SG switches should serve you pretty well. The web interface is pretty easy to work in (waaaay better than Ubiquiti) especially if CLI isn't your preferred switch management method.


----------



## tmcgow1 (Jan 1, 2021)

StradivariusBone said:


> So I posted this in the Music Tribe forums as well, but I'm not sure how much hope I have in getting a solution over there. Here's the deal:
> 
> We have two venues linked to a single broadcast room via an isolated network that supports both DANTE and NDI traffic. Despite DANTE Controller not showing any clock errors, we get popping artifacts in the audio stream on the output side of the DANTE network. The diagram shows the layout of our network:
> View attachment 21303
> ...



This is not a simple problem to deal with. I would suggest looking at your source power as you stated the setup would pop by itself. Nothing in the data diagram is irregular or incompatible. 
You are describing a classic power differential pop that is from items being out of phase and potential differential voltage between neutral and ground. Is everything under common switchgear? Same distro panel? Same phase? - this can be the first problem source of differential voltage on neutral legs. Common earth ground? - all earth ground potentials are not the same when dealing with clocks. ( that's another story) You should verify or self-install 8' copper grounding round as close to main distro as possible. Inspect it yourself and make sure there is not a 10-12 awg copper where there should be 4-8awg for sufficient ground. You can do the math for resistance and distance. In average soil there is 25 Ohms resistance already built into calculations. Plug strips and devices must all have earth grounding connectors. Racks should be connected to the common ground. Any lifted grounds should be primary suspects for induction to the common copper connection either audio and data resulting in a micro-ampere discharge when peak potential is achieved resulting in a POP. Ask why any ground is lifted? Toss any cheap plug strips and use IEEE certified power distribution hardware. Every discharge or pop upsets and/or restarts clocks, drains diodes, flashes memory, and causes havoc on a micro scale. Think of it as a mini bolt of lighting inside your power distro. 
Then there is also rogue sources such as bell/alarm circuits, ethernet, telco/pots, cable tv, cable modem, surveillance that can carry a mish-mosh of voltage, grounding or lack-of that all puts demand on the grounding systems and draining of leaky voltage. External sources commonly cause noise either hums, pops, echo and timing errors between clocks. Have a common demarc ground for all external powered wiring such as a single lash up board. Our theater uses a 3P 200A voltage regulator before my fixed house distro with copper bonding to earth. All distro panels for stage, audio, lighting, data, all have common earth ground to eliminate any potential between devices. Any clock issues, pops, or hums can be tracked to interconnect cables or devices by process of elimination by type of system, lighting, data, video, audio. If you are subject to flooding or very high humidity, create a decision tree to eliminate moisture penetration inside devices or connect points. 
If you do use lifted grounds or non-shielded anything, then consider voltage regulators and./or hum-buckers which should drain off the offset power charge. Once power is eliminated as a potential source for pops, look at your oldest gear that may have a leaky diode or faulty push-button that is on it's way out. 

Clocks can be finicky on their own. The clock itself should not cause a pop sound. For Dante there is internal IEEE 1588 PTP or AES3 or word clock in a console. Only 1 clock can be selected as master clock, all else are slave clocks. Clock Synchronization (audinate.com)


----------



## StradivariusBone (Jan 2, 2021)

That's a potential with the the building we are dealing with. Venue #1 and Broadcast both exist in the older side of the campus where Venue #2 was built about 15 years after. The only thing is we had this problem when we were just running a closed network with one M32-DL32, one laptop, and the Aviom box. All of which were in the same room, fed from the same panel. My troubleshooting gut is pointing towards something in that group causing the faults, but it is never a bad time to check out equipment grounding and make sure it's correct!


----------



## MNicolai (Jan 2, 2021)

StradivariusBone said:


> That's a potential with the the building we are dealing with. Venue #1 and Broadcast both exist in the older side of the campus where Venue #2 was built about 15 years after. The only thing is we had this problem when we were just running a closed network with one M32-DL32, one laptop, and the Aviom box. All of which were in the same room, fed from the same panel. My troubleshooting gut is pointing towards something in that group causing the faults, but it is never a bad time to check out equipment grounding and make sure it's correct!



It's not possible that it's a grounding/bonding issue between the buildings. Your fiber link between the buildings doesn't care about earth potential. For that matter, neither does unshielded CATx cable. The only links you could have an issue are on AES50 runs with shielded CATx cable, and the shield is required to be terminated at both ends by Behringer/Midas for the specific reason of solving their AES50 popping manufacturing defect.

Honestly, I would break it all down into just the one system and work out the issues on an island before even thinking about the interconnections to other rooms. You are seeing a Dante issue but are you absolutely sure it isn't an AES50 problem that's rippling through the entire system? Have you double checked that your STP CATx cables are terminated at both ends and verified with a continuity tester?


----------



## StradivariusBone (Jan 2, 2021)

I made the Cat5e cables myself and they are grounded at both ends. The old wisdom of pulling the ground on one end is something I've always found slightly questionable. I'm hoping to get a chance tomorrow afternoon to do some isolation troubleshooting. We can simplify it to one room, I'm just wondering why I can't run the clock off the card without causing the DL32 to freak out.


----------



## FMEng (Jan 2, 2021)

I like Mike's idea of breaking the system down to it's parts, and making sure the basics are working before looking further.

Be sure to use Neutrik Ethercon shells on both ends of the AES50 cables. The shell is where the critical connection is made.  Just using the ground connection on the RJ-45 causes trouble.

Finding out why the system won't run from the X-Dante card clock is key, because it absolutely must. It would be very useful to run just the console and the DL32, with the X-Dante card as the clock reference, but without any Dante network connection. That would tell us a lot. If that fails, then try the console without the DL32, clocked to the X-Dante card.

You live in humidity and salt central, and the accessory card slot connection does not inspire confidence. Maybe it's time for a spritz of DeoxIT D5 and and Gold G5.

In the analog audio world, dropping the shield at one end is what makes large systems work that would otherwise be a giant mess of hums and buzzes. I've built several radio facilities that way. Behringer's implementation of AES50 is a special case because they goofed on the circuit board design and caused what is called "the pin 1 problem." Basically, they made a high impedance ground that induces noise into adjacent circuitry. The Ethercon shell shunts enough noise to the chassis, preventing data corruption and sync losses.


----------



## StradivariusBone (Jan 3, 2021)

FMEng said:


> The shell is where the critical connection is made.  Just using the ground connection on the RJ-45 causes trouble.



Hmmm.

So that was a light bulb. We did use the shells, but the weird thing was when the cable strain was screwed all the way down it would not work correctly. The DL32 would show up, but sync was red. Here's my solution-




It seems pretty happy with this. So the XLR shield is not bonded to the RJ45 shield? Not sure why it doesn't like the back end screwed down. These are the "pass-through" RJ-45's, so maybe something is making contact when that is jammed in there?

I was able to switch the clock source over to the card on both venues (Venue 2 actually already has the DL32 out of the loop but still had the clock internal). 

So far so good?


----------



## TimMc (Jan 3, 2021)

StradivariusBone said:


> Hmmm.
> 
> So that was a light bulb. We did use the shells, but the weird thing was when the cable strain was screwed all the way down it would not work correctly. The DL32 would show up, but sync was red. Here's my solution-
> View attachment 21317
> ...


I would reterminate the RJ45 and cut back about 4" on the cable... 

Here's my guess: the compression on the cable by the strain relief chuck is likely creating an impedence bump. You may need to modify the chuck (or find the larger, white plastic one) because STP has a greater diameter than UTP.

The other possibility is that when you screw down the tail piece that some mis-alignment of the RJ45 is occurring, whether that's in the contact mating or the shield ground/connector shell interface, I can't say.


----------



## StradivariusBone (Jan 3, 2021)

The RJ45 is very misaligned in the ethercon. That partially led to us deciding to go without them for the time being since they wouldn't really seat properly into the socket with the strain-relief screwed down. I've heard of these pass-through RJ-45's having issues like this, so I terminated them with a bit of the copper inside (not sticking out the other end), but I'm wondering if the angle of the RJ45 is causing some pins to not seat completely? That might explain the console recognizing the DL is connected, but not able to fully sync. That was the other part of abandoning the XLR barrel, it was happier without them. 

In any event, it seems like this is solved. I was able to pull clock from the Dante cards and we had no audio distortions (outside of a funky RF mic issue- unrelated and mildly expected). Thanks to all!


----------



## FMEng (Jan 3, 2021)

Getting the clock from the X-Dante cards is a big step in the right direction. 

I don't think the cable diameter is the issue with the shells. I use direct bury cable for ruggedness in portable use. It is thick because it has double jackets, but it still fits in the ethercon shell. The jacket should be crimped in the plug.

I have found that there are plugs made for solid wires, and plugs made for stranded, and flakey connections result when cable type and plug don't match. Too many manufacturers gloss over this detail. Generally, the pass through plugs are made for stranded. Solid wires go into non-pass through plugs pretty easily. Plugs made for solid wires are often smoke colored.

Back in the ethernet dark ages, I prewired a whole rack of equipment with the wrong RJ-45s, and had flakey connections galor. I ended up redoing all of the terminations and all of the problems were solved.


----------



## Jay Ashworth (Jan 3, 2021)

Quite so. Nearly all RJ-45's you find for sale are solid; you have to hunt for the stranded ones, which are really only needed for building patch and drop cables, and most people don't bother to use stranded wire for that anyway.

All the stranded icecubes I've ever seen were smoke grey, though I don't think that's any kind of standard.


----------



## StradivariusBone (Jan 4, 2021)

Yeah, jacket went into the plug, pulled the drain wire out and soldered it to the RJ45 shell for good luck. I'm not sure why we ended up with passthrough, but that's what Amazon sent. Everything was super stable, but at least we know where the fault lies if it comes back. I'll just gaff some tinfoil around the back of the console next time


----------



## Terence Gray (Jan 6, 2021)

This has been a great thread to read through. 

I'm having similar noise issues in a Dante network setup we want to use for recording, although our audio hardware is all Yamaha. The intended setup is to record individual sources directly on a Mac Mini (and possibly also on a laptop) through DVSC, while at the same time providing a mix to the video recorder. This is basically taking the existing live setup and tapping into the Dante network to record audio. Due to a lack of patchable data ports at FOH, I added a Cisco SG300-20P switch between the console and the dedicated ports to Dante Primary and Secondary switches, located in the amp room, and ran the Dante connections through that. The result was noise in all three audio recordings – the mixed feed to video, the house Mac Mini, and my personal MBP. 

Some of the suggestions above are things I've already considered and/or implemented, but others have given me more things to try. 

I've been using RJ45-EZ (aka pass-through) connectors for nearly 15 years, and have never had a termination issue with them. Most of that work has been done on solid core CAT cable. The only problems to crop up have been with the ports in connected hardware. One of the keys is using a really good crimp tool and making sure the cutting blade is sharp and correctly aligned. While most RJ45 connectors are designed for use with either solid or stranded cable, there are a lot on the market that are designed to work with both. 

Cheers.


----------



## StradivariusBone (Jan 6, 2021)

You might want to explore if you need to enable multicast for some of your flows. If you're sending audio to 3 more more devices it's advised that you enable it. Without knowing the network map it's hard to say if that's the cause or not. 

And I'm not sure if I am understanding correctly, but you have the primary and secondary dante ports on the same switch? The idea with the two ports on the interface is to have two redundant but separate networks so if one fails, you can quickly jump to the backup. They aren't designed to work together over the same subnet. Though I think it does function that way if you hook it up, so I don't know that your problem is that.


----------



## yert33 (Jan 6, 2021)

So what's the lesson from this exciting sleigh ride of community troubleshooting?
1. Behringer's AES50 pin 1 problem?
2. Wrong connector for STP CATx?
3. Faulty termination of RJ45?
Combination of the 3?


----------



## Terence Gray (Jan 6, 2021)

StradivariusBone said:


> You might want to explore if you need to enable multicast for some of your flows. If you're sending audio to 3 more more devices it's advised that you enable it. Without knowing the network map it's hard to say if that's the cause or not.
> 
> And I'm not sure if I am understanding correctly, but you have the primary and secondary dante ports on the same switch? The idea with the two ports on the interface is to have two redundant but separate networks so if one fails, you can quickly jump to the backup. They aren't designed to work together over the same subnet. Though I think it does function that way if you hook it up, so I don't know that your problem is that.


I did have multicast enabled for the flows.

The switch was set up with two VLANs, one for the Primary and one for the Secondary. I know this isn't best practice, but it's my understanding that it's more about the potential failure of a single piece of hardware taking down the entire Dante network, than it is about traffic issues. The Primary and Secondary are on two different subnets – each of which is using static IPs.

We went back to an earlier setup that didn't tap into the Dante flows for the individual channel recordings, and took the switch out of the signal chain, which got us back to clean recordings. The problem is, that requires using my personal MY16-AT card in the QL1, and my FocusRite 18i8 to get to my MBP (there is a FocusRite 4Pre USB going to the Mac Mini). This isn't a problem when I'm the one on hand to do the recording, but we need a set up that anyone can come in and use. The next time we are recording, I plan to use the rehearsal day to try some different configurations and see if I can get a clean signal from the Dante audio flows. One definite change will be not using the Secondary VLAN – the only device at FOH that has a Secondary port is the QL1, so that will go straight into the designated data port to the amp room. That opportunity won't be for at least another month. I'll also be looking at the switch configuration to see if there's anything I missed in the port set up.


----------



## MNicolai (Jan 6, 2021)

Terence Gray said:


> The switch was set up with two VLANs, one for the Primary and one for the Secondary. I know this isn't best practice, but it's my understanding that it's more about the potential failure of a single piece of hardware taking down the entire Dante network, than it is about traffic issues. The Primary and Secondary are on two different subnets – each of which is using static IPs.



This is fine for most users. Switch failures are very rare with exception to someone accidentally unplugging one or roaching the trunk cable between switches. The most common causes of transient Dante network problems are bad cables out to endpoint devices or someone plugging something into the network that causes a major conflict because of bandwidth or a failed network port on a device that corrupts the network. I'm not wild about static IP's on Dante networks because it adds extra steps when your temporarily adding Dante devices or upgrading firmware that may blow out the IP settings, but it's not a big deal so long as you remember you have to set those statics if the system gets changed around. Dante is much more resilient and self-healing than other AV streaming protocols so the general wisdom of static-for-stability is almost always an unnecessary precaution that can create more problems than it prevents.


Terence Gray said:


> The result was noise in all three audio recordings – the mixed feed to video, the house Mac Mini, and my personal MBP.



What kind of noise? Blips, intermittent drop-outs, general distortion, or does it sound more like digital artifacts?

If the audio is playing continuously but there's an issue with a higher noise floor or such, it's unlikely that Dante is the cause because Dante tends to be true/false -- either it's passing audio or it isn't but the actual audio waveform isn't getting manipulated. If the EEE energy efficiency is enabled on those ports in the switch, that can create problems that will lead to blips and digital artifacts. I would probably check that and page through Yamaha's SG300 configuration guide and verify everything is set up the way Yamaha recommends.


----------



## StradivariusBone (Jan 6, 2021)

The idea with the redundancy is that you would duplicate your equipment as well. In reality, it's overkill for most things. Here's the example pic from Audinate- 



How you have it configured will work with what Mike's describing. I do static IP's for our network, but I have a router between us and the main network and have universal control over the addressing scheme. If you have a situation where people are BYO gear and patching in, letting the DHCP handle it might be better. 


Terence Gray said:


> took the switch out of the signal chain, which got us back to clean recordings


That's where I think I would start. If removing that piece of gear seems to fix things, then maybe there's a setting on it that is causing you issues. 

Yamaha has some pretty good resources on Dante (in some cases I found it better than Audinate's own stuff!) Here's a landing page for some of their best practices- https://usa.yamaha.com/products/contents/proaudio/docs/dante_network_design_guide/index.html


----------



## Terence Gray (Jan 6, 2021)

MNicolai said:


> This is fine for most users. Switch failures are very rare with exception to someone accidentally unplugging one or roaching the trunk cable between switches. The most common causes of transient Dante network problems are bad cables out to endpoint devices or someone plugging something into the network that causes a major conflict because of bandwidth or a failed network port on a device that corrupts the network. I'm not wild about static IP's on Dante networks because it adds extra steps when your temporarily adding Dante devices or upgrading firmware that may blow out the IP settings, but it's not a big deal so long as you remember you have to set those statics if the system gets changed around. Dante is much more resilient and self-healing than other AV streaming protocols so the general wisdom of static-for-stability is almost always an unnecessary precaution that can create more problems than it prevents.
> 
> 
> 
> ...


Definitely sounds like digital artifacts, and definitely isn't general distortion or noise floor issues. 

The system was set up by the installer, in early 2019, and literally nothing was added to the Dante network until last October, when we shifted from doing in-person recitals to recording them for subscriber-accessed streaming, and I wanted a convenient way to record each individual mic separately, in addition to the mix I was creating for the video side. The first thing I had to do was track down one of the DVSC tokens that came with the QL1, Rio3224 and Rio1608 and install DVSC on the Mac Mini. 

The video producer we send the recordings to described the waveform as a sudden drop in signal to 0dB, then a sharp jump back up to the program level, which conicides with a digital crackle in the audio. When I listen to the individual tracks, that noise it present in ALL of the input channels at the same time. Again, I suspect it's in the SG300, either something I missed, or an issue in the switch itself, which was taken from the electrics department spares. I will definitely check out the Yamaha site for any notes they have about it.

There's no one plugging or unplugging anything into the Dante network, and everything we are using – with the exception of the Mac Mini and my MBP – is part of the original installation. The cables from the QL1 to the Cisco switch, and from the switch to the FOH data ports, are all brand new CAT6, and when I pulled out the switch, I used the same CAT6 cables from the QL1 directly to the data ports without any trouble.


----------



## Terence Gray (Jan 6, 2021)

StradivariusBone said:


> The idea with the redundancy is that you would duplicate your equipment as well. In reality, it's overkill for most things. Here's the example pic from Audinate-View attachment 21359
> 
> 
> How you have it configured will work with what Mike's describing. I do static IP's for our network, but I have a router between us and the main network and have universal control over the addressing scheme. If you have a situation where people are BYO gear and patching in, letting the DHCP handle it might be better.
> ...


Thank you for the link. I'll definitely be looking at what Yamaha says about the SG300. 

To be clear, we removed the switch AND changed the individual channel recordings to direct out from the QL1 via ADAT optical, so it could still be Dante problem related to the way I had the signal flow routed.


----------



## TimMc (Jan 6, 2021)

What is the Dante master clock? Are there any error messages in Dante Controller?


----------



## Terence Gray (Jan 7, 2021)

TimMc said:


> What is the Dante master clock? Are there any error messages in Dante Controller?


Clock Master is the Rio 3224. No to error messages.


----------



## StradivariusBone (Jan 7, 2021)

If the dropouts you're describing were a symptom of the Dante failing, you would almost certainly see something in the error log. I had an issue once with my DAW (I was using StudioOne) where it's clock or sampling rate got jacked up and it was having all kinds of issues. Recorded at like 1/3 speed or something weird. But it also had issues with dropouts and that was related to the MBP and software I was using.


----------



## Terence Gray (Jan 7, 2021)

StradivariusBone said:


> If the dropouts you're describing were a symptom of the Dante failing, you would almost certainly see something in the error log. I had an issue once with my DAW (I was using StudioOne) where it's clock or sampling rate got jacked up and it was having all kinds of issues. Recorded at like 1/3 speed or something weird. But it also had issues with dropouts and that was related to the MBP and software I was using.


They sounded very much like the ones on the video clip you posted – but louder, and more frequent.

If they were only present on one of the recordings, I'd be looking at the computer or video deck as the problem area, but they are on all three. The common connection point was the switch, so we went back to a setup that eliminated it, and had clean recordings. I did change the recording path on my MBP to an external Seagate HDD (7200 RPM), and have asked for an external SSD for the in-house Mac Mini. 

The Yamaha site has been helpful in pointing to a couple of switch setup things I may have missed. I'm back in next week, but will be using a different setup for the projects over the next month. We do need the ease of a Dante setup, so the SG300 will get reconfigured with the information from Yamaha, and I'll have my personal D-Link DGS1210 as a backup. Plan C is a MOTU828 to get playback from a Mac Trash Can to the QL1.


----------



## TimMc (Jan 7, 2021)

Teerence - I think the issue is how Dante handles distribution of clock via the network. The switch is not passing clock traffic correctly; this is likely a configuration issue. Clocking is the only thing common to all devices. You should see clock errors in Dante Controller.

Edit PS: If you restore the switch, in Controller make the DVSC your master Dante clock and have the surface/Rio sync to it. Does it work?


----------



## Terence Gray (Jan 7, 2021)

TimMc said:


> Teerence - I think the issue is how Dante handles distribution of clock via the network. The switch is not passing clock traffic correctly; this is likely a configuration issue. Clocking is the only thing common to all devices. You should see clock errors in Dante Controller.
> 
> Edit PS: If you restore the switch, in Controller make the DVSC your master Dante clock and have the surface/Rio sync to it. Does it work?


From material on the Yamaha site about switch setup, I've come to the same conclusion that it's something I did, or didn't do, in creating the VLANs. It's likely to be the setting in the ports connecting the main Dante switches in the rack room to the SG300. That will get looked at when I'm back in the building next week, although the Dante setup for the next recording project will be quite different.

DVSC cannot be the clock master in a Dante network. Via can be, but it's really not recommended. Ideally the master clock is a Dante-enabled device that is always present on the network. In our case, that means one of the components in the AV rack room, because the QL1 and Mac Mini are only on when the space they are intended for is in use. When the network was set up no master clock was designated; I set it as the Rio3224 because it's always on and will be used anytime we're using the QL1. The other options were the RF mic receivers or the 70V amps – everything else that gets connected to the Dante network is used in temporary setups and not always present.

I do have Dante Level 3 certification, but the "IT" side of networking is my weak point.


----------



## Jay Ashworth (Jan 8, 2021)

MNicolai said:


> [...] I'm not wild about static IP's on Dante networks because it adds extra steps when your temporarily adding Dante devices or upgrading firmware that may blow out the IP settings, but it's not a big deal so long as you remember you have to set those statics if the system gets changed around. Dante is much more resilient and self-healing than other AV streaming protocols so the general wisdom of static-for-stability is almost always an unnecessary precaution that can create more problems than it prevents.



No reason you can't combine them, is there?

Put your fixed devices on static IPs, and leave a DHCP server enabled on the subnet carefully set not to hand out those static IPs?


----------



## MNicolai (Jan 8, 2021)

Jay Ashworth said:


> No reason you can't combine them, is there? Put your fixed devices on static IPs, and leave a DHCP server enabled on the subnet carefully set not to hand out those static IPs?



There's no compelling reason to spend any time on statics. You can combine them in a reserved area of the subnet and cordon your DHCP range in the other part of that subnet but in my experience it just creates more problems. Somebody upgrades firmware or factory resets a device and now you have to log into the system with Dante Controller or that manufacturer's special app to set the statics.

Part of the reason statics became popular is because it was the only surefire way to connect to a specific device. So on an AV system with a lot of different devices getting controlled, it was more reliable to use statics because hostnames were unreliable and if the system had a hard reboot and all your DHCP addresses changed, you'd have a control system that couldn't control anything. Dante was built to be idiot proof though and work out of the box without special sauce configurations though -- so IP addresses basically get ignored. Dante subscriptions are all based on device and channel names. If your system is recovering from a hard reset, it's *possible* your system will come back online a little faster with statics than with DHCP -- because DHCP means that the router has to reboot first and do it's thing -- but if you're using autonegotiation without DHCP it'll come back online and transport audio about as fast as if you used statics.

My primary issue with statics has been that people forget about them. I've installed Dante systems with them before. Then I disappear and we turn the system over to the owner. 6 months goes by, they have a problem with a Lake LM-44 processor. Tech support tells them to factory reset. They factory reset, and now they've got 2-3 problems: 1) It was a redundant network and the LM-44's default to daisy-chained -- it kills your entire network -- you have to disconnect them in order to connect so you can configure them for redundant, 2) you have to use Lake's special software to change network settings, 3) you have to reconfigure the IP settings. I've also seen people who wanted to use Yamaha's Stagemix app try and stick some Best Buy WAP bridged between the consoles control port and the Dante networks, and now you could suddenly have a DHCP source you didn't plan on that could cause problems. If it's the _only_ DHCP source on the Dante network and nothing is configured for statics, not so much a problem. If there are other DHCP sources or you have statics on _some_ devices, now you have a problem. This gets even more interesting when someone tries to bridge their Dante/Control network with their ETCNet ports so they can use the StageMix app and RFR app on the same WiFi SSID. Suddenly you're in the middle of a tech rehearsal and everything starts to go haywire.

Other problems have been replacing or adding gear. 3-4 years after it's installed, nobody remembers there are statics and they spend several hours troubleshooting why their new piece of equipment doesn't talk to their existing equipment.

IMO, statics have long been a crutch for control/streaming systems because hostnames were unreliable. Dante was built with bulletproof hostnames and channel subscription naming. In almost all cases, that crutch of using statics has become irrelevant.


----------



## Terence Gray (Jan 8, 2021)

MNicolai said:


> There's no compelling reason to spend any time on statics. You can combine them in a reserved area of the subnet and cordon your DHCP range in the other part of that subnet . . .


Everything in your reply is valid and makes sense, but the Dante network I'm dealing with doesn't have a DHCP server, so everything has to be either static or self-assigning. 

For whatever reason, the designer or installer (more likely) set it up as static. Once I figured that out it was a simple matter of assigning the Mac Mini (in-house) and MacBook Pro (mine) IP addresses higher than the existing addresses. There were same gaps in the addresses shown by Dante Controller, but I avoided using them because there are a number of Dante devices that are rarely connected, and I assume they have been given those IPs. At the moment, the entire Dante network is physically separate from the ETCNet, control net, and – most importantly – the building IT infrastructure. There are designated Primary and Secondary Dante data ports at FOH, and in at least two of the floor pockets at the stage.

Other than within the performance space we're currently using for recording, there's only one point in the building where data ports have been connected to the Dante Primary and Secondary switches in the AV rack room, and those ports have devices connected so are very unlikely to have anyone messing with them. The education and outreach department has a camera setup they share with IT – it's a security camera and a small rack. When E&O has it, the camera records directly to a Denon unit in the rack. IT uses it as the security camera for the performance space, and takes the room mic signal, through Dante, and adds it to the video in a closed-circuit feed. There is a dedicated data port at FOH for its Dante connection. One of my ideas when we added the switch at FOH was that IT could simply plug the camera into the Primary VLAN, freeing up a data line to the AV rack, and a port in the main Dante Primary switch.


----------



## StradivariusBone (Jan 9, 2021)

Dante should be fine without DHCP as well. It's perfectly happy dealing with APIPA addresses. We use static because it makes things easier to find on our network. One of my volunteers is a genius at scripting and we've automated a number of things through their respective APIs, so we just have a list of all our devices to reference and since it's just the two of us doing any of the network nitty gritty stuff, it doesn't matter when gear is swapped because we're the ones doing it. BUt Mike's point is very valid in a setup where the end user is not savvy with IP networks.

We did have a wacky instance using Magewell HDMI to NDI converters where they would not detect NDI sources unless they had DHCP turned off. That's a bug though, not a feature LOL. They just like static IPs better. 

I think your best bet is what Mike's advice was to me. Narrow it down and eliminate things. I would definitely check that switch config since it seems like it's the smoking gun right now, but the VLAN setup you've got could also contribute. The weird part is still that you're having audio signal drop outs with no clock errors showing up in controller. THat points to gear on the transmit or receive side being at fault (like in my problem).


----------



## StradivariusBone (Jan 19, 2021)

Here's another fun one. Haven't done much research into this yet, so forgive me if it's obvious. 

When loading presets from Controller, sometimes it will decide that the Venue 2 board is the Venue 1 and load all of the channel info onto that board, which then makes duplicate Venue 1 boards in Controller. I'd assume that the Controller software would identify the cards by MAC, so I'm a bit confused by this one. Not a party stopper, but a weird one.


----------



## StradivariusBone (Jan 5, 2023)

Reviving a dead thread because we're facing some new wine in an old bottle. 

In both venues with the M32-DL32 setup we are now experiencing loss of sync at random (and relatively seldom) intervals. Both venues have professionally made Ethercons with the chassis shields (which is what fixed the problem before), and both have Dante cards with the console set to pull clock from the Dante card. Despite this, every so often we'll still have a sync drop on the AES50. Sometimes it recovers after a pop, sometimes it requires a power cycle or unplugging/replugging the ethercon fixes it. I'd lean towards the AES50 card being faulty, but the fact that it's happening in two identical setups makes that less likely. 

In troubleshooting, I tried switching back to the old cable we made earlier and found an interesting fault with that. Our main cable has been on the AES B port (we switched to that as part of our troubleshooting). The secondary cable I patched in to AES A and both got sync. But if you watch this video, just about every time our drummer hits the kick the secondary cable (AES A) would lose sync. I thought maybe this is a mechanical thing, but beating on the cable chassis does not cause sync loss and we have experienced the fault when there is no sound whatsoever. 

Here's a video of that happening- https://photos.app.goo.gl/Awv5miwvyG6tHASg9

Another thing we tried was putting every piece of gear on voltage regulating UPS. One other thread I found seemed to think that transient voltage drops would cause a loss of sync and one thing I have noticed is that it likes to pop if I bring gain up on something like a bass input relatively quickly (maybe the sudden amp load drops voltage?). 

But if I had to guess at what is causing it, I'd lean toward the clock sync being pulled from the card and not the AES50 bus making it struggle with the sync on the stagebox side. Audio will also disrupt on the Dante stream when it fails, but pulling clock from the AES50 will cause failures on the Dante side. It's lose lose

I'm at my wits end with this one. I'm planning on barking up the Music Tribe tree but I figured see what the brain trust here might throw at it.


----------



## FMEng (Jan 6, 2023)

Time to go back to basics. Delete the Dante card and see how the mixer and DL32 behaves. If it runs OK, then you know the problem is related to the Dante card or network.

I may have mentioned this before, but I think you are struggling with inadequate switches or improper configuration. Buy switches known to work well with Dante and configure them exactly prescribed. I built a small, two switch Dante and NDI network with a Midas M32. I found a good tutorial on configuring Cisco SG-350 series for Dante, so that's what I bought. The configuration is rather complicated, which tells me a Netgear from Bestbuy doesn't cut it. I did the config and have Dante running alongside six cameras of NDI with no problems. It's not a redundant network.


----------



## StradivariusBone (Jan 6, 2023)

Pulling the card is a good next step. I don't think it's switches at this point since it seems confined to the connection between the console and the stage box. Dante Controller is happy and doesn't report anything wrong when the console loses sync to the stage. In your setup do you have a stage box or are you pulling direct from the console?


----------



## FMEng (Jan 6, 2023)

I don't have a stage box, so that is a difference. The Dante card is the clock master.


----------



## StradivariusBone (Jan 6, 2023)

That's another good data point. I think I'll pull the Dante in one and pull the stage box in the other.


----------

