Untagged customer traffic inconsistently sent over trunk

Question

I have a catalyst 3750 which is giving me a headache.

I have several "customers" who all produce an identical mixture of tagged and untagged traffic. I want to take this traffic in and then pass it out on a trunk with a second 802.1Q tag (the customer or 'metro' tag) added on.

The problem I have is that most of the untagged customer traffic is not being sent out the trunk.

vlan dot1q tag native
vlan 201-224,301-324,401-422

interface GigabitEthernet1/0/1
  switchport access vlan 401
  switchport mode dot1q-tunnel
  switchport trunk native vlan 401 !  This doesn't seem to do anything.
  no keepalive
  no cdp enable

interface GigabitEthernet1/0/2
  switchport access vlan 402
  switchport mode dot1q-tunnel
  no keepalive
  no cdp enable

...


interface GigabitEthernet1/0/23
  !other switches trunk in through here to port 24
  switchport trunk encapsulation dot1q
  switchport trunk allowed vlan 201-224,301-324 
  switchport mode trunk
  switchport nonegotiate
  no keepalive
  no cdp enable

interface GigabitEthernet1/0/24
  switchport trunk encapsulation dot1q
  switchport trunk allowed vlan 201-224,301-324,401-422
  switchport mode trunk
  switchport nonegotiate
  no keepalive
  no cdp enable

Edit1: Setting the native vlan on the access ports does not seem to have any effect.

Edit2: SSH traffic traverses the switch as expected (from trunk to access), but multicast traffic from access to trunk (such as RTPS) does not. HOWEVER, I have now noticed that when the cable on the access port is unplugged and replugged, RTPS traffic is sent across the trunk as expected for about a second before disappearing again. I have used an in-line data tap to confirm that the RTPS traffic is still reaching the switch (ie the RTPS traffic is not being turned off at the source)

Edit3: Oops, multicast traffic from the other switches was being dropped.

What works:

• tagged traffic entering port 1,2,... has a customer vlan tag of 401,402,... added and is sent out the trunk port (24)
• untagged traffic entering port 1,2,... of types ARP and MDNS has customer tags added and is sent out the trunk port (24)
• tagged unicast traffic from other switches (vlan range 201-224,301-324) transits the switch and emerges from the trunk port (24) unmolested

What doesn't work:

• untagged traffic (mostly RTPS but also other stuff) entering port 1,2,... is dropped instead of being tagged and sent out the trunk port (24)
• tagged multicast traffic from the other switches (vlan range 201-224,301-324) is dropped instead of emerging from the trunk port 24.

Side Note: this all works perfectly on my SG-300 (also a cisco switch) but the configuration language/syntax is completely different and sooooo much simpler. "switchport mode customer/switchport customer vlan ###"

Answer 1

It seems that your problem is with Multicast traffic. Switches that default to IGMP snooping have a problem with multicast traffic across multiple switches. Cisco has documents, e.g. Multicast Does Not Work in the Same VLAN in Catalyst Switches, which explain the problem and solutions.

The problem is basically that switches with IGMP snooping will listen to IGMP messages in order to learn which switch interfaces are requesting traffic for which IGMP groups. If a switch doesn't have any hosts connected, it will simply drop multicast traffic. The IGMP requests will not span across switch trunks, except in specific scenarios.

Problem

Multicast traffic does not seem to pass across Catalyst switches, even in the same VLAN. Figure 1 depicts a typical scenario:

Figure 1 – Network Setup with Multicast Source and Receivers

The multicast source is connected to Switch 1, which is a Catalyst 6500 Switch with Supervisor Engine 720 that runs Cisco IOS Software. Receiver 1 is connected to Switch 1, and Receiver 2 is connected to Switch 2. Switch 2 is a Catalyst 3750. There is a Layer 2 link, either access or trunk, between Switch 1 and Switch 2.

In this setup, you find that Receiver 1, which is on the same switch as the source, gets the multicast stream with no problems. However, Receiver 2 does not get any multicast traffic. This document aims to resolve this issue.

An explanation of the problem with a link to the solutions:

Understand the Problem and Its Solutions

By default, the Catalyst switches have IGMP snooping enabled. With IGMP snooping, the switch snoops (or listens) for IGMP messages on all the ports. The switch builds an IGMP snooping table that basically maps a multicast group to all the switch ports that have requested it.

Assume that, without any prior configuration, Receiver 1 and Receiver 2 have signaled their intentions to receive a multicast stream for 239.239.239.239 that maps to the L2 multicast MAC address of 01.00.5e.6f.ef.ef. Both Switch 1 and Switch 2 create an entry in their snooping tables for these receivers in response to the IGMP reports that the receivers generate. Switch 1 enters port Gigabit Ethernet 2/48 in its table, and Switch 2 enters port Fast Ethernet 1/0/47 in its table.

Note: At this point, the multicast source has not started its traffic, and none of the switches knows about the switch mrouter port.

When the source on Switch 1 starts to stream multicast traffic, Switch 1 has "seen" the IGMP report from Receiver 1. As a result, Switch 1 delivers the multicast out port Gigabit Ethernet 2/48. But, since Switch 2 "absorbed" the IGMP report from Receiver 2 as part of the IGMP snooping process, Switch 1 does not see an IGMP report (multicast request) on port Gigabit Ethernet 2/46. As a result, Switch 1 does not send any multicast traffic out to Switch 2. Therefore, Receiver 2 never gets any multicast traffic, even though Receiver 2 is in the same VLAN but merely on a different switch than the multicast source.

The reason for this issue is that IGMP snooping is not really supported on any Catalyst platform without an mrouter. The mechanism "breaks down" in the absence of an mrouter port. If you want a fix for this solution, you must have the switches somehow learn or know of an mrouter port. The Solutions section of this document explains the procedure. But how does the presence of an mrouter port on the switches remedy the situation?

Basically, when the switches learn or statically know about an mrouter port, two critical things occur:

• The switch "relays" the IGMP reports from the receivers to the mrouter port, which means that the IGMP reports go toward the multicast router. The switch does not relay all the IGMP reports. Instead, the switch sends only a few of the reports to the mrouter. For the purpose of this discussion, the number of reports is not important. The multicast router only needs to know if there is at least one receiver that is still interested in the multicast downstream. In order to make the determination, the multicast router receives the periodic IGMP reports in response to its IGMP queries.
• In a source-only multicast scenario, in which no receivers have yet "joined" in, the switch only sends the multicast stream out its mrouter port.

When the switches know their mrouter port, Switch 2 relays out the IGMP report that the switch received from Receiver 2 to its mrouter port. This port is Fast Ethernet 1/0/33. Switch 1 gets this IGMP report on the switch port Gigabit Ethernet 2/46. From the perspective of Switch 1, the switch has received merely another IGMP report. The switch adds that port into its IGMP snooping table and begins to send out the multicast traffic on that port as well. At this point, both the receivers receive the requested multicast traffic, and the application works as expected.

But how do the switches identify their mrouter port so that IGMP snooping works as it is expected to work in a simple environment like this? The Solutions section provides some answers.

Cisco lists five different possible solutions to the problem:

1. Enable PIM on the Layer 3 Router/VLAN Interface
2. Enable IGMP Querier Feature on a Layer 2 Catalyst Switch
3. Configure Static Mrouter Port on the Switch
4. Configure Static Multicast MAC Entries on All the Switches
5. Disable IGMP Snooping on All the Switches

Answer 2

You have not set a native VLAN on the interfaces. This would be stopping untagged traffic from traversing the trunk.

See below links for clarification;

http://www.cisco.com/c/m/en_us/techdoc/dc/reference/cli/nxos/commands/l2/switchport-trunk-native-vlan.html

http://www.cisco.com/c/m/en_us/techdoc/dc/reference/cli/nxos/commands/l2/vlan-dot1q-tag-native.html

Answer 3

Ron Maupin's comment was exactly on-target. It also made me realize that multicast traffic from the other switches was NOT traversing this switch as I had previously thought.

no igmp snooping

Adding this in the global config made everything work the way I wanted it to, although depending on your volume of multicast traffic you may want to enable it for selected vlans and ports only. I recommend reading http://www.cisco.com/c/en/us/support/docs/switches/catalyst-6500-series-switches/68131-cat-multicast-prob.html for a better understanding of the options.

Thanks everyone!