4

I've been brought in to troubleshoot an issue on an install performed by a different engineer.

The basic topology is:

1x Dell N2000 as Core Switch 1x Dell N2000 as Access Switch 01 1x Dell N2000 as Access Switch 02

Access switches trunk with a single 1gb interface to the core (Gi1/0/24 on both access switches) so there are no switching loops or anything like that so far as I can tell.

Essentially, the topology looks like this:

(ACSW01)Gi1/0/24 <-> Gi1/0/22 (CORE) Gi1/0/21 <-> Gi1/0/24(ACSW02)

Nothing fancy - The VLANs are:

VLAN 47 - 192.168.47.0/24 VLAN 48 - 192.168.48.0/24 VLAN 49 - 192.168.49.0/24 VLAN 50 - 192.168.50.0/24

At this point, clients on any VLAN get internet connectivity, but clients on VLAN 49 or VLAN 50 cannot ping a printer on VLAN48 located on Edge Switch 01 (ESW01) - but if the printer's connection is physically moved to the core switch, they can. The problem follows the VLANs on edge switches, not the devices; any device connected to an edge switch that is NOT on VLAN 48 will not be able to communicate with any other VLAN - unless the device is on the core switch. Additionally, those clients experience no issues with internet connectivity (and that works fine across all VLANs too).

Below are my troubleshooting steps taken thusfar: 

1: Verify configuration integrity
    Notice that "ip routing" is on both edge switches - issue "no ip routing" command
    Ensure that "ip default gateway 192.168.48.254" is set - OK
        When setting IP default gateway, automatically switch creates an entry with "ip route 0.0.0.0 0.0.0.0 192.168.48.254" 
    Verify trunk tagging for Edge - Core - Edge:
        Appears OK - 48 native, 47,49,50 tagged
    Verify Aerohive trunk encapsulation
        Untagged 47, Tagged 48, 49, 50

2: Verify Layer 2 connectivity path
    Examine Spanning-Tree
        CSW01 reports it is NOT the root bridge
            Root bridge is ESW01 
        Changing CSW01 bridge priority to 8192 - "spanning-tree priority 8192"
            Root bridge is now CSW01 
    Checking MAC address table
        Printer on VLAN 48 on ESW01 has MAC w/last 4 D53C 
            CSW01: D53C appears on Gi1/0/22 (correct trunk link to ESW01) on VLAN 48
        My laptop on VLAN 49 on Aerohive (connected to CSW01) w/last 4 MAC A560
            ESW01: "show mac address-table | inc A560" - shows on Gi1/0/24 leading to Core on VLAN 49 - correct trunk link

3: Verify Layer 3 connectivity path
    Ping 192.168.48.225 (Printer, D53C) 
        FAILS from laptop on wireless VLAN 49 connected to CSW01
        FAILS from Aerohive on VLAN 47 connected to ESW01 
        WORKS from CSW01 command line
        WORKS from ESW01 command line
        WORKS from ESW02 command line 
        FAILS from laptop on wired VLAN 49 connected to CSW01 
        FAILS from laptop on wired VLAN 49 connected to ESW01

    Ping 192.168.48.1 (Sonicwall)
        WORKS from Aerohive connected to ESW01
        WORKS from Core Switch
        WORKS from Laptop on VLAN 49 connected to Core Switch
        WORKS from ESW01 command line

Below are the configs (some output suppressed for redundancy / brevity) with relevant portins:

CORE SWITCH: 

    AS-CSW01#show run

!Current Configuration:
!System Description "Dell Networking N2024P, 6.0.1.3, Linux 3.6.5-320b2282"
!System Software Version 6.0.1.3
!
configure
vlan 47
name "Aerohive Management"
vlan association subnet 192.168.47.0 255.255.255.0
exit
vlan 48
name "Client Company Wired Production"
vlan association subnet 192.168.48.0 255.255.255.0
exit
vlan 49
name "Client Company Wireless Production"
vlan association subnet 192.168.49.0 255.255.255.0
exit
vlan 50
name "Client Company Wireless Guest"
vlan association subnet 192.168.50.0 255.255.255.0
exit
vlan 47-50
exit              
hostname "AS-CSW01"
slot 1/0 8    ! Dell Networking N2024P
slot 2/0 5    ! Dell Networking N2048
stack
member 1 7    ! N2024P
member 2 8    ! N2048
exit
ip routing
ip route 0.0.0.0 0.0.0.0 192.168.48.1
interface vlan 1
exit
interface vlan 47
ip address 192.168.47.254 255.255.255.0
ip helper-address 192.168.48.31
exit
interface vlan 48
ip address 192.168.48.254 255.255.255.0
exit
interface vlan 49
ip address 192.168.49.254 255.255.255.0
ip helper-address 192.168.48.31
exit
interface vlan 50 
ip address 192.168.50.254 255.255.255.0
ip helper-address 192.168.48.31
exit
username ´(output omitted)
ip ssh server
!
interface Gi1/0/1
description "AS-APFL1-01" (Aerohive AP trunk port)
switchport mode general
switchport general pvid 47
switchport general allowed vlan add 47
switchport general allowed vlan add 48-50 tagged
switchport general allowed vlan remove 1
exit
!
(Output omitted for brevity)
!
interface Gi1/0/18 (A regular access port)
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan remove 1
exit
!
!
interface Gi1/0/21
description "AS-ESW02"
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan add 47,49-50 tagged
switchport general allowed vlan remove 1
switchport trunk native vlan 48
exit
!
interface Gi1/0/22
description "AS-ESW01"
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan add 47,49-50 tagged
switchport general allowed vlan remove 1
switchport trunk native vlan 48
exit
!
interface Gi1/0/23
description "SonicWall X0"
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan add 47,49-50 tagged
switchport general allowed vlan remove 1
exit
!
interface Gi1/0/24
description "Uplink to NetGear"
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan add 47,49-50 tagged
switchport general allowed vlan remove 1
exit
!
exit
snmp-server engineid local 800002a203f8b1564d9fb4
snmp-server community "public" ro
exit

Access Switch 01:

AS-ESW01#show run

!Current Configuration:
!System Description "Dell Networking N2024P, 6.0.1.3, Linux 3.6.5-320b2282"
!System Software Version 6.0.1.3
!
configure
vlan 47
name "Aerohive Management"
vlan association subnet 192.168.47.0 255.255.255.0
exit
vlan 48
name "Client Company Wired Production"
vlan association subnet 192.168.48.0 255.255.255.0
exit
vlan 49
name "Client Company Wireless Production"
vlan association subnet 192.168.49.0 255.255.255.0
exit
vlan 50
name "Client Company Wireless Guest"
vlan association subnet 192.168.50.0 255.255.255.0
exit
vlan 47-50
exit              
hostname "AS-ESW01"
slot 1/0 8    ! Dell Networking N2024P
slot 2/0 5    ! Dell Networking N2048
stack
member 1 7    ! N2024P
member 2 8    ! N2048
exit
ip default-gateway 192.168.48.254
ip route 0.0.0.0 0.0.0.0 192.168.48.254 253
interface vlan 1
exit
interface vlan 47
ip address 192.168.47.253 255.255.255.0
exit
interface vlan 48
ip address 192.168.48.253 255.255.255.0
exit
interface vlan 49
ip address 192.168.49.253 255.255.255.0
exit
interface vlan 50
ip address 192.168.50.253 255.255.255.0
exit
username (output omitted)
ip ssh server
!
interface Gi1/0/1
description "AS-APFL3-01"
switchport mode general
switchport general pvid 47
switchport general allowed vlan add 47
switchport general allowed vlan add 48-50 tagged
switchport general allowed vlan remove 1
exit
!
(Output omitted for brevity)
!
interface Gi1/0/24
description "Trunk to Suite 128 (AS-CSW01)"
switchport mode general
switchport general pvid 48
switchport general allowed vlan add 48
switchport general allowed vlan add 47,49-50 tagged
switchport general allowed vlan remove 1
switchport trunk native vlan 48
exit              
!
exit
snmp-server engineid local 800002a203f8b1564da008
snmp-server community "public" ro
exit

I attempted to replicate this problem in my home lab (albeit on Cisco equipment) and was unable to: inter-VLAN routing worked fine on that setup. At this point, I'm kinda stumped. Dell tech support wanted us to create a separate VLAN for each switch, and static-route between them. No thanks.

All thoughts / input appreciated!

Improve this question
4
  • NOTE: It is not my choice to have VLAN 48 as the trunk native VLAN - I'd like to correct that, but I have to schedule a service outage to do so... so not today. :(
    – Panther Modern
    Oct 9, 2014 at 23:33
  • To add: I am beginning to suspect a frame tagging issue, but am not wholly certain either way.
    – Panther Modern
    Oct 9, 2014 at 23:34
  • A colleague pointed out the following command: "vlan associate subnet" under the VLAN configurations - this appears to be a Layer 3 inspection way to assign VLAN tags to frames by looking at source-IP in the encapsulated packets.
    – Panther Modern
    Oct 10, 2014 at 0:05
  • 1
    I am beginning to suspect that the edge switches are re-writing VLAN tags on return traffic from the core.
    – Panther Modern
    Oct 10, 2014 at 0:05

1 Answer 1

Reset to default
2

Solution:

The commands on VLANs on the access switches for VLAN association with subnet are the root cause of the issue. 

vlan associate subnet 192.168.48.0 255.255.255.0

This command tells the switch to associate traffic with the appropriate VLAN by source-IP address in packet headers.

What was happening is that the return traffic from a client would be dumped into the wrong VLAN on the access switch - the VLAN associated with its source IP, different than that of the client.

Removal of these commands on the access switches resulted in immediate desired connectivity across all VLANs.

EDITED to add: Credit for this fix goes to a fellow networking professional who pointed out their function and voiced his concern.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.