7

I should get this figured out by my own, but actually I seem to be a blockedhead.

I have two stacks of Cisco 3750X with two switches in each stack. One Stack is in building A, the other one in Building B. Each stack acts as core switch for the building and as a router for the local traffic. Also, there are some VLANs shared between the two stacks with no routing.

The stacks are connected with a dark fiber and 1Gbps single mode SFP. This is working like a charm.

Now, we want to install a second uplink between the two stacks for backup. Unfortunately, we could only get a 100 Mbit (STM-100) leased ethernet line between the buildings, so no portchannel/etherchannel is possible.

Then I thought: Just configure a second port with same settings (trunk) as the first uplink port and plug in the line. Spanning-Tree will disable it because of higher cost and enable it in case the main uplink (Gbit fiber) fails. And there is the problem:

Spanning-Tree (RSTP) has disabled the "new" port, but detected it as "Backup" and made it discard all packets and don't enables it when the main uplinks fails.

The question is: How can I make the stacks enable the backup line in case of a main line failure OR if this is not possible, what would be a better approach to reach the same goal.

Here is a simple drawing to visualize the problem:

##

The relevant parts of the running config:

Stack in Building A:

interface GigabitEthernet1/0/28
 description downlink trunk BldB
 switchport access vlan 666
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 666
 switchport trunk allowed vlan 2-99,101-199,201-4094
 switchport mode trunk
 switchport nonegotiate
 speed nonegotiate
end


interface GigabitEthernet2/0/28
 description "downlink trunk BldgB 100M-Backup"
 switchport access vlan 666
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 666
 switchport trunk allowed vlan 2-99,101-109,111-4094
 switchport mode trunk
 switchport nonegotiate
 shutdown
 speed 100
 duplex full
 spanning-tree vlan 99 port-priority 128
 spanning-tree vlan 99 cost 50
end

Stack in Building B:

interface GigabitEthernet1/0/1
 description downlink trunk BldgA
 switchport access vlan 666
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 666
 switchport trunk allowed vlan 2-99,101-199,201-4094
 switchport mode trunk
 switchport nonegotiate
 speed nonegotiate
 duplex full
end

interface GigabitEthernet2/0/1
 description "downlink trunk BldgA 100Mbit-Backup"
 switchport access vlan 666
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 666
 switchport trunk allowed vlan 2-99,101-109,111-4094
 switchport mode trunk
 switchport nonegotiate
 speed 100
 duplex full
end

EDIT: Added spanning tree information:

bldga-core# show spanning-tree summary
Switch is in rapid-pvst mode
Root bridge for: VLAN0099, VLAN0101-VLAN0117, VLAN0200
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default  is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
EtherChannel misconfig guard is enabled
UplinkFast                   is disabled
Stack port is StackPort1
BackboneFast                 is disabled
Configured Pathcost method used is short


bldgb-core# show spanning-tree summary
Switch is in rapid-pvst mode
Root bridge for: VLAN0100, VLAN0110, VLAN0118, VLAN0200
EtherChannel misconfig guard is enabled
Extended system ID           is enabled
Portfast Default             is disabled
PortFast BPDU Guard Default  is disabled
Portfast BPDU Filter Default is disabled
Loopguard Default            is disabled
UplinkFast                   is disabled
Stack port is StackPort1
BackboneFast                 is disabled
Configured Pathcost method used is short


bldga-core# show spanning-tree vlan 99

VLAN0099
  Spanning tree enabled protocol rstp
  Root ID    Priority    99
             Address     081f.f31e.9c80
             This bridge is the root
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    99     (priority 0 sys-id-ext 99)
             Address     081f.f31e.9c80
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Po14                Desg FWD 3         128.592  P2p
Po15                Desg FWD 3         128.600  P2p
Po17                Desg FWD 3         128.616  P2p
Po19                Desg FWD 3         128.632  P2p
Po22                Desg FWD 3         128.656  P2p
Po23                Desg FWD 3         128.664  P2p
Po24                Desg FWD 3         128.672  P2p
Po28                Desg FWD 4         128.704  P2p

bldgb-core#show spanning-tree vlan 99

VLAN0099
  Spanning tree enabled protocol rstp
  Root ID    Priority    99
             Address     081f.f31e.9c80
             Cost        4
             Port        616 (Port-channel14)
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    4195   (priority 4096 sys-id-ext 99)
             Address     d072.dc9b.4980
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec
             Aging Time  300 sec

Interface           Role Sts Cost      Prio.Nbr Type
------------------- ---- --- --------- -------- --------------------------------
Po14                Root FWD 4         128.616  P2p
Po15                Desg FWD 4         128.624  P2p
Po16                Desg FWD 4         128.632  P2p
Po17                Desg FWD 4         128.640  P2p
Po18                Desg FWD 4         128.648  P2p
Po19                Desg FWD 4         128.656  P2p
Po20                Desg FWD 4         128.664  P2p
Po21                Desg FWD 4         128.672  P2p
Po22                Desg FWD 4         128.680  P2p
Po23                Desg FWD 4         128.688  P2p
Po25                Desg FWD 4         128.704  P2p

bldga-core#show interfaces gi2/0/28
GigabitEthernet2/0/28 is administratively down, line protocol is down (disabled)
  Hardware is Gigabit Ethernet, address is 081f.f31e.c39c (bia 081f.f31e.c39c)
  MTU 1500 bytes, BW 100000 Kbit, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 100Mb/s, link type is force-up, media type is 10/100/1000BaseTX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 20:35:08, output 20:34:56, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 6039
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     964013646 packets input, 740488440354 bytes, 0 no buffer
     Received 3529649 broadcasts (2477004 multicasts)
     0 runts, 0 giants, 0 throttles
     12 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 2477004 multicast, 0 pause input
     0 input packets with dribble condition detected
     975352039 packets output, 923509356637 bytes, 0 underruns
     0 output errors, 0 collisions, 9 interface resets
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 PAUSE output
     0 output buffer failures, 0 output buffers swapped out


bldgb-core#show interfaces gigabitEthernet 2/0/1
GigabitEthernet2/0/1 is down, line protocol is down (notconnect)
  Hardware is Gigabit Ethernet, address is 74a0.2f7e.3001 (bia 74a0.2f7e.3001)
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive not set
  Full-duplex, 100Mb/s, link type is auto, media type is 10/100/1000BaseTX SFP
  input flow-control is off, output flow-control is unsupported
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input never, output 20:25:27, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 346
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     0 packets input, 0 bytes, 0 no buffer
     Received 0 broadcasts (0 multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog, 0 multicast, 0 pause input
     0 input packets with dribble condition detected
     160167 packets output, 19128660 bytes, 0 underruns
     0 output errors, 0 collisions, 2 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier, 0 pause output
     0 output buffer failures, 0 output buffers swapped out

bldga-core#show etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      f - failed to allocate aggregator

        M - not in use, minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port


Number of channel-groups in use: 15
Number of aggregators:           15

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SD)          -
14     Po14(SU)         -        Gi1/0/14(P) Gi2/0/8(P)
15     Po15(SU)         -        Gi1/0/15(P) Gi2/0/9(P)
16     Po16(SD)         -        Gi2/0/16(D)
17     Po17(SU)         -        Gi1/0/17(P) Gi2/0/17(P)
18     Po18(SD)         -
19     Po19(SU)         -        Gi1/0/22(P) Gi2/0/19(P)
20     Po20(SD)         -
21     Po21(SD)         -
22     Po22(SU)         -        Gi1/0/9(P)  Gi2/0/10(P) Gi2/0/14(P)
                                 Gi2/0/15(P)
23     Po23(SU)         -        Gi1/0/10(P) Gi1/0/18(P) Gi2/0/11(P)
                                 Gi2/0/18(P)
24     Po24(SU)         -        Gi2/0/5(P)  Gi2/0/6(P)
27     Po27(SD)         -        Gi2/0/23(D)
28     Po28(SU)         -        Gi1/0/28(P)
29     Po29(SD)         -

Thanks for any help!

Improve this question
4
  • interface GigabitEthernet2/0/28 is shutdown?
    – richardb
    Apr 29, 2015 at 13:57
  • Only at the moment, of course it is enabled in regular use. But I was trying something yesterday night and shut it down for this
    – Sebastian
    Apr 29, 2015 at 14:00
  • I'm having trouble understanding the relationship between the port channels and the interfaces. Can you post show etherchannel summary?
    – Ron Trunk
    Apr 30, 2015 at 1:08
  • It is simple: Usually, every Gigabit interface is also in one port channel. They have the exact same config each (switchport mode, vlans, negotiation, etc.). This is a relict of the past. But some interfaces are bundled with port-channel, of course, for LACP. Etherchannel summary is above :)
    – Sebastian
    Apr 30, 2015 at 7:54

3 Answers 3

Reset to default
3

First of all you may want to remove the unnecessary spanning-tree configuration on your secondary uplink. STP calculates the cost metric based on the link speed. Your 100Mbit/sec CU uplink has a default cost of 19 and your primary 1Gbit/sec SM Fibre uplink got a cost of 4, so no need to set it manually.

You may also remove the "access vlan configuration" since your trunk configuration with native vlan 666 will be preferred. Also change your allowed vlan configuration to match the primary link. I do not know which vlans you actually use, but inconsistency may become a big pain in case your primary link fails.

 interface GigabitEthernet2/0/28
 description "downlink trunk BldgB 100M-Backup"
 switchport access vlan 666              -> unnecessary
 switchport trunk encapsulation dot1q
 switchport trunk native vlan 666
 switchport trunk allowed vlan 2-99,101-109,111-4094
 switchport mode trunk
 switchport nonegotiate
 shutdown
 speed 100
 duplex full
 spanning-tree vlan 99 port-priority 128 -> unnecessary
 spanning-tree vlan 99 cost 50           -> unnecessary
end

An interesting counter from Gi1/0/28 (Site A) are the input errors on the interface. Might be a layer 1 issue but since you do not have any crc errors and the realibility is 255/255 it may be due to repluging cables.

Since there are no obvious configuration issues I would recommand re-enabling the port and posting the output of:

Site A:

show spanning-tree interface gi1/0/28
show spanning-tree detail

Site B: 

show spanning-tree interface gi2/0/1
show spanning-tree detail

Last but not least could you provide the following information:

  • Which IOS release are you running on the stacks?
  • Does your Service Provider provide transparent L2 services (BPDUs received on both sides? ... you may check if CDP packets are received on both sites via show cdp neighbors)
Improve this answer
1
  • Thank you for your answer. I would love to try this and post the details you requested, but unfortunately I don't have access to this gear anymore. Just to let you know: In a GNS3 - Simulation this all worked as you would expect it. I still don't know what the problem was with the real world implementation but beeing not 100% sure the backup link was absolutely transparent, my best guest is the provider of the backup link (German Telekom) was blocking (at least some of) the BPDUs.
    – Sebastian
    Dec 29, 2015 at 11:32
0

Spanning-Tree (RSTP) has disabled the "new" port, but detected it as "Backup" and made it discard all packets and don't enables it when the main uplinks fails.

STP should enable it when primary fails. It's normal behaviuor of STP. Maybe you didn't enabled Rapid STP, and switching to backup takes about minute (50s).

Run "show spanning-tree summary" command on both switches and on first line it will show operational mode. It should be "Switch is in rapid-pvst mode". If no - than do "spanning-tree mode rapid" on both stacks.

Improve this answer
3
  • Thanks for your help. Switch is in r-pvst+ mode, and I have the main uplinke "failed" (I unplugged it) for about half an hour but the backup link didn't came up. The port is in rstp type "Backup" and state "Discarding"
    – Sebastian
    Apr 29, 2015 at 13:01
  • paste here output of "show spanning-tree vlan 2" from both switches in both cases, if you please. And "show interface" of interfaces involved
    – Karroplan
    Apr 29, 2015 at 13:10
  • VLAN 2 is not in use, instead I added the information for/from one other vlan and the requested interface output in the original posting above
    – Sebastian
    Apr 29, 2015 at 14:08
-1

STP could work but I would probably use EEM and script up a custom solution. You should be able to beat STP in terms of failover and failback speed.

Maybe use IP Device Tracking to detect the switch peer across the primary link and upon failure detection admin up the backup link. Maybe you could kick start the re-routing of frames by scripting in the MAC address of the remote backup link to the CAM table?

I don't have a lab so... ...just an idea.

Improve this answer
5
  • Thanks for your answer, and normally you'd be right. But this copper link is a special active STM connection provided by our carrier. The Link itself is good, if I connect it only, all is perfectly fine. The problem only appears when I use it as the backup connection.
    – Sebastian
    Jun 13, 2015 at 6:14
  • Seriously?! Did you just recommend datacenter technologies instead of helping him solve a simple stp issue? What should he do, buy Nexus gear for a campus and configure FabricPath...
    – kaisero
    Dec 26, 2015 at 22:39
  • He should configure cross stack etherchannel or cross chassis etherchannel or VSS paired with one of those. In any case use an MLAG technology and Etherchannel. STP should be avoided.
    – Ronnie Royston
    Dec 26, 2015 at 22:43
  • He only got VPLS services with 100Mbit/sec as secondary link. Recommanding MLAG considering his circumstances and the hardware in place is plain wrong, STP may not be the sexiest solution but it gets the job done. Just imagine he would have 2x 1Gbit/sec and the ISP got issues... Since the hardware does not support fast bpdus it will take longer to reconverge than spanning-tree...
    – kaisero
    Dec 26, 2015 at 23:15
  • I think the original question must have been modified. Upon rereading it, an Etherchannel not work as the links must be the same speed. I will need to update my answer...
    – Ronnie Royston
    Dec 27, 2015 at 1:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.