Force10 port channel to server that also supports PXE boot

Question

I have an 802.3ad port channel comprised of 2 10GigE links from a Force10 E1200 switch to a Linux server with one dual-ported 10G NIC. This setup works great when Linux speaks LACP and brings up a bonded interface. However, when the PXE option ROM in the Linux NICs runs, before Linux is in play, it doesn't work. That is, as I have things configured, the PXE option ROM isn't able to send or receive any packets on the expected native VLAN, when the switch ports are both set to port channel mode.

What I'd like is a way for the Force10 switch port to fall back to a normal switchport, when LACP hasn't established a portchannel, so that PXE will still work.

How can I do this? Hopefully with just some additional switch config?

I'm aware of work-arounds for this like PXE-booting off a different non-portchannel interface, etc. I'm really constrained by the hardware I have and looking for a solution that involves only the switch I have, along with the two 10G NIC ports. Further, I'm aware of other Linux bonding types that don't require a port channel on the switch side (e.g., balance-rr, active-backup). I've tested these, and they do work both in Linux and with PXE, but what I'm really after is the single 20Gb/s link and a port channel on the switch side.

The Force10 switch is configured like:

interface TenGigabitEthernet 7/32
 no ip address
!  
 port-channel-protocol LACP 
  port-channel 99 mode active 
 no shutdown

and:

interface TenGigabitEthernet 8/32
 no ip address
!  
 port-channel-protocol LACP 
  port-channel 99 mode active 
 no shutdown

and:

#show int switch Po99

Codes:  U - Untagged, T - Tagged
        x - Dot1x untagged, X - Dot1x tagged
        G - GVRP tagged, M - Trunk

Name: Port-channel 99
...
802.1QTagged: Hybrid
Vlan membership:
Q       Vlans
U       123

Native VlanId:    123.

The Linux server has:

# cat /etc/sysconfig/network-scripts/ifcfg-bond0 
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPADDR=...
GATEWAY=...
PREFIX=...
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=802.3ad miimon=100"
DNS1=A.B.C.D
DOMAIN=bar.com foo.bar.com

and:

# cat /etc/sysconfig/network-scripts/ifcfg-enp6s0f0 
TYPE=Ethernet
BOOTPROTO=none
DEVICE=enp6s0f0
NAME=enp6s0f0
ONBOOT=yes
MASTER=bond0
SLAVE=yes

and:

# cat /etc/sysconfig/network-scripts/ifcfg-enp6s0f1
TYPE=Ethernet
BOOTPROTO=none
DEVICE=enp6s0f1
NAME=enp6s0f1
ONBOOT=yes
MASTER=bond0
SLAVE=yes

Answer 1

@Matthew you are on the right way. I have faced this exact problem in the past and I have used this command in S4810 devices.

Unfortunately, I suspect this command is not supported in all Force10 Model. I just tested it in S60 and S4810 and it's working. You should check your model/version cli document.

In fact, according to E-Series CLI, this command is not supported. Sorry about that.

Anyway, a possible workaround would be use a third standadlone interface for PXE provisioning and then use port-channel for normal operations.

Answer 2

Another option would be to use an LACP-capable PXE Driver. Such PXE driver AFAIK, do not really support port grouping in that sense. It simply "Speak" LACP PDUs in response to an Active-mode LACP configuration in switch (and only on the NIC-port which performs the PXE) This is enough to convince the switch to move that port into Membership in the LAG and start trafficking Data. The trick however, is to incorporate this LACP-enhanced PXE code into your currently installed NIC. Fortunately, it is not such a complicated operation. visit www.iPXE.org to learn how to do that. iPXE ROM supports responding to LACP negotiation PDUs and should perform well against Active Mode LACP switch configuration. Such solution would redundant the need for LACP-Suspend-to-Individual kind of configuration in the switch. Any switch.