1

I have a client that someone in the past setup two NICs on the server to be active on vSwitch0. vSwitch0 has load balancing set for Route based on originating virtual port. Switches are D-Link DGS-3120 series (their enterprise? grade switches). Switches are not stacked, but instead linked through Ethernet ports. Spanning Tree is disabled.

The performance of the guest servers is a little off - When I remotely connect to them for remote control, I deal with pauses up to 45 seconds and disconnects (using an RMM tool). I can connect to physical machines at this client site without issues.

End users report issues also - They lost connectivity to their financial application quite frequently during the day.

I moved one guest server over to a new VMware server with a single NIC and it's performing much nicer. The older server has plenty of compute and RAM, so I'm wondering if there are some MAC table issues on the physical switches due to the load balancing. I was concerned with Spanning Tree until I read https://kb.vmware.com/s/article/2047822 where it states:

Note: VMware vSwitches (Standard and Distributed) cannot form loops as there is no way to join two virtual switches together at layer 2 of the OSI layer. As such, no Spanning Tree Protocol functionality has been incorporated into the virtual switches.

I tried to display the config at the CLI but the switches were not giving it up. sh tech-support worked, dumped out to a log. A huge one. Looked through it and there is nothing Cisco/Dell/HP like with sections pertaining to each port configuration beyond:

 MAC Base information : dev_num = 1, phy port = 19, medium = copper

  ====================================

 State:Enable

 Speed:1000

 Auto negotiation:Enable

 Duplex:FULL Duplex

 Mdix:fiber

 Flow control:Disable

  ====================================

  Dump normal register vale: 
  ...

I did verify there is no load balancing setup on the switches.

I'm kind of at a loss as to why there is a network I/O performance difference between servers with the one with a single connected NIC outperforming the one with two connected NICs.

closed as too broad by Ron Maupin Dec 9 '18 at 21:42

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • "I'm wondering if there are some ARP table issues on the physical switches due to the load balancing." Switch use MAC address table, not ARP tables. ARP relate layer-2 (MAC) to alyer-3 (IPv4) addresses, but switches are layer-2 devices. Switches have MAC address table to relate a MAC address and a switch interface where the source of the MAC address was seen. What happens inside the host is off-topic here, but you have not provided the switch configurations, nor how they are connected to each other. Speculation and guessing are off-topic here. – Ron Maupin Nov 19 '18 at 19:51
  • You can refer to the Network Engineering Question Checklist for guidance, then edit your question to include the necessary information. – Ron Maupin Nov 19 '18 at 19:52
  • "Switches are not stacked, but instead linked through Ethernet ports" from above. So connected from an ethernet port to an ethernet port. – Dacid Salin Nov 19 '18 at 19:57
  • I'll have to remote in and dig around to find the switchports and their config. Not sure how to dump a config at CLI on a D-Link but I will figure it out and update this. These switches are L3 switches. But guess that doesn't matter? – Dacid Salin Nov 19 '18 at 19:58
  • Right, but we have no idea of the switch configurations or which interfaces are connected, etc. Load balancing on switches is normally confined to a single switch, so how do you have it configured on multiple switches. You should copy the switch configurations from your terminal application and paste them into the question using the Preformatted-text option ({}). – Ron Maupin Nov 19 '18 at 19:59
3

ESXi's route based on originating virtual port ID is a good way for a somewhat even load distribution with a larger number of VMs - the virtual port ID is more or less a random value, associating a VM with one of the physical NIC semi-permanently (I think the ID changes on host migration). Do not configure anything special on the uplink switch(es), especially no static LAG trunking.

Unless you configure otherwise, the standard NIC teaming will provide automatic failover - in case a link fails all VMs on that link fall back to the remaining link(s). Switches should be notified (Yes) to update their MAC tables.

If you've got only a few running VMs and less optimal 'random' bandwidth distribution you can alternatively set up multiple port groups each with a dedicated NIC (and only failover to the other NICs) and manually connect the VMs to the one of the groups.

All cases where I've seen performance problems, address flapping and such, were due to physical switch misconfigurations (read: LAG). Check the logs for any flapping and the MAC tables for up-to-date associations. Double check the host links and the physical switch interconnect for link problems. Also check the vSwitch and the port group for any 'funny' setup like bandwidth throttling. If unsure, just create a new port group with your test settings and re-attach a VM to it.

There's no reason to keep RSTP/MSTP disabled on the physical switches though. The vSwitch doesn't participate in it, but doesn't need to since it can't ever cause a loop. STP would protect you from a loop on the physical ports though.

ARP shouldn't be a problem here. ARP associates an IP address with a MAC address - these associations don't change unless you've got some L3 load-balancing or failover setup on the VM side (and it's not working correctly).

If you're not sure how the switches are set up you can dump their configs (show config) and add them (sanitized) to your question.

Edit

It's also possible that the host NIC ports are messed up. Make sure they're set to "Auto Negotiate" and show up as expected (e.g. 1000 Mb, Full Duplex). On the D-Link switches, check the port status for the same and the port error counters to rule out any cable errors. Also, check for funny jumbo frame size settings (physical switches, vSwitches, VMs, even clients).

  • Thanks for the info Zac. With no LAG and being D-Link being so non-standard, I was kind of at a loss. The load distribution is pretty random and there are only 6 guest servers on this host, so no idea why performance was such an issue. New host with single link it's okay. Since it won't hurt, I'ma gonna try Route Based on Physical NIC Load and hope it gets better. Seems like the best option among the four choices. IP Hash won't work because I can't do EtherChannel on the switches. Source MAC Hash seems like a good alternate to try if that doesn't help. – Dacid Salin Nov 26 '18 at 18:40

Not the answer you're looking for? Browse other questions tagged or ask your own question.