Trying to get 10 gigabit connection over a switch with 100 megabit standard gateway

Question

I have limited knowledge of this and I'm still learning, so I hope someone can give me advice on this.

I have a 10GbE NAS, a 10GbE NIC in my PC and a switch with two 10GbE ports, which connect all together with fiber. Then there is also another 100Mbit cable attached which connects the switch to the router, which is also the standard gateway. This one also provides the internet connection for all connected devices.

I noticed that a direct connection to the NAS gives me speeds that 10GbE can handle, but when I put the switch in between (all standard VLAN, same subnet and standard gateway), the speeds to the NAS is limited to a max of 1GbE.

Is because the standard gateway is the central where all traffic comes together? How can I manage a peer to peer connection directly to the NAS via the switch, so the traffic goes through the 10GbE cable to the switch and from there through the 10GbE to the NAS?

Extra details:

IP NAS: 192.168.10.120
IP switch: 192.168.10.125
IP PC: 192.168.10.103
Standard gateway: 192.168.10.1
Subnet: 255.255.255.0
Switch: D-link 1510-52

Thank you

Answer 1

I don't think the default gateway has anything to do with your problem. Local traffic between two devices on the same subnet doesn't flow through the default gateway and in any case the speeds you are reporting for your local traffic are 10 times higher than the speed you report for the link to your default gateway.

A couple of possibilities spring to mind.

Firstly are you in the correct ports? from some searching it seems only two of the four fibre ports on that switch are 10G.

Secondly I have heard reports of some early switches which have 10G ports but which can't handle a full 10G for a single flow because of their internal forwarding design.

Answer 2

Looking at the Data Sheet for the D-Link 1510-52 switch:

It mentions switch being capable of "140Gbps" of throughput - using typical marketing "full-duplex" numbers this would mean that on a 50x1G and 2x10G switch like the 1510-52 that the switch is capable of running line-rate through every interface simultaneously. Eg: (1Gbps x 50) + (10Gbps x 2) = 70Gbps x 2 (Full Duplex) = 140Gbps.

Similarly the Packets-Per-Second (pps) numbers for the box are listed as 104.16Mpps, which equates nicely to line rate:

1Gbps = 1,000,000,000 bits/s = (1,000,000,000 bits/s) / (8 bits/byte)= 125,000,000 bytes/s

10Gbps = 10,000,000,000 bits/s = (10,000,000,000 bits/s) / (8 bits/byte)= 1,250,000,000 bytes/s

PPS on 1G port = (125,000,000 bytes/s) / (84 bytes/packet) = 1,488,095 pps

PPS on 10G port = (1,250,000,000 bytes/s) / (84 bytes/packet) = 14,880,952 pps

50 x 1,488,095 pps + 2 x 14,880,952 pps = 104,166,654 pps ~ 104.16Mpps

So, if the data sheet is to be believed, the switch should be capable of this easily.

The one thing you might want to confirm though is that there are two 1G SFP ports and two 10G SFP+ ports, so make sure that you are connected to ports 51 and 52 on the switch and that you are seeing Orange LED status on both ports (which means 10G). If you're seeing Green LEDs this means you're only getting a 1G link.

Answer 3

When switching, throughput between two ports is completely independent of the link speed of another interface. With the non-blocking switches of today, it's even independent of the throughput on any other ports.

There are several possible reasons for the low throughput:

• SFP+ transceivers: they need to support 10GBASE; SFPs/mini GBICs only support 1G rate - check link status in switch
• interface link configuration: may be limited to 1G speed
• other interface configuration: rate limiting, maximum frame size mismatch, ...
• physical link problems: wrong fiber type (-SR = MM, -LR = SM), damaged fiber, dirty ports (FCS and - if indicated - FEC will show errors counting up)