Cisco performance issues C890 series

Question

I'm working on a complex project that includes Linux and Cisco routers. I am fairly new to networking and i'm using cisco command lines for the first time, so please be nice and don't hesitate to suggest/correct me.

A simplified network scheme could be this one The computers are running under Linux/Xenomai. They have two network interfaces. Real-time data are using 192.168.1.1 address and is only UDP. Monitoring/ssh is using 192.168.2.1 address.

Those two interfaces are connected to a Cisco router, which WAN address is 10.11.151.X. The cisco router is doing NAT on our packets, so they can cross the network and acces the distant router, which NAT backwards. 192.168.1.1 using port 2100 goes in router, is natted to 10.11.151.1 using port 2100 and vice-versa.

I have two different pairs of routers: a Cisco RV325 and a Cisco C891f-k9. The C891f-k9 router configuration is the following (I removed users/security and unused interfaces before posting) :

version 15.3
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
boot-start-marker
boot-end-marker
!
aqm-register-fnf
!
no ip domain lookup
ip cef
no ipv6 cef
!
multilink bundle-name authenticated
!
crypto isakmp policy 1
!
interface FastEthernet0
 no ip address
 shutdown
 duplex auto
 speed auto
!
interface GigabitEthernet4
 switchport access vlan 11
 no ip address
!
interface GigabitEthernet5
 switchport access vlan 21
 no ip address
!
interface GigabitEthernet8
 description PrimaryWANDesc_
 ip address 10.11.151.6 255.255.255.0
 ip nat outside
 duplex auto
 speed auto
!
interface Vlan1
 no ip address
 shutdown
!
interface Vlan11
 ip address 192.168.1.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!
interface Vlan21
 ip address 192.168.2.254 255.255.255.0
 ip nat inside
 ip virtual-reassembly in
!

ip forward-protocol nd
no ip http server
no ip http secure-server
no ip http path flash
!
ip nat inside source static tcp 192.168.2.1 22 10.11.151.1 2044
!
ip nat inside source static udp 192.168.1.1 2100 10.11.151.1 2100
ip nat inside source static udp 192.168.1.1 2101 10.11.151.1 2101
ip nat inside source static udp 192.168.1.1 2102 10.11.151.1 2102
!
ip route 0.0.0.0 0.0.0.0 10.11.151.253
!
snmp-server community public RO
!
control-plane
!
mgcp behavior rsip-range tgcp-only
mgcp behavior comedia-role none
mgcp behavior comedia-check-media-src disable
mgcp behavior comedia-sdp-force disable
!
mgcp profile default
!
line con 0
 no modem enable
line aux 0
line 3
 modem InOut
 speed 115200
 flowcontrol hardware
line vty 0 4
 privilege level 15
 password cisco
 login local
 transport input telnet ssh
!
scheduler allocate 20000 1000
!
end

The RV325 is a very basic router configured with web interface.

I did some tests using the RV325 pair of routers and the same tests using the C891f-K9. With this configuration, I am observing poor performances with C891F compared to the RV325. I can't explain the differences. I was hoping that the high-end router would do better than the RV325, but it's the opposite. I checked the CPU usage of the C891f when in use and it's barely 10%.

So my questions are: Is this configuration good? Do you have any tips that could help me with performance issues? Are there some services that I could shut down to improve router efficiency/latency?

As I said, I'm new to network and routing, so don't hesitate to ask for explanations or to advise me. Thank you for the help.

EDIT:

Thank you for the help. I managed to do some tests and our performances are still not good.

How tests are done: We have a computer plugged on each server we have. This computer generates traffic and records the incoming traffic. Our machines are configured to just pack/unpack the packets. Then we compare the sent data and the receive: we find the corresponding data and we compare the timestamp. This timestamp difference gives us the total amount of time needed to cross our system.

With the cisco rv325, the mean is 200µs. With the C891F, the mean is 450µs. I also checked the ping from the machine that is directly connected to the router: rv325 ping is >400µs and c891f-k9 is ~600µs.

From now, I believe that there is nothing more I can do on the configuration. I believe that the issue is the router itself that does not suit our needs.

EDIT 2:

Thank you all for your help. It helped me a lot. The conclusion is that this series of router does not suit our needs. The most stable and efficient solution we found is to install pfSense on a standard computer (intel i5 with 8Go RAM and intel i350 4 ethernet ports). With this, we have a latency of ~0.2ms with no pikes and no lost packets. This is, by far, the best we have done for the moment and probably the best we could achieve.

Answer 1

Well, the (corrected) config is straightforward and there's nothing much you could do differently or in a better way.

But first, let's confirm that you have the younger model. The fact that the LAN switch ports are "gigabit" is a strong hint that this is the case, but let's verify.

Issue the "show inventory" command and look for the difference of "CISCO89x" vs "C89x" in the Product ID (PID). On a younger C89x, you would find (example taken from a C892FSP)

C892#show inventory

NAME: "C892FSP-K9", DESCR: "C892FSP-K9 chassis, Hw Serial#: xxxxxxxxxxx, Hw Revision: 1.0" PID: C892FSP-K9 , VID: V02, SN: xxxxxxxx

On an older CISCO89x, that would be

CISCO891#show inv

NAME: "891", DESCR: "891 chassis, Hw Serial#: xxxxxxxx, Hw Revision: 1.0" PID: CISCO891-K9 , VID: V02, SN: xxxxxxxx

Should you have the older model and be looking for performance: don't. Actually, don't look for "high-performance" in the 800 Series at all.

These are branch office models well suited to provide feature-rich connectivity in the 25-50Mbps market, where the WAN circuit has latency characteristics in the a-few-tens-of-milliseconds to start with.

What makes them look "high-end" is their rich feature set (QoS, Dynamic Routing, virtualised Routing (VRFs), Firewalling, NAT, IDS, WAN Accel., MPLS, IPSec etc.). But they are not known for outstanding performance or low latency.

At my employer's, we use the C89x family to provide WAN connectivity to customers with WAN circuits up to 50Mbps (making use of VRFs, MPLS-o-GRE-o-IPSEC tunneling and QoS). We see the 50Mbps being fully-used on a regular basis. We are aware that this is already above what Cisco says this device is intended for.

I have a C892 on my homebrew testbed right now. In "dragstrip routing" [1] with large packets, it can take the full gigabit pipe at ca 80kpps with IPv4 and IPv6, both TCP and UDP.

With small packet payloads (88bytes MSS for TCP), there seems to be some limit at ~280kpps (IPv4) and ~80kkpps (IPv6) when routing between L3-Switch and WAN-interface. Oddly, these values are better when routing between gig8 and gig9 of the C892FSP: >600kpps for IPv4, ~144kpps for IPv6;

My homebrew testbed does not allow for generating more than ~120kpps of UDP reliably, so the results are a bit shady, and I pefer to not share them.

A router's performance is generally characterized by packet rate (often related to CPU power, resp. CPU load) and delay introduced. Throughput and "Performance" just follow by "multiplicator by packet size" and how good the upper layer protocols in question can cope with the unavoidably increased delay.

Depending on which set of features ou activate on a low end platform such as the 800 Series, the router's CPU and therefore packet rate is taking massive hits - especially NAT is notorious for that.

To follow-up on Teun Vinks answer - please define what you consider the "performance issue" to be. Is it only the latency difference of 0.2ms vs 0.45ms between RV325 and C891?

If latency differences in the sub-ms range are your "hot topic", that puts you in the low-latency networking game, and you should consider designing your network in a way that routing nor NATing can be avoided at all, or you'll have to consider specialized products. The 800 Series are definitely not in that league.

I'm not at expert at host based firewalling or NATting, but couldn't the same goal be achieved with an additional LAN interface on the Linux machine (connected to where the Cisco's gig8 is plugged into), with some "iptable magic" added?

Best regards Marc

[1] no NAT, no ACLs, no CBAC-FW, no ZB-FW, no IPS, no Tunneling, no IPSec encap/decaps, no QoS)

Answer 2

Your Cisco configuration is incorrect for NAT. It should not work at all. The problem is that you have ip nat inside configured on the layer-2 interfaces (GigabitEthernet4 and GigabitEthernet5) instead of the layer-3 interfaces (Vlan11 and Vlan21).

Also, you should not use the default-gateway command on a router that is routing because it is the gateway. Instead, use a static default route, which you have also configured.

Try something like:

interface GigabitEthernet4
 no ip nat inside
!
interface GigabitEthernet5
 no ip nat inside
!
interface Vlan11
 ip nat inside
!
interface Vlan21
 ip nat inside
!
no ip default-gateway 10.11.151.253
!
ip nat inside source list 10 interface GigabitEthernet8 overload
!
access-list 10 permit 192.168.1.0 0.0.0.255
access-list 10 permit 192.168.2.0 0.0.0.255
!

Get that to work before you start getting fancy with static translations. I really don't see how you were getting NAT to work at all since you didn't have the NAT inside on the layer-3 inside interfaces.