Recently the routing engine CPU utilization on two of our Juniper peering routers increased from ~10-20% average load to 80+%. I'm trying to figure out what's causing this (and how to get this high load back down).
Some info on the routers: both run the same JunOS version, both are connected to the same two peering IXP LANs and have a large number (several hundreds) of (almost identical) IPv4 and IPv6 sessions. Both routers have a connection to a different IP transit provider and are connected in the same way to the rest of our network. The routing engines' CPU load isn't flatline on 80+%, there are drops back to normal levels for minutes to hours, but these drops are not that often.
Things I've checked:
- no configuration changes have been made at the moment the increase started
- there's no increase in non-unicast traffic directed at the control plane
- there's no (substantial) change in the amount of traffic being forwarded (though even a increase shouldn't matter)
show system processes summaryindicates the
rpdprocess is causing the high CPU load
- there are no rapidly flapping BGP peers causing a large amount of BGP changes
One possible explanation I can come up with is a peer (or more than one) on one of the IXP's both routers are connected to sending a large number of BGP updates. Currently I only have statistics on the number of BGP messages for my transit sessions (showing no abnormal activity) and with several hundreds of BGP sessions on the peering LANs it's not that easy to spot the problematic session(s) if I should create graphs for all sessions.
My questions are:
- are there any other things I should check to find the cause of this increase in CPU load on the routing engines?
- how can I easily find out which sessions are causing these problems (if my assumption is right)? Enabling BGP traceoptions generates huge amounts of data, but I'm not sure if it gives me any real insights.