6

I am receiving the following syslog messages on a 7609-S.

Jun 17 11:52:27.560 BST: %CONST_DIAG-SP-4-ERROR_COUNTER_DATA: ID:47 IN:4 PO:255 RE:169 RM:255 DV:5 EG:2 CF:10 TF:472
Jun 17 11:52:27.560 BST: %CONST_DIAG-SP-4-ERROR_COUNTER_WARNING: Module 6 Error counter exceeds threshold, system operation continue.

The card in slot 6 is as follows:

router1#show module 6 
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  6   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     XXXXXXXX

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  6  000e.d771.8550 to 000e.d771.857f  10.1   7.2(1)       8.7(0.22)FW2 Ok

Mod  Online Diag Status 
---- -------------------
  6  Pass

router1#show ver
Cisco IOS Software, c7600rsp72043_rp Software (c7600rsp72043_rp-ADVENTERPRISEK9-M), Version 12.2(33)SRE3, RELEASE SOFTWARE (fc1)
  • 2013-06-02 : I received this message, once, for the first time
  • 2013-06-06 : I received the message again, only once
  • 2013-06-11 : I received the message again, only once
  • 2013-06-17 : I have received this message three times today, in a 2 hour period

Searching on the Internet I see other people reporting this issue and it seems to be an indication of hardware failure on the horizon. Has anyone experienced this error before? It simply means (to the best of my knowledge) that that line card is receiving a high volume of errors, above a certain threshold which causes the system to log a syslog message. Should I be worried about this line card?

I do have some graphs I will post here when I get some time over the next day or two showing interface error counters and traffic etc, although I'm not finding much correlation at this point!

  • 1
    Dumb question: why aren't you talking to Cisco TAC? – Mike Pennington Jun 17 '13 at 16:25
  • @MikePennington No, +1 from me, good question IMO! I should have mentioned in the posting, it's not my box, no TAC available, I'm just the bozo that has been tasked with looking at it :) – jwbensley Jun 17 '13 at 16:28
  • 1
    Then perhaps the best you can do is 1) reseat the LC... 2) replace the LC... 3) Upgrade IOS and pray it was a bug... running HW without support is something you should hammer your management about – Mike Pennington Jun 17 '13 at 16:31
3

Worst case scenario, your HW has gone bad
Best case scenario, it's cosmetic failure due to software defect, luckily you are in SRE which will be supported until 2015, so maybe upgrade it to latest rebuild.

There are two bugID which will cause this error in very benign way.

  • CSCsk03373, due to large packets, fixed in SXH
  • CSCsw32280, due to CRC errors, fixed in SXH

You should probably check 'show diag events', it should correlate with these messages.

GOLD gives us description for 'TestErrorCounterMonitor', which gives us some data on understanding the message

ID -- Asic Identification
IN -- Asic Instance
PO -- Asic Port Number
RE -- Register Identification
RM -- Register Identification More
EG -- Error Group
DV -- Delta Value
CF -- Consecutive Failure
TF -- Total Failure

I don't unfortunately have CEF256 cards, so I can't check which ASIC it was, but you should be able to do it with:

remote command switch show platform hardware asic-versions | i 47

IN will which of the ASIC it is, I'm guessing as there is at least 4 of them, it is 'pinnacle' ASIC, which is port-asic in CEF256, as I don't think CEF256 has 4 of any other ASIC.

If it is pinnacle, you should be able to use 'sh int capabilities module X' and 'sh int X capabilities' to determine which ports are sharing the 4th port ASIC.

However as the 'Asic Port Number' is 255, it seems to contradict it being 'pinnacle' as no physical port would have this number.
There are some special ports in the card like EOBC, RBUS, DBUS and fabric. Unfortunately I don't know what 255 means, it might mean some of these special ports, it might be just place-holder value.

If 'Total Failure' or TF correlates with interface CRC errors, it might be CSCsw32280, otoh CSCsw32280 should show sensible PO number.

If everything else fails, buy smartnet for the card for a year. I'd be curious if you'd answer your own question when you solve this as to what was the root cause. And especially if you can find out what is port 255.

  • 1
    CSCsw32280 is fixed in 12.2(33)SRE4... no need to jump to 12.2(33)SXH – Mike Pennington Jun 17 '13 at 17:40
  • Excellent advise as always Ytti. I will post back here what I find out. Thanks! – jwbensley Jun 19 '13 at 10:00
5

You need to open a case with Cisco TAC... obligatory Cisco Support Forums reference to a similar problem

Without TAC support perhaps the best you can do is

  1. reseat the LC...
  2. replace the LC...
  3. Upgrade IOS and pray it was a bug...

Be sure the screws on the LC are snugged on tight when you do maintenance... loose screws can cause more problems than you might imagine (think repetitive heating / cooling cycles wiggling the LC loose)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.