How to get per-process memory use from Cisco switches via SNMP

Question

I'm trying to get per-process memory use from Cisco switches via SNMP.

I've found an article suggesting you can subtract CISCO-PROCESS-MIB.cpmProcExtMemAllocatedRev and CISCO-PROCESS-MIB.cpmProcExtMemFreedRev (Cisco process memory usage) but that doesn't seem to result in sane values.

Often the values are the same (resulting in a zero) and sometimes the freed is greater than the allocated (resulting in a negative number)- though I think this is probably to do with memory being freed between when I pull the allocated results and when I pull the freed results.

The output of show processes memory on a switch shows the same results that I see via SNMP (insane values if the allocated - freed logic is correct), but it also shows a Holding column that looks like it has what I need.

    Switch1#show processes memory
    Processor Pool Total:  175382376 Used:   47922940 Free:  127459436
          I/O Pool Total:   16777216 Used:   13591380 Free:    3185836
    Driver te Pool Total:    4194304 Used:         40 Free:    4194264

     PID TTY  Allocated      Freed    Holding    Getbufs    Retbufs Process
       0   0  118863872   66054580   48365728          0          0 *Init*
       0   0      12476 2411645460      12476          0          0 *Sched*
       0   0 3937788032 4286508576    3540184   15562527    1490354 *Dead*
       0   0          0          0     394476          0          0 *MallocLite*
       1   0     348672     187988     175856          0          0 Chunk Manager
       2   0        232        232       4160          0          0 Load Meter
       3   0         76          0       9236          0          0 hulc_entropy_thr
       4   0          0          0      10080          0          0 Connection Mgr
       5   0       4712       4520      11692          0          0 Check heaps
       6   0   16741228   29720504      38796   14834428   22351307 Pool Manager

I can't find any references to Holding in the CISCO-PROCESS-MIB and I've had no luck searching on the internet.

Does anyone know how to get this Holding field via SNMP?

Answer 1

I've just spent a good half an hour looking at this, and I don't think there is a way to get an accurate number from SNMP.

This is the data available from the 3 tables in cpmProcess (there are many more OIDs, but they don't seem to be populated on the devices i was looking at)

CISCO-PROCESS-MIB::cpmProcessPID.1.5 = Gauge32: 5
CISCO-PROCESS-MIB::cpmProcessName.1.5 = STRING: Pool Manager
CISCO-PROCESS-MIB::cpmProcessuSecs.1.5 = Gauge32: 215 microseconds
CISCO-PROCESS-MIB::cpmProcessTimeCreated.1.5 = Timeticks: (267) 0:00:02.67
CISCO-PROCESS-MIB::cpmProcessAverageUSecs.1.5 = Gauge32: 215 microseconds
CISCO-PROCESS-MIB::cpmProcExtMemAllocated.1.5 = Gauge32: 23223664 bytes
CISCO-PROCESS-MIB::cpmProcExtMemFreed.1.5 = Gauge32: 1784383920 bytes
CISCO-PROCESS-MIB::cpmProcExtInvoked.1.5 = Counter32: 418471
CISCO-PROCESS-MIB::cpmProcExtRuntime.1.5 = Counter32: 90380 microseconds
CISCO-PROCESS-MIB::cpmProcExtUtil5Sec.1.5 = Gauge32: 0
CISCO-PROCESS-MIB::cpmProcExtUtil1Min.1.5 = Gauge32: 0
CISCO-PROCESS-MIB::cpmProcExtUtil5Min.1.5 = Gauge32: 0
CISCO-PROCESS-MIB::cpmProcExtPriority.1.5 = INTEGER: critical(1)
CISCO-PROCESS-MIB::cpmProcExtMemAllocatedRev.1.5 = Gauge32: 23223664 bytes
CISCO-PROCESS-MIB::cpmProcExtMemFreedRev.1.5 = Gauge32: 1784383920 bytes
CISCO-PROCESS-MIB::cpmProcExtInvokedRev.1.5 = Counter32: 418471
CISCO-PROCESS-MIB::cpmProcExtRuntimeRev.1.5 = Counter32: 90380 microseconds
CISCO-PROCESS-MIB::cpmProcExtUtil5SecRev.1.5 = Gauge32: 0 percent
CISCO-PROCESS-MIB::cpmProcExtUtil1MinRev.1.5 = Gauge32: 0 percent
CISCO-PROCESS-MIB::cpmProcExtUtil5MinRev.1.5 = Gauge32: 0 percent
CISCO-PROCESS-MIB::cpmProcExtPriorityRev.1.5 = INTEGER: critical(1)

I wrote a script to snmpget all of the values to prevent timing issues, and sorted them by Allocated - Freed :

(the middle bit removed)

PDU DISPATCHER        3446519088
IP SNMP               1742301596
SNMP ENGINE           792807060
IP Input              635031724
Logger                27247348
TCP Protocols         15053688
Per-Second Jobs       10510488

...

SSM connection manager  0
Crypto IKE Dispatcher  -472
IP Background         -1880
DHCPD Timer           -47424
TTY Background        -103736
LOCAL AAA             -126696
Transport Port Agent  -181900
IP Cache Ager         -1120096
crypto sw pk proc     -2327016
TCP Timer             -606529520
Pool Manager          -1761160256

The processes with negative values don't seem to have much in common, and it's strange that PDU DISPATCHER and IP SNMP would be using the most memory.

I think it might be down to shared memory and the original OIDs pre-dating modern memory management in IOS.

It's possible that the data needed is contained in the OIDs Cisco don't seem to often populate :

1.3.6.1.4.1.9.9.109.1.2.3.1.15 cpmProcessTextSegmentSize
1.3.6.1.4.1.9.9.109.1.2.3.1.16 cpmProcessDataSegmentSize
1.3.6.1.4.1.9.9.109.1.2.3.1.17 cpmProcessStackSize
1.3.6.1.4.1.9.9.109.1.2.3.1.18 cpmProcessDynamicMemorySize

I can't find any other Cisco MIBs that contain better data, so I don't think there is a reliable way to get this information.