SNMPD occasionally consumes a high CPU level

Product

VSX (Traditional)

Version

R80.40 (EOS), R81 (EOS), R81.10 (EOS), R81.20

Last Modified

2023-03-07

Symptoms

• The output of top command shows that the main SNMP thread spikes to 100%.

• Perf top on PID of SNMP shows:

  # perf top -p $(pidof snmpd)

  9% libOS.so fw_malloc 9% libdatastruct.so hash_find_hashent

• Changing SNMP mode from VS to default does not resolve the issue.

• Running strace on the snmpd process shows high amount of read cycles on $FWDIR/database/netobj_objects.C across all the configured virtual-systems:

  open("/opt/CPsuite-R80.40/fw1/CTX/CTX<VSID>/database/netobj_objects.C", O_RDONLY) = 168 <0.000025>

• The output of " fw vsx" commands takes several seconds to be printed to the terminal.

• The output of the " fw -d vsx" command shows:

  [xxxx] hash_do_resize: Resizing hash from 16384 to 32768 (n_elements=32768)

  [xxxx] reference_resolving_hash created

  [xxxx] hash_do_resize: Resizing hash from 8192 to 16384 (n_elements=16384)

Cause

The parent process of SNMPD reads the data of $FWDIR/database/netobj_objects.C from all the virtual systems configured and causes it to consume high CPU utilization. Because it is loading more information than it should, the output of  the " fw vsx" command takes a longer time to be printed.

Solution

This problem was fixed. The fix is included in:

• Jumbo Hotfix Accumulator for R81.20 starting from Take 8
• Jumbo Hotfix Accumulator for R81.10 starting from Take 82
• Jumbo Hotfix Accumulator for R81 starting from Take 79
• Jumbo Hotfix Accumulator for R80.40 starting from Take 196

If you choose not to upgrade, use the workaround:

Note: The workaround might disable Check Point's related OIDs.

Comment out the line "dlmod nstAgentPluginObject /usr/lib/nstAgentPluginObject.so" in /etc/snmp/snmpd.conf file.

• To comment the line above, run this command:
  • [Expert@admin]# sed -i 's/dlmod/#dlmod/g' /etc/snmp/snmpd.conf
• To revert the changes:
  • `[Expert@admin]#sed -i 's/#dlmod/dlmod/g' /etc/snmp/snmpd.conf``

NOTE

This solution has been verified for the specific scenario, described by the combination of Product, Version and Symptoms. It may not work in other scenarios.

Article Properties

Access Level: General
Status: Approved by TAC
Date Created: 2022-11-28
Last Modified: 2023-03-07