SNMP Configuration and Verification

You can configure and verify the Simple Network Management Protocol (SNMP). This section explains how to configure and verify the SNMP.

SNMP Traps

Hawkeye server supports the generation of SNMP traps for tests originated from endpoints. Tests configured to generate SNMP traps on certain error/failure conditions sends the SNMP traps in MIB format to a designated SNMP server.

There are six steps involved:

How to use SNMP traps
Verify Hawkeye Server MIBs file can be interpreted by the User’s SNMP server
Configure HawkeyeServer in Preferences to send SNMP trap to a specific User’s SNMP server
Configure User’s SNMP server to receive Hawkeye server originated SNMP traps
Enable SNMP traps to be triggered for endpoint scheduled tests
Verify SNMP trap successfully received and decoded by SNMP server
Advanced Troubleshooting – Monitor Hawkeye server SNMP debug logs

How to use SNMP Traps

SNMP traps can be configured to be triggered by test results on end points. The time between tests and consequently the possible reporting time between SNMP traps is Customer configurable. The shortest time between tests on an endpoint is one minute.

SNMP traps are supported for all test types. When configuring a test, select Show Alarm options to see the SNMP configuration.

Each test can be enabled to trigger SNMP traps, based on a failure or error result. There is the option to only report an SNMP trap on a change. This means if a test on an endpoint is scheduled to run every 5 minutes and it is failing every time, it is only reported as an SNMP trap the very first time it fails, then it generates another SNMP trap once the endpoint test result changes to be an error or passes.

Verify Hawkeye Server MIBs file can be interpreted by the User’s SNMP server

Hawkeye SNMP traps encoding is identified by three files. Retrieve the following three MIBs files from the Hawkeye server so that they can be verified:

/usr/share/snmp/mibs /NET-SNMP-MIB.txt
/opt/hawkeye/includes/MIBS/Hawkeye-MIB.txt
/opt/hawkeye/includes/MIBS/IXIA-SMI.txt

There are a number of independent free web sites that verify that custom MIB files confirm to correct standards.

Suggested free independent web sites to verify Hawkeye MIBs format:

http://www.simpleweb.org/ietf/mibs/validate/upload.php

http://www.muonics.com/Tools/smicheck.php

From these web sites pull in the retrieved two Hawkeye files and validate to MIBs standards. There are three supported versions of MIB files (v1, v2 and v3).

Configure Hawkeye Server to send SNMP trap to a specific SNMP server

Enable SNMP traps to be generated by the Hawkeye server.

To enable the SNMP traps, do the following:

Go to Administration > preferences > Alarms toggle the Enable/disable the alarms received using SNMP.

Also configure the IP address of the your SNMP trap server, to which the SNMP traps are to be sent to.

Multiple SNMP trap receivers can be supported by using a third party application that will forward received UDP packets from the Hawkeye Server IP address/port to a defined list of destination SNMP trap receivers (IP/port). There are many free applications that are available for this purpose, such as the free linux samplicator application. The SNMP server IP defined on the Hawkeyeserver to achieve this will be the IP of the third party application doing the forwarding.

Configure your SNMP server to receive Hawkeye server originated SNMP traps

There are many SNMP servers (receiver) available but the required basic configuration is as follows:

Load in the two Hawkeye MIB modules identified and retrieved from the Hawkeye server in the above steps (NET-SNMP-MIB.txt, IXIA-SMI.TXT and Hawkeye-MIB.txt). This enables your SNMP trap server to decode the Hawkeye SNMP traps.
Configure the host address of the Hawkeye server that will be the source of the SNMP traps.
Set the host port used to generate and be monitored for the SNMP traps to 162.
Set the community to ixia.
Following a Hawkeye Server upgrade, remove, flush, or delete any Hawkeye MIB files from the SNMP server (receiver) and pull in the latest version as new test types. Else, changed test metrics will cause decode issues.

Enable SNMP traps to be triggered for endpoint scheduled tests

Schedule a test (example, node to node Skype4B traffic test) on probes expected to generate SNMP traps on error/failure conditions.
When configuring the test, set thresholds to expected values additionally select Show Alarm Options.
Go to Show Alarm Options and enable the SnmpTrap.
Also select the conditions for the alarm to be triggered that generates the SNMP Trap (Error/Failed/Status change).

Verify SNMP trap successfully received and decoded by SNMP server

Invoke SNMP trapviewer from your SNMP server. Generate SNMP trap by running Hawkeye test that fails. Confirm that the SNMP trap received in the trap viewer is as expected and can be decoded.

Advanced Troubleshooting – Debug SNMP packet routing issues

Putty (ssh) to Hawkeye server and monitor SNMP trap logs to confirm that tests are triggering sending of SNMP traps to SNMP server.

On the Hawkeye server go to directory /opt/hawkeye/logs/Hawkeye and monitor server daemon logs with the following command.

tail -f ServerDaemon.log

Confirm able to see logs such as the one below where the first IP address is the IP address of the SNMP server to receive the SNMP trap.

20:24:58 UTC (Alarm send) sending snmptrap -

snmptrap -M /usr/share/snmp/mibs/:/opt/hawkeye/includes/MIBS -c ixia -v 2c 10.220.120.11 "" HAWKEYE-MIB::hawkeye-notification trapID s "xr2000autoqa2xrpiWi-Fiqa2Skype4BTrafficDelaymsAudioRTPfromto12kbpsFAIL" summary s "xr2000autoqa2 to xrpiWi-Fiqa2 Skype4B Traffic Audio RTP fromto 12 kbps Delay ms Failed" runID s "8285" TimeStamp s "2016-11-30 20:24:57" TestType s "Skype4B Traffic" TestStatus s "Failed" From s "xr2000-auto-qa-2" To s "xrpi-Wi-Fi-qa-2" PAIRNAME s "Audio RTP from->to 12 kbps" MetricName s "Delay (ms)" METRICVALUE s "2.5" DefaultThresholdType s "1" THRESHOLD s "9000" METRICSTATUS s "Failed" FailReason s " Threshold failed on Audio RTP from->to 12 kbps Delay (ms) "

Additionally use TCP dump command to confirm that the SNMPtrap packet is not being blocked in the network. The TCP logs below (taken from Hawkeye server) show that a SNMP trap is generated from Hawkeye server (10.220.120.127) and sent to the SNMP trap server/receiver (10.220.20.45) but is blocked as unreachable.

TCP dump from Hawkeye server:

tcpdump -n -i eth0 | grep -i "10.220.120.127"

21:11:38.246127 IP 10.220.120.127.52365 > 10.220.20.45.snmptrap: C=ixia V2Trap(183) .1.3.6.1.2.1.1.3.0=148999358 .1.3.6.1.6.3.1.1.4.1.0=.1.3.6.1.4.1.8072.2.1.0.3 .1.3.6.1.4.1.8072.2.1.2.1=3 .1.3.6.1.4.1.8072.2.1.2.2="Amazon-KFFOWI" .1.3.6.1.4.1.8072.2.1.2.3="Amazon-KFFOWI" .1.3.6.1.4.1.8072.2.1.2.4="0" .1.3.6.1.4.1.8072.2.1.2.5="TestAgent1"

21:11:38.247491 IP 10.220.20.45 > 10.220.120.127: ICMP 10.220.20.45 udp port snmptrap unreachable, length 234

In the above example, unlike the Network Unreachable and Host Unreachable messages which come from routers, the Port Unreachable message comes from the SNMP server/receiver. The primary implication for troubleshooting is that the frame was successfully routed across the communications infrastructure, the last router ARP'ed for the host, got the response, and sent the frame. Furthermore, the intended SNMP server (destination host) was on-line and willing to accept the frame into its communications buffer. The frame was then processed and an attempt was made to send the data up to the destination port number (UDP port 162) and the port process (SNMP server) did not exist. The protocol handler then reports Destination Port Unreachable.

For SNMP traps generated by Hawkeye Server to reach a third party SNMP trap receiver the UDP ports 161 and 162 need to be open for UDP traffic on the Hawkeye server and switches between the two parties. One way to open the ports on the Hawkeye Server is to putty to the Hawkeye Server then using the following CLI commands to open the firewall for SNMP traffic:

You need privilege escalation for the following steps.

ksuser@he-qa-deb:~$ su -

Password:

root@he-qa-deb:~# firewall-cmd --add-service=snmp --permanent

success

root@he-qa-deb:~# firewall-cmd --add-service=snmptrap --permanent

success

root@he-qa-deb:~# firewall-cmd --reload

success

Hawkeye SNMP MIB Information

This section covers information about SNMP MIB provided by Hawkeye

MIB file location

MIB files can be found into the following directory /opt/hawkeye/includes/MIBS

Associated MIB Descriptors

Associated MIB descriptors can be needed to import MIB. The associated MIB descriptor are available in the same subdirectory:

/usr/share/snmp/mibs