Understanding Node to Node Metrics

The Test Totals are divided into three major sections that correspond to the three basic types of measurements taken in an Hawkeye test. Following are explanations of how Hawkeye calculates throughput, transaction rate, and response time.

Throughput

Hawkeye measures the throughput associated with packet payload, ignoring headers. This is referred to as Goodput in RFC 2647.

The throughput is calculated for pairs with the following equation:

Bytes Received (by E2) / (Throughput_Units) /Measured time

For pairs that are bidirectional (TCP based application type of traffic), throughput is aggregated in two directions:

(Bytes_Sent + Bytes_Received_By_Endpoint_1) / (Throughput_Units) / Measured_Time

In all calculations, elapsed time is the elapsed time of the longest-running pair in the test. The measured time is the sum, in seconds, of all the timing record durations returned for the endpoint pair.

Here's how each of the script variables is defined:

• Bytes_Sent: the number of bytes sent by Endpoint 1 of a pair.
• Bytes_Received: the number of bytes received by the endpoint of a pair.
• Throughput_Units: the current throughput units value, in bytes per second. For example, if the throughput units are KBps, 1024 is the Throughput_Units value. In this example, the throughput units number is shown in the column heading as Mbps, which is 125,000 bytes per second (that is, 1,000,000 bits divided by 8 bits per byte). See Raw Data Totals.

Transaction Rate

The calculations are shown in transactions per second. This rate is calculated as follows:

Transaction_Count / Measured_Time

Response Time

The response time is the inverse of the transaction rate. The calculations are shown in seconds per transaction. This value is calculated as follows:

Measured_Time / Transaction_Count

Lost Data

The lost data is the difference between the number of bytes sent by Endpoint 1 and the number of bytes Endpoint 2 actually received. Lost data is only calculated when a pair is running a streaming script (for example, in a VoIP or video). Only payload data is included in the calculations.

Max Loss burst

Maximum number of packets lost in a row during test.

Jitter

Hawkeye helps you determine the quality of a multimedia transmission or voice over IP call using measurements of jitter. RFC 1889 jitter (calculated for all RTP pairs) shows mean statistical deviance of packet inter-arrival times over a period of time.

The jitter (delay variation) maximum reveals when the greatest variation in delay seen for a timing record occurred during the test.

When a datagram is sent, the sender gives it a timestamp. When it is received, the receiver adds another timestamp. These two timestamps are used to calculate the datagram's transit time. If the transit times for datagrams within the same test are different, the test contains jitter. In a video application, it manifests itself as a flickering image, while in a telephone call, its effect may be similar to the effect of packet loss; some words may be missing or garbled.

The amount of jitter in a test depends on the degree of difference between the datagrams' transit times. If the transit time for all datagrams is the same (no matter how long it took for the datagrams to arrive), the test contains no jitter. If the transit times differ slightly, the test contains some jitter. Jitter values in excess of 50 ms probably indicate poor call quality.

Jitter statistics let you see a short-term measurement of network congestion and can also show the effects of queuing within the network. The jitter value is reset for each timing record, so the jitter statistic for a specific timing record shows the jitter for that timing record only.

Almost all data transfers experience jitter, but it is not necessarily a problem. Jitter occurs in several patterns. If the delay time for each datagram steadily increases, jitter values increase and the throughput decreases. But it is also possible for jitter to increase while throughput remains constant. In this case, the delay variation fluctuates widely, which could mean poor performance for delay-sensitive applications. Finally, "bursty" jitter—occurring in bursts of datagrams—has the most significannot effect on call quality in voice over IP transmissions. If jitter occurs in bursts, it can lead to data loss and a degradation of call clarity. Hawkeye measures this "bursty" quality; refer to Max loss burst.

RFC 1889 Jitter

When calculated according to the specification for RTP, jitter (J) is defined as the mean deviation (smoothed absolute value) of the difference (D) in datagram spacing at the receiver compared to the sender for a pair of datagrams. As shown below, this is equivalent to the difference in the "relative transit time" for the two datagrams; the relative transit time is the difference between a datagram's RTP timestamp and the receiver's clock at the time of arrival, measured in the same units. If Si is the RTP timestamp from datagram I, and Ri is the time of arrival in RTP timestamp units for Datagram I, then for two datagrams (I and j), D may be expressed as follows:

D(i,j)=(Rj-Ri)-(Sj-Si)=(Rj-Sj)-(Ri-Si)

The inter-arrival jitter is the jitter calculated continuously as each datagram (I) is received from the source. The jitter is calculated according to the formula defined in RFC 1889:

J=J+(¾D(I-1,I)¾-J)/16

Jitter is measured in timestamp units and is expressed as an unsigned integer. Whenever the endpoint creates a timing record, the current value of J is sampled.

Delay

Endpoints can calculate delay statistics in a single direction for VoIP pairs and for pairs using the RTP protocol. Delay is calculated for each timing record in a test. These measurements are useful in testing time-sensitive applications because they can help to pinpoint sources of delay. One-way or network delay excludes sources of delay apart from the "wire" itself, while end-to-end delay includes all sources of delay, such as the codec used, jitter buffers, and fixed delays.

MOS

Media Delivery Index (MDI) Calculation

The Media Delivery Index (MDI) is a proposed metric, defined in RFC 4445, that can be used as a diagnostic tool or a quality indictor for monitoring the delivery of streaming media on a network. It specifically focuses on the measurement of packet jitter and packet loss in networks carrying streaming media, such as MPEG video, Voice over IP, and other information that is sensitive to arrival time and media loss. Hawkeye collects and reports MDI statistics for all Video Pairs.

The two major factors impacting the quality of a video stream transmitted over an IP network are packet loss and packet jitter. Packet loss can be caused by many factors, including data corruption, insufficient bandwidth, and out-of-order packet delivery. Any packet loss will adversely affect the quality of the delivered video. Packet jitter causes buffer overflows and underflows, either of which will cause unacceptable time distortions in the video streams.

Packet Jitter

Packet jitter is a measure of the variation in arrival rates between individual packets in a media stream. Streaming media requires a consistent and predictable time delay between successive packets as they are received at a destination node. Variations in this interpacket delay causes packet jitter. If packets are delayed by the network, some packets will arrive in bursts with diminished interpacket delays while other packets will arrive with longer interpacket delays. Therefore, the node at the receiving end (the decoder) must buffer the video data to ensure that it can be displayed at its nominal rate. The size of the buffer determines the maximum amount of packet jitter than can be accommodated without experiencing buffer underrun or overrun.

Delay Factor

The MDI consists of two components: the Delay Factor (DF) and the Media Loss Rate (MLR).

The Delay Factor (DF) indicates the maximum difference between the arrival of streaming data and the drain of that data, as measured at the end of each media stream packet. The drain rate refers to the payload media rate. For example, for a typical 3.75 Mbps MPEG video stream, the drain rate is 3.75 Mbps—the rate at which the payload is displayed at a decoding node. The DF is computed at the time that each packet arrives at the Hawkeye endpoint, and is recorded for each timing record. The default timing record duration is three seconds.

The DF is computed at the arrival time of each packet at the point of measurement. For Hawkeye, the point of measurement is Endpoint 2. It is recorded at set time intervals, typically about one second. For Hawkeye, it is measured when each timing record is generated.

If X is a virtual buffer where DF is measured,

X = |Bytes Received – Bytes Drained|

Then, DF is calculated as follows:

DF = [max(X) – min(X)] / media rate

where media rate is expressed in bytes per second and max(X) and min(X) are the maximum and minimum values measured in an interval.

The largest difference is recorded for all intervals in a measurement period. That is, DF is the maximum observed value of the flow rate imbalance over the calculated interval.

Media Loss Rate

The second component in the MDI is the Media Loss Rate (MLR).

The MLR is the count of lost or out-of-order packets, measured over a selected time interval (such as three seconds). That is,

MLR = (Packets Expected – Packets Received) / Interval

There may be zero or more streaming packets in a single IP packet. For example, it is common to carry seven 188 byte MPEG Transport Stream packets in an IP packet. In such a case, loss of a single IP packet would result in seven lost MPEG Transport Stream packets. Counting out-of-order packets is important because many streaming media applications do not attempt to reorder packets that are received out of order.

Media Delivery Index

The Media Delivery Index (MDI) combines the Delay Factor (DF) and the Media Loss Rate (MLR) values for presentation, and is expressed as:

DF:MLR

The DF provides an indication of the size of the buffer needed to accommodate the packet jitter in the network. The MLR gives an indication of the extent of media loss as the streaming media traverses the network.