Real Time Transactional Applications Tests

Transactional application tests are all based on the same principle:

Tests consist of a number of transactions between endpoints based on a defined network pattern.

The unitary transaction consists on exchange of information - typically from client (E1, or endpoint from) to server (E2, endpoint to).

This transaction is based on simple exchange (one GET and file reception) or multiple exchange.

The defined throughput will take into account TOTAL throughput on both sides (adding upstream and downstream traffic) and is mostly used to limit the impact on transactions on the network.

  RTSP
Description

Simulate RTSP streaming—client (from probe) requests video to server (to probe) - to probe sends video traffic to client (RTP/UDP). It is a simple transaction (one packet) with get from client (endpoint from-E1) and response from server (endpoint to-E2). And it defines a video stream from E2 to E1 with defined bit rate and MPEG2 or customizable codec. The client is the 'From' endpoint and the server is the 'To' endpoint.

Purpose

Measure simple TCP response time and / or transaction rate. Allows to understand response time for TCP through very simple transactions on network. Also generates a unidirectional video flow from E2 to E21. This can simulate video based on RTP/UDP flows going through the network.

Available Options
  • Test Duration

  • Generated bitrate

  • DSCP setting

Metrics
  • Delay (ms)

  • Jitter (ms)

  • Jitter max (ms)

  • Loss (%)

  • Max loss burst

  • Throughput video (Mbps)

RTSP Control:

  • Response Time (ms)

Advanced Information

This test sends tcp sync and waits for response. Then 100 bytes are sent from E1 to E2 and 100 bytes are sent from E2 to E1. Typically, only 2 packets are exchanged for full transaction.

This test allows generating video rtp streams—bitrate defines corresponding codec (HD or SD).

Results interpretation

The response time is the expected time to transfer very simple packets in both directions into two payloads with TCP. In case of retransmissions, the response time increases significantly.

By default, no objective is set to throughput: any result is good enough, because the total throughput must reflect the response time and transaction rate (transaction rate is 1/response time).

This test generates video stream on the selected path. It can be optimized with setting up send low jitter in the preferences to be 1 if the endpoints support it.

Potential Errors

If the configured bitrate is set too high compared to the expected line capacity:

  • The test may not be able to accomplish first measurement. This configuration results in an error with time out ('Some pairs in tests failures complete').

If the test result pair cannot be collected:

  • Check that your E2 endpoint can open 10115 port toward E1 for management (this is needed for stream pairs).

  • Check that synchronization port between pairs is opened (management UDP 10115).

  • Check that port in use is in the range. See preferences/Test Controller/Destination Port Conf option and preferences/Test Controller/First Destination Port, so that correct destination port for test traffic is used.

 

  TCP response time
Description

Simple transaction (one packet) with get from client (endpoint from) and response from server (endpoint to).

Purpose

Measure simple TCP response time and / or transaction rate. Allows to understand response time for TCP very simple transactions on network.

Available Options
  • Test duration , DSCP setting
Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

This test sends tcp sync and waits for response.

then 100 bytes are sent E1 to E2 and 100 bytes are sent from E2 to E1. Typically only 2 packets are exchanged for full transaction.

Results interpretation

The response time is the expected time to transfer very simple packets in both directions into 2 payloads with TCP. In case of retransmissions the response time will increase significannotly.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  http/https response time
Description

http or https transactions from E1 (client ) to E2 (server) - download file or gif.

Purpose

Measure simple TCP response time and / or transaction rate for http requests. Request is 300 bytes, response is 25 kbytes which is size of typical web page.

Available Options
  • Test duration
  • Test bit rate
  • http format (txt or gif)
Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing http or https traffic with same network pattern and footprint to measure network capability to transport this type of traffic.

Results interpretation

The response time is the expected time to transfer web page.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  pop3/smtp response time
Description

pop3 or smtp transactions from E1 (client ) to E2 (server) - download file or gif.

Purpose

Measure simple TCP response time and / or transaction rate for pop3 or smtp requests. E1 is email client, E2 is email server. The traffic reproduces synthetic transactions.

Available Options
  • Test duration
  • Test bit rate
Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing pop3 or smtp traffic with same network pattern and footprint to measure network capability to transport this type of traffic.

Results interpretation

The response time is the expected time to transfer email.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  ftp response time
Description

ftp transactions from E1 (client ) to E2 (server) - download or upload file of 100kbytes.

Purpose

Measure simple TCP response time and / or transaction rate for FTP requests. E1 is ftp client, E2 is ftp server. The traffic reproduces synthetic transactions.

Available Options
  • Test duration
  • Test bit rate
  • ftp get or ftp put
Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing ftp traffic with same network pattern and footprint to measure network capability to transport this type of traffic.

File size for transactions is 100 kbytes.

Results interpretation

The response time is the expected time to transfer ftp file.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  DNS response time
Description

DNS transactions from E1 (client ) to E2 (server).

Purpose

Measure simple TCP response time and / or transaction rate for DNS requests. E1 is DNS client, E2 is DNS server. The traffic reproduces synthetic transactions.

Available Options
  • Test duration
  • Test bit rate
Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing DNS queries with same network pattern and footprint to measure network capability to transport this type of traffic.

Results interpretation

The response time is the expected time to accomplish DNS request.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  Exchange traffic
Description

Exchange protocol transactions from E1 (client) to E2 (email server).

Purpose

Measure simple Exchange transactions based on sending and receiving traffic from remote site to central site hosting server.

Available Options
  • Number of users
  • Send traffic bitrate
  • Receive traffic bitrate
  • Emails size (sent)
  • Emails size (Received)

The total throughput is distributed amongst number of users !

Metrics
  • Per traffic type (sent or received)
  • Response time Average (ms)
  • throughput total transactions (Mbps) - bidirectional throughput
  • Transaction rate (per second)
  • Total bytes sent and received
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing Exchange messages sent and received. The maximum throughput is the limit set to allow this traffic to go through on each direction.

Results interpretation

The response time is the expected time to transfer email or received.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  SIP response time
Description

SIP protocol transactions from E1 (caller) to E2 (call server).

Purpose

Measure simple TCP response time and / or transaction rate for SIP requests. E1 is SIP client, E2 is SIP server. The traffic reproduces synthetic transactions that are representative of typical SIP based functions.

Available Options
  • Test duration
  • Test bit rate
  • type of SIP transactions

 

Metrics
  • Response time Average (ms)
  • Throughput (Mbps)
  • Transaction rate (per second)
Advanced Information

Synthetic transactions are simulated between E1 and E2 reproducing SIP transactions with same network pattern and footprint to measure network capability to transport this type of traffic.

Results interpretation

The response time is the expected time to accomplish SIP transaction.

By default no objective is set to throughput : any result is good enough as total throughput should reflect response time and transaction rate (transaction rate is 1/response time).

Potential

Errors

If the configured bitrate is set too high compared to expected line capacity, this might result in the test not able to accomplish first measurement. This would result in an error with time out ("Some pairs in tests failures complete"). If results cannot be collected Check that port in use is in range (see "preferences/Traffic Port Management/Destination Port Conf Option" and "preferences/Traffic Port Management/First Destination Port" so that correct destination port for test traffic is used.

 

  Multicast video over RTP
Description

Broadcast of RTP (over UDP) video stream to single.

Purpose

Broadcast of video stream to single (node to node test) or multiple probes (hub and spoke mesh test).

Available Options

Define multicast source IP address and port. Configure stream bit rate.

 

Metrics
  • Datagrams out of order
  • Delay
  • Jitter
  • Loss throughput video rate
  • Total bytes lost
Advanced Information

Can be used for N2N or Mesh tests.Only available for hub and spoke mesh.

Results interpretation

Confirm all probes receive video stream at acceptable rate and quality.

Potential

Errors

Port blocked errors.

 

  Speedtest from->to
Description

Determines average speed between two endpoints in one direction (upload).

Purpose

Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported.

Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate. The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection. As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test. During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest to->from
Description

Determines average speed between two endpoints in one direction (download).

Purpose

Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported.

Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest bidir
Description

Determines average speed between two endpoints in both directions.

Purpose

Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported.

Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest from->to 2 COS
Description

Determines average speed between two endpoints in one direction (download) using two Class of Service (COS) TCP streams.

Purpose

Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported. Each of the two TCP streams will be using a different QOS setting impacting priority of data packets and across the network which will be reflected in throughput.

Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction for each COS
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

The lower the COS number the lower the priority of the TCP stream (QOS).

The software endpoint is dependent upon endpoint base OS to add COS to ipV4 packets and it is possible the base OS (particularly if windows) will not always do this. Additionally, COS is layer 2 traffic, just the ToS 1 byte value in the IP header.

QOS is layer 3 traffic where COS and traffic type are both taken into consideration.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest from->to 3 COS
Description

Determines average speed between two endpoints in one direction (download) using three Class of Service (COS) TCP streams.

Purpose Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported. Each of the three TCP streams will be using a different QOS setting impacting priority of data packets and across the network which will be reflected in throughput.
Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction for each COS
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over a TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

The lower the COS number the lower the priority of the TCP stream (QOS).

The software endpoint is dependent upon endpoint base OS to add COS to ipV4 packets and it is possible the base OS (particularly if windows) will not always do this. Additionally, COS is layer 2 traffic, just the ToS 1 byte value in the IP header.

QOS is layer 3 traffic where COS and traffic type are both taken into consideration.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest from->to 4 COS
Description

Determines average speed between two endpoints in one direction (download) using four Class of Service (COS).

Purpose Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported. Each of the four TCP streams will be using a different QOS setting impacting priority of data packets and across the network which will be reflected in throughput.
Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction for each COS
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

The lower the COS number the lower the priority of the TCP stream (QOS).

The software endpoint is dependent upon endpoint base OS to add COS to ipV4 packets and it is possible the base OS (particularly if windows) will not always do this. Additionally, COS is layer 2 traffic, just the ToS 1 byte value in the IP header.

QOS is layer 3 traffic where COS and traffic type are both taken into consideration.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest from->to 5 COS
Description

Determines average speed between two endpoints in one direction (download) using five Class of Service (COS).

Purpose Performs TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported. Each of the five TCP streams will be using a different QOS setting impacting priority of data packets and across the network which will be reflected in throughput.
Available Options
  • Test duration (sec)
  • Specify which QOS (DSCP setting) to request between switches in network
  • Number of pairs
Metrics
  • Identifies from IP address
  • Identifies to IP address
  • Average TCP throughput (Mbps) in one direction for each COS
Advanced Information

If the test duration is too short the time taken to ramp up TCP throughput will reflect in a lower average throughput rate.

The endpoint establishes multiple connections with the server over TCP port. An initial chunk of data is sent. The endpoint then adjusts the chunk size and buffer size based on TCP metrics to maximize usage of the network connection.

As the chunks are received by the other endpoint, the endpoint will request more chunks throughout the duration of the test.

During the initial stages of the test, the endpoint will establish extra TCP connections (streams) to the other endpoint if it determines additional threads are required to more accurately measure the throughput speed.

The lower the COS number the lower the priority of the TCP stream (QOS).

The software endpoint is dependent upon endpoint base OS to add COS to ipV4 packets and it is possible the base OS (particularly if windows) will not always do this. Additionally, COS is layer 2 traffic, just the ToS 1 byte value in the IP header.

QOS is layer 3 traffic where COS and traffic type are both taken into consideration.

Results interpretation

Speed test between endpoints.

Potential

Errors

See Common Errors - Real Services.

 

  Speedtest from->to with KPI
Description

Speedtest along with network KPI measures speed with TCP + latency/jitter/loss with RTP/UDP packets. It determines average speed between two endpoints in one direction (upload), along with the test network delivery KPI with low footprint (100 Mbps) and 50 packets per second—from E1 to E2 direction.

Purpose The purpose of this test is to perform TCP traffic using several TCP pairs. The test attempts to generate as much TCP throughput as possible using several TCP streams. The average throughput is reported during the test. While the throughput test is run, key KPI loss/jitter/latency are also measured.
Available Options
  • Test duration (sec)

  • Specify which QOS (DSCP setting) to request between switches in network

  • Number of pairs

Metrics
  • Identifies from IP address

  • Identifies to IP address

  • Average TCP throughput (Mbps) in one direction

Network KPI Metrics:

  • One way delay (ms)

  • Jitter (ms)

  • Max jitter (ms)

  • Loss (%)

  • Voice MOS score

  • Packet loss burst

  • Max delay variation(ms)

Advanced Information

If the test duration is too short, the time taken to ramp up TCP throughput reflects in a lower average throughput rate. The endpoint establishes multiple connections with the server over TCP port. An initial set of data is sent. The endpoint then adjusts the data size and buffer size based on the TCP metrics to maximize usage of the network connection. As the data set is received by the other endpoint, the endpoint requests more data sets throughout the duration of the test. During the initial stages of the test, the endpoint establishes extra TCP connections (streams) to the other endpoint; if it determines that additional threads are required to more accurately measure the throughput speed.

This test generates unidirectional traffic with heartbeats to measure the key metrics. The traffic is like voice traffic—MOS score is an indicator of capacity of the link to transport voice in a good quality.

The following definitions help you to understand key metrics:

  • Max delay variation: The time from when a packet is sent from a probe to when it is received, end to end, is the delay. The max delay variation is the difference between the shortest time and the longest time for this end to end delay for exchanging the packets.

  • Max jitter: This parameter measures the variance in spacing between packets sent end to end. If the difference in time (spacing) between 1st and 2nd packet sent from the first probe is 2 ms, and the time on far end probe receiving 1st and 2nd packets is 3.7 ms, the jitter is (3.7-2) ms = 1.7 ms.

Results interpretation

Speed test between endpoints:

  • The network KPI pair generates packets in RTP stream.

  • These packets traverse the network from one probe to the other and provides indication based on the time sent and received, how many are lost and so on.

  • The packets are sent similarly to a G711 voice stream, because this behavior is expected to be a key application of the network's ability to transport.

  • Voice MOS score gives an indication of the quality of transport network for voice G711 application.

Potential Errors

If test result pair cannot be collected:

  • Check that your E2 endpoint can open 10115 port toward E1 for management (this is needed for stream pairs).

  • Check that synchronization port between pairs is opened (management UDP 10115).

  • Check that port in use is in range (see "preferences/Test Controller/Destination Port Conf Option" and "preferences/Test Controller/First Destination Port" so that correct destination port for test traffic is used.

Also see Common Errors – N2N.