Level(3) Lambda TCP / UDP Measurements After Acceptance (4 * STS 12C)

Topology
TCP Tests
- Iperf V. 1.6.2
- Netperf
UDP Tests
- Iperf V. 1.6.2
Other Result Links:
- Results Before Acceptance
- Index of All Lambda Results

In this document the results from TCP and UDP measurements are presented that were executed at the Lambda between SURFnet, Amsterdam, and StarLight, Chicago, provided by Level(3). The results are displayed from the moment the Lambda had been accepted by SURFnet. All were performed after the hosts upgrade of the SARA cluster to 512 Mbyte memory and Linux V.2.4.16-web100.

Topology

For the tests described in this document the topology has been used that is shown in .

    SARA

+----------+                                               EVL
| gwgsara2 |-\
+----------+  \                                +---+     +----------+
               \   +---+   STS 12C   +---+     | 6 |   /-|  prusin  |
+----------+    \--|   |- ......... -|   |-----| 5 |--/  +----------+
| gwgsara4 |--\    | 1 |             | 1 |     | 0 |
+----------+   \---| 5 |- ......... -| 5 |-----| 9 |--\  +----------+
                   | 4 |             | 4 |     +---+   \-| reynolds |
+----------+   /---| 5 |- ......... -| 5 |--\            +----------+
| gwgsara3 |--/    | 4 |             | 4 |  |
+----------+    /--|   |- ......... -|   |--/  hard
               /   +---+             +---+     loop-back
+----------+  /
| gwgsara5 |-/
+----------+

Topology of the used connection scheme. All hosts are directly connected with the ONS15454. Hosts gwgsara2 and gwgsara4 are connected with the LSD6509, while the hosts gwgsara3 and gwgsara5 are internally connected via a hard loop-back at the ONS15454 in Chicago. Four STS 12C channels are used between both ONS15454's.

TCP Tests

Setup

TCP throughput tests were performed between the hosts connected by the scheme displayed in . At each source host one to twelve streams were started to the equivalent destination host. During these tests the sum of the TCP window sizes over the streams was maximal 12 Mbyte per source - destination pair. The duration of each test was 60 s.

Iperf V. 1.6.2

Results

In this section the results of the throughput tests are given that are performed with Iperf V. 1.6.2. The advantage of this tools is that it can easily handle multiple streams using the pthread library.

In the following figures the sum of the TCP throughput values, taken over all streams, has been presented as a function of the sum of the TCP window sizes also taken over the streams, and as a function of the total # streams. In these data are given for the direction gwgsara4 => prusin and in for the reverse direction. The same data are presented in for the directions gwgsara2 => reynolds, and gwgsara4 => prusin. shows the corresponding data for the reverse direction. In the sum throughput values are displayed for the respective directions directions: gwgsara2 => reynolds, gwgsara4 => prusin, and gwgsara3 => gwgsara5 (via the hard loop-back interface in the ONS15454). shows the corresponding data for the reverse directions.

Sum throughput for the direction gwgsara4 => prusin as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

. Sum throughput for the direction prusin => gwgsara4 as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

. Sum throughput for the directions gwgsara2 => reynolds and gwgsara4 => prusin as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

. Sum throughput for the directions reynolds => gwgsara2 and prusin => gwgsara4 as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

. Sum throughput for the directions gwgsara2 => reynolds, gwgsara4 => prusin and gwgsara3 => gwgsara5 (via hard loop-back in ONS15454, Chicago) as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

. Sum throughput for the directions reynolds => gwgsara2, prusin => gwgsara4 and gwgsara5 => gwgsara3 (via hard loop-back in ONS15454, Chicago) as a function of the total window size and the # streams. Generator: Iperf V. 1.6.

In the figures displayed below the average throughput per stream has been given as a function of the TCP window size per stream. In these results are given for the direction gwgsara4 => prusin and in for the reverse direction. shows these data for the combined directions gwgsara2 => reynolds and gwgsara4 => prusin, while the results for the reverse directions are presented by .

Throughput per stream in the direction gwgsara4 => prusin as a function of the window size per stream. Generator: Iperf V. 1.6.

. Throughput per stream in the direction prusin => gwgsara4 as a function of the window size per stream. Generator: Iperf V. 1.6.

. Throughput per stream in the directions gwgsara2 => reynolds and gwgsara4 => prusin as a function of the window size per stream. Generator: Iperf V. 1.6.

. Throughput per stream in the directions reynolds => gwgsara2 and prusin => gwgsara4 as a function of the window size per stream. Generator: Iperf V. 1.6.

Conclusions

From these results the following conclusions can be drawn:

The throughput values obtained with Iperf V. 1.6.2 are comparable with the values obtained with the previously used version 1.2.1 in the sense that the throughput per stream with comparable source and destination hosts is about the same. See for instance the "TCP Tests" section from the "Lambda TCP Tests after Host Upgrade to Linux 2.4" document. The comparison is somewhat difficult, because in these tests the maximum # streams per host is larger than in the mentioned example.
The obtained throughput values are comparable with previous results. Again it is remarkable that the throughput along the longer route SARA - Chicago - SARA is much lower than for the shorter route SARA - EVL v.v.
Despite the fact that maximal twelve streams for each host pair are used, the throughput per stream still is constant. This implies that the network is not yet saturated. Concerning the achieved throughput values, this could not be expected.

Netperf

Results

In this section the results of throughput tests have been given that were executed with the Netperf tool. The same parameters have been used as for the Iperf TCP tests that are described in the "Setup" subsection.

In the figures listed below the sum of the TCP throughput values, taken over all streams, has been shown as a function of the sum of the TCP window sizes (also taken over the streams), and as a function of the total # streams. shows the TCP throughput data for the direction gwgsara4 => prusin and in for the reverse direction. The equivalent data are presented in for the directions gwgsara2 => reynolds, and gwgsara4 => prusin. shows the corresponding data for the reverse direction. In the sum throughput values are displayed for the respective directions directions: gwgsara2 => reynolds, gwgsara4 => prusin, and gwgsara3 => gwgsara5 (via the hard loop-back interface in the ONS15454). shows the corresponding data for the reverse directions.

Sum throughput for the direction gwgsara4 => prusin as a function of the total window size and the # streams. Generator: Netperf.

. Sum throughput for the direction prusin => gwgsara4 as a function of the total window size and the # streams. Generator: Netperf.

. Sum throughput for the directions gwgsara2 => reynolds and gwgsara4 => prusin as a function of the total window size and the # streams. Generator: Netperf.

. Sum throughput for the directions reynolds => gwgsara2 and prusin => gwgsara4 as a function of the total window size and the # streams. Generator: Netperf.

In the following figures the average throughput per stream has been presented as a function of the TCP window size per stream. In these data are shown for the direction gwgsara3 => prusin and in for the reverse direction. displays these results for the directions gwgsara2 => reynolds and gwgsara4 => prusin, while these data are presented in for the reverse directions.

Throughput per stream in the direction gwgsara4 => prusin as a function of the window size per stream. Generator: Netperf.

. Throughput per stream in the direction prusin => gwgsara4 as a function of the window size per stream. Generator: Netperf.

. Throughput per stream in the directions gwgsara2 => reynolds and gwgsara4 => prusin as a function of the window size per stream. Generator: Netperf.

. Throughput per stream in the directions reynolds => gwgsara2 and prusin => gwgsara4 as a function of the window size per stream. Generator: Netperf.

Conclusions

Comparison of the Netperf throughput results with the results obtained with Iperf V. 1.6.2 () leads to the following conclusions:

The performances are better than with the Iperf streams. This is especially true for the route SARA => EVL. An explanation might be that the higher processor load, due to the fact that a fork() is used to start the multiple processes, leads to better shaped TCP streams.
For larger # streams per host pair saturation has been found. For the single host pair tests their follows that this is an indication of too heavy processor, because the network still has sufficient capacity.

UDP Tests

Setup

UDP bulk transfer tests were performed between the hosts connected by the scheme displayed in . At each source host one to eight streams were started to the equivalent destination host. The sum of the maximum shaped UDP bandwidth send was 800 Mbit/s for each source - destination host pair. The duration of each test was 60 s.

Iperf V. 1.6.2

Results

In this section some results are displayed that are obtained with Iperf V. 1.6.2. The percentage total packets has been shown as a function of the total bandwidth send. The results for each # streams are represented by a separate plot traces. In these data are displayed for the single host pair direction gwgsara4 => prusin and in for the reverse direction. shows the data described above for the dual host pair directions gwgsara2 => prusin, gwgsara4 => reynolds and in for the reverse directions. In the packets lost data are given for the triple host pair direction gwgsara2 => reynolds, gwgsara4 => prusin, gwgsara3 => gwgsara5 and in for the reverse directions.

. Total packets lost in the direction gwgsara4 => prusin as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

. Total packets lost in the direction prusin => gwgsara4 as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

. Total packets lost in the directions gwgsara2 => reynolds, gwgsara4 => prusin as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

. Total packets lost in the directions reynolds => gwgsara2, prusin => gwgsara4 as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

. Total packets lost in the directions gwgsara2 => reynolds, gwgsara4 => prusin, gwgsara3 => gwgsara5 as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

. Total packets lost in the directions reynolds => gwgsara2, prusin => gwgsara4, gwgsara5 => gwgsara3 as a function of the sum of the bandwidth send. Generator: Iperf V. 1.6.

Conclusions

To check the UDP performance after acceptance, the current results are compared with previous results at the Teleglobe Lambda, for instance:

For one host pair with the "UDP Tests" section from the "Lambda TCP / UDP Measurements" document. The hosts were during these tests still in the original configuration, running Linux V. 2.2.15 with 128 MB memory.
For two host pairs with the "UDP Tests" section from the "Lambda TCP / UDP Tests after Host Upgrade to 512 MB" document. Also only the memory was upgraded when these tests were executed.
For three host pairs with the "UDP Tests" section from the "Lambda TCP / UDP Measurements after Upgrade of the ONS" document. During these tests the hosts were still in the original configuration.

Comparison with a.o. these tests shows that:

The UDP performance stayed relatively constant. Only the current performance with three host pairs is slightly better, but that can be explained that in the topology during these tests the traffic is better separated.
Compared with the older results the current tests do show some small fluctuations in the # packets lost. The reason for that is unclear.

Results Before Acceptance | Index of All Lambda Results