Back 2 Back TCP Tests

Method

It appears that the effect of txqueuelen does not affect noticeably the flow of udp packets at GigE rates. In order to give a full analysis, we now test the effects of tcp with various txqueuelen sizes.

In order to give a full account of tcp performance, we will be using tcp with web100. The tcp stack under analysis is the standard (vanilla) tcp that comes with the 2.4.19 kernel. The web100 version used is 2.1.

This investigation was primarily initiated to deduce the optimal size of the txqueuelen to set the number of SendStalls (interface stalls) for tcp transfers.

These tests were performed using 2minute iperfs with large socket buffers (2m) on both the sender and recver. The test was repeated 3 times and the averages show.

Results A: Dynamic Right Sizing

Results: xls tsv

As you can see, there is a dramatic decrease in the number of sendstalls as we increase the size of the txqueuelen. This change is less dramatic at about txqueuelen of 15. Past a txqueuelen of 200, we get no sendstalls.

It is noteworthy that the default txqueuelen size if set to 100.

Looking at the throughput reported by both iperf and logvars, we can see that we get a steady rate of 941ish mbit/sec. It is also worth noting that logvars grabs values from web100.

It is also interesting that tcp appears to get higher throughput than the updmon results shown prior. Using iperf udp shows that we can recieve at 957mbit/sec which equates to line rate.

In order to see an effect of the txqueuelen on the tcp performance, we really need much higher latencies than that provided by a back to back link.

Looking indepth into the effects of tcp, we show the cwnd values as a function of the txqueuelen. We can see that there is a sharp increase in the values of cwnd. Typically, we would expect a proportional increase in throughput with cwnd. However as we are already going at line rate, this is not the case.

The variation of the value of cwnd (at 1 sec intervals) shows that it is pretty constant; until we get to large cwnd values. Then we appear to be less variable in the values of cwnd.

Looking at the number of dupacks recieved, we appear to get a lot more for larger txqueulen's. This flattens off at the optimal txqueuelen value. Unforuntately, for tcp, we would preferably want less dupacks as this may cause a FastRetrans causing our cwnd to half and our throughput to decrease.

It is also a bit strange that we get any dupacks at all; dupacks occur either through reordering or through loss. If there was a loss, then we would get a fast retransmit (loss is accounted for when there are 3 consequtive? dupacks recieved).

There does not appear to be any increase or decrease in the number of fast retransmits; indicating that the loss (3 consequtive? dupacks) are not recieved.

Note that we get fractional numbers because we are taking an average of the tests.

This shows the number of packets going out and coming in from the sender. As you can see, we appear to get half the number of packets coming back in; indicating that we are getting delayed acks from the recv. The number of packets appear constant throughout - now surprising as we are getting line rates. We send out about 10million packets for the duration of the 2 minute test for each txqueuelen.

This shows the total number of slowstarts (acks which cause the slow start algorithm to initiate on the sender) that each txqueuelen (on average) experience. There appears to be a high number of slow starts for small txqueuelens, which dip between teh range 15-200. This then increases to a mid level for any txqueuelens higher.

The really odd thing this graph shows is that there does not appear to be any packets at all which cause congestion avoidance to increase past a txqueuelen of 300. This correlates to a higher number of packets going into slow start.

The graph showing that the number of congAvoids tend to zero for larger txqueuelens is matched by the fact that the above graph shows that the number of CongestionSignals also tend to zero at the same point. However, we get an increase in the number of OtherReductions (either Cwnd Moderation of Cwnd Validation). The latter would cause the cwnd to be capped to a value of min( cwnd, packets_in_flight + 3). Therefore, should the number of packets in flight be at least 3 less than cwnd, we would reduce to that value and hence reduce the value of cwnd.

Just for completeness, this shows the average rtt of the different queuelens. However, due to the granularity of hte way that the kernel adds up rtts, it's not very meaninggul.

Mon, 7 April, 2003 17:01