Investigation into txqueuelen across the MBNG WAN - 14th February 2003

Purpose

The txqueuelen is another buffer in the kernel stack that can affect performance; especially of tcp transfers. When the system is sending out too much data from the ip layer to the ethernet device driver layer, this buffer, txqueuelen, may overflow. In tcp, this has the effect of a congestion event causing the cwnd to halve.

In terms of frequency of the sendstalls (at Gig rates), the above graph shows for a 10 minute tcp transfer, the accumulative number of a sendstalls experienced. This roughly equates to about 92/600 = 0.15 sendstalls per second, or one sendstall every 6.5 seconds.

Method

A series of iperfs using vanilla 2.4.19 tcp will be conducted from mbng1 to gig1 and monitored with web100 (2.1). It is recommended that 5 minute transfers take place with various sizes txqueuelens, and the results plotted.

Txqueuelen's of 1,5,10,15,20,25,50,100 were choosen initially.

Results

table: tsv

logs: zip

Hmmm... not very good results :( Need a lot more tests!

Almost no difference.... :(

The average throughput isn't exactly great either....

And the variation worse....

Indepth Analysis of txqueuelen effect on SendStalls - 20th/21st Feb 2003

Method

Single TCP streamed tests were performed on the MB-NG network using iperf with socket buffers of 2m on both ends. Test duration was 300seconds. The txqueuelen was varied each time and the resutls plotted. The tests were conducted on a clean (ie empty) network.

test1 tsv zip

test2 tsv zip

test3 tsv zip

average tsv

The above graph shows the effects of the txqueuelen upon the average throughput achieved by the 5min tcp flow. As you can see, for txqueuelens of below about 200, the throughput is smaller than that of it over. There is then a step increase in throughput as we move from 200->400 txqueuelen. Past this region, there is very little difference in the achieved throughput.

This graph shows the number of Sendstalls (interface buffer overflows) that each test experiences. The general trend is that as the value of txqueuelen increases, the number of sendstalls decrease; rapidly in the region 100->200. Correlated with the above graph, one can see that as the number of sendstall decreases, the throughput increases.

The below graph shows a scatter plot of the throughput and the number of sendstalls.

Conclusion

For WAN transfers, it was discovered that a setting of 2,000 for the txqueuelen is sufficient to prevent any sendstalls from occuring on the MB-NG network. This matches to back to back results conducted here.

Wed, 23 July, 2003 13:07