24h VanillaTCP iperf Transfer - 8th-9th February 2003

Purpose & Method

The purpose of this transfer is to determine the potential problems associated with long transfers. This will be conducted by running a 24h iperf transfer at gig rates over the MBNG network. There will be no other traffic on the network at the time of transfer.

The test will be conducted from lon02 to man03. Both sides will have socket buffers set to 2mbytes. Dynamic Right Sizing will be enabled to give an indication of what a normal user will perceive.

Web100 logs will be collected using logvars at 1 second intervals. Also iperf will also report the throughput every second. As we are looking at the protocol and not the kernel problems, we increased txqueuelen on the interface to 2000.

Results

logs zip: sender recv

One funny thing is that there appears to be periodic drops in the cwnd. This causes slow starts to occur as can be seen from the graph on the right. The effect of the capped value of cwnd is due to the dynamic right sizing.

There were no timeouts during this period. So it could not have been due to slow start.

The effect is also apparent in the increase of the total number of SACKS recv (red), SACKBlocks Rcv (greeen). blue is DSACKS. the increase in SACKS show that there is some information regarding the recv's buffer being sent back to the sender. Each SACK Block specifies that a range of segments has been recieved. As such, if we had two sack blocks every sack, then we are missing a range of segments from the recv. Unless we actually look inside of the sack blocks, we can not determine whether its just one packet or a range of packets.

We send at about 83,000 pkts/sec. Just before the initial dip, we are at about 34sec. This is about 2,822,000 million packets.

From the graph on the right, it shows the number of fast retransmits (The number of invocations of the Fast Retransmit algorithm) that we experience. Everytime we get a decrease in the cwnd value, we also get an invocation of the FastRetrans. As a fast retrans occurs when there are 3 dupacks recieved;

The above graph shows the pktsin (red), pktsout (green), pktsretrans (blue), dupacksin (purple) and ackafterfr (pink). As the same point as when the cwnd shrinks, we can see that there is a decrease in both the number of pkts in and out. Correspondingly, there is also an increase in the number of pktsretrans which is roughly the same value as the number of dupacks in. Notice that the number of dupacks is actually quite high; in one second, we get about 200-300 dupacks for the two peaks. This value is much greater than that of the 3 dupacks required for fast retransmit.

However, there is no indication of sequential dupacks that are required for the fast retransmit. Throughput the entire transfer, we get 2508 fastretransmits.

According to web100, at 34 seconds (when the first dip occurs), we have sent out approximately 2,760,224 pkts. If it were a lost packet, then we should only need to resent a single packet, not almost 300 packets. However, the dip is very periodic; every ~35 seconds. This suggests that there is something which is overfilling in the system.

The maximum number of packets out per second was 104,856, with an average of 79,789 and stdev of 5,372 pkts.

Looking at the rates involved, it would have taken half a rtt for the initial dupack to reach us. Assuming that the second and third come immediately after, this is still roughly half a rtt. Duing this period, the cwnd can no longer grow as the ack clocking is no longer sustained. However, upon reciept of the 3rd dupack, it will still take half an rtt for the retransmitted packet to reach the sink. Given that the cwnd is about 800,000k, and the mtu is about 1500, there is about 533 packets in the system. During the time it takes the for the retransmitted packet to get back to the sink, we would have got acks for about half of them.

Web100 reports that the average throughput was 924.25mbit/sec with a stdev of 62.19mbit/sec. With a peak throughput of 941.98mbit/sec.

There were 2 timeouts and 2 abrupttimeouts. The ave rtt was 6.75msec with stdev of 17.60msec, the max was 3808.24msec. There were no sendstalls as the txqueuelen was set to 2000. There were no quenchrcv either.

Wed, 23 July, 2003 13:07