Personal Miscellaneous TCP/IP GRID Quality of ServiceMulti-Cast  
Background high-speed tcp Transfer Tests Web 100TCP Tuning  

TCP Tuning

Even though there are flaws in the design of TCP in long distance, high speed networks, there are many otehr factors that can lead to poor performance. I detail the important ones below. This focuses on Linux specific hosts - although the ideas and some of the problems defined are applicable for all TCP implementations.

Application Level

  Description
TCP Socket Buffer Sizes

The window size is a very important variable as it limits the amount of data that is buffered from the application level to the TCP protocol for a particular transfer. This involves both the sender and the receiver buffer.

A User's Guide to TCP Windows
TCP Tuning Guide for Distributed Application on Wide Area Networks

Automatic TCP Socket Buffer Size Tuning

A 'simple' way of increasing the transfer rate is to increase the socket buffer size of the transfer. However, this method can lead to memory problems on servers as connections take up unnecessary memory. As such, by automatically adjusting the size of the socket buffer to just the necessary memory will use resources more effeciently.

Dynamic Adjustment of TCP Window Sizes
Automatic TCP Window Tuning and Applications
Automatic TCP Buffer Tuning Jeffrey Semke, Jamshid Mahdavi, and Matthew Mathis, Computer Communications Review, a publication of ACM SIGCOMM, volume 28, number 4, October 1998.
Enabling Network-Ware Applications
Facilitating High-Performace Network Use

Kernel Level

  Description
Memory Limits on Socket Buffer Size  
Txqueuelens

The TXQueueLen determines the maximum size of packets that can be buffered on the egress queue of a linux net interface. Higher queues means that more packets can be buffered and hence not lost. In TCP, this an overflow of this queue will cause loss and hence TCP will enter its congestion control algorithms.

Tests here.

Max backlog

At the opposing end of transport, packets will be queued, once past the driver in a queue which length is defined by this variable max_backlog. As it is a recv side queue, if only has an effect when recving packets. It is similar in function to the txqueuelen variable on the sender.

 

Driver Level

  Description
interrupt Coalesce Values

The interupt coalesce determine the frequency in which the NIC driver tells the kernel that packets are waiting in the input/output queues. The effect of a high value often lowers throughput as packets could be dropped. When set too low, high CPU utilisation occurs due to the necessary context switching between kernel processes to absorb/transmit the packet .

Tests with 4.4.12 e1000

Tests with 5.2.20 e1000

Descriptor Sizes

The descriptor sizes describe how long the queue in on the recv and sender queues on the interface. it is a driver specific parameters and can aid with bursts of packets.

Tests with 5.2.20 e1000

 


TCP Tuning Documents

Other documents relating to tuning transfers.

TCP Tuning Guide for Distributed Application on Wide Area Networks
Enabling High Performance Data Transfers on Hosts also here
Official Networking Documents (RFC) also
MStructure of Management Information for Version 2 of the Simple Network Management Protocol (SNMPv2)
http://www-didc.lbl.gov/tcp-wan.html http://www-didc.lbl.gov/ENABLE/enable-HPDC01.pdf http://www.epm.ornl.gov/~dunigan/netperf/bulk.html (includes some discussion of UDP use)
http://www-didc.lbl.gov/papers/visapult-sc00.pdf

 

Wed, 19 November, 2003 16:20 Previous PageNext Page
 
 
    email me!
2001-2003, Yee-Ting Li, email: ytl@hep.ucl.ac.uk, Tel: +44 (0) 20 7679 1376, Fax: +44 (0) 20 7679 7145
Room D14, High Energy Particle Physics, Dept. of Physics & Astronomy, UCL, Gower St, London, WC1E 6BT