logvars

The web100 package includes a gui interface called Diagnostic Tool Builder (dtb) that allows access to view any the progress of any TCP connection taking place. The newer versions also incorporates a feature to alter the local window sizes `on-the-fly' using either dtb or the supplied API's (Future versions hope to be able to 'automatically' adjust the window size to maintain optimum performance for the connection).

Whilst dtb is a good tool for watching the connection, what we needed in our tests was a way of outputting the Web100 updates such that we could analyse the transfers at a later date. This is especially important as file transfers less than about 5 to 10 megabytes at Gig speeds would literally terminate before dtb could be opened and used to monitor the connection. As such we created a custom program which allows the output of the TCP-KIS variables into a file which could then be imported into a spreadsheet for easy data manipulation and analysis.

The custom program in question is called lovars, having been heavily influenced by the supplied API demo code 'readvar' included with the Web100 package. The program reads in a list of variables that the user wishes to analyse and requires the Connection ID (CID) of the transfer that the user wishes to monitor. The current version is able to wait for an un-open connection before logging, although this feature is somewhat primitive at present.

The program is run from the command line with the following command,

[ytl@lon02 ytl]$ ./logvars
Logvars polls all Web100 variables and outputs them in tabular format
for easy analysis. Current versions outputs results to STDOUT, so simply
pipe the results to a file.

Version 1.3 for Web100 versions alpha1.2 and above.

Usage: logvars -c {cid,new} [OPTIONS]

-c {cid,'new',+n}
specifies the ConnectionID of the TCP stream to be monitored. cid is
a positive integer value that is the ConnectionID. the literal 'new'
defines the next ConnectionID to open. '+n' defines an offset number
of new connections from the next new one, where n is a number 0-9.
This argument defines the single tcp stream to be logged.

[OPTIONS] are:

-s
terminates monitoring immediately when the ConnectionID specified
connection closes - Default on.

-t <msec>
time-step of monitoring in milliseconds, minimum 10 millisec.

-e
prints out only a summary of the start and the end snapshots.

-o <file>
output path to store the logvars trace. _TODO_

-v <file>
is the path to a plain ASCII, comma seperated file containing a specific
list of Web100 Variables to be monitored. Exclusion of this option will
result in all Variables being monitored. _TODO_

-h
prints this help screen.

Report bugs to <ytl@hep.ucl.ac.uk>.

The program itself monitors the /proc/web100 directory in which information about each connection is temporally stored whilst open. Each connection creates a new directory in /proc/ which represents the CID of each item.

Output of the program is to the screen by default. In order to output the data of the connection to a file, one should use the standard pipe commands in Unix, e.g.

./readvars -c <CID> [OPTIONS]> Web100_output.dat

Upon execution of the program, if the CID does not exist, it will wait until the directory for that CID is created by Web100 in the /proc/web100 directory. The advantage of this method of monitoring is that the entire connection can be monitored from start to finish. Output from the program is in tab-delimited format and can be monitored using a simple cgi script which is currently located at pc35.

Limitations

The current release relays heavily on while loops to both wait for the web100 directory to be created when a TCP/IP connection initiates, and also during transfer polling. This can have a heavy load on the processor of the machine as a consequence. On the Dell Precision machines that i have for MB-NG, the %CPU load as shown by top is 99.9 - however, this is spread over the two processors so it's not so bad. This could have a major effect on a single processor system.

Due to the structure of the program (and inefficient coding), the current release can only provide a time resolution as low as 30millisecs on a Pentium 3 500Mhz. (when it’s outputting to a file rather than the screen). One hopes to decrease this value to allow ’live’ preview of packet arrival. However, this value is also limited by the Web100 software, of which a study is to be performed.

During transfer monitoring of a web100 connection, readvars takes up a lot less processor time - reaching a max of about 16% CPU, with a vage average of about 8%. For example of comparision, iperf utilises approximately the same amount of processor load when conducting a transfer.

Other Programs

Web based visualisatin package of logvars log files. TBA.

Download

Source file in C, current version is 0.4. here. (web100 source files required)
Executable for Linux/Intel (1 processor) machines. here.

Wed, 23 July, 2003 14:14