REQUIREMENTS

In order to obtain information about the network, certain tests may need to be performed. These tests can be broadly classified into two categories:

·        Active: Where test data is sent through the network in order to discover the properties of the end-to-end connection.
·        Passive: Where useful data, such as a required file, is transferred across the network and the resultant transfer properties used as a test benchmark.

This current proposal only implements active forms of monitoring, with the resultant test results stored on the 'sending' machine. The specifics of the storage requirements are discussed later in this paper.

To update the information from network monitoring on the LDAP server, two forms of networking monitoring initialisation are proposed:

·        Streamed: Periodic updates of each network property is updated,
·        Query/Response: Where a 'user' can ask for specific network properties and be supplied an immediate response (or as long as it takes to conduct that measurement).

Our initial implementation shall use a query/response scheme to query network-monitoring information collated from logs and store them in an LDAP server. Future incarnations shall also implement automatic streamed updates either through,

·        Push: Where a LDAP 'backend' will automatically interface with the applications and update accordingly - this may also be considered as 'piping'.
·        Pull: Where information will be stored locally (or remotely) and then periodically polled by the GRIS to update the tree information.

There is current concern over the feasibility of using a 'push' technique in real life GRID environments, as servers may be flooded with so much information they have little resources to do anything else.

The process of data gathering in our proposal is similar to a method adopted by Alex Martin with his ftree program: it is suggested that the programs that gather or produce the data shall be separate from the GRIS and interfaced via a LDAP backend.

For early implementations, the network monitoring application(s) would produce a log file from which the network metric values are extracted and converted to a format (such as LDIF) that can then be published on the LDAP server of that machine. This is shown in Figure 1.

Figure 1: The relation between data produced by network monitoring programs and the LDAP representation of that data.

LDAP APPLICATION

By adopting the scheme shown in Figure 3, the use of any user specified network-monitoring package could be used and the information stored on the LDAP directory - as long as an appropriate backend is implemented on the LDAP. This has the benefit of flexibility as administrators can choose to implement preferred network monitoring tools, however, for the initial testbeds the type and number of applications should be limited to certain widely available tools such as those described later in this document.

Different performance measuring tools produce different results for the "same" test, due to the difference in the exact method and internal parameters they use. For this reason, it may be beneficial to classify performance measurements within the DIT according to the tool used. It is expected that this would produce a deeper tree, but it may be useful to consumers to know how a piece of information was measured so that they can act accordingly.

By allowing the user to specify the network monitoring applications used, it is necessary to identify the type of tools that could be used. This may, for example, pertain to bandwidth measurements, one-way delay etc. In LDAP, we can classify the type of network monitoring data using object classes. By specifying the basic form of data that is required, such as the RTT value of a measurement, we can inherit these attributes and possibly include further information measured by the program such as the deviation of the measured sample. This therefore means that in order to implement the use of a network-monitoring program, one must inherit the metric type (e.g. RTT) object class - adding it as a subclass. It is therefore important to store information such as the network monitoring application used and also the version number of that program. This also has the benefit of storing performance metrics of different tools separately, without conflict. This feature may also become valuable in comparing the quality of different tools under the same conditions.

BACKENDS

The production of log files will no doubt differ in terms of content. It is the purpose of a LDAP backend to convert the log information containing networking metrics into something that can be published on the LDAP. This, in principle implies that every network-monitoring program will require its own LDAP backend to convert the data for publication on to the GRIS.

Is it proposed by groups in the US to adopt a standard NetLogger log file format to store the networking monitoring information. This, if implemented, should allow a single backend to enter network-monitoring information into the LDAP.

Other groups also propose to use 'application wrappers' that can be used to encapsulate each network monitoring application. This has the benefit of a centralised interface to setup update frequency and maintenance. Applications such as NIMI support this, whilst NWS required specifically written modules to gather such information.

INFORMATION STORAGE

Our current proposal is to only store the most recent data for each metric. This may also contain basic statistical information such as min, max and averages. It is recognised that historical information is important for network monitoring and forecasting should also be implemented on a later release.

In terms of historical information storage, the current idea is to include an attribute that contains the number of records (or individual measurements) to keep for that 'branch' and to simply overwrite the earliest entry should the number of records become larger than this value. While this is in principle possible, it may require hacking of the LDAP Protocol. This type of queuing can be envisaged to one that is similar to one used by NWS and would require each entry to be identified by date as part of its dn. The format of this historical recording should also be compatiable with the archiving service that is being developed to store network monitoring readings.

For forecasting, it is proposed that perhaps the information can be set up on the GIIS machine and not locally on the GRIS. This is similar to the methods used for centrally gathering information on NWS networks. Currently, Rich Wolski and Martin Swany in the US are implementing a method of publishing such information on LDAP format, and it is hoped that this can be incorporated into our scheme.

 

Fri, 17 August, 2001 4:59