next up previous
Next: Preselecting and analysis Up: Data and MC reconstruction Previous: Reconstructed data in UCL

Reconstructing the data on batch

In the /unix/nemo2/n3/soft/N3Nemos_batchrun/local/tools there is a collection of useful PERL tools to run NEMO3 reconstruction in the batch mode. The whole thing lacks detailed documentation and is not very user friendly, but the ideas behind it are very simple.

First on need to prepare a list of files to reconstruct. It can be a list of files on a NFS disk, like /unix/nemo2, or a list of files on HPSS. The first line is a parent directory, the rest are filenames relative to the parent directory. Thus one can use absolute pathnames, if the first line kept empty, or use all files from a directory doing smth like:

pwd > reco.list
ls *.ntup.gz >> reco.list

list2run.pro.pl is a script that processes the list and for each line launches a separate job on the batch system. Please see inside the script for the parameters such as nemor executable location, runlist location etc.

runnemor.pro.pl is the main script to run nemor reconstruction on the batch system. It copies all the data to scratch location, runs nemor, converts it to ROOT with h2root and copies the result to provided TMP path, organising every file reconstruction into the separate folder.

Note, that if HPSS pathnames are used, the user has to be registered as a bbftp user with a privilege of key SSH login (w/o typing a password). Please contact CC support staff in Lyon to arrange those. Currently runnemor.pro.pl configured to be used for CC user vasiliev, from the node pc131.hep.ucl.ac.uk, please put yours instead. The key SSH login w/o typing the password between HEP machines has to be enabled either to make the whole thing work.

Finally copyresults.pl script is coping the results of simulation from TMP subdirs into some permanent location.

Please consider this as an example of how to organise data reconstruction on the batch farm, rather than a finished tool. Taking into account amount of the data and MC files (tens thousands), the reasonable atomisation of the process is inevitable.


next up previous
Next: Preselecting and analysis Up: Data and MC reconstruction Previous: Reconstructed data in UCL
Vladimir Vasiliev 2008-12-02