Notes on Monte Carlo generation & ntupling  9.9.2004
====================================================

(1) Building the tar-ball. Sunny has written decent instructions here :

  http://www-cdf.fnal.gov/internal/mcProduction/mcprod_5_3_3_howto_v2.txt

Remember that the dataset definition file must be in the tarball, even if not
yet in CVS.

(2) Submitting MCProd job. Instructions are above and also in CDF-7055. 
A very useful script for CAF submission of MCProd jobs is :

  cdfopr/scripts/submit_MCProd

For example :

  Dave(nucl05)-> ./cdfopr/scripts/submit_MCProd mewkpp caf 1 735

Note that the username at the top of the script and the output file location
may have to be modified by hand first.

At the moment many unnecessary files are transferred back to the specified
output location - but they are small and easily deleted afterwards.

(3) Concatenation -> *** run as "cdfopr" on fcdflnx ***

    (i) Prepare the concatenation tcl files using the following script :

        cdfopr/scripts/make_prod_concat_tcl

        This script seems to want to be run on fcdflnx3. The output directory
        should be specified directly. Also had some problems with setting of
        CAF_CURRENT just before the "cafhostdir" command - needed to comment
        out in the script the setting of CAF_CURRENT (NB - this might just 
        need .cafrc file in the right place).

        For example :
  
          [fcdflnx3:~/testRel/5.3.3] cdfopr> pwd
          /cdf/opr/cdfopr/testRel/5.3.3

          ./make_prod_concat_tcl mewkpp                                                       \      dataset name
                                 fcdfdata091.fnal.gov:/cdf/scratch/cdfdata/ewk/mcprod/mewkpp  \      containing MCProd output
                                 /cdf/opr2/cdfopr/datasets/cdfpewk/mewkpp/requests            \      where tcl fragments will be put
                                 mp                                                                  regular expression - make
                                                                                                     sure log files not accidentally
                                                                                                     selected

    (ii) Run the concatenation job using the following script :

         cdfopr/scripts/submit_prod_concat

         By default (always used so far), it wants to put the output files 
         on fcdfsgi2:

          [fcdfsgi2:ewk/datasets/cdfpewk] cdfopr> pwd
          /cdf/data36a/ewk/datasets/cdfpewk

         Make a sub-directory here with the dataset name if required.

        
         The other defaults are also good. Number of sections should correspond to
         the number of tcl fragments generated in the previous step.
         The pattern should correspond to the stem of the tcl fragment files names. 
	 Modify the script if necessary to :
          (i)  point to a tarball (e.g. fcdflnx3:/cdf/opr/cdfopr/testRel/5.3.1_physics/stnmaker_5.3.1.tgz)
          (ii) 
         Use CONDOR-CAF. For example :

           [fcdflnx3:~/testRel/5.3.1_physics] cdfopr> pwd
           /cdf/opr/cdfopr/testRel/5.3.1_physics

           [fcdflnx3:~/testRel/5.3.1_physics] cdfopr> setenv CAF_CURRENT cafcondor

           [fcdflnx3:~/testRel/5.3.1_physics] cdfopr>  ./cdfopr/scripts/submit_prod_concat -b cdfpewk -d mewkpp        \
                                                       -s 1:1 -m caf                                                   \   1:N_TCL
                                                       -p 2004_09_09.16_22_19                                          \   
                                                       -o fcdfsgi2:/cdf/data36a/ewk/datasets/cdfpewk/mewkpp            \   specify output location explicitly
                                                       -r fcdflnx3:/cdf/opr2/cdfopr/datasets/cdfpewk/mewkpp/requests       Specify tcl location explicitly


           ("-m fcdflnx3" also works if on fcdflnx3)



(4) Ntupling -> *** run as "cdfopr" on fcdflnx ***

     Use the following script :

       cdfopr/scripts/submit_stnmaker_prod

     First, need to :

     (i)  Make a tcl file for the dataset in question. Copy the examples in "cdfopr/tcl".
          There are examples there for datasets that are and are not already in the
          data-handling system.

     (ii) Make sure the tar-ball to be used (which must be specified in the script) 
          contains the new tcl file. This is a useful command (only works on 
          uncompressed tarballs though I think) :

          tar --append --file=stnmaker_5.3.1.tar cdfopr/tcl/mewkpp.tcl

     (iii) Modify the above script to :
           - point to the new tarball
           - add a section corresponding to the new dataset (copy existing sections)
           - specify output directory (currently fcdfdata034:/cdf/scratch/cdfopr/datasets/$book/$dataset)
           - specify CAF email address

     
     Use CONDOR-CAF. For example :

       [fcdflnx3:~/testRel/5.3.1_physics] cdfopr> pwd
       /cdf/opr/cdfopr/testRel/5.3.1_physics

       [fcdflnx3:~/testRel/5.3.1_physics] cdfopr> ./cdfopr/scripts/submit_stnmaker_prod mewkpp 1 21 

       (section range should correspond to the number of files & files/job specified in the tcl
        file).