Tags:
tag this topic
create new tag
view all tags
<span style="background-color: transparent;">---+ </span><span style="background-color: transparent;">%NOTOC%</span><span style="background-color: transparent;"> </span> *xAOD* <span style="background-color: transparent;"> Analysis in !</span><span style="background-color: transparent;">EventLoop</span> <div id="_mcePaste">%TOC% <!-- this line is optional --></div> <div id="_mcePaste">%STARTINCLUDE%</div> <div id="_mcePaste">---+ Recent updates</div> <div id="_mcePaste">29.09.2016</div> <div id="_mcePaste">* creating a new =ALRB_TutorialData= (cern-oct2016)</div> <div id="_mcePaste">12.07.2016</div> <div id="_mcePaste">* updating to use 2.4.14</div> <div id="_mcePaste">08.06.2016</div> <div id="_mcePaste">* switching from =scanDQ2()= to =scanRucio()= in [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Running_on_the_grid][running on the Grid]] (although scanDQ2 uses rucio behind the scene now), more information [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/SampleHandler#Grid_Data_Discovery_Using_Rucio][here]]</div> <div id="_mcePaste">07.06.2016</div> <div id="_mcePaste">* new =$ALRB_TutorialData= (cern-june2016)</div> <div id="_mcePaste">* updated decoration name for jets from "signal" to "mySignal"</div> <div id="_mcePaste">* updated input !DxAOD to use a 20.7 SUSY1 derivation</div> <div id="_mcePaste">* updated Analysis Release to 2.4.9</div> <div id="_mcePaste">05.04.2016</div> <div id="_mcePaste">* fixed small typo, thanks Katharina Ecker!</div> <div id="_mcePaste">04.03.2016</div> <div id="_mcePaste">* new =$ALRB_TutorialData= (chicago-mar2016)</div> <div id="_mcePaste">* using a new !DxAOD as input (DAOD_SUSY1)</div> <div id="_mcePaste">29.02.2016</div> <div id="_mcePaste">* updating to Analysis Base, 2.3.45</div> <div id="_mcePaste">* migrating the [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Return_codes][return code]] check to the new and more central Asg provided tool ( =ANA_CHECK=)</div> <div id="_mcePaste">28.01.2016</div> <div id="_mcePaste">* updating to Analysis Base 2.3.41</div> <div id="_mcePaste">* updating the input xAOD to use a [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/TopDerivations][top derivation]]</div> <div id="_mcePaste">* changing the jet input collection from =AntiKt4LCTopoJets= to =AntiKt4EMTopoJets= (to use a collection actually in the top derivation)</div> <div id="_mcePaste">* updating the GRL from 2012 to 2015</div> <div id="_mcePaste">* creating a new =ALRB_TutorialData= (cern-feb2016)</div> <div id="_mcePaste">30.10.2015</div> <div id="_mcePaste">* updating to Analysis Base 2.3.32</div> <div id="_mcePaste">19.10.2015</div> <div id="_mcePaste">* moving to new [[https://twiki.atlas-canada.ca/bin/view/AtlasCanada/ATLASLocalRootBase2][ATLASLocalRootSetup]] commands, for example =lsetup 'rcsetup Base,xxx'== </div> <div id="_mcePaste">Older updates to this tutorial can be found [[SoftwareTutorialxAODAnalysisInROOT#Older_updates_to_this_tutorial][below]]</div> <div id="_mcePaste">---+ Introduction</div> <div id="_mcePaste">This hands-on tutorial will lead you through some typical analysis tools used for doing an xAOD analysis in the EventLoop framework, with the RootCore build system. Please also note that you can do ROOT-based xAOD analysis in the Athena framework and the CMT build system as [[SoftwareTutorialxAODAnalysisInAthena][documented here]].</div> <div id="_mcePaste">---+ 1. Setup the Analysis Release</div> <div id="_mcePaste">We will work on lxplus, so let's log in to it:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">ssh -Y lxplus.cern.ch</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Note: From a linux laptop you should instead use =-X in the ssh command.</div> <div id="_mcePaste">Let's setup the ATLAS software environment:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">setupATLAS</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">This alias is already defined for you when you log in to lxplus. After you type it you will see a list of commands you can type to setup various ATLAS computing tools.</div> <div id="_mcePaste"><blockquote></div> <div id="_mcePaste">If you are *not* working on lxplus: you will need to define these variables: <br/></div> <div id="_mcePaste"><verbatim></div> <div id="_mcePaste">export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase</div> <div id="_mcePaste">alias setupATLAS='source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh'</div> <div id="_mcePaste">setupATLAS</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste"></blockquote></div> <div id="_mcePaste">We will use the ATLAS [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AnalysisRelease][Analysis Release]] (new for Run 2) to setup our environment and other !RootCore packages with correct package tags (such as those recommended by the CP groups). You can use the Analysis Release if you have access to cvmfs (lxplus, local clusters, machines, and the Grid).</div> <div id="_mcePaste"><blockquote></div> <div id="_mcePaste">It is also possible to checkout all packages in a given Analysis Release and manually compile them on your working machine. This alternative setup is described below. But for this tutorial we recommend you just use the release from cvmfs on lxplus.</div> <div id="_mcePaste"></blockquote></div> <div id="_mcePaste">Let's create a new directory for this analysis from where we will work. When we setup the Analysis Release (or just !RootCore), !RootCore will create a new directory called =RootCoreBin= that will store information about your setup and local packages. Let's create our analysis working directory and change directory there:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">mkdir ROOTAnalysisTutorial</div> <div id="_mcePaste">cd ROOTAnalysisTutorial</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Now let's setup the Analysis Release, we will use a 'flavor' called !AnalysisBase (which is the general purpose one maintained by ASG. Other version and series are documented on [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AnalysisRelease][AnalysisRelease page]]) :</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">lsetup 'rcsetup Base,2.4.14'</div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---++ Define ALRB_TutorialData</div> <div id="_mcePaste">%INCLUDE{"SoftwareTutorialSoftwareBasics"pattern="^.*?(<!--STARTSTTUTORIALDATA-->.*?<!--STOPSTTUTORIALDATA-->).*"}%</div> <div id="_mcePaste">---++ Checking out extra packages</div> <div id="_mcePaste">The command</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc version</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">shows you the package tags in your release, including the full path.</div> <div id="_mcePaste">In case you want a different tag in your release, copy the path and change the package tag. Then use</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc checkout_pkg <new path></div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">If you checked out any package you should run:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">which will find the local version of this package and compile it.</div> <div id="_mcePaste">---++ What to do every time you log in</div> <div id="_mcePaste">The next time you start a new session to start working, you'll have to setup the ATLAS software environment again:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">setupATLAS</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Then navigate to your working directory (the one containing the =RootCoreBin/= directory) and setup the same analysis release:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">lsetup rcsetup</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">The script will automatically setup the same release as you previously defined in that directory (by looking in =RootCoreBin/=).</div> <div id="_mcePaste">And to follow this tutorial define the environment variable =ALRB_TutorialData= as explained [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Define_ALRB_TutorialData][above]].</div> <div id="_mcePaste">---++ Alternative setup on local machine (advanced/optional)</div> <div id="_mcePaste">See the instructions on: <br/></div> <div id="_mcePaste">https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODEDM#On_your_local_machine_optional <br/></div> <div id="_mcePaste">on how to set up !AnalysisBase-2.0.2 on your laptop. The change now is that you should add/update the packages listed above to/in the =packages.txt= file described in the instructions. A similar prescription can be followed for any numbered release.</div> <div id="_mcePaste">---++ How to update to a newer Analysis Release (advanced/optional)</div> <div id="_mcePaste">It is likely that in the course of your analysis development you will need to update to use a newer version of the Analysis Release. It can be very easy to do this (of course this comes with caveats, especially if you have manually checked out specific package tags, and/or made modifications to those packages).</div> <div id="_mcePaste">If you want to update to a newer Analysis Release, first navigate to your working directory (in the case of this tutorial =ROOTAnalysisTutorial/=, where you can see the =RootCoreBin/= directory), and *unsetup* the last Analysis Release:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">lsetup 'rcsetup -u'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Now you need to stop and think a bit ....</div> <div id="_mcePaste">* You may have checked out additional packages on top of the previous Analysis Release you were using (not including your analysis package), as we may have done [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Extra_packages_needed][above]]:</div> <div id="_mcePaste">* Do you still need a local setup of those additional packages?</div> <div id="_mcePaste">* Did you make any local changes to those packages?</div> <div id="_mcePaste">* How will the tags of those packages be different in the newer Analysis Release?</div> <div id="_mcePaste">* Other issues to be aware of is the possibility the tools you are using in your analysis code have changed (if the packages tags are different between the old and new release).</div> <div id="_mcePaste">* A good piece of advice is to join the atlas-sw-pat-releaseannounce@cern.ch mailing list, where new Analysis Release's are announced, along with the relevant updates to that release.</div> <div id="_mcePaste">Second, assuming the points above are not issues, you can simply setup a new Analysis Release from the working directory as usual, find packages, and re-compile. If you wanted to updated to !AnalysisBase 9.9.99 (which doesn't actually exist), you would do:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">lsetup 'rcsetup Base,9.9.99'</div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc clean</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---+ 2. xAOD samples</div> <div id="_mcePaste">In this tutorial we are using an example derived xAOD file. For your analysis it is highly recommended (dare I say required) that you use a derived xAOD (due to corrections that take place in going from the primary xAOD to the derived version). Information about what derivations exist can be found here:</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PhysicsAnalysisWorkBookRel20DxAODs][Physics Analysis Workbook: Which DxAODs exist]]</div> <div id="_mcePaste">---+ 3. Creating your analysis package and algorithm</div> <div id="_mcePaste">---++ Creating your analysis package</div> <div id="_mcePaste">Now it is time to create your own package. !RootCore provides you a script that creates a skeleton package, and we highly recommend you use it. We'll call the package =MyAnalysis=, but you can name it anything you want. From your working directory (the one containing =RootCoreBin/=) execute:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc make_skeleton MyAnalysis</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">If you list the contents of the package directory, you'll see:</div> <div id="_mcePaste">* cmt directory: Where the package Makefile sits. You'll have to modify this Makefile if your package depends on other packages (we'll do that soon).</div> <div id="_mcePaste">* !MyAnalysis directory: This is where all of your C++ header files go.</div> <div id="_mcePaste">* Root directory: This is where all of your C++ source files go. It also holds the =LinkDef.h= file that ROOT uses to build dictionaries.</div> <div id="_mcePaste">Now let's just ask !RootCore to pick up this new package:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Technically you don't have to rebuild after every step, since you won't be running code until the very end of this tutorial, but it is good practice to check in regular intervals whether your code still compiles.</div> <div id="_mcePaste">---++ Creating your Event Loop algorithm</div> <div id="_mcePaste">[[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop][EventLoop]] is a package maintained by the ASG group for looping over events. You first start out by making an algorithm class that will hold your analysis code. We'll call it =MyxAODAnalysis=, but you can name it anything you want. You could create this class by hand, but the !EventLoop package provides a script to create the skeleton of the class for you inside the package you specify (here =MyAnalysis=). From your working directory (the one containing =RootCoreBin/=)</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">$ROOTCOREBIN/user_scripts/EventLoop/make_skeleton MyAnalysis MyxAODAnalysis</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Now we should be able to build our package with our empty algorithm:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">We had to rerun =find_packages= here, because =make_skeleton= updated the dependencies for our package. Normally you can recompile using just =compile.sh=.</div> <div id="_mcePaste">Of course you don't have to use !EventLoop to loop through your events, you can write your own analysis code to do that. But !EventLoop has some nice features that will be shown later, and has full support from the ASG group (meaning this group will help you debug your !EventLoop code, add extra features, etc.).</div> <div id="_mcePaste">If you manually add a new class to your package, you will need to manually add that class to the =MyAnalysis/Root/LinkDef.h= file.</div> <div id="_mcePaste">*Basic notes about !EventLoop:* <br/></div> <div id="_mcePaste">* You will see this script has created a skeleton !EventLoop code for you, with a source code =MyAnalysis/Root/MyxAODAnalysis.cxx= and header code =MyAnalysis/MyAnalysis/MyxAODAnalysis.h== </div> <div id="_mcePaste">* In this !EventLoop algorithm you will have (among others) the following functions:</div> <div id="_mcePaste">* =initialize(): called once, before the first event is executed</div> <div id="_mcePaste">* =execute()=: called once per event</div> <div id="_mcePaste">* =finalize()=: called once, after the final event has completed</div> <div id="_mcePaste">* You may notice there is a return code for each !EventLoop function, to let you know how successful (or not) the function ended</div> <div id="_mcePaste">*Creating member variables in !EventLoop:* <br/></div> <div id="_mcePaste">As with all c++ code, if you want to create a member variable of your algorithm you add that variable (with the appropriate include statement if necessary) to your header code =MyAnalysis/MyAnalysis/MyxAODAnalysis.h=.</div> <div id="_mcePaste">Now what is different with !EventLoop is the distinction between a member variable that will be initialized either when the =initialize()= method of your algorithm is called *or* when the steering macro is executed. The notation for each case is shown below:</div> <div id="_mcePaste">| member variable initialized in =initialize()= | add =//!= after member variable declaration | Ex: m_counter; //! |</div> <div id="_mcePaste">| member variable initialized in steering macro | nothing special | Ex: m_counter; |</div> <div id="_mcePaste">%X% Forgetting the =//!= is the most common problem when using !EventLoop. It tends to manifest itself as a crash when you try to run the job. In effect what it does is to tell !EventLoop that the corresponding variable will not be initialized until =initialize= gets called. So you should add this to all variables in your algorithm, except for configuration parameters.</div> <div id="_mcePaste">%H% The exact meaning of =//!=: What happens internally is that !EventLoop will store a copy of your algorithm object on disk, and then for every sample it reads back a copy, initializes it and processes the samples. Saving objects and reading them back happens through a so called streamer. These get automatically generated for you by ROOT. Adding =//!= to a variable tells ROOT to ignore that variable. If you were to forget the =//!= in this case, ROOT would try to store a copy of the event structure on disk, but since it hasn't been initialized yet it will (most likely) crash.</div> <div id="_mcePaste">---+ 4. Accessing xAOD quantities</div> <div id="_mcePaste">---++The xAODRootAccess package</div> <div id="_mcePaste">In ROOT if you want to do any sort of real analysis you will access the xAOD objects using a package called [[https://svnweb.cern.ch/trac/atlasoff/browser/Control/xAODRootAccess][xAODRootAccess]]. The good news is this is already included in the Analysis Release (so no need to check it out!). The most important class is the =xAOD::TEvent= class which is used to read and write information from the xAOD (which acts similar to the !D3PDReader, for those that used on !D3PDs). We will also use the =xAOD::Init= class to handle any xAOD access "magic" that will happen behind the scenes to the user (basically so we can talk to the interface classes).</div> <div id="_mcePaste">To include this package in our analysis, first we have to update the package dependencies by adding =xAODRootAccess=. So in =MyAnalysis/cmt/Makefile.RootCore= make sure the line =PACKAGE_DEP= looks something like:</div> <div id="_mcePaste">%CODE{"cpp"}%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop xAODRootAccess</div> <div id="_mcePaste">%ENDCODE%</div> <div id="_mcePaste">(it probably already has =EventLoop= listed there).</div> <div id="_mcePaste">Now we are going to add the hooks needed for the =xAODRootAccess= classes to our algorithm. First we'll need to add the =xAODRootAccess= header files to the includes in at the top of =MyAnalysis/Root/MyxAODAnalysis.cxx=:</div> <div id="_mcePaste">%CODE{"cpp"}%</div> <div id="_mcePaste">// Infrastructure include(s):</div> <div id="_mcePaste">#include "xAODRootAccess/Init.h"</div> <div id="_mcePaste">#include "xAODRootAccess/TEvent.h"</div> <div id="_mcePaste">%ENDCODE%</div> <div id="_mcePaste">---+++ Using xAODRootAccess in our !EventLoop package</div> <div id="_mcePaste">Now we will focus on the !EventLoop-specific way of using the TEvent class. In every function of our algorithm that we will need information from the xAOD, either reading or writing it, we need to define a variable of type =xAOD::TEvent= to access this information. In this tutorial were are using !EventLoop, and it has a special way of connecting to this event store, so the syntax is:</div> <div id="_mcePaste">%CODE{"cpp"}%</div> <div id="_mcePaste">xAOD::TEvent* event = wk()->xaodEvent();</div> <div id="_mcePaste">%ENDCODE%</div> <div id="_mcePaste">To prepare ourselves for the rest of the tutorial (because I know where we're going...) add the line above to =MyAnalysis/Root/MyxAODAnalysis.cxx= at the top of the following three functions:</div> <div id="_mcePaste">* =initialize()== </div> <div id="_mcePaste">* =execute()=</div> <div id="_mcePaste">* =finalize()=</div> <div id="_mcePaste">Now we need to tell !EventLoop that we actually want to use the xAODRootAccess in our job. The easiest way to do that is to add the request in the =setupJob function in =MyAnalysis/Root/MyxAODAnalysis.cxx=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::StatusCode MyxAODAnalysis :: setupJob (EL::Job& job)</div> <div id="_mcePaste">{</div> <div id="_mcePaste">// Here you put code that sets up the job on the submission object</div> <div id="_mcePaste">// so that it is ready to work with your algorithm, e.g. you can</div> <div id="_mcePaste">// request the D3PDReader service or add output files. Any code you</div> <div id="_mcePaste">// put here could instead also go into the submission script. The</div> <div id="_mcePaste">// sole advantage of putting it here is that it gets automatically</div> <div id="_mcePaste">// activated/deactivated when you add/remove the algorithm from your</div> <div id="_mcePaste">// job, which may or may not be of value to you.</div> <div id="_mcePaste">// let's initialize the algorithm to use the xAODRootAccess package</div> <div id="_mcePaste">job.useXAOD ();</div> <div id="_mcePaste">xAOD::Init(); // call before opening first file</div> <div id="_mcePaste">return EL::StatusCode::SUCCESS;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now let's slightly modify the =initialize()= function in =MyAnalysis/Root/MyxAODAnalysis.cxx= to print out the number of events, modify it so it looks like this (basically adding the =Info= line, everything else should be there):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::StatusCode MyxAODAnalysis :: initialize ()</div> <div id="_mcePaste">{</div> <div id="_mcePaste">// Here you do everything that you need to do after the first input</div> <div id="_mcePaste">// file has been connected and before the first event is processed,</div> <div id="_mcePaste">// e.g. create additional histograms based on which variables are</div> <div id="_mcePaste">// available in the input files. You can also create all of your</div> <div id="_mcePaste">// histograms and trees in here, but be aware that this method</div> <div id="_mcePaste">// doesn't get called if no events are processed. So any objects</div> <div id="_mcePaste">// you create here won't be available in the output if you have no</div> <div id="_mcePaste">// input events.</div> <div id="_mcePaste">xAOD::TEvent* event = wk()->xaodEvent(); // you should have already added this as described before</div> <div id="_mcePaste">// as a check, let's see the number of events in our xAOD</div> <div id="_mcePaste">Info("initialize()", "Number of events = %lli", event->getEntries() ); // print long long int</div> <div id="_mcePaste">return EL::StatusCode::SUCCESS;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">We've updated our package dependencies, so from our working directory we have to rerun =rc find_packages= before compiling:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">More information about using !EventLoop to access xAOD quantities can be read here: <br/></div> <div id="_mcePaste">[[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#Access_the_Data_Through_xAOD_EDM][EventLoop: Access the Data through the xAOD EDM]].</div> <div id="_mcePaste">---+++ Return codes</div> <div id="_mcePaste">You may have noticed that the !EventLoop functions return a code (a !StatusCode, similar to the !StatusCode of Athena). The options are failure, success, and recoverable. Actually every time you use =xAOD::TEvent= (or xAOD::TStore) you need to check this return code thing. You should always check the status of these return codes, these basically make sure your code is doing what you want it to do ... however in ROOT if you don't explicitly check these return codes you will only be pinged only with warnings when you execute your code (it will compile fine) with error messages like:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">Warning in <xAOD::TReturnCode>:</div> <div id="_mcePaste">Warning in <xAOD::TReturnCode>: Unchecked return codes encountered during the job</div> <div id="_mcePaste">Warning in <xAOD::TReturnCode>: Number of unchecked successes: 502</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Since there are many many commands we will execute that have a return code, let's use a more central tool that will check the return codes for us (this works with any type of status code return).At the top of our source code =MyAnalysis/Root/MyxAODAnalysis.cxx= add this header</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// ASG status code check</div> <div id="_mcePaste">#include <AsgTools/MessageCheck.h></div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Since we are now relying on a new package we have to update our package dependencies in =MyAnalysis/cmt/Makefile.RootCore= to include:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] AsgTools</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Let's start with this good practice right away and take advantage of this tool. In the last section we modified the =setupJob()= function in =MyAnalysis/Root/MyxAODAnalysis.cxx= to initialize using the xAOD, in fact this returns a status code that we should check. So in =setupJob()= function in =MyAnalysis/Root/MyxAODAnalysis.cxx= replace this line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">xAOD::Init(); // call before opening first file</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">with this line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">ANA_CHECK_SET_TYPE (EL::StatusCode); // set type of return code you are expecting (add to top of each function once)</div> <div id="_mcePaste">ANA_CHECK(xAOD::Init());</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">We will take advantage of this =ANA_CHECK= throughout the rest of the tutorial.</div> <div id="_mcePaste">Add the following line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">ANA_CHECK_SET_TYPE (EL::StatusCode); // set type of return code you are expecting (add to top of each function once)</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">to the top of each of the following functions in =MyAnalysis/Root/MyxAODAnalysis.cxx=:</div> <div id="_mcePaste">* initialize()</div> <div id="_mcePaste">* execute()</div> <div id="_mcePaste">* finalize()</div> <div id="_mcePaste">(I know already we will need to check return codes in each of these functions later on in this tutorial... I'd rather get us setup now than forget later :-) )</div> <div id="_mcePaste">*Alternative:* <br/></div> <div id="_mcePaste">Alternatively you can check each return code 'by-hand', without using the macro we have created. For the event-level material we will cover later ([[SoftwareTutorialxAODAnalysisInROOT#Event_level][see Section 6 below]]) that would look something like this:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">if( ! event->retrieve( eventInfo, "EventInfo").isSuccess() ){</div> <div id="_mcePaste">Error("execute()", "Failed to retrieve event info collection. Exiting." );</div> <div id="_mcePaste">return EL::StatusCode::FAILURE;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Instead of something like this using this macro:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">ANA_CHECK(event->retrieve( eventInfo, "EventInfo") );</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">---++ Knowing what information is in the xAOD</div> <div id="_mcePaste">One question everyone will have is: how do I know what information/variables are actually stored in my xAOD for each container type? You can be sure for "particles" (inheriting from =IParticle=) you will have things like pt, eta, and phi. But what other variables are associated to the different containers? We'll try to answer that question...</div> <div id="_mcePaste">---+++ Containers and key names</div> <div id="_mcePaste">In order to "retrieve" the information stored in the xAOD containers we need to know the container type and the container key name. We will use a handy script called =checkxAOD.py= which is available in both the Athena-world and when using the !RootCore Analysis Release with 2.X.Y where Y>=29. If you have an xAOD file, say =xAOD.pool.root= and want to know the containers and associated key names do the following:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">checkxAOD.py xAOD.pool.root</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">(Note: You need to replace the fake =xAOD.pool.root= with the full path to an xAOD sample, for example =$ALRB_TutorialData/p2622/mc15_13TeV.410000.PowhegPythiaEvtGen_P2012_ttbar_hdamp172p5_nonallhad.merge.DAOD_SUSY1.e3698_s2608_s2183_r7725_r7676_p2622/DAOD_SUSY1.08377960._000012.pool.root.1=)</div> <div id="_mcePaste">The last column will show you the xAOD container names and types. When you are retrieving information you usually need to know the container type (for example =xAOD::CaloCluster=) and the key name for the particular instance of that container you are interested in (for example ="egammaClusters"=).</div> <div id="_mcePaste">In your analysis you can ignore the "Aux" containers (for Auxiliary store), these hold some behind-the-scenes magic. You can also "mostly" ignore the versions like =_v1=.</div> <div id="_mcePaste">Most information in the xAOD is stored and retrieved via the Auxillary store. The user doesn't need to worry about this Auxillary store, and only interacts with the interface (so something called !TEvent for ROOT standalone analysis.) So now you should know the container type and key name. If you use the wrong key name the code will compile, but it will crash at run-time.</div> <div id="_mcePaste">---+++ Variables inside the containers</div> <div id="_mcePaste">Now to know what variables are associated to this container, the trick I use at the moment (again maybe something official will come along...) is to use interactive ROOT. So back into your Analysis Release shell (with ROOT automatically setup), you can simply do this:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">root -l xAOD.pool.root</div> <div id="_mcePaste">root [1] CollectionTree->Print("egammaClusters*")</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">You will get a printout of all variables you can access from that container (aka collection). Note that the variable name you will use in your code is the one that comes after the ".", so for example you might see:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">egammaClusters.rawEta : vector<float></div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">So in your analysis code (after setting up the TEvent and interface magic), you can access this variable from the xAOD::CaloCluster object by calling =rawEta=.</div> <div id="_mcePaste">If you try to request a variable associated to a particular xAOD object that does not exist the code will crash at compile-time, complaining the xAOD object =has no member named ‘whatever’=.</div> <div id="_mcePaste">---+++ Accessing object variables</div> <div id="_mcePaste">To access variables associated to objects in your analysis code there are often special accessor functions available to help. These are associated to the objects of the containers. At the moment the best place to find these accessors is by browsing the code. All of the xAOD EDM classes live in [[https://svnweb.cern.ch/trac/atlasoff/browser/Event/xAOD][atlasoff/Event/xAOD]], and the naming should be obvious to find the object type you are interested in. Alternatively you can access the variables directly without using the accessors, but this is slow as it depends on string comparisons.</div> <div id="_mcePaste">Here is one example that might clarify these points (you don't have to copy and paste this anywhere, it's just a qualitative description). Let's say you have a pointer to an xAOD muon object for a particular event, called =( *muon_itr)*= <strong> (we'll actually do this later on in complete detail), and now we want to access the ptcone20 isolation variable for this muon. </strong></div> <strong> <div id="_mcePaste">To access the variable with the help of the muon accessor you can do:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">float muPtCone20 = 0.; // your variable that will be filled after calling the isolation function</div> <div id="_mcePaste">(*muon_itr)->isolation(muPtCone20, xAOD::Iso::ptcone20); // second arg is an enum defined in xAODPrimitives/IsolationType.h</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Alternatively you can access that same variable by simply doing:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">(*muon_itr)->auxdata< float >("ptcone20");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">For the muons you can find the complete list of accessors in the [[https://svnweb.cern.ch/trac/atlasoff/browser/Event/xAOD/xAODMuon/trunk/xAODMuon/versions/Muon_v1.h][xAOD Muon class]] (version 1)</div> <div id="_mcePaste">---+ 5. Creating and running our steering macro</div> <div id="_mcePaste">To actually run this !EventLoop algorithm we need some steering code. This can be a root macro in either C++ or python or some compiled C++ code. For this tutorial we will create a C++ macro.</div> <div id="_mcePaste">We will use another ASG tool called [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/SampleHandler][SampleHandler]] which is a nice tool that allows for easy sample management. In this example we will create and configure a !SampleHandler object. We will specify the path to the main directory, under which there could be several subdirectories (typically representing datasets) and within those the individual input files. Here we will tell !SampleHandler we are only interested in one input xAOD file (specified by the exact name, but wildcards are accepted to use several specific inputs). More information and options for using !SampleHandler to 'find' your data is found on the dedicated [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/SampleHandler][SampleHandler]] wiki.</div> <div id="_mcePaste">First let's create a =Run= directory for it. From your main working directory execute:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">mkdir Run</div> <div id="_mcePaste">cd Run</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">And in that directory we will create a new file called =ATestRun.cxx=. Fill this new file, =ATestRun.cxx=, with the following:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">void ATestRun (const std::string& submitDir)</div> <div id="_mcePaste">{</div> <div id="_mcePaste">//===========================================</div> <div id="_mcePaste">// FOR ROOT6 WE DO NOT PUT THIS LINE</div> <div id="_mcePaste">// (ROOT6 uses Cling instead of CINT)</div> <div id="_mcePaste">// Load the libraries for all packages</div> <div id="_mcePaste">// gROOT->Macro("$ROOTCOREDIR/scripts/load_packages.C");</div> <div id="_mcePaste">// Instead on command line do:</div> <div id="_mcePaste">// > root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste">// The above works for ROOT6 and ROOT5</div> <div id="_mcePaste">//==========================================</div> <div id="_mcePaste">// Set up the job for xAOD access:</div> <div id="_mcePaste">xAOD::Init().ignore();</div> <div id="_mcePaste">// create a new sample handler to describe the data files we use</div> <div id="_mcePaste">SH::SampleHandler sh;</div> <div id="_mcePaste">// scan for datasets in the given directory</div> <div id="_mcePaste">// this works if you are on lxplus, otherwise you'd want to copy over files</div> <div id="_mcePaste">// to your local machine and use a local path. if you do so, make sure</div> <div id="_mcePaste">// that you copy all subdirectories and point this to the directory</div> <div id="_mcePaste">// containing all the files, not the subdirectories.</div> <div id="_mcePaste">// use SampleHandler to scan all of the subdirectories of a directory for particular MC single file:</div> </strong> <div id="_mcePaste"> *const char* inputFilePath = gSystem->ExpandPathName ("$ALRB_TutorialData/p2622/");</div> <div id="_mcePaste">SH::ScanDir().filePattern("DAOD_SUSY1.08377960._000012.pool.root.1").scan(sh,inputFilePath);</div> <div id="_mcePaste">// set the name of the tree in our files</div> <div id="_mcePaste">// in the xAOD the TTree containing the EDM containers is "CollectionTree"</div> <div id="_mcePaste">sh.setMetaString ("nc_tree", "CollectionTree");</div> <div id="_mcePaste">// further sample handler configuration may go here</div> <div id="_mcePaste">// print out the samples we found</div> <div id="_mcePaste">sh.print ();</div> <div id="_mcePaste">// this is the basic description of our job</div> <div id="_mcePaste">EL::Job job;</div> <div id="_mcePaste">job.sampleHandler (sh); // use SampleHandler in this job</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optMaxEvents, 500); // for testing purposes, limit to run over the first 500 events only!</div> <div id="_mcePaste">// add our algorithm to the job</div> <div id="_mcePaste">MyxAODAnalysis *alg = new MyxAODAnalysis;* </div> <strong> <div id="_mcePaste">// later on we'll add some configuration options for our algorithm that go here</div> <div id="_mcePaste">job.algsAdd (alg);</div> <div id="_mcePaste">// make the driver we want to use:</div> <div id="_mcePaste">// this one works by running the algorithm directly:</div> <div id="_mcePaste">EL::DirectDriver driver;</div> <div id="_mcePaste">// we can use other drivers to run things on the Grid, with PROOF, etc.</div> <div id="_mcePaste">// process the job using the driver</div> <div id="_mcePaste">driver.submit (job, submitDir);</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Read over the comments carefully to understand what is happening. Notice that we will only run over the first 500 events (for testing purposes). Obviously if you were doing a real analysis you would want to remove that statement to run over all events in a sample.</div> <div id="_mcePaste">Ok, now the big moment has come. Within your =Run= directory execute your !ATestRun.cxx macro with root:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">You can quit ROOT by typing ".q" at the ROOT prompt.</div> <div id="_mcePaste">%I%Note that =submitDir= is the directory where the output of your job is stored. If you want to run again, you either have to remove that directory or pass a different name into =ATestRun.cxx=.</div> <div id="_mcePaste"><blockquote></div> </strong> <div id="_mcePaste"> **Please Note:* For actual work you should give more thought about how you want to organize your submission directories. Since those contain the output of your jobs you probably want to keep old versions around, so that you can compare. Also, you may not want to keep them in your source directory tree, but instead put them into a separate data directory tree. However, try to avoid putting them inside a RootCore managed package. Doing so may result in them getting copied around when submitting to the grid or a batch system.</div> <div id="_mcePaste"></blockquote></div> <div id="_mcePaste">---++ Alternative: Run the job from a compiled application (optional)</div> <div id="_mcePaste">In order to debug problems with the code, it is often not practical to run the job from ROOT's interpreter. So, if you encounter any problems, you should create a directory like =MyAnalysis/util/=, and in there put an executable source file like =MyAnalysis/util/testRun.cxx= for instance, with the content:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODRootAccess/Init.h"</div> <div id="_mcePaste">#include "SampleHandler/SampleHandler.h"</div> <div id="_mcePaste">#include "SampleHandler/ScanDir.h"</div> <div id="_mcePaste">#include "SampleHandler/ToolsDiscovery.h"</div> <div id="_mcePaste">#include "EventLoop/Job.h"</div> <div id="_mcePaste">#include "EventLoop/DirectDriver.h"</div> <div id="_mcePaste">#include "SampleHandler/DiskListLocal.h"</div> <div id="_mcePaste">#include <TSystem.h></div> <div id="_mcePaste">#include "MyAnalysis/MyxAODAnalysis.h"</div> <div id="_mcePaste">int main( int argc, char* argv[] ) {</div> <div id="_mcePaste">// Take the submit directory from the input if provided:</div> <div id="_mcePaste">std::string submitDir = "submitDir";</div> <div id="_mcePaste">if( argc > 1 ) submitDir = argv[ 1 ];</div> <div id="_mcePaste">// Set up the job for xAOD access:</div> <div id="_mcePaste">xAOD::Init().ignore();</div> <div id="_mcePaste">// Construct the samples to run on:</div> <div id="_mcePaste">SH::SampleHandler sh;</div> <div id="_mcePaste">// use SampleHandler to scan all of the subdirectories of a directory for particular MC single file:</div> <div id="_mcePaste">const char* inputFilePath = gSystem->ExpandPathName ("$ALRB_TutorialData/p2622/");</div> <div id="_mcePaste">SH::ScanDir().filePattern("DAOD_SUSY1.08377960._000012.pool.root.1").scan(sh,inputFilePath);</div> <div id="_mcePaste">// Set the name of the input TTree. It's always "CollectionTree"</div> <div id="_mcePaste">// for xAOD files.</div> <div id="_mcePaste">sh.setMetaString( "nc_tree", "CollectionTree" );</div> <div id="_mcePaste">// Print what we found:</div> <div id="_mcePaste">sh.print();</div> <div id="_mcePaste">// Create an EventLoop job:</div> <div id="_mcePaste">EL::Job job;</div> <div id="_mcePaste">job.sampleHandler( sh );</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optMaxEvents, 500);</div> <div id="_mcePaste">// Add our analysis to the job:</div> <div id="_mcePaste">MyxAODAnalysis* alg = new MyxAODAnalysis();</div> <div id="_mcePaste">job.algsAdd( alg );</div> <div id="_mcePaste">// Run the job using the local/direct driver:</div> <div id="_mcePaste">EL::DirectDriver driver;</div> <div id="_mcePaste">driver.submit( job, submitDir );</div> <div id="_mcePaste">return 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">From your working directory recompile everything:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">And you can execute the compiled steering macro by doing:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">testRun submitDir</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Which should give the same results as shown previously by running interactively in ROOT.</div> <div id="_mcePaste">---+ 6. Objects and tools for analysis</div> <div id="_mcePaste">This section covers the various reconstructed objects and associated tools that you may use for your analysis. For the actual tutorial, which is of limited time, please try a handful of these, but be aware of the full content of this section, as it may come in handy in your own analysis.</div> <div id="_mcePaste">---++ Event-level</div> <div id="_mcePaste">---+++ General event information</div> <div id="_mcePaste">Now let's get some event-level information, like how many events we've processed, and if the event is data or MC.</div> <div id="_mcePaste">First add the xAOD EDM event package to our =MyAnalysis/cmt/Makefile.RootCore= file:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] xAODEventInfo</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">In our analysis header file =MyAnalysis/MyAnalysis/MyxAODAnalysis.h=, let's define a variable that we will use to count the number of events we have processed, you can put it under public:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">int m_eventCounter; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">(Again, remember the =//!=!)</div> <div id="_mcePaste">Now, to our analysis source code =MyAnalysis/Root/MyxAODAnalysis.cxx= let's include the corresponding xAOD EDM class header file (add it near the top with all the other include statements):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// EDM includes:</div> <div id="_mcePaste">#include "xAODEventInfo/EventInfo.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And in the =initialize()= method let's initialize our event counting variable to zero:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// count number of events</div> <div id="_mcePaste">m_eventCounter = 0;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And one last thing - let's actually do something event-by-event, which happens in the =execute()= method:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// print every 100 events, so we know where we are:</div> <div id="_mcePaste">if( (m_eventCounter % 100) ==0 ) Info("execute()", "Event number = %i", m_eventCounter );= </div> <div id="_mcePaste">m_eventCounter++;</div> <div id="_mcePaste">//----------------------------</div> <div id="_mcePaste">// Event information</div> <div id="_mcePaste">//---------------------------</div> <div id="_mcePaste">const xAOD::EventInfo* eventInfo = 0;</div> <div id="_mcePaste">ANA_CHECK(event->retrieve( eventInfo, "EventInfo"));</div> <div id="_mcePaste"></div> <div id="_mcePaste">// check if the event is data or MC</div> <div id="_mcePaste">// (many tools are applied either to data or MC)</div> <div id="_mcePaste">bool isMC = false;</div> <div id="_mcePaste">// check if the event is MC</div> <div id="_mcePaste">if(eventInfo->eventType( xAOD::EventInfo::IS_SIMULATION ) ){</div> <div id="_mcePaste">isMC = true; // can do something with this later</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Since we've updated our package dependencies from our working directory we have to rerun =rc find_packages before compiling:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---+++ The Good Runs List</div> <div id="_mcePaste">The Good Runs List (GRL) is an xml file that selects good luminosity blocks from within the data runs (spanning 1-2 minutes of data-taking). Luminosity blocks which are not listed in this xml file are considered bad, and should not be used in your analysis.</div> <div id="_mcePaste">We will need to use a new tool that searches this xml file and returns a boolean as to whether the luminosity block provided is good or not. This tool lives in the [[https://svnweb.cern.ch/trac/atlasoff/browser/DataQuality/GoodRunsLists][GoodRunsList]] package, so we must tell !RootCore and our analysis where to find it. Add the following to your =MyAnalysis/cmt/Makefile.RootCore= file:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] GoodRunsLists</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">Going to our header file, =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= , near the top we will add the !GoodRunsList package header:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// GRL</div> <div id="_mcePaste">#include "GoodRunsLists/GoodRunsListSelectionTool.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And still in =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= in our class definition (so after the lines =class !MyxAODAnalysis : public EL::Algorithm=) add a new member variable:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">GoodRunsListSelectionTool *m_grl; //!* </div> <strong> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now let's move to our source code =MyAnalysis/Root/MyxAODAnalysis.cxx=, and initialize the tool for usage in our =initialize()= method:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// GRL</div> <div id="_mcePaste">m_grl = new GoodRunsListSelectionTool("GoodRunsListSelectionTool");</div> </strong> <div id="_mcePaste"> *const char* GRLFilePath = "$ALRB_TutorialData/data15_13TeV.periodAllYear_DetStatus-v73-pro19-08_DQDefects-00-01-02_PHYS_StandardGRL_All_Good_25ns.xml";</div> <div id="_mcePaste">const char* fullGRLFilePath = gSystem->ExpandPathName (GRLFilePath);</div> <div id="_mcePaste">std::vector<std::string> vecStringGRL;</div> <div id="_mcePaste">vecStringGRL.push_back(fullGRLFilePath);</div> <div id="_mcePaste">ANA_CHECK(m_grl->setProperty( "GoodRunsListVec", vecStringGRL));</div> <div id="_mcePaste">ANA_CHECK(m_grl->setProperty("PassThrough", false)); // if true (default) will ignore result of GRL and will just pass all events</div> <div id="_mcePaste">ANA_CHECK(m_grl->initialize());</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">At the top of your source code (with all the other includes) we have to include the appropriate header file to use this =gSystem= class:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include <TSystem.h></div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Next in the =execute()= method, let's check if the event is in fact a data event, and if it is, then retrieve the result of the GRL for this event. *After* you have retrieved the !EventInfo object (done above) and after you have set the flag =isMC= (also shown above), add these lines:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// if data check if event passes GRL</div> <div id="_mcePaste">if(!isMC){ // it's data!</div> <div id="_mcePaste">if(!m_grl->passRunLB( *eventInfo)){* </div> <strong> <div id="_mcePaste">return EL::StatusCode::SUCCESS; // go to next event</div> <div id="_mcePaste">}</div> <div id="_mcePaste">} // end if not MC</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And last but not least, remember to clean up the memory in =finalize()=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// GRL</div> <div id="_mcePaste">if (m_grl) {</div> <div id="_mcePaste">delete m_grl;</div> <div id="_mcePaste">m_grl = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Finally let's compile! Note we have to run =rc find_packages= again, as we updated the package dependencies in our Makefile.</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Choices of GRLs are described on this page: [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/GoodRunListsForAnalysisRun2][Good Run Lists for Analysis]]</div> <div id="_mcePaste">To add: Determine the integrated luminosity of our data sample <br/></div> <div id="_mcePaste">---+++ Removing additional detector imperfections</div> <div id="_mcePaste">The GRL helps remove detector problems that affect entire luminosity blocks (1-2 minutes of data-taking). Note that even one luminosity block can be thousands of events. To avoid throwing away perfectly fine events there are some additional event-level detector flags, to help single events that are plagued with some detector problem. For the 2015 data-taking period there are three flags to remove problematic events:</div> <div id="_mcePaste">* due to the liquid argon system</div> <div id="_mcePaste">* due to the tile calorimeter system</div> <div id="_mcePaste">* due to the SCT inner detector system</div> <div id="_mcePaste">* due to incomplete events (event information missing after TTC restarts)</div> <div id="_mcePaste">Here we will show you how to remove these problematic events. Here we assume the !EventInfo EDM classes have already been defined and used from instructions at the top of this [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Basic_event_information][page]].</div> <div id="_mcePaste">We can count the number of 'clean' events, by defining a member variable in our header file =MyAnalysis/MyAnalysis/MyxAODAnalysis.h=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">int m_numCleanEvents; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now in our source code let's initialize this event count to zero, so in the =initialize()= method in =MyAnalysis/Root/MyxAODAnalysis.cxx= simply add this line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">m_numCleanEvents = 0;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now for the cleaning! This is something done event-by-event (think =execute()= method!), and only for data:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//------------------------------------------------------------</div> <div id="_mcePaste">// Apply event cleaning to remove events due to</div> <div id="_mcePaste">// problematic regions of the detector</div> <div id="_mcePaste">// or incomplete events.</div> <div id="_mcePaste">// Apply to data.</div> <div id="_mcePaste">//------------------------------------------------------------</div> <div id="_mcePaste">// reject event if:</div> <div id="_mcePaste">if(!isMC){</div> <div id="_mcePaste">if( (eventInfo->errorState(xAOD::EventInfo::LAr)==xAOD::EventInfo::Error ) || (eventInfo->errorState(xAOD::EventInfo::Tile)==xAOD::EventInfo::Error ) || (eventInfo->errorState(xAOD::EventInfo::SCT)==xAOD::EventInfo::Error ) || (eventInfo->isEventFlagBitSet(xAOD::EventInfo::Core, 18) ) )</div> <div id="_mcePaste">{</div> <div id="_mcePaste">return EL::StatusCode::SUCCESS; // go to the next event</div> <div id="_mcePaste">} // end if event flags check</div> <div id="_mcePaste">} // end if the event is data</div> <div id="_mcePaste">m_numCleanEvents++;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">You can print out the final number of clean events from the =finalize()= method:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">Info("finalize()", "Number of clean events = %i", m_numCleanEvents);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">This will do nothing for MC files, but could actually remove some events from the 2015 data samples (you could try using the data xAOD file as input in your steering macro).</div> <div id="_mcePaste">You can compile your package as before (we do not need to run =rc find_packages= again as we did not update the package dependencies), and you can run your job in ROOT from the =Run/= directory as before. From your working directory:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run/</div> <div id="_mcePaste">rm -rf submitDir/</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---++ Jets</div> <div id="_mcePaste">Let's create a basic loop over a jet container, for the !AntiKt4EMTopoJets jet collection:</div> <div id="_mcePaste">First, let's add the jet xAOD EDM package to =MyAnalysis/cmt/Makefile.RootCore=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] xAODJet</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">Now in our source code =MyAnalysis/Root/MyxAODAnalysis.cxx=, add the jet xAOD EDM container class to =include= statement near the top:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODJet/JetContainer.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Still in the source code, in =execute()= (run for every event) retrieve the jet container and loop over all jets for this event:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// get jet container of interest</div> </strong> <div id="_mcePaste"> *const xAOD::JetContainer* jets = 0;</div> <div id="_mcePaste">ANA_CHECK(event->retrieve( jets, "AntiKt4EMTopoJets" ));</div> <div id="_mcePaste">Info("execute()", " number of jets = %lu", jets->size());</div> <div id="_mcePaste">// loop over the jets in the container</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_itr = jets->begin();</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_end = jets->end();</div> <div id="_mcePaste">for( ; jet_itr != jet_end; ++jet_itr ) {</div> <div id="_mcePaste">Info("execute()", " jet pt = %.2f GeV", (( *jet_itr)->pt() * 0.001)); // just to print out something* </div> <strong> <div id="_mcePaste">} // end for loop over jets</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">If you are not sure of the container type and/or container name see [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Knowing_what_information_is_in_t][above]].</div> <div id="_mcePaste">You can do the usual !RootCore thing to update/find the (new) package dependencies and compile:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---+++ Jet cleaning tool</div> <div id="_mcePaste">Let's add the jet cleaning tool to our code. This jet selector tool applies the jet cleaning described on this page: [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HowToCleanJets2015][How to clean jets 2015]]. It is a cleaning that you could apply to jets in both data and MC (see the dedicated wiki page for the recommended use).</div> <div id="_mcePaste">Below describe the steps necessary to including this jet cleaning in our analysis code:</div> <div id="_mcePaste">First, add the package dependency to =MyAnalysis/cmt/Makefile.RootCore=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] JetSelectorTools</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">Second, add the class dependency to our header file, at the top near where all the other "includes" are, we will add:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "JetSelectorTools/JetCleaningTool.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Still in the header file, let's create an instance of the tool. In the class definition (so somewhere within =class !MyxAODAnalysis : public EL::Algorithm{...}=) :</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">JetCleaningTool *m_jetCleaning; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now moving to our source code,=MyAnalysis/Root/MyxAODAnalysis.cxx= ,we need to initialize this tool, in the =initialize()= method add these lines:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// initialize and configure the jet cleaning tool</div> <div id="_mcePaste">m_jetCleaning = new JetCleaningTool("JetCleaning");</div> <div id="_mcePaste">m_jetCleaning->msg().setLevel( MSG::DEBUG );</div> <div id="_mcePaste">ANA_CHECK(m_jetCleaning->setProperty( "CutLevel", "LooseBad"));</div> <div id="_mcePaste">ANA_CHECK(m_jetCleaning->setProperty("DoUgly", false));</div> <div id="_mcePaste">ANA_CHECK(m_jetCleaning->initialize());</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Then, in =finalize()=, don't forget to delete the tool from memory.</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">if( m_jetCleaning ) {</div> <div id="_mcePaste">delete m_jetCleaning;</div> <div id="_mcePaste">m_jetCleaning = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">In the above we have set a DEBUG level of text information to be printed out, and we have set the cut level to "MediumBad", there are several other options; the choice however is usually analysis dependent.</div> </strong> <div id="_mcePaste"> *Now in the =execute()= method, just *before* we loop over the jets let's add an integer variable to count the number of good jets passing this cleaning:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">int numGoodJets = 0;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And *in* the loop over the jets, check if the jet passes the cleaning criteria by adding:</div> <div id="_mcePaste">%CODE{"cpp"}%</div> <div id="_mcePaste">if( !m_jetCleaning->accept( **jet_itr )) continue; //only keep good clean jets* </div> <strong> <div id="_mcePaste">numGoodJets++;</div> <div id="_mcePaste">%ENDCODE%</div> <div id="_mcePaste">After the jet loop you could print out the number of good jets.</div> <div id="_mcePaste"><blockquote></div> <div id="_mcePaste">Apart from the setup and configuration, here is a quick comparison of how this tool was called for jets using !D3PDs and how it is now called using xAODs:</div> </strong> <div id="_mcePaste"> **D3PDs* <br/></div> <div id="_mcePaste">We showed you how to do this in the !D3PD-version of this tutorial (using the !D3PDReader) here: [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/SoftwareTutorialAnalyzingD3PDsInROOT#Using_ObjectSelectors][SoftwareTutorialAnalyzingD3PDsInROOT: Using Object Selectors]]</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">my_JetCleaningTool->accept(event->jet_AntiKt4EMTopo[i].eta(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].NegativeE(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].hecf(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].HECQuality(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].emfrac(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].sumPtTrk_pv0_500MeV(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].LArQuality(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].Timing(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].fracSamplingMax(),</div> <div id="_mcePaste">event->jet_AntiKt4EMTopo[i].AverageLArQF() ))</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">which was error prone (in fact for a few editions of the tutorial I realized we had one of the arguments wrong, but everything still ran!!).</div> <div id="_mcePaste">*xAODs* <br/></div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">m_jetCleaning->accept( **jet_itr );* </div> <strong> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">%BLUE% Ta-da!%ENDCOLOR% Isn't that just SO much nicer :-)</div> <div id="_mcePaste"></blockquote></div> <div id="_mcePaste">---+++ Jet energy resolution</div> <div id="_mcePaste">This tool takes jets as inputs and smears the MC such that the energy resolution matches that of data (note in 2015 the data and MC agreed pretty well, so this smearing was not necessary). The tool also provides the uncertainty associated with the energy resolution. More official information about this tool (including configuration options) can be found on the following jet pages:</div> <div id="_mcePaste"><!--[[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/JetEnergyResolutionProvider2012][Jet energy resolution provider 2012]]--></div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/JetEnergyResolutionXAODTools][JetEnergyResolutionXAODTools]]</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/JetResolution2015Prerecom][JetResolution2015Prerecom]]</div> <div id="_mcePaste">Here we will show you how to implement the tool. Like all tools, there are several steps:</div> </strong> <div id="_mcePaste"> **1.* Add the package to =MyAnalysis/cmt/Makefile.RootCore=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] JetResolution</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">*2.* Create a member variable to our class. In =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= add the following lines in the appropriate place (hopefully you know by now where that appropriate place is):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "JetResolution/JERTool.h"</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// JER</div> <div id="_mcePaste">JERTool *m_JERTool; //!* </div> <strong> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste"></div> </strong> <div id="_mcePaste"> **3.* In the source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, initialize the tool, do something with the tool every event, and finally delete the new tool. Add these lines to the appropriate places, but if you're not sure please ask us!</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include <TSystem.h> // used to define JERTool calibration path (you may already have this from the GRL part)</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// instantiate and initialize the JER (using default configurations)</div> <div id="_mcePaste">m_JERTool = new JERTool("JERTool");</div> <div id="_mcePaste">ANA_CHECK(m_JERTool->initialize());</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// event-by-event (execute) and in a loop over jets:</div> <div id="_mcePaste">// JER and uncert</div> <div id="_mcePaste">if(isMC){ // assuming isMC flag has been set based on eventInfo->eventType( xAOD::EventInfo::IS_SIMULATION )</div> <div id="_mcePaste">// Get the MC resolution</div> <div id="_mcePaste">double mcRes = m_JERTool->getRelResolutionMC( *jet_itr);* </div> <strong> <div id="_mcePaste">// Get the resolution uncertainty</div> <div id="_mcePaste">double uncert = m_JERTool->getUncertainty(*jet_itr); // you can provide a second argument specify which nuisance parameter, the default is all</div> <div id="_mcePaste">Info("execute()", "jet mcRes = %f , uncert = %f", mcRes, uncert);</div> <div id="_mcePaste">} // end if MC</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// in finalize, delete tool:</div> <div id="_mcePaste">if(m_JERTool){</div> <div id="_mcePaste">delete m_JERTool;</div> <div id="_mcePaste">m_JERTool = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">We've done a lot of stuff - let's recompile the package and test that we didn't break anything:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run/</div> <div id="_mcePaste">rm -rf submitDir/</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---++ Muons</div> <div id="_mcePaste">In this introductory part we will show you how to make a basic loop over muon objects. This is very much the same thing we did for jets already.</div> <div id="_mcePaste">Add the muon xAOD EDM package to =MyAnalysis/cmt/Makefile.RootCore=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] xAODMuon</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= is any other package dependencies you may already have included.</div> <div id="_mcePaste">Now in our source code =MyAnalysis/Root/MyxAODAnalysis.cxx=, add the muon xAOD EDM container class to =include= statement near the top:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODMuon/MuonContainer.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Still in the source code, in =execute()= (run for every event) retrieve the muon container and loop over all muons for this event:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//------------</div> <div id="_mcePaste">// MUONS</div> <div id="_mcePaste">//------------</div> <div id="_mcePaste">// get muon container of interest</div> </strong> <div id="_mcePaste"> *const xAOD::MuonContainer* muons = 0;</div> <div id="_mcePaste">ANA_CHECK(event->retrieve( muons, "Muons" ));</div> <div id="_mcePaste">// loop over the muons in the container</div> <div id="_mcePaste">xAOD::MuonContainer::const_iterator muon_itr = muons->begin();</div> <div id="_mcePaste">xAOD::MuonContainer::const_iterator muon_end = muons->end();</div> <div id="_mcePaste">for( ; muon_itr != muon_end; ++muon_itr ) {</div> <div id="_mcePaste">Info("execute()", " original muon pt = %.2f GeV", (( *muon_itr)->pt() * 0.001)); // just to print out something* </div> <strong> <div id="_mcePaste">} // end for loop over muons</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">If you are not sure of the container type and/or container name see [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Knowing_what_information_is_in_t][above]].</div> <div id="_mcePaste">You can do the usual !RootCore thing to update/find the (new) package dependencies and compile:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">More information about using muons for DC14 can be found here: <br/></div> <div id="_mcePaste">[[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/MCPAnalysisGuidelinesDC14][MCP: Analysis Guidelines for DC14]]</div> <div id="_mcePaste">---+++ Muon calibration and smearing tool, and associated systematics</div> <div id="_mcePaste">Now let's use the !MuonCalibrationAndSmearingTool found in the [[https://svnweb.cern.ch/trac/atlasoff/browser/PhysicsAnalysis/MuonID/MuonIDAnalysis/MuonMomentumCorrections][MuonMomentumCorrections package]] which is used to correct the MC to look like the data.</div> <div id="_mcePaste">As with all tools we have to do the basics:</div> </strong> <div id="_mcePaste"> **1.* Let !RootCore know where to find the package that we will use, by adding it to =MyAnalysis/cmt/Makefile.RootCore:== </div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] MuonMomentumCorrections</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...] is any other package dependencies you may already have included.</div> <div id="_mcePaste">*2.* Create our member variable in your class =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= by adding the following lines in the appropriate place (hopefully you know by now where that appropriate place is):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "MuonMomentumCorrections/MuonCalibrationAndSmearingTool.h"</div> <div id="_mcePaste">…</div> <div id="_mcePaste">// MuonCalibrationAndSmearing</div> <div id="_mcePaste">CP::MuonCalibrationAndSmearingTool *m_muonCalibrationAndSmearingTool; //!* </div> <strong> <div id="_mcePaste">%ENDSYNTAX%</div> </strong> <div id="_mcePaste"> **3.* In the source code, =!MyAnalysis/Root/MyxAODAnalysis.cxx=, initialize the tool, and finally delete the new tool (we'll show you in a second what we will do event-by-event). Add these lines to the appropriate places, but if you're not sure please ask us!</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// initialize the muon calibration and smearing tool</div> <div id="_mcePaste">m_muonCalibrationAndSmearingTool = new CP::MuonCalibrationAndSmearingTool( "MuonCorrectionTool" );</div> <div id="_mcePaste">//m_muonCalibrationAndSmearingTool->msg().setLevel( MSG::DEBUG );</div> <div id="_mcePaste">ANA_CHECK(m_muonCalibrationAndSmearingTool->initialize());</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// in finalize, delete tool:</div> <div id="_mcePaste">if(m_muonCalibrationAndSmearingTool){</div> <div id="_mcePaste">delete m_muonCalibrationAndSmearingTool;</div> <div id="_mcePaste">m_muonCalibrationAndSmearingTool = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">*Tool usage* <br/></div> <div id="_mcePaste">Ok, now we have the basics out of the way, so let's actually do something with this tool, event by event. Here we will show you a nice example of a tool that implements all the dual-use guidelines, from smearing the pt of MC muons, to applying systematic uncertainties associated to this correction.</div> <div id="_mcePaste">In this example the tool will actually 'update' the muon's pt to the corrected value (leaving all other variables the same). We can't actually override values of the input (const) objects, so we must create a new (non-const) muon object that is a copy of the orignal muon.</div> <div id="_mcePaste">There are two ways to do this:</div> <div id="_mcePaste">1. Create a [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Shallow_copy][shallow copy]] of the original muon object and apply the correction directly to this copy, via the applyCorrection(xAOD::Muon& mu) method.</div> <div id="_mcePaste">1. Create a new Muon object and pass this to the correctedCopy(const xAOD::Muon& input, xAOD::Muon*& output) method, which will automatically create a [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Deep_copy][deep copy]] of the original muon and modify the copy.</div> <div id="_mcePaste">Here we will show you the first method.</div> <div id="_mcePaste">Near the top of =MyAnalysis/Root/MyxAODAnalysis.cxx= add an include statement so we can check the return status of the tool:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "PATInterfaces/CorrectionCode.h" // to check the return correction code status of tools</div> <div id="_mcePaste">#include "xAODCore/ShallowAuxContainer.h"</div> <div id="_mcePaste">#include "xAODCore/ShallowCopy.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Still in our source code, in =execute()=, somewhere after you have already retrieved =xAOD::MuonContainer* muons= from TEvent, let's create the shallow copy of the muon container and loop over this shallow copy.</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// create a shallow copy of the muons container</div> <div id="_mcePaste">std::pair< xAOD::MuonContainer*, xAOD::ShallowAuxContainer* > muons_shallowCopy = xAOD::shallowCopyContainer( *muons );* </div> <strong> <div id="_mcePaste">// iterate over our shallow copy</div> <div id="_mcePaste">xAOD::MuonContainer::iterator muonSC_itr = (muons_shallowCopy.first)->begin();</div> <div id="_mcePaste">xAOD::MuonContainer::iterator muonSC_end = (muons_shallowCopy.first)->end();</div> <div id="_mcePaste">for( ; muonSC_itr != muonSC_end; ++muonSC_itr ) {</div> <div id="_mcePaste">if(m_muonCalibrationAndSmearingTool->applyCorrection(**muonSC_itr) == CP::CorrectionCode::Error){ // apply correction and check return code= </div> <div id="_mcePaste">// Can have CorrectionCode values of Ok, OutOfValidityRange, or Error. Here only checking for Error.</div> <div id="_mcePaste">// If OutOfValidityRange is returned no modification is made and the original muon values are taken.</div> <div id="_mcePaste">Error("execute()", "MuonCalibrationAndSmearingTool returns Error CorrectionCode");</div> <div id="_mcePaste">}</div> <div id="_mcePaste">Info("execute()", " corrected muon pt = %.2f GeV", ((*muonSC_itr)->pt() * 0.001));</div> <div id="_mcePaste">} // end for loop over shallow copied muons</div> <div id="_mcePaste">delete muons_shallowCopy.first;</div> <div id="_mcePaste">delete muons_shallowCopy.second;</div> <div id="_mcePaste">%ENDSYNTAX%</div> </strong> <div id="_mcePaste"> **Tool systematic uncertainties* <br/></div> <div id="_mcePaste">The dual-use tools infrastructure has set in place a systematic 'registry', where all tools can list (by strings) the systematics associated with that particular tool. When a tool gets initialized for use in your analysis, you can setup this registry and get the list of all systematics recommended by the tool developers.</div> <div id="_mcePaste">The ideal use case is that you:</div> <div id="_mcePaste">* loop over the events</div> <div id="_mcePaste">* loop over the systematics</div> <div id="_mcePaste">* use the CP tool, which will be automatically configured for you for the systematic you are evaluating (or no systematic which defaults to the nominal tool use)</div> <div id="_mcePaste">You can then decide what you actually do with the outcome of these systematics: use the results in event/object selections, write out individual ntuples or trees per systematic, etc.</div> <div id="_mcePaste">First we will show you how to setup the systematic registry. In your header file, =MyAnalysis/MyAnalysis/MyxAODAnalysis.h, add the following lines in the appropriate spots (of course ask if you're not sure):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// header for systematics:</div> <div id="_mcePaste">#include "PATInterfaces/SystematicRegistry.h"</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// list of systematics</div> <div id="_mcePaste">std::vector<CP::SystematicSet> m_sysList; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now to the source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, let's add the following with the other headers near the top:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// header files for systematics:</div> <div id="_mcePaste">#include "PATInterfaces/SystematicVariation.h"</div> <div id="_mcePaste">#include "PATInterfaces/SystematicsUtil.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And in =initialize()= let's get the list of systematics from the systematic registry associated to our initialized tools. This must be done *after* all the other tools have been initialized.</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// get the systematics registry and add the recommended systematics into our list of systematics to run over (+/-1 sigma):</div> <div id="_mcePaste">const CP::SystematicRegistry& registry = CP::SystematicRegistry::getInstance();</div> <div id="_mcePaste">const CP::SystematicSet& recommendedSystematics = registry.recommendedSystematics(); // get list of recommended systematics</div> <div id="_mcePaste">m_sysList = CP::make_systematics_vector(recommendedSystematics);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">In =execute()= we should modify the part where we loop over the muons and apply the muon calibration and smearing tool to look something like this instead (where first we loop over the systematics, apply the systematic to the appropriate tool, and then loop over the muon objects that are smeared according to this systematic):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">std::vector<CP::SystematicSet>::const_iterator sysListItr;</div> <div id="_mcePaste">// loop over recommended systematics</div> <div id="_mcePaste">for (sysListItr = m_sysList.begin(); sysListItr != m_sysList.end(); ++sysListItr){</div> <div id="_mcePaste">if(( *sysListItr).name()=="") std::cout << "Nominal (no syst) " << std::endl;* </div> <strong> <div id="_mcePaste">else std::cout << "Systematic: " << (*sysListItr).name() << std::endl;</div> <div id="_mcePaste"></div> <div id="_mcePaste">// apply recommended systematic for muonCalibrationAndSmearingTool</div> <div id="_mcePaste">if( m_muonCalibrationAndSmearingTool->applySystematicVariation( *sysListItr ) != CP::SystematicCode::Ok ) {</div> <div id="_mcePaste">Error("execute()", "Cannot configure muon calibration tool for systematic" );</div> <div id="_mcePaste">continue; // go to next systematic</div> <div id="_mcePaste">} // end check that systematic applied ok</div> <div id="_mcePaste">// create a shallow copy of the muons container</div> </strong> <div id="_mcePaste"> *std::pair< xAOD::MuonContainer*, xAOD::ShallowAuxContainer* > muons_shallowCopy = xAOD::shallowCopyContainer( *muons );* </div> <strong> <div id="_mcePaste">// iterate over our shallow copy</div> <div id="_mcePaste">xAOD::MuonContainer::iterator muonSC_itr = (muons_shallowCopy.first)->begin();</div> <div id="_mcePaste">xAOD::MuonContainer::iterator muonSC_end = (muons_shallowCopy.first)->end();</div> <div id="_mcePaste">for( ; muonSC_itr != muonSC_end; ++muonSC_itr ) {</div> <div id="_mcePaste">m_muonCalibrationAndSmearingTool->applyCorrection(**muonSC_itr);</div> <div id="_mcePaste">Info("execute()", "corrected muon pt = %.2f GeV", ((*muonSC_itr)->pt() * 0.001));</div> <div id="_mcePaste">} // end for loop over shallow copied muons</div> <div id="_mcePaste">delete muons_shallowCopy.first;</div> <div id="_mcePaste">delete muons_shallowCopy.second;</div> <div id="_mcePaste"></div> <div id="_mcePaste">} // end for loop over systematics</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now you can recompile to see this new tool in action:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run/</div> <div id="_mcePaste">rm -rf submitDir/</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">---++ Taus</div> <div id="_mcePaste">More information about looping over taus and accessing tau xAOD quantities can be found on this page: <br/></div> <div id="_mcePaste">[[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/TauXAODInstructions][Tau xAOD Instructions]]</div> <div id="_mcePaste">Information about tau-related tools for:</div> <div id="_mcePaste">* !TauSelectionTool: generic tool to apply a set of requirements on tau candidates</div> <div id="_mcePaste">* !TauSmearingTool: currently supports tau energy corrections</div> <div id="_mcePaste">* !TauEfficiencyCorrectionsTool: currently provides jet identification scale factors and the associated uncertainties</div> <div id="_mcePaste">can be read in this very nice documentation included in the package: <br/></div> <div id="_mcePaste">https://svnweb.cern.ch/trac/atlasoff/browser/PhysicsAnalysis/TauID/TauAnalysisTools/tags/TauAnalysisTools-00-00-04/README.rst <br/></div> <div id="_mcePaste">(Note I have linked to !TauAnalysisTools-00-00-04 as this is the tagged version in !AnalysisBase 2.0.6., but this can/will change with release)</div> <div id="_mcePaste"></div> <div id="_mcePaste">---++ MC Truth</div> <div id="_mcePaste">Lots of goodies about how to extract MC truth information in this talk: [[https://indico.cern.ch/event/307549/session/7/contribution/31/material/slides/0.pdf][Monte Carlo truth in the xAOD]]</div> <div id="_mcePaste">---++ Trigger</div> <div id="_mcePaste">Here we will show you how to do some simple things with the trigger (namely if the trigger passed/failed and the final trigger prescale factor). First let me mention that the trigger is a complicated beast... It relies on meta-data, which is "data about the data", like which trigger menu was used which contains info about the thresholds, chains, and prescales. And it relies on event-by-event level information, like which triggers passed/failed and the associated trigger objects that fired these triggers (like the object that was identified as an electron at the trigger level that fired a particular electron trigger). Because of this level of complication you can't just browse the trigger decision in a !TBrowser in ROOT.</div> <div id="_mcePaste">The primary tool you must use to interact with the trigger is called the !TrigDecisionTool. Note that if you want to use the !TrigDecisionTool in !EventLoop (which is what we are doing here in this tutorial), you must also make use of another tool: the !TrigConf::xAODConfigTool. Both of which must be declared on the "heap" (meaning a member variable declared in your header file). But we'll lead you through all that below!</div> <div id="_mcePaste">Let's do the basics :</div> </strong> <div id="_mcePaste"> **1.* Let !RootCore know where to find the packages containing the tools that we will use, by adding the following lines to =MyAnalysis/cmt/Makefile.RootCore:== </div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] TrigDecisionTool TrigConfxAOD</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...] is any other package dependencies you may already have included.</div> <div id="_mcePaste">*2.* Create our member variables in =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= by adding the following lines in the appropriate place (hopefully you know by now where that appropriate place is):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// include files for using the trigger tools</div> <div id="_mcePaste">#include "TrigConfxAOD/xAODConfigTool.h"</div> <div id="_mcePaste">#include "TrigDecisionTool/TrigDecisionTool.h"</div> <div id="_mcePaste">…</div> <div id="_mcePaste">// trigger tools member variables</div> <div id="_mcePaste">Trig::TrigDecisionTool *m_trigDecisionTool; //!* </div> <strong> <div id="_mcePaste">TrigConf::xAODConfigTool *m_trigConfigTool; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> </strong> <div id="_mcePaste"> **3.* In the source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, we will initialize the tool in the =initialize()= function. This requires a few more steps than other tools, as we are setting up both tools: !TrigDecisionTool and !TrigConf::xAODConfigTool.</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// Initialize and configure trigger tools</div> <div id="_mcePaste">m_trigConfigTool = new TrigConf::xAODConfigTool("xAODConfigTool"); // gives us access to the meta-data</div> <div id="_mcePaste">ANA_CHECK( m_trigConfigTool->initialize() );</div> <div id="_mcePaste">ToolHandle< TrigConf::ITrigConfigTool > trigConfigHandle( m_trigConfigTool );</div> <div id="_mcePaste">m_trigDecisionTool = new Trig::TrigDecisionTool("TrigDecisionTool");</div> <div id="_mcePaste">ANA_CHECK(m_trigDecisionTool->setProperty( "ConfigTool", trigConfigHandle ) ); // connect the TrigDecisionTool to the ConfigTool</div> <div id="_mcePaste">ANA_CHECK(m_trigDecisionTool->setProperty( "TrigDecisionKey", "xTrigDecision" ) );</div> <div id="_mcePaste">ANA_CHECK(m_trigDecisionTool->initialize() );</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">*4.* Still in the source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, let's delete the variables (and associated memory allocation) in the =finalize()= method:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// cleaning up trigger tools</div> <div id="_mcePaste">if( m_trigConfigTool ) {</div> <div id="_mcePaste">delete m_trigConfigTool;</div> <div id="_mcePaste">m_trigConfigTool = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">if( m_trigDecisionTool ) {</div> <div id="_mcePaste">delete m_trigDecisionTool;</div> <div id="_mcePaste">m_trigDecisionTool = 0;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">*5.* Let's actually do something now, event-by-event. Here we will get all triggers matching the string "HLT_xe80*", and for each of those we will get the associated trigger chain (the L1 input). And for each chain group we will ask if the chain passed or failed the trigger selection (so like an AND of all parts of that chain, L1 and HLT), and also the total chain prescale (so like the multiplication of the L1 and HLT trigger prescales). For MC the prescales are 1, but for data this can very often be not equal to 1. So in the =execute()= function in =MyAnalysis/Root/MyxAODAnalysis.cxx= (after you have defined an xAOD::TEvent object) add the following lines:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// examine the HLT_xe80* chains, see if they passed/failed and their total prescale</div> <div id="_mcePaste">auto chainGroup = m_trigDecisionTool->getChainGroup("HLT_xe80.*");</div> <div id="_mcePaste">std::map<std::string,int> triggerCounts;</div> <div id="_mcePaste">for(auto &trig : chainGroup->getListOfTriggers()) {</div> <div id="_mcePaste">auto cg = m_trigDecisionTool->getChainGroup(trig);</div> <div id="_mcePaste">std::string thisTrig = trig;</div> <div id="_mcePaste">Info( "execute()", "%30s chain passed(1)/failed(0): %d total chain prescale (L1*HLT): %.1f", thisTrig.c_str(), cg->isPassed(), cg->getPrescale() );</div> <div id="_mcePaste">} // end for loop (c++11 style) over chain group matching "HLT_xe80*"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now you can recompile to see this new tool in action:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run/</div> <div id="_mcePaste">rm -rf submitDir/</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Other references:</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerxAODTutorialJan2015][xAOD trigger analysis in ROOT (January 2015)]] (Note that at the time of that tutorial the trigger information was not accessible from within ROOT in the primary xAODs (the xAODs used in this tutorial), so this dedicated trigger tutorial used a special xAOD that had a fix applied so that trigger information can be accessed.)</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerAnalysisTools][TriggerAnalysisTools]]: a wiki page with more complex examples in both RootCore and Athena</div> <div id="_mcePaste">---++ Metadata</div> <div id="_mcePaste">Metadata is basically "data about data" and includes things like the trigger menu, MC channel number, number of events processed in a file, and luminosity information. The (D)xAOD is being improved to include more and more of this information. And more tools are becoming available to be able to access this information from within your analysis code. More information (and some examples) about accessing this metadata can be found here: [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AnalysisMetadata][Analysis Metadata]]</div> <div id="_mcePaste">---++ More tools and status'</div> <div id="_mcePaste">You can find more documentation about each xAOD object and associated tools in the appropriate workbook:</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PhysicsAnalysisWorkBookRel19#Documentation_for_specific_xAOD][Physics Analysis Workbook: Release 19 (DC14)]].</div> <div id="_mcePaste">* [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PhysicsAnalysisWorkBookRel20CPRec][Physics Analysis Workbook: Release 20 (2015 data, MC)]]</div> <div id="_mcePaste">The status of CP tool migration to the xAOD can be found on the [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/ASGUserAnalysisToolsxAODMigration][Tools xAOD Migration]] page. Please check here to see if the tool you need has been migrated. You should be able to configure and use these tools in a similar way to those shown above.</div> <div id="_mcePaste">---+ 7. Creating and saving histograms</div> <div id="_mcePaste">Usually you will want to create and save histograms to a root file. With !EventLoop this is really easy to do. !EventLoop handles much of the heavy lifting, and as the user you just need to define what histogram you want (type, range, etc.), and fill it appropriately. When working with PROOF (feature to come), !EventLoop will take care of collecting the histograms from all the worker nodes, merging them and saving them on the submission node.</div> <div id="_mcePaste">In this simple example (to demonstrate how it is done) we will plot the pt of one jet collection. In the header file =MyAnalysis/MyAnalysis/MyxAODAnalysis.h= add an include for the TH1 class file (before the class statement):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include <TH1.h></div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Within our algorithm class add a histogram pointer as a member to our =MyxAODAnalysis= algorithm class (in =MyxAODAnalysis.h=). You can make this either public or private, it doesn't matter. Please note that the =//!= is important:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">TH1 *h_jetPt; //!* </div> <strong> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now inside the source file =MyAnalysis/Root/MyxAODAnalysis.cxx=, we need to book the histogram and add it to the output. That happens in the =histInitialize= function:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::StatusCode MyxAODAnalysis :: histInitialize ()</div> <div id="_mcePaste">{</div> <div id="_mcePaste">// Here you do everything that needs to be done at the very</div> <div id="_mcePaste">// beginning on each worker node, e.g. create histograms and output</div> <div id="_mcePaste">// trees. This method gets called before any input files are</div> <div id="_mcePaste">// connected.</div> <div id="_mcePaste">h_jetPt = new TH1F("h_jetPt", "h_jetPt", 100, 0, 500); // jet pt [GeV]</div> <div id="_mcePaste">wk()->addOutput (h_jetPt);</div> <div id="_mcePaste">return EL::StatusCode::SUCCESS;</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">This method is called before processing any events. Note that the =wk()->addOutput= call is a mechanism !EventLoop uses for delivering the results of an algorithm to the outside world. When running in PROOF, ROOT will merge all of the objects in this list. In principle you can place any object that inherits from =TObject= in there, but if you put anything other than histograms there you need to take special precautions for the merging step.</div> <div id="_mcePaste">Now that we have the histogram, we also need to fill it. For that we can use the =execute()= method (called for every event). You probably have a loop over</div> <div id="_mcePaste">the =AntiKt4EMTopoJets= collection (as described [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Jets][above]]), in that jet loop let's fill this histogram, so the loop looks something like:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// loop over the jets in the container</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_itr = jets->begin();</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_end = jets->end();</div> <div id="_mcePaste">for( ; jet_itr != jet_end; ++jet_itr ) {</div> <div id="_mcePaste">Info("execute()", " jet pt = %.2f GeV", ((*jet_itr)->pt() * 0.001)); // just to print out something</div> <div id="_mcePaste">h_jetPt->Fill( ( (*jet_itr)->pt()) * 0.001); // GeV</div> <div id="_mcePaste">} // end for loop over jets</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now we should be able to build our package with the full algorithm:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">And rerun from the =Run/= directory:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> </strong> <div id="_mcePaste"> **Outputs* <br/></div> <div id="_mcePaste">Once your job is finished you can find the histogram(s) inside the unique directory you created at job submission ( =submitDir/=). There is a different file for each sample you submitted ( =hist-label.root=), so in our case we have only one =submitDir/hist-AOD.05352803._000031.pool.root.1.root=. Please note that the second part of the histogram file name will correspond directly to the name of the sample in !SampleHandler, while the first part ( =hist-=) is set by !EventLoop and can not be changed.</div> <div id="_mcePaste">A second output is made in the =submitDir/hist/= directory such that you can access these histograms through !SampleHandler, e.g.:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">SH::SampleHandler sh;</div> <div id="_mcePaste">sh.load ("submitDir/");</div> <div id="_mcePaste">sh.get ("hist")->readHist ("h_jetPt");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">If you want, at the very end of your steering macro, =Run/ATestRun.cxx=, (after the job has been submitted with the driver) you can ask !SampleHandler to plot this histogram (making a canvas and the histogram pop us), by adding the following line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// Fetch and plot our histogram</div> <div id="_mcePaste">SH::SampleHandler sh_hist;</div> <div id="_mcePaste">sh_hist.load (submitDir + "/hist");</div> <div id="_mcePaste">TH1 *hist = (TH1*) sh_hist.get ("AOD.05352803._000031.pool.root.1")->readHist ("h_jetPt");</div> <div id="_mcePaste">hist->Draw ();</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Note that if you submit your job to the Grid (shown later) you should remove these lines.</div> <div id="_mcePaste">---+ 8. Creating and saving ntuples with trees</div> <div id="_mcePaste">In this section we'll build a simple example of writing a new n-tuple. You may want to write out an ntuple (with a tree and branches connected to that tree) after you have finished working within the xAOD format. So for example, after you have applied the object tools and applied systematic uncertainties, for subsequent steps of your analysis (like preparing input to a statistical package like !HistFitter).</div> <div id="_mcePaste">For this example we will use !EventLoop to define an output stream and we will use the [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#Creating_output_n_tuples][EventLoop NTupleSvc]] to create an ntuple (root file) format. This will be a trivial example showing you how to fill one tree with one very simple quantity (event number), but you can reproduce this example to produce multiple trees in one root file.</div> <div id="_mcePaste">First let's open our header file, =MyAnalysis/MyAnalysis/MyxAODAnalysis.h=, and add the following lines which will define the name of output file (which will be supplied when the macro is executed), the TTree, and the branch we will fill in the ntuple (just the event number, not very exciting):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// defining the output file name and tree that we will put in the output ntuple, also the one branch that will be in that tree</div> <div id="_mcePaste">std::string outputName;</div> <div id="_mcePaste">TTree *tree; //!* </div> <strong> <div id="_mcePaste">int EventNumber; //!</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Notice =outputName= does not have a =//!= beside it, that's because we will defined it at run time (in the macro). And before closing this header file we need to include the TTree header file near the top of our code:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include <TTree.h></div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now in our source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, let's do the actual implementation. First in the =histInitialize()= method, add these lines:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// get the output file, create a new TTree and connect it to that output</div> <div id="_mcePaste">// define what braches will go in that tree</div> <div id="_mcePaste">TFile *outputFile = wk()->getOutputFile (outputName);</div> <div id="_mcePaste">tree = new TTree ("tree", "tree");</div> <div id="_mcePaste">tree->SetDirectory (outputFile);</div> <div id="_mcePaste">tree->Branch("EventNumber", &EventNumber);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now in the =execute()= method let's actually fill the branch. You should do this somewhere after you have retrieved the !EventInfo object:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// fill the branches of our trees</div> <div id="_mcePaste">EventNumber = eventInfo->eventNumber();</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And then somewhere near the end of the =execute()= method be sure to fill the tree:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">tree->Fill();</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Finally near the top of the source code add the include statement for the TFile object we are using:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include <TFile.h></div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">To =MyAnalysis/cmt/Makefile.RootCore= add the dependency for =EventLoopAlgs= which where the !NtupleSvc lives:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] EventLoopAlgs</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">After these modifications we should re-compile our package and make sure it works:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc find_packages</div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Moving now to our =Run/= directory and our macro =ATestRun.cxx=, after you have defined the =job= but before you have added your algorithm to the job, add these lines which will create an output and an ntuple in that output stream:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// define an output and an ntuple associated to that output</div> <div id="_mcePaste">EL::OutputStream output ("myOutput");</div> <div id="_mcePaste">job.outputAdd (output);</div> <div id="_mcePaste">EL::NTupleSvc *ntuple = new EL::NTupleSvc ("myOutput");</div> <div id="_mcePaste">job.algsAdd (ntuple);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">After this line =job.algsAdd (alg);= add the following line, which is basically letting your algorithm know the name of the output file:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">alg->outputName = "myOutput"; // give the name of the output to our algorithm</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">(If you are using the compiled macro you will need to add two include statements for the !NtupleSvc and Output ntuple, =#include <EventLoopAlgs/NTupleSvc.h>= and =#include <EventLoop/OutputStream.h>=.)</div> <div id="_mcePaste">And that's it. You can rerun your macro again:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Your new ntuple will appear in the directory =submitDir/data-myOutput/=.</div> <div id="_mcePaste">---+ 9. Playing with xAOD containers</div> <div id="_mcePaste">Here we will show you how to modify or create new xAOD containers/objects, and then how to write these to another xAOD in ROOT.</div> <div id="_mcePaste">Note that if you do it this way you will not be able to read the output xAOD back into Athena. We will make use of some methods in the xAODRootAccess package to create our new xAOD.</div> </strong> <div id="_mcePaste"> **IMPORTANT NOTE*: If you are creating a new smaller xAOD file that is used by your analysis group, consider setting up a derivation in the Derivation Reduction Framework, which will automatically generate this smaller xAOD in the production system for you. See the [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/DerivationFramework][Derivation Framework]] for more information.</div> <div id="_mcePaste">---++ Creating a new xAOD output in !EventLoop</div> <div id="_mcePaste">Since we are using !EventLoop we need to tell it we are writing a new output. In =MyAnalysis/Root/MyxAODAnalysis.cxx= go into the =setupJob (EL::Job& job)= function and let it know about our new output:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// tell EventLoop about our output xAOD:</div> <div id="_mcePaste">EL::OutputStream out ("outputLabel", "xAOD");</div> <div id="_mcePaste">job.outputAdd (out);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And put this class into our list of included header files:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "EventLoop/OutputStream.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">The second argument to the !OutputStream constructor is needed to correctly merge xAOD outputs (including the metadata) on the grid. !AnalysisBase-2.3.15 (or !AnalysisBase 2.1.35 for the DC14 version) or later is needed to use this option.</div> <div id="_mcePaste">Now in =initialize()= let's tell our instance of the !TEvent that we are writing an output file, and to be ready to set some output containers/branches active</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// output xAOD</div> <div id="_mcePaste">TFile *file = wk()->getOutputFile ("outputLabel");* </div> <strong> <div id="_mcePaste">ANA_CHECK(event->writeTo(file));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">---++ Copying full container(s) to a new xAOD</div> <div id="_mcePaste">Here we will show you how to copy the contents of a full container, un-modified, for every event. We assume you have followed the instructions above to [[SoftwareTutorialxAODAnalysisInROOT#Creating_a_new_xAOD_output_in_EventLoop][define a new output xAOD in !EventLoop]].</div> <div id="_mcePaste">We will create this copy in the event loop, so in =MyAnalysis/Root/MyxAODAnalysis.cxx= in the =execute()= method add the following line to copy the full container for !AntiKt4EMTopoJets:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// copy full container(s) to new xAOD</div> <div id="_mcePaste">// without modifying the contents of it:</div> <div id="_mcePaste">ANA_CHECK(event->copy("AntiKt4EMTopoJets"));</div> <div id="_mcePaste">%ENDSYNTAX%</div> </strong> <div id="_mcePaste"> *At the *end* of =execute()= add this line to fill the xAOD with the content we have specified in the event loop:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// Save the event:</div> <div id="_mcePaste">event->fill();</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Finally, in =finalize()= let's tell the job to close up the output xAOD by adding:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// finalize and close our output xAOD file:</div> <div id="_mcePaste">TFile *file = wk()->getOutputFile ("outputLabel");* </div> <strong> <div id="_mcePaste">ANA_CHECK(event->finishWritingTo( file ));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Compile like usual and test your code:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">(Remember that if you already have a =submitDir= directory the job will crash.)</div> <div id="_mcePaste">If you have followed the instructions above you will find your output xAOD in =submitDir/data-outputLabel/=.</div> <div id="_mcePaste">Note that you can only copy xAOD objects and/or containers. You can determine if the container is of xAOD type by running =checkxAOD.py= on the xAOD and seeing which objects are of type =xAOD::x= (where =x= is the container of interest).</div> <div id="_mcePaste">---++ Creating modified container(s)</div> <div id="_mcePaste">Sometimes you don't want to just blindly copy the entire contents of a container for every event into a new xAOD, sometimes you want to modify the (content of the) containers themselves. You could imagine you might want to do some of the following:</div> </strong> <div id="_mcePaste"> ** *deep copy:* create a new container with the same variables as an existing container, but with only for a subset of objects/events passing some selection criteria (example: create a new jet collection that only contains jets with a pt greater than 50 GeV)</div> <div id="_mcePaste">* *shallow copy*: create a light-weight container that only "updates" some variables from an original container to new values (example: apply some energy correction and override existing reconstructed energy)</div> <div id="_mcePaste">* *adding new variables:* add variables/attributes to objects (example: adding a variable that identifies the jet as a bjet)</div> <div id="_mcePaste">Each of these are described in more detail below.</div> <div id="_mcePaste">Many of the CP tools take advantage of one of these features to either apply corrections to existing containers or copied containers, or to decorate objects with new information.</div> <div id="_mcePaste">---+++ Deep copy</div> <div id="_mcePaste">Deep copying will create new objects/containers that have all the attributes (aka variables) of the original container. This is useful when you are only interested in objects that pass certain criteria.</div> <div id="_mcePaste">You previously saw how to write out the entire original jet container to an output file, and how you can select good jets, let's go one step further. Let's create a new jet container of selected, good jets. First, you need to create the new jet container in your =execute()= function. For this, remember what you heard about the xAOD EDM in the tutorial. An xAOD container is *always* described by two objects. An "interface container" that you interact with, and an auxiliary store object that holds all the data. Now that we want to create a new container, you need to create both of these objects.</div> <div id="_mcePaste">The naming convention for the auxiliary stores is that if an interface container is called =xAOD::BlaContainer=, then its auxiliary store type is called =xAOD::BlaAuxContainer=, where Bla is the object of interest.</div> <div id="_mcePaste">So for jets will look like:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODJet/JetContainer.h"</div> <div id="_mcePaste">#include "xAODJet/JetAuxContainer.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">However instead of using the =JetAuxContainer= we will use the generic =AuxContainerBase= this is because if we were using a derivation as input some of the original (so-called "auxillary") variables may have been slimmed away (removed to make the container smaller), so if we were to do a deep-copy of the full !JetAuxContainer then we would make our container larger than necessary (by creating a bunch of variables that were not even in the original input !DxAOD). Instead let's add these lines instead:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODJet/JetContainer.h"</div> <div id="_mcePaste">#include "xAODCore/AuxContainerBase.h"</div> <div id="_mcePaste">...</div> <div id="_mcePaste">EL::StatusCode MyxAODAnalysis::execute() {</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// Create the new container and its auxiliary store.</div> <div id="_mcePaste">xAOD::JetContainer* goodJets = new xAOD::JetContainer();</div> <div id="_mcePaste">xAOD::AuxContainerBase* goodJetsAux = new xAOD::AuxContainerBase();</div> <div id="_mcePaste">goodJets->setStore( goodJetsAux ); //< Connect the two</div> <div id="_mcePaste">...</div> <div id="_mcePaste">for( ; jet_itr != jet_end; ++jet_itr ) {</div> <div id="_mcePaste">if( ! m_jetCleaning->accept( **jet_itr ) ) continue;* </div> <strong> <div id="_mcePaste">// Copy this jet to the output container:</div> </strong> <div id="_mcePaste"> *xAOD::Jet* jet = new xAOD::Jet();</div> <div id="_mcePaste">goodJets->push_back( jet ); // jet acquires the goodJets auxstore</div> <div id="_mcePaste"> *jet= **jet_itr; // copies auxdata from one auxstore to the other* </div> <strong> <div id="_mcePaste">}</div> <div id="_mcePaste">...</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Also make sure you have updated your package dependencies in !MyAnalysis/cmt/Makefile.RootCore to include:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">PACKAGE_DEP = EventLoop [...] xAODJet xAODCore</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where =[...]= represents any other package dependencies you may already have included.</div> <div id="_mcePaste">Now to write this new container to a new xAOD output, we assume you have first setup the new xAOD output in !EventLoop as we just described above:</div> <div id="_mcePaste">[[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Creating_a_new_xAOD_output_in_Ev][Creating a new xAOD output in !EventLoop ]]</div> <div id="_mcePaste">And then, still in =execute()= in the source code, after you have created and filled the new =goodJet= collection above, add these lines:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// Record the objects into the output xAOD:</div> <div id="_mcePaste">ANA_CHECK(event->record( goodJets, "GoodJets" ));</div> <div id="_mcePaste">ANA_CHECK(event->record( goodJetsAux, "GoodJetsAux." ));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Of course all of this needs to be put before the =event->fill();= call somewhere. Also take note that you can call the interface container whatever you want (within reason, it should be an alphanumeric name), but the accompanying auxiliary store object must always have the same name, postfixed by "Aux.". So, for any "Bla" interface container you would record a "BlaAux." auxiliary container.</div> <div id="_mcePaste">Notice that we create the output objects on the heap. (With =new=.) This mimics the behaviour of StoreGate. Just like StoreGate, TEvent also takes ownership of the objects recorded in it, so you must not delete the containers that you successfully recorded into TEvent.</div> <div id="_mcePaste">You can compile and test your code:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">The new output xAOD in =submitDir/data-outputLabel/= will now contain our new "GoodJets" container. You can check that it has fewer jets than the original =AntiKt4EMTopoJets= from which we deep copied.</div> <div id="_mcePaste">---+++ Shallow copy</div> <div id="_mcePaste">Another way of making copies of containers is called the shallow copy. If you want to apply some sort of modification/calibration to all objects in an input container, but you don't want to perform an object selection at the same time, then the best idea is to make a "shallow copy" of the input container. This is done with the help of [[https://svnweb.cern.ch/trac/atlasoff/browser/Event/xAOD/xAODCore/trunk/xAODCore/ShallowCopy.h][xAOD::shallowCopyContainer]]. Note that you can not add/remove objects to/from a shallow copy container (but you can decorate it with new variables associated to each object - see decorations in the next section). However it's absolutely possible to make deep copies of some selected shallow copies later on in your code, and put these deep copies into a container of selected objects.</div> <div id="_mcePaste">This creates a copy which only overrides default values with new ones that you set, while keeping all other unaffected quantities unchanged from the original. The shallowCopyContainer will return a pair of xAOD objects (one for the interface and one for the auxiliary store).</div> <div id="_mcePaste">Let's create a shallow copy of the !AntiKt4EMTopoJets and shift the pt of all jets up by 5%. In our source code, =MyAnalysis/Root/MyxAODAnalysis.cxx=, near the top add the include statement to do this shallow copy:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODCore/ShallowCopy.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now in =execute()=, we assume you already have some lines like:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> </strong> <div id="_mcePaste"> *const xAOD::JetContainer* jets = 0;</div> <div id="_mcePaste">ANA_CHECK(event->retrieve( jets, "AntiKt4EMTopoJets" ));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">So, now somewhere below these lines let's create our shallow copy:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//--------------</div> <div id="_mcePaste">// shallow copy</div> <div id="_mcePaste">//--------------</div> <div id="_mcePaste">// "jets" jet container already defined above</div> <div id="_mcePaste">std::pair< xAOD::JetContainer*, xAOD::ShallowAuxContainer* > jets_shallowCopy = xAOD::shallowCopyContainer( *jets );* </div> <strong> <div id="_mcePaste">// iterate over the shallow copy</div> <div id="_mcePaste">xAOD::JetContainer::iterator jetSC_itr = (jets_shallowCopy.first)->begin();</div> <div id="_mcePaste">xAOD::JetContainer::iterator jetSC_end = (jets_shallowCopy.first)->end();</div> <div id="_mcePaste">for( ; jetSC_itr != jetSC_end; ++jetSC_itr ) {</div> <div id="_mcePaste">// apply a shift in pt, up by 5%</div> <div id="_mcePaste">double newPt = (*jetSC_itr)->pt() * (1 + 0.05);</div> <div id="_mcePaste">// from: https://svnweb.cern.ch/trac/atlasoff/browser/Event/xAOD/xAODJet/trunk/xAODJet/versions/Jet_v1.h</div> <div id="_mcePaste">xAOD::JetFourMom_t newp4(newPt, (*jetSC_itr)->eta(), (*jetSC_itr)->phi(), (*jetSC_itr)->m());</div> <div id="_mcePaste">(*jetSC_itr)->setJetP4( newp4); // we've overwritten the 4-momentum</div> <div id="_mcePaste">} // end iterator over jet shallow copy</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">By default when you create a shallowCopyContainer you take ownership of the pairs (meaning you need to take care to delete the both). You can give ownership to either the TStore or TEvent, which will then handle deletion for you. If you want to work with the pair internally to your algorithm or pass it around to different algorithms without writing it out to an output xAOD, you should give it to the TStore. If you plan to write it out to an output xAOD you can give ownership to TEvent. Each of these cases is described below:</div> </strong> <div id="_mcePaste"> **Shallow copy: record to output xAOD* <br/></div> <div id="_mcePaste">There are two options here, dictated by how you set the flag =setShallowIO=:</div> <div id="_mcePaste">1. Save a real shallow copy only writing to the xAOD container the variables you have overwritten while still pointing to the original for all other variables. In this case you must also write the original container ( =setShallowIO= is true) as [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT#Copying_full_container_s_to_a_ne][previously described]].</div> <div id="_mcePaste">1. Save an actually deep copy; in this case you do not need to also write the original container to the xAOD ( =setShallowIO= is false).</div> <div id="_mcePaste">For either 1. and 2. add these lines below our iterator over the shallow copied jets:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">jets_shallowCopy.second->setShallowIO( false ); // true = shallow copy, false = deep copy</div> <div id="_mcePaste">// if true should have something like this line somewhere:</div> <div id="_mcePaste">// event->copy("AntiKt4EMTopoJets");</div> <div id="_mcePaste">ANA_CHECK(event->record( jets_shallowCopy.first, "ShallowCopiedJets" ));</div> <div id="_mcePaste">ANA_CHECK(event->record( jets_shallowCopy.second, "ShallowCopiedJetsAux." ));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">*Shallow copy: record to TStore* <br/></div> <div id="_mcePaste">You can record the shallow copy to TStore, in a very similar way we did above by storing it to TEvent. First in the source code in =execute()= you will need to define an instance of a TStore object (making use of the !EventLoop worker object):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">xAOD::TStore* store = wk()->xaodStore();</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Then simply record your shallowed jet container (and aux container) to the store:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">ANA_CHECK(store->record( jets_shallowCopy.first, "ShallowCopiedJets" ));</div> <div id="_mcePaste">ANA_CHECK(store->record( jets_shallowCopy.second, "ShallowCopiedJetsAux." ));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">!EventLoop takes care of clearing the memory for you.</div> <div id="_mcePaste">(Tip: At any point you can see what is stored to your TStore by doing =store->print()=.)</div> <div id="_mcePaste">Compile like usual and test your code:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">rc compile</div> <div id="_mcePaste">cd Run</div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">And depending on how you set =setShallowIO= you will have more or less variables in your new xAOD associated to the !ShallowCopiedJets. You can try changing the flag, recompiling and checking the alternative content of the xAOD.</div> <div id="_mcePaste">More examples of how this code works, over [[https://svnweb.cern.ch/cern/wsvn/atlas-krasznaa/AODUpgrade/xAODCopyTest/trunk/util/readWriteShallowDecoratedObjects.cxx][here]].</div> <div id="_mcePaste">---+++ New variables</div> <div id="_mcePaste">You might want to modify an object by adding a new attribute (or variable) to it's container. There are two ways to do this, depending on the "const" state of the container you want to modify:</div> <div id="_mcePaste">* for =const= objects (for example adding variables to containers from the input xAOD) you will "decorate" the container using the =auxdecor= function</div> <div id="_mcePaste">* for =nonconst= objects (for example possibly like a shallow or deep copied container) you will add auxillary data to this container using the =auxdata= function</div> <div id="_mcePaste">Now we will show you a simple example of each below.</div> <div id="_mcePaste">*auxdecor* <br/></div> <div id="_mcePaste">Recall we [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Jets][looped over the "AntiKt4EMTopoJets" jet container]] and simply wrote to screen the jet pt (pretty boring). But now let's decorate this (const) jet container by adding a variable called "mySignal" that just checks if the jet pt is greater than 40 !GeV. Modify the loop over the jets so it looks like:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// loop over the jets in the container</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_itr = jets->begin();</div> <div id="_mcePaste">xAOD::JetContainer::const_iterator jet_end = jets->end();</div> <div id="_mcePaste">for( ; jet_itr != jet_end; ++jet_itr ) {</div> <div id="_mcePaste">Info("execute()", " jet pt = %.2f GeV", (( *jet_itr)->pt() * 0.001)); // just to print out something* </div> <strong> <div id="_mcePaste">if((*jet_itr)->pt() > 40000 ){</div> <div id="_mcePaste">( *jet_itr )->auxdecor< int >( "mySignal" ) = 1; // 1 = yes, it's a signal jet!</div> <div id="_mcePaste">}</div> <div id="_mcePaste">else{</div> <div id="_mcePaste">( *jet_itr )->auxdecor< int >( "mySignal" ) = 0; // 0 = nope, not a signal jet</div> <div id="_mcePaste">}</div> <div id="_mcePaste">} // end for loop over jets</div> <div id="_mcePaste">%ENDSYNTAX%</div> </strong> <div id="_mcePaste"> **auxdata* <br/></div> <div id="_mcePaste">Here we will add a variable to the (nonconst) jet container we did the shallow copy of [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Shallow_copy][above]]. In the loop over the shallow copied jets, after we have shifted the pt up by 5% you can add the following line:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// iterate over the shallow copy</div> <div id="_mcePaste">xAOD::JetContainer::iterator jetSC_itr = (jets_shallowCopy.first)->begin();</div> <div id="_mcePaste">xAOD::JetContainer::iterator jetSC_end = (jets_shallowCopy.first)->end();</div> <div id="_mcePaste">for( ; jetSC_itr != jetSC_end; ++jetSC_itr ) {</div> <div id="_mcePaste">...</div> <div id="_mcePaste">// adding a (silly) variable: checking if shallow copied pt is greater than 40 GeV, after 5% shift up (classify as signal or not)</div> <div id="_mcePaste">if(( *jetSC_itr)->pt() > 40000 ){* </div> <strong> <div id="_mcePaste">( *jetSC_itr )->auxdata< int >( "mySignal" ) = 1; // 1 = yes, it's a signal jet!</div> <div id="_mcePaste">}</div> <div id="_mcePaste">else{</div> <div id="_mcePaste">( *jetSC_itr )->auxdata< int >( "mySignal" ) = 0; // 0 = nope, not a signal jet</div> <div id="_mcePaste">}</div> <div id="_mcePaste">} // end iterator over jet shallow copy</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Here we have added an integer variable called 'mySignal' to the shallow copied jets</div> <div id="_mcePaste">---+++ Future feature: Easy container slimming</div> <div id="_mcePaste">As a final ingredient to writing out modified objects, you can select which of their properties should be written to the output file. The xAOD design was based around the idea that objects/containers may be slimmed during the analysis easily.</div> <div id="_mcePaste">As you should know, all the data payload of xAOD objects/containers is in their auxiliary store objects. Because of this the way to specify which variables should be written out, is to set a property for the auxiliary store in question. This is done using the [[https://svnweb.cern.ch/trac/atlasoff/browser/Control/xAODRootAccess/trunk/xAODRootAccess/TEvent.h#L102][xAOD::TEvent::setAuxItemList]] function. By putting something like this into your algorithm's =initialize()= function:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// Set which variables not to write out:</div> <div id="_mcePaste">event->setAuxItemList( "AntiKt4EMTopoJetsAux.", "-NumTrkPt1000.-NumTrkPt500" );</div> <div id="_mcePaste">// Set which variable to do write out:</div> <div id="_mcePaste">event->setAuxItemList( "GoodJetsAux.", "JetGhostArea.TrackCount" );</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Unfortunately at the time of writing this still has some issues when using multiple input files (code crashing on a few files into the job), but the formalism is going to be this once the code works as intended...</div> <div id="_mcePaste">---+ 10. More !EventLoop features</div> <div id="_mcePaste">!EventLoop is an ASG supported ROOT tool for handling optimized event looping and management. Below are some nice features of using !EventLoop in your analysis. One nice feature is the different drivers available to easily switch between running your code locally, on the Grid, using PROOF, or on a batch system.</div> <div id="_mcePaste">A more complete description of the tool can be found on the dedicated [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop][EventLoop]] wiki page.</div> <div id="_mcePaste">---++ Running on the grid</div> <div id="_mcePaste">In this section of the tutorial we will teach you how to run on the grid. There are two main advantages to running on the grid: First, you have access to the vast computing power of the grid, which means even very lengthy jobs can finish within hours, instead of days or weeks. Second, you have direct access to all the datasets available on the grid, which means you don't need to download them to your site first, which can save you hours if not days of time. There are also two main disadvantages: First, for all but the simplest jobs your turnaround time will be measured in hours, if not days. And this time can vary depending on the load at the various grid sites. Second, there are more things beyond your control that can go wrong, e.g. the only grid site with your samples may experience problems and go offline for a day or two thereby delaying the execution of your jobs.</div> </strong> <div id="_mcePaste"> *As a first step, set up the Panda client which will be needed for running on the grid. You should set these up %RED%before%ENDCOLOR% setting up root, so it's probably best to *start from a clean shell* and issue the commands:</div> <div id="_mcePaste">%SYNTAX{ syntax="sh" }%</div> <div id="_mcePaste">setupATLAS</div> <div id="_mcePaste">lsetup panda</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now navigate to your working area, and *setup your Analysis Release*, following the recommendations above [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#What_to_do_every_time_you_log_in][What to do everytime you log in]].</div> <div id="_mcePaste">The nice thing about using !EventLoop is that you don't have to change any of your algorithms code, we simply change the driver in the steering macro. It is recommended that you use a separate submit script when running on the grid.</div> <div id="_mcePaste">Let's copy the content of =ATestRun.cxx= all the way up to, and including, the =driver.submit= statement into a new file =ATestSubmit.cxx=. Don't forget to change the name of the macro at the beginning of the file to =ATestSubmit=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">void ATestSubmit (const std::string& submitDir)</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">If you did the section on FAX the configuration for !SampleHandler should already be set, otherwise open =ATestSubmit.cxx= and comment out the directory scan and instead scan using DQ2 (shown below). Note since we are just testing this functionality we will use a very small input dataset (a SUSY signal point) so your job will run quickly and you can have quick feedback regarding the success (let's hope it's a success) of your job.</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//const char* inputFilePath = gSystem->ExpandPathName ("$ALRB_TutorialData/p2622/");</div> <div id="_mcePaste">//SH::ScanDir().filePattern("DAOD_SUSY1.08377960._000012.pool.root.1").scan(sh,inputFilePath);</div> <div id="_mcePaste">SH::scanRucio (sh, "mc15_13TeV.370900.MadGraphPythia8EvtGen_A14NNPDF23LO_GG_direct_200_0.merge.DAOD_SUSY1.e4008_a766_a821_r7676_p2666/");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Next, replace the driver with the !PrunDriver:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//EL::DirectDriver driver;</div> <div id="_mcePaste">EL::PrunDriver driver;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">We actually need to specify a structure to the output dataset name, as our input sample has a really really long name, and by default the output dataset name will contain (among other strings) this input dataset name which is too long for the Grid to handle. So after you've defined this !PrunDriver add:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">driver.options()->setString("nc_outputSampleName", "user.lheelan.test.%in:name[2]%.%in:name[6]%");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">where you should replace =lheelan= with your Grid nickname (usually same as your lxplus username), the argument [2] will put the datasetID (MC ID or run number) and [6] will put the AMI tags (basically we are removing this "physics short" text). Note that if you wanted to rerun this again with the same input but a slightly different analysis setting you would need to come up with a different output dataset name (or you will get an error telling you this output dataset name already exists). You need to have unique dataset names.</div> <div id="_mcePaste">The !PrunDriver supports a number of optional configuration options that you might recognize from the prun program. If you want detailed control over how jobs are submitted, please consult this page for a list of options: https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#Grid_Driver</div> <div id="_mcePaste">Finally, submit the jobs as before:</div> <div id="_mcePaste">%SYNTAX{ syntax="sh" }%</div> <div id="_mcePaste">root -l -b -q '$ROOTCOREDIR/scripts/load_packages.C' 'ATestSubmit.cxx ("myGridJob")'</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">This job submission process may take a while to complete - do not interrupt it! You will be prompted to enter your Grid certificate password. When all jobs are submitted to panda, a monitoring loop will start. Output histograms for a each input dataset will be downloaded as soon as processing of that dataset is completed. You can stop the monitoring loop with ctrl+c and restart it later by calling</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::Driver::wait("myGridJob");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">or do a single check on the status of the jobs and retrieve and new output:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::Driver::retrieve("myGridJob");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">See [[AtlasProtected.EventLoop#Submitting_a_Job_in_Batch_Mode][here]] for more info on how to create a separate retrieve script which could contain any post-processing code that you want to run once the jobs are finished. If you do not want to enter the monitoring loop in the first place you can, at the end of =ATestSubmit.cxx=, replace =driver.submit= with:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// submit the job, returning control immediately if the driver supports it</div> <div id="_mcePaste">driver.submitOnly (job, submitDir);</div> <div id="_mcePaste">}</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">If you need to log out from your computer but you still want output to be continuously downloaded so that it is immediately available when you come back, a somewhat more advanced !GridDriver exists which will use Ganga and !GangaService to keep running in the background, see the [[AtlasProtected.EventLoop][EventLoop]] twiki page for more info.</div> <div id="_mcePaste">You can follow the evolution of your jobs by going to http://bigpanda.cern.ch, clicking on users and then finding yourself in the list.</div> <div id="_mcePaste">%I% *Specifying paths to local files:* When running on the grid, your local file system is of course inaccessible. All axuiliary files needed by your job must be placed in the =share/= directory in one of your !RootCore packages and accessed in your source code using paths of the form $ROOTCOREBIN/data/PackageName/FileName.root. When the package is compiled (via =rc compile=) !RootCore will create a symbolic link in =$ROOTCOREBIN/data/PackageName/= to the actual file in the package.</div> <div id="_mcePaste">(take a look around inside the !RootCore directory after compiling to see how symlinks on this form are automatically created). It is a good practice to always use this system, as it works with all drivers. For example, with the Good Runs List we showed you earlier, copy the GRL from it's afs area to your area: <br/></div> <div id="_mcePaste">=MyAnalysis/share/= <br/></div> <div id="_mcePaste">( =$ROOTCOREBIN= should be defined after you setup the Analysis Release, you may need to manually create the =share/= directory). Then in your source code in the initialization of the GRL tool specify the path to this file with something like (put these lines in the appropriate places):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">const char* grlFilePath = "$ROOTCOREBIN/data/MyAnalysis/data15_13TeV.periodAllYear_DetStatus-v73-pro19-08_DQDefects-00-01-02_PHYS_StandardGRL_All_Good_25ns.xml";</div> <div id="_mcePaste">const char* fullGRLFilePath = gSystem->ExpandPathName (grlFilePath);</div> <div id="_mcePaste">vecStringGRL.push_back(fullGRLFilePath);</div> <div id="_mcePaste">ANA_CHECK(m_grl->setProperty( "GoodRunsListVec", vecStringGRL));</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Don't forget to recompile!</div> <div id="_mcePaste">*Note: If you are using the [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Alternative_Run_the_job_from_a_c][compiled application to run your macro]], you need to add two things:*</div> <div id="_mcePaste">* to =MyAnalysis/cmt/Makefile.RootCore= add =EventLoopGrid= to the list of =PACKAGE_DEP= </div> <div id="_mcePaste">* in your compiled steering macro =MyAnalysis/util/testRun.cxx= add this near the top with the other header include statements: =#include "EventLoopGrid/PrunDriver.h"== </div> <div id="_mcePaste">Don't forget to run =rc find_packages before compiling since you've just updated the package dependencies!</div> <div id="_mcePaste">If you need more information on options available for running on the grid, check out the [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#Grid_Driver][grid driver documentation]].</div> <div id="_mcePaste">---++ Using !TTreeCache</div> <div id="_mcePaste">The speed with which your jobs will run will in most cases be dominated by the speed with which you can read your input data. So it is often worthwhile to try to maximize that speed.</div> <div id="_mcePaste">One way to improve read performance is to use !TTreeCache, which will predict the data that you are likely to access and preload it in big chunks. This is mostly important when reading files over the network, and it is virtually mandatory in case you read the files directly from the grid (see the next section on FAX).</div> <div id="_mcePaste">Using !TTreeCache with !EventLoop is very straightforward, just specify the size of the cache you would like for your job before submitting it (in this case 10MB):</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optCacheSize, 10*1024*1024);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">The default way in which !TTreeCache predicts what data your job will access is by looking at the first n events and assume that the pattern will hold for the rest of your jobs. If you want to, you can change the number of events to be read to suit your needs:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optCacheLearnEntries, 20);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">You may have to play around a little to determine which number works best for you. There is a definite tradeoff, in that a too large number will mean that you read too many events without the benefit of the cache, while a too small number means that you will not be able to cache variables that you do not access on every event.</div> <div id="_mcePaste">---++ Using FAX</div> <div id="_mcePaste">FAX is a mechanism that allows you to read data directly from the grid without downloading it first. Depending on your usage pattern this may not only be more convenient, but also faster than downloading the files directly: If you download a file, you have to download it in its entirety, whereas if you read the file directly from the grid you may read only a small fraction of it. So if you are reading the file only once (or only a few times) this may be an option to improve your analysis workflow.</div> <div id="_mcePaste">%W% *Warning:* While not strictly necessary, it is *strongly* recommended that you use !TTreeCache when using FAX, otherwise your performance is likely to be very poor. So if you haven't done so already, you should work through the section above on !TTreeCache first.</div> <div id="_mcePaste">*Exit your shell and start a new session on lxplus (or your working environment).* As always, we have to setup the atlas environment:</div> <div id="_mcePaste">%SYNTAX{ syntax="sh" }%</div> <div id="_mcePaste">setupATLAS</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Now in your new shell source the environments to use the FAX tools and establish a voms proxy:</div> <div id="_mcePaste">%SYNTAX{ syntax="sh" }%</div> <div id="_mcePaste">lsetup fax</div> <div id="_mcePaste">voms-proxy-init -voms atlas</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">And when prompted enter your grid certificate password. The first line here takes care of setting up FAX and other necessary tools. If you do not work on lxplus you may need to do something different. The second line will initialize the VOMS proxy that holds your grid certificate. If you have problems with that command you should ask your resident grid expert. There are too many things that can go wrong with that to discuss them here.</div> <div id="_mcePaste">Navigate to your working area, and from there setup your Analysis Release, following the recommendations above [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#What_to_do_every_time_you_log_in][What to do everytime you log in]].</div> <div id="_mcePaste">Now it's time to actually use FAX. For that, in =ATestRun.cxx= comment out the part where we scan the local directory for samples, and instead scan DQ2:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">//const char* inputFilePath = gSystem->ExpandPathName ("$ALRB_TutorialData/p2622/");</div> <div id="_mcePaste">//SH::ScanDir().filePattern("DAOD_SUSY1.08377960._000012.pool.root.1").scan(sh,inputFilePath);</div> <div id="_mcePaste"></div> <div id="_mcePaste">SH::scanRucio (sh, "mc15_13TeV.370900.MadGraphPythia8EvtGen_A14NNPDF23LO_GG_direct_200_0.merge.DAOD_SUSY1.e4008_a766_a821_r7676_p2666/");</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">(Note we are using a small SUSY input dataset so your test job can run quickly.)</div> <div id="_mcePaste">That should do it. You can now just run your script the same way you did before and with a little luck you will get the same result as before. The initialization will take a little longer, as this will actually query dq2 to find the datasets matching your request, and then again for each dataset to locate the actual files. However, compared to the overall run-time this overhead should be small, and the power you gain is most likely worth it.</div> <div id="_mcePaste">---++ Using PROOF Lite</div> <div id="_mcePaste">%RED% Warning: %ENDCOLOR% PROOF-Lite is not currently working with ROOT6. A jira ticket has been filed here:</div> <div id="_mcePaste">https://its.cern.ch/jira/browse/ATLASG-26</div> <div id="_mcePaste">PROOF lite is a fairly effective and easy way of improving the performance of your analysis code. PROOF lite will execute your code in parallel on every CPU core available on your local machine, so if your machine has 16 cores, your analysis will run up to 16 times as fast. The actual amount of speed-up will depend on a variety of things, the number of cores in your machine, the speed of your hard drive, how you read your data, the amount of processing you do for each event, etc.</div> <div id="_mcePaste">Using PROOF lite is very straight-forward, you just change the driver statement in =ATestRun.cxx= to:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::ProofDriver driver;</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">That's it. And run as usual:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">root -l '$ROOTCOREDIR/scripts/load_packages.C' 'ATestRun.cxx ("submitDir")'</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">Note you may get an error like:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">Function SETUP_e89fa50d() busy. loaded after "/cvmfs/atlas.cern.ch/repo/sw/ASG/AnalysisBase/2.0.14/RootCore/scripts/load_packages.C"</div> <div id="_mcePaste">Error: G__unloadfile() Can not unload "/cvmfs/atlas.cern.ch/repo/sw/ASG/AnalysisBase/2.0.14/RootCore/scripts/load_packages.C", file busy ...</div> <div id="_mcePaste">Note: File "/cvmfs/atlas.cern.ch/repo/sw/ASG/AnalysisBase/2.0.14/RootCore/scripts/load_packages.C" already loaded</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">But it does not cause any problems.</div> <div id="_mcePaste">There are a couple of practical changes though. The main one is that any messages your job prints while processing events no longer appear on the screen. Instead they go into a log file in the =~/.proof= directory hierarchy. In general it is a little hard to debug jobs in PROOF, so if something goes wrong there is an advantage to running with =DirectDriver= instead.</div> <div id="_mcePaste">For PROOF farm support see [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#PROOF_Farm_Support][here]]. (Note I have not tested this with the xAOD ... yet.)</div> <div id="_mcePaste">---++ Using Lxbatch and Other Batch Systems</div> <div id="_mcePaste">If you work on lxplus, you can run your job on the local batch system: lxbatch. This allows you to run on many nodes in parallel, improving the turnaround time on your job drastically. If you work on your institutions tier 3 site you may be able to do something similar there by using the correct driver for your batch system. For lxbatch just change the driver statement in =ATestRun.cxx= to:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">EL::LSFDriver driver;</div> <div id="_mcePaste">driver.options()->setString (EL::Job::optSubmitFlags, "-L /bin/bash"); // or whatever shell you are using</div> <div id="_mcePaste">driver.shellInit = "export ATLAS_LOCAL_ROOT_BASE=/cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase && source ${ATLAS_LOCAL_ROOT_BASE}/user/atlasLocalSetup.sh";</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Note the =shellInit= parameter, which is used to set up AtlasLocalSetup on each of the worker nodes. If you run somewhere else than lxplus you will have to update that. Other supported batch drivers can be found here: <br/></div> <div id="_mcePaste">[[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/EventLoop#Batch_System_Drivers][Event Loop: Batch System Drivers]]</div> <div id="_mcePaste">*Note: If you are using the [[AtlasComputing.SoftwareTutorialxAODAnalysisInROOT#Alternative_Run_the_job_from_a_c][compiled application to run your macro]], you need to do one thing:*</div> <div id="_mcePaste">* in your compiled steering macro =MyAnalysis/util/testRun.cxx= add this near the top with the other header include statements: =#include "EventLoop/LSFDriver.h"== </div> <div id="_mcePaste">---++ Variable usage statistics</div> <div id="_mcePaste">If you would like to know how often each of the variables inside the xAOD are accessed by your analysis code, you can simply add the following lines to your steering macro (=ATestRun.cxx) after you have defined the =EL::job= object:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">// print to screen information about xAOD variables used</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optXAODPerfStats, 1);</div> <div id="_mcePaste">job.options()->setDouble (EL::Job::optPrintPerFileStats, 1);</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">When you run the code you will get a long print out listing all the variables inside the xAOD and how many times you've accessed each of them. You probably don't want to run this by default, but it might be useful if you are trying to determine which variables to save to a derived xAOD.</div> <div id="_mcePaste">---+ 11. More !SampleHandler features</div> <div id="_mcePaste">This section (will) describe some nice features of using !SampleHandler in your ROOT analysis. It is used to handle the mass amounts of data and MC typically used in your analysis. It can assign 'metadata' to the samples (luminosity, color schemes, etc.), that are easily retrieved by simply referring to the sample.</div> <div id="_mcePaste">A complete description of this tool can be found on the dedicated [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/SampleHandler][SampleHandler]] wiki.</div> <div id="_mcePaste">---+ 12. More example code</div> <div id="_mcePaste">* Example xAOD example code: [[https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/CPAnalysisExamples][CPAnalysisExamples]]</div> <div id="_mcePaste">* group-dependent analysis frameworks: [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/RunIIAnalysisFrameworks][RunIIAnalysisFrameworks]]</div> <div id="_mcePaste"></div> <div id="_mcePaste">---+ 13. Notes and caveats</div> <div id="_mcePaste">---++ Systematics</div> <div id="_mcePaste">This is a brief summary of how working with systematics and CP tools is expected to work.</div> <div id="_mcePaste">All CP tools that implement some sort of systematic variation(s) will be required to implement a common interface. A first version of this is available [[https://svnweb.cern.ch/trac/atlasoff/browser/PhysicsAnalysis/AnalysisCommon/PATInterfaces/trunk/PATInterfaces/ISystematicsTool.h][here]]. This will first of all make it simpler to hard-code the usage of systematics into the analysis code if the user wants to do that. (Since all CP tools will provide the same interface for setting them up to use different systematics.)</div> <div id="_mcePaste">Then, the tools will be required to "register themselves" into a central registry. This registry will know about all the CP tools that have been created by the analysis code, and have a central list of all the possible systematic variations that can be applied to them. The user will then be able to just interact with this singleton registry to set up his/her code for applying different systematics. Without having to interact with each CP tool one by one. This registry is being developed [[https://svnweb.cern.ch/trac/atlasoff/browser/PhysicsAnalysis/AnalysisCommon/PATInterfaces/trunk/PATInterfaces/SystematicRegistry.h][here]], but is not ready for prime time just yet.</div> <div id="_mcePaste">An example of a tool setting up the systematics registry, and user code using this tool and associated systematic is found in the package =PhysicsAnalysis/AnalysisCommon/CPAnalysisExamples= and is described on this wiki: https://twiki.cern.ch/twiki/bin/view/AtlasComputing/CPAnalysisExamples</div> <div id="_mcePaste">---++ Checklist: Accessing EDM objects</div> <div id="_mcePaste">If you want to access and loop over EDM containers (electrons, taus, etc.), there is a general prescription, similar to what has been shown above:</div> <div id="_mcePaste">* *Source code* =MyAnalysis/Root/MyxAODAnalysis.cxx= </div> <div id="_mcePaste">* in the =execute()= method (that loops event-by-event) retrieve the container type and key name, and iterate over the contents of that container</div> <div id="_mcePaste">* at the very top, include the header file to the xAOD EDM container class of interest</div> <div id="_mcePaste">* *Makefile* =MyAnalysis/cmt/Makefile.RootCore== </div> <div id="_mcePaste">* add the appropriate package to the =PACKAGE_DEP list to point the compilation to the right location of the xAOD EDM package (usually somewhere in =Event/xAOD=)</div> <div id="_mcePaste">---++ Note: xAODVertex</div> <div id="_mcePaste">The xAODVertex package is depreciated. You can access =xAOD::Vertex= through the xAODTracking package:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODTracking/VertexContainer.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">---++ Note: Including xAOD headers</div> <div id="_mcePaste">ROOT CINT has some trouble with the xAOD code.</div> <div id="_mcePaste">If you include an xAOD header file in your algorithms header file, like adding the following to your =MyAnalysis/MyAnalysis/MyxAODAnalysis.h=:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#include "xAODJet/JetContainer.h"</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">You will receive an extremely cryptic error message upon compiling ( =rc compile=); something to the affect of:</div> <div id="_mcePaste"><verbatim style="background: #e0ebf6;"></div> <div id="_mcePaste">Error: class,struct,union or type __int128 not defined /afs/cern.ch/user/l/lheelan/public/ROOT/MT_July2014_2.0.3/RootCoreBin/include/boost/config/suffix.hpp:496:</div> <div id="_mcePaste">...</div> <div id="_mcePaste">Error: Symbol hash_node<unsigned long,0> is not defined in current scope /afs/cern.ch/user/l/lheelan/public/ROOT/MT_July2014_2.0.3/RootCoreBin/include/CxxUtils/hashtable.h:347:</div> <div id="_mcePaste"></verbatim></div> <div id="_mcePaste">This is ROOT CINT attempting to make a dictionary for your algorithm but having trouble dealing with the xAODJet EDM....</div> <div id="_mcePaste">The moral of this story is: *Do not include the xAOD EDM header files in any headers for which you want to use for CINT dictionary generation* (so like your algorithm). You can include them in your source code.</div> <div id="_mcePaste">If you absolutely need access to these classes in your header file, you should hide these includes from CINT. With something like this:</div> <div id="_mcePaste">%SYNTAX{ syntax="cpp" }%</div> <div id="_mcePaste">#ifndef __MAKECINT__</div> <div id="_mcePaste"># include "xAODJet/Jet.h"</div> <div id="_mcePaste">#endif // not __MAKECINT__</div> <div id="_mcePaste">...</div> <div id="_mcePaste">#ifndef __MAKECINT__</div> <div id="_mcePaste">void someFunction( const xAOD::Jet& jet ) const;</div> <div id="_mcePaste">#endif // not __MAKECINT__</div> <div id="_mcePaste">%ENDSYNTAX%</div> <div id="_mcePaste">Note that like this you can have functions that depend on xAOD parameter/return types, but you can't really declare an xAOD member variable to your class. (At least, it can only be done *very* carefully.) But in any case, declaring an xAOD member for an !EventLoop algorithm doesn't seem like a smart idea anyway.</div> <div id="_mcePaste">Forward declaring xAOD types could in principle be another way to go, but that's not any simpler than hiding the includes. And that also results in your code explicitly depending on a certain version of the xAOD EDM.</div> <div id="_mcePaste">A final note, this will not be a problem with ROOT6. Hopefully soon (timescale?) the Analysis Release will switch to be a ROOT6 based release and this problem will be solved (as dictionary generation will not be done with CINT, instead clang).</div> <div id="_mcePaste">---++ Note: TEvent and TStore</div> <div id="_mcePaste">To have a container in xAOD::TEvent it must be coming from an input file, or going to an output file. It does not act like a generic whiteboard like !StoreGate in Athena. If you want to work with containers in your analysis that are not in xAOD::TEvent (so not associated to the input or output file, or transient data) you can record them to the xAOD::TStore. When objects/containers are recorded to xAOD::TStore it takes ownership of them, and handles the memory management for you (the deletion).</div> <div id="_mcePaste">---+ Getting Help</div> <div id="_mcePaste">If you have questions about this tutorial or doing analysis on an xAOD please post your questions to the following mailing list: <br/></div> <div id="_mcePaste">hn-atlas-PATHelp@cern.ch</div> <div id="_mcePaste">Sometimes it is best to browse the code to figure out what's going on. There are two places I find useful to search for code:</div> <div id="_mcePaste">* [[http://acode-browser2.usatlas.bnl.gov//lxr-rel20/search][LXR]] which allows you to browse code which is in some official Athena version. This can be useful as most (all?) of the CP tools are dual-use, meaning they exist in Athena-land too.</div> <div id="_mcePaste">* [[https://svnweb.cern.ch/trac/atlasoff/browser][svn offline browser]] from which you can navigate through the packages to find the code of interest. The challenge here is knowing in which package the code you are interested in lives. After setting up your analysis release you can type =rc version= to see the package layout (and tags) of all the packages included in the release. This svn offline browser holds the code for both Athena and the Analysis Releases.</div> <div id="_mcePaste">---+ If you've got this far...</div> <div id="_mcePaste">... Congratulations! :-)</div> <div id="_mcePaste">And if you have got this far then feel free to start (migrating) your own analysis code, and take advantage of the experts around to help you!!!</div> <div id="_mcePaste">---+ Older updates to this tutorial</div> <div id="_mcePaste">05.08.2015:</div> <div id="_mcePaste">* fixed typo in arguments of JER tool =getUncertainty= (thanks O.Brandt)</div> <div id="_mcePaste">28.07.2015:</div> <div id="_mcePaste">* updated to 2.3.21</div> <div id="_mcePaste">* updated commands necessary to [[SoftwareTutorialxAODAnalysisInROOT#Running_on_the_grid][submit to the Grid]]</div> <div id="_mcePaste">14.07.2015:</div> <div id="_mcePaste">* updated implementation of [[SoftwareTutorialxAODAnalysisInROOT#Muon_calibration_and_smearing_to][systematics]] to use =make_systematics_vector= to create list of systematics to run over</div> <div id="_mcePaste">* updated to 2.3.18</div> <div id="_mcePaste">* this required updating the instantiation of the [[SoftwareTutorialxAODAnalysisInROOT#Jet_energy_resolution][jet energy resolution tool]]</div> <div id="_mcePaste">* this required updating the initialization of properties in the [[SoftwareTutorialxAODAnalysisInROOT#Jet_cleaning_tool][jet cleaning tool]] to set the property ="DoUgly"= and use one of the two =CutLevels= to either ="LooseBad"= or ="TightBad"= (previously had ="MediumBad"= which is not an option anymore)</div> <div id="_mcePaste">25.06.2015:</div> <div id="_mcePaste">* using Analysis Base 2.3.15</div> <div id="_mcePaste"></div> <div id="_mcePaste">08.05.2015:</div> <div id="_mcePaste">* updated to MC15 input samples, and correspondingly to Analysis Base 2.3.11 (some hints on updating your analysis code from DC14/Release19 samples to MC15/Release20 can be found [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PhysicsAnalysisWorkBookRel20xAODContent#Differences_between_Release_19_a][here]])</div> <div id="_mcePaste">07.05.2015:</div> <div id="_mcePaste">* removed the forward declarations to just including the regular old xAOD headers, as we are using ROOT6 there are no more CINT (ROOT5) problems.</div> <div id="_mcePaste">26.03.2015:</div> <div id="_mcePaste">* updating to use 2.1.29</div> <div id="_mcePaste">* using checkxAOD.py script in the Analysis Release (2.X.Y, where Y>=29) to find the container key names and types (instead of checkSG.py in Athena)</div> <div id="_mcePaste">19.03.2015:</div> <div id="_mcePaste">* changed the =xAOD::TEvent= pointer, previously was member variable =m_event=, now there is one created in each method where it is used (better coding practice, improves flexibility) [[SoftwareTutorialxAODAnalysisInROOT#Using_x_AODRootAccess_in_our_Eve][(details)]]</div> <div id="_mcePaste">* added =EL_CHECK_RETURN= macro, instead of the mix-and-match status code check previously implemented [[SoftwareTutorialxAODAnalysisInROOT#Return_codes][(details)]]</div> <div id="_mcePaste">27.02.2015:</div> <div id="_mcePaste">* Using 2.1.27 (ROOT6!!!); a description of some of the differences can be found [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AnalyseBaseRoot6][here]]</div> <div id="_mcePaste">17.02.2015:</div> <div id="_mcePaste">* Using 2.1.25 (ROOT6!!!); a description of some of the differences can be found [[https://twiki.cern.ch/twiki/bin/view/AtlasProtected/AnalyseBaseRoot6][here]]</div> <div id="_mcePaste">* Currently the instructions below will work in both ROOT5 (Analysis Release 2.0.X) and ROOT6 (Analysis Release 2.1.X) - this may change shortly.</div> <div id="_mcePaste">* New xAOD samples: Data with AMI tag p1814 and DC14 MC 13 !TeV with AMI tag r5787. These samples should have the correct trigger information.</div> <div id="_mcePaste">* Use !EventLoop to only run over the first 500 events.</div> <div id="_mcePaste">* FYI: The last stable revision of this tutorial (2.0.22 and older xAODs was [[https://twiki.cern.ch/twiki/bin/view/AtlasComputing/SoftwareTutorialxAODAnalysisInROOT?rev=112][r112]])</div> <div id="_mcePaste"><!-- <strong>*********************************************************</strong> --> <!-- Do NOT remove the remaining lines, but add requested info as appropriate --> <!-- <strong>*********************************************************</strong> --></div> <div id="_mcePaste">---</div> <div id="_mcePaste"><!-- For significant updates to the topic, consider adding your 'signature' (beneath this editing box) --> *Major updates*:%BR% -- Main.LouiseHeelan - 21 May 2014</div> <div id="_mcePaste"><!-- Person responsible for the page:</div> <div id="_mcePaste">Either leave as is - the creator's name will be inserted;</div> <div id="_mcePaste">Or replace the complete REVINFO tag (including percentages symbols) with a name in the form Main.TwikiUsersName --></div> <div id="_mcePaste">%RESPONSIBLE% %REVINFO{"$wikiusername" rev="1.1"}% %BR% <!-- Once this page has been reviewed, please add the name and the date e.g. Main.StephenHaywood - 31 Oct 2006 --></div> <div id="_mcePaste">%REVIEW% *Never reviewed*</div> <div id="_mcePaste">%STOPINCLUDE%</div>
E
dit
|
A
ttach
|
Watch
|
P
rint version
|
H
istory
: r1
|
B
acklinks
|
V
iew topic
|
Ra
w
edit
|
M
ore topic actions
Topic revision: r1 - 2016-12-06
-
AmalVaidya
Home
Site map
Main web
ATLAS_2010_S8894728 web
Sandbox web
TWiki web
Sandbox Web
Create New Topic
Index
Search
Changes
Notifications
RSS Feed
Statistics
Preferences
P
P
View
Raw View
Print version
Find backlinks
History
More topic actions
Edit
Raw edit
Attach file or image
Edit topic preference settings
Set new parent
More topic actions
Account
Log In
Register User
E
dit
A
ttach
Copyright © 2008-2025 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki?
Send feedback