CCana1

Data/MC Comparisons for Low Hadronic Energy CC-like Events

Outline for Webpage

Introduction to and motivation for the study
'Executive Summary'
Cuts, PoTs, reweighting and samples of the kinematic plane
Data/MC comparisons for low hadronic energy CC-like events
Discussion of CC-like selection and comparison with standard CC selection
Implication of 'detector effects' for the data/MC comparisons (fiducial studies)
Implication of a shift in the hadronic energy scale in MC
Additional thoughts, conclusions and further work
Description of analysis chain (included for completeness and reference)

Introduction and Motivation

In this study I shall investigate data/MC disagreements for low hadronic energy CC-like events.
Low hadronic energy CC-like event samples are dominated by quasi-elastic interactions and it is important to understand any differences between the data and MC before using QEL-like event selections in an analysis.
QEL-like event selections can form the basis for several interesting measurements:
- Extraction of ND neutrino flux via method described in DocDB.
- Extraction of QEL axial-vector mass via QEL-like sample Q^2 distribution. The QEL axial-vector mass is the only free parameter in the differential cross section (for QEL interactions) with respect to Q^2 (see DocDB talk by Rik Gran on the K2K analysis).
- QEL-only oscillation analysis that takes advantage of the improved neutrino energy resolution for these events (uncertainties in hadronic energy can be avoided).

'Executive Summary'

CC-like samples are selected with cuts on the hadronic shower energy. Data and MC show large differences when these low hadronic energy CC-like samples are considered.
The differences are not large for zero hadronic energy CC-like samples but increase as events with larger and larger showers are included up to a 'peak' data/MC disagreement on the order of 25-30% for events with no more than 200 MeV of hadronic energy.
Data and MC start to agree better again as the hadronic energy cut is allowed increase beyond 0.5-1 GeV.
The data excess occurs at low reconstructed neutrino energy (~3 GeV) and larger reconstructed muon angles with respect to the neutrino direction (~0.25 radians).
Correspondingly the data excess occurs mainly in the QEL/Delta region of the kinematic plane as confirmed by samples with reconstructed invariant mass below 1.3 GeV.
A series of fiducial volume studies rule out 'detector effects' as the cause of this disagreement.
A 10% downward shift of the hadronic energy scale in MC can account for some of the difference between data and MC but is has very little effect for CC-like events with very small hadronic showers (50-150 MeV).
Particulars of the nuclear model could be responsible for (some of) the disagreement but this idea has not been investigated yet.

Cuts, PoTs, Reweighting and Samples of the Kinematic Plane

I apply the following pre-selection cuts to data and MC:
- Reconstructed neutrino energy less than 120 GeV.
- Event has 1 and only 1 track.
- 'Pittsburgh' fiducial volume cut (plots of this volume later on).
- The SR gives the track fit a pass.
- The chi^2 per DoF for this fit is less than 20.
- The difference between the track beginning in the U and V views is no more than 5 planes.
- If muon momentum is measured from curvature the fractional error on this value is no more than 30%.
- Anti-neutrino cut - the muon charge sign is zero or negative.
- Rock muon cut - the track starts at least 0.5m into the detector.
- Runt event cut - either there is no event within 50ns of the current event or the current event has at least 95% of the PH of itself and the nearest event summed together.
Data has to pass the standard beam quality cuts and the special horn current runs are excluded.
CC-like samples are then selected by placing a cut on the CC PID parameter at -0.1.
The exposures for the data and MC samples are as follows:
- LE-10 Data: 7.12e19 PoTs
- LE-10 MC: 8.16e18 PoTs
- pME Data: 1.13e18 PoTs
- pME MC: 3.86e18 PoTs
- pHE Data: 1.60e18 PoTs
- pHE MC: 7.21e17 PoTs
In the following the label 'tuned MC' refers to MC that has been weighted using the 12 parameter fit SKZP2 beam and hadron production weights and the MODBYRS3 Neugen weights. When truth information is shown it always corresponds to the tuned MC.
In some plots I have selected samples of the kinematic plane defined as follows:
- 'Safe' DIS: W^2 > 4, Q^2 > 1
- Low Q^2 DIS: W^2 > 4, Q^2 <= 1
- QEL/Delta: W^2 <= 1.69
- Transition: 1.69 < W^2 <= 4
These samples correspond roughly to kinematic regions where Neugen uses a different interaction model and so can help to diagnose problems. Any plot in this webpage can be remade for any of these samples so please let me know if there is something that you wish to see (med@hep.ucl.ac.uk).

Data/MC Comparisons for Low Hadronic Energy CC-like Events

The following plot shows the reconstructed neutrino energy spectra of my CC-like samples for the 3 beam configurations.
- The reweighting of the MC has the same effects on the spectra as can be seen in the MINOS blessed plots and this gives me confidence that my reweighting procedure is working as expected.
- There are some differences in the spectral shapes as compared to the blessed plots that stem from the different pre-selection cuts that I have used. In particular the low energy part of the spectra for the LE-10 beam shows larger disagreement between the data and the MC than in the blessed plots.
- The next section will investigate where these differences come from and show that low hadronic energy samples with (my best approximation of) the standard CC pre-selection cuts exhibit similar data/MC comparisons as in the figures in this section.
Figure 1. CC-like Reconstructed Neutrino Energy Spectra (No Hadronic Energy Cut)

(Larger version of Figure 1)
In this study I have chosen to focus on 2 quantities, the reconstructed neutrino energy and the muon angle with respect to the neutrino direction. The following plot shows the muon angle distributions of my CC-like samples for the 3 beam configurations.
- It can be seen in the LE-10 plots that the reweighting is most affecting the low muon angle events - these are events from the high energy tail of the energy spectra above.
- The slopes of the ratio plots are indicitive of a shift to higher muon angles in data.
Figure 2. CC-like Reconstructed Muon Angles (No Hadronic Energy Cut)

(Larger version of Figure 2)
Figures 3-6 show the effects of placing cuts on the hadronic energy of events in my CC-like samples. Figure 3 shows how the peak of the reconstructed neutrino energy spectra in data and MC agrees reasonably well for zero hadronic energy events but that significant differences occur when small hadronic shower events are also included. Figure 4 shows the data/MC ratios corresponding to these 4 values of the hadronic shower energy cut.
Figure 3. CC-like Reconstructed Neutrino Energy Spectra With Tight Cuts on Hadronic Energy

(Larger version of Figure 3)
Figure 4. Data / MC Ratios With Tight Cuts on Hadronic Energy
Figure 5 shows that as the cut on the hadronic shower energy is loosened to 500 MeV and then 1 GeV the data/MC agreement begins to get better again. Figure 6 shows the data/MC ratios corresponding to the 4 samples shown in Figure 5.
Figure 5. CC-like Reconstructed Neutrino Energy Spectra With Looser Cuts on Hadronic Energy

(Larger version of Figure 5)
Figure 6. Data / MC Ratios With Looser Cuts on Hadronic Energy
I have tried to summarise the effects shown in the preceding 4 plots by showing the peak and the mean of the data/MC ratios for events with reconstructed neutrino energy between 0 and 5 GeV (where the statistics are reasonable) as a function of the cut placed on the hadronic shower energy. As such these plots are cumulative along the x-axis.
Figure 7. Peaks and Means of the Data / MC Ratios as a Function of Hadronic Energy Cut
Figures 8-12 are repeats of the previous 5 plots but for the reconstructed muon angle distributions. They show excesses in data at larger muon angles for low hadronic energy CC-like samples. The zero hadronic energy data and MC samples do not agree as well in the peak as in the energy spectra plots.
Figure 8. CC-like Reconstructed Muon Angles With Tight Cuts on Hadronic Energy

(Larger version of Figure 8)
Figure 9. Data / MC Ratios With Tight Cuts on Hadronic Energy

Figure 10. CC-like Reconstructed Muon Angles With Looser Cuts on Hadronic Energy

(Larger version of Figure 10)
Figure 11. Data / MC Ratios With Looser Cuts on Hadronic Energy

Figure 12. Peaks and Means of the Data / MC Ratios as a Function of Hadronic Energy Cut
For completeness I include here links to some other distributions:
- Data/MC comparisons of the reconstructed hadronic shower energy:
- Data/MC comparisons of the reconstructed muon energy:
- Data/MC comparisons of the reconstructed Q^2 (reconstructed as 2*Enu*Emu*(1-cosThetaMu)):
- Data/MC comparisons of the reconstructed x (reconstructed as Q^2/2M*Ehad where M is average nucleon mass):
- Data/MC comparisons of the reconstructed y (reconstructed as Ehad/Enu):
- Data/MC comparisons of the reconstructed W^2 (reconstructed as M^2-Q^2+2M*Ehad):
There are many possibilities for the cause of the data/MC disagreements:
- 'Detector' effects coming from, for example, 'missed' hadronic energy leaking out of the edges of the detector. I have considered several different fiducial volumes and their effect on the data/MC ratios below.
- Possible miscalibration of the hadronic shower energy. Below I consider a CalDet motivated +-10% shift to the hadronic energy scale in MC and repeat the analysis of low hadronic energy data and MC CC-like samples.
- Problem with the modelling of nuclear effects. I would like to understand how the details of the nuclear model can affect hadronic shower energies and muon angles. One option is to try to run Neugen with different values of the Fermi momentum used in the Fermi gas model for iron. This value determines whether or not a quasi-elastic interaction is Pauli blocked and so changing it could lead to more/less QEL events and change the energy spectrum of the hadronic showers associated with those events.

Effect of CC-like Sample Pre-Selection Cuts

My pre-selection cuts are not identical to the standard CC pre-selection and were initially developed with QEL samples in mind. I was a little worried as my LE-10 reconstructed neutrino energy spectra data/MC ratio is higher below 4 GeV than the standard CC blessed plot and this is exactly the regime where I am looking into data/MC disagreements.
I have implemented as close as possible a pre-selection to the standard CC cuts (as defined in the CC analysis proto-paper) and compared the distributions to those derived with my pre-selection cuts.
- 'Standard CC Cut' ||||| 'My Version of this Cut'
- # tracks > 0 ||||| # tracks > 0
- 1 < z < 5 and r < 1 ||||| 1 < z < 5 and r < 1
- track fit pass = 1 ||||| track fit pass = 1 and fit chi^2/DoF < 20 and UV difference < 6 planes (unfortunately these are combined in my PAN ntuples)
- reco. muon angle > 0.6 ||||| reco. muon angle > 0.6
- track qp <= 0 ||||| track qp <= 0
The following plots shows the reconstructed neutrino energy spectra. These distributions now almost exactly replicate the equivalent CC blessed plot.
Figure 13. CC-like Reconstructed Neutrino Energy Spectra - Standard CC Pre-Selection

(Larger version of Figure 13)
Figures 14-18 show the effects of cutting on hadronic energy with this standard CC pre-selection. These plots exhibit the same features as with my pre-selection.
Figure 14. Spectra with Tight Cuts on Hadronic Energy - Standard CC Pre-Selection

(Larger version of Figure 14)
Figure 15. Data / MC Ratios With Tight Cuts on Hadronic Energy - Standard CC Pre-Selection

Figure 16. Spectra with Looser Cuts on Hadronic Energy - Standard CC Pre-Selection

(Larger version of Figure 16)
Figure 17. Data / MC Ratios With Looser Cuts on Hadronic Energy - Standard CC Pre-Selection

Figure 18. Peaks and Means of the Data / MC Ratios - Standard CC Selection
I need to investigate further the underlying cause of the differences in the neutrino energy spectra with the 2 pre-selections and decide whether one or the other (or both or neither) is including 'bad' events.

Fiducial Studies

I have investigated the possibility of a systematic 'detector effect' by considering different sub-regions of the 'Pittsburgh' fiducial volume and looking at the data/MC ratios for a different cuts on the hadronic energy. I have considered 3 sets of fiducial region that decompose the radial, x-y and z symmetries of the whole fiducial volume:
- 'Anulii'
- Quadrants with the beam spot removed
- 'Chunks' in z
and 3 CC-like samples within each region:
- All CC-like events
- Zero hadronic energy CC-like events
- CC-like events with hadronic energy less than 250 MeV (where the data/MC disagreements are at their worst)
For each region and sample I have plotted the double ratio of data/MC for the whole Pitt. fiducial volume to data/MC for that region. Deviations of these double ratios from unity indicate the presence of a systematic effect.
Figure 19. Definitions of Anulii

Figure 20. Double Ratio for All CC-like Events

(More detail for Figure 20)
Figure 21. Double Ratio for Zero Hadronic Energy CC-like Events

(More detail for Figure 21)
Figure 22. Double Ratio for Hadronic Energy Less Than 250 MeV CC-like Events

(More detail for Figure 22)
Figure 23. Definitions of Quadrants

Figure 24. Double Ratio for All CC-like Events

(More detail for Figure 24)
Figure 25. Double Ratio for Zero Hadronic Energy CC-like Events

(More detail for Figure 25)
Figure 26. Double Ratio for Hadronic Energy Less Than 250 MeV CC-like Events

(More detail for Figure 26)
Figure 27. Definitions of z 'Chunks'

Figure 28. Double Ratio for All CC-like Events

(More detail for Figure 28)
Figure 29. Double Ratio for Zero Hadronic Energy CC-like Events

(More detail for Figure 29)
Figure 30. Double Ratio for Hadronic Energy Less Than 250 MeV CC-like Events

(More detail for Figure 30)
Although some of the plots (in particular those for the zero hadronic energy samples) have low statistics I cannot see any obvious systematic effect coming from the choice of fiducial volume.

Effect of Hadronic Energy Scale Shift in MC

Here I consider the effects of shifting the hadronic energy scale in the MC by +-10%. This figure was suggested to me by Mike and motivated by CalDet. I have implemented the shift by sending:
- reco_eshw -> reco_eshw * X (where X is either 0.9 or 1.1)
- reco_enu -> reco_emu + (reco_eshw * X)
As before I then consider CC-like samples for data and the shifted MC with cuts on the hadronic shower energy. In the following plots I am just showing data and the tuned MC. The first figure shows the effect of the hadronic energy shifts on the data/MC comparisons of the reconstructed neutrino energy for the 3 beam configurations. The shaded boxes represent the value of the MC sample with shifted hadronic energy in that bin but it should be noted that the statistical errors on the extent of these shaded boxes are about the same as the error bars shown for the corresponding tuned MC point in that bin.
Figure 31. Reconstructed Neutrino Energy for CC-like Events

(Larger version of Figure 31)
The zero hadronic energy CC-like sample will not be affected by a scaling of the hadronic energy and so the next figures show how the shifts affect the reconstructed neutrino energy spectra for different cuts on the hadronic energy.
Figure 32. CC-like Events With Hadronic Energy Less Than 150 MeV

(Larger version of Figure 32)
Figure 33. CC-like Events With Hadronic Energy Less Than 250 MeV

(Larger version of Figure 33)
Figure 34. CC-like Events With Hadronic Energy Less Than 500 MeV

(Larger version of Figure 34)
Figure 35. CC-like Events With Hadronic Energy Less Than 1 GeV

(Larger version of Figure 35)
Shifting the hadronic energy scale in MC down by 10% does help with the data/MC disagreements but cannot account for all of the differences and it's effects are negligable for the very low hadronic energy CC-like samples (as would be expected).
The shift will not affect the total CC-like reconstructed muon angle distributions but can have an effect when cuts are placed on the hadronic energy. The following links show plots of the muon angles for different cuts:

Additional Thoughts, Conclusions and Further Work

Significant differences are observed between data and MC when CC-like events with small showers are considered.
These cannot be accounted for by either detector effects stemming from the choice of fiducial volume or a reasonable size shift in the hadronic energy scale of the MC.
The one remaining investigation that I hope to undertake is the effect that the details of the nuclear model can have on the low hadronic energy CC-like samples. At the moment this may take the form of changing the value of the Fermi momentum in Neugen but Daikon improvements and the possibility of a full spectral function treatment of the nuclear model could also prove useful.
Should I try to investigate the effects of a change in cross-section values and repeat the analysis?
Please let me know if there are further studies that I can undertake to investigate these discrepancies.

Analysis Chain

Generate PAN ntuples from SNTP ntuples.
- I use the Mad framework and several classes that have not yet made it to CVS; MadMedQEAnalysis and NewMadQEID. These create a 'UCL/MadMK style' PAN ntuple with additional branches for variables from my QEL event selection.
Filter the PAN ntuples to reduce their size.
- I use a root macro (MedFilterTree) to only keep events that pass certain cuts (for example that all events have one well reconstructed track and are in a fiducial volume).
For MC PAN ntuples add branches that contain weights corresponding to the 12 parameter fit SKZP2 beam and hadron production reweighting and the MODBYRS3 Neugen reweighting.
- I use another macro that runs in loon (MedReweight) to add these branches.
Use TTree 'MakeClass' formalism to produce histogram files from the PAN ntuples.
- For example the MedPanAna class.
Run configurable root macros to produce plots from the histogram files.
- For example the ThreeBeamEnu macro.