=================================================================== MICAC Sub-Committee Minutes - 20th Dec 2005 ------------------------------------------- Attendees: Jim, Jon, Brett, Caius, Sue, Sergei, Rustem, George, Nathaniel, Chris Absent: Robert a) Wishlist Discussion - led by Sergei Numbers refer to items in table at: http://www.hep.ucl.ac.uk/~cbs/Talks/Reco/NewFramework/MICAC_New_Reco_Wishlist.html 1) Agreed. Where Overriding = reimplementation (functions in the base class are blocked by those in the implementation) 2) Agreed. Objects are modifiable for some well defined time. However, need to agree when to fix each type of object. e.g. strips should be fixed before track finding starts. 3) Agreed. 4) Agreed. Handles and Lists are not in the new framework. Brett: Would like to know which Alg produced which reco object. This information is available from existing candidates. Would not want to lose this functionality. Jim: Primary use is production processing, so should know what is being run. Jon: Minerva + Nova will also need a framework, so it's important to ensure software is fully functional. A CandConfig record, written once per job, was suggested as a possible solution. 5) Completely agree. In fact, coding rules will be strictly enforced and documentation will be provided to help code writers. Nathaniel: Can I create temporary candidates? Probably yes, just don't give it to mom to prevent it from being written out to cand file - needs testing. 6) Changes to interfaces of an object necessarily affect all implementations of that object. Can always add functions to an implementation without affecting anyone else, but then to use that method you would need to cast. This wishlist item (from Jim) is not a show stopper since MINOS is in a regime where interface changes should be rare. 7) Agree but orthogonal to central discussion. Can be implemented if necessary. 8) Conflicts with 17) but essentially orthogonal. Can do either. 8) is designed to avoid confusion 17) allows easy comparison between different algs + adding new lists etc. Nathaniel: Can have both, by having multiple event- or snarl-lists. Just need a rigid way to put in/get out events to/from mom. Then can have a pointer to the primary list. Jim: Just need to avoid confusion. 9) Up for discussion. Sergei: Event-centric is preferable:- run slicer to get events, then build tracks + showers within events. Can then use striplist to fix up slicing problems. Sue: Store snarl-based records since I/O is still snarl based. 10) Possible. Going top->bottom is trivial, but no system is built in to go daughter->parent. Requires searches, loops, etc to go upstream, so will be slow, but can and should write generic helper functions that can do this. Nathaniel: Problem is that daugherlists are not generic - need to know what the daughter is. Brett: Can make search functions generic by adding a member which is the type of the daughter. 11) Only proposing changes to candidate files, not to ntuples. Can still have ntuples as they are but need to re-write ntuple maker to use new reco objects. This wishlist item (from Jim) assumed that I/O from new candidates would be fast enough that ntuples would be obsolete. Still too early to decide whether or not a two-step process will be needed - depends on how fast new candidates can be. Nevertheless, moving over to the new framework would likely be a slow transition, so having the current ntuples available from the new framework would be necessary. 12) Agreed. Sergei suggested that some benchmark be defined for I/O speed. At the moment 1 month of ND data can be read in in ~4 hours using ntuples; with candidates, it is 20 times slower. Jon suggested that the current speed of the ntuples would be a good benchmark. Sergei has contacted Phillipe on the ROOT team to try to understand why the I/O is a factor of 10 slower than might be expected. He has replied that he is looking into it and that he is also surprised by the factor of 10. 13) Can be done. Perhaps by adding a Registry to reco objects; then users can add arbitrary variables with names. This would only be allowed for development jobs, not for production. Nathaniel: Could also add a TObject* which is usually 0. Jim: Need to be careful that this does not become the default way to add new variables. Nathaniel: If something is useful it can be promoted later. But not allowed in production running. Rustem: Can make sure that these are not filled/written out for production jobs. Brett: This sounds like a white board and shouldn't be in candidates. Could instead write to a user-designed ntuple record that can be aligned with other objects using mom. Nathaniel: Record alignment is only useful per snarl, not per strip for example. 14) Against this. This is a wishlist item from Nathaniel who objects against the need in the current framework to have a RunAlg() call to make each reco object. In the new framework, algorithms are still the only way to make persistent candidates, but a single algorithm MakeBlah() call will make the entire list. i.e. you do not need an alg for every candidate. 15) Can be done. 16) Agree. Can be done. 17) See 8). 18) Agreed. End of wishlist items. b) Volunteers to work on new framework - Jon Jon reported that there was not significant interest from the people he spoke to and that the majority of the effort in the short term would likely have to come from committee members. However, once the initial CC measurement has been made, people may become more interested in working on the reconstruction. c) Plans for the Collaboration Meeting at Oxford. In light of b) it is important to keep people informed of the status of this subcommittee. A few slides in the reco plenary session and a short presentation in the core software group parallel session was suggested to highlight the motivation for doing this. The parallel session talk would only be useful if enough interested people attend. Chris agreed to contact the ND convenors to find a good time. d) ABC Design Discussion - Sergei, Rustem In light of the discussions at the last meeting, Sergei and Rustem were asked to think about the ABC design and it's role in their proposal. Essentially they have had no change in thinking. The point was made that since promoting new members to the ABCs can significantly impact other parts of the code, the basic problem with switching to the new framework would be organizational - there would need to be a committee that decides if and when things get added to the ABC class. e) Test of TClonesArray vs STL vectors - Sue Sue reported that she has started doing this. She has performed a baseline run with the new code and modified the data structure in order to do an I/O comparison with TClonesArrays. From the baseline run, she sees the factor of 20 improvement in candidate I/O but would like to know how much of this is related to the data structure and how much to the change in size of the candidates. Rustem: TClonesArray's used properly will improve I/O, but it creates specialization and we don't want to break the generality of the proposed framework by forcing use of TObjects. Addendum from Sue: ----------------- Just to be clear, the factor of 20 I referred to was based on a comparison of standard ntuple (NtpSt) i/o speed to that of Cand record. I haven't yet attempted to introduce TClonesArrays to the proposed Reco records, although I plan to do this. Summary: ------- >From the wishlist excercise it has been shown that the new framework provides essentially all the features requested by committee members. There are still details to be worked out, but most of these are orthogonal to the proposed framework design. The major issue related to the design of the new framework is I/O. Work still needs to be done in order to decide on the optimal structure for the candidates. The other major issue is manpower. Action items for the committee: ------------------------------ 1) How do we converge on making a decision to work on this - All 2) TClonesArray vs STL - Sue 3) ROOT team feedback - Sergei [ Part 2, "" Application/PDF 789KB. ] [ Unable to print this part. ]