trackreco / cmssw Goto Github PK

in a follow up to Mar 28 discussion in TRK POG, some special treatment may be useful for regions where the overlapping hits/layers are expected. A solution could be to split the layers.
The issue may be more significant in the pixel forward region where it's possible to get even 4 hits per layer.

So far we've been using the approximate sincos and did not see any problems. So, this may in the end stay, unless we suddenly decide to look at loopers or close to that.

There was also a longish-standing idea to improve the propToR to reduce the number of iterations. Depending on how that goes, perhaps nIter will actually be useful.

I made some progress making the propToR in one step.

• Sep 23, 2022 : So far the helix angle parameter computation is available
• todo: check if jacobian derivatives need to be computed as well; and then implement this to replace the 5-iter variant that we have now

• in part, this was already discussed in the mkFit meeting
• in part, to follow up on 07b466d which suggests that IceFPU.h is not used
  • this was discovered in a followup to warnings that show up during compilation RecoTracker/MkFitCore/src/Ice/IceFPU.h:36:25: warning: dereferencing type-punned pointer will break strict-aliasing rules [-Wstrict-aliasing]

@osschar @makortel

Something isn't quite working about rootcling's include paths during standalone build. The easy workaround is just to leave WITH_ROOT undefined for make.

...
make[1]: Entering directory '/home/users/slantz/mkFit-standalone/objs-CMS'
rootcling -f ShellDict.cc /home/users/slantz/mkFit-standalone/RecoTracker/MkFitCMS/standalone/Shell.h /home/users/slantz/mkFit-standalone/RecoTracker/MkFitCMS/standalone/ShellLinkDef.h
In file included from input_line_9:3:
/home/users/slantz/mkFit-standalone/RecoTracker/MkFitCMS/standalone/Shell.h:4:10: fatal error: 'RecoTracker/MkFitCore/interface/Hit.h' file not found
#include "RecoTracker/MkFitCore/interface/Hit.h"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Error: rootcling: compilation failure (./ShellDict8148ac5312_dictUmbrella.h)
make[1]: *** [/home/users/slantz/mkFit-standalone/RecoTracker/MkFitCMS/standalone/Makefile:31: ShellDict.cc] Error 1
make[1]: Leaving directory '/home/users/slantz/mkFit-standalone/objs-CMS'
make: *** [Makefile:15: all] Error 2

The supposedly missing include file really does exist at the given location. The mkFit code version I'm using is from a recent head of CMSSW, commit cbe3152 on 2/21/23. ROOT is obtained from cvmfs by source-ing the following script (the if condition is slightly modified for cgpu-1, based on the current RecoTracker/MkFitCore/standalone/xeon_scripts/init-env.sh):

[1539] slantz@cgpu-1 ~/mkFit-standalone$ cat RecoTracker/MkFitCore/standalone/xeon_scripts/init-env.sh
#!/bin/bash
if [[ `[ -f /usr/bin/lsb_release ] && lsb_release -si` == \
        "Fedora" ||
      `[ -f /etc/redhat-release ] && awk '{print $1}' /etc/redhat-release` == \
        "AlmaLinux" ]]
then
  source /cvmfs/cms.cern.ch/el8_amd64_gcc10/lcg/root/6.24.07-da610b2b7ed663a0a05d3605f3d83ceb/etc/profile.d/init.sh
  export TBB_GCC=/cvmfs/cms.cern.ch/el8_amd64_gcc10/external/tbb/v2021.5.0-e966a5acb1e4d5fd7605074bafbb079c/
else
  source /cvmfs/cms.cern.ch/slc7_amd64_gcc10/lcg/root/6.24.07-f52350f4e0b802edeb9a2551a7d00b92/etc/profile.d/init.sh
  export TBB_GCC=/cvmfs/cms.cern.ch/slc7_amd64_gcc10/external/tbb/v2021.4.0-d0152ca29055e3a1bbf629673f6e97c4
fi

The resulting rootcling isn't causing the problem, I think, but I don't know much about it:

[1541] slantz@cgpu-1 ~/mkFit-standalone$ which rootcling
/cvmfs/cms.cern.ch/el8_amd64_gcc10/lcg/root/6.24.07-da610b2b7ed663a0a05d3605f3d83ceb/bin/rootcling

Otherwise I am following the instructions in RecoTracker/MkFitCore/standalone/README_buildFromCMSSW.txt, which I believe should still be valid (given the above modification to init-env.sh), and which seem to work when WITH_ROOT is left undefined.

In presence of overlapping module hits we create candidates that are essentially identical and each of them roughly follows the same path in the following layer combinatorics.
IIUC, CKF does not branch in this case and simply adds both hits on one candidate.

This reduces the combinatorial coverage for mkFit.
I'm proposing to dynamically increase maxcand for comb candidates with overlapping hits (perhaps provisional on relatively similar picked up hit chi2).

@osschar @mmasciov we discussed this in 2022.12.07 meeting

This is for discussion of the points raised in PR #122. Solutions can be added to that PR if desired.

Currently mkFitOutputConverter drops all non-valid hits in the produced TrackCandidates. Some of the useful information is lost: we could pass invalid,bad, or other status hits that could be potentially useful in the final fit.

Most of this will likely require additional level of detail added inside of mkFit, e.g.,

• ability to provide a DetId per dead hit
• ability to provide a full hit index for potentially useful hits which were skipped in the pattern reco
  • currently we just assign a -x to the hit index; -> hit index and hit status would need to be tracked separately

to follow up on the discussion in 7fc06ca

To avoid clang format applied on the /standalone/ subdirectories, while the mkFit code was still only partially reincarnated in the cmssw repo, the formatting was disabled in #70

This issue is proposing to review the files with // clang-format off in the /standalone/ subdirectories and apply the formatting where it seems applicable.

In MkFitOutputConverter, sanity check at L.279 causes a crash related to trackreco/mkFit#321

Solutions (in increasing order of preference):
(1) disable sanity check;
(2) make a separate seed-merging module to produce a new collection of TrajectorySeeds, to be used as input to MkFitOutputConverter (output is a TrackCandidate).

DNN configs are inconsistent when adding individual mkFit iterations on top of Run3_noMkFit. Need to refactor qualityCutsDictionary to pick up correct values and not use copy-paste

all CKF iterations (at least the pixel-based) require max consecutive holes of 1, while we have 2 (except in pixelLess or tobTec).
Apparently mkFIt choice goes back to the days before the dead modules were detected.

This issue is to suggest to investigate if going to 1 is feasible, at least in some of the iterations where fakes/combinatorics is significant.

Motivation: moderate outlier hits can distort the trajectory (after update) during building.

The idea is to rescale the hit covariance by a function of the initial hit chi2. The function could just be 1+ chi2 or something smoothed extending to around 2 sigma equivalent, e.g. chi2 < 4 ? 1 : chi2-3 (to be a continuous function).
The rescale would be applied in the Kalman update math and should be straightforward during search where we compute chi2 first and then run the update.

I see build failures that are apparently coming from WriteMemFileDict detection in the rules

Use the following steps:

git clone --branch CMSSW_13_1_0_pre2 --single-branch --no-checkout --reference /cvmfs/cms-ib.cern.ch/git/cms-sw/cmssw.git [email protected]:cms-sw/cmssw.git 1310p2-2023H1-bin
cd 1310p2-2023H1-bin/
git config core.sparsecheckout true
echo -e "/.gitignore\n/.clang-tidy\n/.clang-format" > .git/info/sparse-checkout
echo -e "/RecoTracker/MkFit/\n/RecoTracker/MkFitCMS/\n/RecoTracker/MkFitCore/" >> .git/info/sparse-checkout
echo -e "/FWCore/Utilities/interface/" >> .git/info/sparse-checkout
git checkout CMSSW_13_1_0_pre2
./RecoTracker/MkFitCore/standalone/configure $PWD
source ./RecoTracker/MkFitCore/standalone/xeon_scripts/init-env.sh
make -j 12  AVX2:=1 WITH_ROOT=yes >& a.log &

the last lines in the log (on the first try, actual explicit path in the log is replaced by $BPATH)

rootcling -I=$BPATH -f WriteMemFileDict.cc $BPATH/RecoTracker/MkFitCMS/standalone/tkNtuple/DictsLinkDef.h
rootcling -I=$BPATH -f WriteMemFileDict.cc $BPATH/RecoTracker/MkFitCMS/standalone/tkNtuple/DictsLinkDef.h
mv WriteMemFileDict_rdict.pcm ../WriteMemFileDict_rdict.pcm
mv WriteMemFileDict_rdict.pcm ../WriteMemFileDict_rdict.pcm
mv: cannot stat ‘WriteMemFileDict_rdict.pcm’: No such file or directory
make[1]: *** [WriteMemFileDict.cc] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `$BPATH/objs-CMS'
make: *** [all] Error 2

@osschar

prompted by needs for HeavyIons tracking https://indico.cern.ch/event/1133724/#5-mkfit-usage-for-heavy-ion-tr
where large differences in the number of reconstructed tracks with and without mkFit was found.
These were related to the additional trajectory filter selections enabled by pp_on_AA modifiers.

@mmasciov collected the differences/selections:

It looks like we can have just the pt cut.
As discussed in UCSD meeting we can have a json patch injection via a json-formatted string in the mkFit configuration, e.g. in https://github.com/cms-sw/cmssw/blob/4c8958415ca756c3589be1f92ef0e71ecf200b2e/RecoTracker/IterativeTracking/python/DetachedTripletStep_cff.py#L231-L234
we could have an extra
pp_on_AA.toModify(detachedTripletStepTrackCandidatesMkFitConfig, configPatchString = 'something in json format')

according to @sgnoohc
there is a problem to run the trackingNtuple in a phase-2 setup.
The problem comes up in the clusterMasks configuration which is apparently not (auto)updated to use the phase-2 iterations.

A kludge to make things run is to modify the config with process.trackingNtuple.clusterMasks = []