Add a pt-dependent mean(response) plot about cmssw

[jpata]

I added the pt-dependent mean-of-response plot. For the moment, I do not compute errors, as the standard error of the mean (stddev/sqrt(n), which estimates how far the sample mean is from the population mean), is ~1-2, such that the error bars eclipse everything else on the plot.
@juska do you have a corresponding plot from the NanoAOD side?

response_pt.pdf

[hatakeyamak]

Looks pretty good. Thanks @jpata .
Well, except pt=20 GeV point. I don't think Juska saw this jump. Any clue?

[jpata]

This is also visible in the raw response distributions (attached), which are asymmetrical and skewed to large R. So for low-pt gen-jets, the matched reco-jet contains extra stuff. Actually I was looking at the sample again and contrary to what I said earlier, I'm using a sample with pileup, as that was the only one available at CERN: /RelValQCD_FlatPt_15_3000HS_13/CMSSW_9_4_9-PU25ns_94X_mc2017_realistic_v14_rsb-v1/GEN-SIM-DIGI-RAW.
Even though I'm using the slimmed jets from MiniAOD with CHS, possibly this is from PU that @juska didn't have? I'm also setting up other samples now (#17).
response_10_24.pdf

[hatakeyamak]

Ha. you were using PU samples. Interesting.

Yes, let's look at zero PU. sample may be available for newer releases.

[jpata]

Implemented.

[juska]

Hi all, I modified my code to follow the CMSSW convention in jet matching. In practice I now to rely on the match index stored to nanoAOD - need to still check from nanoAOD producer codes that the smallest delta R convention is indeed used. I reproduced response plots with 1000 and 82500 event statistics for comparing with Joosep's latest resolution results: https://jpata.web.cern.ch/jpata/pfvalidation/2019_01_29/plots/FlatQCD_ParticleFlow/reso_pt.pdf http://juska.web.cern.ch/juska/reso_naodmatch_1k.pdf http://juska.web.cern.ch/juska/reso_naodmatch_83k.pdf In the DQM plot the resolution starts from 0.3, where in my nanoAOD-based plots it starts from 0.2. As the behavior (i.e. shape) is otherwise similar, I think it's a good assumption that resolution in nanoAOD samples is smaller due to not having PU. I'm still baffled why the error bars do not work - the uncertainty estimate is similar in the plots with 1000 event and 82500 event statistics. I need to dig into this closer - maybe where the TH1:GetStdDev does a good job in estimating the width with low-statistics histograms, it for some reason fails to estimate the uncertainty. Cheers, Juska

…

On 1/30/19 10:08 PM, Kenichi Hatakeyama wrote: Ha. you were using PU samples. Interesting. Yes, let's look at zero PU. sample may be available for newer releases. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHKcoDIKaXXPLW7pmeEI29Xh_SaroIE-ks5vIgnmgaJpZM4aa5gb>.

[juska]

Hi Joosep, here's the corresponding version straight from nanoAOD: http://juska.web.cern.ch/juska/response_qcd83k.pdf For me the errors are visible and reasonable, but are mostly hidden behind the markers. I suspect that you calculate the response using corrected reco jet pt as your plot looks more like JEC closure plot, am I right? I get also this weird jump to the first bin, I suspect this is some fluke due to the low-end pT cut in the sample. Need to think a bit more. Cheers, Juska

…

On 1/30/19 8:31 PM, Joosep Pata wrote: I added the pt-dependent mean-of-response plot. For the moment, I do not compute errors, as the standard error of the mean (stddev/sqrt(n), which estimates how far the sample mean is from the population mean), is ~1-2, such that the error bars eclipse everything else on the plot. @juska <https://github.com/juska> do you have a corresponding plot from the NanoAOD side? response_pt.pdf <https://github.com/jpata/cmssw/files/2814134/response_pt.pdf> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHKcoC4XlvyqKJmosInu4gf4aA5PQxw6ks5vIfMKgaJpZM4aa5gb>.

[juska]

Hi, we looked at the resolution uncertainty calculation with Chad again and I think that we solved the problem. Now the errors are small and do scale with event count. See here for resolution studies with 1k, 10k and 83k events: http://juska.web.cern.ch/juska/reso_naodmatch_1k_fixerr.pdf http://juska.web.cern.ch/juska/reso_naodmatch_10k_fixerr.pdf http://juska.web.cern.ch/juska/reso_naodmatch_83k_fixerr.pdf The problem was that I was using the SetBinError method for TProfiles, and this does something very different than setting the bin error in the case of a TProfile, obviously: https://root.cern.ch/root/roottalk/roottalk03/3755.html I switched to using TH1D and now can really set the bin errors. The formula for error that I use (and that is used in CMSSW) is: JER_unc = width/mean * sqrt(width_error**2 / width**2 + mean_error**2 / mean**2) Or as implemented in the JetMET code: https://github.com/cms-jet/JetMETAnalysis/blob/master/JetAnalyzers/bin/jet_response_and_resolution_x.cc#L422 Cheers, Juska

…

On 1/30/19 10:08 PM, Kenichi Hatakeyama wrote: Ha. you were using PU samples. Interesting. Yes, let's look at zero PU. sample may be available for newer releases. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#18 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHKcoDIKaXXPLW7pmeEI29Xh_SaroIE-ks5vIgnmgaJpZM4aa5gb>.