GithubHelp home page GithubHelp logo

xroot: improve performances about hep HOT 2 OPEN

sbinet avatar sbinet commented on August 17, 2024
xroot: improve performances

from hep.

Comments (2)

Moelf avatar Moelf commented on August 17, 2024

I was just about to post a performance question (apparently not the same file, but the branch is exactly the same type):

julia> using UnROOT
[ Info: Precompiling UnROOT [3cd96dde-e98d-4713-81e9-a4a1b0235ce9]

julia> const txrd = LazyTree("root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root", "Events");

julia> @time sum(txrd.nMuon[1:10^5])
  1.723536 seconds (3.45 M allocations: 173.713 MiB, 2.67% gc time, 29.63% compilation time)
0x0000000000039bbb

julia> @time sum(txrd.nMuon[1:10^5])
  1.174588 seconds (1.27 k allocations: 1.049 MiB)
0x0000000000039bbb

julia> const thttp = LazyTree("https://jiling.web.cern.ch/jiling/public/Run2012B_DoubleMuParked.root", "Events");

julia> @time sum(thttp.nMuon[1:10^5])
  0.847792 seconds (146.22 k allocations: 9.412 MiB, 8.69% compilation time)
0x0000000000039647

julia> @time sum(thttp.nMuon[1:10^5])
  0.287621 seconds (752 allocations: 1.516 MiB)
0x0000000000039647

julia> @time sum(thttp.nMuon[1:10^5])
  0.279659 seconds (719 allocations: 1.516 MiB)
0x0000000000039647

both of these have the same underlying logic beyond I/O source/sink. The HTTP one is not using Multipart GET so it's a fair comparison. Both of these are "CERN" but evidently not the same network, before I've noticed eospublic have pretty good infrastructure for the root:// so it's telling that it's this much slow.

One thing is very different is root:// doesn't seem to benefit from async call, I wonder if it's because I'm not using Go library correctly: should ReadAt() be capable of being called in parallel?

from hep.

sbinet avatar sbinet commented on August 17, 2024

#923 brought the gap of performances to:

  $> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null

  real	0m7.279s
  user	0m8.221s
  sys	0m1.256s

  $> time root-dump https://cern.ch/binet/big-file.root > /dev/null

  real	0m5.454s
  user	0m6.156s
  sys	0m0.228s

(and that's with the https plugin downloading the whole file locally and serving it as a local file)

so there are still some performance bits to recoup.
but the 10x drop has been fixed.

from hep.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.