Comments (2)
I was just about to post a performance question (apparently not the same file, but the branch is exactly the same type):
julia> using UnROOT
[ Info: Precompiling UnROOT [3cd96dde-e98d-4713-81e9-a4a1b0235ce9]
julia> const txrd = LazyTree("root://eospublic.cern.ch//eos/root-eos/cms_opendata_2012_nanoaod/Run2012B_DoubleMuParked.root", "Events");
julia> @time sum(txrd.nMuon[1:10^5])
1.723536 seconds (3.45 M allocations: 173.713 MiB, 2.67% gc time, 29.63% compilation time)
0x0000000000039bbb
julia> @time sum(txrd.nMuon[1:10^5])
1.174588 seconds (1.27 k allocations: 1.049 MiB)
0x0000000000039bbb
julia> const thttp = LazyTree("https://jiling.web.cern.ch/jiling/public/Run2012B_DoubleMuParked.root", "Events");
julia> @time sum(thttp.nMuon[1:10^5])
0.847792 seconds (146.22 k allocations: 9.412 MiB, 8.69% compilation time)
0x0000000000039647
julia> @time sum(thttp.nMuon[1:10^5])
0.287621 seconds (752 allocations: 1.516 MiB)
0x0000000000039647
julia> @time sum(thttp.nMuon[1:10^5])
0.279659 seconds (719 allocations: 1.516 MiB)
0x0000000000039647
both of these have the same underlying logic beyond I/O source/sink. The HTTP one is not using Multipart
GET so it's a fair comparison. Both of these are "CERN" but evidently not the same network, before I've noticed eospublic
have pretty good infrastructure for the root://
so it's telling that it's this much slow.
One thing is very different is root://
doesn't seem to benefit from async
call, I wonder if it's because I'm not using Go library correctly: should ReadAt()
be capable of being called in parallel?
from hep.
#923 brought the gap of performances to:
$> time root-dump root://ccxrootdgotest.in2p3.fr:9001/tmp/rootio/testdata/SMHiggsToZZTo4L.root > /dev/null
real 0m7.279s
user 0m8.221s
sys 0m1.256s
$> time root-dump https://cern.ch/binet/big-file.root > /dev/null
real 0m5.454s
user 0m6.156s
sys 0m0.228s
(and that's with the https
plugin downloading the whole file locally and serving it as a local file)
so there are still some performance bits to recoup.
but the 10x drop has been fixed.
from hep.
Related Issues (20)
- distributed use cases? HOT 3
- Xrootd HOT 1
- xrootd: how to read raw bigendian bytes? HOT 8
- hplot: implement interactivity w/ a plot
- hplot: design interface(s) to handle plot interactivity
- pawgo: add support for groot files
- Rename package fwk/testdata (Go reserved name) HOT 2
- groot: go generate failure HOT 2
- Invalid argument xrootd HOT 4
- root-split HOT 2
- cmd/hepmc2root: consider implement this conversion tool HOT 1
- groot: duplicate streamer info entries for "This" streamers
- hepmc: consider adding groot persistency to Event
- hepmc: I/O benchmarks (ASCII and ROOT)
- correctionlib: consider adding a package for applying corrections
- groot: finding child object (multilevel TDirectory) in a ROOT file and conversion of plots to JSON HOT 5
- hplot: consider adding a JSON backend for hplot.Save HOT 4
- groot: handle files produced by mt-G4
- groot/rhist: consider adding JSON serialization for TH{1,2}x
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hep.