go-hep / hep Goto Github PK
View Code? Open in Web Editor NEWhep is the mono repository holding all of go-hep.org/x/hep packages and tools
Home Page: https://go-hep.org
License: BSD 3-Clause "New" or "Revised" License
hep is the mono repository holding all of go-hep.org/x/hep packages and tools
Home Page: https://go-hep.org
License: BSD 3-Clause "New" or "Revised" License
From @sbinet on November 16, 2016 9:15
right now, opening a CSV connection is done like so:
func foo() {
// Open the CSV file as a database table.
db, err := csvdriver.Conn{
File: "../data/iris.csv",
Comment: '#',
Comma: ',',
}.Open()
if err != nil {
log.Fatal(err)
}
defer db.Close()
// Start a database transaction.
tx, err := db.Begin()
if err != nil {
log.Fatal(err)
}
defer tx.Commit()
// Define a SQL query that we will execute against the CSV file.
query := "SELECT var3, var4, var5 FROM csv WHERE var5 = \"Iris-versicolor\""
// etc...
}
this makes a query rather cumbersome to read.
it should be possible to open a CSV connection with an optional Columns []string
field:
// Open the CSV file as a database table.
db, err := csvdriver.Conn{
File: "../data/iris.csv",
Comment: '#',
Comma: ',',
Columns: []string{"sepal_length", "sepal_width", "petal_length", "petal_width", "species"},
}.Open()
such that the query reads:
// Define a SQL query that we will execute against the CSV file.
query := `SELECT petal_length, petal_width, species FROM csv WHERE species = "Iris-versicolor"`
What do you think @dwhitena ?
Copied from original issue: go-hep/csvutil#2
From @sbinet on August 15, 2014 9:18
blog.golang.org has a nice write-up on contexts:
especially the part about hierarchical contexts.
would this be a better fit for fwk.Ctxt
:
Copied from original issue: go-hep/fwk#11
From @sbinet on August 21, 2014 11:56
using job configurations from issue go-hep/fwk#25 , it should be possible to submit jobs to a web server with simple GET/POST queries.
Copied from original issue: go-hep/fwk#26
From @ebusato on October 12, 2016 15:10
Hi,
I'd like to retrieve the position on the x axis of the y maximum of the histogram within a certain range provided by the user. Right now you have the Max() method that returns the y maximum. What do you think would be the best option to implement what I need :
?
Let me know and I'll send a PR.
Emmanuel
Copied from original issue: go-hep/hbook#8
From @sbinet on August 13, 2014 6:45
fwk
should support the ability to abort the processing of an event when it reaches a given timeout
Copied from original issue: go-hep/fwk#5
From @sbinet on July 7, 2016 15:41
we may need to support (a subset of?) SQL queries.
Copied from original issue: go-hep/csvutil#1
@aphecetche wants them
From @sbinet on December 7, 2016 16:9
Copied from original issue: go-hep/hbook#11
From @sbinet on December 15, 2016 9:50
Copied from original issue: go-hep/pawgo#4
From @sbinet on September 16, 2014 13:12
rio
filesCopied from original issue: go-hep/rio#2
From @sbinet on December 7, 2016 16:33
Copied from original issue: go-hep/hbook#16
From @sbinet on September 16, 2014 13:12
rio
filesCopied from original issue: go-hep/rio#2
From @sbinet on December 15, 2016 9:47
Copied from original issue: go-hep/fwk#43
From @sbinet on August 19, 2014 7:2
investigate whether usage of reflect
on a Task.Process
code would be possible to automagically extract the data flow.
Copied from original issue: go-hep/fwk#15
From @sbinet on August 18, 2014 14:48
do we need a virtual (r/w) filesystem ?
Copied from original issue: go-hep/fwk#13
this would allow to use rootio.Tree
from hbook/ntup.Ntuple
(and use hbook/ntup.Ntuple.Scan()
and friends)
From @sbinet on September 13, 2014 8:49
make sure consistency is upheld in face of double-put:
store := ctx.Store()
data := 42
// store data
err := store.Put("my-data", &data)
// oops, another time, under the same key
err = store.Put("my-data", &data)
otherwise, this could wreak havoc in the sync'ing of achan
-s.
Copied from original issue: go-hep/fwk#39
From @sbinet on December 15, 2016 9:49
Copied from original issue: go-hep/pawgo#3
From @sbinet on August 19, 2014 13:34
Copied from original issue: go-hep/fwk#19
From @sbinet on December 15, 2016 9:48
Copied from original issue: go-hep/fwk#44
From @sbinet on September 16, 2014 9:44
Requirements:
http
?http://git.kernel.org/cgit/linux/kernel/git/mason/btrfs-progs.git/tree/
https://btrfs.wiki.kernel.org/index.php/Btrfs_design
http://www.cs.ucdavis.edu/~pandey/Teaching/ECS150/Lects/07fs.pdf
https://ext4.wiki.kernel.org/index.php/Main_Page
http://foss.in/2006/cfp/slides/ext4-foss.pdf
https://ols.fedoraproject.org/OLS/Reprints-2008/kumar-reprint.pdf
https://github.com/torvalds/linux/tree/master/fs/ext4
http://atrey.karlin.mff.cuni.cz/~jack/papers/lk2009-ext4-btrfs.pdf
http://maczfs.googlecode.com/files/ZFSOnDiskFormat.pdf
https://blogs.oracle.com/bonwick/en_US/entry/zfs_block_allocation
https://blogs.oracle.com/bonwick/entry/space_maps
https://github.com/zfsonlinux/zfs/
http://oss.sgi.com/projects/xfs/papers/xfs_filesystem_structure.pdf
http://www.hdfgroup.org/HDF5/doc/H5.format.html
https://github.com/root-mirror/root/tree/master/io/doc/TFile
http://root.cern.ch/root/htmldoc/TFile.html
ftp://root.cern.ch/root/doc/11InputOutput.pdf
https://github.com/a4/a4
http://arxiv.org/abs/1208.1600
http://www.blosc.org/
http://www.blosc.org/blosc-in-depth.html
http://en.m.wikipedia.org/wiki/Linear_Tape_File_System
http://tukaani.org/xz/xz-file-format.txt
https://indico.in2p3.fr/event/10875/session/7/material/0/3.pdf
http://www.ietf.org/id/draft-alakuijala-brotli-07.txt
https://github.com/google/brotli/
Copied from original issue: go-hep/rio#1
right now, root-srv
will fail to plot foo.root/dir1/dir2/h1
.
From @sbinet on December 15, 2016 9:49
Copied from original issue: go-hep/pawgo#2
it should be possible to combine multiple rootio.Tree
s (with the same layout) from multiple ROOT files into one rootio.Tree
.
right, now, to get at a rootio.Tree
, one does:
func main() {
f, err := rootio.Open("testdata/chain.1.root")
if err != nil {
log.Fatal(err)
}
defer f.Close()
obj, err := f.Get("tree")
if err != nil {
log.Fatal(err)
}
tree := obj.(rootio.Tree)
fmt.Printf("entries= %v\n", tree.Entries())
}
the rootio.Tree
interface is:
// Tree is a collection of branches of data.
type Tree interface {
Named
Entries() int64
TotBytes() int64
ZipBytes() int64
Branch(name string) Branch
Branches() []Branch
Leaves() []Leaf
getFile() *File
loadEntry(i int64) error
}
it should be possible to have a type that implements the rootio.Tree
interface but logically concatenates multiple rootio.Tree
s together.
Something like the io.MultiReader
function: https://godoc.org/io#MultiReader
Initially, one could just require the user to pass already created/retrieved rootio.Tree
to a, e.g., Chain
function:
// Chain returns a new Tree that is the logical concatenation of all the input Trees.
func Chain(trees ...Tree) Tree { ... }
one should think about resource ownership:
chain
type with the Scanner
s ?Chain
function that takes a tree name and a list of file names ? (so resource ownership is clearer). it's less composable...ChainScanner
that opens and closes files as needed when each Tree
has been consumed ?From @sbinet on August 22, 2014 16:19
gaudi
had issues with initialization of components (especially cycles in services/tools: ToolA -> ToolB -> ToolC -> ... -> ToolA
)
the lack of resource dependencies declaration (component A
needs component B
) prevented to efficiently discover such issues.
it also prevented to initialize components in parallel.
investigate whether fwk
couldn't do better here.
Copied from original issue: go-hep/fwk#28
From @sbinet on August 19, 2014 6:58
boltdb
has a nice interface for transactions (updates
and read-only
):
https://github.com/boltdb/bolt
https://github.com/boltdb/bolt#transactions
fwk
Copied from original issue: go-hep/fwk#14
From @sbinet on September 11, 2014 13:27
concrete components may have properties.
should we provide a way to expose/enforce/attach documentation for them ?
godoc is useless for them.
Copied from original issue: go-hep/fwk#37
to reduce the amount of dependencies, consider using "gonum/graph/formats/dot"
instead of "awalterschulze/gographviz"
.
Scanner does not properly seal user from implementation details of Tree.
if the underlying file pointer is not at the correct position (ie pos=0), Scanner will fail.
see: c58ed3e.
From @sbinet on October 20, 2015 11:48
to provide an NlnN
combinatorics (instead of O(N^3)
) fastjet
should implement Delaunay triangulations and voronoi tesselation (as is done in the C++
version of fastjet
via the CGAL
library)
https://en.wikipedia.org/wiki/Delaunay_triangulation
https://en.wikipedia.org/wiki/Voronoi_diagram
https://github.com/sbinet-staging/poly2tri.golang/blob/master/testbed/main.go
Copied from original issue: go-hep/fastjet#1
From @sbinet on December 15, 2016 9:47
Copied from original issue: go-hep/fwk#42
From @sbinet on September 12, 2014 18:27
detect and prevent components from applying r/w updates of items in the data-store
Copied from original issue: go-hep/fwk#38
From @sbinet on August 13, 2014 6:46
fwk
should (perhaps) support task-level timeout (and then abort event) when the task takes too much time to process.
Copied from original issue: go-hep/fwk#6
From @sbinet on December 9, 2016 18:5
Copied from original issue: go-hep/hbook#24
From @sbinet on December 7, 2016 16:33
Copied from original issue: go-hep/hbook#15
From @sbinet on September 10, 2014 15:32
it should be possible to dump a fwk.App configuration into:
a human-readable format
a machine-readable format (serialization/gob/json/...)
for later re-use, comparison or regression-tests purposes.
it should also work for user-defined types. (currently, the JSON
based format only works for builtins)
Copied from original issue: go-hep/fwk#34
From @sbinet on December 15, 2016 9:49
Copied from original issue: go-hep/pawgo#1
right now, when a user uploads a file, root-srv
gobbles it up all in memory and keeps it there (until the user session expires).
From @sbinet on August 14, 2014 19:17
investigate feasibility of a multi-process, multi-node fwk.Task
process.
rpc
Copied from original issue: go-hep/fwk#8
From @sbinet on December 12, 2016 16:23
add a helper package to read/write "archives" of YODA files, so one can read/write a complete collection of hbook
values (H1D
, H2D
, Scatter2D
, ...) from/to a single YODA file.
Copied from original issue: go-hep/hbook#26
From @sbinet on December 7, 2016 16:35
Copied from original issue: go-hep/hbook#17
From @sbinet on August 20, 2014 14:55
people might cringe a bit at configuring a fwk
job in go
.
even if the github.com/go-hep/fwk/job.Job
interface can be considered quite nice.
also, providing a "Turing-complete" language as a configuration language introduces problems of its own (one has to debug the program which creates the program one wants to run.)
c.f. all the issues with e.g. the Athena/Gaudi
python
joboptions.
Using a declarative language like hcl might fit the bill.
Copied from original issue: go-hep/fwk#24
From @sbinet on December 7, 2016 16:9
Copied from original issue: go-hep/hbook#10
From @sbinet on December 7, 2016 16:11
add support for reading/writing from/to AIDA files
Copied from original issue: go-hep/hbook#12
From @sbinet on December 9, 2016 18:5
Copied from original issue: go-hep/hbook#25
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.