GithubHelp home page GithubHelp logo

exp's People

Contributors

cznic avatar myitcv avatar pmezard avatar schleibinger avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

pmezard pallat

exp's Issues

Race condition

lldb commit bacccb2
ql commit 4091083ba6a77128c16327ce0b2e2cfe8599e33f

jnml@r550:~/src/github.com/cznic/ql$ time go test -bench . -race -timeout 24h -cpu 1,2,4
PASS
BenchmarkSelectMem1kBx1e2       5000        335254 ns/op    298281.12 MB/s
BenchmarkSelectMem1kBx1e2-2     5000        320026 ns/op    312474.57 MB/s
BenchmarkSelectMem1kBx1e2-4     5000        331976 ns/op    301225.89 MB/s
...
BenchmarkCrossJoinFile1e2NoX1e1-4        500       5450806 ns/op    1467.67 MB/s
BenchmarkCrossJoinFile1e2X1e1       1000       1895028 ns/op    4221.57 MB/s
BenchmarkCrossJoinFile1e2X1e1-2 ==================
WARNING: DATA RACE
Read by goroutine 23:
  github.com/cznic/exp/lldb.(*RollbackFiler).ReadAt()
      /home/jnml/src/github.com/cznic/exp/lldb/xact.go:521 +0x54
  github.com/cznic/exp/lldb.(*ACIDFiler0).ReadAt()
      /home/jnml/src/github.com/cznic/exp/lldb/2pc.go:1 +0x8b
  github.com/cznic/exp/lldb.(*Allocator).read()
      /home/jnml/src/github.com/cznic/exp/lldb/falloc.go:951 +0x8c
  github.com/cznic/exp/lldb.(*Allocator).Get()
      /home/jnml/src/github.com/cznic/exp/lldb/falloc.go:683 +0x656
  github.com/cznic/exp/lldb.btree.seek()
      /home/jnml/src/github.com/cznic/exp/lldb/btree.go:2199 +0x147
  github.com/cznic/exp/lldb.(*BTree).seek()
      /home/jnml/src/github.com/cznic/exp/lldb/btree.go:335 +0x2a9
  github.com/cznic/exp/lldb.(*BTree).Seek()
      /home/jnml/src/github.com/cznic/exp/lldb/btree.go:315 +0x92
  github.com/cznic/ql.(*fileIndex).Seek()
      /home/jnml/src/github.com/cznic/ql/file.go:1200 +0x1ab
  github.com/cznic/ql.(*whereRset).tryBinOp()
      /home/jnml/src/github.com/cznic/ql/ql.go:516 +0x6cb
  github.com/cznic/ql.(*whereRset).tryUseIndex()
      /home/jnml/src/github.com/cznic/ql/ql.go:689 +0x11b8
  github.com/cznic/ql.(*whereRset).do()
      /home/jnml/src/github.com/cznic/ql/ql.go:722 +0x138
  github.com/cznic/ql.(*selectRset).do()
      /home/jnml/src/github.com/cznic/ql/ql.go:898 +0x6fb
  github.com/cznic/ql.(*selectStmt).do()
      /home/jnml/src/github.com/cznic/ql/stmt.go:484 +0x78
  github.com/cznic/ql.func·025()
      /home/jnml/src/github.com/cznic/ql/ql.go:1224 +0x40d
  github.com/cznic/ql.(*crossJoinRset).do()
      /home/jnml/src/github.com/cznic/ql/ql.go:1226 +0x80f
  github.com/cznic/ql.(*selectRset).do()
      /home/jnml/src/github.com/cznic/ql/ql.go:864 +0x2ad
  github.com/cznic/ql.(*DB).do()
      /home/jnml/src/github.com/cznic/ql/ql.go:1785 +0x225
  github.com/cznic/ql.recordset.Do()
      /home/jnml/src/github.com/cznic/ql/ql.go:99 +0x9f
  github.com/cznic/ql.(*recordset).Do()
      /home/jnml/src/github.com/cznic/ql/blob.go:1 +0xdc
  github.com/cznic/ql.benchmarkCrossJoin()
      /home/jnml/src/github.com/cznic/ql/all_test.go:1844 +0xf82
  github.com/cznic/ql.benchmarkCrossJoinFile()
      /home/jnml/src/github.com/cznic/ql/all_test.go:1893 +0x5ac
  github.com/cznic/ql.BenchmarkCrossJoinFile1e2X1e1()
      /home/jnml/src/github.com/cznic/ql/all_test.go:1975 +0x4b
  testing.(*B).runN()
      /home/jnml/go/src/pkg/testing/benchmark.go:119 +0xc0
  testing.(*B).launch()
      /home/jnml/go/src/pkg/testing/benchmark.go:207 +0x1ba

Previous write by goroutine 20:
  github.com/cznic/exp/lldb.(*RollbackFiler).EndUpdate()
      /home/jnml/src/github.com/cznic/exp/lldb/xact.go:472 +0x1dc
  github.com/cznic/exp/lldb.(*ACIDFiler0).EndUpdate()
      /home/jnml/src/github.com/cznic/exp/lldb/2pc.go:1 +0x5b
  github.com/cznic/ql.(*file).Commit()
      /home/jnml/src/github.com/cznic/ql/file.go:717 +0xa3
  github.com/cznic/ql.(*DB).timeout()
      /home/jnml/src/github.com/cznic/ql/ql.go:1714 +0x104
  github.com/cznic/ql.*DB.(github.com/cznic/ql.timeout)·fm()
      /home/jnml/src/github.com/cznic/ql/ql.go:1671 +0x33
  runtime.gosched0()
      ?:0 +0xaf
  os.(*File).WriteAt()
      /home/jnml/go/src/pkg/os/file.go:160 +0xec
  github.com/cznic/exp/lldb.(*OSFiler).WriteAt()
      /home/jnml/src/github.com/cznic/exp/lldb/osfiler.go:129 +0x1b4
  github.com/cznic/exp/lldb.(*InnerFiler).WriteAt()
      /home/jnml/src/github.com/cznic/exp/lldb/filer.go:191 +0x1f7
  github.com/cznic/exp/lldb.func·002()
      /home/jnml/src/github.com/cznic/exp/lldb/2pc.go:172 +0x7b5
  github.com/cznic/exp/lldb.(*RollbackFiler).EndUpdate()
      /home/jnml/src/github.com/cznic/exp/lldb/xact.go:491 +0x371
  github.com/cznic/exp/lldb.(*ACIDFiler0).EndUpdate()
      /home/jnml/src/github.com/cznic/exp/lldb/2pc.go:1 +0x5b
  github.com/cznic/ql.(*file).Commit()
      /home/jnml/src/github.com/cznic/ql/file.go:717 +0xa3
  github.com/cznic/ql.(*DB).timeout()
      /home/jnml/src/github.com/cznic/ql/ql.go:1714 +0x104
  github.com/cznic/ql.*DB.(github.com/cznic/ql.timeout)·fm()
      /home/jnml/src/github.com/cznic/ql/ql.go:1671 +0x33

Goroutine 23 (running) created at:
  testing.(*B).run()
      /home/jnml/go/src/pkg/testing/benchmark.go:170 +0x3c
  testing.RunBenchmarks()
      /home/jnml/go/src/pkg/testing/benchmark.go:303 +0x695
  testing.Main()
      /home/jnml/go/src/pkg/testing/testing.go:411 +0x1e4
  testing.runExample()
      /home/jnml/go/src/pkg/testing/example.go:98 +0x4e5
  testing.RunExamples()
      /home/jnml/go/src/pkg/testing/example.go:36 +0x2f7
  testing.Main()
      /home/jnml/go/src/pkg/testing/testing.go:404 +0xe4
  main.main()
      github.com/cznic/ql/_test/_testmain.go:329 +0xdc

Goroutine 20 (running) created at:
  time.goFunc()
      /home/jnml/go/src/pkg/time/sleep.go:123 +0x55
==================
--- FAIL: BenchmarkCrossJoinFile1e2X1e1-2
    all_test.go:1845: Error, off: 0xd0, EOF
BenchmarkCrossJoinFile1e2X1e1-4     1000       1850207 ns/op    4323.84 MB/s
BenchmarkCrossJoinFile1e3NoX1e2        5     331323330 ns/op    2444.74 MB/s
BenchmarkCrossJoinFile1e3NoX1e2-2          5     334071044 ns/op    2424.63 MB/s
BenchmarkCrossJoinFile1e3NoX1e2-4          5     334680907 ns/op    2420.22 MB/s
BenchmarkCrossJoinFile1e3X1e2         50      61281066 ns/op    13217.79 MB/s
BenchmarkCrossJoinFile1e3X1e2-2       50      60735363 ns/op    13336.55 MB/s
BenchmarkCrossJoinFile1e3X1e2-4       50      60517227 ns/op    13384.62 MB/s
BenchmarkCrossJoinFile1e4NoX1e3        1    49052124848 ns/op   2179.56 MB/s
BenchmarkCrossJoinFile1e4NoX1e3-2          1    49296038181 ns/op   2168.77 MB/s
BenchmarkCrossJoinFile1e4NoX1e3-4          1    49065576124 ns/op   2178.96 MB/s
BenchmarkCrossJoinFile1e4X1e3          1    5437764549 ns/op    19661.02 MB/s
BenchmarkCrossJoinFile1e4X1e3-2        1    5268351488 ns/op    20293.25 MB/s
BenchmarkCrossJoinFile1e4X1e3-4        1    5131618378 ns/op    20833.97 MB/s
Found 1 data race(s)
exit status 66
FAIL    github.com/cznic/ql 7531.251s
$

Silly typo causes few tests to silently panic(nil) and pass with release 1.2.1 

go version devel +eeb3547ccd1b Tue Feb 04 11:53:13 2014 -0800 linux/amd64
exp commit: 6017180e3b896f7456d3777822883869296fdf3d

--- FAIL: TestFilerNesting (0.00 seconds)
        filer_test.go:100: /tmp/lldb-test-file579927411
        filer_test.go:107: /tmp/lldb-test-file032489014
        filer_test.go:119: /tmp/lldb-test-file604776477
        filer_test.go:128: /tmp/lldb-test-file554511576
        filer_test.go:100: /tmp/lldb-test-osfile541635159
        filer_test.go:107: /tmp/lldb-test-osfile310435274
        filer_test.go:119: /tmp/lldb-test-osfile147011233
        filer_test.go:128: /tmp/lldb-test-osfile419791244
        filer_test.go:100: 0xc20ec66880.memfiler
        filer_test.go:107: 0xc20ec66900.memfiler
        filer_test.go:119: 0xc20ec669c0.memfiler
        filer_test.go:128: 0xc20ec66a80.memfiler
panic: nil [recovered]
        panic: test executed panic(nil) or runtime.Goexit

goroutine 46 [running]:
runtime.panic(0x5eb7e0, 0xc20ece2460)
        /home/mpl/gotip/src/pkg/runtime/panic.c:250 +0xb4
testing.func·005()
        /home/mpl/gotip/src/pkg/testing/testing.go:392 +0x150
runtime.panic(0x0, 0x0)
        /home/mpl/gotip/src/pkg/runtime/panic.c:232 +0x110
github.com/cznic/exp/lldb.func·006(0xc20ec66a80, 0x0)
        /home/mpl/gocode/src/github.com/cznic/exp/lldb/filer_test.go:83 +0x28f
github.com/cznic/exp/lldb.testFilerNesting(0xc20ed58240, 0x685440)
        /home/mpl/gocode/src/github.com/cznic/exp/lldb/filer_test.go:99 +0x31
github.com/cznic/exp/lldb.TestFilerNesting(0xc20ed58240)
        /home/mpl/gocode/src/github.com/cznic/exp/lldb/filer_test.go:94 +0x85
testing.tRunner(0xc20ed58240, 0xad99b0)
        /home/mpl/gotip/src/pkg/testing/testing.go:398 +0x8b
created by testing.RunTests
        /home/mpl/gotip/src/pkg/testing/testing.go:479 +0x978

goroutine 16 [chan receive]:
testing.RunTests(0x6857c8, 0xad9740, 0x2e, 0x2e, 0x1)
        /home/mpl/gotip/src/pkg/testing/testing.go:480 +0x9a8
testing.Main(0x6857c8, 0xad9740, 0x2e, 0x2e, 0xada6e0, ...)
        /home/mpl/gotip/src/pkg/testing/testing.go:411 +0x8c
main.main()
        /tmp/go-build115088548/github.com/cznic/exp/lldb/_test/_testmain.go:291
+0x9c

goroutine 17 [syscall, 1 minutes]:
runtime.goexit()
        /home/mpl/gotip/src/pkg/runtime/proc.c:1438

goroutine 21 [finalizer wait, 1 minutes]:
runtime.park(0x418900, 0xafd878, 0xad9fa1)
        /home/mpl/gotip/src/pkg/runtime/proc.c:1364 +0x66
runtime.parkunlock(0xafd878, 0xad9fa1)
        /home/mpl/gotip/src/pkg/runtime/proc.c:1380 +0x3b
runfinq()
        /home/mpl/gotip/src/pkg/runtime/mgc0.c:2335 +0x7a
runtime.goexit()
        /home/mpl/gotip/src/pkg/runtime/proc.c:1438
exit status 2
FAIL    github.com/cznic/exp/lldb       102.104s

Patch ready, now going to test with tip.

A question about Dump

@cznic I use a array to creat a sub-array, and calle it's Dump function. It dumps all informations from root array.

lldb.remove panics

Hi, I'm testing camlistore, and after some camput-ting, it panics:

2013/09/20 15:58:48 http: panic serving 127.0.0.1:35186: runtime error: invalid memory address or nil pointer dereference
goroutine 1718 [running]:
net/http.func·007()
/usr/lib/go/src/pkg/net/http/server.go:1022 +0xac
camlistore.org/pkg/blobserver/diskpacked.func·001()
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/pkg/blobserver/diskpacked/diskpacked.go:339 +0xd7
camlistore.org/third_party/github.com/cznic/exp/lldb.(_lst).remove()
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/falloc.go:1861 +0xa
camlistore.org/third_party/github.com/cznic/exp/lldb.(_lst).removeBack(0xc200163ab8, 0x1c)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/falloc.go:1876 +0x2e
camlistore.org/third_party/github.com/cznic/exp/lldb.(_Allocator).cadd(0xc2001639a0, 0xc2002c3e7b, 0x22, 0x22, 0x53, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/falloc.go:380 +0x10e
camlistore.org/third_party/github.com/cznic/exp/lldb.(_Allocator).Alloc(0xc2001639a0, 0xc2002c3e7b, 0x22, 0x22, 0x53, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/falloc.go:420 +0x1f6
camlistore.org/third_party/github.com/cznic/exp/lldb.btreeDataPage.setContent(0xc200bf90cd, 0x2f8, 0x20151, 0xc200282bc0, 0xc2001639a0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:1272 +0x50a
camlistore.org/third_party/github.com/cznic/exp/lldb.btreeDataPage.setKey(0xc200bf9000, 0x3c5, 0x2021e, 0xc200282bc0, 0xc2001639a0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:1308 +0xa9
camlistore.org/third_party/github.com/cznic/exp/lldb.btreeDataPage.insertItem(0xc200bf9000, 0x3c5, 0x2021e, 0xc200282bc0, 0xc2001639a0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:1350 +0x1aa
camlistore.org/third_party/github.com/cznic/exp/lldb.btree.put2(0x1, 0xc200f04000, 0x1010f, 0x2021e, 0xc200282bc0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:1888 +0xecf
camlistore.org/third_party/github.com/cznic/exp/lldb.btree.put(0x1, 0xc200f04000, 0x1010f, 0x2021e, 0xc200282bc0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:1784 +0x122
camlistore.org/third_party/github.com/cznic/exp/lldb.(_BTree).Set(0xc2001c4000, 0xc2002c3e70, 0x2d, 0x2d, 0xc200776140, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/exp/lldb/btree.go:429 +0x1ca
camlistore.org/third_party/github.com/cznic/kv.(_DB).Set(0xc200176a50, 0xc2002c3e70, 0x2d, 0x2d, 0xc200776140, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/third_party/github.com/cznic/kv/kv.go:756 +0xcb
camlistore.org/pkg/index/kvfile.(_kvis).Set(0xc20011f790, 0xc2002c3e40, 0x2d, 0xc200776120, 0x11, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/pkg/index/kvfile/kvfile.go:84 +0xc3
camlistore.org/pkg/blobserver/diskpacked.(_storage).finishAppend(0xc200181510, 0xc200231940, 0xc2007763c0, 0x12523, 0x12523, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/pkg/blobserver/diskpacked/diskpacked.go:342 +0x28e
camlistore.org/pkg/blobserver/diskpacked.(_storage).append(0xc2001566c0, 0xc200231940, 0xc2007763c0, 0x12523, 0xc20056b5a0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/pkg/blobserver/diskpacked/diskpacked.go:385 +0x5d2
camlistore.org/pkg/blobserver/diskpacked.(_storage).ReceiveBlob(0xc2001566c0, 0xc200231940, 0xc2007763c0, 0xc20056bbd0, 0xc2003c57e0, ...)
/home/tgulacsi/src/camlistore.org/tmp/build-gopath/src/camlistore.org/pkg/blobserver/diskpacked/diskpacked.go:293 +0x1a0

It seems that l.back is nil. Any ideas?

configure compressor in lldb

As i see lldb uses zappy for compression, does it possible to get compressor via interface to get ability to use different compressor for data (i want to try lz4)...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.