goinstant / delayd Goto Github PK

View Code? Open in Web Editor NEW

13.0 13.0 4.0 1.57 MB

Replicated setTimeout()

License: BSD 3-Clause "New" or "Revised" License

Go 100.00%

delayd's People

Contributors

Stargazers

Watchers

Forkers

alisheikh hfeeki nabeken johnhenry

delayd's Issues

delayd -v

Can we get an option to display the current version? Thanks.

Wrap timer in a mutex

Or otherwise prevent race conditions.

At the moment, the timer can be changed from two places:

After the emission of events
When a new event is added

There's a race condition there, since they run on different goroutines. Eliminate this race condition.

Implement log compaction

Required to prevent unbounded log growing

Use build tags to exclude integration tests

It should be possible to just run the unit tests, for faster feedback while developing. Use a build tag to mark the integration tests, and add a Makefile that will run them.

The unit tests should probably require not having that tag set, so they don't run as well, but that might be overkill.

next time race condition

see https://travis-ci.org/goinstant/delayd/builds/27535069

Schema versioning/upgrades for persistent storage

The meta db stores a schema field, containing the schema version for persistent storage.
When we lock down the format then increase the value, we'll need to:

read the value from a db on open.
if it is less than the current version, perform an upgrade on the data.

Listen to adds and removes from storage to reset the timer

The existing model updates the timer based on timer triggers, and add commands. When the system is running, this is fine.

On a server restart, we become the raft leader before reloading all of the commands, meaning there is a window when we are the leader, but storage is not populated, so we won't notify. The server will have to wait for the next command to appear from amqp before resetting the timer.

Instead, trigger timer resets based on a notification from storage.

==================
WARNING: DATA RACE
Read by goroutine 19:
  main.(*Storage).Add()
      /repos/delayd/storage.go:275 +0xc54
  main.(*FSM).Apply()
      /repos/delayd/raft.go:68 +0xa24
  github.com/hashicorp/raft.(*Raft).runFSM()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/hashicorp/raft/raft.go:483 +0xe73
  github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runFSM)·fm()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/hashicorp/raft/raft.go:223 +0x33
  github.com/hashicorp/raft.func·007()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/hashicorp/raft/state.go:152 +0x7b

Previous write by goroutine 16:
  main.(*Storage).startTimerLoop()
      /repos/delayd/storage.go:168 +0x8f0

Goroutine 19 (running) created at:
  github.com/hashicorp/raft.(*raftState).goFunc()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/hashicorp/raft/state.go:153 +0xf1
  github.com/hashicorp/raft.NewRaft()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/hashicorp/raft/raft.go:223 +0xdf9
  main.NewRaft()
      /repos/delayd/raft.go:135 +0xd7b
  main.(*Server).Run()
      /repos/delayd/server.go:43 +0x88c
  main.execute()
      /repos/delayd/main.go:58 +0x50c
  github.com/codegangsta/cli.Command.Run()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/codegangsta/cli/command.go:97 +0xef3
  github.com/codegangsta/cli.(*App).Run()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/codegangsta/cli/app.go:125 +0xc10
  main.main()
      /repos/delayd/main.go:89 +0x3fc


Goroutine 16 (running) created at:
  main.(*Storage).initTimer()
      /repos/delayd/storage.go:114 +0xf6
  main.NewStorage()
      /repos/delayd/storage.go:49 +0xca
  main.(*Server).Run()
      /repos/delayd/server.go:38 +0x6e7
  main.execute()
      /repos/delayd/main.go:58 +0x50c
  github.com/codegangsta/cli.Command.Run()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/codegangsta/cli/command.go:97 +0xef3
  github.com/codegangsta/cli.(*App).Run()
      /home/vagrant/.gvm/pkgsets/go1.2.2/global/src/github.com/codegangsta/cli/app.go:125 +0xc10
  main.main()
      /repos/delayd/main.go:89 +0x3fc
==================

SIGSEGV: segmentation violation
PC=0x40daf8
signal arrived during cgo execution
2014/06/10 12:44:47 Lost raft leadership
runtime.cgocall(0x4035c0, 0x2b9e2c6629b8)
    /home/travis/.gvm/gos/go1.2/src/pkg/runtime/cgocall.c:149 +0x11b fp=0x2b9e2c6629a0
github.com/armon/gomdb._Cfunc_mdb_txn_commit(0x2b9e30008dc0, 0x0)
    github.com/armon/gomdb/_obj/_cgo_defun.c:347 +0x31 fp=0x2b9e2c6629b8
github.com/armon/gomdb.(*Txn).Commit(0xc210000de8, 0x2, 0xc210000e80)
    /home/travis/gopath/src/github.com/armon/gomdb/txn.go:66 +0x2a fp=0x2b9e2c6629e8
github.com/goinstant/delayd.(*Storage).Remove(0xc210071330, 0xc2100c51a2, 0x10, 0x10, 0x9, ...)
    /home/travis/gopath/src/github.com/goinstant/delayd/storage.go:362 +0x302 fp=0x2b9e2c662b00
github.com/goinstant/delayd.(*FSM).Apply(0xc21009cdd8, 0xc2100552e8, 0xe87b00, 0x1)
    /home/travis/gopath/src/github.com/goinstant/delayd/raft.go:70 +0x936 fp=0x2b9e2c662d80
github.com/hashicorp/raft.(*Raft).runFSM(0xc2100b41c0)
    /home/travis/gopath/src/github.com/hashicorp/raft/raft.go:500 +0x9f1 fp=0x2b9e2c662f78
github.com/hashicorp/raft.*Raft.(github.com/hashicorp/raft.runFSM)·fm()
    /home/travis/gopath/src/github.com/hashicorp/raft/raft.go:226 +0x26 fp=0x2b9e2c662f88
github.com/hashicorp/raft.func·008()
    /home/travis/gopath/src/github.com/hashicorp/raft/state.go:152 +0x4d fp=0x2b9e2c662fa0
runtime.goexit()
    /home/travis/.gvm/gos/go1.2/src/pkg/runtime/proc.c:1394 fp=0x2b9e2c662fa8
created by github.com/hashicorp/raft.(*raftState).goFunc
    /home/travis/gopath/src/github.com/hashicorp/raft/state.go:153 +0x9c
goroutine 1 [chan receive]:
testing.RunTests(0x8f4da0, 0xe70780, 0x17, 0x17, 0x1)
    /home/travis/.gvm/gos/go1.2/src/pkg/testing/testing.go:472 +0x8d5
testing.Main(0x8f4da0, 0xe70780, 0x17, 0x17, 0xe85a60, ...)
    /home/travis/.gvm/gos/go1.2/src/pkg/testing/testing.go:403 +0x84
main.main()
    github.com/goinstant/delayd/_test/_testmain.go:91 +0x9c
goroutine 4 [syscall]:
os/signal.loop()
    /home/travis/.gvm/gos/go1.2/src/pkg/os/signal/signal_unix.go:21 +0x1e
created by os/signal.init·1
    /home/travis/.gvm/gos/go1.2/src/pkg/os/signal/signal_unix.go:27 +0x31
goroutine 6 [sleep]:
time.Sleep(0x4c4b40)
    /home/travis/.gvm/gos/go1.2/src/pkg/runtime/time.goc:31 +0x31
github.com/hashicorp/raft.(*shutdownFuture).Error(0xc210000fd8, 0x2b9e205a0680, 0xc210000fd8)
    /home/travis/gopath/src/github.com/hashicorp/raft/future.go:83 +0x47
github.com/goinstant/delayd.(*Raft).Close(0xc21009d320)
    /home/travis/gopath/src/github.com/goinstant/delayd/raft.go:153 +0x56
github.com/goinstant/delayd.(*Server).Stop(0xc210073a20)
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:90 +0x151
github.com/goinstant/delayd.TestInAndOut(0xc210055360)
    /home/travis/gopath/src/github.com/goinstant/delayd/main_test.go:95 +0x287
testing.tRunner(0xc210055360, 0xe70798)
    /home/travis/.gvm/gos/go1.2/src/pkg/testing/testing.go:391 +0x8b
created by testing.RunTests
    /home/travis/.gvm/gos/go1.2/src/pkg/testing/testing.go:471 +0x8b2
goroutine 7 [chan receive]:
github.com/goinstant/delayd.(*Server).Run(0xc210073a20, 0xc21006a420, 0x10, 0xc210000530, 0x6, ...)
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:58 +0x8bf
created by github.com/goinstant/delayd.TestInAndOut
    /home/travis/gopath/src/github.com/goinstant/delayd/main_test.go:86 +0x1af
goroutine 11 [syscall]:
runtime.goexit()
    /home/travis/.gvm/gos/go1.2/src/pkg/runtime/proc.c:1394
goroutine 16 [chan send]:
github.com/goinstant/delayd.func·001()
    /home/travis/gopath/src/github.com/goinstant/delayd/amqp.go:127 +0x1d5
created by github.com/goinstant/delayd.NewAmqpReceiver
    /home/travis/gopath/src/github.com/goinstant/delayd/amqp.go:130 +0x9e5
goroutine 28 [chan receive]:
github.com/hashicorp/raft.(*deferError).Error(0xc2100552d0, 0xc2100c51a0, 0x12)
    /home/travis/gopath/src/github.com/hashicorp/raft/future.go:51 +0x9f
github.com/goinstant/delayd.(*Raft).Remove(0xc21009d320, 0xc21000aa10, 0x10, 0x10, 0xdf8475800, ...)
    /home/travis/gopath/src/github.com/goinstant/delayd/raft.go:175 +0x17c
github.com/goinstant/delayd.(*Server).timerSend(0xc210073a20, 0xecb28f4bf, 0x2d3b7257, 0xe87b00, 0x0, ...)
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:154 +0x4ed
github.com/goinstant/delayd.*Server.(github.com/goinstant/delayd.timerSend)·fm(0xecb28f4bf, 0x2d3b7257, 0xe87b00, 0x0, 0x0, ...)
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:54 +0x64
github.com/goinstant/delayd.(*Timer).timerLoop(0xc210072300)
    /home/travis/gopath/src/github.com/goinstant/delayd/timer.go:87 +0xbf
created by github.com/goinstant/delayd.NewTimer
    /home/travis/gopath/src/github.com/goinstant/delayd/timer.go:40 +0x104
goroutine 29 [chan receive]:
github.com/goinstant/delayd.(*Server).observeLeaderChanges(0xc210073a20)
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:108 +0x60
created by github.com/goinstant/delayd.(*Server).Run
    /home/travis/gopath/src/github.com/goinstant/delayd/server.go:55 +0x881
rax     0x9
rbx     0x2b9e30008dc0
rcx     0x0
rdx     0x2b9e
rdi     0x0
rsi     0x0
rbp     0x0
rsp     0x2b9e2c1ffb50
r8      0x7
r9      0x1
r10     0x2b9e2c971010
r11     0x0
r12     0x2b9e30001700
r13     0x2b9e30008dc0
r14     0x2b9e30001700
r15     0x0
rip     0x40daf8
rflags  0x10246
cs      0x33
fs      0x0
gs      0x0
FAIL    github.com/goinstant/delayd 3.499s
make: *** [test] Error 1

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

Jobs

Jooble