gholt / ring

Consistent Hashing Partition Ring for Go (golang)

Ring

Development Repository

Experimental: No stable version of this package yet exists; it is still in early development.

Package ring provides a way to distribute replicas of partitioned items to nodes.

An example would be a distributed storage system, storing duplicate copies of each file on different drives, servers, or even data centers based on the assignments given by the Ring.

If you're not entirely sure what consistent hashing is, reading Basic Hash Ring might help.

API Documentation
Basic Hash Ring
Partition Ring vs. Hash Ring

Other interesting ideas in this space:
Jump consistent hashing - dgryski implementation also dgryski shared key-value store
Multi-probe consistent hashing - dgryski implementation
GreenCHT replication scheme

This is the latest development area for the package.
Eventually a stable version of the package will be established but, for now, all things about this package are subject to change.

Copyright See AUTHORS. All rights reserved.
Use of this source code is governed by a BSD-style
license that can be found in the LICENSE file.

ring's Issues

We miss you!

calling Stop() on tcp_msg_ring.go never returns

2015/09/02 16:50:11 triggering stop
DebugStore 2015/09/02 16:50:58 compaction candidates submitted: 0
DebugStore 2015/09/02 16:50:58 compaction pass took 111.746µs
DebugStore 2015/09/02 16:51:01 tombstone discard pass took 9.190017ms
DebugStore 2015/09/02 16:51:06 out pull replication pass took 6.867206ms
DebugStore 2015/09/02 16:51:08 out push replication pass took 3.779081ms
DebugStore 2015/09/02 16:52:00 out push replication pass took 19.817µs
DebugStore 2015/09/02 16:52:00 compaction candidates submitted: 0
DebugStore 2015/09/02 16:52:00 compaction pass took 100.709µs
DebugStore 2015/09/02 16:52:06 out pull replication pass took 6.441041ms
DebugStore 2015/09/02 16:52:09 tombstone discard pass took 22.669µs
DebugStore 2015/09/02 16:52:59 out pull replication pass took 5.307018ms
DebugStore 2015/09/02 16:52:59 out push replication pass took 16.951µs
DebugStore 2015/09/02 16:53:01 compaction candidates submitted: 0
DebugStore 2015/09/02 16:53:01 compaction pass took 167.305µs
DebugStore 2015/09/02 16:53:18 tombstone discard pass took 23.044µs
DebugStore 2015/09/02 16:53:58 out pull replication pass took 6.096998ms
DebugStore 2015/09/02 16:53:59 out push replication pass took 17.566µs

Bring some of the previous test code back

project status?

Does this project maintained and why after so much time it in not production ready?

Also i want to use it for my toy storage system:
Volume - disk attached to vm
Volume represented as fixed sized blocks (4M) ech block have own id.
Physical server have 3 disks , each block have replication count 2. So it placed on two of three disks on current server. Also i have not one server but may (3 for example) Also i want to have each block not only current physical server, but also on nearest. Does it possible to do that using replica count and tiering?
Also if i want to use utp for communication how hard to do that ?

tcp msg ring panic

...attempting to close nonexistent net.Conn obj.

2015/08/28 05:12:11 msgToNode error: write tcp 10.X.X.X:8001: connection reset by peer
2015/08/28 05:12:11 conn struct: &ring.ringConn{state:2, addr:"10.X.X.X:8001", conn:(*net.TCPConn)(0xc465116380), reader:(*ring.timeoutReader)(0xc448bc2fe0), writerLock:sync.Mutex{state:1, sema:0x0},
 writer:(*ring.timeoutWriter)(0xc448bc30a0)}
015/08/28 05:12:11 handleForever error: read tcp 10.X.X.X:8001: use of closed network connection
2015/08/28 05:12:11 conn struct: &ring.ringConn{state:1, addr:"10.X.X.X:8001", conn:net.Conn(nil), reader:(*ring.timeoutReader)(nil), writerLock:sync.Mutex{state:0, sema:0x0}, writer:(*ring.timeoutWr
iter)(nil)}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x20 pc=0x51bd43]

goroutine 1485 [running]:
github.com/gholt/ring.(*TCPMsgRing).disconnection(0xc20807a180, 0xc208079830, 0x10)
        /root/go/src/github.com/gholt/ring/tcp_msg_ring.go:138 +0x1d3

That error's actually on line 137 at https://github.com/gholt/ring/blob/master/tcp_msg_ring.go#L137 but I added some debug info locally.

Create webhook for pandemicsyn

builder rename fails

The rename doubles up on the dir for the tmp file:

ring /tmp/ort.builder create replicas=3
ring /tmp/ort.builder add active=true capacity=1000 tier0=server1 tier1=z1 address0=127.0.0.1:8001 address1=127.0.0.2:8001 meta=onmetalv1
ID:        181f00653e5281fc
Active:    true
Capacity:  1000
Tiers:     server1
           z1
Addresses: 127.0.0.1:8001
           127.0.0.2:8001
Meta:      onmetalv1
/tmp/ort.builder.187494569
Rename /tmp/tmp/ort.builder.187494569 /tmp/ort.builder: no such file or directory.
make: *** [ring] Error 1

How swift compatible Alibaba oss？could you give me a demo？