rqlite / rqlite Goto Github PK

The lightweight, distributed relational database built on SQLite.

License: MIT License

Go 90.68% Shell 0.57% Python 8.71% Dockerfile 0.04%

go sqlite distributed-systems database relational-database raft consensus distributed-database sql

rqlite's Introduction

rqlite is a relational database which combines SQLite's simplicity with the power of a robust, fault-tolerant, distributed system. It's designed for easy deployment and lightweight operation, offering a developer-friendly and operator-centric solution for Linux, macOS, and Windows, as well as various CPU platforms.

Check out the rqlite FAQ.

Why run rqlite?

rqlite is your solution for a rock-solid, fault-tolerant, relational database with effortless installation, deployment, and operation. It's ideal as a lightweight, distributed relational data store for both developers and operators. Think Consul or etcd, but with relational modeling available.

You can use rqlite to store your important data reliably, ensuring it's always available. If you're interested in understanding how distributed systems actually work, it's a good example to study. A lot of thought has gone into its design, separating storage, consensus, and API clearly.

Key features

Easy Deployment: Up and running in seconds, with no separate SQLite installation.
Developer-Friendly: Straightforward HTTP API, CLI, and client libraries.
Fully Replicated: SQL database with full-text search and JSON support.
Large data set support: rqlite works well, even when managing multi-GB data sets.
Dynamic Clustering: Integrates with Kubernetes, Consul, etcd, and DNS for automatic clustering.
Robust Security: Extensive encryption and TLS support.
Flexible Consistency: Customize read/write performance and durability.
Scalable Reads: Read-only nodes for enhanced scalability.
Transactions: Supports a form of transactions.
Easy Backups: Hot backups, including automatic backups to AWS S3 and MinIO, as well as restore directly from SQLite.

Quick Start

Get up and running quickly with our Quick Start guide.

Pronunciation

Common pronunciations of rqlite include "R Q lite" and "ree-qwell-lite".

rqlite's People

Contributors

Stargazers

Watchers

Forkers

wheelcomplex jonsen zhuweijava vonwenm tml 2mia ctrlrsf jamesclonk godeep jmcarbo kmad oscillatingworks fayizk1 sekimura huayl dessalines insionng jeffatix jmptrader flyeven mem iexperi slowfranklin zmedico rtvt123 chancez applied-duality magictour dengweiwen fnet123 les-mockingjay skyformat99 qian19876025 m4ce martindale hcxiong vamsu linearregression cbforks zyh329 jiashilei-1985 cdfpaz glycerine abhishekshivanna rindvieh strogo mkideal jburnham bkeroackdsc gaobinwang tomzhang euforia w4ngyi jiapei100 fengweijp wangjun socialpercon bkeroack duzhanyuan modulexcite anchoo2kewl ruiaylin chenzhongtao tomas-fp m-kuhn maniacs-db crestone ubhisat sreenadhp chanwit lygstate elotta1223 zhoudaqing tych0 rudimk bobquest33 cuulee kofj mrtomyum wanlitian lamproae timerope happy-ferret mikalv patrick-park gsdu8g9 ianwoolf izagz litalidev malvania secsecsec lubaoyilang kimerran-archive ariesdevil sourceinsight leanhd emrul fowum josejamilena joseph-hurtado

rqlite's Issues

Non-standard use of double quotes around string literals in Readme

Although SQLite does support that notation, it is invalid SQL, and supported only for legacy reasons, and Dr. Richard Hipp has several time expressed regret from this decision.

That syntax further requires escaping the double-quotes in JSON, while single-quote do not need escaping, leading to more readable example.

Re-enable 'explain' URL query param

This is currently non-functional and needs to be re-enabled.

Is GET the right method for bulk queries?

This example from README.md fails to work with curl (tested with curl versions 7.45.0 and 7.47.1):

$ curl -L -G 'localhost:4001/db?pretty' -H "Content-Type: application/json" -d '[
    "SELECT * FROM foo",
    "SELECT * FROM bar"
]'
curl: (3) Illegal characters found in URL

Given that the HTTP/1.1 spec says the body of a GET request is supposed to be ignored, I'm not sure that embedding queries in the body was your intention (but I'll go along if you'd like to ignore the spec).

Looking at the Service.handleQuery method, I see that it checks for an empty q= query parameter. However, I have not yet found a curl command both curl and rqlite (current master branch) will successfully respond to, given a JSON body.

I understand that the master branch is under development, and it may be that this is intentional breakage. Anyway, I'd like some clarification on the intended request format, and I'd be happy to send a pull request once the intention is clear to me.

return 'SELECT last_insert_rowid()' result executed just before transaction commit

When inserting a row into a table with an automatically generated primary key, it can be very useful to return the newly allocated primary key value which is returned by SELECT last_insert_rowid(), so it can be inserted as a foreign key in another table. Unless the SELECT last_insert_rowid() statement is executed in the same transaction as the INSERT statement, it is needlessly complex to determine the value of the newly allocated primary key.

I have a python driver for rqlite, which I will release soon, and I would like to use this feature to implement the Cursor.lastrowid attribute.

Decoupling SQL Syntax in Post / Get APIs

Do you have any plans of making the SQL agnostic ReST APIs and do a thin API resource transformation into SQL Call within the server to decouple the clients from SQL Syntax?

For trivial data modification / retrieval it will ease the Client side query generation and give a more ReSTFul feel to the API Layer.

For post the table name can go as a rest url and data in form of json array payload there by merging the single insert & batch insert into a single rest endpoint interaction.

Also for retrieval the Column key val can be query params and pk be path params

Changing the API

This is a heads-up that I will soon change the API. It won't be a huge change, but I've realised that the current API is quite awkward, and tedious to unit test. It should be very easy to update client software.

Multiple query statements contained within a bulk update will no longer be delimited by a newline. Instead the delimiter will be a semicolon. For example, a bulk update will look like this:

curl -L -XPOST 'localhost:4001/db?pretty' -d 'INSERT INTO foo(name) VALUES("fiona"); INSERT INTO foo(name) VALUES("sinead");INSERT INTO foo(name) VALUES("laura")'

When performing a GET, the query will no longer be read from the body of the request. Instead, it will be read using the URL param q. An example curl command showing this in action is shown below.

curl -L -G localhost:4001/db --data-urlencode 'q=SELECT * from foo'

Create CLI for rqlite

A simple command-line tool should be written. This doesn't have to be written in Go, it could be written in, say, Python.

Store commit index in sqlite file

On startup rqlite unconditionally deletes any sqlite file, gets any snapshot, and applies outstanding log entries. However if, with every change to the underlying SQLite database, rqlite also stored the commit index of the Raft command in the database file (store it in a special table and do this in a transactional manner) rqlite may drastically shorten the start-up times by a) not deleting any existing database file, and b) ignoring Raft commands with an index lower than that recorded in the database. If this is a valid procedure, then start-up times could be reduced ever further.

Run rqlite through Go's race detector

Try this with rqlite:

http://blog.golang.org/race-detector

Automatically redirect XPOSTS to the leader

As I stated in issue #15 , it would be extremely helpful if I could send my XPOSTS/inserts and updates to my own node, which is always available. Since it knows the leader anyway, it should be able redirect these. So in your 3 node example, I'd run the line curl -XPOST localhost:4003/db -d 'yada yada', and it could redirect the XPOST to the current leader as shown in curl localhost:4003/raft?pretty

Right now, if I try an XPOST on my own node, I get the error:
"error":"raft.Server: Not current leader"

Allow rqlite to use in-memory only database

Started with https://github.com/otoolep/rqlite/issues/55

With better support for DSNs, it should be possible to support in-memory SQLite databases. Persistency isn't lost since the Raft log is the ultimate source of truth. This might result in much faster performance.

Create ability to backup node

The system should expose an API endpoint that streams out a consistent snapshot of the database.

Followers and leaders keep switching after using -L command

I can't completely verify this, but would love if someone would help test this out.

I'm finding that after writing to a follower using the curl -L command, it suddenly becomes the leader(as shown by the raft endpoint)

Its not causing any problems really, because the DBs appear to show the same data correctly, but I'm not sure that's correct behavior.

Edit: They both seem to be switching alternately to candidate, follower, and leader.

Peers should be an array

I'm starting to do some json templating of a network status page of rqlite, and I'm noticing something that doesn't seem right. The 'raft' endpoint that returns json has a peers element, that should be a json array, not a map like it currently is.

Right now its:

{
    "peers": {
        "2456e8c": {
            "name": "2456e8c",
            "connectionString": "http://localhost:4002",
            "isConnected" : "false" 
       ...

And it should be:

{
    "peers": [{
            "name": "2456e8c",
            "connectionString": "http://localhost:4002",
            "isConnected" : "false" 
       ...

This way I could iterate over the peers array correctly using mustache/handlebars.

error

Hi,
when i execute "go get github.com/otoolep/rqlite" i get the following error:

src/code.google.com/p/goprotobuf/proto/text.go:39:2: no Go source files in /usr/lib/go/src/pkg/encoding

Whats the problem?

Several linter errors and warnings to be fixed

gometalinter is an exceptional tool for checking errors and warnings into a source code written in Go.

After running the tool for rqlite, I found that there are many small things to be fixed:

$ gometalinter ./...
main.go:101::error: possible formatting directive in Error call
main.go:119::error: possible formatting directive in Error call
command/execute_command.go:64::error: possible formatting directive in Error call
server/server.go:214::error: arg s.connectionString for printf verb %s of wrong type: func() string
server/server.go:240::error: possible formatting directive in Error call
server/server.go:482::error: possible formatting directive in Error call
db/db.go:15::warning: unused global variable dbName
main.go:78:14:warning: error return value not checked (os.MkdirAll(logFileDir, 0744))
main.go:101:13:warning: error return value not checked (log.Error("Unable to create path: %s", err.Error()))
main.go:103:16:warning: error return value not checked (defer f.Close())
main.go:105:24:warning: error return value not checked (pprof.StartCPUProfile(f))
main.go:114:12:warning: error return value not checked (log.Error("No data path supplied -- aborting"))
main.go:119:12:warning: error return value not checked (log.Error("Unable to create path: %s", err.Error()))
main.go:124:12:warning: error return value not checked (log.Error(s.ListenAndServe(join)))
main.go:141:16:warning: error return value not checked (go client.Post("https://logs-01.loggly.com/inputs/8a0edd84-92ba-46e4-ada8-c529d0f105af/tag/reporting/",)
command/execute_command.go:64:14:warning: error return value not checked (log.Error("Failed to rollback transaction: %s", err.Error))
command/execute_command.go:71:12:warning: error return value not checked (log.Error("Failed to start transaction:", err.Error()))
command/execute_command.go:77:13:warning: error return value not checked (log.Error("Failed to execute statement within transaction", err.Error()))
command/execute_command.go:83:12:warning: error return value not checked (log.Error("Failed to commit transaction:", err.Error()))
db/db.go:30:11:warning: error return value not checked (os.Remove(dbPath))
db/db.go:39:12:warning: error return value not checked (log.Error(err))
db/db.go:56:11:warning: error return value not checked (log.Warn("Query \"%s\" may modify the database", query))
db/db.go:60:12:warning: error return value not checked (log.Error("failed to execute SQLite query", err.Error()))
db/db.go:63:18:warning: error return value not checked (defer rows.Close())
db/db.go:77:13:warning: error return value not checked (log.Error("failed to scan SQLite row", err.Error()))
server/server.go:142:21:warning: error return value not checked (m.registry.Register("join.success", m.joinSuccess))
server/server.go:143:21:warning: error return value not checked (m.registry.Register("join.fail", m.joinFail))
server/server.go:144:21:warning: error return value not checked (m.registry.Register("query.Received", m.queryReceived))
server/server.go:145:21:warning: error return value not checked (m.registry.Register("query.success", m.querySuccess))
server/server.go:146:21:warning: error return value not checked (m.registry.Register("query.fail", m.queryFail))
server/server.go:147:21:warning: error return value not checked (m.registry.Register("execute.Received", m.executeReceived))
server/server.go:148:21:warning: error return value not checked (m.registry.Register("execute.tx.received", m.executeTxReceived))
server/server.go:149:21:warning: error return value not checked (m.registry.Register("execute.success", m.executeSuccess))
server/server.go:150:21:warning: error return value not checked (m.registry.Register("execute.fail", m.executeFail))
server/server.go:151:21:warning: error return value not checked (m.registry.Register("snapshot.created", m.snapshotCreated))
server/server.go:233:12:warning: error return value not checked (log.Error("Failed to create new Raft server", err.Error()))
server/server.go:240:12:warning: error return value not checked (log.Error("Error loading snapshot: %s", err.Error()))
server/server.go:244:20:warning: error return value not checked (s.raftServer.Start())
server/server.go:252:13:warning: error return value not checked (log.Error("Cannot join with an existing log"))
server/server.go:256:13:warning: error return value not checked (log.Error("Failed to join leader", err.Error()))
server/server.go:270:13:warning: error return value not checked (log.Error("Failed to join to self", err.Error()))
server/server.go:311:28:warning: error return value not checked (json.NewEncoder(&b).Encode(command))
server/server.go:316:23:warning: error return value not checked (defer resp.Body.Close())
server/server.go:396:10:warning: error return value not checked (w.Write([]byte(b)))
server/server.go:482:12:warning: error return value not checked (log.Error("Database mutation failed: %s", err.Error()))
server/server.go:498:10:warning: error return value not checked (w.Write([]byte(b)))
server/server.go:509:13:warning: error return value not checked (log.Error("failed to get " + k + " stats"))
server/server.go:516:9:warning: error return value not checked (w.Write(ensurePrettyPrint(req, statistics)))
server/server.go:531:9:warning: error return value not checked (w.Write(ensurePrettyPrint(req, diagnostics)))
server/server.go:542:9:warning: error return value not checked (w.Write(ensurePrettyPrint(req, info)))
server/statemachine.go:33:12:warning: error return value not checked (log.Error("Failed to save state: ", err.Error()))
server/statemachine.go:45:12:warning: error return value not checked (log.Error("Failed to recover state: ", err.Error()))

Many of them are not easy to fix, as they require bigger changes in the code, but the idea here is to decrease the length of this list, and try not to increment it on new changes. We could eventually add gometalinter as part of the CI system.

Leader Election?

Is leader election implemented? I've tried killing off the leader but can't figure out if one of the other nodes has been elected.

Should I be able to find the new leader and do inserts to it? If so, would an api of /leader to get the IP/PORT of the leader be possible?

Injections and Authentication

Hi,

I'm wondering how to sanitize queries to be safe from SQL injections. Indeed since there is only the HTTP API, we can't prepare so any query might not be safe. Is there a recommended pattern to apply?

Also, since there is no users or authentication process before running queries, should a rqlite cluster be run behind a proxy, or did a miss something?

Thanks for your work!

Row columns ordered in JSON response differently from SQL statement, tricky to map to database APIs

Many database APIs return row objects which are expected to have the columns in a particular order. Given the ordering of columns in rqlite JSON responses, it is tricky to map the columns into the order implied by the SELECT statement, since it involves parsing SQL. It would greatly simplify the implementation of database APIs if rqlite would return columns in the order implied by the SELECT statement.

Support configurable consistency levels

rqlite should support the following levels of consistency for reads:

Any node can service a read as long as it believes it remains in the cluster.
Only the leader should serve reads.
The leader should check before serving any read if it is actually still the leader. It can happen, that a leader has not detected a network partition since heatbeat failures are not detected instantaneously.

These levels of consistency should chosen by a switch at the command-line.

Version and Changelog

This project lacks of versioning. It might help users and developers on tracking changes, making it easier to take decisions on possible upgrades. It will ease the release process, when it comes of native packages (i.e. apt, brew, rpm...).

This versioning should adheres to Semantic Versioning and bring a CHANGELOG file following http://keepachangelog.com/ practices.

Main question here is, which versions do we want to start with? Does 0.1.0 sound good? Do we want to include any specific work in progress to 0.1.0 before releasing?

[BUG] If you set a snapshot '-s' parameter, no one can join that master node

Hey @otoolep ,

I just spent a few minutes trying to figure out why I couldn't join a master node. It would stall on:

2015/04/12 20:27:58 [ERROR] Error loading snapshot: open data/snapshot: no such file or directory
2015/04/12 20:27:58 [INFO ] Attempting to join leader at xxx.xx.xx.xx:4002

I finally figured out that if I remove the -s snapshot parameter on the master startup script, it works fine:

This won't accept joiners:
$GOPATH/bin/rqlite -s 50 -p 4002 data

This will:
$GOPATH/bin/rqlite -p 4002 data

Error when using rqlite to work over the net

Edit: don't bother reading through this, I figured it out, scroll to the bottom comment.

I'm getting a strange error when using rqlite over the net. I'll post my commands for the leader and follower, and then the go error output.

Leader commands:

export GOPATH=$PWD
go get github.com/otoolep/rqlite
go get gopkg.in/check.v1;
$GOPATH/bin/rqlite data

Follower commands to join:

export GOPATH=$PWD
go get github.com/otoolep/rqlite
go get gopkg.in/check.v1;
$GOPATH/bin/rqlite -join xx.xx.xx.xx:4001 data

Note: I tried this twice in a row, and the second time, I didn't get an error, but there was no db.sqlite file for the follower, and the administration showed some weird things. I'll put these below the error output.

leader output:

[04/13/15 09:09:46] [INFO] Redirectoring logging to stdout
[04/13/15 09:09:46] [INFO] Raft random seed initialized
[04/13/15 09:09:46] [INFO] Raft commands registered
[04/13/15 09:09:46] [INFO] Initializing Raft Server: data
[04/13/15 09:09:46] [INFO] Loading latest snapshot, if any, from disk
[04/13/15 09:09:46] [EROR] Error loading snapshot: open data/snapshot: no such file or directory
[04/13/15 09:09:46] [INFO] Initializing new cluster
[04/13/15 09:09:46] [INFO] Initializing HTTP server
[04/13/15 09:09:46] [INFO] Listening at http://localhost:4001
panic: assertion failed: leader.elected.at.same.term.0


goroutine 13 [running]:
github.com/goraft/raft._assert(0x8de100, 0x93af70, 0x1f, 0xc208295cb0, 0x1, 0x1)
    /home/tyler/test/src/github.com/goraft/raft/util.go:60 +0xcf
github.com/goraft/raft.(*server).processAppendEntriesRequest(0xc20808cc60, 0xc2082788c0, 0xc2082788c0, 0xc2082788c0)
    /home/tyler/test/src/github.com/goraft/raft/server.go:948 +0x413
github.com/goraft/raft.(*server).leaderLoop(0xc20808cc60)
    /home/tyler/test/src/github.com/goraft/raft/server.go:849 +0x761
github.com/goraft/raft.(*server).loop(0xc20808cc60)
    /home/tyler/test/src/github.com/goraft/raft/server.go:609 +0x444
github.com/goraft/raft.func·007()
    /home/tyler/test/src/github.com/goraft/raft/server.go:470 +0x60
created by github.com/goraft/raft.(*server).Start
    /home/tyler/test/src/github.com/goraft/raft/server.go:471 +0x49e

goroutine 1 [chan receive]:
main.main()
    /home/tyler/test/src/github.com/otoolep/rqlite/main.go:133 +0x71a

goroutine 5 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc20803a060, 0x7f9b49b28c58, 0xc208044008)
    /home/tyler/test/src/code.google.com/p/log4go/termlog.go:27 +0x7a
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/tyler/test/src/code.google.com/p/log4go/termlog.go:19 +0x6b

goroutine 7 [syscall]:
os/signal.loop()
    /home/tyler/.gvm/gos/go1.4.2/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /home/tyler/.gvm/gos/go1.4.2/src/os/signal/signal_unix.go:27 +0x35

goroutine 21 [IO wait]:
net.(*pollDesc).Wait(0xc208010bc0, 0x72, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010bc0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010b60, 0xc2080d5000, 0x1000, 0x1000, 0x0, 0x7f9b49b28ac8, 0xc2082731a8)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208044028, 0xc2080d5000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc208264728, 0xc2080d5000, 0x1000, 0x1000, 0x2, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc208277360, 0xc2080d5000, 0x1000, 0x1000, 0x2, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc2081db440)
    /home/tyler/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2081db440, 0xc20829bb0a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc2081db440, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc20825bfb0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc20825bfb0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc2081db440, 0xc20829e340, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc2082646e0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc2082646e0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 9 [chan receive]:
code.google.com/p/log4go.ConsoleLogWriter.run(0xc20803a1e0, 0x7f9b49b28c58, 0xc208044008)
    /home/tyler/test/src/code.google.com/p/log4go/termlog.go:27 +0x7a
created by code.google.com/p/log4go.NewConsoleLogWriter
    /home/tyler/test/src/code.google.com/p/log4go/termlog.go:19 +0x6b

goroutine 10 [chan receive]:
database/sql.(*DB).connectionOpener(0xc20807e000)
    /home/tyler/.gvm/gos/go1.4.2/src/database/sql/sql.go:589 +0x4c
created by database/sql.Open
    /home/tyler/.gvm/gos/go1.4.2/src/database/sql/sql.go:452 +0x31c

goroutine 11 [IO wait]:
net.(*pollDesc).Wait(0xc208011cd0, 0x72, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011cd0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208011c70, 0x0, 0x7f9b49b28ac8, 0xc208273430)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208044138, 0x50d08e, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/tcpsock_posix.go:234 +0x4e
net/http.tcpKeepAliveListener.Accept(0xc208044138, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1976 +0x4c
net/http.(*Server).Serve(0xc20803a540, 0x7f9b49b2a988, 0xc208044138, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1728 +0x92
net/http.(*Server).ListenAndServe(0xc20803a540, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1718 +0x154
github.com/otoolep/rqlite/server.(*Server).ListenAndServe(0xc208094120, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/test/src/github.com/otoolep/rqlite/server/server.go:294 +0x163e
main.func·002()
    /home/tyler/test/src/github.com/otoolep/rqlite/main.go:128 +0x42
created by main.main
    /home/tyler/test/src/github.com/otoolep/rqlite/main.go:129 +0x62d

goroutine 22 [select]:
net/http.(*persistConn).roundTrip(0xc20824b080, 0xc208273330, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc208094360, 0xc20829e680, 0xc2082a0420, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:235 +0x558
net/http.send(0xc20829e680, 0x7f9b49b28eb0, 0xc208094360, 0x28, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xc208094328, 0xc20829e680, 0x28, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xc208094328, 0xc20829e680, 0x99e1a8, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/client.go:367 +0xb25
net/http.(*Client).Post(0xc208094328, 0xc2082a03c0, 0x28, 0x912370, 0x14, 0x7f9b49b291c0, 0xc208011650, 0xc208011284, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/client.go:441 +0x10d
github.com/goraft/raft.(*HTTPTransporter).SendAppendEntriesRequest(0xc2080942d0, 0x7f9b49b29370, 0xc20808cc60, 0xc2080113b0, 0xc208278820, 0x0)
    /home/tyler/test/src/github.com/goraft/raft/http_transporter.go:125 +0x4ec
github.com/goraft/raft.(*Peer).sendAppendEntriesRequest(0xc2080113b0, 0xc208278820)
    /home/tyler/test/src/github.com/goraft/raft/peer.go:193 +0x303
github.com/goraft/raft.(*Peer).flush(0xc2080113b0)
    /home/tyler/test/src/github.com/goraft/raft/peer.go:178 +0x29e
github.com/goraft/raft.(*Peer).heartbeat(0xc2080113b0, 0xc2081db6e0)
    /home/tyler/test/src/github.com/goraft/raft/peer.go:163 +0x5de
github.com/goraft/raft.func·005()
    /home/tyler/test/src/github.com/goraft/raft/peer.go:97 +0x79
created by github.com/goraft/raft.(*Peer).startHeartbeat
    /home/tyler/test/src/github.com/goraft/raft/peer.go:98 +0x1ae

goroutine 18 [select]:
net/http.(*persistConn).readLoop(0xc20805c160)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:928 +0x9ce
created by net/http.(*Transport).dialConn
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /home/tyler/.gvm/gos/go1.4.2/src/runtime/asm_amd64.s:2232 +0x1

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc20805c160)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 24 [select]:
github.com/goraft/raft.(*server).send(0xc20808cc60, 0x861ae0, 0xc2082788c0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/test/src/github.com/goraft/raft/server.go:630 +0x327
github.com/goraft/raft.(*server).AppendEntries(0xc20808cc60, 0xc2082788c0, 0xc20826edc0)
    /home/tyler/test/src/github.com/goraft/raft/server.go:933 +0x48
github.com/goraft/raft.func·001(0x7f9b49b2fb78, 0xc208264960, 0xc20829e750)
    /home/tyler/test/src/github.com/goraft/raft/http_transporter.go:246 +0x29e
net/http.HandlerFunc.ServeHTTP(0xc20802b0c0, 0x7f9b49b2fb78, 0xc208264960, 0xc20829e750)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1265 +0x41
github.com/gorilla/mux.(*Router).ServeHTTP(0xc20804c320, 0x7f9b49b2fb78, 0xc208264960, 0xc20829e750)
    /home/tyler/test/src/github.com/gorilla/mux/mux.go:98 +0x297
net/http.serverHandler.ServeHTTP(0xc20803a540, 0x7f9b49b2fb78, 0xc208264960, 0xc20829e750)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc2082648c0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 25 [IO wait]:
net.(*pollDesc).Wait(0xc2082a60d0, 0x72, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2082a60d0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2082a6070, 0xc2082b1000, 0x1000, 0x1000, 0x0, 0x7f9b49b28ac8, 0xc208273478)
    /home/tyler/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208044180, 0xc2082b1000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7f9b49b2aac0, 0xc208044180, 0xc20824b0d8, 0xc2082b1000, 0x1000, 0x1000, 0x7f9b49b17010, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2082777c0, 0xc2082b1000, 0x1000, 0x1000, 0xc207ffbbe7, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2081dbb60)
    /home/tyler/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2081dbb60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /home/tyler/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc20824b080)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 26 [select]:
net/http.(*persistConn).writeLoop(0xc20824b080)
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /home/tyler/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

Second try errors(no db.sqlite file in follower)

leader raft(appears to be correct, except for maybe the connection string)

{
    "leader": "229a633",
    "name": "229a633",
    "peers": {
        "3c37b08": {
            "name": "3c37b08",
            "connectionString": "http://localhost:4001"
        }
    },
    "state": "leader"

follower raft(shows no leader or peers)

{
    "leader": "",
    "name": "3c37b08",
    "peers": null,
    "state": "follower"
}

edit: I just tried this on multiple ports(just to make sure forwarding wasn't an issue) and have verified that ports aren't the issue.

Error in main.go:140

I'm getting an error when pulling rql:

src/github.com/otoolep/rqlite/main.go:140: unknown http.Client field 'Timeout' in struct literal

-join command to a follower node? Maybe redirect/forward to the leader.

I've been testing what happens when a node uses the -join command on another node that is not the leader of his network. IE, what happens when you run -join on a node that is currently a follower?

Here's how I'm going about it:

Create a two-node network. Node.1(Leader) and Node.2(Follower)
Create a Node.3, and use the -join command to connect to Node.2

When I do this, I get some strange results.

Node 1 shows only Node 2 as its peer follower.
Node 2 shows only Node 1 as the leader, and leaves out node 3
Node 3 shows no peers at all.

My preferred solution to this, is that if the -join command is trying to connect to a follower, then to automatically redirect/forward that join command to that network's leader. Other solutions might be to throw an error(it didn't),with the output address of that node's leader.

Here's the output of of the raft endpoints:

curl localhost:4001/raft?pretty 
{
    "leader": "38ce511",
    "name": "38ce511",
    "peers": {
        "6731b4c": {
            "name": "6731b4c",
            "connectionString": "http://localhost:4002"
        }
    },
    "state": "leader"
}
curl localhost:4002/raft?pretty 
{
    "leader": "38ce511",
    "name": "6731b4c",
    "peers": {
        "38ce511": {
            "name": "38ce511",
            "connectionString": "http://localhost:4001"
        }
    },
    "state": "follower"
}
curl localhost:4003/raft?pretty 
{
    "leader": "",
    "name": "7519049",
    "peers": {},
    "state": "follower"

Changing leader after initialization? IE, what if the leader disconnects?

Hey Philip, I have a question regarding leader disconnects. My use case is not just a replicated data, but a decentralized data store.

So Lets say I have 10 nodes, the first one being a leader. For some reason or another, the leader drops out(network reasons or other). But the network has to stay functional. Because of this, lets say I want to elect a new leader, IE node #2 .

There are several ways to handle. Here's a list of solutions that I have in order of preference.

Option 1) If the leader goes down, then change the leader to the 2nd peer in the list. All nodes have the list of peers anyway, and can (once you handle issue #13 ), see which ones are active.
- Users could then either manually change their curl XPOST statements to point to a new leader, or what would be a lot better, is if XPOST statements went to their own local node and redirected to the current leader. They already knows the leader location anyway, so I don't see why this wouldn't be doable. Currently if I try this, I get the following error: "error":"raft.Server: Not current leader"
Option 2) Notice that the leader requests are failing, and have the other nodes do a -join to the second node, which they can get from their own peer lists. This is really not optimal, because right now, a -join requires a completely clear directory, so the DB would need to be resynced.

BTW, here's what happens if I try to simulate the above conditions with your 3 node example:

Start up 3 nodes
Node #1 goes down, nodes 2 and 3 still show #1 being the leader.(Their /raft endpoints)
Try to make node 3 point to a new leader node 2, unfortunately have to clear the directory and do a whole DB resync because of the -join command. Connect to node 2.

Here's the raft endpoint outputs of node 2 and 3. Some things to notice are that node 2 does correctly say that its the leader, and the info appears to be correct. IMO though, its not really an acceptable solution because it requires all the other nodes to delete their directories.
EDIT: oddly node 3 shows itself to be its own peer in this case, and has two names. I'm not sure why.

~/rqlite$ curl localhost:4002/raft?pretty
{
    "leader": "26403fc",
    "name": "26403fc",
    "peers": {
        "1669380": {
            "name": "1669380",
            "connectionString": "http://localhost:4003"
        },
        "4630386": {
            "name": "4630386",
            "connectionString": "http://localhost:4001"
        }
    },
    "state": "leader"
}
curl localhost:4003/raft?pretty
{
    "leader": "26403fc",
    "name": "6ff833e",
    "peers": {
        "1669380": {
            "name": "1669380",
            "connectionString": "http://localhost:4003"
        },
        "26403fc": {
            "name": "26403fc",
            "connectionString": "http://localhost:4002"
        },
        "4630386": {
            "name": "4630386",
            "connectionString": "http://localhost:4001"
        }
    },
    "state": "follower"
}

Best way to implement a whitelist of nodes for a cluster(IE only let certain IPs join the rqlite cluster)?

Hey @otoolep , I'm not sure if this is an issue or not, but I'd like your expertise on the best way to implement a whitelist of nodes.

So lets say I only want certain IP addresses or connection strings to be able to join the network, for security reasons.

One solution I thought about is to just use an abstraction layer in my own project, to pass a parameter like:
-whitelist IP_ADDR_1 IP_ADDR_2 , etc.

I can do this in my own project as an abstraction layer, but let me know if this is something you'd be interested in building, and I won't do this.

Reconnecting gives an error : Cannot join with an existing log

I've been testing this out a bit, am really excited about it. But I'm having an issue, that whenever a client disconnects, or I cancel the process then try to reconnect to the master node, it gives me the following error:

[03/01/15 14:56:14] [INFO] Initializing Raft Server: node.3
[03/01/15 14:56:14] [INFO] Loading latest snapshot, if any, from disk
[03/01/15 14:56:14] [INFO] Attempting to join leader at localhost:4001
[03/01/15 14:56:14] [EROR] Cannot join with an existing log
[03/01/15 14:56:14] [EROR] Cannot join with an existing log

Does this mean that rqlite has no ability to reconnect to the master, without starting from scratch? What if the .sqlite DB has gigabytes worth of data. Obviously you wouldn't want to have to resync the whole thing just because of a disconnection.

My use case for this consists of many clients connecting and dropping out at will.

Thanks in advance.

Reconnecting to a leader fails from the same connection String. (IE replacing peers)

One thing is making debugging really difficult.

If a peer attaches to a leader, then has an error for some reason, and I need that same peer to rejoin from a specific connectionstring(an ip address in my case), I have to completely delete the leader and the peer, to get things working again. Otherwise it will stall on attempting to join peer, and the old leader node still shows the old connection string and old name.

I see either two solutions. I like 1) better.

If a leader receives a join command from that connetion string, replace it correctly with the new name and connection string. This scales better too.
It also might be useful, if you create an endpoint that can clear a specific peer, or all the peers. The code for this might have to be done to complete the first solution anyway.

Thanks.

Add native binaries and package distribution

As of now, you need Go installed to install rqlite. Given the nature of this project, this might be inadequate in many cases, as you just want to start a sqlite cluster without further dependencies.

Go comes to the rescue here, as it makes it super easy to cross compile the program and generate binaries for different OSs and architectures. Tools like gox make it even easier to automate.

First question we need to answer is, is there currently any restriction to run rqlite on any of the most popular platforms (linux, darwin, windows)? Is there something to improve first on this matter (i.e. specific build tags)?

Allow leader redirection to be disabled

Leader re-direction was recently enabled. It should be possible to disable this via a command line-switch.

semicolon in sql

[root@cloud rqlite]# curl -L -XPOST localhost:4001/db?pretty -d '
>CREATE TABLE "server" ("id" INTEGER DEFAULT NULL PRIMARY KEY AUTOINCREMENT,"servername" TEXT DEFAULT NULL,"setting" TEXT DEFAULT NULL,"created_at" INTEGER NOT NULL  DEFAULT CURRENT_TIMESTAMP,UNIQUE (servername));
>INSERT INTO "server" ("id", "servername", "setting")VALUES(2,"www.test1.com",'{"gzip": 1, "upstream": "server 127.0.0.1:81; server 127.0.0.1:82;"}');'
{
    "failures": [
        {
            "sql": "\nINSERT INTO \"server\" (\"id\", \"servername\", \"setting\")VALUES(2,\"www.test1.com\",{gzip:",
            "error": "unrecognized token: \"{\""
        }
    ]
curl: (6) Couldn't resolve host '1,'
curl: (6) Couldn't resolve host 'upstream:'

panic: assertion failed: leader.elected.at.same.term.283

I've been getting this error occasionally, what does it mean? panic: assertion failed: leader.elected.at.same.term.283

panic: assertion failed: leader.elected.at.same.term.283


goroutine 10 [running]:
github.com/goraft/raft._assert(0x8a9200, 0x9004d0, 0x1f, 0xc2080b3cb0, 0x1, 0x1)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/util.go:60 +0xcf
github.com/goraft/raft.(*server).processAppendEntriesRequest(0xc208001b00, 0xc2082328c0, 0xc2082328c0, 0xc2082328c0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:948 +0x413
github.com/goraft/raft.(*server).leaderLoop(0xc208001b00)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:849 +0x761
github.com/goraft/raft.(*server).loop(0xc208001b00)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:609 +0x444
github.com/goraft/raft.func·007()
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:470 +0x60
created by github.com/goraft/raft.(*server).Start
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:471 +0x49e

goroutine 1 [chan receive]:
main.main()
    /root/.openmarket/testnet/db/src/github.com/otoolep/rqlite/main.go:108 +0x62e

goroutine 6 [syscall]:
os/signal.loop()
    /root/.gvm/gos/go1.4.2/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /root/.gvm/gos/go1.4.2/src/os/signal/signal_unix.go:27 +0x35

goroutine 7 [chan receive]:
database/sql.(*DB).connectionOpener(0xc208044000)
    /root/.gvm/gos/go1.4.2/src/database/sql/sql.go:589 +0x4c
created by database/sql.Open
    /root/.gvm/gos/go1.4.2/src/database/sql/sql.go:452 +0x31c

goroutine 8 [IO wait]:
net.(*pollDesc).Wait(0xc2080c0610, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080c0610, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080c05b0, 0x0, 0x7fc350001a88, 0xc208293c40)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc208038040, 0x51054e, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/tcpsock_posix.go:234 +0x4e
net/http.tcpKeepAliveListener.Accept(0xc208038040, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1976 +0x4c
net/http.(*Server).Serve(0xc208033200, 0x7fc350003a68, 0xc208038040, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1728 +0x92
net/http.(*Server).ListenAndServe(0xc208033200, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1718 +0x154
github.com/otoolep/rqlite/server.(*Server).ListenAndServe(0xc208066120, 0x7fff78f59534, 0x13, 0x0, 0x0)
    /root/.openmarket/testnet/db/src/github.com/otoolep/rqlite/server/server.go:321 +0x145b
main.func·002()
    /root/.openmarket/testnet/db/src/github.com/otoolep/rqlite/main.go:99 +0x42
created by main.main
    /root/.openmarket/testnet/db/src/github.com/otoolep/rqlite/main.go:100 +0x533

goroutine 166 [IO wait]:
net.(*pollDesc).Wait(0xc20833b2c0, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20833b2c0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20833b260, 0xc2081ff000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293e40)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038398, 0xc2081ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc2082edb28, 0xc2081ff000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc2082a6b40, 0xc2081ff000, 0x1000, 0x1000, 0xe, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc2080e8d20)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080e8d20, 0xc20823600a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc2080e8d20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2080e30e0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2080e30e0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc2080e8d20, 0xc20834a9c0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc2082edae0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc2082edae0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 123 [IO wait]:
net.(*pollDesc).Wait(0xc208010f40, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010f40, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010ee0, 0xc208043800, 0x400, 0x400, 0x0, 0x7fc350001a88, 0xc2081c7de0)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038058, 0xc208043800, 0x400, 0x400, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
crypto/tls.(*block).readFromUntil(0xc2081378f0, 0x7fc350003740, 0xc208038058, 0x5, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/crypto/tls/conn.go:454 +0xe6
crypto/tls.(*Conn).readRecord(0xc2080e0000, 0x17, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/crypto/tls/conn.go:539 +0x2da
crypto/tls.(*Conn).Read(0xc2080e0000, 0xc2080d6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/crypto/tls/conn.go:904 +0x166
net/http.noteEOFReader.Read(0x7fc350009c58, 0xc2080e0000, 0xc2080463c8, 0xc2080d6000, 0x1000, 0x1000, 0x7fc34fff0010, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081bd5c0, 0xc2080d6000, 0x1000, 0x1000, 0xc207ffc7fa, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208092a20)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208092a20, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208046370)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 170 [select]:
github.com/goraft/raft.(*Peer).heartbeat(0xc2080c1260, 0xc20833ea80)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:148 +0x834
github.com/goraft/raft.func·005()
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:97 +0x79
created by github.com/goraft/raft.(*Peer).startHeartbeat
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:98 +0x1ae

goroutine 18 [IO wait]:
net.(*pollDesc).Wait(0xc208010ed0, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010ed0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010e70, 0xc2080ba000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc20802b570)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038118, 0xc2080ba000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc350003740, 0xc208038118, 0xc2080465d8, 0xc2080ba000, 0x1000, 0x1000, 0xc2080306c0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801ede0, 0xc2080ba000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208033080)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208033080, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208046580)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 17 [syscall, locked to thread]:
runtime.goexit()
    /root/.gvm/gos/go1.4.2/src/runtime/asm_amd64.s:2232 +0x1

goroutine 14 [IO wait]:
net.(*pollDesc).Wait(0xc208010b50, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208010b50, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc208010af0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc20802b3a8)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080380d0, 0xc20800f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc350003740, 0xc2080380d0, 0xc208046318, 0xc20800f000, 0x1000, 0x1000, 0xc208030640, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc20801eaa0, 0xc20800f000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc208032c60)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc208032c60, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc2080462c0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 15 [select]:
net/http.(*persistConn).writeLoop(0xc2080462c0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 19 [select]:
net/http.(*persistConn).writeLoop(0xc208046580)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 171 [select]:
github.com/goraft/raft.(*server).send(0xc208001b00, 0x85db00, 0xcad3f0, 0x0, 0x0, 0x0, 0x0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:630 +0x327
github.com/goraft/raft.(*server).Do(0xc208001b00, 0x7fc350009ca8, 0xcad3f0, 0x0, 0x0, 0x0, 0x0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:897 +0x78
github.com/goraft/raft.func·010()
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:828 +0xaf
created by github.com/goraft/raft.(*server).leaderLoop
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:829 +0x2de

goroutine 22 [IO wait]:
net.(*pollDesc).Wait(0xc2080c0060, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080c0060, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080c0000, 0xc208090000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293b80)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038018, 0xc208090000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc208044408, 0xc208090000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc20801e4c0, 0xc208090000, 0x1000, 0x1000, 0xe, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc2080324e0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc2080324e0, 0xc20823600a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc2080324e0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2081d2360, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2081d2360, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc2080324e0, 0xc20834a4e0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc2080443c0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc2080443c0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 23 [IO wait]:
net.(*pollDesc).Wait(0xc2080c00d0, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080c00d0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080c0070, 0xc20809d000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293a90)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038028, 0xc20809d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc208044688, 0xc20809d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc20801e560, 0xc20809d000, 0x1000, 0x1000, 0xe, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc208032c00)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208032c00, 0xc20823600a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc208032c00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2081d2000, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2081d2000, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc208032c00, 0xc20834a270, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc208044640, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc208044640)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 162 [select]:
net/http.(*persistConn).writeLoop(0xc208287760)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 43 [IO wait]:
net.(*pollDesc).Wait(0xc2080c1100, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080c1100, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc2080c10a0, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293908)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038158, 0xc20810a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc350003740, 0xc208038158, 0xc208046aa8, 0xc20810a000, 0x1000, 0x1000, 0x7e92e0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2081a3ea0, 0xc20810a000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc2080e97a0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc2080e97a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208046a50)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

goroutine 45 [select]:
github.com/goraft/raft.(*server).send(0xc208001b00, 0x831920, 0xc2082328c0, 0x0, 0x0, 0x0, 0x0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:630 +0x327
github.com/goraft/raft.(*server).AppendEntries(0xc208001b00, 0xc2082328c0, 0xc2083449c0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/server.go:933 +0x48
github.com/goraft/raft.func·001(0x7fc350003b70, 0xc2082ede00, 0xc208335ee0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/http_transporter.go:246 +0x29e
net/http.HandlerFunc.ServeHTTP(0xc20802ade0, 0x7fc350003b70, 0xc2082ede00, 0xc208335ee0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1265 +0x41
github.com/gorilla/mux.(*Router).ServeHTTP(0xc20803a190, 0x7fc350003b70, 0xc2082ede00, 0xc208335ee0)
    /root/.openmarket/testnet/db/src/github.com/gorilla/mux/mux.go:98 +0x297
net/http.serverHandler.ServeHTTP(0xc208033200, 0x7fc350003b70, 0xc2082ede00, 0xc208335ee0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1703 +0x19a
net/http.(*conn).serve(0xc208045e00)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1204 +0xb57
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 169 [select]:
net/http.(*persistConn).roundTrip(0xc208287760, 0xc208242540, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc2080662d0, 0xc20834b040, 0xc2080e39e0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:235 +0x558
net/http.send(0xc20834b040, 0x7fc350001e40, 0xc2080662d0, 0x2a, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xc208066298, 0xc20834b040, 0x2a, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xc208066298, 0xc20834b040, 0x95d0a8, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:367 +0xb25
net/http.(*Client).Post(0xc208066298, 0xc2080e3950, 0x2a, 0x8d8dd0, 0x14, 0x7fc3500025d0, 0xc20833b880, 0xc2080ea901, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:441 +0x10d
github.com/goraft/raft.(*HTTPTransporter).SendAppendEntriesRequest(0xc208066240, 0x7fc3500022f0, 0xc208001b00, 0xc2080c0f50, 0xc2082327d0, 0x0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/http_transporter.go:125 +0x4ec
github.com/goraft/raft.(*Peer).sendAppendEntriesRequest(0xc2080c0f50, 0xc2082327d0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:193 +0x303
github.com/goraft/raft.(*Peer).flush(0xc2080c0f50)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:178 +0x29e
github.com/goraft/raft.(*Peer).heartbeat(0xc2080c0f50, 0xc20833e960)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:163 +0x5de
github.com/goraft/raft.func·005()
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:97 +0x79
created by github.com/goraft/raft.(*Peer).startHeartbeat
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:98 +0x1ae

goroutine 44 [select]:
net/http.(*persistConn).writeLoop(0xc208046a50)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 167 [IO wait]:
net.(*pollDesc).Wait(0xc20833b330, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20833b330, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20833b2d0, 0xc2080c8000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293eb0)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc2080383a0, 0xc2080c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.(*liveSwitchReader).Read(0xc2082edbc8, 0xc2080c8000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:214 +0xab
io.(*LimitedReader).Read(0xc2082a6ba0, 0xc2080c8000, 0x1000, 0x1000, 0xe, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/io/io.go:408 +0xce
bufio.(*Reader).fill(0xc208093620)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).ReadSlice(0xc208093620, 0xc20834450a, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:295 +0x257
bufio.(*Reader).ReadLine(0xc208093620, 0x0, 0x0, 0x0, 0xc207ff1c00, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:324 +0x62
net/textproto.(*Reader).readLineSlice(0xc2080e3560, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:55 +0x9e
net/textproto.(*Reader).ReadLine(0xc2080e3560, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/textproto/reader.go:36 +0x4f
net/http.ReadRequest(0xc208093620, 0xc20834ab60, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/request.go:598 +0xcb
net/http.(*conn).readRequest(0xc2082edb80, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:586 +0x26f
net/http.(*conn).serve(0xc2082edb80)
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1162 +0x69e
created by net/http.(*Server).Serve
    /root/.gvm/gos/go1.4.2/src/net/http/server.go:1751 +0x35e

goroutine 168 [select]:
net/http.(*persistConn).roundTrip(0xc208046a50, 0xc208242360, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1082 +0x7ad
net/http.(*Transport).RoundTrip(0xc2080662d0, 0xc20834aea0, 0xc2080e3890, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:235 +0x558
net/http.send(0xc20834aea0, 0x7fc350001e40, 0xc2080662d0, 0x2c, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:219 +0x4fc
net/http.(*Client).send(0xc208066298, 0xc20834aea0, 0x2c, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:142 +0x15b
net/http.(*Client).doFollowingRedirects(0xc208066298, 0xc20834aea0, 0x95d0a8, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:367 +0xb25
net/http.(*Client).Post(0xc208066298, 0xc2080e37d0, 0x2c, 0x8d8dd0, 0x14, 0x7fc3500025d0, 0xc20833b650, 0xc2080ea901, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/client.go:441 +0x10d
github.com/goraft/raft.(*HTTPTransporter).SendAppendEntriesRequest(0xc208066240, 0x7fc3500022f0, 0xc208001b00, 0xc2080c0d90, 0xc208232780, 0x0)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/http_transporter.go:125 +0x4ec
github.com/goraft/raft.(*Peer).sendAppendEntriesRequest(0xc2080c0d90, 0xc208232780)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:193 +0x303
github.com/goraft/raft.(*Peer).flush(0xc2080c0d90)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:178 +0x29e
github.com/goraft/raft.(*Peer).heartbeat(0xc2080c0d90, 0xc20833e840)
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:163 +0x5de
github.com/goraft/raft.func·005()
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:97 +0x79
created by github.com/goraft/raft.(*Peer).startHeartbeat
    /root/.openmarket/testnet/db/src/github.com/goraft/raft/peer.go:98 +0x1ae

goroutine 124 [select]:
net/http.(*persistConn).writeLoop(0xc208046370)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:945 +0x41d
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:661 +0xcbc

goroutine 161 [IO wait]:
net.(*pollDesc).Wait(0xc20833a450, 0x72, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc20833a450, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).Read(0xc20833a3f0, 0xc20833d000, 0x1000, 0x1000, 0x0, 0x7fc350001a88, 0xc208293ee8)
    /root/.gvm/gos/go1.4.2/src/net/fd_unix.go:242 +0x40f
net.(*conn).Read(0xc208038348, 0xc20833d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/net.go:121 +0xdc
net/http.noteEOFReader.Read(0x7fc350003740, 0xc208038348, 0xc2082877b8, 0xc20833d000, 0x1000, 0x1000, 0x7e92e0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:1270 +0x6e
net/http.(*noteEOFReader).Read(0xc2082a6400, 0xc20833d000, 0x1000, 0x1000, 0xc208012000, 0x0, 0x0)
    <autogenerated>:125 +0xd4
bufio.(*Reader).fill(0xc20833e2a0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:97 +0x1ce
bufio.(*Reader).Peek(0xc20833e2a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /root/.gvm/gos/go1.4.2/src/bufio/bufio.go:132 +0xf0
net/http.(*persistConn).readLoop(0xc208287760)
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:842 +0xa4
created by net/http.(*Transport).dialConn
    /root/.gvm/gos/go1.4.2/src/net/http/transport.go:660 +0xc9f

Return "rows affected" when database modified

The Go SQL interface supportes RowsAffected in reponse to an database-change. This value could be returned in the JSON response.

Swap out Raft implementation

go-raft is no longer maintained. Swapping out the Raft implementation for that in etcd could be an option. Another is the new implementation in InfluxDB 0.9.

error: raft.Server: Not current leader with leader redirection turned on

I have XPOST leader redirection turned on, but sometimes I still get this error:

 "failures": [
        {
            "sql": "UPDATE Seller...",
            "error": "raft.Server: Not current leader"
        }
    ]

It only happens occasionally, sometimes the writes go through without error. What might be causing it?

Deprecate code.google.com/p/log4go

https://code.google.com is shutting down. Also, https://code.google.com/p/log4go has been unmaintained for more than three years. Furthermore, Go's standard library is awesome and the community is embracing it, and using third-party libraries only when mandatory (see Three falacies of dependencies).

The options here:

Let log4go rest in peace and use log package instead.
Grab one of the log4go clones on Github (e.g https://github.com/alecthomas/log4go) and use it, so only the import path must be changed.
Find another high-level log implementation and replace it (e.g. https://github.com/golang/glog)

Thoughts?

Add extensive unit tests at Server-level

The system needs many more tests at the server-level. Basic integration testing has been performed, but more extensive testing is needed.

Run rqlite through Go's GC profiler

GC analysis should be performed of rqlite.

How to do a restore after updating to a new version of rqlite?

Hey @otoolep , whats the best way to do a restore?

I needed to update to a new version of rqlite, so I saved the db.sqlite file.

Then I reinstalled rqlite and replaced the db.sqlite file, but it doesn't appear to work correctly.

Will copying the whole data/db directory work?

Thanks.

Good for multiple clients/server sync?

I have hunting for a way to sync safely sqlite databases. However I need to do multiple iOS - Server.

I understand that I need to build a raft client, so the question is if this approach (ie: using raft) could work for sync ios-server...

Go database/sql driver?

Is there a standard way to use this from Go code? Maybe a database/sql compatible driver?

Include "connected" state in nodes output

From issue #12, seems like a worthy idea.

Something like:

{
    "peers": {
        "2456e8c": {
            "name": "2456e8c",
            "connectionString": "http://localhost:4002",
            "isConnected" : "false" 
       ...

Report of timings via text and embedded unit (uom), using unicode chars even

Numbers are best represented as numbers in JSON, not text.

And the semantic and unit of that number should be either fixed, for example milliseconds, or the unit of measure should be specified in its own JSON bit.

The current design makes for nicely readable output, but is a poor choice for machine parsable output, since the text value must currently be parsed itself, despite the document returned already having been parsed by JSON.

There could be any number of alternative output, like

{
"time": 0.0123,
"time-sec": 0.0123,
"time": [0.0123, "s"],
"time": [12.3, "ms"],
"time": [0.0123, 1.0],
"time": [12.3, 1e-3],
"time": { "value": 12.3, "uom": "ms"}
}

I first simplest is best, and sticking to the second, the SI unit, the first solution is best IMHO.

My $0.02 :)

DB always locked when running rqlite, even for reads.

I have a case where I prefer to read from the local sqlite db file directly with my own sqlite jdbc driver. I want to do this mainly for speed, and because I'm using an ORM framework that makes reading and constructing write statements for rql easier.

But whenever RQL is running, it completely locks the db.sqlite file from reading.

Here's the relevant part of the stack trace error I'm getting:

Caused by: java.sql.SQLException: [SQLITE_BUSY]  The database file is locked (database is locked)
    at org.sqlite.DB.newSQLException(DB.java:383)
    at org.sqlite.DB.newSQLException(DB.java:387)
    at org.sqlite.DB.throwex(DB.java:374)
    at org.sqlite.NativeDB.prepare(Native Method)
    at org.sqlite.DB.prepare(DB.java:123)
    at org.sqlite.PrepStmt.<init>(PrepStmt.java:42)

Node getting stalled on snapshotting / empty connection strings

Sometimes I'll have a node disconnect, and just try to restart it in the correct way, but its raft log will say that its snapshotting:

{
    "leader": "38567ec",
    "name": "57e97c8",
    "peers": [
        {
            "name": "7da05d3",
            "connectionString": "http://104.236.44.89:4570"
        },
        {
            "name": "38567ec",
            "connectionString": ""
        }
    ],
    "state": "snapshotting"

Once its like this it can never rejoin again.

Support specifying the SQLite DSN

rqlite should support query parameter, for example

db, err := sql.Open("sqlite3", "test.db?cache=shared&mode=rwc")

can't build rqlite; goprotobuf error

I'm getting this error when I run go get github.com/otoolep/rqlite:

package github.com/otoolep/rqlite
    imports github.com/goraft/raft
    imports code.google.com/p/gogoprotobuf/proto
    imports code.google.com/p/goprotobuf/proto: unable to detect version control system for code.google.com/ path

Since google code is going away, I'm guessing the correct repo is here,
https://github.com/golang/protobuf

Descriptions of column data types for query results in JSON response would be useful

In order for a database API to map query results to specific types other than string, such as DATETIME, it would be useful to include the data types in the JSON response. Without this information, the database API implementation has to do a separate PRAGMA table_info query, and then parse the SQL statement to determine how those types map to result columns. It would greatly simplify database API implementations if the JSON response described the result column types.

Statistics endpoint, dots in variable names

In most templating engines, dots are illegal to use for variable names, since they traverse sub-objects.

So as a result, I'm unable to access variables like execute.Received

  "statistics" : {
    "server" : {
      "execute.Received" : {
        "count" : 2
      },

Could these dots be changed to a different delimiter? Maybe underscores, or just make another embedded object.