ssdb with rocksdb
ideawu / ssdb-rocks Goto Github PK
View Code? Open in Web Editor NEWssdb with rocksdb, rocksdb client-server support
License: BSD 3-Clause "New" or "Revised" License
ssdb with rocksdb, rocksdb client-server support
License: BSD 3-Clause "New" or "Revised" License
lua:
location /ssdb1 {
default_type 'text/plain';
content_by_lua '
local ssdb = require "resty.ssdb"
local db = ssdb:new()
db:set_timeout(1000) -- 1 sec
local ok, err = db:connect("127.0.0.1", 8888)
if not ok then
ngx.say("failed to connect: ", err)
return
end
for i=1,100000,1
do
db:init_pipeline()
db:set("key"..tostring(i), i)
local results, err = db:commit_pipeline()
end
';
}
spend about 42 seconds
php:
pconnect('127.0.0.1', 8888); $time_start = microtime_float(); //保存数据 for($i = 0; $i < 100000; $i++){ $redis->sadd("key$i",$i); } spend about 24 secondsis ssdbrocksdb production ready?
benchmarked ssdb is slower than leveldb using sata hdds, is this normal/expected?
If the system has gflags with both google
and gflags
namespaces installed, we try to define GFLAGS as two things. This breaks the compile.
facebook/rocksdb@742fa9e
I am running multiple reader/writer worker threads and seeing 15ms latency spikes. Based on various test results, I think the cause of high tail latency is contention for the lock associated with the BinlogQueue object (acquired when a Transaction object is created in functions like SSDB::set()). An SSDB instance has a single BinlogQueue object, so the lock acts as a global lock. If a worker thread is de-scheduled when it holds the lock, no other threads can make progress until this thread is re-scheduled, finishes its transaction, and releases the lock.
I am wondering about the purpose of BinlogQueue (which uses RocksDB's WriteBatch object). Each "batch" contains only a single Put() or Delete() operation, followed by a Put() associated with the BinlogQueue log (why?), and then db->Write() is called. My questions are:
What is the purpose of the BinlogQueue log?
I see there is an option to set no_log = true for the BinlogQueue. When is this recommended?
Given that a transaction seems to only contain a single operation, is a lock operation in the Transaction constructor necessary? If yes, why?
Thanks.
compilation environment:
CentOS release 6.6 and Ubuntu 14.04.4 LTS
/root/ali/ssdb-rocks/src/util does not compile and gives long list of error:
`g++ -std=c++0x -DNDEBUG -D__STDC_FORMAT_MACROS -Wall -O2 -Wno-sign-compare -I "/root/ali/ssdb-rocks/deps/rocksdb-master/include" -I "/root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include" -c log.cpp
In file included from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/stringfwd.h:40:0,
from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/string:39,
from ./strings.h:5,
from /root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include/jemalloc/jemalloc.h:8,
from ../version.h:7,
from ../include.h:24,
from log.h:4,
from log.cpp:1:
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:63:3: error: template with C linkage
template
^
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:66:3: error: template specialization with C linkage
template<>
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/memoryfwd.h:70:3: error: template with C linkage
template<typename, typename>
^
In file included from /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/string:39:0,
from ./strings.h:5,
from /root/ali/ssdb-rocks/deps/jemalloc-3.3.1/include/jemalloc/jemalloc.h:8,
from ../version.h:7,
from ../include.h:24,
from log.h:4,
from log.cpp:1:
/opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/stringfwd.h:52:3: error: template with C linkage
template
.
.
[snip]
.
.
^
./strings.h:235:32: error: declaration of C function ‘uint64_t big_endian(uint64_t)’ conflicts with
uint64_t big_endian(uint64_t v){
^
./strings.h:225:11: error: previous declaration ‘uint16_t big_endian(uint16_t)’ here
uint16_t big_endian(uint16_t v){
^
make[1]: *** [log.o] Error 1
make[1]: Leaving directory /root/ali/ssdb-rocks/src/util' make: *** [all] Error 2
i have found some segmentation fault with value when adjusting write_buffer_size(4<<20) from 4 to 8 or num_levels(7) changed from 7 to 10
it compiles but when running ssdb-server will get segmentation fault
for rocksdb, ssdb-server and ssdb.conf is all you need + the directory folder of the data right?
by the way, the configuration for ssdbROCKSDB is under "leveldb"?
same configuration settings?
leveldb:
# in MB
cache_size: 500
# in KB
block_size: 32
# in MB
write_buffer_size: 64
# in MB
compaction_speed: 1000
# yes|no
compression: no
Now ssdb-rocks supports all commands SSDB supports.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.