xjdr / chicago Goto Github PK
View Code? Open in Web Editor NEWHigh Performance Distributed Key Value Store
High Performance Distributed Key Value Store
Potential NPE:
if (finalMsg == null) {
needsToWrite = true;
}
switch (finalMsg.getOp()) {
When we do performance testing with large amount of Data, sometimes the server goes into a state where it keeps writing data over and over again even when client has been shut down.
The client throws an NPE when the server(s) are not up
Since the replication happens based on quorum defined in the config, there might be a mismatch between the client quorum and server quorum.
When a channel is not writeable anymore, the connection is re-established by the client in the following code, but the timeout happens before the connection is re-established.
private ChannelFuture _getNode(String node, long startTime) throws ChicagoClientTimeoutException {
.
.
.
ChannelFuture cf = connectionMap.get(node);
if (cf.channel().isWritable()) {
return cf;
}
cf.channel().close();
cf.cancel(true);
connectionMap.remove(node);
log.debug("removing and reconnecting to "+ node);
connect(new InetSocketAddress(node, 12000), listenerMap.get(node));
return getNode(node);
}
When the node is being put in traffic for read/write for a key/colFam, there could be a small duration when the data is not going to that node as well as not being replicated from good nodes. In this case, the node might not have a small amount of data. The time for this data loss is the time taken by ZK to send the events to all the clients.
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x000000012db21f1e, pid=18947, tid=226327
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b27) (build 1.8.0_40-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C [librocksdbjni6871628254650430731.jnilib+0x11f1e] rocksdb_get_helper(JNIEnv_*, rocksdb::DB*, rocksdb::ReadOptions const&, rocksdb::ColumnFamilyHandle*, _jbyteArray*, int)+0x5e
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/jefrose/src/xjdr/chicago/hs_err_pid18947.log
Compiled method (nm) 66797 3581 n 0 org.rocksdb.RocksDB::get (native)
total in heap [0x000000011162e110,0x000000011162e480] = 880
relocation [0x000000011162e238,0x000000011162e278] = 64
main code [0x000000011162e280,0x000000011162e478] = 504
oops [0x000000011162e478,0x000000011162e480] = 8
#
# If you would like to submit a bug report, please visit:
# http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
Need to create a DB Provider Abstraction
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.