GithubHelp home page GithubHelp logo

xjdr / chicago Goto Github PK

View Code? Open in Web Editor NEW
9.0 9.0 10.0 11.29 MB

High Performance Distributed Key Value Store

Java 95.82% Shell 3.02% Groovy 0.16% Makefile 1.00%
database distributed-database key-value-database

chicago's Issues

NPE on server on reachable

The client throws an NPE when the server(s) are not up

  • simulate by bringing down the cluster and it should be a proper message / response to the client

RocksDB JNI failures during unit tests

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000012db21f1e, pid=18947, tid=226327
#
# JRE version: Java(TM) SE Runtime Environment (8.0_40-b27) (build 1.8.0_40-b27)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C  [librocksdbjni6871628254650430731.jnilib+0x11f1e]  rocksdb_get_helper(JNIEnv_*, rocksdb::DB*, rocksdb::ReadOptions const&, rocksdb::ColumnFamilyHandle*, _jbyteArray*, int)+0x5e
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/jefrose/src/xjdr/chicago/hs_err_pid18947.log
Compiled method (nm)   66797 3581     n 0       org.rocksdb.RocksDB::get (native)
 total in heap  [0x000000011162e110,0x000000011162e480] = 880
 relocation     [0x000000011162e238,0x000000011162e278] = 64
 main code      [0x000000011162e280,0x000000011162e478] = 504
 oops           [0x000000011162e478,0x000000011162e480] = 8
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

Need to create a DB Provider Abstraction

Infinte loop of writes to exhaust DB

When we do performance testing with large amount of Data, sometimes the server goes into a state where it keeps writing data over and over again even when client has been shut down.

ChicagoClientTimeout during getNode for non-writeable channel

When a channel is not writeable anymore, the connection is re-established by the client in the following code, but the timeout happens before the connection is re-established.

private ChannelFuture _getNode(String node, long startTime) throws ChicagoClientTimeoutException {
.
.
.
    ChannelFuture cf = connectionMap.get(node);

    if (cf.channel().isWritable()) {
      return cf;
    }

    cf.channel().close();
    cf.cancel(true);
    connectionMap.remove(node);
    log.debug("removing and reconnecting to "+ node);
    connect(new InetSocketAddress(node, 12000), listenerMap.get(node));
    return getNode(node);

  }

Small data loss when replication has completed

When the node is being put in traffic for read/write for a key/colFam, there could be a small duration when the data is not going to that node as well as not being replicated from good nodes. In this case, the node might not have a small amount of data. The time for this data loss is the time taken by ZK to send the events to all the clients.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.