GithubHelp home page GithubHelp logo

gwhalin / memcached-java-client Goto Github PK

View Code? Open in Web Editor NEW
662.0 119.0 449.0 3.07 MB

Information about this project lives on the wiki

Home Page: http://wiki.github.com/gwhalin/Memcached-Java-Client/

Java 100.00%

memcached-java-client's People

Contributors

gwhalin avatar newroot avatar scottkiss avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

memcached-java-client's Issues

getMulti return null value in map in 2.5.3

for your java docs, getMulti return a map, if key are not found, the key is not entered into the hashmap.
It's no problem at 2.5.1, but in 2.5.3 getMulti return null value in a map that key are not found.

eg. client.getMulti(new String[] {"xxx"}) , results are:
{} in 2.5.1,
{xxx=null} in 2.5.3

BufferOverflowException when getMulti with lots of keys

java.nio.BufferOverflowException
at java.nio.Buffer.nextPutIndex(Buffer.java:501)
at java.nio.DirectByteBuffer.putLong(DirectByteBuffer.java:738)
at com.schooner.MemCached.BinaryClient$NIOLoader$Connection.(Unknown Source)
at com.schooner.MemCached.BinaryClient$NIOLoader.doMulti(Unknown Source)
at com.schooner.MemCached.BinaryClient$NIOLoader.doMulti(Unknown Source)
at com.schooner.MemCached.BinaryClient.getMulti(Unknown Source)
at com.danga.MemCached.MemCachedClient.getMulti(Unknown Source)
at com.danga.MemCached.MemCachedClient.getMulti(Unknown Source)

should calculate the size of keys, instead of hard code 10K bytes

Detect When SASL was Not Used for Connection?

It appears that if the client is configured to use SASL but the Memcached server does not require it, the connection succeeds. Is there any way to tell that the server does not require SASL, other than to try connecting without it first? Even a way to query the pool to see if the SASL credentials were actually used to make the connection would be useful.

byte[] is not properly stored when client.setPrimitiveAsString(true)

Once the configuration setPrimitiveAsString is set true, the following code:
byte[] bytes = new byte[];
client.set(bytes);

will actually act as:
client.set(bytes.toString().getBytes[]); //according to line 392, AscIIClient.java
while "bytes.toString()" will return something like "[B@1837697", which is obviously wrong.

Is this a bug, or I'm using MemcacheClient in the wrong way?

incompatibility with a python client

2.6.6 stores a String value with flag=32 even with setPrimitiveAsString(true).
a python client (installed with pip install python-memcached) can not get the value.

MemCached: unknown flags on get: 20
13: handler exception: local variable 'val' referenced before assignment

i guess it's better to use flag=0 for setPrimitiveAsString(true) clients as it seems that
older versions used to do.

Possible native resource leak when number of connections exceeds pool size

This is happening in the performance branch.

If I read the code correctly, a one time socket will be created if the pool is full and all the sockets in it are being used. Then in AsciiClient class, sock.close() is called in the finally clause of all the command methods. Since sock.close() will not actually close the socket and will only put pooled sockets back in the pool, these one time sockets are never closed. This is reproducible by making the number of concurrent clients much more than the pool size. I was getting "Too many open files" within two minutes. The problem disappeared after I applied a fix.

java.lang.OutOfMemoryError Attempt to allocate stack guard pages failed

Exception in thread "http-bio-8080-exec-14" java.lang.OutOfMemoryError
Java HotSpot(TM) Server VM warning: Attempt to allocate stack guard pages failed.
at sun.misc.Unsafe.allocateMemory(Native Method)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:127)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source)
at com.schooner.MemCached.SchoonerSockIOFactory.createSocket(Unknown Source)
at com.schooner.MemCached.SchoonerSockIOFactory.makeObject(Unknown Source)

运行环境 java_memcached-release_2.5.2, tomcat7, jdk7
配置参数-server -Xms1548m -Xmx1548m -Xmn1000m -XX:PermSize=400m -XX:MaxPermSize=400m -XX:ReservedCodeCacheSize=100m
操作系统 Linux version 2.6.18-164.el5 ([email protected]) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Tue Aug 18 15:51:48 EDT 2009
cpu 单核 内存 4g,使用2g左右

cas unique data type?

This is not really an issue but a question instead. It's said in memcached protocol that "cas unique" is a 64 bit integer. I have two questions regarding this:

  1. I assume that it's a signed integer. So it's value could be negative?
  2. Is 0 the "cas unique" returned for non existing entries?

Thanks.

client unable to set objects greater than 1 Mb in version 2.5.1, was able to in 2.0.1

I just noticed the latest update of the memcached-java-client after playing around with spymemcached and then reading about the performance increases. Great work!

I do have one issue...

With 2.0.1, I was able to set objects > 1 Mb by increasing the slab max (passing in the -I flag to memcached). With 2.5.1, the set command returns false if the object size is > 1 Mb. I checked the source code and didn't see a limit being set.

Any ideas? Thanks for you time.
Owen

Problem with gets and cas

This thread is a continuation of issue #6

Mengli as you said, gets will return null in case of non existing value.

I guess the reason for introducing gets and cas is to avoid concurrency problems during add/update, Now if suppose 2 machines wants to add an entry simultaneously without race condition it is not possible as gets will return null (initial value will be null as the key wont exist. So we are forced to use put).

So present gets and sets address concurrent update problem and not Add.

One solution I could think of is to store the key and generate a cas value, if the key is not present, in memcache server itself

Out Of Memory, when host fails.

I've just been running some tests on 2.5.0, to investigate the binary protocol, and I've hit an Out of Memory exception that occurs if a node failure is simulated, while the application is running, i.e.:

  • start memcached daemon
  • write key
  • running 500 get() on key
  • stop daemon
  • runing 500 more get() on key

During the second lot of 500 get() you will recieve an OOM:

Exception in thread "main" java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:633)
    at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
    at com.schooner.MemCached.SchoonerSockIO.(SchoonerSockIO.java:51)
    at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(SchoonerSockIOPool.java:1369)
    at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(SchoonerSockIOPool.java:404)
    at com.schooner.MemCached.SchoonerSockIOPool.getConnection(SchoonerSockIOPool.java:545)
    at com.schooner.MemCached.SchoonerSockIOPool.getSock(SchoonerSockIOPool.java:480)
    at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:1519)
    at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:750)
    at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:746)
    at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)

Looking at the code, it seems to be coming from the ByteBuffer.allocateDirect in the SchoonerSockIO, constructor. The following patch in SchoonerSockIOPool.java (1375), seems to sort it out:

from

sock = getSocket(ip[0], Integer.parseInt(ip[1]), connectTimeout);

to

// get socket: default is to use non-blocking connect
try
{
    sock = getSocket(ip[0], Integer.parseInt(ip[1]), connectTimeout);
}
catch(Exception e)
{           
    writeBuf.clear();
    writeBuf = null;
    throw new IOException(e);
}

Seems to sort it out; but not sure if that causes any other issues.

Sorry if this post is messy, not sure of the markup to be used.
/dom

client unable to set objects greater than 1 Mb in version 2.5.1, was able to in 2.0.1

I just noticed the latest update of the memcached-java-client after playing around with spymemcached and then reading about the performance increases. Great work!

I do have one issue...

With 2.0.1, I was able to set objects > 1 Mb by increasing the slab max (passing in the -I flag to memcached). With 2.5.1, the set command returns false if the object size is > 1 Mb. I checked the source code and didn't see a limit being set.

Any ideas? Thanks for you time.
Owen

I'm running memcached version 1.4.3 with the following settings

PORT="11211"
USER="nobody"
MAXCONN="1024"
OPTIONS="-I 8m"
CACHESIZE=1536

To test, I created a Junit test and assertTrue on the set command for byte arrays of size 1000000 and 2000000.

SockIOPool.SockIO.clearEOL() has some wrong

This "continue" need to be deleted?

// only stop when we see
// \r (13) followed by \n (10)
if (b[0] == 13) {
eol = true;
continue;
}

            if (eol) {
                if (b[0] == 10)
                    break;

                eol = false;
            }

PortRange feature Request

There are some parts of our code which specify a port and connect. This clashes with open mem-cached ports sometimes. We handle it the following way currently

SockIOPool.java
protected static Socket getSocket(String host, int port, int timeout) throws IOException {

        SocketChannel sock = SocketChannel.open();
        sock.socket().setReuseAddress(true);
        sock.socket().bind(new InetSocketAddress(getNextPortFromRange()));
        sock.socket().connect(new InetSocketAddress(host, port), timeout);
        return sock.socket();

}

TCPSockIO.java
public final static Socket getSocket(String host, int port, int timeout) throws IOException {
SocketChannel sock = SocketChannel.open();
sock.socket().setReuseAddress(true);
sock.socket().bind(new InetSocketAddress(getNextPortFromRange()));
sock.socket().connect(new InetSocketAddress(host, port), timeout);
return sock.socket();
}

It would be great if we can expose an optional port range specifier as a config option to use.

Can't Find JavaDocs

Can't find them here, don't see them in the download package. I can't find the site which used to host this library either.

NullPointerException in getMulti while some concurrent calls to a SchoonerSockIOPool

Hi, Everyone:
I made a minor change on MemcachedPerfTest to test 'getMulti' instead of 'get', and then when I run the tests using thread number 50, runs number 100, I got the following exception sometimes:

Exception in thread "Thread-0" java.lang.NullPointerException
    at com.schooner.MemCached.AscIIClient$NIOLoader.doMulti(AscIIClient.java:1522)
    at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1082)
    at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1005)
    at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1207)
    at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1188)
    at com.schooner.MemCached.MemcachedPerfTest$bench.run(MemcachedPerfTest.java:123)

Then I did some researches and found there's a bug:

        try {
#1458       selector = Selector.open();

            // get the sockets, flip them to non-blocking, and set up data
            // structures
#1462       conns = new Connection[sockKeys.keySet().size()];
            ...
    } catch (IOException e) {
#1508       e.printStackTrace();
            return;
        } finally {
            ....
#1522       for (Connection c : conns) {
                if (c != null)
                    c.close();
                }
            }
        }

The exception occurred at #1522, it means conns is null, the possible reason is #1458 threw an exception and was not successfully run, so I added code #1508 for tracing the reason, and I got the stack of real exception:

 java.io.IOException: Unable to establish loopback connection
    at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.nio.ch.PipeImpl.(PipeImpl.java:122)
    at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:27)
    at java.nio.channels.Pipe.open(Pipe.java:133)
    at sun.nio.ch.WindowsSelectorImpl.(WindowsSelectorImpl.java:104)
    at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:26)
    at java.nio.channels.Selector.open(Selector.java:209)
    at com.schooner.MemCached.AscIIClient$NIOLoader.doMulti(AscIIClient.java:1458)
    at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1082)
    at com.schooner.MemCached.AscIIClient.getMulti(AscIIClient.java:1005)
    at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1207)
    at com.danga.MemCached.MemCachedClient.getMulti(MemCachedClient.java:1188)
    at com.schooner.MemCached.MemcachedPerfTest$bench.run(MemcachedPerfTest.java:123)
Caused by: java.net.BindException: Address already in use: connect
    at sun.nio.ch.Net.connect(Native Method)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:507)
    at java.nio.channels.SocketChannel.open(SocketChannel.java:146)
    at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78)
    ... 13 more

Not sure why nio throws out this exception, probably the concurrent load is too high?(but I don't think 50 concurrent is very high ), anyway I think we should not let null pointer exception happen, it's a bug.

Socket null. Failed to connect

When SchoonerSockIOPool.getSoket() returns null we return false.

In case of normal commands (set replace append ...) this is of no problem but for commands like add and cas we cant differentiate cas and add failure with connection failure.

Instead we could throw an RuntimeException which would be a configurable data

Memcached 的更新

com.whalin Memcached-Java-Client 3.0.0

我通过Maven依赖Memcached,发现与提供的java_memcached-release_2.6.6版本中的包名不一致,但是里面的接口好像一致。
com.whalin.MemCached
com.danga.MemCached
这给我带来了许多困扰,让我不知道该如何选择,总之我认为这不是一个好的现象。

Mc socket is keep reading but no data is readed

Hi All,
I got a strange problem using java memcached client. I have a resin which has 512 resin-tcp-connection, all are runnable, but they are hanging at mc socket reading. So the resin is no response, for it reaches max 512 http request. And it just happens on busy resin, which means the request num is >30/s, and the network traffic is >100Mb/s. There are 5*50=250 MC connections.It seemed that readFromChannel sometimes not work. the possiblity is less than 1/100000.
But once it ocurr, the resin thread will keep on reading but read on data. So these fake dead threads will be more and more.

   I also find someone else suffered similar problem about sun.nio.ch.FileDispatcher.read0, such as hadoop, jboss.

How to solve it? As the thread keep on reading but no timeout? Now I can only restart resin every day to avoid resin no responding.

And I was confuse about the code on com.schooner.MemCached.SchoonerSockIO.java

// the datagram sent from memcached mustn't beyond 1400 bytes.
public ByteBuffer readBuf = ByteBuffer.allocateDirect(8 * 1024); 

comment: the MTU on linux is 1500 by default. why you say 1400 bytes?
if the datagram is about 1400 bytes, why sets the buff 8_1024, not 4_1024?

"resin-tcp-connection-10.67.222.61:8388-19351" daemon prio=10 tid=0x00002aabba08f800 nid=0x13c0 runnable [0x00002aabf8906000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.FileDispatcher.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233)
at sun.nio.ch.IOUtil.read(IOUtil.java:200)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236)
- locked <0x00002aaadb561880> (a java.lang.Object)
at com.schooner.MemCached.SockInputStream.readFromChannel(SockInputStream.java:124)
at com.schooner.MemCached.SockInputStream.read(SockInputStream.java:109)
at com.schooner.MemCached.SockInputStream.getLine(SockInputStream.java:173)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:771)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:613)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:609)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at cn.org.dbcache.BaseManager.getFromMemCachedServer(BaseManager.java:1806)
at cn.org.dbcache.BaseManager.findById(BaseManager.java:434)
at cn.org.dbcache.BaseManager.getFromMemCachedServerList(BaseManager.java:1883)
at cn.org.dbcache.BaseManager.loadList(BaseManager.java:1317)

storeCounter

For performance branch:

In master the storeCounter overloads set() with primitiveAsString as true. The performance branch currently doesn't do this so storeCounter will cause the server to throw the following error message unless the flag is set for the entire instance of the client:

CLIENT_ERROR cannot increment or decrement non-numeric value

File descriptor leak in SchoonerSockIOPool

When creating a MemCachedClient against a bad/nonexistent memcached server, every call to the client will result in a file descriptor being claimed for a socket, but then never closed. These open FDs will stick around for the length of the process, even if the memcached server becomes valid.

Using 3.0.1, with tcp connection.

Looks to be caused by the following:

https://github.com/gwhalin/Memcached-Java-Client/blob/performance/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java#L1476

When requesting a connection from the pool, it will attempt to create a new TCPSockIO object (via GenericObjectPool.borrowObject()). Instantiation of TCPSockIO attempts to create and connect to a socket for the memcached server. But it does this as a two-step process: first opening the socket, then attempting to connect to it. If the connection throws an exception the socket is never closed. And since the TCPSockIO object was never fully instantiated, the object pool simply registers the exception and continues - with the next request repeating the process.

Fix should be relatively simple. Either use the SocketChannel convenience open method (which does not accept a timeout):

http://docs.oracle.com/javase/6/docs/api/java/nio/channels/SocketChannel.html#open%28java.net.SocketAddress%29

Or wrap the socket connect() call in a try/catch, and close if needed:

diff --git a/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java b/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java
index 396894a..7f4abe0 100644
--- a/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java
+++ b/src/main/java/com/schooner/MemCached/SchoonerSockIOPool.java
@@ -1474,7 +1474,14 @@ public class SchoonerSockIOPool {
                 */
                protected final static Socket getSocket(String host, int port, int timeout) throws IOException {
                     SocketChannel sock = SocketChannel.open();
+                    try {
                        sock.socket().connect(new InetSocketAddress(host, port), timeout);
+                     } catch (IOException e) {
+                        sock.socket().close();
+                        throw e;
+                    }
                     return sock.socket();
                }
 

About the big data operation using memcached-java-client!!help me out!!

We use memcached-java-client as one of our memcached client in our project.we found that when we operation big data some thing goes wrong:

[2012-03-05 22:58:29.872]-[ERROR] com.danga.MemCached.MemCachedClient ++++ exception thrown while writing bytes to server on get
[2012-03-05 22:58:29.874]-[ERROR] com.danga.MemCached.MemCachedClient An existing connection was forcibly closed by the remote host
java.io.IOException: An existing connection was forcibly closed by the remote host
at sun.nio.ch.SocketDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:33)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:26)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:1547)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:1482)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:738)
at com.schooner.MemCached.BinaryClient.get(BinaryClient.java:734)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:917)
at com.eshore.crm.memcache.client.SimpleTest$1.run(SimpleTest.java:91)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
null

Test code:
memcached server:
/usr/local/bin/memcached -d -m 4096m -u mem -vv -p 11212 -I 10m

client test:

import java.util.UUID;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

import com.danga.MemCached.MemCachedClient;
import com.danga.MemCached.SockIOPool;

public class SimpleTest {

// create a static client as most installs only need
// a single instance
protected static MemCachedClient mcc;

// set up connection pool once at class load
static {

    // server list and weights
    String[] servers = { "10.17.13.49:11212" };

    Integer[] weights = { 3, 3, 2 };

    // grab an instance of our connection pool
    SockIOPool pool = SockIOPool.getInstance();

    // set the servers and the weights
    pool.setServers(servers);
    pool.setWeights(weights);

    // set some basic pool settings
    // 5 initial, 5 min, and 250 max conns
    // and set the max idle time for a conn
    // to 6 hours
    pool.setInitConn(5);
    pool.setMinConn(5);
    pool.setMaxConn(250);
    pool.setMaxIdle(1000 * 60 * 60 * 6);

    // set the sleep for the maint thread
    // it will wake up every x seconds and
    // maintain the pool size
    pool.setMaintSleep(30);

    // set some TCP settings
    // disable nagle
    // set the read timeout to 3 secs
    // and don’t set a connect timeout
    pool.setNagle(false);
    pool.setSocketTO(3000);
    pool.setSocketConnectTO(0);

    pool.setBufferSize(30000000);

    // initialize the connection pool
    pool.initialize();

    // lets set some compression on for the client
    // compress anything larger than 64k
    //mcc.setCompressEnable(true);
    //mcc.setCompressThreshold(64 * 1024);
    mcc = new MemCachedClient(true);
}

public static void main(String args[]){
    int k=1;
    String[] values= new String[8000];
    int j = 0;
    for (int i = 0; i < keys.length; i++) {
        keys[j] = "3";
        j++;
    }
    BlockingQueue<Runnable> startStopQueue = new LinkedBlockingQueue<Runnable>();
    ThreadPoolExecutor taskExecutor = new ThreadPoolExecutor(10,
            10, 10, TimeUnit.SECONDS,
            startStopQueue);
    taskExecutor.allowCoreThreadTimeOut(true);
    final String[] vs=values;
    final MemCachedClient mc = mcc;
    while(k>0){
        //mcc.delete("foo");
        taskExecutor.submit(new Runnable(){

            @Override
            public void run() {
                    //mcc = new MemCachedClient(true);
                    String key = "foo"+UUID.randomUUID();
                    mcc.set(key, vs);
                    Object obj = mcc.get(key);
                    System.out.println(obj);
            }

        });
        try {
            TimeUnit.MICROSECONDS.sleep(100);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        k--;
    }

    taskExecutor.shutdown();
    while(!taskExecutor.isTerminated()){
        try {
            TimeUnit.SECONDS.sleep(1);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
    }
    System.exit(1);
}

}

when I change the values size to 1000 the error won't happend.did something wrong with my code or config .please help me out !!

BinaryClient should use primitiveAsString mode for all counter operations

It's easy to make mistakes with counter operations. The client should use primitive as String at all times with counters:

    final MemCachedClient client = new MemCachedClient( true, true );

    // fails without this line below
    client.setPrimitiveAsString( true );

    final String key = "testKey" + Math.random();
    client.storeCounter( key, 3 );
    client.incr( key );

    // prints 4 if asString, -1 if not
    System.out.println( client.getCounter( key ) );

New Feature Request.

Hi Mengli,

We are currently using the Memcached client and providing the pool parameters through our custom configuration system. We are using the client object as a spring singleton wired into our main codebase.
The issue we are facing is, we want to refresh the pool when any config value is changed in our config system.

In order to do so there should be a method in SchoonerSockIOPool class which will remove the pool object from "pools" HashMap. This method will take in poolname as argument and will remove that object from the HashMap.

If we can do this then we can remove the pool object if any config value is changed and then reinitialize the pool at runtime with new values. With addition of this feature we don't have to recycle our servers everytime a config value is changed.

I was wondering how difficult it would be for you to add this feature?

Thank you for your time.

Regards,
bhargav

getMultiArray() does not return all objects cacehed

Hi all,

Has anybody experienced problems using java client's getMultiArray()
method(Schooner memcached client 2.5.0)?

We run into this problem recently - getMultiArray() does not return
all objects specified. But when trying to load the missing objects
using get() method one by one, they are all fine. Those objects also
exist in memcached server with proper expiration time when checking
with 'stats cachedump'.

Any help appreciated, Thanks!

Failover issue.

Hi,

We are trying to use your memcached-java client to connect to the memcacheD server and we want to test following failover scenario.

  • We are pulling the ethernet cable from the server while the client is writing entries to the memcacheD.
    In this scenario, the client is not successfully switching over the the next host in the list, rather it hangs there. We waited for 180secs which is default socket time out at the kernel level but still it was hanging.

Then when we reconnect the cable, it successfully resumes. So my question was, are we missing any configuration values?

btw I am setting the "setFailback" and "setFailover" flags to true.

here is my code snippet:

SchoonerSockIOPool pool = null;
try {
pool = SchoonerSockIOPool.getInstance("default", AuthInfo.typical("cacheuser", "password"));
pool.setServers( servers );
pool.setWeights( weights );
pool.setInitConn( 10 );
pool.setMinConn( 5 );
pool.setMaxConn( 250 );
pool.setMaintSleep( 10 );
pool.setNagle( false );
pool.setSocketTO( readTO );
pool.setSocketConnectTO( 0 );
pool.setMaxIdle( maxIdle );
pool.setFailback(true);
pool.setFailover(true);
pool.initialize();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}

So is this client behavior expected?

We will highly appreciate your help on this issue.

Thank you for your time.

Unclosed GZIPOutputStream cause mass memory occupation while stress testing

Recently our developer was evaluating Memcached as an alternative solution on site specific problems. When we started stress test on Memcached with compress enabled, something went wrong.
We call the set(...) method of MemcachedClient within a loop with a large base, like one million times per batch; and we limited the Java heap size up to 2gb, the stack size up to 1024k. After the test began, the Resident Memory of this Java process inflated dramaticly in liner pattern, without the acknowledgement of heap limit. Our test server has 16GB physical memory, but in a short while, within 40 seconds approximately, this test Java process had taken up all the physical memory and throws OOM with the exception shows the zlib was just trying to allocate memory.
Then we've profiled the test process but found nothing abnormal, so we dumped the heap, and it was okay, only occupies 123mb, and the stack was also okay; as the GZIPOurputStream utilizing JNI call, we believe this problem was caused by over-inflation of native heap, the memory malloced by JNI function.
So we examined the GZIPOutputStream and found it extends DeflaterOutputStream and it use Deflater to compress stream, according to Java-doc of Deflater, the caller should invoke the end() method of Deflater object when the compressor is no longer being used or it will also be called automatically by the finalize() method. So here is the problem, now the MemcachedClient code does not explicitly invoke the end() method(which is indirectly invoked by the close() method of GZIPOutputStream), then the Java VM may not realize the mass memory occupied by JNI portion, and the Deflator object may not be GCed at first priority(in previously dumped heap, the Deflator class has approximately 6 thousand instances), at this point, the native heap memory was out of control.
To sum up, we insert a simple line after line 800:
gos.finish();
gos.close();
then the memory problem definitely gone, all seems good afterward, the Java process Resident Memory had never gone too far, just around 800mb.

So should we add this line to the trunk code?

infinite loop caused by IdentityHashMap

Memcached-Java-Client 1.6
java version "1.6.0_03"
Java(TM) SE Runtime Environment (build 1.6.0_03-b05)
Java HotSpot(TM) Client VM (build 1.6.0_03-b05, mixed mode, sharing)

It's IdentityHashMap bug in the IdentityHashMapIterator.remove() method in jdk5, 6,
can also see https://issues.apache.org/jira/browse/DIRMINA-452, http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6612102

can change the use of Memcached-Java-Client?

call stack
"worker-15" daemon prio=10 tid=0x080f1800 nid=0x2d6b runnable [0xb483f000..0xb483ffb0]
java.lang.Thread.State: RUNNABLE
at java.util.IdentityHashMap.put(IdentityHashMap.java:421)
at com.danga.MemCached.SockIOPool.addSocketToPool(SockIOPool.java:961)
at com.danga.MemCached.SockIOPool.checkIn(SockIOPool.java:1045)
- locked <0x727fccc0> (a com.danga.MemCached.SockIOPool)
at com.danga.MemCached.SockIOPool.checkIn(SockIOPool.java:1060)
at com.danga.MemCached.SockIOPool$SockIO.close(SockIOPool.java:1520)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:780)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:473)

Logger is too weak

gwhalin's memcached client implements a Logger itself.

But the logger is really weak. Could it support some mechanism which enables users to apply an alternative Logger implementation such as log4j?

Why there're lots of warning logs for getMulti?

Hi, Everyone:
I always get warnings when I call getMulti, the message is as follows:
WARN com.danga.MemCached.MemCachedClient - Caught the aforementioned exception on Connection to localhost:11211 with 1 bufs; done is true

Is it normal or any bug caused this?

Stats Cache Dump Only Works with the ASCII Protocol?

I'm able to use MemCachedClient.statsCacheDump successfully when connecting to a Memcached server using the ASCII protocol, but no items are returned when connecting using the binary protocol with SASL. Other operations work over the binary protocol with SASL, including the stats, statsSlabs and statsItems.

I'm using java-memcached-2.6.2.jar from the performance branch with JDK 6u26 32-bit on Linux. The Memcached server is 1.4.6 with libevent 2.0.12-stable. It is running with the following options:

memcached -d -p 11211 -u nobody -m 256 -c 1024 -P /var/run/memcached/memcached.pid -S

2.5.0 performance bottleneck above 10240B?

I’ve been running the numbers to compare 2.0.1 and 2.5.0 and I’ve noticed something. This is against a simple single thread unit test with 64B, 1024B, 10240B, and 102400B. The first 3 perform as I would expect but at 102400b, it goes to crap. Not sure where the actual threshold is but above 10240B, each additional 10240B adds about an additional 2-3 seconds to the test.

With client 2.0.1
testCaching64B

## Total time: 3074ms
## Total time: 2813

testCaching1024B

## Total time: 3008
## Total time: 2958

testCaching10240B

## Total time: 4531
## Total time: 4473

testCaching102400B

## Total time: 6776
## Total time: 6835

Now with client 2.5.0 (same server):
testCaching64B

## Total time: 2457
## Total time: 2420

testCaching1024B

## Total time: 2673
## Total time: 2637

testCaching10240B

## Total time: 3600
## Total time: 3551

testCaching102400B

## Total time: 20908
## Total time: 20977

(This was on an 8 way, linux, 64bit box with 16gb ram)

Are others seeing this? Do you have any idea what might cause this? Based on these numbers, we won’t be able to upgrade.

incorrect conditional statement for NOT_FOUND status

Memcached-Java-Client 2.6.3

In code com.schooner.MemCached.AscIIClient

line 580: String line = new SockInputStream(sock, Integer.MAX_VALUE).getLine().split("\r\n")[0];
line 585: } else if (NOTFOUND.equals(line)) {

And com.danga.MemCached.MemCachedClient.NOTFOUND = "NOT_FOUND\r\n", but variable line = "NOT_FOUND", so the result of line 585 is always FALSE.

I use "NOTFOUND.startsWith(line)" to instead of "NOTFOUND.equals(line)" temporarily.

NumberFormat exception in AsciiClient

After upgrading to 2.6.2, we're intermittently getting the below exceptions (against 1.4.5 server).


ERROR 2011-11-28 23:00:01,141 [com.danga.MemCached.MemCachedClient 740206481@qtp-1696446139-18735] For input string: "command"
java.lang.NumberFormatException: For input string: "command"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)

at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:734)

Looked into the code and found that it should be due to a breakdown of the below typical 'get' conversation with the server. Eg. a HIT yields

setting a key

set the_key 1 0 13
Just a number

retrieving the key

get the_key
VALUE the_key 32 13
Just a number
END

For the exception scenario, it seems that the AsciiClient was trying to pass a response from the server that looked like "VALUE the_key command xxx". Wondering where did the 'command' keyword came from.

Anyone facing the same issue?

SockIOPool.shutDown() seems has a dead lock with selfMaint()

When I run the class MemCachedBench.java, I find sometimes the programme can't exit normally.
Then I debug it with eclipse, I found the "maintThread" is still waiting at line 1298, seems like dead lock.

protected void selfMaint() {
    if ( log.isDebugEnabled() )
        log.debug( "++++ Starting self maintenance...." );

    // go through avail sockets and create sockets
    // as needed to maintain pool settings
    Map needSockets =
        new HashMap();

************** synchronized( this ) {
// find out how many to create
for ( Iterator i = availPool.keySet().iterator(); i.hasNext(); ) {

I think maybe you shouldn't use "synchronized" in Method shutDown(), or you should use "this.wait(500);" to replace "Thread.sleep( 500 );".

the bin.jar and source.jar mismatch

I can't debug Memcached-Java-Client with the source.jar,because I can't set breakpoint in my IDE,the code line in class and source is not the same?

NullPointerException of memcached-client2.5.1 in Jmeter testing

Test tools:jakarta-jmeter-2.3.2
Thread number: 100,500,1000
Error:
java.lang.NullPointerException
at com.schooner.MemCached.SchoonerSockIOPool.getConnection(SchoonerSockIOPool.java:590)
at com.schooner.MemCached.SchoonerSockIOPool.getSock(SchoonerSockIOPool.java:514)
at com.schooner.MemCached.AscIIClient.set(AscIIClient.java:376)
at com.schooner.MemCached.AscIIClient.set(AscIIClient.java:241)
at com.danga.MemCached.MemCachedClient.set(MemCachedClient.java:603)
at test.TestMemcachedSet.setkey(TestMemcachedSet.java:126)
at test.TestMemcachedSet.runTest(TestMemcachedSet.java:101)
at org.apache.jmeter.protocol.java.sampler.JavaSampler.sample(JavaSampler.java:161)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:290)
at java.lang.Thread.run(Unknown Source)

TestCode:

public class TestMemcachedSet extends AbstractJavaSamplerClient {
private SampleResult results;
private String key;
private String value;
static SockIOPool pool;
static MemCachedClient mc;
private String serverlist;
private static int timeout;

public void setupTest(JavaSamplerContext arg0) {        

    results = new SampleResult();
    timeout = arg0.getIntParameter("timeout");
    key = arg0.getParameter("key");
    value = arg0.getParameter("value");
    serverlist = arg0.getParameter("serverlist");

    if (key != null && key.length() > 0) {
        results.setSamplerData(key);

    }   
    if (value != null && value.length() > 0) {
        results.setSamplerData(value);

    }
    if (serverlist != null && serverlist.length() > 0) {
        results.setSamplerData(serverlist);

    }   
}



public Arguments getDefaultParameters() {
    Arguments params = new Arguments();

    params.addArgument("serverlist", "172.20.57.167:11211,172.20.57.112:11211");
    params.addArgument("timeout", "5000"); 
    params.addArgument("key", "key"); 
    params.addArgument("value", "value");                                   

    return params;
}



public SampleResult runTest(JavaSamplerContext arg0) {

    results.sampleStart();  

    boolean result=false;

    try{
      results.setRequestHeaders(key);
      result = setkey(key,value);   
      results.setResponseMessage("set data:"+key+" is "+result);
      results.setSuccessful(result);
    }catch(Exception e){
        e.printStackTrace();
        results.setSuccessful(result);
    }finally {
        // Record end time and populate the results.
      results.sampleEnd();

    }

    return results;
}

public void teardownTest(JavaSamplerContext arg0) {
    //nothing todo  
}

private boolean setkey(String key,String value)
{
    pool = SockIOPool.getInstance();
    pool.setServers(serverlist.split(","));
    Integer[] weights={1,2};
    pool.setWeights(weights);

    pool.setNagle( false );
    pool.setSocketTO(timeout );

    pool.setHashingAlg(2);

    pool.initialize();
    mc = new MemCachedClient();

    mc.setPrimitiveAsString(true); 
    mc.setSanitizeKeys(false);
    boolean result = mc.set(key, value);
    pool.shutDown();
    return result ;
}

}

java memcached client 2.5 and 2.6 get() and getmulti() bug?

When mysql memcached client set some data to memcached, other language memcached client can get the value,but Java memcached client whth version 2.5 and 2.6 occuer following error and return null (version 2.0.1 runs well):

com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - ++++ exception thrown while trying to get object from cache for key: test117
com.danga.MemCached.MemCachedClient Tue Aug 16 11:17:56 CST 2011 - invalid stream header: 76616C75
java.io.StreamCorruptedException: invalid stream header: 76616C75
at java.io.ObjectInputStream.readStreamHeader(Unknown Source)
at java.io.ObjectInputStream.(Unknown Source)
at com.schooner.MemCached.ObjectTransCoder.decode(ObjectTransCoder.java:57)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:761)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:612)
at com.schooner.MemCached.AscIIClient.get(AscIIClient.java:608)
at com.danga.MemCached.MemCachedClient.get(MemCachedClient.java:1047)
at TestM.main(TestM.java:56)

I found this problem caused by the code :

AscIIClient.java at line 755:
...
if(transCoder != null)
{
InputStream in = input;
if((flag & 2) == 2)
in = new GZIPInputStream(in);
if(classLoader == null)
o = transCoder.decode(in);
else
o = ((ObjectTransCoder)transCoder).decode(in, classLoader);
}
}
...

the code can't judge 'primitiveAsString',so the error happened.I modified the code like this, everything is normal:
...
else if (transCoder != null) {
// decode object with default transcoder.
InputStream in = input;
//
byte[] buf = ((SockInputStream) in).getBuffer();

                if ((flag & F_COMPRESSED) == F_COMPRESSED)
                    in = new GZIPInputStream(in);

                if (primitiveAsString || asString) {
                    o = new String(buf, defaultEncoding);
                } else{
                    if (classLoader == null)
                      o = transCoder.decode(in);
                    else
                      o = ((ObjectTransCoder) transCoder).decode(in, classLoader);
                ...

and getMulti() and doMulti() method is the same reason.

setAliveCheck(true) flag hangs the execution.

Hi,

I am trying to use the setAliveCheck(true) flag so that when a memcacheD server goes down, the flag makes the client check for the connection and if not available then failover to the next server. During my execution I noticed that it hangs the application and the appln does nothing. I am trying to use the client with SASL support.

Thanks.

here is my code snippet:

    SchoonerSockIOPool pool = null;
    try {
                    pool = SchoonerSockIOPool.getInstance("default", AuthInfo.typical("cacheuser", "password"));
                    pool.setServers( servers );
                    pool.setWeights( weights );
                    pool.setInitConn( 10 );
                    pool.setMinConn( 5 );
                    pool.setMaxConn( 250 );
                    pool.setMaintSleep( 10 );
                    pool.setNagle( false );
                    pool.setSocketTO( readTO );
                    pool.setSocketConnectTO( 0 );
                    pool.setMaxIdle( maxIdle );

// pool.setAliveCheck(true);
pool.setFailback(true);
pool.setFailover(true);
pool.initialize();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();

Thanks.

Always return null when get data from tokyo tyrant

Hi,when I try to get data from tokyo tyrant by java memcached,it allways return null,but correct when by spymemcached,so should I implement my own decoder for tokyo tyrant ? or is there any exsited solution for java memcached ? thank you very much~~

failback is not working?

I am working on the latest release java_memcached-release_2.5.3.jar.

I have added lots of cache in Memche,
and at least 50 get call per request.

recently, my site get very slow,
and get lots of error message:
com.schooner.MemCached.SchoonerSockIOPool Tue May 03 21:18:37 PDT 2011 - ++++ failed to get SockIO obj for:

it seems it try to connect the memcached server for every get request when server DOWN.
although I have config :
pool.setFailback(false);

so how can I stop it trying to connect dead server?

another issue:

    protected final SchoonerSockIO createSocketWithAdd(String host) {

        SchoonerSockIO socket = null;
        try {
            poolCurrentConn.get(host).addAndGet(1);
            if (isTcp) {
                socket = new TCPSockIO(this, host, bufferSize, this.socketTO, this.socketConnectTO, this.nagle, true);
            } else {
                socket = new UDPSockIO(this, host, bufferSize, socketTO, true);
            }
        } catch (Exception ex) {
            log.error("++++ failed to get SockIO obj for: " + host);
            // log.error(ex.getMessage(), ex);
            socket = null;
            poolCurrentConn.get(host).decrementAndGet();
        }

        return socket;
    }

will it clean up the direct memory if connection fail?

I have got some of this exception:

Caused by: java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:632)
    at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
    at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source)
    at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(Unknown Source)
    at com.schooner.MemCached.SchoonerSockIOPool.getConnection(Unknown Source)
    at com.schooner.MemCached.SchoonerSockIOPool.getSock(Unknown Source)
    at com.schooner.MemCached.AscIIClient.get(Unknown Source)
    at com.schooner.MemCached.AscIIClient.get(Unknown Source)
    at com.schooner.MemCached.AscIIClient.get(Unknown Source)
    at com.danga.MemCached.MemCachedClient.get(Unknown Source)
    at tws.webapp.cache.MemcachedCache.get(Unknown Source)

Errors and bugs when memcached server(s) recycled.

Hi,

We are using the memcached client as part of our client which maintains dedicated connection via memcached client pool to memcached servers.

We observed that when the memcached servers are recycled, the connection pool on the client side becomes stale. The client will understand this only when its ready to write an entry to the recycled memcached server(s). The process of re-establishing/refreshing the client pool takes around 2 failed requests. The third consecutive request goes through.

First Request:

The first request fails with the following exception which is caught but swallowed in the catch(RuntimeException e) block in BinaryClient::set(byte opcode, String key, Object value, Date expiry, Integer hashCode, long casUnique, boolean asString) method.

Exception:
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:474)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:208)
at com.schooner.MemCached.SockInputStream.read(SockInputStream.java:110)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:420)
at com.schooner.MemCached.BinaryClient.add(BinaryClient.java:249)
at com.danga.MemCached.MemCachedClient.add(MemCachedClient.java:568)

Second Request:

The second request fails with exception

java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:72)
at sun.nio.ch.IOUtil.write(IOUtil.java:28)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(SchoonerSockIOPool.java:3077)
at com.schooner.MemCached.BinaryClient.set(BinaryClient.java:417)
at com.schooner.MemCached.BinaryClient.add(BinaryClient.java:249)
at com.danga.MemCached.MemCachedClient.add(MemCachedClient.java:568)

We tried to set the errorhandler but were not able to receive the callback.

Third Request: Success.

The current implementation leaves at least two failed requests per memcached server, when the sevrers are recycled.

Is it possible to:

  1. modify the code and throw the RuntimeException and IOException out so that we can catch it in our code and mitigate it ?
  2. Add an api through which we can check if a particular server is up or not ?

Thank you for your help.

Regards.
Bhargav.

OutOfMemory stack trace

In our production system, we have seen the following stack trace recently. We running the latest Memcached-Java-Client binary version 2.5.1.

2010-07-02 18:43:03 [TP-Processor61] ERROR c.s.j.s.c.ContainerResponse - The exception contained within MappableContainerException could not be mapped to a response, re-throwing to the HTTP container
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633) ~[na:1.6.0_18]
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:95) ~[na:1.6.0_18]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288) ~[na:1.6.0_18]
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.createSocketWithAdd(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.getConnection(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.SchoonerSockIOPool.getSock(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.schooner.MemCached.BinaryClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]
at com.danga.MemCached.MemCachedClient.get(Unknown Source) ~[memcached-client-2.5.1.jar:na]

Catch Exception would swallow the interrupted exception

Hi, Everyone:
I took a look at the source code of Memcached-Java-Client 2.6.1 last night, and found it catches Exception in many places. I think which would probably cause some bugs when the thread is interrupted. a typical scene is:
The client application needs a timeout for calling each memcached get, so it will call the memcached 'get' in a separated child thread and wait it completed in main thread, if it's timed out, then the main thread needs to ignore the cache data(treat it as not hit cache) and terminate the child thread.
The current implementation in Memcached-Java-Client always just catches Exception but doesn't throw out interrupted exception again or restore the interrupted status of the thread when the caught exception is an interrupted exception. which would cause the child thread can not be terminated correctly and would cause system slow when there're lots of slow 'get' need to be terminated.
We are using Memcached-Java-Client 2.5.1 in a very big project which has more than 200 concurrent 'get' and 'set' very often, with more than 500K data for most of the access, there're performance issues for more than half a year. we have made some small changes on source code to fix some small bugs, but can not resolve the performance issues, and we are planning to use other clients recently. last night I was excited to see 2.6.1 was released a few days ago, hopefully it would resolve the performance issues we have.

memcached-client-java_3.0.1 exception

Hi, recently, when using memcached-client-java-3.01, I encountered some exceptions as follows:
Exception one:
31065671 [http-10.108.72.170-8080-exec-7] ERROR com.whalin.MemCached.MemCachedClient - ++++ exception thrown while trying to get object from cache for key: _22_channel_id=800000_query=|婴|乐|岛_sort=1_state=0_0
31065671 [http-10.108.72.170-8080-exec-7] ERROR com.whalin.MemCached.MemCachedClient - For input string: "command"
java.lang.NumberFormatException: For input string: "command"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:449)
at java.lang.Integer.parseInt(Integer.java:499)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.whalin.MemCached.MemCachedClient.get(Unknown Source)

Exception two:
22153145 [http-10.108.72.170-8080-exec-6] ERROR com.whalin.MemCached.MemCachedClient - Broken pipe
java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcher.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:69)
at sun.nio.ch.IOUtil.write(IOUtil.java:26)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
at com.schooner.MemCached.SchoonerSockIOPool$TCPSockIO.flush(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.schooner.MemCached.AscIIClient.get(Unknown Source)
at com.whalin.MemCached.MemCachedClient.get(Unknown Source)

Exception three:
22152935 [ async update multicache] ERROR com.whalin.MemCached.MemCachedClient - ++++ exception thrown while writing bytes to server on set
22152935 [ async update multicache] ERROR com.whalin.MemCached.MemCachedClient - null
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:472)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:219)
at com.schooner.MemCached.SockInputStream.read(Unknown Source)
at com.schooner.MemCached.SockInputStream.getLine(Unknown Source)
at com.schooner.MemCached.AscIIClient.set(Unknown Source)
at com.schooner.MemCached.AscIIClient.set(Unknown Source)
at com.whalin.MemCached.MemCachedClient.set(Unknown Source)

When using version 2.5.3, all the best! Now don't know the reason, please help me!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.