Catch Exception would swallow the interrupted exception about memcached-java-client

[mengli]

Hi dracoyang,

I have some question about your report.

1. 200 concurrent 'get' and 'set' means 200 TPS? If so, it is not a havery load. If you refer to the wiki https://github.com/gwhalin/Memcached-Java-Client/wiki/PERFORMANCE you will find that our client can handle it easily.
   So why you make sure that the performance bottleneck is memcached client? Did you do some performace tuning to your application?

2. Our client is synchronized, so it will return immediately even though it encounts some exception. In other words, it will not hang up the child thread.

If I misunderstand your report, couid give me some more information, so that I can figure out a solution to help you.

ps. Are you work for fengxing software? Could you tell me in which city this company located? I am interesting in it.

Thank you.

Meng Lee

[dracoyang]

Hi, mengli:
For your questions, below are my answers:

1) 200 concurrent 'get' and 'set' means 200 TPS? If so, it is not a havery load. If you refer to the wiki https://github.com/gwhalin/Memcached-Java-Client/wiki/PERFORMANCE you will find that our client can handle it easily.
So why you make sure that the performance bottleneck is memcached client? Did you do some performace tuning to your application?
I think our load is more than 200TPS, we use a thread for each memcached 'get' and 'set', that means we have more than 200 threads(we're using thread pool) to do this, from our monitor page we see most of these threads are busy at most of time. If a memcached 'get' or 'set' can be done within 100ms in average, then the load will be reach to 2000TPS or more.
we have done performance tuning to our application for more than half a year, we have resolved some bottlenecks such as synchronized lock, infinite loop, dead lock, etc, the clue we think there's a bottleneck in memcached client is: from the thread dump we found many of these memcached 'get'/'set' threads were blocked at 'socket.connect()' of SocketChannel when load is high, we are not sure why here is a bottleneck, we can quickly telnet to any of the memcached server from the server the application runs on, so the network is no problem, but it does not work well in our application. from the source code of 2.5.1, we know it will always create a new connection to memcached server if the connection pool is out of use, if many requests come at the same time, it will be a problem since there's not good synchronization. in new version 2.6.1 it uses apache commons pool to manage the socket connections, I think it will probably resolve this issue, I'm doing more tests for this. Mengli, any suggestion?
2) Our client is synchronized, so it will return immediately even though it encounts some exception. In other words, it will not hang up the child thread.
It looks like the current implementation does will not hang up the child thread, but the best practice is just catching concrete exception or need to process the interrupted exception, here is a reference, hope it can be helpful to explain my suggestion, http://www.ibm.com/developerworks/java/library/j-jtp05236/index.html

ps. Are you work for fengxing software? Could you tell me in which city this company located? I am interesting in it.
Yes, our company is located in San Francisco, and we have branch offices in Changsha, Chengdu, Inner Mongolia. :)

Thanks a lot for your reply!

Draco Yang

[mengli]

HI dracoyang,

Our client is thread safe, so why not just get rid of the thread pool for memcached client? I think it will improve the performance obviously.

Hey, my hometown is Changsha. Right now, you work in Changsha? I am hunting a job in Changsha, so I just want to know whether there is any technical position in Changsha branch office.

Meng Li

[dracoyang]

We need to give up waiting for the cache 'get' if it's timed out(e.g. 500ms or 1 sec), if get rid of the thread pool, do you have better idea to implement this?
Yes, we are looking for new developers, I think there should be technical positions fit for you, I can give you a call for the details, please send your cell number to my email [email protected] when you feel free.

[mengli]

Hi dracoyang,

It's my pleasure to help you in tuning the performance of memcahced client.

To my understanding, your memcached server is deployed in WAN. So the network latency is very high and you need to set a timeout for get operation. Am I right?
If so, can you deploy your memcached server in LAN?

If your memcached server is already in LAN, and the latency is low, I suggest you call our client directly in application. There is no need to set a timeout for each get. Asynchronized usage of our client will obviously drop the performance of get operation.

Meng Lee

[newroot]

Shall I close this issue?

Anyway, the exception handling mechanism is not perfect in this client.

[mengli]

Let's close this issue at this moment.