vert-x / mod-redis Goto Github PK

View Code? Open in Web Editor NEW

39.0 39.0 25.0 741 KB

Vert.x 2.x is deprecated - use instead

Home Page: http://vertx.io/docs/vertx-redis-client/java/

License: Other

Groovy 35.90% Java 52.22% JavaScript 11.88%

mod-redis's People

Stargazers

Watchers

mod-redis's Issues

OutOfBoundsException when calling a method without parameters on the client

Calling methods without parameters (e.g. ping, info) causes and exception. This probably is not a big problem because you'd usually provide a Handler.

client.info();

causes

java.lang.ArrayIndexOutOfBoundsException: -1
    at io.vertx.java.redis.AbstractRedisClient.send(AbstractRedisClient.java:137)
    at io.vertx.java.redis.RedisClient.info(RedisClient.java:801)

hgetall fails on key with large number of fields

Hi,

I've run into an issue with mod-redis 1.1.4 where if I attempt to do a hgetall on a key with a large number of fields it fails with various exceptions. I initially saw the issue in some code with only about 15 fields, but have written this test code that demonstrates the issue:

final String testKey = "testKey";
JsonObject command = new JsonObject();
command.putString("command", "hmset");

JsonArray args = new JsonArray();
args.add(testKey);
for (int i=0; i<100; i++) {
    args.add("field"+i);
    args.add("1234567890");
}
command.putArray("args", args);

vertx.eventBus().send("my.redis", command, new Handler<Message<JsonObject>>() {
    @Override
    public void handle(Message<JsonObject> result) {
        //key should be stored

        //now hgetall the key
        JsonObject command = new JsonObject();
        JsonArray args = new JsonArray();
        args.add(testKey);
        command.putString("command", "hgetall");
        command.putArray("args", args);

        vertx.eventBus().send("my.redis", command, new Handler<Message<JsonObject>>() {
            @Override
            public void handle(Message<JsonObject> result) {
                //never get here - lots of exceptions output instead!!!
            }
        });
    }
});

The exact number of fields that it fails at seems to vary. The exceptions generated are along the lines of:

SEVERE: Exception in Java verticle
java.lang.IndexOutOfBoundsException: index: 2049 (expected: range(0, 2048))
    at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1118)
    at io.netty.buffer.AbstractByteBuf.getByte(AbstractByteBuf.java:331)
    at io.netty.buffer.WrappedByteBuf.getByte(WrappedByteBuf.java:205)
    at org.vertx.java.core.buffer.Buffer.getByte(Buffer.java:115)
    at io.vertx.redis.ReplyParser.parseResult(ReplyParser.java:100)
    at io.vertx.redis.ReplyParser.handle(ReplyParser.java:141)
    at io.vertx.redis.ReplyParser.handle(ReplyParser.java:7)
    at org.vertx.java.core.net.impl.DefaultNetSocket.handleDataReceived(DefaultNetSocket.java:237)
    at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:47)
    at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:31)
    at org.vertx.java.core.net.impl.VertxHandler.channelRead(VertxHandler.java:155)
    at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:338)
    at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:324)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:485)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
    at java.lang.Thread.run(Thread.java:722)

followed by several along the lines of:

SEVERE: Exception in Java verticle
java.lang.RuntimeException: Received a non pub/sub message without reply handler waiting:field69
    at io.vertx.redis.RedisConnection.handleReply(RedisConnection.java:262)
    at io.vertx.redis.ReplyParser.handle(ReplyParser.java:147)
    at io.vertx.redis.ReplyParser.handle(ReplyParser.java:7)
    at org.vertx.java.core.net.impl.DefaultNetSocket.handleDataReceived(DefaultNetSocket.java:237)
    at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:47)
    at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:31)
    at org.vertx.java.core.net.impl.VertxHandler.channelRead(VertxHandler.java:155)
    at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:338)
    at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:324)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:126)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:485)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
    at java.lang.Thread.run(Thread.java:722)

I've tried re-writing the code using RedisClient and experience the same issue.

Looks to me like the response from Redis is being broken up into several messages and not handled correctly (just a guess).

Any suggestions would be greatly appreciated as without this working I'm not going to be able to use mod-redis.

Many Thanks

Implement transaction support in the client lib

Socket client error, broken pipe

Hello.
I have experienced the following (after weeks of running fine). Can you explain what is causing this and the workaround for it, or fix the issue, please?
Thank you.
-Uppsax

SEVERE [io.vertx~~mod-redis~~1.1.4-SNAPSHOT-io.vertx.redis.RedisMod-139461122] Socket client error
java.io.IOException: Broken pipe
..
at io.vertx.redis.RedisConnection.send(RedisConnection.java:217)
at io.vertx.redis.RedisMod.handle(RedisMod.java:176)
at io.vertx.redis.RedisMod.handle(RedisMod.java:14)

..

What is expected blocking behavior with blpop?

I'm having trouble with blpop blocking redis access in other verticles in the same Vertx instance and am wondering whether this is not supported or is as expected?

The exact scenario is that I have a verticle that deploys several other verticles including a worker verticle. The worker verticle is reading a queue using blpop. Whilst it is blocked all other calls to redis in other verticles timeout. This is in mod redis 1.1.4 & vertx 2.1.4.

I've tried a few other code combinations but so far have not found a way around it. If this is not expected behavior I can see if I can produce a simple test case. I can work around the issue by using lpop although this is not ideal and so any advice would be greatly appreciated.

connection pooling

Hi Paulo,

I'm a Jedis user and I'd like to move to mod-redis. One of the features I like about Jedis is that I can keep a pool of pre-opened connections so I can send multiple commands in a single connection without the overhead of having one TCP handshake per command.

So I have two questions:

Will I be able to reuse a connection using the current mod-redis implementation?
Will I be able to rely on a pool of asynchronous connections? And if not, do you think it will be possible in the future to have something like that implemented natively in Vert.x?

Thanks!

Runnable module io.vertx~mod-redis-client~1.1.3 mod.json must contain a "main" field

Hi,

I'm trying yo use mod-redis in a Java vert.x project, and I have this error when I'm trying to load the module : Runnable module io.vertx~mod-redis-client~1.1.3 mod.json must contain a "main" field

If I try to load the module as done in the readme :

container.deployModule("io.redis.mod-redis", config, 1)

I have an error too :
Invalid module identifier: io.redis.mod-redis. Should be of form owner~name~version

Here is my code :

public class PingVerticle extends Verticle {

 public void start() {        

        final String address = "vertx.mod-redis-io";
        JsonObject config = new JsonObject();

        config.putString("address", address);
        config.putString("host", "localhost");
        config.putNumber("port", 6379);

        container.deployModule("io.vertx~mod-redis-client~1.1.3", config, 1);       

        RouteMatcher matcher = new RouteMatcher();

        matcher.get("/:keyword", new Handler<HttpServerRequest>() {
            public void handle(final HttpServerRequest request) {                

                JsonObject redisParams = new JsonObject();
                redisParams.putString("command", "get");
                redisParams.putString("args", request.params().get("keyword"));
                vertx.eventBus().send(address, redisParams, new Handler<Message<JsonObject>>() {  
                  @Override
                  public void handle(Message<JsonObject> reply) {
                    request.response().putHeader("content-type", "text/plain");
                    request.response().end(reply.body().encodePrettily()); 
                  }
                });
            }       
        });       

    }

What did I do wrong?
I'm a vert.x beginner , sorry if my question is stupid :) :)

Thanks,

Loïc

reduce the amount of compile warnings

gradle build not working on master branch

I got the master branch and tried to build the module locally. But I got the following error and could not figure out why. Please suggest. Thanks.

$ gradle compile

FAILURE: Build failed with an exception.

* Where:
Script '/Users/stuart/dev/mod-redis/gradle/maven.gradle' line: 67

* What went wrong:
A problem occurred evaluating script.
> No signature of method: org.gradle.api.publication.maven.internal.ant.DefaultGroovyMavenDeployer.configurePom() is applicable for argument types: (org.gradle.api.publication.maven.internal.DefaultMavenPom) values: [org.gradle.api.publication.maven.internal.DefaultMavenPom@1e79c125]

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 4.583 secs

clojure error: mod.json must contain a "main" field

Hi..sorry but I'm trying use mod-redis inside a clojure verticle and I'm getting this error

Unexpected exception in Clojure verticle 
org.vertx.java.platform.PlatformManagerException: Runnable module io.vertx~mod-redis-client~1.1.4 mod.json must contain a "main" field
    at org.vertx.java.platform.impl.DefaultPlatformManager.deployModuleFromModJson(DefaultPlatformManager.java:1032)

my code is pretty simple and I'm using vertx directly for running it...

(ns redisvertx.testbus2
  (:require [vertx.core :as vertx]
            [vertx.eventbus :as eb]))

(vertx/deploy-module "io.vertx~mod-redis-client~1.1.4")
(def address "io.vertx.mod-redis")


(vertx/periodic 5000
                (eb/send address {:command "set" :args ["user1:1212" "jhon@google"]}
                         (fn [reply] (println "respuesta> " reply)))

also I try pass arguments to deply-module as the json file and verticl number..but it didnt make any difference ..thanks!!!.....

Mulitple Redis connection from a single verticle

Hi,

I am trying to connect to two different redis instances.

I deployed two vertx redis modules with different addresses and different configuration names contains these two redis servers.

It's succesfull intially but with in 5 mins, connection to redis seems dropped and my verticle was not able to query redis....

Any help on this to find out wrong happening...!!

Thanks,,,,

Unstable on high load with complex results

Hi Paulo,
first of all thank you for this great piece of code. We plan to use mod-redis on production, unfortunately we observed an issue with v1.1.2 while sending multiple read queries to redis in very short time. IMO it happens only with commands like "lrange", "smembers", where the result is quite complex and may contain many elements.

The stacktrace of the exception is as follows:

Exception in Java verticle
java.lang.ArrayIndexOutOfBoundsException: 128
        at io.netty.buffer.UnpooledHeapByteBuf._getByte(UnpooledHeapByteBuf.java:291)
        at io.netty.buffer.UnpooledHeapByteBuf.getByte(UnpooledHeapByteBuf.java:286)
        at io.netty.buffer.UnreleasableByteBuf.getByte(UnreleasableByteBuf.java:220)
        at org.vertx.java.core.buffer.Buffer.getByte(Buffer.java:115)
        at io.vertx.redis.reply.ReplyParser.packetEndOffset(ReplyParser.java:215)
        at io.vertx.redis.reply.ReplyParser.parsePacketSize(ReplyParser.java:195)
        at io.vertx.redis.reply.ReplyParser.parseResult(ReplyParser.java:68)
        at io.vertx.redis.reply.ReplyParser.parseResult(ReplyParser.java:111)
        at io.vertx.redis.reply.ReplyParser.handle(ReplyParser.java:147)
        at io.vertx.redis.reply.ReplyParser.handle(ReplyParser.java:7)
        at org.vertx.java.core.net.impl.DefaultNetSocket.handleDataReceived(DefaultNetSocket.java:200)
        at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:47)
        at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:31)
        at org.vertx.java.core.net.impl.VertxHandler.channelRead(VertxHandler.java:156)
        at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337)
        at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785) 
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:101)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:485)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
        at java.lang.Thread.run(Thread.java:744)

and then

Exception in Java verticle
java.lang.RuntimeException: Received a non pub/sub message without reply handler waiting:io.vertx.redis.reply.BulkReply@28c39e18
        at io.vertx.redis.RedisConnection.handleReply(RedisConnection.java:255)
        at io.vertx.redis.reply.ReplyParser.handle(ReplyParser.java:153)
        at io.vertx.redis.reply.ReplyParser.handle(ReplyParser.java:7)
        at org.vertx.java.core.net.impl.DefaultNetSocket.handleDataReceived(DefaultNetSocket.java:200)
        at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:47)
        at org.vertx.java.core.net.impl.VertxNetHandler.channelRead(VertxNetHandler.java:31)
        at org.vertx.java.core.net.impl.VertxHandler.channelRead(VertxHandler.java:156)
        at io.netty.channel.DefaultChannelHandlerContext.invokeChannelRead(DefaultChannelHandlerContext.java:337)
        at io.netty.channel.DefaultChannelHandlerContext.fireChannelRead(DefaultChannelHandlerContext.java:323)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:785)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:101)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:485)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346)
        at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
        at java.lang.Thread.run(Thread.java:744)

I've created a very simple Verticle (in Groovy) that may reproduce the error: https://github.com/dankraw/redis-mod-issue
If you'd like to try it, just clone the repo and put some sample data to redis that the Verticle will ask for:

echo "rpush list0 0 10 12 5 3 4 8 9 13 1 6 4" | src/redis-cli
echo "sadd set0 joe mark drew paul wario" | src/redis-cli
echo "sadd set1 mark drew wario marie" | src/redis-cli
echo "set string0 15" | src/redis-cli
echo "set string1 7" | src/redis-cli

then run the Verticle

./gradlew runMod

It will start a HTTP server at localhost:8080, just hit F5 fast multiple times and an exception should be fired after few seconds.

Regards,
Daniel

timeout

When the connection to remote redis is lost, the module seems to wait for a minute.
Is there any way to make the time-out a lot smaller?
Thank you.
uppsax

implement a client API to hide the complexity of json commands

setex command support

Hi i want to use setex command in vertx with your module can you please let me if it supports and if yes than how can i call?

Extraction of handlers based on patterns

@purplefox @pmlopes

Hi,

while I was looking for redis module for vert.x, I found this one and I am definitely going to try it.

I was looking into the code and the concept of the handlers which are registered against pattern-like addresses (as I understand it) is very sound to me and I would like to use this feature in different context from this module.

Basically I am more or less forced to use vert.x 2.x stream and I am looking for a way how to register a handler which accepts messages which are sent to address of some pattern and as far as I know there is not such functionality in vert.x 2.x (not sure about vert.x 3).

Do you think it would be possible to reuse your code or be inspired by it and code just API which envelops this functionality and can be reused as standard Maven / Gradle artifact?

Is there any existing way how to register pattern-like handler in vert.x 2.x at this moment?

Thanks

Binary support using BSON

Implement binary support using mod-bson

Getting rid of Groovy dependency

@pmlopes @purplefox

Hi,

I want to use this module in my project which will be delivered as a fat jar hence, as I understand it right, once this module is added into mod.json and once fat jar is about to be created, fat jar will contain also all Groovy dependencies and it makes that jar unnecessarily bloated.

Is there any motivation to strip down all Groovy features (removing all code and marks of Groovy) hence I can just bundle pure Java module into the fat jar?

I am ok with putting it into the branch (not master) and it can be released separately.

If there is not any will to do so, I think you will be forked and I just do it on my own anyway but I do not want to split the effort.

Thoughts?

Does vert-x3/vertx-redis-client replace mod-redis?

Looks like mod-redis is not being actively worked on. So going forward, we should all use the new redis client built in vertx 3? Please confirm. Thanks.

db paramater name missmatch in module and client

With the latest 1.1.2 release you can specify a db to be selected when a connection is established. The parameter the module expect is called "select" but the client set a "db" parameter (deployModule).

On the documentation of the module there is also a mismatch between db and select.

Implement transaction support in the driver lib

Add pending sends to queue instead of use a timer

Looking at the code:

https://github.com/vert-x/mod-redis/blob/master/src/main/java/io/vertx/redis/RedisClientBase.java#L227

Might be better and more efficient to add any sends that come in while the module is connecting to a queue, and iterate through the queue and send them one it connects.

SEVERE: ERR Client sent AUTH, but no password is set

I have configured redis server to use AUTH, and tested AUTH command working in the redis-cli. However, when I set the AUTH, it won't pass auth. I'm using Redis 2.8.13 (00000000/0) 64 bit on localhost:6379:

<vertx.version>2.1.1</vertx.version>
<mod-redis.version>1.1.4</mod-redis.version>

        JsonObject redisConfig = new JsonObject();
        redisConfig.putString(Constants.REDIS_VERTX_ADDRESS, config.getRedisVertxAddressSubscribe());
        redisConfig.putString(Constants.REDIS_HOST, config.getRedisHost());
        redisConfig.putNumber(Constants.REDIS_PORT, config.getRedisPort());
        redisConfig.putString(Constants.REDIS_AUTH, config.getRedisAuthPassword());
        container.deployModule(redisModuleName, redisConfig, 1);

I got the following error when connecting to the redis server:

SEVERE: ERR Client sent AUTH, but no password is set

Could you please share if I have done something incorrectly? By reading the implementation, it looks like the auth is supported. What else I might have missed?

Thanks.

Best approach for multiple access to redis

I need to use 1st redis result to decide if to access redis another data or to return error, and sometimes it requires multiple accesses. What are the best approaches to complete this? Should mod-redis provide synchronous approach to access data?

vert-x / mod-redis Goto Github PK

mod-redis's People

Stargazers

Watchers

Forkers

mod-redis's Issues

..

Recommend Projects

Recommend Topics

Recommend Org

Jobs