stackexchange / stackexchange.redis Goto Github PK

at System.Net.EndPoint.Serialize () [0x00000] in /private/tmp/source/bockbuild-mono-3.2.6/profiles/mono-mac-xamarin/build-root/mono-3.2.6/mcs/class/System/System.Net/EndPoint.cs:51
  at System.Net.Sockets.Socket.BeginConnect (System.Net.EndPoint end_point, System.AsyncCallback callback, System.Object state) [0x000fa] in /private/tmp/source/bockbuild-mono-3.2.6/profiles/mono-mac-xamarin/build-root/mono-3.2.6/mcs/class/System/System.Net.Sockets/Socket_2_1.cs:1499 
  at StackExchange.Redis.SocketManager.BeginConnect (System.Net.EndPoint endpoint, ISocketCallback callback) [0x00000] in :0 
  at StackExchange.Redis.PhysicalConnection..ctor (StackExchange.Redis.PhysicalBridge bridge) [0x00000] in :0 
  at StackExchange.Redis.PhysicalBridge.GetConnection () [0x00000] in :0

Got this when calling var multiplexer = ConnectionMultiplexer.Connect("localhost");

mono --version
Mono JIT compiler version 3.2.6 ((no/9b58377 Thu Jan 16 17:49:56 EST 2014)

Is this tested under Mono and therefore something is wrong on my end, or does it need a patch to work? :)

Hey!

As an extension of a StackOverflow Q&A I've published today (http://stackoverflow.com/questions/23180765/redis-keyspace-notifications-with-stackexchange-redis), I'm adding here the issue in order to request keyspace notifications.

Thanks in advance, either if it gets implemented or not 👍

EDIT

According to the StackOverflow Q&A linked above, we arrived to the conclusion that keyspace notifications are just pubsub channels, and it would be nice that StackExchange.Redis would implement helpers, since keyspace event channel names would require to work with strings (not that elegant).

These helpers would look something like:

    IDatabase.KeySpaceEvents.OnSet("users:matias:messages", (channel, value) => { });

Another good point would be specifying which command to listen:

    IDatabase.KeySpaceEvents.OnSet("users:matias:messages", (channel, value) => { }).When(RedisCommand.SAdd);

It's just a suggestion ;)

Hello
We have 2 node redis server setup where 1 node is master and the other is a slave. We have observed that StackExchange.Redis can take anything from few seconds to several minutes to reconnect to the server if master/slave failover happens in the background. i.e. if we keep on using the same ConnectionMultiplexer object as the one created before the failover was triggered we get keep on getting a connection exception. The connectionMultiplexer object takes atleast 1 minute(sometimes more) to recover from this condition. Is this expected? If we create a new ConnectionMultiplexer object during this time its able to connect to the redis server successfully
Thanks

I'm using StackExchange.Redis version 1.0.247.

I'm running a test to kill redis and then restart it to make sure the ConnectionMultiplexer reconnects. The test works, but there are hundreds of internal errors events being raised.

Here's the stack trace:

InternalError with args: System.NullReferenceException: Object reference not set to an instance of an object.
at StackExchange.Redis.PhysicalConnection.Flush() in c:\TeamCity\buildAgent\work\18a91a3757cef937\StackExchange.Redis\StackExchange\Redis\PhysicalConnection.cs:line 140
at StackExchange.Redis.PhysicalBridge.WriteQueue(Int32 maxWork) in c:\TeamCity\buildAgent\work\18a91a3757cef937\StackExchange.Redis\StackExchange\Redis\PhysicalBridge.cs:line 543

Here's my test code:

[TestClass]
[DeploymentItem("redis-server.exe")]
[DeploymentItem("redis.conf")]
[DeploymentItem("APIBridge.dll")]
[DeploymentItem("redis-startup-firewall.cmd")]
public class ConnectivityTest {

    private const string RedisAddr = "127.0.0.1";
    private const int RedisPort = 6379;

    /// <summary>
    /// Tests:
    /// 1) Connecting to an instance of Redis that is not yet running.  
    /// 2) Reconnecting after a Redis instance goes down.
    /// </summary>
    [TestMethod]
    public void RedisConnectivityTest() {

        // setup firewall and kill any running redii
        TestUtils.RunCmdSilently("redis-startup-firewall.cmd");
        RedisManagementUtils.KillRedis();

        // create a connection before redis is running
        var config = new ConfigurationOptions {
            EndPoints = { { RedisAddr, RedisPort } },
            KeepAlive = 5,
            SyncTimeout = 1000,
            AllowAdmin = true,
            AbortOnConnectFail = false
        };
        ConnectionMultiplexer conn = ConnectionMultiplexer.Connect(config, Console.Out);

        // log events
        conn.ConfigurationChanged += (sender, args) => Console.WriteLine("ConfigurationChanged with endpoint: {0}", args.EndPoint);
        conn.ConfigurationChangedBroadcast += (sender, args) => Console.WriteLine("ConfigurationChangedBroadcast with endpoint: {0}", args.EndPoint);
        conn.ConnectionFailed += (sender, args) => Console.WriteLine("ConnectionFailed with args: {0}", args.Exception.Message);
        conn.ConnectionRestored += (sender, args) => Console.WriteLine("ConnectionRestored with args: {0}", args.EndPoint);
        conn.ErrorMessage += (sender, args) => Console.WriteLine("ErrorMessage with args: {0}", args.Message);
        conn.HashSlotMoved += (sender, args) => Console.WriteLine("HashSlotMoved with args: {0}", args.NewEndPoint);
        conn.InternalError += (sender, args) => Console.WriteLine("InternalError with args: {0}", args.Exception);

        // start redis
        const string redisExe = "redis-server.exe";
        const string redisConf = "redis.conf";
        var redisWork = Directory.GetCurrentDirectory();
        RedisManagementUtils.StartRedis(RedisPort, redisExe, redisConf, redisWork);

        // ping
        Thread.Sleep(5000);
        TestUtils.RunUntilSuccessOrTimeout(() => Assert.IsTrue(conn.IsConnected));

        // kill redis
        RedisManagementUtils.KillRedis();

        // check for event
        TestUtils.RunUntilSuccessOrTimeout(() => Assert.IsFalse(conn.IsConnected));

        // start redis
        RedisManagementUtils.StartRedis(RedisPort, redisExe, redisConf, redisWork);
        Thread.Sleep(5000);
        TestUtils.RunUntilSuccessOrTimeout(() => Assert.IsTrue(conn.IsConnected));
    }
}

If I do not set resolveDns=true, then I will sometimes get a AccessViolationException (System.Net.Dns.ResolveCallback ... System.Net.Dns.TryGetAddrInfo). This seems to be a thread-safety issue, since it doesn't always happen.

But if I do set resolveDns=true, then I don't get this exception.

ConnectionMultiplexer line 1086 seems to be where the the initial decision is being made - if thread safety cannot be guaranteed, then I think if configuration.HasDnsEndPoints() and !configuration.ResolveDns an exception should be thrown.

http://redis.io/commands/scan

why ?

I’m trying to find how to use StackExchange.Redis to lock a specified Key to prevent concurrency problem for multiple apps/clients
e.g. what I need to do:

1. Get value for the a specified Key
2. Set increased value back to Redis

Sometimes I have deadlocks related to async methods.
I think it happens because sometimes async methods finish in "reading" thread.
Here is example:

var task1 = _db.SetAddAsync(key1, val1);
var task2 = _db.SetAddAsync(key2, val2);

await Task.WhenAll(new[]{task1, task2}).ConfigureAwait(false);

//if you check current thread name sometimes it has name "computername:Read".
Console.WriteLine(System.Threading.Thread.CurrentThread.Name);

//if this happens the next code will newer finish.
// actually it could be any async call
await _db.SetAddAsync(key2, val3).ConfigureAwait(false);
// if you replace it with sync method the timeout will happen

I use redis to save about 1 milions keys, I want to get all keys from page 1 to page 10, then page 11 to page 20 ( pagesize = 1000).
Why StackExchange.Redis not support scan command ???
http://redis.io/commands/scan

Hi Marc,

Version 1.0.289.0 (nuget, .net 4.0).

When I leave my workstation alone for five minutes, after calling Connect() I get a RedisConnectionException the first time. Immediately retrying does work with no delay at a 90% success rate. No info is written to the supplied TextWriter.

var oCli = ConnectionMultiplexer.Connect("srv-flux-01:14144", log);

Complete exception message:

An unhandled exception of type 'StackExchange.Redis.RedisConnectionException' occurred in StackExchange.Redis.dll
Additional information: It was not possible to connect to the redis server(s); to create a disconnected multiplexer, disable AbortOnConnectFail

I connect from Win7 Pro X64 to Redis 2.8.9 (x64) on a Debian machine on our LAN (same subnet). No hardware firewalls in between, only Windows firewall.
Redis connections from redis-py or RedisDesktopManager(which uses hiredis) never fail, and always connect within milliseconds.

I use a simple Windows Forms project, VS2013 Express, for testing.
I use (and need) the .NET Framework 4 Client Profile.

Any ideas?

Kind regards, TW

Hi,

I was not able to locate the method to flush the cache across all databases (FLUSHALL)?

Can someone help?

Thanks.

var connection = ConnectionMultiplexer.Connect("redacted.redis.cache.windows.net,ssl=true,password=redacted");

---- x86 target build
---- looks like library targets wrong .net version

System.InvalidProgramException was unhandled
HResult=-2146233030
Message=JIT Compiler encountered an internal limitation.
Source=StackExchange.Redis
StackTrace:
at StackExchange.Redis.ConfigurationOptions.OptionKeys.TryNormalize(String value)
at StackExchange.Redis.ConfigurationOptions.DoParse(String configuration, Boolean ignoreUnknown) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConfigurationOptions.cs:line 465
at StackExchange.Redis.ConfigurationOptions.Parse(String configuration) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConfigurationOptions.cs:line 258
at StackExchange.Redis.ConnectionMultiplexer.CreateMultiplexer(Object configuration) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 705
at StackExchange.Redis.ConnectionMultiplexer.Connect(String configuration, TextWriter log) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 725
at ConsoleApplication22.Program.Main(String[] args) in c:\Users\Admin\Documents\Visual Studio 2013\Projects\ConsoleApplication22\ConsoleApplication22\Program.cs:line 14
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:

--- x64 target build
--- looks like DoParse.TryNormalize returns null

System.ArgumentException was unhandled
HResult=-2147024809
Message=Keyword 'ssl' is not supported
Source=StackExchange.Redis
StackTrace:
at StackExchange.Redis.ConfigurationOptions.OptionKeys.Unknown(String key) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConfigurationOptions.cs:line 67
at StackExchange.Redis.ConfigurationOptions.DoParse(String configuration, Boolean ignoreUnknown) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConfigurationOptions.cs:line 533
at StackExchange.Redis.ConfigurationOptions.Parse(String configuration) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConfigurationOptions.cs:line 259
at StackExchange.Redis.ConnectionMultiplexer.CreateMultiplexer(Object configuration) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 705
at StackExchange.Redis.ConnectionMultiplexer.Connect(String configuration, TextWriter log) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 725
at ConsoleApplication22.Program.Main(String[] args) in c:\Users\Admin\Documents\Visual Studio 2013\Projects\ConsoleApplication22\ConsoleApplication22\Program.cs:line 14
at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ThreadHelper.ThreadStart()
InnerException:

We use TwemProxy as a layer between C# and redis for load balancing between several redis instances. Unfortunately, it does not support echo or ping. When I try adding

{ "echo", null }

to the command map for my connection, the AssertAvailable call on line 798 of ConnectionMultiplexer.cs throws an error. I was wondering if it would be possible to add a toggle for whether or not to check for these commands? I know this request falls squarely into the "the community needs that weren’t internal priorities" category from Marc's blog post, but it would make the switch to StackExchange.Redis from BookSleeve easier.

Thanks again for providing such awesome libraries, BookSleeve was great as it was but the presence of synchronous calls in StackExchange.Redis made the move a no-brainer.

KEYS command which returns a set of keys matching a given pattern, likely missing.

ConnectionMultiplexer may deliver notifications about connection failure (and some other infrastructure errors). In some scenarios it might be important to process these notifications synchronously or client may blindly start working with the different Redis instance (e.g. if crashed server was automatically restarted).

Internally StackExchange tries to deliver "connection failed" synchronously by invoking ConnectionMultiplexer.TryCompleteHandler(,,,. false /* isAsync*/). However it looks like isAsync is not handled correctly inside TryCompleteHandler(). In fact "isAsync == true" does synchronous processing while isAsync == false just returns control to the caller hence triggering asynchronous event processing.

Should it be "if (NOT isAsync)" here?

        if (isAsync)
        {
            foreach (EventHandler<T> sub in handler.GetInvocationList())
            {
                try
                { sub.Invoke(sender, args); }
                catch
                { }
            }
            return true;
        }
        return false;

Mark I've converted a BookSleeve project to SE.redis - so far so good. I use Lua scripts. One minor thing I ran into was a couple of my scripts return different types depending on inputs - e.g., a script that might return an int, null or byte[]. With BookSleeve's Eval, you could use is semantics - e.g., if (ret is Int64). With RedisResult, you wrap the returned object within the internals of the class with cast operators. So the only way I can distinguish is to cast and deal with InvalidCastException (although you do have an IsNull property). Would you consider adding some sort of Is(Type) function of something along those lines?

Steps to reproduce:

1. Open connection to two servers (master and slave)
2. Send get request
3. Stop master, and quickly send get request
4. Stop slave, start master
5. Send get request again

Expected result is valid data returned, because master server is up.
But actual result is exception:

StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: GET key1
   at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 1687
   at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\RedisBase.cs:line 79
   at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:line 1286

I've debugged code a little bit, and I found out that on step 3 master endpoint sets DidNotRespond to unselectableReasons, and so this endpoint will never be used for requests. I didn't find any normal way for endpoint to remove this state.

Hi Marc,

First of all thanks for this awesome redis client.
But I have some issues :)

I have the following issues in our production environment (Web-Farm - 4 nodes, on top of it Load balancer):

1. Timeout performing HGET key, inst: 3, queue: 29, qu=0, qs=29, qc=0, wr=0/0
   at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor``1 processor, ServerEndPoint server) in ConnectionMultiplexer.cs:line 1699 This happens 3-10 times in a minute

2. No connection is available to service this operation: HGET key at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor``1 processor, ServerEndPoint server) in ConnectionMultiplexer.cs:line 1666

I tried to implement as Marc suggested (Maybe I interpreted it incorrectly) - better to have fewer connections to Redis than multiple. I made the following implementation:

public class SeRedisConnection
{
private static ConnectionMultiplexer _redis;

private static readonly object SyncLock = new object();

public static IDatabase GetDatabase()
{
    if (_redis == null || !_redis.IsConnected || !_redis.GetDatabase().IsConnected(default(RedisKey)))
    {
        lock (SyncLock)
        {
            try
            {
                var configurationOptions = new ConfigurationOptions
                {
                    AbortOnConnectFail = false
                };
                configurationOptions.EndPoints.Add(new DnsEndPoint(ConfigurationHelper.CacheServerHost,
                    ConfigurationHelper.CacheServerHostPort));

                _redis = ConnectionMultiplexer.Connect(configurationOptions);
            }
            catch (Exception ex)
            {
               IoC.Container.Resolve<IErrorLog>().Error(ex);
                return null;
            }
        }
    }
    return _redis.GetDatabase();
}

public static void Dispose()
{
    _redis.Dispose();
}

}

Actually dispose is not being used right now. Also I have some specifics of the implementation which might cause such behavior (I'm only using hashes):

1. Add, Remove hashes - async
2. Get -sync

Original question you can find here:
http://stackoverflow.com/questions/22930856/stackexchange-redis-timeout-and-no-connection-is-available-to-service-this-oper

Thanks a lot in advance!

Looking forward for your response.

BTW: Here what I can see from my redis cache server (TCP connections never goes up):

Also I have stress-testing web-farm (2 nodes - Microsoft NLB). If I run load test for 200 req/s - it handles it ok in this environment and I noticed 5-8 open TCP connections and no timeouts at all. But the issue is with Microsoft NLB if you are making stress-test from 2-3 ip addresses it allocates requests only to one node.

So in production environment case it should be 4 nodes * 8 (active connections) = 32 open connections rather than 200.

StackExchange.Redis v1.0.187 (from nuget & from repo on March 18).
Started Redis locally, ran the following piece:

using (var c = await ConnectionMultiplexer.ConnectAsync("127.0.0.1"))
{
    c.ConnectionFailed += (s, e) => Console.Write("failed");
    c.ErrorMessage += (s, e) => Console.Write("error");
    c.ConnectionRestored += (s,e) => Console.Write("restored");
    var db = c.GetDatabase();
    while (true)
    {
        try
        {
            byte[] value = await db.StringGetAsync("123");
            Console.Write(".");
        }
        catch (Exception ex)
        {
            Console.Write("!");
        }
        await Task.Delay(TimeSpan.FromSeconds(1));
    }
}

When Redis gets stopped while running the code, event handlers for ConnectionFailed and ErrorMessage are not raised. I get RedisConnectionException on StringGetAsync call. When Redis is started again, ConnectionRestored event handler is called twice (I guess there's two connections being kept around per node?).

We use StackExchange.Redis in SignalR Redis scale-out, I found after Redis restart in ConnectionRestored event sometimes ConnectionMultiplexer.IsConnected is False, we need reading Redis Key in ConnectionRestored event handler, in this case it throw RedisConnectionException.

Here is simple repro app:

       private static ConnectionMultiplexer _connection;
        static void Main(string[] args)
        {
           string connectionString = "......";
            _connection = ConnectionMultiplexer.Connect(connectionString);

            _connection.ConnectionRestored += OnConnectionRestored;

            Console.ReadLine();
        }

        private static void OnConnectionRestored(object sender, ConnectionFailedEventArgs args)
        {
            Console.WriteLine("_connection.IsConnected is " + _connection.IsConnected);
        }

Repro steps:
1). Update connectionString for your Redis, build the app and run
2). Stop and Restart the Redis, look the console output, repeat this step until in output see: _connection.IsConnected is False

Expected result:
in output it always should display "_connection.IsConnected is True" after Redis restart

Actual result:
On my machine, after I restart Redis about 3 times, I can see " _connection.IsConnected is False" in output.

reply unknown on redis 2.6

We use Redis' server-side Lua scripting a lot, but we have split the responsibilities. Clients aren't allowed to inject Lua code into Redis. Instead, Lua code is automatically injected into Redis instances by a (custom) Lua script manager, after commits / pushes to a DVCS repository. How we use this client-side: see this diagram for a gist (the Lua scripts return SHA1 checksums themselves).

Therefore we need to call EVALSHA directly, supplying only the SHA1 checksum and the parameters.

Is this possible in StackExchange.Redis? Sidenote: we have no problems using this Lua setup in redis-py (python) from various AIX/Linux servers.

Kind regards, TW

After updating to the latest NuGet package 1.0.297 in case of an incorrect authentication, I have started seeing the following exception been thrown, instead of the unabletoconnect exception "It was not possible to connect to the redis server(s)"
An unhandled exception of type 'System.IO.IOException' occurred in System.dll

Additional information: Unable to read data from the transport connection: Cannot access a disposed object.

Object name: 'System.Net.Sockets.Socket'..

Background

We run the latest stable redis on linux and our .NET stuff is on Windows Server 2012 all in windows azure, all within the same region.

How do i plan on using Redis? Well, I plan on having it cache the layer that feeds the http Response. Its not output caching because i'm not caching the actual JSON that the api is returning-- it's the layer that builds the object that will end up being returned as JSON.

Story

I'm testing out using StackExchange.Redis for caching instead of our current ObjectCache-based impl. For full disclosure I'm going to provide both here:

ObjectCache Based

using Printmee.Infra.Caching;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.Caching;
using System.Text;
using System.Threading.Tasks;

namespace Widget.Caching
{
    public class HttpCache:IBigCache
    {
        private ObjectCache _cache
        {
            get
            {
                if (__cache == null)
                {
                    __cache = System.Runtime.Caching.MemoryCache.Default;
                }
                return __cache;
            }
        }
        private ObjectCache __cache = null;
        public async Task<T> Get<T>(string key) where T : class
        {
            return _cache.Get(key) as T;
        }

        public async Task<T> Get<T>(string key, params object[] args) where T : class
        {
            return _cache.Get(string.Format(key,args)) as T;
        }

        public async Task<T> Get<T>(CacheKey key) where T : class
        {
            return _cache.Get(key.Full) as T;
        }

        public async Task<T> GetCategorized<T>(string category, string key) where T : class
        {
            return _cache.Get(category+key) as T;
        }

        public async Task Remove(string key)
        {
            _cache.Remove(key);
        }

        public async Task<T> Set<T>(string key, T obj, TimeSpan keepAlive)
        {
            _cache.Set(key, obj, new CacheItemPolicy() { AbsoluteExpiration = DateTime.UtcNow.Add(keepAlive) });
            return obj;
        }

        public async Task<T> Set<T>(CacheKey key, T obj) where T : class
        {
            _cache.Set(key.Full, obj, new CacheItemPolicy() { AbsoluteExpiration = DateTime.UtcNow.AddDays(1) });
            return obj;
        }

        public async Task<T> SetCategorized<T>(string category, string key, T obj) where T : class
        {
            _cache.Set(category+key, obj, new CacheItemPolicy() { AbsoluteExpiration = DateTime.UtcNow.AddDays(1) });
            return obj;
        }
    }
}

StackExchange.Redis

using Printmee.Infra.Caching;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using StackExchange.Redis;
using ServiceStack.Text;
using ServiceStack.Text.Json;
using NLog;

namespace Widget.Caching
{
    public class RedisCache : Widget.Caching.IBigCache 
    {
        private static Logger _Log = LogManager.GetCurrentClassLogger();
        private StackExchange.Redis.IDatabase __Redis;
        private StackExchange.Redis.IDatabase _Redis 
        { 
            get 
            {
                if (__Redis == null)
                    __Redis = _Connection.GetDatabase();
                return __Redis;
            } 
        }
        private RedisConfig _Config;
        private ConnectionMultiplexer _Connection;
        public RedisCache(ConnectionMultiplexer con, RedisConfig config)
        {
            _Connection = con;
            _Config = config;
        }

        public async Task<T> Get<T>(string key) where T: class
        {
            if (!_Config.IsOn)
                return null;
            var val= await _Redis.StringGetWithExpiryAsync(key);
            if (val.Value.IsNull)
                return null;
            return JsonSerializer.DeserializeFromString<T>(val.Value);
        }



        public async Task<T> Get<T>(string key, params object[] args) where T:class
        {
            return await Get<T>(string.Format(key, args));
        }

        public async Task<T> GetCategorized<T>(string category, string key) where T : class
        {
            if (!_Config.IsOn)
                return null;
            _Log.Debug("lookup {0}{1}", category, key);
            var val = await _Redis.HashGetAsync(category, key);
            _Log.Debug("done {0}{1}", category, key);
            if (val.IsNull)
                return null;
            _Log.Debug("ser {0}{1}", category, key);
            var final = JsonSerializer.DeserializeFromString<T>(val);
            _Log.Debug("done ser {0}{1}", category, key);
            return final;
        }

        public async Task<T> Get<T>(CacheKey key) where T:class
        {
            if (key.IsCategorized)
                return await GetCategorized<T>(key.Category, key.Key);
            return await Get<T>(key.Key);
        }

        public async Task Remove(string key)
        {
            if (!_Config.IsOn)
                return;
            await _Redis.KeyDeleteAsync(key);
        }

        public async Task<T> Set<T>(string key, T obj, TimeSpan keepAlive)
        {
            if (!_Config.IsOn)
                return obj;

            await _Redis.StringSetAsync(key, JsonSerializer.SerializeToString<T>(obj));
            await _Redis.KeyExpireAsync(key, DateTime.Now.ToUniversalTime().Add(keepAlive));

            return obj;
        }
        public async Task<T> SetCategorized<T>(string category, string key, T obj) where T : class
        {
            if (!_Config.IsOn)
                return obj;

            await _Redis.HashSetAsync(category,key, JsonSerializer.SerializeToString<T>(obj));
            return obj;
        }

        public async Task<T> Set<T>(CacheKey key, T obj) where T : class
        {
            if (key.IsCategorized)
                return await SetCategorized<T>(key.Category, key.Key,obj);
            return await Set<T>(key.Key,obj, TimeSpan.FromDays(2));
        }
    }
}

Here's an example of me using this IBigCache interface:

var key = new CacheKey()
    .SetCategory("ProductVariant-{0}", sku)
    .SetKey("BuildInfo");

var cached = await _BigCache.Get<ProductBuildInfoResponseV1>(key);
if (cached != null) 
    return cached;

// the computation
var res = await GetBuildInfo(sku);

//proxying the result through cache setting
return await _BigCache.Set<ProductBuildInfoResponseV1>(key,res);

Ok ok ok. So the thing that is bothering me is the performance change between the two. Things I know I need to account for when comparing the two:

1. Object cache isnt json serializing, obviously this will come at a cost
2. Network latency and the fact that my data isn't on the same box anymore

So I'm thinking "object cache will be faster" as I go into this. The thing that surprise me though is the cost of using Redis here--

Benchmark of ObjectCache at -c 50

Percentage of the requests served within a certain time (ms)
  50%    211
  66%    337
  75%    388
  80%    423
  90%    500
  95%    519
  98%    587
  99%    592
 100%    592 (longest request)

Benchmark of StackExchange.Redis (async) at -c 50

Percentage of the requests served within a certain time (ms)
  50%   6899
  66%   6941
  75%   7838
  80%   8381
  90%   9904
  95%  10448
  98%  12104
  99%  12607
 100%  12607 (longest request)

It's a lot higher than I expected. Am I doing this wrong?

Hi,

I'm running few threads with Redis Locking mechanism against an Azure Redis Cache Server, and once in a while I get a TimeoutException (even thou there has not been any Idle time during the past minute), not sure if this is by design and how would one avoid this?

My Current solution is to Close the Connection, and create a new one, and continue let the threads continue. (cMgr.Close; cMgr = ConnectionMultiplexer.Connect())

Mark it would be great if you'd provide a signed NuGet package.

thx

Hi, Marc.
I just started use your library and like it very much. Nice work

This is rather question than issue.

I need read all keys from DB and then get all values for this keys.
I found that i can get all keys through IServer.Keys. IServer.Keys returns IEnumerable. but IDatabase.StringGet accepts RedisKey[]. So i need copy keys from IEnumerable to array.
So question is: can IServer.Keys return array or IDatabase.StringGet accept IEnumerable?

Hi

This is not a new topic but I would like to bring it to the discussion again. I wonder if Nuget package could contain strongly named assemblies so other strong named assemblies could reference them.

I'm aware of the solution you proposed in #18. However I personally don't like to use third party tools to surge assembly potentially adding bugs. Neither I'm happy with using ildasm/ilasm to rebuild/sign dll.

It would be much easier for the package consumers to just import package without post-processing it.

Normally, the muxer.ConnectionRestored is null when connection established.

ex:
var muxer = ConnectionMultiplexer.Connect(...);
// has connected, but now muxer.ConnectionRestored doesn't be set
// then
muxer.ConnectionRestored += myConnectedFunc;

1. make redis memory full with keys having no expiry set
2. now try setting a key
   Timeout performing SET b416dacb-8466-4f2a-a527-ae55121a27cb, inst: 0, queue: 484998, qu=482315, qs=2683, qc=0, wr=1/1, in=0/0
3. Try it from cli
   actual from redis-cli
   OOM command not allowed when used memory > 'maxmemory'.

expected:
stackexchange.redis to throw exception with error message similar to redis-cli

Do you think it's wise to put server.FlushDatabase(); to the sample code in KeyScan.md without any comments? For a beginner user not aware of all Redis commands, and who searches for a sample code to get the list of the keys, it can be very deceiving (as it is not available anywhere else in the documentation).

I have a question on the SSL connection option and if it really does anything. Wouldn't defining the port that uses SSL be adequate?

When storing integers and using database.KeyType(key) it always returns a string.

Just a reminder to port the Sentinel code from Booksleeve to StackExchange.Redis.

ConfigurationOptions allows you to specify the Sentinel ServiceName, but connecting to a Sentinel doesn't work:

SENTINEL_IP:26379,serviceName=mymaster,keepAlive=5,syncTimeout=1000,allowAdmin=True,connectTimeout=5000,abortConnect=False
1 unique nodes specified
Requesting tie-break from SENTINEL_IP:26379 > __Booksleeve_TieBreak...
Allowing endpoints 00:00:05 to respond...
SENTINEL_IP:26379 faulted: ProtocolFailure on PING
SENTINEL_IP:26379 failed to nominate (Faulted)
> UnableToResolvePhysicalConnection on GET
No masters detected
SENTINEL_IP:26379: Sentinel v2.8.4, master; keep-alive: 00:00:05; int: Disconnected; sub: Connecting; not in use: DidNotRespond
SENTINEL_IP:26379: int ops=5, qu=0, qs=0, qc=0, wr=0, sync=5, socks=1; sub ops=2, qu=0, qs=0, qc=0, wr=0, sync=2, socks=2
Circular op-count snapshot; int: 0+5=5 (0.50 ops/s; spans 10s); sub: 0+2=2 (0.20 ops/s; spans 10s)
Sync timeouts: 0; fire and forget: 0; last heartbeat: -1s ago
Starting heartbeat...
ConfigurationChanged endpoint: SENTINEL_IP:26379
ConnectionFailed endpoint: SENTINEL_IP:26379, exception: ProtocolFailure on SENTINEL_IP:26379/Interactive, input-buffer: 663, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 5
s, pending: 0, state: ConnectedEstablishing, last-heartbeat: never, last-mbeat: 0s ago, global: 0s ago

Do you plan on supporting .net 4.0?

Thanks

• Is there any alternative to ServiceStack's Pipeline concept. Or there is no need, because you are pipelining all commands silently?
• I have HashItem class. Is there any function in your library like LoadHashItemFromDb (HashItem[]) ? (the method fills value fields of hashItems)

 public class HashItem
    {
        public string HashId { get; private set; }
        public string Key { get; private set; }
        public object Value { get; private set; 
   }

• Do you have any benchmark between ServiceStack.Redis and yours?
• This code takes advantage of async performance and sync simplicity ?

public long GetAid(string code)
        {
            var ret = db.StringIncrementAsync("aid:" + code, 1);
            ret.Wait();
            return ret.Result;
        } 

PS: As a result: I've started to replace ServiceStack.Redis with yours.

• Performance advantage (async api, multiplexer)
• Continous development (ex:2.8.4 support (Tcp loopback))
• Free of charge

Thanks.

When adding a the nuget package to a WebAPI project or to a project referenced by the WebAPI project the System.Net.Http assemblyBinding gets changed to version 2.0.0.0. My project uses System.Net.Http 4.0.0.0 and every time I add the nuget package to any of the referenced proejcts the System.Net.Http gets changed, asp.net web app doesn't start and I have to manually revert the assemblyBinding.

All my projects are .NET 4.5.

Does this happen to anyone else?

SET.addasync, set.remove are non-functional why?
set.add times out quite often also....

In order to prevent the scenario that the scripts cached on the client are not available on the server after the server restart, we should clear the client cache on disconnect. This issue is currently handled for sync requests but with async requests, the error can't be reliably handled, so this change needs to be made.

Hi, there.

I use StackExchange.Redis.1.0.281.0
It is turned out that lib can't restore connection to redis in some cases.

My application executes next code every few seconds

try
{
      var server = this.application.Multiplexer.GetServer(RedisServerAddress);
      var keys = server.Keys(Properties.Settings.Default.BackendRedisDBId);
...............
}
catch(Exception e)
{
  //log exception
}

In order to check how this code works when connection lost and restored, i just stopped Redis and started it again. Most of the time connection restored, but some times does not. The code above fails with exception until i restart Redis again. After restart lib restores connection.

I create ConnectionMultiplexer next way

 var config = ConfigurationOptions.Parse(RedisServerAddress);

 config.KeepAlive = 5;
 config.SyncTimeout = 1000;
 config.AbortOnConnectFail = false;
 config.AllowAdmin = true;

 var muxer = ConnectionMultiplexer.Connect(config);

 muxer.ConfigurationChanged += (sender, args) => log.DebugFormat("ConfigurationChanged with endpoint: {0}", args.EndPoint);
 muxer.ConfigurationChangedBroadcast += (sender, args) => log.DebugFormat("ConfigurationChangedBroadcast with endpoint: {0}", args.EndPoint);
 muxer.ConnectionFailed += (sender, args) => log.DebugFormat("ConnectionFailed with args: {0}", args.Exception.Message);
 muxer.ConnectionRestored += (sender, args) => log.DebugFormat("ConnectionRestored with args: {0}", args.EndPoint);
 muxer.ErrorMessage += (sender, args) => log.DebugFormat("ErrorMessage with args: {0}", args.Message);
 muxer.HashSlotMoved += (sender, args) => log.DebugFormat("HashSlotMoved with args: {0}", args.NewEndPoint);
 muxer.InternalError += (sender, args) => log.DebugFormat("InternalError with args: {0}", args.Exception);

What information yet should i provide in order to help find reason?

Example scenarios that should have a blindingly obvious message:

• server did not respond
• wrong password

1: case-insensitive keywords
2: throw a clear error if an unknown keyword is found

This would be consistent with ADO.NET, which throws exceptions like:

ArgumentException: Keyword not supported: 'foo'.

I added the SIO_LOOPBACK_FAST_PATH option in MSOpenTech Redis 2.8.4 in commit a2ef6a4fdc9b76816641c4652595e75f77c9e960. This option can significantly speed up loopback communication on Win8/Server2012+ with clients that also enable this option. This can be enabled as follows:

    internal void SetFastLoopbackOption(Socket socket)
    {
        // SIO_LOOPBACK_FAST_PATH (http://msdn.microsoft.com/en-us/library/windows/desktop/jj841212%28v=vs.85%29.aspx)
        // Speeds up localhost operations significantly. OK to apply to a socket that will not be hooked up to localhost, 
        // or will be subject to WFP filtering.
        const int SIO_LOOPBACK_FAST_PATH = (-1744830448);

        // Win8/Server2012+ only
        if (Environment.OSVersion.Version.Major > 6 || Environment.OSVersion.Version.Major == 6 && Environment.OSVersion.Version.Minor >= 2)
        {
            Byte[] optionInValue = BitConverter.GetBytes(1);
            socket.IOControl(SIO_LOOPBACK_FAST_PATH, optionInValue, null);
        }
    }

    internal SocketToken BeginConnect(EndPoint endpoint, ISocketCallback callback)
    {
        var socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
        SetFastLoopbackOption(socket);

We use StackExchange.Redis in SignalR Redis scale-out, I found that ConnectionRestored event always raised two times after Redis restart.

Here is simple repro app:

       private static ConnectionMultiplexer _connection;
        static void Main(string[] args)
        {
           string connectionString = "......";
            _connection = ConnectionMultiplexer.Connect(connectionString);

            _connection.ConnectionRestored += OnConnectionRestored;

            Console.ReadLine();
        }

        private static void OnConnectionRestored(object sender, ConnectionFailedEventArgs args)
        {
            Console.WriteLine("_connection.IsConnected is " + _connection.IsConnected);
        }

Repro steps:
1). Update connectionString for your Redis, build the app and run
2). Stop and Restart the Redis

Expected result:
OnConnectionRestored rasie one time, in output should just display one line "_connection.IsConnected is ..."

Actual result:
OnConnectionRestored rasie two times, in output displays two lines "_connection.IsConnected is ..."

Hi,
While converting my code from Booksleeve to StackExchange.Redis, I found that the new transaction system does not support publishing inside of a transaction like Booksleeve did.
I guess this is by design (changes for Redis Cluster?), but is there some kind of way to combine the actions from GetDatabase and GetSubscriber? In my use case, I have to (kind of batch processing like) set lots of keys and publish a message for each of them. This seemed way more straight-forward in Booksleeve, but maybe I missed something?

This may just be me doing something completely wrong, but any help is appreciated.

Here is my order of operations:

• nuget install-package stackexchange.redis
• git pull, git reset head --hard
• Set StackExchange.Redis_Net40 project to be strongly named
• Rebuild with VS2013.
• Dropped dll into packages folder, overwriting the original one obtained from nuget

Then I get build errors indicating the StackExchange namespace could not be found.

Server Error in '/' Application.

The free-quota limit on '6000 Redis requests per hour' has been reached. Please see https://servicestack.net to upgrade to a commercial license or visit https://github.com/ServiceStackV3/ServiceStackV3 to revert back to the free ServiceStack v3.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: ServiceStack.LicenseException: The free-quota limit on '6000 Redis requests per hour' has been reached. Please see https://servicestack.net to upgrade to a commercial license or visit https://github.com/ServiceStackV3/ServiceStackV3 to revert back to the free ServiceStack v3.

Source Error:

Line 137: {
Line 138: inventory = redisClient.As<List>();
Line 139: inventory.SetEntry(keyid.ToString(), inv);
Line 140: }
Line 141:

Source File: E:\Knowledge\Cache\Redis\RedisTry\RedisClient\RedisClient\Default.aspx.cs Line: 139

Stack Trace:

[LicenseException: The free-quota limit on '6000 Redis requests per hour' has been reached. Please see https://servicestack.net to upgrade to a commercial license or visit https://github.com/ServiceStackV3/ServiceStackV3 to revert back to the free ServiceStack v3.]
ServiceStack.LicenseUtils.ApprovedUsage(LicenseFeature licenseFeature, LicenseFeature requestedFeature, Int32 allowedUsage, Int32 actualUsage, String message) +227
ServiceStack.LicenseUtils.AssertValidUsage(LicenseFeature feature, QuotaType quotaType, Int32 count) +656
ServiceStack.Redis.RedisNativeClient.SendCommand(Byte[][] cmdWithBinaryArgs) +109
ServiceStack.Redis.RedisNativeClient.SendExpectLong(Byte[][] cmdWithBinaryArgs) +50
ServiceStack.Redis.RedisNativeClient.SAdd(String setId, Byte[] value) +172
ServiceStack.Redis.RedisClient.AddItemToSet(String setId, String item) +74
ServiceStack.Redis.RedisClient.RegisterTypeId(String typeIdsSetKey, String id) +118
ServiceStack.Redis.RedisClient.RegisterTypeId(T value) +206
ServiceStack.Redis.Generic.RedisTypedClient`1.SetEntry(String key, T value) +225
RedisClient._Default.Button2_Click(Object sender, EventArgs e) in E:\Knowledge\Cache\Redis\RedisTry\RedisClient\RedisClient\Default.aspx.cs:139
System.Web.UI.WebControls.Button.OnClick(EventArgs e) +9628114
System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) +103
System.Web.UI.WebControls.Button.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument) +10
System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) +13
System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData) +35
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +1724

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.34209

Can the API be used against other similar key-value db with similar structure? and how can the configuration be modified for this?

Hi, great package!

I am trying to use Redis 3.0 (on Linux) in a cluster configuration, same setup (6 nodes) as you use in the Cluster.cs tests. The underlying requirement is to get one of the ASP.NET session providers working with a cluster configuration. They typically use "KeyExpire()" in some fashion. However, if I execute a "CLUSTER FAILOVER" command on one of the slaves, KeyExpire() (and maybe other methods) falls apart. Here is a test I added to "cluster.cs":

    [Test]
    public void KeyExpireWithClusterFailover()
    {
        using (var conn = Create())
        {
            var db = conn.GetDatabase();
            RedisKey key = "abcabc";
            db.StringSet(key, "myvalue");
            for (var i = 0; i < 1000; i++)
            {
                db.KeyExpire(key, TimeSpan.FromHours(1));
                Thread.Sleep(5000);
            }
        }
        Assert.IsTrue(true);
    }

While this code is running, execute a "CLUSTER FAILOVER" from port 7004. After a little while, the code fails with a

StackExchange.Redis.RedisServerException : MOVED 6207 <ip-address>:7004

Sniffing around with the debugger, the code doesn't seem to pick up the new master/slave configuration after the failover.

Any help would be much appreciated! If this is resolved, we would have the perfect scale out / failover solution for ASP.NET session state.

Regards, Peter Myklebust

I wonder if there is a way to avoid caching password in the ConnectionMultiplexer. Today ConnectionMultiplexer keeps reference to the ConfigurationOptions object that has plain text password inside. If process crashes creating minidump then password may be exposed in it.

Probably password could be stored as SecureString in the ConfigurationOptions.