Comments (6)
Can you be specific about what happens in this failover scenario? Socket
death can be quite... particular. Most socket terminations should be
immediately detected, but there is a heartbeat to catch others. Also: what
exact error is it reporting?
On 1 Jul 2014 22:53, "Hrishikesh Pathak" [email protected] wrote:
Hello
We have 2 node redis server setup where 1 node is master and the other is
a slave. We have observed that StackExchange.Redis can take anything from
few seconds to several minutes to reconnect to the server if master/slave
failover happens in the background. i.e. if we keep on using the same
ConnectionMultiplexer object as the one created before the failover was
triggered we get keep on getting a connection exception. The
connectionMultiplexer object takes atleast 1 minute(sometimes more) to
recover from this condition. Is this expected? If we create a new
ConnectionMultiplexer object during this time its able to connect to the
redis server successfully
Thanks—
Reply to this email directly or view it on GitHub
#65.
from stackexchange.redis.
So for a duration of approximately 1 minute i see the following error in logs:
System.TimeoutException: Timeout performing GET key1, inst: 0, queue: 0, qu=0, qs=0, qc=0, wr=
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProc
nt server) in c:\Users\hrishp\Documents\GitHub\StackExchange.Redis\StackExchange.Redis\StackEx
xer.cs:line 1773
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processo
\Users\hrishp\Documents\GitHub\StackExchange.Redis\StackExchange.Redis\StackExchange\Redis\Red
at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in c:\User
Exchange.Redis\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:line 1346
at SampleConsoleApp.Program.Main(String[] args) in c:\Users\hrishp\Documents\Visual Studio
SampleConsoleApp\Program.cs:line 64
StackExchange.Redis.RedisConnectionException: No connection is available to service this opera
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProc
nt server) in c:\Users\hrishp\Documents\GitHub\StackExchange.Redis\StackExchange.Redis\StackEx
xer.cs:line 1738
If i create a new connection in this interval, everything works as expected. If it helps here is my sample console app code:
static ConnectionMultiplexer Connection
{
get
{
if ((_connection == null))
{
Console.WriteLine("Connection to the server is broken. Trying to connect...");
try
{
string configurationString = "<conn_str with hostname and password>";
_connection = ConnectionMultiplexer.Connect(configurationString);
Console.WriteLine("Connected successfully to the server : {0}", configurationString);
}
catch (Exception e)
{
Console.WriteLine("Exception in connecting to server : {0}", e);
_connection = null;
}
}
return _connection;
}
}
static void Main(string[] args)
{
Connection.GetDatabase().StringSet("key1","value1");
while (true)
{
try
{
{
Console.WriteLine(Connection.GetDatabase().StringGet("key1"));
}
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
Console.ReadKey();
}
from stackexchange.redis.
Any luck reproing this?
from stackexchange.redis.
What I understood is that when Redis Cache is under load and with the failover happening, Stackexchange.redis would try only once to be able to re-establish the connection. I have a pull request that would help to make it honor configuration.retry (which is 3 be default). Please review.
from stackexchange.redis.
There have been many, many improvements on connections and retry since this was logged - I'm going to close this one out. If someone can repro on a current version we'll open it right back up.
from stackexchange.redis.
Using StackExchange.Redis 1.2.1
I have three redis nodes, with Sentinel promoting another to master if one fails. I log into the Redis master and cause it to fail vis debug sleep 30
. The following client code will receive a System.TimeoutException
when trying to write to the master that has failed:
public static void Main(string[] args)
{
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("redis1-exp.aclens.local:6300,redis2-exp.aclens.local:6300,redis3-exp.aclens.local:6300");
while (true)
{
var db = redis.GetDatabase();
db.StringSet("mytime", DateTime.Now.ToLongTimeString()); // throws exception
Console.WriteLine(db.StringGet("mytime"));
Thread.Sleep(5000);
}
}
The following code fixed this for me:
public static void Main(string[] args)
{
ConnectionMultiplexer redis = ConnectionMultiplexer.Connect("redis1-exp.aclens.local:6300,redis2-exp.aclens.local:6300,redis3-exp.aclens.local:6300");
while (true)
{
try
{
var db = redis.GetDatabase();
db.StringSet("mytime", DateTime.Now.ToLongTimeString());
Console.WriteLine(db.StringGet("mytime"));
Thread.Sleep(5000);
}
catch (StackExchange.Redis.RedisConnectionException)
{
// reconnect
redis = ConnectionMultiplexer.Connect("redis1-exp.aclens.local:6300,redis2-exp.aclens.local:6300,redis3-exp.aclens.local:6300");
}
catch (System.TimeoutException)
{
// reconnect
redis = ConnectionMultiplexer.Connect("redis1-exp.aclens.local:6300,redis2-exp.aclens.local:6300,redis3-exp.aclens.local:6300");
}
}
}
from stackexchange.redis.
Related Issues (20)
- Assembly mismatch HOT 2
- Reading is not allowed after reader was completed HOT 2
- Pipelines.Sockets.Unofficial Thread Pool HOT 3
- Cannot convert to RedisValue: `MultiBulk` HOT 3
- threads hanging on production when using transaction
- Confusing nullability of `IDatabase.ListRightPop()` (and possibly more?) HOT 4
- The message timed out in the backlog attempting to send because no connection became available HOT 9
- ListLeftPushAsync doesn't use AsyncTimeout? HOT 4
- Timeout awaiting response - random short streaks HOT 3
- Massive SocketClosed Exceptions and ConnectionRestored with hanging requests HOT 4
- SocketFailure on xx.redis.cache.windows.net:6380/Interactive, Idle/Faulted, last: GET, origin: ReadFromPipe, outstanding: 12, last-read: 22s ago HOT 1
- UnableToConnect on http://redis:6379/Interactive, Initializing/NotStarted Docker HOT 1
- StackExchange.Redis.RedisTimeoutException HOT 3
- Feature proposal: high integrity mode
- RedisTimeoutException with high inbound HOT 1
- Error Connecting to Elasticache Redis Serverless - How to use TextWriter for Details HOT 2
- mixing synchronous code with asynchronous code with ScriptEvaluate
- Feature: Stall Detection
- Protocol error: invalid bulk length HOT 2
- ERR wrong number of arguments for 'expire' command - Azure Cache for Redis HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stackexchange.redis.