Comments (7)
Did the redis server get restarted at that time?
The most common reason for this socket error (returned from the operating system) is that the remote server is (temporarily) not listening on the unix domain socket.
Regards,
-agentzh
from lua-resty-redis.
it wasn't restarted to my knowledge
I will increase logging level and check further
thanks
from lua-resty-redis.
BTW, you may want to increase the backlog setting for the listening socket in the redis server. The redis server hardcodes the backlog value 511 in its source:
$ ack 511 src/
src/anet.c
265: if (listen(s, 511) == -1) { /* the magic 511 constant is from nginx */
The accept() queue on your redis server side can easily overflow when your redis is really busy at some moments. Ironically, nginx does provide ways to adjust the backlog setting for its listening socket (that is, the backlog parameter for the listen configure directive).
from lua-resty-redis.
excellent I will definitely try that, I really bet that's the problem!
There may be an influx of connections when the redis replies which I cache in ngx.shared.dict expire, which could easily all be at the same time!
Never the less, I have managed to tidy my lua code a lot with regard to redis while trying to find the problem which is always good :)
from lua-resty-redis.
BTW, have you enabled the connection pool for redis? See
https://github.com/agentzh/lua-resty-redis#set_keepalive
It could help by saving lots of connect() and close() syscalls.
Also, I'm pondering adding a built-in wait queue to the cosockets to prevent such peaks of new connections when the frontend cache entries are expired.
Another thing you could probably do here is to use a shdict entry to limit concurrent fetches when some cache entry is expired.
from lua-resty-redis.
yeah I'm using set keepalive.. I could increase the keepalive idle timeout to be greater than shdict expire time also I suppose (currently it's less than) which would probably help
I forked redis and set the backlog to -1 which nginx also does apparently, since I'm using FreeBSD, can do this:
The sysctl(3) MIB variable kern.ipc.somaxconn specifies a hard limit on
backlog; if a value greater than kern.ipc.somaxconn or less than zero is
specified, backlog is silently forced to kern.ipc.somaxconn.
Will see how it goes like this 👍
from lua-resty-redis.
Hi
This issue was solved. I was using "KEYS" irresponsibly causing the redis worker to be busy for longer than a few seconds and then waiting new connections would build up and overflow the backlog
Many thanks as always!
from lua-resty-redis.
Related Issues (20)
- SSL not working
- The performance of redis pipeline mode is unstable HOT 3
- CLIENT SETNAME?
- redis:get(key) question HOT 2
- opm package outdated HOT 1
- Connection pool not creating HOT 2
- How to create a single redis connection HOT 5
- this support two-way authentication?
- Add keepalive for redis client to make the connections reliable
- Nginx 1.24 HOT 2
- Nginx creates more connection than poolsize, backlog to redis HOT 5
- lua tcp socket read timed out, context: ssl_certificate_by_lua*, client: 172.69.58.204, server: 0.0.0.0:443 HOT 2
- redis set_keepalive doesn't work HOT 1
- lua entry thread aborted: runtime error: /usr/local/openresty/lualib/resty/redis.lua:357: bad request HOT 1
- no request found HOT 1
- Add the host:port to the error message
- Reading from client: error:0A000126:SSL routines::unexpected eof while reading
- red:smembers("key"), sometime not include the new member? HOT 6
- [Feature Request] Pipeline coroutine safety HOT 6
- Redis: timeout HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lua-resty-redis.