Comments (9)
Hello!
Thanks for the feedback.
There are two main reasons that may cause the problem:
- there is something that creates many
ets
'es in your code, so that the creation of the next mc session fails when system limits are exhausted; - all of the
ets
'es are consumed bySMPPEX
itself, in this case there should be many (unstopped for some reason) mc sessions.
So there are several questions I would like to ask to get the situation more clear:
- What is the number of simultaneous client connections that your server has when the crash occurs? Have you specified custom
max_connections
transport option when starting MC? - What are the names of
ets
'es that pollute theets
space when the crash occurs? (This info can be obtained by running:ets.i()
).
from smppex.
Closing due to no reply.
from smppex.
@savonarola Hi, we just ran into the same issue. My max_connections is at 600 (while the ETS table limit should be around 1400 by default) and I had a health checker try and open (and close) a socket every 10 seconds.
12:47:58.117 [info] mc_conn #PID<0.1832.0>, socket closed, stopping
12:48:08.117 [info] mc_conn #PID<0.1838.0>, socket closed, stopping
12:48:08.117 [info] mc_conn #PID<0.1841.0>, socket closed, stopping
12:48:18.117 [info] mc_conn #PID<0.1844.0>, socket closed, stopping
12:48:18.117 [info] mc_conn #PID<0.1847.0>, socket closed, stopping
12:48:28.117 [info] mc_conn #PID<0.1850.0>, socket closed, stopping
12:48:28.117 [info] mc_conn #PID<0.1853.0>, socket closed, stopping
12:48:38.117 [info] mc_conn #PID<0.1856.0>, socket closed, stopping
...
After a few hours, of this though, any time the health checker opens a socket, we encounter this issue:
16:54:48.121 [error] Ranch listener #Reference<0.0.2.571> connection process start failure; SMPPEX.Session:start_link/4 returned: {:error, {{:badmatch, {:error, {:system_limit, [{:ets, :new, [:pdu_storage_by_sequence_number, [:set]], []}, {SMPPEX.PduStorage, :init, 1, [file: 'lib/smppex/pdu_storage.ex', line: 43]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 328]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}}}, [{SMPPEX.MC, :init, 1, [file: 'lib/smppex/mc.ex', line: 386]}, {:gen_server, :init_it, 6, [file: 'gen_server.erl', line: 328]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 247]}]}}
So it seems that the ETS table is not getting cleaned up properly when a Ranch socket is closed.
from smppex.
Do note that we have no active connections to the instance, except for the health-check opening and closing the socket (so this is not a case of it being over-saturated with traffic).
from smppex.
Hello!
Trying to reproduce the issue.
from smppex.
from smppex.
from smppex.
Hello!
I have reproduced the issue; the reason was that peer closing socket is not considered to be an abnormal case, so MC session stopped with :normal
leaving child PduStorage
alive and keeping its ets
.
I have added the necessary cleanup.
from smppex.
@savonarola as always, thank you for the swift fix! 🍻
from smppex.
Related Issues (20)
- Support semantic versioning of dependencies
- `deliver_sm_resp` Factory incomplete HOT 2
- Process name for SMPPEX.ESME.start_link HOT 2
- Generic nack - recognized as pdu - not as pdu response HOT 2
- SMPPEX.MC doesn't send enquire_link automatically HOT 4
- Why is the IF Version not sended? HOT 2
- Receive SMS HOT 2
- Set default_call_timeout to SMPPEX.Session.Defaults or config HOT 4
- Q: Any recommendations on how to process submit_sm asynchronously? HOT 3
- Q: Why are the body paramters in submit_multi_resp mandatory?
- Q: Is there any ready to use SMPP server? HOT 2
- Upgrade to Ranch >= 2 HOT 1
- Question: Wrong use of handle_socket_error? HOT 1
- Usage of functions that require session/pid HOT 4
- Why is poison a dependency? HOT 2
- Sending multipart message HOT 9
- External sequence number generation and pdu storage HOT 1
- log noise due to timeouts HOT 2
- support starting MC in the client app supervision tree HOT 1
- Receiving multipart SMS HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from smppex.