Comments (8)
Hey @summersonne, that's an interesting but pesky bug you've encountered there. My theory is that because you're running 128 instances, the amount of metrics increase because you get them for each OpenSIPS instance, thus bumping into the 65K datagram limit. Bumping the datagram size over 65K is not an option AFAIK though.
Can exporter work with mi_xmlrpc or mi_fifo instead of mi_datagram ?
Yes it could though no such functionality exists in the codebase yet. I am currently in the process of implementing the mi_json endpoint to support OpenSIPS > 3.1 (#6), but the OpenSIPS 1.10 that you're running does not have that module. I have no plans to implement mi_xmlrpc myself at this point. Though I'd take pull requests to do so and would be willing to help out to make it happen.
Possible solution for now
Something that might sidestep the 65K limit you're running into would be to try and get metrics by targeting specific modules. For example, try to just run curl localhost:9434/metrics?collect[]=core:
and see if that does come back without reaching the limit. If that works for all the modules that you want metrics from, you can setup prometheus to have seperate jobs scraping the seperate modules for you. I know it's not ideal, but that could be a workaround in your situation until mi_xmlrpc is implemnted.
from opensips_exporter.
Thank you for your answer.
I have another question. Is it possible to filter out some modules? pkmem for example?
Also curl -g localhost:9434/metrics?collect[]=core:&collect[]=usrloc: gives error curl localhost:9434/metrics?collect[]=core:&collect[]=usrloc:
[1] 46613
-bash: collect[]: bad array subscript
from opensips_exporter.
Yes, you're able to filter out modules by selecting the modules that you want metrics for. So if you don't want pkmem, just don't include it in the collect[]
array you sent through HTTP parameters.
It seems that the error you're getting with curl is a bash error, not so much an exporter error. It's because bash sees []
as a shell builtin. If you escape it with backslashes it should work like so:
curl "localhost:9434/metrics?collect\[\]=core:"
Though if you set it up with Prometheus params, it should work just fine, so it's just an issue with bash that you're seeing there.
from opensips_exporter.
Thank you for your answer.
I tried to configure Prometheus scrape config like this:
- job_name: "opensips-modules"
metrics_path: /metrics
scrape_interval: 30s
params:
module:
- 'core:'
relabel_configs:
- source_labels: [__address__]
separator: ':'
regex: '(.*):(9434)'
replacement: '${1}'
target_label: 'instance'
file_sd_configs:
- files:
- "/etc/prometheus/file_sd/sip-proxy.yml"
But I still can see metrics like opensips_pkmem_free_size and as far as I understood its from pkmem module
I am not sure it's my config wrong or module filtering not working. Can you help?
Also changing config like
params:
module:
- core
not worked as well
from opensips_exporter.
That is not supposed to happen if you have set the param correctly. Could you try manually running curl for just the core metrics and post the output here? (curl "localhost:9434/metrics?collect\[\]=core:"
) When I run it I am only getting the core metrics and not the pkmem ones.
The Prometheus scrape config looks good to me though so this might be a bug somewhere.
from opensips_exporter.
When I do curl "localhost:9434/metrics?collect[]=core:" on localhost exporter return only core metrics
So it's Prometheus config or bug?
from opensips_exporter.
I figured out what was the problem. Config must be like this:
- job_name: "opensips-modules"
metrics_path: /metrics
scrape_interval: 30s
params:
collect[]:
- 'core:'
- 'dialog:'
- 'load:'
- 'net:'
- 'registrar:'
- 'shmem:'
- 'sl:'
- 'tm:'
- 'tmx:'
- 'uri:'
- 'usrloc:'
relabel_configs:
- source_labels: [__address__]
separator: ':'
regex: '(.*):(9434)'
replacement: '${1}'
target_label: 'instance'
file_sd_configs:
- files:
- "/etc/prometheus/file_sd/sip-proxy.yml"
from opensips_exporter.
Great, glad that you figured it out. 👍 Looking at what you're collecting I'm assuming that pkmem was what's causing the datagram to be too large. Which makes sense because it returns information for all PID's, and if you have 128 of them, that's a lot of data. 😅
Closing this issue since your issue is fixed.
from opensips_exporter.
Related Issues (13)
- Opensips 3.* support HOT 1
- OpenSIPS can bind to network sockets for mi_datagram instead of unix domain sockets HOT 1
- Add information about profiles
- exporter creating to many empty directories like shown below
- opensips_exporter crashes with panic: inconsistent label cardinality error
- Exporter crashes when scraping load metrics on OpenSIPS 2.4.2 HOT 1
- Metrics missing with OpenSIPS 2.4.2 HOT 2
- OpenSIPS 2.4.5 does not parse metrics correctly HOT 2
- Support other Management Interface protocols HOT 2
- Getting Error HOT 5
- 'i/o timeout' and mi_datagram 'failed to send the response' errors HOT 5
- Check socket file permissions before starting
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opensips_exporter.