Can exporter work with mi_xmlrpc instead of mi_datagram about opensips_exporter

[RubenHoms]

Hey @summersonne, that's an interesting but pesky bug you've encountered there. My theory is that because you're running 128 instances, the amount of metrics increase because you get them for each OpenSIPS instance, thus bumping into the 65K datagram limit. Bumping the datagram size over 65K is not an option AFAIK though.

Can exporter work with mi_xmlrpc or mi_fifo instead of mi_datagram ?
Yes it could though no such functionality exists in the codebase yet. I am currently in the process of implementing the mi_json endpoint to support OpenSIPS > 3.1 (#6), but the OpenSIPS 1.10 that you're running does not have that module. I have no plans to implement mi_xmlrpc myself at this point. Though I'd take pull requests to do so and would be willing to help out to make it happen.

Possible solution for now
Something that might sidestep the 65K limit you're running into would be to try and get metrics by targeting specific modules. For example, try to just run curl localhost:9434/metrics?collect[]=core: and see if that does come back without reaching the limit. If that works for all the modules that you want metrics from, you can setup prometheus to have seperate jobs scraping the seperate modules for you. I know it's not ideal, but that could be a workaround in your situation until mi_xmlrpc is implemnted.

[summersonne]

Thank you for your answer.

I have another question. Is it possible to filter out some modules? pkmem for example?

Also curl -g localhost:9434/metrics?collect[]=core:&collect[]=usrloc: gives error curl localhost:9434/metrics?collect[]=core:&collect[]=usrloc:
[1] 46613
-bash: collect[]: bad array subscript

[RubenHoms]

Yes, you're able to filter out modules by selecting the modules that you want metrics for. So if you don't want pkmem, just don't include it in the collect[] array you sent through HTTP parameters.

It seems that the error you're getting with curl is a bash error, not so much an exporter error. It's because bash sees [] as a shell builtin. If you escape it with backslashes it should work like so:
curl "localhost:9434/metrics?collect\[\]=core:"

Though if you set it up with Prometheus params, it should work just fine, so it's just an issue with bash that you're seeing there.

[summersonne]

Thank you for your answer.

I tried to configure Prometheus scrape config like this:

  - job_name: "opensips-modules"
    metrics_path: /metrics
    scrape_interval: 30s
    params:
      module:
      - 'core:'
    relabel_configs:
    - source_labels: [__address__]
      separator: ':'
      regex: '(.*):(9434)'
      replacement: '${1}'
      target_label: 'instance'
    file_sd_configs:
    - files:
      - "/etc/prometheus/file_sd/sip-proxy.yml" 

But I still can see metrics like opensips_pkmem_free_size and as far as I understood its from pkmem module
I am not sure it's my config wrong or module filtering not working. Can you help?

Also changing config like

params:
  module:
  - core

not worked as well

[RubenHoms]

That is not supposed to happen if you have set the param correctly. Could you try manually running curl for just the core metrics and post the output here? (curl "localhost:9434/metrics?collect\[\]=core:") When I run it I am only getting the core metrics and not the pkmem ones.

The Prometheus scrape config looks good to me though so this might be a bug somewhere.

[summersonne]

When I do curl "localhost:9434/metrics?collect[]=core:" on localhost exporter return only core metrics

So it's Prometheus config or bug?

[summersonne]

I figured out what was the problem. Config must be like this:

  - job_name: "opensips-modules"
    metrics_path: /metrics
    scrape_interval: 30s
    params:
      collect[]:
        - 'core:'
        - 'dialog:'
        - 'load:'
        - 'net:'
        - 'registrar:'
        - 'shmem:'
        - 'sl:'
        - 'tm:'
        - 'tmx:'
        - 'uri:'
        - 'usrloc:'
    relabel_configs:
    - source_labels: [__address__]
      separator: ':'
      regex: '(.*):(9434)'
      replacement: '${1}'
      target_label: 'instance'
    file_sd_configs:
    - files:
      - "/etc/prometheus/file_sd/sip-proxy.yml"

[RubenHoms]

Great, glad that you figured it out. 👍 Looking at what you're collecting I'm assuming that pkmem was what's causing the datagram to be too large. Which makes sense because it returns information for all PID's, and if you have 128 of them, that's a lot of data. 😅
Closing this issue since your issue is fixed.