Allow mutations on data about grok_exporter

[fstab]

Done. Rel v0.2.6. Documentation still needs to be updated though.

[C-Duv]

(I failed some Markdown in my issue and my web mobile client does not support the "Edit" button, I'll fix this once I get access to a full featured client.) Done

[fstab]

Thanks for the feature request. I think it's a good idea. grok_exporter uses go templates to define label values. The go template language is extensible: go provides some predefined functions, but grok_exporter could also provide some custom functions. It should be straightforward to implement some functions from the logstash mutate filter (like gsub) as custom functions for label templates.

I will look into it.

Apart from that: The cases you describe (only keep base url, extract query string labels) can also be achieved with a grok pattern.

[C-Duv]

Thanks for the positive reply, happy to hear that you find the idea is relevant :)

I knew adapting the line matching regex could help for some mutations but for the query string problem I cannot see how would it works for matching all query strings (and name them accordingly).
The best I can think of is matching a preset number of query parameters.

Bonus: Decode URL encoded query strings value cannot be done using regex matching ;)

Disclosure: I'll pair logstash to grok_exporter for now (logstash will ingest file, mutate it and feed it to grok_exporter), but as soon as I can drop logsatsh and perform my mutations directly into grok_exporter I will :)

Other disclosure: I opened a similar feature request on mtail's bugtracker ;)

[fstab]

I implemented an experimental gsub function for label templates. It's not in a release yet and it's not documented, but if you compile grok_exporter from source you can try it.

Example: If .url is a value containing your example url from above http://example.com/foo.asp?id=42&source=github&foo=bar and you want to use just the value 42 from the id as a label, it should work as follows:

labels:
    id: '{{gsub .url ".*id=([^&]*).*" "\\1"}}'

The syntax is {{gsub input pattern replacement}} (see golang's text/template for general info on templates).

The pattern and replacement syntax is similar to Elastic's mutate filter's gsub (derived from Ruby's String.gsub()), except that you need to double-escape backslashes (\\ instead of \).

In the example, \\1 is the first capture group, which is ([^&]*) in the pattern. This matches 42, so the match is replaced with 42.

Let me know if this is helpful.

[andreyev]

Great! This is very helpful to us, do you plans to release this?

Thanks in advance!

[fstab]

Documentation updated.

[eskornev]

Can you please explain how can I transform text strings than contains "True" and "False" to boolean metrics with "gsub"?

[fstab]

You should use conditionals for that instead of gsub. Example:

grok:
    additional_patterns:
    - 'BOOLEAN True|False'
metrics:
    - type: gauge
      name: boolean_test
      help: boolean test
      match: '%{BOOLEAN:bool}'
      value: '{{if eq .bool "True"}}1{{else}}0{{end}}'

[eskornev]

Thanks a lot!

[mad-ady]

@fstab please add the following as an example to the documentation where you feel it's appropriate. I'm using it to convert units of measurement to base units (e.g. GB to bytes):
{{ if eq .unit "G" }}1000000000{{ else if eq .unit "M" }}1000000{{else if eq .unit "K"}}1000{{else}}{{.heap}}{{end}}
The data looks like heap.memory.used=8.9G,, and I'm breaking it up with this regex:

    -   help: Process heap memory used
        labels:
            host: myhost
            logfile: '{{.logfile}}'
        match: 'heap.memory.used=(?<heap>[0-9\.]+)(?<unit>[GMK]),'
        name: heap_memory_used
        type: gauge
        value: '{{ if eq .unit "G" }}{{ multiply .heap 1000000000 }}{{ else if eq .unit "M" }}{{ multiply .heap 1000000 }}{{else if eq .unit "K"}}{{ multiply .heap 1000}}{{else}}{{.heap}}{{end}}'

The tricky part is - I'm maintaining the config files from ansible, and I need to escape every '{{' and '}}' and it looks awful... Any suggestions on forcing a different separator? :)

[HarinathReddyA]

@fstab I was able to get a constant value from my logfile and set it to gauge metric , but I want it to be updated every time with latest value.