Replace Function in Matched Captures about grok_exporter

[fstab]

Hi, sorry for the late reply. Yes, this is possible with the gsub function. Example:

Logfile:

INFO: request to path /abc/def/ghi/number/123123/jkl/mno
INFO: request to path /abc/def/ghi/number/123124/jkl/mno
INFO: request to path /abc/def/ghi/number/987654/jkl/mno
INFO: request to path /abc/def/ghi/number/654763/jkl/mno

config

global:
    config_version: 2
input:
    type: file
    path: ./logfile.log
    readall: true
grok:
    patterns_dir: ../logstash-patterns-core/patterns
metrics:
    - type: counter
      name: requests_total
      help: Total number of requests, partitioned by path.
      match: '.* %{URIPATH:path}'
      labels:
          path: '{{gsub .path "(.*)/[0-9]*/(.*)" "\\1/{id}/\\2" }}'
host: localhost
port: 9144

explanation:

gsub input pattern replacement searches for the pattern in input and replaces it with replacement. In the example, it searches for "(.*)/[0-9]*/(.*)" which is a pattern with two capture groups: (.*) is the first capture group, capturing everything befor the id, and (.*) is the second capture group, capturing everything after the id. The replacement "\\1/{id}/\\2" references the first capture group \\1 and the second capture group \\2 but the id is replaced with the fixed string {id}.

result

requests_total{path="/abc/def/ghi/number/{id}/jkl/mno"} 4

See gsub in CONFIG.md.

[varundmishra]

Hey,
Many thanks for the reply.
This looks great, I am going to try this over the weekend.
Will update this thread post that.

Cheers,
V

[kfreem02]

Is it possible to perform multiple gsub substitutions on the same metric label? How do I refer to the output of gsub 1 within gsub 2? In this example I want to replace both a UUID and a serial number on the same log line, but only the final gsub output is retained.

  labels:
    request: '{{gsub .request "(\\/)(?i)(prefix)(\\w+)(\\/?)" "\\1\\2_SERIAL\\4"}}'
    request: '{{gsub .request "(?i)[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}" "UUID"}}'

[varundmishra]

@fstab : Thank you for the inputs in the previous comment, I just got around testing this.
I have updated to the latest release 0.27 and have implemented this.
I will update the thread with all the results before closing the same.

Thank You.

[fstab]

@kfreem02 I added support for this. If you build grok_exporter from source it should work now, otherwise you need to wait for the next release. Example:

Input: message = Sender verify failed

      labels:
          error_message: '{{gsub (gsub .message "e" "a") "r" "x"}}'

Resulting error_message label: "Sandax vaxify failad"

In the example, first all e's are replaced with a's, then all r's replaced with x's.

See https://github.com/fstab/grok_exporter/blob/master/template/gsub_test.go#L39-L55

[varundmishra]

@fstab : This works great, thank you!!! Is the multiple substitution feature limited to two subs, or does it support more than two substitutions as well??

[kfreem02]

I confirm support of nested gsub when built from source. Thanks for the fast response!

[fstab]

@varundmishra it should work for any number of substitutions. If it doesn't please open a new issue.

[varundmishra]

Great, Many Thanks @fstab.