Comments (12)
Done. Rel v0.2.6. Documentation still needs to be updated though.
from grok_exporter.
(I failed some Markdown in my issue and my web mobile client does not support the "Edit" button, I'll fix this once I get access to a full featured client.) Done
from grok_exporter.
Thanks for the feature request. I think it's a good idea. grok_exporter
uses go templates to define label values. The go template language is extensible: go provides some predefined functions, but grok_exporter
could also provide some custom functions. It should be straightforward to implement some functions from the logstash mutate filter (like gsub) as custom functions for label templates.
I will look into it.
Apart from that: The cases you describe (only keep base url, extract query string labels) can also be achieved with a grok pattern.
from grok_exporter.
Thanks for the positive reply, happy to hear that you find the idea is relevant :)
I knew adapting the line matching regex could help for some mutations but for the query string problem I cannot see how would it works for matching all query strings (and name them accordingly).
The best I can think of is matching a preset number of query parameters.
Bonus: Decode URL encoded query strings value cannot be done using regex matching ;)
Disclosure: I'll pair logstash to grok_exporter for now (logstash will ingest file, mutate it and feed it to grok_exporter), but as soon as I can drop logsatsh and perform my mutations directly into grok_exporter I will :)
Other disclosure: I opened a similar feature request on mtail
's bugtracker ;)
from grok_exporter.
I implemented an experimental gsub
function for label templates. It's not in a release yet and it's not documented, but if you compile grok_exporter
from source you can try it.
Example: If .url
is a value containing your example url from above http://example.com/foo.asp?id=42&source=github&foo=bar and you want to use just the value 42
from the id
as a label, it should work as follows:
labels:
id: '{{gsub .url ".*id=([^&]*).*" "\\1"}}'
The syntax is {{gsub input pattern replacement}}
(see golang's text/template for general info on templates).
The pattern and replacement syntax is similar to Elastic's mutate filter's gsub (derived from Ruby's String.gsub()), except that you need to double-escape backslashes (\\ instead of \).
In the example, \\1
is the first capture group, which is ([^&]*)
in the pattern. This matches 42
, so the match is replaced with 42
.
Let me know if this is helpful.
from grok_exporter.
Great! This is very helpful to us, do you plans to release this?
Thanks in advance!
from grok_exporter.
Documentation updated.
from grok_exporter.
Can you please explain how can I transform text strings than contains "True" and "False" to boolean metrics with "gsub"?
from grok_exporter.
You should use conditionals for that instead of gsub
. Example:
grok:
additional_patterns:
- 'BOOLEAN True|False'
metrics:
- type: gauge
name: boolean_test
help: boolean test
match: '%{BOOLEAN:bool}'
value: '{{if eq .bool "True"}}1{{else}}0{{end}}'
from grok_exporter.
Thanks a lot!
from grok_exporter.
@fstab please add the following as an example to the documentation where you feel it's appropriate. I'm using it to convert units of measurement to base units (e.g. GB to bytes):
{{ if eq .unit "G" }}1000000000{{ else if eq .unit "M" }}1000000{{else if eq .unit "K"}}1000{{else}}{{.heap}}{{end}}
The data looks like heap.memory.used=8.9G,
, and I'm breaking it up with this regex:
- help: Process heap memory used
labels:
host: myhost
logfile: '{{.logfile}}'
match: 'heap.memory.used=(?<heap>[0-9\.]+)(?<unit>[GMK]),'
name: heap_memory_used
type: gauge
value: '{{ if eq .unit "G" }}{{ multiply .heap 1000000000 }}{{ else if eq .unit "M" }}{{ multiply .heap 1000000 }}{{else if eq .unit "K"}}{{ multiply .heap 1000}}{{else}}{{.heap}}{{end}}'
The tricky part is - I'm maintaining the config files from ansible, and I need to escape every '{{' and '}}' and it looks awful... Any suggestions on forcing a different separator? :)
from grok_exporter.
@fstab I was able to get a constant value from my logfile and set it to gauge metric , but I want it to be updated every time with latest value.
from grok_exporter.
Related Issues (20)
- Feature Request: Option to specify ciphers or minimum TLS version HOT 2
- fail_on_missing_logfile for missing directories HOT 3
- Feature Request : Config option to expose metrics as a .prom file
- Issue reading Multiple files:
- how to match regex upper case/lower case insensitively
- Is grok able to watch only the latest log in dir?
- how to use regex group match and use matched group name label? HOT 1
- Can the latest version cover multi-line pattern match?
- if string exists HOT 7
- Use Gauge metric to find DB connectivity status HOT 1
- Exclude char HOT 2
- Does it support Elastic Search Open Distro HOT 1
- Configure for AWS ALB/CLB Logs or Cloudfront
- grok timestamp diffrent
- grok fails on insufficient permissions when fail_on_missing_logfile is false
- mutiple depth doc support HOT 1
- How can I use FuncMap to help transform data from the log, inside a template?
- Always report counter metric (initialize to zero) HOT 1
- match fails after encountering binary text within file
- Update or let someone else maintain this repo HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grok_exporter.