Comments (12)
Hi,
Great question, we certainly need to improve the documentation around filtering somewhat...
The interface we have here is just a very thin wrapper over what you would do to filter entries with journalctl
...
I made an attempt to explain this better below, by borrowing the examples from the journalctl docs, let me know if they help? And I will work on a properly written page of documentation...
AFACT systemd does not support wildcards in these filters ... so to get everything that contains any value in CONTAINER_NAME
you would probably want to read all the messages from the journal, then filter things down futher e.g. with a grep filter in fluentd https://docs.fluentd.org/v1.0/articles/filter_grep
https://www.freedesktop.org/software/systemd/man/journalctl.html
Without arguments, all collected logs are shown unfiltered:
journalctl
This is the default if you don't specify any filters
in the config
filters []
With one match specified, all entries with a field matching the expression are shown:
journalctl _SYSTEMD_UNIT=avahi-daemon.service
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service"}]
If two different fields are matched, only entries matching both expressions at the same time are shown:
journalctl _SYSTEMD_UNIT=avahi-daemon.service _PID=28097
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}]
If two matches refer to the same field, all entries matching either expression are shown:
journalctl _SYSTEMD_UNIT=avahi-daemon.service _SYSTEMD_UNIT=dbus.service
Fields with Arrays as values are treated as an OR statement, since a ruby hash can only have one value per key.
filters [{"_SYSTEMD_UNIT": ["avahi-daemon.service", "dbus.service"]}]
This could also be expressed as two separate filter hashes...
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service"}, {"_SYSTEMD_UNIT": "dbus.service"}]
The form you choose only matters if you need to filter on multiple fields
If the separator "+" is used, two expressions may be combined in a logical OR. The following will show all messages from the Avahi service process with the PID 28097 plus all messages from the D-Bus service (from any of its processes):
journalctl _SYSTEMD_UNIT=avahi-daemon.service _PID=28097 + _SYSTEMD_UNIT=dbus.service
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}, {"_SYSTEMD_UNIT": "dbus.service"}]
Show all logs generated by the D-Bus executable:
journalctl /usr/bin/dbus-daemon
filters [{"_exe": "/usr/bin/dbus-daemon"}]
from fluent-plugin-systemd.
Thanks so much for the excellent description!
from fluent-plugin-systemd.
Honestly I haven't thought about it much ... but it is a goal for 1.0 to have better documentation.
/docs
on master seems reasonable to start with, I think the README is already a bit too long... so think we should start to split off some topic pages and index them all in the README...
Honestly though if you want to spend some time on this ... do whatever you feel works best...
from fluent-plugin-systemd.
If I was to put in a PR to expand the documentation around this particular topic, which branch should I base it off of? Also, would you like to have documentation branch out into separate files in a /docs
directory, or just continue to expand off of README.md
?
from fluent-plugin-systemd.
I'll submit separate PRs to both.
from fluent-plugin-systemd.
I'm writing up the documentation now, but I found an edge case that you didn't address. Is there any way to specify an OR
condition between two expressions instead of the default AND
?
IE:
# journalctl _PID=2345 _SYSTEMD_UNIT=docker.service
... <logical AND result here> ...
# journalctl _PID=2345 + _SYSTEMD_UNIT=docker.service
... <logical OR result here> ...
from fluent-plugin-systemd.
I think one of your examples might actually hit this question.
You gave an example of:
filters [{"_SYSTEMD_UNIT": "avahi-daemon.service", "_PID": 28097}, {"_SYSTEMD_UNIT": "dbus.service"}]
Would the two separate hashes define a logical OR
condition @errm? IE: [{"THING1": "value"}, {"THING2": "value"}]
would match any logs with THING1=value
OR
THING2=value
?
from fluent-plugin-systemd.
Correct:
- Within a single Hash the field matches are logical AND
- Each hash in the array is a logical OR
- Array values are logical OR (for that value)
from fluent-plugin-systemd.
There, I've created 2 PRs, one against the master branch and the other cherry-picking commits into the 1.0.0 branch.
from fluent-plugin-systemd.
Sure I wouldn't worry about the v1.0.0 branch I am going to merge it into master just before we release v1 anyway ... people will see the docs on master when they look for them ...
from fluent-plugin-systemd.
Since my issue has been resolved and master documentation has been merged, I'm happy that this issue is fully dealt with. Thanks again for your help!
from fluent-plugin-systemd.
Thanks for your help :)
from fluent-plugin-systemd.
Related Issues (20)
- Error: Is a directory @ io_getpartial - /var/log/journal HOT 2
- Is it possible to transfer the ownership to fluent-plugins-nursery? HOT 7
- Does support Journal Namespaces? HOT 4
- Parser type filter not working with systemd logs
- Unable to read and output journal files from default path var/log/journal HOT 1
- No logs are read on systemd 253+ potentially HOT 2
- Can't configure storage with the conf.arg parameter HOT 1
- Segmentation fault on Ubuntu 22.04 HOT 4
- How to get Gke node reboot and shutdown logs using journald Systemd service unit HOT 1
- Do not install the plugin HOT 1
- Not able to ship logs even after adding td-agent user to systemd-journal group HOT 2
- heads up! - systemd 246 journal logs produced is not readable by older versions of systemd-journald HOT 2
- support for Fedora based systems? HOT 1
- Logs getting delayed
- This plugin does not seem to work when systemd rotates the journal
- Plugin crash during graceful shutdown
- Does this plugin support zstd compression compression?
- Latest 1.0.5 version is missing some gem dependencies
- Unable to read journal files from subdirectories under default path var/log/journal
- Using systemd entry in record_modifier
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fluent-plugin-systemd.