Comments (10)
and the button I was talking about:
from quickwit-datasource.
I think it might also be caused by the name of the fields, so here is another example that doesn't show any results, althought the data is for sure there:
app.kubernetes.io/managed-by:"trivy-operator"
Even when escaping characters:
app\.kubernetes\.io\/managed\-by:"trivy-operator"
from quickwit-datasource.
thanks a lot @damyan90 for all those feedback. @ddelemeny started working on it.
from quickwit-datasource.
Thanks!
I also noticed that it's impossible to get any results when using following queries:
azure.workload.identity/use:"true"
Also when trying to escape the dots and slashes:
Maybe I screw up the whole index config, especially when it comes to dynamic mapping:
from quickwit-datasource.
Thank you @damyan90 for your detailed feedback, much appreciated.
There's a few things to unpack here:
Quickwit returns an error on syntax-correct queries (upstream bug)
Parsing -fieldA:term1 AND -fieldB:term2
should not fail.
The plugin itself handles it correctly, but Quickwit fails to parse.
Parsing equivalent queries NOT fieldA:term1 AND NOT fieldB:term2
or (-fieldA:term1) AND (-fieldB:term2)
seem to work as expected.
Plugin-side I'll add a couple "()" around the filters as a quick fix, but the parsing failure needs to be followed-up upstream.
Quickwit returns unwanted results
That's the case shown by your first screenshot and possibly by your last comment
I haven't been able to reproduce the issue, it looks like the request sent by the plugin to Quickwit is ok. I could get a correct behavior from a test case with an "a.b/c"-shaped field name.
There is a seemingly unrelated linting error on the main branch, but that's not affecting releases and it doesn't prevent the request to be sent.
@fmassot do you have any insight there ? Smells like something out of my reach, indexing issue maybe ?
from quickwit-datasource.
Thanks for the quick check!
And yes, you're right with the parsing issue. The "()" or "NOT" instead of "-" resolves it - waiting for it to be implemented in the plugin then ;)
Regarding the "a.b/c" shaped fields, do you mind sharing the doc mapping for such fields which works for you?
from quickwit-datasource.
Sure, I tried to reproduce by adding a test value in attributes.test/field
.
Here are the relevant parts :
{
"field_mappings": [
...
{
"name": "attributes",
"type": "json",
"expand_dots": true,
"fast": {
"normalizer": "raw"
},
"indexed": true,
"record": "basic",
"stored": true,
"tokenizer": "raw"
},
...
],
"tag_fields": [],
"store_source": false,
"index_field_presence": false,
"timestamp_field": "timestamp_nanos",
"mode": "strict",
"max_num_partitions": 200,
"tokenizers": []
}
Not using a dynamic mapping though, that part could be investigated further
from quickwit-datasource.
Hmm this is weird. I think it only affects these few extracted fields (i.e. "app", "app.kubernetes.io/name"). I have to double check my fluentbit config cause I don't remember it being extracted.
However, for instance the "annotations.x.x./restartedAt" works fine..
Green are OK, Red ones are failing.
Will let you know if I find something.
from quickwit-datasource.
So as suspected, I'm not doing anything in the fluentbit to extract those fields. Also they're not actually extracted in qucikwit itself. That's only Grafana that shows them additionally, which is causing the whole confusion (if it is for me now when testing, then for the devs later when trying to filter by fields which don't exists in the backend - will be frustrating too)
So here's what I mean, the exact same log in both places.
Grafana:
So do you know which part is responsible for such extraction of these fields? Is it Grafana itself or the datasource?
What's weird also is that there is a field "Annotations" but not "labels".
I think it would be perfect if it's 1:1 as in quickwit. Because then I can extract these fields myself (in fluentbit config) if I want and they would be queryable as well.
from quickwit-datasource.
I researched a bit more about what could be going on in your last comments.
It appears that grafana treats the labels
field of a dataframe as a collection of transformation hints, specifically Extract Fields.
Not sure yet what to do about that...
from quickwit-datasource.
Related Issues (20)
- Valid query considered invalid by the editor HOT 2
- No graph in the explore view on a metric query
- Keeping query context while changing query types HOT 1
- Console warn "Duplicate field names in DataFrame: message" seen on a Grafana instance with 0.4.0-beta.2 version: HOT 2
- Unknown field `end_timestamp` HOT 3
- Make sure older logs can be explored
- Add sort direction HOT 4
- Support datasource config jsonData: {oauthPassThru: true} HOT 3
- Check if the plugin works with grafana 9.3 HOT 1
- Consider template variables valid in the query editor
- Remove `AND` on a positive filter in explore view and use simple quotes HOT 2
- Query editor flickers between old and new state when running query
- Cancelled requests for multiple variable queries
- Plugin freezes on hovering "Ad-hoc statistics" in explore view HOT 6
- Cannot create a panel on a dashboard with the pre-release v0.4.0 HOT 1
- Revert default search limit to 100 and make it configurable HOT 2
- `Shift-Enter` doesn't trigger runQuery if the editor is not focused
- Multisearch response unmarshalling loses nano precision from numeric timestamps HOT 1
- entering the Log context interface may feel it with invalid query HOT 3
- Allow using private datasource connect (PDC) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from quickwit-datasource.