Comments (41)
The whole point of a promfmt
tool is to, if not enforce, at least "endorse" a certain "canonical" formatting, very much like gofmt
, with a whole string of arguments behind it.
from prometheus.
v3 of the YAML library is out, which should allow for all of this.
from prometheus.
Yeah, I had strong opinions about this initially. At first Prometheus only supported sum(x) by(y)
because it read more like English to me and thus I much preferred it, but for any non-trivial x
it becomes really hard to see what dimensions you are aggregating over / keeping. I wish I had just made it sum by(y) (x)
from the beginning and made that the only legal variant.
I still don't like at all the exact formatting sum by (y)(x)
and much prefer sum by(y) (x)
(makes much more sense to me), not sure why we chose to go for the latter one in the docs.
from prometheus.
👍 * 💯
from prometheus.
Let's start with the PromQL part. The YAML part should be almost trivial once the library preserves comments and things like the |2
indentation hint for multiline strings.
from prometheus.
The final representation after formatting can be discussed and decided during the GSoC period (maybe even now if any maintainer wants to chime in, but I don't have the bandwidth to review it at the moment). I would suggest (to all GSoC aspirants) to focus on how would you achieve this.
from prometheus.
https://prometheus.io/docs/practices/rules/ uses the best practices.
from prometheus.
i wrote a rough prototype to preserve comments by augmenting the goyacc grammar rules a little. commit here [haibeey/prometheus@8855667]
Putting the comments in the AST seems like a reasonable idea.
I've left some comments about the implementation there.
from prometheus.
In relation to YAML comments, there's problems with v3 that's causing issues for Cortex so we should avoid further rollout of that until that's all resolved.
That said i think the formatting is not only to be done in rules files but in some places too like the web ui. is this true
The web ui is the only place it's done.
from prometheus.
Yep, that'd be great. Preserving comments is the biggest (and basically only) problem. And agreement on how to split expressions over multiple lines.
from prometheus.
Aren't we in the middle of adding this to promtool?
from prometheus.
Related: #1779
from prometheus.
Some observations:
- With the 2.x rule format (YAML embedding PromQL), we want this to act on YAML files, too.
- It should preserve both comments in YAML and in PromQL.
- Indentation is crucial (not just line breaks).
- It has to be seen if certain line breaks in the input should be preserved in the output (cf. how
gofmt
does it). - Wherever a rule is rendered, the same formatting should apply, in particular on the Alerts and Rules tabs of the Prometheus web UI. This implies that the AST needs to keep comments (and possibly certain line breaks, see previous point).
from prometheus.
It should preserve both comments in YAML
Our current YAML library doesn't do this, but is hoping to "soon".
from prometheus.
I'd like to include this in my gsoc proposal. @brian-brazil, can you link to the YAML library you're referring to?
from prometheus.
https://github.com/go-yaml/yaml, but I'd scope this to just the PromQL as this stands.
from prometheus.
@brian-brazil But since the PromQL from rules is always embedded in YAML files, it at least has to preserve YAML comments, right? Otherwise the tool wouldn't be very useful as nobody wants to lose their comments the benefit of formatting.
from prometheus.
That'd be nice, but we can get benefits without that. I'd rather also not depend on something with an unclear timeline for a gsoc project.
from prometheus.
Ok!
from prometheus.
FYI perfectly readable rule in configuration
(
kube_job_status_failed > 0
UNLESS kube_job_status_succeeded > 0
)
* on (namespace, job_name) group_left(maintainer) label_replace(kube_job_labels, "maintainer", "$1", "label_maintainer", "(.*)")
* on (namespace, job_name) group_left(pager) label_replace(kube_job_labels, "pager", "$1", "label_pager", "(.*)")
* on (namespace, job_name) group_left(paging) label_replace(kube_job_labels, "paging", "$1", "label_paging", "(.*)")
VS its garbage representation in the Web UI
(kube_deployment_status_replicas_available
/ kube_deployment_spec_replicas * 100) * on(namespace, deployment) group_left(maintainer)
label_replace(kube_deployment_labels, "maintainer", "$1", "label_maintainer",
"(.*)") * on(namespace, deployment) group_left(pager) label_replace(kube_deployment_labels,
"pager", "$1", "label_pager", "(.*)") * on(namespace,
deployment) group_left(paging) label_replace(kube_deployment_labels, "paging",
"$1", "label_paging", "(.*)") < 100
from prometheus.
@sylr I think you posted some other rule from the web UI. also @beorn7 I am starting to work on this issue.
here's what it shows for me for your rule:
(kube_job_status_failed
> 0 unless kube_job_status_succeeded > 0) * on(namespace, job_name) group_left(maintainer)
label_replace(kube_job_labels, "maintainer", "$1", "label_maintainer",
"(.*)") * on(namespace, job_name) group_left(pager) label_replace(kube_job_labels,
"pager", "$1", "label_pager", "(.*)") * on(namespace,
job_name) group_left(paging) label_replace(kube_job_labels, "paging", "$1",
"label_paging", "(.*)")
from prometheus.
@geekodour Yes, I mixed up two rules.
from prometheus.
since this is still open and added in project ideas for upcoming GSOC (https://github.com/cncf/soc#rule-formatting-tool) i would like to work on the completion of this during GSOC. Any recent update and additional info regarding this would be appreciated @codesome @brian-brazil @juliusv @geekodour @simonpasquier @sylr .
From reading the comment and other reference link it seems the next action point is building a formatter for promQL expression with the promQL AST?
from prometheus.
@haibeey There already has been some progress in that direction.
When building such a formatter it probably would be best to start with what is already implemented here and add proper white space and comment handling.
It would be also nice, when such features could be integrated with the upcoming PromQL language server. Inside the language server code a lot of the stuff that is needed to format YAML files is already implemented, and it might make sense to reuse some of it.
from prometheus.
alright! Thanks @slrtbtfs . i will start perusing the links shared ASAP .
from prometheus.
Can I take up this issue? Would love to implement this.
from prometheus.
It looks like @haibeey is already somehow planning working on this.
from prometheus.
@roidelapluie I have already implemented a part of it way before. It's just that I had not mentioned here.
from prometheus.
FYI perfectly readable rule in configuration
( kube_job_status_failed > 0 UNLESS kube_job_status_succeeded > 0 ) * on (namespace, job_name) group_left(maintainer) label_replace(kube_job_labels, "maintainer", "$1", "label_maintainer", "(.*)") * on (namespace, job_name) group_left(pager) label_replace(kube_job_labels, "pager", "$1", "label_pager", "(.*)") * on (namespace, job_name) group_left(paging) label_replace(kube_job_labels, "paging", "$1", "label_paging", "(.*)")
VS its garbage representation in the Web UI
(kube_deployment_status_replicas_available / kube_deployment_spec_replicas * 100) * on(namespace, deployment) group_left(maintainer) label_replace(kube_deployment_labels, "maintainer", "$1", "label_maintainer", "(.*)") * on(namespace, deployment) group_left(pager) label_replace(kube_deployment_labels, "pager", "$1", "label_pager", "(.*)") * on(namespace, deployment) group_left(paging) label_replace(kube_deployment_labels, "paging", "$1", "label_paging", "(.*)") < 100
Is this formatting style acceptable? @codesome @juliusv @brian-brazil @beorn7
I'm writing my proposal based on this style.
from prometheus.
i wrote a rough prototype to preserve comments by augmenting the goyacc grammar rules a little. commit here haibeey@8855667
Rules file
recording rule in web ui
from prometheus.
That doesn't look right, the operators aren't on their own line
from prometheus.
oh yes. that commit doesn't do formmating. it is just to preserve comments in promql expr after evaluation for printing.
previous versions ignores all comments after evaluation.
from prometheus.
https://prometheus.io/docs/practices/rules/ uses the best practices.
Please don't enforce:
sum without (instance)(instance_path:requests:rate5m{job="myjob"})
over
sum(instance_path:requests:rate5m{job="myjob"}) without (instance)
from prometheus.
Having grouping modifiers first is the best practice, and is already how we print it - it's very difficult to read non-trivial expressions otherwise.
from prometheus.
I think everyone is entitled to its own opinion about this, I personally find the "best practice" to be hideous, but, as all syntaxes are legal, please do not enforce one over the others.
from prometheus.
Yeah, gofmt
's formatting is nobody's favorite, but at least it creates one consistent style rather than many different ones, which is a value in itself.
from prometheus.
As a side note, this is actively been sought after for the GSoC. @haibeey I see that you are also interested in GSoC. While I appreciate the upfront work that you have put now, I would appreciate some open discussions and design sharing, in the GSoC proposal or otherwise, for easier coordination of projects in GSoC :)
from prometheus.
Comment preserving is already supported very well in the new go-yamlv3 library. The implementation is so flawless that there is no loss of comments. Hence, we can keep preserving mechanism independent from the Prometheus code without any worries.
from prometheus.
alright. i guess it is time share my proposal draft GSoC proposal .
Reviews would be appreciated before the final draft. Thanks
from prometheus.
@Harkishen-Singh There are two levels of comments: YAML comments and comments within a PromQL expression. #21 (comment) was talking about the PromQL ones (but YAML ones would be good to preserve as well).
from prometheus.
Comment preserving is already supported very well in the new go-yamlv3 library. The implementation is so flawless that there is no loss of comments. Hence, we can keep preserving mechanism independent from the Prometheus code without any worries.
During evaluation of a PromQL expression is when the comments are removed.
That said i think the formatting is not only to be done in rules files but in some places too like the web ui. is this true @juliusv ?
So as pointed out by @slrtbtfs most of the formatting and preserving would be done in the Printer module.
from prometheus.
Related Issues (20)
- Prometheus high memory usage(>960Gib) with 1200+ ServiceMonitors HOT 12
- PromQL/storage: Require selectors to always return matching results HOT 6
- Implement New Metric Type for Constant Value Metrics to Optimize Storage in Prometheus HOT 2
- Allow configuration of partial evaluation strategies for rule group evaluation failures
- add support for http2 over tcp aka h2c
- Prometheus scraped pod metrics is not matching with kubectl top pod result
- promql: setting multiple regexp's for `__name__` doesn't work anymore HOT 5
- New Prometheus UI HOT 2
- Update oauth2 options description HOT 1
- CPU, RAM, Storage Requirements for Prometheus (Storage answered)
- Promote new-service-discovery-manager HOT 2
- puppetdb_sd_configs - TTL for resources HOT 1
- Eager compaction of stale series
- prometheus v2.49.1 has too many wal files ,wal replay too slow
- confuse of function quantile HOT 1
- Random scrape targets get stuck HOT 11
- Proposal: PromQL arithmetic with default value HOT 27
- [Docs] Hidden command line flag HOT 3
- Pause and Resume Rule Group Evaluation HOT 8
- Metadata API JSON respones broken after commit 8065bef HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from prometheus.