Comments (26)
Run buf check lint --error-format=config-ignore-yaml
, which will print out a per-file per-checker ignore config you can copy/paste into the buf.yaml file.
We disable the comment ability as this would allow individual developers in large organizations to disregard lint rules, which defeats the purpose.
We could theoretically have a mode where we enable comment ignores as an option through the config, but I’m not sure why the above doesn’t work? Let us know.
from buf.
It doesn't really work in a large repository where the generated file is thousands of lines long and it's a pain to modify it because:
- Large size means IDEs will be slow
- Also means painful merge conflicts
- And there will be a lot of merge conflicts when all developers start fixing lint errors
- Also the whole repo will depend on that file so when it changes the all protos in the repo will be rebuilt and relinted, which will just take too much time for all developers and CI. And it's for each change, and there will be thousands of changes to slowly fix everything
Ultimately I like the idea, but it doesn't pass the reality check.
from buf.
Allowing comment ignores causes the exact same issue - it allows new bad code to not be rejected, except with no visibility at the top-level.
* Large size means IDEs will be slow
What IDE would be slow for a 1000-3000 line YAML file?
* Also means painful merge conflicts
Not sure this would be that bad, and allowing comment-driven ignores is a bigger issue (in that it in effect defeats the purpose of the linter).
* And there will be **a lot** of merge conflicts when all developers start fixing lint errors
* Also the whole repo will depend on that file so when it changes the all protos in the repo will be rebuilt and relinted, which will just take too much time for all developers and CI. And it's for each change, and there will be thousands of changes to slowly fix everything
It takes 0.8s to lint all 2300 files in googleapis on a four core laptop, how many files are we talking here?
from buf.
Ultimately I like the idea, but it doesn't pass the reality check.
We didn't make this decision without experience, in fact it was based on our real-world experience seeing comment-driven ignores effectively turn off enforced linting, resulting in inconsistent (and often incorrect) Protobuf usage across teams. Despite any downsides (merges, for example), buf check lint --error-format=config-ignore-yaml
provides the ability to migrate over time, and given that Buf takes less than a second to lint thousands of files, we don't see this holding up CI processes (in fact, Buf is usually the fastest test).
from buf.
Unfortunately buf check lint
can not work in our repo for various reasons and I have to use it as a protoc
plugin.
Still, I don't understand the issue here. Developers can just not enable the linter altogether and be done with it. As long as developers want the linter you can be sure they will use it. In my case I'm building the tooling and it's up to each individual team to use it or not.
In any case, of course it's your decision, thanks for a prompt reply!
from buf.
We can consider enabling it in the future as an option if you'd like - I can see some developers wanting it, although it's extremely not recommended. I'd really encourage trying out the buf.yaml
workflow, even if you're (I'm assuming :-) ) using Bazel/Blaze - I don't think it'll be as painful as you think, and way less painful than watching devs all decide their lint exception is correct.
Happy to rename this issue to be something like "enable option to allow commen-driven ignores", because then at least, a Protobuf monorepo owner can have their top-level OWNERS file disallow modifications to the buf.yaml
, including the enabling of this option. How does that sound?
from buf.
I do indeed use Bazel and I indeed can't force everyone to use the linter. In fact, I need to provide a tool for each team to configure it as they wish but with recommended defaults. We try not to be authoritarian in that respect :)
from buf.
Are you looking for something like:
// buf:ignore MESSAGE_PASCAL_CASE
message who_uses_snake_case_for_protobuf {}
Feel free to propose an alternative format, this is what makes the most sense to me though.
Re: authoritarianism with linters - our opinion is there are way too many options available, and creativity in your APIs should not come down to style, instead the creativity should come in the design. And we want to encourage this behavior. We thought that the --error-format=config-ignore-yaml
option would be helpful for migrations (and a little inside baseball, that flag is actually single-handedly messing with our package dependency tree until we finish paying down some tech debt, but alas). Additionally, the comment-driven ignores have been a massive problem in organizations we've seen. But being user-friendly, we could enable it as an option.
from buf.
Looks perfect!
One important feature here is that a user should be able to run the build, parse the errors and generate the comments to send out in a few PRs.
For some more context, we have some weird proto-like DSLs that are not really protos but use protoc for parsing the DSL. It's a big mess, but saves many-many klocs of boring code without creating a zoo of DSLs. One size can't really fit all :)
from buf.
I'm in the middle of a 10000 line refactor on the internal repo (this repo is actually just a mirror of the public code in the internal repo), so I'll keep this in mind and see how hard it is to add. Knowing the codebase, it's likely trivial actually.
One note - we're likely going to add a required version
tag to each config file and/or the lint and breaking sections, and hopefully this will be the only breaking change before 1.0 - we don't ever want to make a breaking change, even during beta - but this tag will make it so that the same lint rule set will be the same forever, while allowing us to iterate on the rule sets for the future.
from buf.
Good idea! Thanks for the warning! Although you could just say that the default version is the current one and that won't be a breaking change any more. I'm assuming you can make version
optional
:)
from buf.
Yea that is an option, although if we do that, then new users who don't have a config file yet will always get the beta rule set. It's a tough spot, but we figure since we are technically beta, it's fair to ask users for this single change, so that we don't have changes in the future - we don't think it's fair that linters change their rules sets on upgrade (which happens all the time), you should be able to rely on buf producing the same errors now as in 10 years.
FYI (note for us), we likely want to make the comments scoped to the linter, i.e.:
// buf:check:lint:ignore MESSAGE_PASCAL_CASE
So we could theoretically add comment controls for the breaking change detector (or other functionality) in the future.
from buf.
Personally I'm listing all rules explicitly anyway so unless you delete them I won't be affected at all.
from buf.
@Monnoroch see #73
There's instructions in there, I will keep them updated. We need to:
- Add tests.
- Decide on the prefix. For now, it's
buf:lint:ignore
- Decide on the configuration option name.
To install:
go get github.com/bufbuild/buf/cmd/buf@comment-driven-ignores
Or in your case:
go get github.com/bufbuild/buf/cmd/protoc-gen-buf-check-lint@comment-driven-ignores
Let me know what you think.
from buf.
Thank you, will definitely try it out!
from buf.
Test added, you can see the file for now at https://github.com/bufbuild/buf/blob/comment-driven-ignores/internal/buf/bufcheck/buflint/testdata/comment_driven_ignores/a.proto
from buf.
@Monnoroch I think this is ready to go now, the open questions:
- The prefix
buf:lint:ignore
- we think this is the best we have. The idea is this is scopedPROGRAM:FUNCTION:DIRECTIVE
. Alternatives were justnolint
, but this could overlap with other programs,lint:ignore
, but same thing,buf:check:lint:ignore
to match the CLI, but the fact thatlint
is scoped undercheck
in the CLI is just an implementation detail, andbuf::lint::ignore
, but this seemed to C++-ey/Ruby-ey. - The configuration option
allow_comment_driven_ignores
- This might be better asallow_comment_ignores
, what do you think? This is less of an issue as we can version the configuration file.
from buf.
I'm happy with your naming choices. A few small suggestions though:
- In addition to
buf:lint:ignore
you could addbuf:lint:warn
that would print the warning but still return 0 exit code - You could also add
buf:lint:ignore ALL
to ignore all lints - You could also add
buf:lint:ignore_file X
to ignore a lint for the whole file (to be placed on top)
Love the comment in the test file though xD
from buf.
Interesting, I think we want to keep it simple for now though, it’s easier to add things later than take things out
from buf.
Could be open to extending later?
On the specific suggestions above:
- Not big on differentiating warning/error - buf check lint is meant to be a binary proposition
- If you’re ignoring all linters in a file, then I think the argument for comment-driven ignores breaks down, and it should be added to config
- I’m not sure I understand difference between 2 and 3, can you explain?
from buf.
Of course, please treat those as feature requests to be implemented whenever.
- There are some violations that can not be fixed, such as RPC names. Changing those is incompatible for some clients. I would like to just ignore those as there's nothing we can do for that code. Fixing other is possible and I would like builds to remind the user that something can be improved though
- (+3)
ALL
means all lints on THAT line,ignore_file
means this one lint for the whole file. And depends on the argument. Linting the file partially argument does indeed break. Having distributed configuration vs one huge file the whole repo depends on does not break. Eliminating bottleneck "god" dependencies is very important to me.
from buf.
Ahhh got it. Ok, both interesting and valid cases for sure. Let us think about em - I think what we will do is get the PR as is merged, do a release, and sit on those for a bit? Want to think about it for a bit.
Also I think the option should be allow_comment_ignores now, thoughts?
from buf.
LGTM on all fronts :)
from buf.
This is published as v0.15.0, and the documentation on the website is updated!
from buf.
Thank you for such a quick response!
from buf.
We would generally want to be quicker haha, feel free to email [email protected] if you want to discuss more!
from buf.
Related Issues (20)
- csharp plugin fails in managed mode with same filename in different versions HOT 1
- Specifying Folders In `buf generate --path` Arguments Causes Massive Performance Hit HOT 3
- Feature request: Option on buf convert to set `DiscardUnknown` to false when unmarshalling input
- Question: Already defined dependency makes `buf generate` failed
- [feat] Option to generate enum with full prefix HOT 2
- mod update panics when dependencies are from multiple registries HOT 1
- Failure: 403 Forbidden HOT 4
- `buf generate` does not use correct `buf.lock` when using `--config` with `git` source HOT 2
- Generate implicit dependencies HOT 4
- unknown extension grpc.gateway.protoc_gen_openapiv2.options.openapiv2_schema HOT 2
- WARN bufimagebuild File imports not found in your local dependencies HOT 4
- Fails to install on BSDs HOT 1
- Field plugin not found in type bufgen.ExternalPluginConfigV1 HOT 1
- Stubs are stored under the initial path where proto files can be found HOT 2
- Clarification on RPC and Request/Response Names HOT 2
- Error installing buf via go install HOT 1
- buf generate does not appear to respect buf.yaml excludes rules HOT 7
- `ignore`/`build.excludes` with glob/regex/pattern HOT 1
- language_version not specified in .pre-commit-hooks.yaml HOT 3
- buf generate get an error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from buf.