terrencepreilly / darglint Goto Github PK
View Code? Open in Web Editor NEWA python documentation linter which checks that the docstring description matches the definition.
License: MIT License
A python documentation linter which checks that the docstring description matches the definition.
License: MIT License
Currently, if we have a docstring with no space in between sections, darglint fails hard:
def double(x):
"""Double the number.
Args:
x: The number to double.
Returns:
The number, doubled.
"""
return x * 2
Really, this should just be a warning (a stylistic error.)
Hey there, cool package!
I was wondering how difficult it would be to add a numpy docstring parser. Numpy docstrings are formatted similarly to Google docstrings. Is this something that I could help on?
If the following code has a docstring
def check_module_installed(name):
"""Temp
Args:
name (str): module name
Returns:
bool: Whether the module can be imported
"""
try:
__import__(name)
except ImportError:
return False
else:
return True
This snipped raises I202 Excess "Returns" in Docstring: + return
I have example code in my docstring like:
>>> gpu = '0' # The 0th GPU
This produces this error:
S001: s Failed to parse noqa statement. Expected "# noqa" but received "# The"
Quick fix: add # noqa: S001 to the docstring.
However darglint should not treat these inline comments as noqa statements.
Much like pydocstyle, a feature where internal documentation is ignored would be much appreciated.
It isn't as prominent in the the pydocstrings
documentation as it should be, but they added basic numpydoc checking in v2.0.0 (April 2017, under setting convention=numpy
) and Google style in v4.0.0 (July 2019, under setting convention=google
):
http://www.pydocstyle.org/en/latest/release_notes.html
Is it realistic to port any of darglint's code into pydocstrings instead of maintaining it as a separate library?
Cross reference #28.
I have this piece of code:
def __init__(self, node: ErrorNode, text: Optional[str] = None) -> None:
"""
Creates a new instance of an abstract violation.
Parameters:
node: violation was raised by this node. If applied.
text: extra text to format the final message. If applied.
"""
self._node = node
self._text = text
When I run flake8
with strictness = short
that's what I get:
./wemake_python_styleguide/violations/base.py
77:1 I101 Missing parameter(s) in Docstring: - text
"""
^
77:1 I101 Missing parameter(s) in Docstring: - node
"""
^
Looks like a false positive to me. What do you think?
darglint
overlaps with flake8-pep3101
in some error codes: https://github.com/gforcada/flake8-pep3101
S001: Describes that something went wrong in parsing the docstring.
S002: An argument/exception lacks a description.
I suggest to use DAL
, ARG
, or any other three letter codes for issues. flake8
supports it and the overlap chance is minimal.
Flake8 has a very useful flag: --isolated
that does not read any config values.
Currently darglint
still relies on its config value.
How to illustrate this problem?
# ex.py
def some(arg):
"""
Test.
Returns some value.
"""
return 'some'
And config:
# setup.cfg
[darglint]
strictness = long
Let's run it:
» flake8 ex.py --select=DAR
Done! No problems.
Let's run it with --isolated
:
» flake8 ex.py --select=DAR --isolated
Done? Why? The default value of darglint
should not allow this code.
Let's try to remove the config value from setup.cfg
:
» flake8 ex.py --select=DAR
with_lb.py
2:1 DAR101 Missing parameter(s) in Docstring: - arg
"""
^
2:1 DAR201 Missing "Returns" in Docstring: - return
"""
^
Boom! I expect the same to happen with --isolated
mode.
This is possibly the best docstring linter I've seen.
The only thing holding me back from using it is the lack of support for the numpydoc docstring format. If support for this is included, darglint will support all 3 major Python docstring formats.
This isn't necessarily a huge issue, since a higher verbosity level will output a helpful description. However, it might be helpful to have a list of all the error codes in the readme/docs.
darglint
raises an error (I203 Return type mismatch: ~Return: expected str but was
) when return type annotations are provided without a declared docstring return type. This is likely an oversight, this check should only fire if both a type annotation and docstring type declaration are present.
Failure example:
def foo() -> str:
"""Just a standard foobar.
Returns: Bar.
"""
return "bar"
First of all thank you for the great package. I would like to make a proposal for another validation that checks whether the docstring specifies the type for each parameter. This in addition to DAR203
which already checks that if type hints are used that the docstrings parameter types and the type hints match up. This new error would check for the specification of parameter types even in the absence of type hints.
Personally this would help my projects a lot since we don't use type hints in our function defintions, but we do want to enforce that types are documented in the docstring. I've taken the liberty to open #48 for this introducing DAR104
which reads The docstring parameter has no type specified
that may hopefully be a start for a discussion.
This isn't super critical, but it looks like error reporting is repeated?
Function:
def get_envlist(ini, factors):
"""Get the list of env names from the tox config that match the factors.
See `match_envs` for more details on env name matching.
Args:
ini: The parsed tox ini file. This should be the `._cfg` attribute of
the `config` passed to the `tox_configure` hook.
factors: The list of env factors to match against.
"""
...
flake8+darglint output
lint run-test: commands[0] | flake8 src
src/tox_factor/factor.py:5:1: I201 Missing "Returns" in Docstring: - return
src/tox_factor/factor.py:5:1: I201 Missing "Returns" in Docstring: - return
ERROR: InvocationError for command /Users/bagel/Documents/projects/tox-factor/.tox/lint/bin/flake8 src (exited with code 1)
The repo is currently private, but I can link/reproduce in a bit when I make it public (is about to go live).
The tox.ini
indicates that this package is tested against python 3.4, 3.5, and 3.6, however typing.Deque
was only added in 3.6.1, so its import breaks these older versions.
There are quite popular tools to lint existing docstrings:
flake8-docstrings
flake8-rst-docstrings
It would be awesome to have a note in the README if this project is compatible with them or not.
It would be also awesome to include a test of some kind if they should be compatible.
flake8 and similar tools return non-zero exit codes when they encounter errors. These failure codes are useful, since they can in turn be used to fail a build in CI. e.g., a user submitted an incorrectly formatted docstring, so fail the build.
cat foo.py
def foo(arg=None):
"""Some summary.
Args:
arg (NoneType, optional): Some description here.
"""
pass
Produces this output:
foo.py:foo:2: S001: s Exected type TokenType.COLON, but was TokenType.WORD: "optional)": You are either missing a colon or have underindented the second line of a description.
I was always under the impression that , optional):
was allowed for Google-style but I could be wrong, because I got that idea from sphinx-napoleon and napoleon may be taking some liberties.
http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html
As an example, this is how it might render, in Sphinx: http://ways.readthedocs.io/en/latest/ways.html#ways.get_action
Complains about newline between list of Args and Returns. https://github.com/google/styleguide/blob/gh-pages/pyguide.md#383-functions-and-methods has an example that is structured like this.
def f(x):
"""summary
more text
here
yeeeee
Args:
x: something
Returns:
something else
"""
pass
$ darglint -v 2 /tmp/d.py
/tmp/d.py:f:10: S001: Syntax error: s Unable to parse word: expected TokenType.WORD but received TokenType.NEWLINE
PS: Thanks for this awesome tool!
This docstring
"""Test.
Args:
input (:obj:`DataFrame <pandas.DataFrame>`, \
:obj:`ndarray <numpy.ndarray>`, list): test
"""
Raises this issue: S001 Syntax error: s Unable to parse word: expected TokenType.WORD but received TokenType.NEWLINE
The line continuation is used to respect PEP-8 guidelines.
I'm trying out wemake-python-styleguide, and part of its configuration is to enable low-number checks from darglint. Given the following definition, DAR101 requires that the arg specification in the docstring starts with an asterisk, which is the beginning of inline emphasis in ReST.
def collapse_many_logs(*args: typing.List[str]) -> typing.List[str]:
"""Concatenate an arbitrary number of lists, with a separator.
Args:
args: Any number of lists of strings.
Returns:
A list of strings made up of the input lists, skipping empty lists and
putting a separator between the remainder.
"""
Linting the above results in
rolls/output.py:22:1: DAR101 Missing parameter(s) in Docstring: - *args
rolls/output.py:23:1: DAR102 Excess parameter(s) in Docstring: + args
If I add the asterisk, the checks are satisfied. (But then flake8-rst-docstrings flags it as invalid ReST.)
This output was produced with version 1.1.0.
I seem to be getting this weird exception that I've isolated to darglint, it doesn't cause any issues but thought I'd bring it up in case it was.
flake8 throws:
__init__: 'Raise' object has no attribute 'ecx'
When I run flake8 with the highest verbosity, I am still unable to trace the error.
Here is a minimum example that raises the error:
class NewException(Exception):
pass
def wrapper():
class SomeClass():
def __init__(self, *args, **kwargs):
try:
raise NewException
except Exception as e:
raise type(e)(
str(e) + " Custom Message"
)
return SomeClass
While not explicitly stated in the Google style guide, the following way of formatting argument descriptions works with all tools I have tested:
Args:
foo:
Description text
However, darglint complains with a DAR101 about a missing parameter documentation.
Hi! I was just wondering if you are planning a 0.5.3 release soon. If not, can I request one from the current master? Just so pip sees all the latest changes :) Thanks!
def false_positive() -> None:
"""summary
:raises ValueError: description
"""
try:
raise ValueError("233")
except ValueError as e:
raise e from None
DAR401 Missing exception(s) in Raises section: -r e
Hi.
I use darglint with flake8:
> poetry run darglint --version
1.1.2
> poetry run flake8 --version
3.7.9 (flake8-annotations-complexity: 0.0.2, flake8-bandit: 2.1.2, flake8-broken-line: 0.1.1, flake8-bugbear: 19.8.0, flake8-comprehensions: 3.2.2, flake8-darglint: 0.4.1, flake8-debugger: 3.2.1, flake8-docstrings: 1.5.0, pydocstyle: 5.0.2, flake8-eradicate: 0.2.4, flake8-executable: 2.0.3, flake8-print: 3.1.4, flake8-string-format: 0.2.3, flake8_builtins: 1.4.1, flake8_coding: 1.3.2, flake8_commas: 2.0.0, flake8_isort: 2.3, flake8_pep3101: 1.2.1, flake8_quotes: 2.1.1, logging-format: 0.6.0, mccabe: 0.6.1, naming: 0.9.1, pycodestyle: 2.5.0, pyflakes: 2.1.1, radon: 4.1.0, rst-docstrings: 0.0.12, wemake-python-styleguide: 0.13.4) CPython 3.7.6 on Linux
When i start checking code like this (there should be no characters after return):
def some_fn():
"""Some fn.
:return x: some value
"""
return 3
I have the traceback:
Traceback (most recent call last):
File "/usr/lib64/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(*args, **kwds))
File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 666, in _run_checks
return checker.run_checks()
File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 598, in run_checks
self.run_ast_checks()
File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 507, in run_ast_checks
text=text,
File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 410, in report
error_code, text = text.split(" ", 1)
ValueError: not enough values to unpack (expected 2, got 1)
The reason in https://github.com/terrencepreilly/darglint/blob/master/darglint/flake8_entry.py#L58. If some exception happen - checker return text of exception.
But in many places darglint have assert checks like:
https://github.com/terrencepreilly/darglint/blob/master/darglint/docstring/sphinx.py#L172
In this case, for report message checker return str(AssertionError()) == ''
But flake expect something like "ERRORCODE errormessage" string
Hi, thanks for this project! It is awesome!
My question is about two different modes of docstrings that I use in my apps.
I can either write a short one like:
class SimpleViolation(BaseViolation):
"""Violation for cases where there's no associated nodes."""
_node: None
def __init__(self, node=None, text: Optional[str] = None) -> None:
"""Creates new instance of simple style violation."""
super().__init__(node, text=text)
Or full one like here:
class BaseViolation(object):
"""
Abstract base class for all style violations.
It basically just defines how to create any error and how to format
this error later on.
Each subclass must define ``error_template`` and ``code`` fields.
Attributes:
error_template: message that will be shown to user after formatting.
code: violation unique number. Used to identify the violation.
"""
error_template: ClassVar[str]
code: ClassVar[int]
def __init__(self, node: ErrorNode, text: Optional[str] = None) -> None:
"""
Creates new instance of abstract violation.
Parameters:
node: violation was raised by this node. If applied.
text: extra text to format the final message. If applied.
"""
self._node = node
self._text = text
Full source here: https://github.com/wemake-services/wemake-python-styleguide/blob/master/wemake_python_styleguide/violations/base.py
The problem is that I get an error from the short version:
./wemake_python_styleguide/violations/base.py:168:1: I101 Missing parameter(s) in Docstring: - text
./wemake_python_styleguide/violations/base.py:168:1: I101 Missing parameter(s) in Docstring: - node
What do I suggest? Allow short docstring (= docstrings without ANY arguments/returns/raises/etc declarations) or require to write a full one (in case ANY formatters are found).
It might be a configuration option turned off by default.
How does it sound to you?
I am not sure this is a bug or by design. If a summary line is too long darglint will not complain about later errors, in this case an incomplete "Returns".
def f(x):
"""summary
is too long
Args:
x: x
Returns"""
return ""
$ darglint -v 2 /tmp/r.py
/tmp/r.py:f:3: S001: Syntax error: s Expected blank line after short description, but found TokenType.WORD: 'is'.
def f(x):
"""summary
Args:
x: x
Returns"""
return ""
$ darglint -v 2 /tmp/r.py
/tmp/r.py:f:7: S001: Syntax error: s Unable to parse colon: stream was unexpectedly empty.
When the noqa is placed outside of the docstring, it is not respected as it works in pydocstyle http://www.pydocstyle.org/en/2.1.1/usage.html. Would greatly appreciate if this could be implemented as otherwise the noqa shows up in the html of the doc.
Have you considered adding a flake8 extension? If you're not aware, flake8 is a general framework for executing all sorts of style checks.
An example of a similar plugin would be flake8-docstrings
, which integrates pycodestyle
with flake8
.
As an added benefit, this plugin would indirectly solve the exitcode handling issue, as well as handling file inputs (eg, no longer necessary to use xargs to feed multiple packages to darglint
.
Hello Terrence,
I googled a lot and found this beautiful library, parsing the docstring with EBNF.
Quick question: are there plans to implement a formatter that automatically formats the docstring?
For a random docstring, it is almost impossible to get any useful output. But for an already structured docstring, it may be good to automatically wrap lines, fix some indentation (if possible), remove extra whitespaces, newlines, etc.
What do you think?
def f(foo):
"""foobar foobars foobar
Args:
foo: foobar
Returns:
bar
"""
return "bar"
results in a parsing error:
$ darglint -v 2 f.py
f.py:f:6: S001: Syntax error: s Unable to parse indent: expected TokenType.INDENT but received TokenType.WORD
Note the single word return description. If I add another word, it works fine:
def f(foo):
"""foobar foobars foobar
Args:
foo: foobar
Returns:
bar bar
"""
return "bar"
$ darglint -v 2 f.py
$
I am on 0.5.1.
Having a multiline return in sphinx is broken: any line breaks result in no return being parsed.
See #56 (comment) for an example.
I have installed darglint (0.6.0)
, that's what I see:
» flake8 .
"flake8-darglint" failed during execution due to "__init__() missing 1 required positional argument: 'strictness'"
Run flake8 with greater verbosity to see more details
I don't have any extra config. When I specify:
[darglint]
strictness=short
It works as expected.
I guess we can make short
the default one. Related to #26
"Sections should be indented two spaces, except for the heading." says https://google.github.io/styleguide/pyguide.html#383-functions-and-methods (and I wonder why it says two while their example uses four...)
Consider
def f(foo):
"""foobar foobars foobar
Args:
foo: foobar
Returns:
foobarred bar
"""
return "bar"
I would have expected a error about the messed up indentation. But got a successful linting run instead:
$ darglint -v 2 f.py
$
I am on 0.5.1.
Just a small message to say your tool is a really good idea. I wasn't able to find anything like it and I think it's essential. Keep up the good work! I hope you're still working on it.
according to sphinx document(search napoleon_use_param)
def func(a, b, **kwargs):
"""example function
:param a: parameter a
:param b: parameter b
:keyword arguments: * **arg1** (*str*) --
Description of `arg1`
* **arg2** (*int, optional*) --
Description of `arg2`, defaults to 0
"""
pass
darglint tmp/a.py
tmp/a.py:func:8: DAR101: - **kwargs
And, I think it should be :param kwargs: description
instead of :param **kwargs: description
darglint trade second one as correct doc
Looking through the git history, there are a few versions that are missing tags. Namely, 0.2.0
and later. Additionally, there are a few missing releases on PyPI (0.1.0
, 0.2.0
, and 0.3.1
).
Thanks for the awesome tool. I have started using it on a Python project and am very happy with it. There are however a couple of things I'd like to report to see how difficult they are to fix.
Talking only Google docstring style.
If I understand it right, there should be a blank line before the closing triple quotes, however it is not reported.
def do_sum1(var1, var2):
"""Sums two numbers.
Description is here Description is here Description is here Description is here.
Description is here Description is here Description is here Description is here.
Description is here Description is here Description is here Description is here.
Args:
var1: Variable 1
var2 (float): Variable 2
Returns:
int: sum of two ints
Raises:
ValueError: If passing non integer values
"""
if not all(
[ isinstance(var, int) for var in [var1, var2]]
):
raise ValueError("Only integers are supported")
return var1 + var2
I except the docstring to be
......
Raises:
ValueError: If passing non integer values
"""
When I have a method like this:
@abstractmethod
def foo(self, bar: str) -> int:
"""foo.
Args:
bar (str): bar.
Returns:
int: spam.
"""
darglint
gives me a DAR202
, because I don't have a return
statement in my function definition. Event if I have it in derived classes. This is a bug, which should be fixed, since we don't define abstract methods, we just declare them.
I don't see anyone talking about this so it has to be something I'm doing wrong...
Using pyenv to provide 3.8.1 with pipenv.
With a file containing:
def tryme():
pass
Running darglint doesn't complain...
$ pipenv install --dev darglint
$ pipenv run darglint dummy.py
$ echo $?
0
In my real project I also have flake8 installed and it notes that darglint is installed, but also doesn't complain about the lack of docstrings.
Hi! First off, I love this linter so thank you for the work on it. FYI I actually just set up a plugin for SublimeText integration here. On to the actual question:
Do you know of a formal style for exception names in the Raises section of the docstrings in the google style guide? When using a builtin exception I use the normal style of:
Raises:
BuiltinException: blah
But when I have a function that raises some nonstandard exception class (sometimes which is in a 3rd party module) I usually use the full path to the exception since there may be duplicates in various modules and clarity in docs is nice imo. IE:
Raises:
foo.bar.SomeOtherException: blah
I'm really not sure if this goes against the official style guide, the docs ive seen seem to be a little unclear there. If it doesn't go around it, how would you feel about adding functionality to match a raise SomeOtherException
to the foo.bar.SomeOtherException
declaration? I don't mind sending a PR but I just want to check first. Thanks
@abstract.abstractmethod
def __init__(self, config: dict):
"""
:param config: config dict user defined in config file.
"""
@abstract.abstractmethod
def __init__(self, config: dict):
"""
:param config: config dict user defined in config file.
"""
DAR101 Missing parameter(s) in Docstring: - config
worked with
@abstract.abstractmethod
def __init__(self, config: dict):
- """
+ """anything
:param config: config dict user defined in config file.
"""
I want to help you with some minor documentation features: like code hightlight, typos, etc.
But, for some reason there are two README
files: rst
and md
. Why?
Which one can be deleted?
Currently darglint completely ignores async functions.
It seems that darglint currently does not support the Google style documentation of documenting class params in the class documentation rather than the __init__.
Is this planned to be fixed? Not sure how technically feasible this would be based on the conversation from #21.
E.g.
class Test(object):
"""Tests darglint.
Params:
p1 (int): param 1
p2 (str): param 2
"""
def __init__(self, p1, p2):
self.p1 = p1
self.p2 = p2
I have this code:
def optional(filename: str) -> str:
"""
This functions is used for compatibility reasons.
It masks the old `optional` class with the name error.
Now `invalid-name` is removed from `pylint`.
Args:
filename: the filename to be optional.
Returns:
New instance of :class:`_Optional`.
"""
return _Optional(filename)
class _Optional(str):
"""
Wrap a file path with this class to mark it as optional.
Optional paths don't raise an :class:`IOError` if file is not found.
"""
...
Full source: https://github.com/sobolevn/django-split-settings/blob/master/split_settings/tools.py#L22
But, darglint
raises this problem:
./split_settings/tools.py
31:1 DAR203 Return type mismatch: ~Return: expected str but was New instance of
I am pretty sure that this is a false positive, because I return str
subclass and mypy
is happy with that.
Checking types is a very complex and ungrateful job in python. Maybe we can remove this feature from this tool by default? And make it configurable. Like so:
[darglint]
typecheck = true
And darglint --typecheck
Because it can work for simple cases, but will fail for most ones.
» flake8 .
"flake8-darglint" failed during execution due to "Unrecognized stricteness amount. Should be one of ['SHORT_DESCRIPTION', 'LONG_DESCRIPTION', 'FULL_DESCRIPTION']"
Run flake8 with greater verbosity to see more details
I have these settings in my setup.cfg
:
[darglint]
# darglint configuration: https://github.com/terrencepreilly/darglint
strictness = long
The PR with the fix is incoming.
My source file are utf-8
encoded, and I'm using this tool on windows, but I got 'gbk' codec can't decode byte 0x80 in position 608: illegal multibyte sequence
.
consider adding chardet
support?
Rich Structured Text format requires a blank line after a definition list. However, darglint splits the arguments section in two if there is a blank line.
Possible solutions:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.