terrencepreilly / darglint Goto Github PK

Currently, if we have a docstring with no space in between sections, darglint fails hard:

def double(x):
    """Double the number.

    Args:
        x: The number to double.
    Returns:
        The number, doubled.

    """
    return x * 2

Really, this should just be a warning (a stylistic error.)

Hey there, cool package!

I was wondering how difficult it would be to add a numpy docstring parser. Numpy docstrings are formatted similarly to Google docstrings. Is this something that I could help on?

If the following code has a docstring

def check_module_installed(name):
    """Temp

    Args:
        name (str): module name

    Returns:
        bool: Whether the module can be imported

    """
    try:
        __import__(name)
    except ImportError:
        return False
    else:
        return True

This snipped raises I202 Excess "Returns" in Docstring: + return

I have example code in my docstring like:
>>> gpu = '0' # The 0th GPU

This produces this error:
S001: s Failed to parse noqa statement. Expected "# noqa" but received "# The"
Quick fix: add # noqa: S001 to the docstring.

However darglint should not treat these inline comments as noqa statements.

Much like pydocstyle, a feature where internal documentation is ignored would be much appreciated.

It isn't as prominent in the the pydocstrings documentation as it should be, but they added basic numpydoc checking in v2.0.0 (April 2017, under setting convention=numpy) and Google style in v4.0.0 (July 2019, under setting convention=google):

http://www.pydocstyle.org/en/latest/release_notes.html

Is it realistic to port any of darglint's code into pydocstrings instead of maintaining it as a separate library?

Cross reference #28.

I have this piece of code:

    def __init__(self, node: ErrorNode, text: Optional[str] = None) -> None:
        """
        Creates a new instance of an abstract violation.

        Parameters:
            node: violation was raised by this node. If applied.
            text: extra text to format the final message. If applied.

        """
        self._node = node
        self._text = text

When I run flake8 with strictness = short that's what I get:

./wemake_python_styleguide/violations/base.py

  77:1     I101  Missing parameter(s) in Docstring: - text
  """
  ^

  77:1     I101  Missing parameter(s) in Docstring: - node
  """
  ^

Real source: https://github.com/wemake-services/wemake-python-styleguide/blob/master/wemake_python_styleguide/violations/base.py#L76-L86

Looks like a false positive to me. What do you think?

darglint overlaps with flake8-pep3101 in some error codes: https://github.com/gforcada/flake8-pep3101

S001: Describes that something went wrong in parsing the docstring.
S002: An argument/exception lacks a description.

I suggest to use DAL, ARG, or any other three letter codes for issues. flake8 supports it and the overlap chance is minimal.

Flake8 has a very useful flag: --isolated that does not read any config values.

Currently darglint still relies on its config value.

How to illustrate this problem?

# ex.py
def some(arg):
    """
    Test.

    Returns some value.
    """
    return 'some'

And config:

# setup.cfg
[darglint]
strictness = long

Let's run it:

» flake8 ex.py --select=DAR

Done! No problems.

Let's run it with --isolated:

» flake8 ex.py --select=DAR --isolated

Done? Why? The default value of darglint should not allow this code.

Let's try to remove the config value from setup.cfg:

» flake8 ex.py --select=DAR           

with_lb.py

  2:1      DAR101 Missing parameter(s) in Docstring: - arg
  """
  ^

  2:1      DAR201 Missing "Returns" in Docstring: - return
  """
  ^

Boom! I expect the same to happen with --isolated mode.

This is possibly the best docstring linter I've seen.

The only thing holding me back from using it is the lack of support for the numpydoc docstring format. If support for this is included, darglint will support all 3 major Python docstring formats.

This isn't necessarily a huge issue, since a higher verbosity level will output a helpful description. However, it might be helpful to have a list of all the error codes in the readme/docs.

darglint raises an error (I203 Return type mismatch: ~Return: expected str but was ) when return type annotations are provided without a declared docstring return type. This is likely an oversight, this check should only fire if both a type annotation and docstring type declaration are present.

Failure example:

def foo() -> str:
    """Just a standard foobar.

    Returns: Bar.
    
    """
    return "bar"

First of all thank you for the great package. I would like to make a proposal for another validation that checks whether the docstring specifies the type for each parameter. This in addition to DAR203 which already checks that if type hints are used that the docstrings parameter types and the type hints match up. This new error would check for the specification of parameter types even in the absence of type hints.

Personally this would help my projects a lot since we don't use type hints in our function defintions, but we do want to enforce that types are documented in the docstring. I've taken the liberty to open #48 for this introducing DAR104 which reads The docstring parameter has no type specified that may hopefully be a start for a discussion.

This isn't super critical, but it looks like error reporting is repeated?

Function:

def get_envlist(ini, factors):
    """Get the list of env names from the tox config that match the factors.

    See `match_envs` for more details on env name matching.

    Args:
        ini: The parsed tox ini file. This should be the `._cfg` attribute of
            the `config` passed to the `tox_configure` hook.
        factors: The list of env factors to match against.

    """
    ...

flake8+darglint output

lint run-test: commands[0] | flake8 src
src/tox_factor/factor.py:5:1: I201 Missing "Returns" in Docstring: - return
src/tox_factor/factor.py:5:1: I201 Missing "Returns" in Docstring: - return
ERROR: InvocationError for command /Users/bagel/Documents/projects/tox-factor/.tox/lint/bin/flake8 src (exited with code 1)

The repo is currently private, but I can link/reproduce in a bit when I make it public (is about to go live).

The tox.ini indicates that this package is tested against python 3.4, 3.5, and 3.6, however typing.Deque was only added in 3.6.1, so its import breaks these older versions.

There are quite popular tools to lint existing docstrings:

• flake8-docstrings
• flake8-rst-docstrings

It would be awesome to have a note in the README if this project is compatible with them or not.

It would be also awesome to include a test of some kind if they should be compatible.

flake8 and similar tools return non-zero exit codes when they encounter errors. These failure codes are useful, since they can in turn be used to fail a build in CI. e.g., a user submitted an incorrectly formatted docstring, so fail the build.

cat foo.py

def foo(arg=None):
    """Some summary.

    Args:
        arg (NoneType, optional): Some description here.

    """
    pass

Produces this output:

foo.py:foo:2: S001: s Exected type TokenType.COLON, but was TokenType.WORD: "optional)": You are either missing a colon or have underindented the second line of a description.

I was always under the impression that , optional): was allowed for Google-style but I could be wrong, because I got that idea from sphinx-napoleon and napoleon may be taking some liberties.
http://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html

As an example, this is how it might render, in Sphinx: http://ways.readthedocs.io/en/latest/ways.html#ways.get_action

Complains about newline between list of Args and Returns. https://github.com/google/styleguide/blob/gh-pages/pyguide.md#383-functions-and-methods has an example that is structured like this.

def f(x):
    """summary
    
    more text
    here
    yeeeee
    
    Args:
        x: something
        
    Returns:
        something else
    """
    pass

$ darglint -v 2 /tmp/d.py 
/tmp/d.py:f:10: S001: Syntax error: s Unable to parse word: expected TokenType.WORD but received TokenType.NEWLINE

PS: Thanks for this awesome tool!

This docstring

"""Test.

Args:
    input (:obj:`DataFrame <pandas.DataFrame>`, \
        :obj:`ndarray <numpy.ndarray>`, list): test

"""

Raises this issue: S001 Syntax error: s Unable to parse word: expected TokenType.WORD but received TokenType.NEWLINE

The line continuation is used to respect PEP-8 guidelines.

I'm trying out wemake-python-styleguide, and part of its configuration is to enable low-number checks from darglint. Given the following definition, DAR101 requires that the arg specification in the docstring starts with an asterisk, which is the beginning of inline emphasis in ReST.

def collapse_many_logs(*args: typing.List[str]) -> typing.List[str]:
    """Concatenate an arbitrary number of lists, with a separator.

    Args:
        args: Any number of lists of strings.

    Returns:
        A list of strings made up of the input lists, skipping empty lists and
        putting a separator between the remainder.
    """

Linting the above results in

rolls/output.py:22:1: DAR101 Missing parameter(s) in Docstring: - *args
rolls/output.py:23:1: DAR102 Excess parameter(s) in Docstring: + args

If I add the asterisk, the checks are satisfied. (But then flake8-rst-docstrings flags it as invalid ReST.)
This output was produced with version 1.1.0.

I seem to be getting this weird exception that I've isolated to darglint, it doesn't cause any issues but thought I'd bring it up in case it was.
flake8 throws:
__init__: 'Raise' object has no attribute 'ecx'

When I run flake8 with the highest verbosity, I am still unable to trace the error.

Here is a minimum example that raises the error:

class NewException(Exception):
    pass

def wrapper():
    class SomeClass():
        def __init__(self, *args, **kwargs):
            try:
                raise NewException
            except Exception as e:
                raise type(e)(
                    str(e) + " Custom Message"
                )
    return SomeClass

While not explicitly stated in the Google style guide, the following way of formatting argument descriptions works with all tools I have tested:

    Args:
        foo:
            Description text

However, darglint complains with a DAR101 about a missing parameter documentation.

Hi! I was just wondering if you are planning a 0.5.3 release soon. If not, can I request one from the current master? Just so pip sees all the latest changes :) Thanks!

Hi.

I use darglint with flake8:

> poetry run darglint --version
1.1.2

> poetry run flake8 --version
3.7.9 (flake8-annotations-complexity: 0.0.2, flake8-bandit: 2.1.2, flake8-broken-line: 0.1.1, flake8-bugbear: 19.8.0, flake8-comprehensions: 3.2.2, flake8-darglint: 0.4.1, flake8-debugger: 3.2.1, flake8-docstrings: 1.5.0, pydocstyle: 5.0.2, flake8-eradicate: 0.2.4, flake8-executable: 2.0.3, flake8-print: 3.1.4, flake8-string-format: 0.2.3, flake8_builtins: 1.4.1, flake8_coding: 1.3.2, flake8_commas: 2.0.0, flake8_isort: 2.3, flake8_pep3101: 1.2.1, flake8_quotes: 2.1.1, logging-format: 0.6.0, mccabe: 0.6.1, naming: 0.9.1, pycodestyle: 2.5.0, pyflakes: 2.1.1, radon: 4.1.0, rst-docstrings: 0.0.12, wemake-python-styleguide: 0.13.4) CPython 3.7.6 on Linux

When i start checking code like this (there should be no characters after return):

def some_fn():
    """Some fn.
    :return x: some value
    """
    return 3

I have the traceback:

Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 666, in _run_checks
    return checker.run_checks()
  File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 598, in run_checks
    self.run_ast_checks()
  File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 507, in run_ast_checks
    text=text,
  File "/home/user/.cache/pypoetry/virtualenvs/env/lib64/python3.7/site-packages/flake8/checker.py", line 410, in report
    error_code, text = text.split(" ", 1)
ValueError: not enough values to unpack (expected 2, got 1)

The reason in https://github.com/terrencepreilly/darglint/blob/master/darglint/flake8_entry.py#L58. If some exception happen - checker return text of exception.

But in many places darglint have assert checks like:

https://github.com/terrencepreilly/darglint/blob/master/darglint/docstring/sphinx.py#L172

In this case, for report message checker return str(AssertionError()) == ''
But flake expect something like "ERRORCODE errormessage" string

Hi, thanks for this project! It is awesome!

My question is about two different modes of docstrings that I use in my apps.
I can either write a short one like:

class SimpleViolation(BaseViolation):
    """Violation for cases where there's no associated nodes."""

    _node: None

    def __init__(self, node=None, text: Optional[str] = None) -> None:
        """Creates new instance of simple style violation."""
        super().__init__(node, text=text)

Or full one like here:

class BaseViolation(object):
    """
    Abstract base class for all style violations.

    It basically just defines how to create any error and how to format
    this error later on.

    Each subclass must define ``error_template`` and ``code`` fields.

    Attributes:
        error_template: message that will be shown to user after formatting.
        code: violation unique number. Used to identify the violation.

    """

    error_template: ClassVar[str]
    code: ClassVar[int]

    def __init__(self, node: ErrorNode, text: Optional[str] = None) -> None:
        """
        Creates new instance of abstract violation.

        Parameters:
            node: violation was raised by this node. If applied.
            text: extra text to format the final message. If applied.

        """
        self._node = node
        self._text = text

Full source here: https://github.com/wemake-services/wemake-python-styleguide/blob/master/wemake_python_styleguide/violations/base.py

The problem is that I get an error from the short version:

./wemake_python_styleguide/violations/base.py:168:1: I101 Missing parameter(s) in Docstring: - text
./wemake_python_styleguide/violations/base.py:168:1: I101 Missing parameter(s) in Docstring: - node

What do I suggest? Allow short docstring (= docstrings without ANY arguments/returns/raises/etc declarations) or require to write a full one (in case ANY formatters are found).
It might be a configuration option turned off by default.

How does it sound to you?

I am not sure this is a bug or by design. If a summary line is too long darglint will not complain about later errors, in this case an incomplete "Returns".

def f(x):
    """summary
    is too long
    
    Args:
        x: x
        
    Returns"""

    return ""

$ darglint -v 2 /tmp/r.py
/tmp/r.py:f:3: S001: Syntax error: s Expected blank line after short description, but found TokenType.WORD: 'is'.

def f(x):
    """summary
    
    Args:
        x: x
        
    Returns"""

    return ""

$ darglint -v 2 /tmp/r.py
/tmp/r.py:f:7: S001: Syntax error: s Unable to parse colon: stream was unexpectedly empty.

When the noqa is placed outside of the docstring, it is not respected as it works in pydocstyle http://www.pydocstyle.org/en/2.1.1/usage.html. Would greatly appreciate if this could be implemented as otherwise the noqa shows up in the html of the doc.

Have you considered adding a flake8 extension? If you're not aware, flake8 is a general framework for executing all sorts of style checks.

An example of a similar plugin would be flake8-docstrings, which integrates pycodestyle with flake8.

As an added benefit, this plugin would indirectly solve the exitcode handling issue, as well as handling file inputs (eg, no longer necessary to use xargs to feed multiple packages to darglint.

Hello Terrence,
I googled a lot and found this beautiful library, parsing the docstring with EBNF.

Quick question: are there plans to implement a formatter that automatically formats the docstring?

For a random docstring, it is almost impossible to get any useful output. But for an already structured docstring, it may be good to automatically wrap lines, fix some indentation (if possible), remove extra whitespaces, newlines, etc.

What do you think?

def f(foo):
    """foobar foobars foobar
    
    Args:
        foo: foobar
    Returns:
        bar
    """
    return "bar"

results in a parsing error:

$ darglint -v 2 f.py 
f.py:f:6: S001: Syntax error: s Unable to parse indent: expected TokenType.INDENT but received TokenType.WORD

Note the single word return description. If I add another word, it works fine:

def f(foo):
    """foobar foobars foobar
    
    Args:
        foo: foobar
    Returns:
        bar bar
    """
    return "bar"

$ darglint -v 2 f.py
$

I am on 0.5.1.

Having a multiline return in sphinx is broken: any line breaks result in no return being parsed.

See #56 (comment) for an example.

I have installed darglint (0.6.0), that's what I see:

» flake8 .                 
"flake8-darglint" failed during execution due to "__init__() missing 1 required positional argument: 'strictness'"
Run flake8 with greater verbosity to see more details

I don't have any extra config. When I specify:

[darglint]
strictness=short

It works as expected.
I guess we can make short the default one. Related to #26

"Sections should be indented two spaces, except for the heading." says https://google.github.io/styleguide/pyguide.html#383-functions-and-methods (and I wonder why it says two while their example uses four...)

Consider

def f(foo):
    """foobar foobars foobar

         Args:
             foo: foobar
            
           Returns:
              foobarred bar
    """
    return "bar"

I would have expected a error about the messed up indentation. But got a successful linting run instead:

$ darglint -v 2 f.py 
$ 

I am on 0.5.1.

Just a small message to say your tool is a really good idea. I wasn't able to find anything like it and I think it's essential. Keep up the good work! I hope you're still working on it.

according to sphinx document(search napoleon_use_param)

def func(a, b, **kwargs):
    """example function
    :param a: parameter a
    :param b: parameter b
    :keyword arguments: * **arg1** (*str*) --
               Description of `arg1`
             * **arg2** (*int, optional*) --
               Description of `arg2`, defaults to 0
    """
    pass

darglint tmp/a.py
tmp/a.py:func:8: DAR101: - **kwargs

And, I think it should be :param kwargs: description instead of :param **kwargs: description darglint trade second one as correct doc

Looking through the git history, there are a few versions that are missing tags. Namely, 0.2.0 and later. Additionally, there are a few missing releases on PyPI (0.1.0, 0.2.0, and 0.3.1).

Thanks for the awesome tool. I have started using it on a Python project and am very happy with it. There are however a couple of things I'd like to report to see how difficult they are to fix.

Talking only Google docstring style.

If I understand it right, there should be a blank line before the closing triple quotes, however it is not reported.

def do_sum1(var1, var2):
    """Sums two numbers.

     Description is here Description is here Description is here Description is here.
     Description is here Description is here Description is here Description is here.
     Description is here Description is here Description is here Description is here.

    Args:
        var1: Variable 1
        var2 (float): Variable 2

    Returns:
        int: sum of two ints

    Raises:
        ValueError: If passing non integer values
    """
    if not all(
        [ isinstance(var, int) for var in [var1, var2]]
        ):
        raise ValueError("Only integers are supported")

    return var1 + var2

I except the docstring to be

    ......
    Raises:
        ValueError: If passing non integer values

    """

When I have a method like this:

@abstractmethod
def foo(self, bar: str) -> int:
    """foo.

    Args:
        bar (str): bar.

    Returns:
        int: spam.

    """

darglint gives me a DAR202, because I don't have a return statement in my function definition. Event if I have it in derived classes. This is a bug, which should be fixed, since we don't define abstract methods, we just declare them.

I don't see anyone talking about this so it has to be something I'm doing wrong...

Using pyenv to provide 3.8.1 with pipenv.

With a file containing:

def tryme():
    pass

Running darglint doesn't complain...

$ pipenv install --dev darglint
$ pipenv run darglint dummy.py
$ echo $?
0

In my real project I also have flake8 installed and it notes that darglint is installed, but also doesn't complain about the lack of docstrings.

Hi! First off, I love this linter so thank you for the work on it. FYI I actually just set up a plugin for SublimeText integration here. On to the actual question:

Do you know of a formal style for exception names in the Raises section of the docstrings in the google style guide? When using a builtin exception I use the normal style of:

Raises:
    BuiltinException: blah

But when I have a function that raises some nonstandard exception class (sometimes which is in a 3rd party module) I usually use the full path to the exception since there may be duplicates in various modules and clarity in docs is nice imo. IE:

Raises: 
    foo.bar.SomeOtherException: blah

I'm really not sure if this goes against the official style guide, the docs ive seen seem to be a little unclear there. If it doesn't go around it, how would you feel about adding functionality to match a raise SomeOtherException to the foo.bar.SomeOtherException declaration? I don't mind sending a PR but I just want to check first. Thanks

    @abstract.abstractmethod
    def __init__(self, config: dict):
        """
        :param config: config dict user defined in config file.
        """

    @abstract.abstractmethod
    def __init__(self, config: dict):
        """

        :param config: config dict user defined in config file.
        """

DAR101 Missing parameter(s) in Docstring: - config

worked with

    @abstract.abstractmethod
    def __init__(self, config: dict):
-        """
+        """anything

        :param config: config dict user defined in config file.
        """

I want to help you with some minor documentation features: like code hightlight, typos, etc.
But, for some reason there are two README files: rst and md. Why?

Which one can be deleted?

Currently darglint completely ignores async functions.

It seems that darglint currently does not support the Google style documentation of documenting class params in the class documentation rather than the __init__.

Is this planned to be fixed? Not sure how technically feasible this would be based on the conversation from #21.

E.g.

class Test(object):
"""Tests darglint.

Params:
    p1 (int): param 1
    p2 (str): param 2

"""

    def __init__(self, p1, p2):
        self.p1 = p1
        self.p2 = p2

I have this code:

def optional(filename: str) -> str:
    """
    This functions is used for compatibility reasons.

    It masks the old `optional` class with the name error.
    Now `invalid-name` is removed from `pylint`.

    Args:
        filename: the filename to be optional.

    Returns:
        New instance of :class:`_Optional`.

    """
    return _Optional(filename)


class _Optional(str):
    """
    Wrap a file path with this class to mark it as optional.

    Optional paths don't raise an :class:`IOError` if file is not found.
    """
    ...

Full source: https://github.com/sobolevn/django-split-settings/blob/master/split_settings/tools.py#L22

But, darglint raises this problem:

./split_settings/tools.py

  31:1     DAR203 Return type mismatch:  ~Return: expected str but was New instance of

I am pretty sure that this is a false positive, because I return str subclass and mypy is happy with that.

Checking types is a very complex and ungrateful job in python. Maybe we can remove this feature from this tool by default? And make it configurable. Like so:

[darglint]
typecheck = true

And darglint --typecheck

Because it can work for simple cases, but will fail for most ones.

» flake8 .
"flake8-darglint" failed during execution due to "Unrecognized stricteness amount.  Should be one of ['SHORT_DESCRIPTION', 'LONG_DESCRIPTION', 'FULL_DESCRIPTION']"
Run flake8 with greater verbosity to see more details

I have these settings in my setup.cfg:

[darglint]
# darglint configuration: https://github.com/terrencepreilly/darglint
strictness = long

The PR with the fix is incoming.

My source file are utf-8 encoded, and I'm using this tool on windows, but I got 'gbk' codec can't decode byte 0x80 in position 608: illegal multibyte sequence.

consider adding chardet support?

Rich Structured Text format requires a blank line after a definition list. However, darglint splits the arguments section in two if there is a blank line.

Possible solutions:

• Accept a configuration option for the docstring format, so that RST can be specified. This
  would allow us to parse the block quickly and accurately in both cases. However, this would
  require different splitting algorithms based on the format. We would also have to add an
  optional long description to the end of the arguments section, to handle the case where the
  arguments section doesn't continue after the line break.
• Assume, once the first section is encountered, that further sections will follow. Then add long
  description grammar to the end of the section grammars. This option would make the parser
  slightly lower (by increasing the grammar size), but would work correctly in most cases.
• Add the concept of block scope to the parser. So, later lines at the same indentation level
  would be assumed to be a continuation of the previous line at that same level. This would
  probably be the most natural solution, and would also fix having code examples in the
  argument descriptions. However, it would probably also be the most difficult to write. It may
  also make parallelization harder in the future. Currently, blocks are separated and treated
  independently, and so would easy to parallelize.