pydantic / pytest-examples

Pytest plugin for testing examples in docstrings and markdown files.

Pydantic

Data validation using Python type hints.

Fast and extensible, Pydantic plays nicely with your linters/IDE/brain. Define how data should be in pure, canonical Python 3.8+; validate it with Pydantic.

Pydantic Company 🚀

We've started a company based on the principles that I believe have led to Pydantic's success. Learn more from the Company Announcement.

Pydantic V1.10 vs. V2

Pydantic V2 is a ground-up rewrite that offers many new features, performance improvements, and some breaking changes compared to Pydantic V1.

If you're using Pydantic V1 you may want to look at the pydantic V1.10 Documentation or, 1.10.X-fixes git branch. Pydantic V2 also ships with the latest version of Pydantic V1 built in so that you can incrementally upgrade your code base and projects: from pydantic import v1 as pydantic_v1.

Help

See documentation for more details.

Installation

Install using pip install -U pydantic or conda install pydantic -c conda-forge. For more installation options to make Pydantic even faster, see the Install section in the documentation.

A Simple Example

from datetime import datetime
from typing import List, Optional
from pydantic import BaseModel

class User(BaseModel):
    id: int
    name: str = 'John Doe'
    signup_ts: Optional[datetime] = None
    friends: List[int] = []

external_data = {'id': '123', 'signup_ts': '2017-06-01 12:22', 'friends': [1, '2', b'3']}
user = User(**external_data)
print(user)
#> User id=123 name='John Doe' signup_ts=datetime.datetime(2017, 6, 1, 12, 22) friends=[1, 2, 3]
print(user.id)
#> 123

Contributing

For guidance on setting up a development environment and how to make a contribution to Pydantic, see Contributing to Pydantic.

Reporting a Security Vulnerability

See our security policy.

pytest-examples's Issues

Python 3.12 support

There are DeprecationWarnings when using pytest-examples with Python 3.12.

We need to run `print_callback` before formatting print args

See pydantic/pydantic-settings#56.

This is required since repr() which isn't valid code is currently breaking black formatting of output.

Use `ruff format` instead of `black`

@Luca-Blight, for some reason I don't have assignee privs in this repo, but feel free to work on this when you have a chance. Thanks for the help 😄

Adding `py.typed` to avoid missing type stubs with type checkers

Could py.typed be added as the project is already typed?

`tests/test_run_examples.py::test_ruff_error` fails with latest `ruff` 0.1.2

Yesterday ruff 0.1.2 was released.
With this new version, the following test fails:

=================================== FAILURES ===================================
_______________________________ test_ruff_error ________________________________

pytester = <Pytester PosixPath('/build/pytest-of-nixbld/pytest-0/test_ruff_error0')>

    def test_ruff_error(pytester: pytest.Pytester):
        pytester.makefile(
            '.md',
            my_file='```py\nimport sys\nprint(missing)\n```',
        )
        # language=Python
        pytester.makepyfile(
            """
    from pytest_examples import find_examples, CodeExample, EvalExample
    import pytest

    @pytest.mark.parametrize('example', find_examples('.'), ids=str)
    def test_find_run_examples(example: CodeExample, eval_example: EvalExample):
        eval_example.lint_ruff(example)
    """
        )

        result = pytester.runpytest('-p', 'no:pretty', '-v')
        result.assert_outcomes(failed=1)

        output = '\n'.join(result.outlines)
        output = re.sub(r'(=|_){3,}', r'\1\1\1', output)
>       assert (
            '=== FAILURES ===\n'
            '___ test_find_run_examples[my_file.md:1-4] ___\n'
            'ruff failed:\n'
            '  my_file.md:2:8: F401 [*] `sys` imported but unused\n'
            '  my_file.md:3:7: F821 Undefined name `missing`\n'
            '  Found 2 errors.\n'
            '  [*] 1 potentially fixable with the --fix option.\n'
            '=== short test summary info ===\n'
        ) in output
E       AssertionError: assert '=== FAILURES ===\n___ test_find_run_examples[my_file.md:1-4] ___\nruff failed:\n  my_file.md:2:8: F401 [*] `sys` impo...me `missing`\n  Found 2 errors.\n  [*] 1 potentially fixable with the --fix option.\n=== short test summ
ary info ===\n' in '=== test session starts ===\nplatform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.2.0 -- /nix/store/ffll6glz3gwx34... info ===\nFAILED test_ruff_error.py::test_find_run_examples[my_file.md:1-4] - Failed: r...\n=== 1 failed in 0.02s =
=='

/build/source/tests/test_run_examples.py:108: AssertionError
----------------------------- Captured stdout call -----------------------------
============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.2.0 -- /nix/store/ffll6glz3gwx342z0ch8wx30p5cnqz1z-python3-3.11.5/bin/python3.11
cachedir: .pytest_cache
rootdir: /build/pytest-of-nixbld/pytest-0/test_ruff_error0
plugins: examples-0.0.10
collecting ... collected 1 item

test_ruff_error.py::test_find_run_examples[my_file.md:1-4] FAILED        [100%]

=================================== FAILURES ===================================
____________________ test_find_run_examples[my_file.md:1-4] ____________________
ruff failed:
  my_file.md:2:8: F401 [*] `sys` imported but unused
  my_file.md:3:7: F821 Undefined name `missing`
  Found 2 errors.
  [*] 1 fixable with the `--fix` option.
=========================== short test summary info ============================
FAILED test_ruff_error.py::test_find_run_examples[my_file.md:1-4] - Failed: r...
============================== 1 failed in 0.02s ===============================
=========================== short test summary info ============================
FAILED tests/test_run_examples.py::test_ruff_error - AssertionError: assert '=== FAILURES ===\n___ test_find_run_examples[my_fil...
=================== 1 failed, 71 passed, 1 xfailed in 2.07s ====================

Continuing examples

Would you consider supporting "continuing" examples that a feature text in between code fences?

For example:

foo = "bar"

We set foo to "bar" to indicate bar. We can then print "Quux!"

if foo == "bar":
    print("Quux!")
#> "Quux!"

Currently, I'm trying some sort of hybrid setup with pytest-markdown-docs that supports it, but that more or less has to run every example twice. Also their ```python continuation thing doesn't quite render right in MkDocs.

0.0.10: Tests missing in sdist tarball

Hi! I'm packaging this project for Arch Linux.
When trying to run tests I realized that the sdist tarball on pypi.org does not offer any tests.

Could you include them with the next version? :)

A lot of downstreams still rely on sdist tarballs (for various reasons).

broken by single line docstrings

pytest-examples gets very messed up by single line docstrings, and starts finding the python code between docstrings, not the docstrings.

This results in examples being missed.

Nö9

L