colin-b / pytest_httpx Goto Github PK

if json parameter is used, set to it to application/json

Document migration from responses module (for those using it to mock requests)

environment.yml

name: api
channels:
  - conda-forge
dependencies:
  - python==3.9
  - fastapi>=0.73.0
  - uvicorn[standard]>=0.17.4
  - SQLAlchemy>=1.4.31
  - psycopg2>=2.9.3
  - httpx>=0.22.0
  - alembic>=1.7.6
  - SQLAlchemy-Utils>=0.38.2
  - pre-commit>=2.9.3
  - pytest>=7.0.1
  - requests>=2.27.1
  - pytest-httpx>=0.20.0 # conflicts begin when adding this package.

❯ conda env update
Collecting package metadata (repodata.json): done
Solving environment: | 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed                                                                                                                                                                        

UnsatisfiableError: The following specifications were found to be incompatible with a past
explicit spec that is not an explicit spec in this operation (sqlalchemy):

  - alembic[version='>=1.7.6'] -> sqlalchemy[version='>=1.3.0']
  - httpx[version='>=0.22.0']
  - psycopg2[version='>=2.9.3']
  - pytest-httpx[version='>=0.20.0'] -> httpx=0.22
  - pytest-httpx[version='>=0.20.0'] -> pytest=6
  - pytest[version='>=7.0.1']
  - requests[version='>=2.27.1']
  - sqlalchemy-utils[version='>=0.38.2'] -> sqlalchemy[version='>=1.0']
  - sqlalchemy[version='>=1.4.31']

From this CHANGELOG entry it appears as though pytest 7 should be supported.

I'm pretty new to python package management, so I'm not sure if I'm doing something dumb. Also since we're using conda-forge, I checked that artifact repository and I see 0.20 was uploaded 19 days ago, so I'm assuming they have the most up-to-date version.

If I switch from
pytest>=7.0.1 => pytest>=6.2.5

❯ conda env update
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
pytest-httpx-0.20.0  | 15 KB     | ################################################################################################################################### | 100% 
pytest-6.2.5         | 434 KB    | ################################################################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Any pointers would be appreciated.

Thanks!

Hi all, Thanks for the great library; httpx released 0.20 yesterday I believe, if pytest-httpx could support that it would be great for those of us using both of these libraries; Happy to take a look myself - would it also be possible to have the hacktoberfest topic added to the repository? :)

Thanks again.

edit: Tests are all failing for me locally even on develop

httpx-0-17.0 was released but at the moment only 0.16.x is supported.

It would be nice to the version constrain could be updated. Thanks.

First iteration on this feature should allow to discard a specified set of headers from the requests (defaulting to whatever might change in between runs, such as the agent)
First iteration will only allow storage as a file provided by a path.

I work on a tool that supports all currently supported python versions. I would like to be able to test each version with the latest release of pytest-httpx, but in #100 you say:

Python 3.7 and 3.8 are no longer supported.

httpx still supports 3.8 and according to https://endoflife.date/python Python 3.8 is supported for one more year (end of life: 14 Oct 2024).

Hi! I'm trying your pytest fixture, unfortunately I get errors like this:

 AssertionError: The following responses are mocked but not requested: [(b'HTTP/1.1', 200, b'', [], <httpx._content_streams.ByteStream object at 0x7f05af8d0160>), (b'HTTP/1.1', 200, b'', [], <httpx._content_streams.ByteStream object at 0x7f05af8d0358>)]

My tests look like this:

import pytest
from pytest_httpx import httpx_mock

from my_package import my_module

@pytest.mark.asyncio
async def test_get_user_from_sso_with_empty_cookie(httpx_mock):
    httpx_mock.add_response()
    with pytest.raises(ConnectionError):
        await my_module.my_function(cookie="")


@pytest.mark.asyncio
async def test_get_user_from_sso_with_missing_cookie(httpx_mock):
    httpx_mock.add_response()
    with pytest.raises(ConnectionError):
        await my_module.my_function(cookie=None)

my_module.my_function is then using HTTPX async clients to send requests to another service.

Any idea why this is happening?

I'm using HTTPX v0.13.1 and pytest_httpx v0.3.0

That we will use and will offer to clients the ability to create responses without having to rely on httpx internals themselves

I recently encountered this error in our tests, however it happens only occasionally and I haven't been able to reliably reproduce it. At first glance it seems like the status_code of the Response is somehow getting set to a re.Pattern object.

Hopefully someone more knowledge of pytest_httpx may recognize the problem; however in the meantime I'll continue to investigate and see if I can come up with a minimal reproducible example.

Here is the line that is (presumably) causing the error:

httpx_mock.add_response(re.compile(r"http://www.example.com.*"), text="wat")

Here is the full pytest traceback:

httpx_mock = <pytest_httpx._httpx_mock.HTTPXMock object at 0x7fd13a696710>, helpers = <bbot.core.helpers.helper.ConfigAwareHelper object at 0x7fd13161b340>

    @pytest.mark.asyncio
    async def test_web_http_compare(httpx_mock, helpers):
        httpx_mock.add_response(re.compile(r"http://www.example.com.*"), text="wat")
        compare_helper = helpers.http_compare("http://www.example.com")
>       await compare_helper.compare("http://www.example.com", headers={"asdf": "asdf"})

bbot/test/test_step_2/test_web.py:155: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
bbot/core/helpers/diff.py:137: in compare
    await self._baseline()
bbot/core/helpers/diff.py:28: in _baseline
    baseline_1 = await self.parent_helper.request(
bbot/core/helpers/web.py:99: in request
    response = await client.request(*args, **kwargs)
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py:1530: in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py:1617: in send
    response = await self._send_handling_auth(
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py:1645: in _send_handling_auth
    response = await self._send_handling_redirects(
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py:1682: in _send_handling_redirects
    response = await self._send_single_request(request)
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py:1729: in _send_single_request
    logger.info(
/usr/lib/python3.10/logging/__init__.py:1477: in info
    self._log(INFO, msg, args, **kwargs)
/usr/lib/python3.10/logging/__init__.py:1624: in _log
    self.handle(record)
/usr/lib/python3.10/logging/__init__.py:1634: in handle
    self.callHandlers(record)
/usr/lib/python3.10/logging/__init__.py:1696: in callHandlers
    hdlr.handle(record)
/usr/lib/python3.10/logging/__init__.py:968: in handle
    self.emit(record)
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/_pytest/logging.py:350: in emit
    super().emit(record)
/usr/lib/python3.10/logging/__init__.py:1108: in emit
    self.handleError(record)
/usr/lib/python3.10/logging/__init__.py:1100: in emit
    msg = self.format(record)
/usr/lib/python3.10/logging/__init__.py:943: in format
    return fmt.format(record)
../../../.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/_pytest/logging.py:114: in format
    return super().format(record)
/usr/lib/python3.10/logging/__init__.py:678: in format
    record.message = record.getMessage()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <LogRecord: httpx, 20, /home/bls/.cache/pypoetry/virtualenvs/bbot-yxGMlPK5-py3.10/lib/python3.10/site-packages/httpx/_client.py, 1729, "HTTP Request: %s %s "%s %d %s"">

    def getMessage(self):
        """
        Return the message for this LogRecord.
    
        Return the message for this LogRecord after merging any user-supplied
        arguments with the message.
        """
        msg = str(self.msg)
        if self.args:
>           msg = msg % self.args
E           TypeError: %d format: a real number is required, not re.Pattern

/usr/lib/python3.10/logging/__init__.py:368: TypeError

I love this package but was wondering if we could also have a conda version of it on conda-forge as I work almost exclusively in conda environments and even though I can still pip install it there upgrades would work more smoothly if pytest_httpx was available as a conda package too.

The process to create - and maintain - a package on conda-forge is relatively easy and straight-forward for pure Python pip packages. Happy to help with this if you're open to the idea. From a quick check of the dependencies these all appear to available on conda-forge too so I don't anticipate any problems there.

Adding a callback mock for an URL which matched previously to any static response mock has no effect.
Seems that pytext-httpx chooses to use last (even already used) static response, even though there is an unused callback mock waiting to be picked.
This is counter-intuitive and incompatible at least with aioresponses.

@pytest.mark.asyncio
async def test_callback_order(httpx_mock: HTTPXMock):
    async with httpx.AsyncClient() as client:
        httpx_mock.add_response(url='http://localhost/api', json={'response': 'first'})
        assert 'first' == (await client.get('http://localhost/api')).json()['response']

        httpx_mock.add_response(url='http://localhost/api', json={'response': 'second'})
        assert 'second' == (await client.get('http://localhost/api')).json()['response']

        def _callback(req: httpx.Request):
            return httpx.Response(status_code=200, json={'response': 'third'})

        httpx_mock.add_callback(_callback, url='http://localhost/api')
        assert 'third' == (await client.get('http://localhost/api')).json()['response']

The result:

>           assert 'third' == (await client.get('http://localhost/api')).json()['response']
E           AssertionError: assert 'third' == 'second'
E             - second
E             + third

add_response()
stream first request
stream second request // failure because stream is already consumed by first read of the same response, even if this is the same request

Similar to #18, but it wasn't resolved there. Suppose we have

def test_no_requests(httpx_mock):
    httpx_mock.add_response()
    my_function(...)
    assert not len(httpx_mock.get_requests())

This will throw at teardown unless you override the assert_all_responses_were_requested fixture to return False. However, this behavior might be desirable for other tests in the file. From what I can tell, there's no way to define the return value of the foregoing fixture depending on the test.

Maybe there should be an API on HTTPXMock which disables this check so that we have:

def test_no_requests(httpx_mock):
    httpx_mock.assert_all_responses_were_requested = False  # default is True
    httpx_mock.add_response()
    my_function(...)
    assert not len(httpx_mock.get_requests())

Take this example:

httpx_mock.add_response(
    url=re.compile(r"http://example.com/"),
    status_code=404,
)

httpx_mock.add_response(
    url="http://example.com/test/123",
    status_code=200,
)

In this case, my expected behavior is:

First, match the more specific URL, otherwise fallback to more general URL.

However, the code above ends up creating flaky tests; most of the time, it works, but sometimes it fails when it matches the regex first and not the specific case.

As far as I understand, this can be solved by defining specific matchers before the general ones. And that's fine.

I have two suggestions:

1. Document this behavior and keep it as is.
2. Raise some exception when two different matchers match the same thing.

I'm developing a FastAPI app that uses async route handlers. Because of this I'm using HTTPX as the test client as described here https://fastapi.tiangolo.com/advanced/async-tests/ . The app also uses httpx internally to call other endpoints.

When I tried to use the httpx_mock targeting just the external URLs, I noticed that it was also blocking the Test Client requests.

Is there some way to do this?

I had a scenario where I wanted to all request to be verified about over non-request, but just ONE request I wanted to be free verification.

With this new feature I can achieve this logic.

I created this PR #76.

hi,

I just wonder how to test if a request is sent via proxy,
it's not reporting CONNECT nor the request is being proxied.

Hi there,
Quick question. I'm using httpx for testing a webserver AND making external requests inside the webserver too.

I want requests to my webserver to go untouched. But I want to mock out all sub-requests made by my server.

Example:
Normal: http://testserver
Mocked: https://wikipedia...

I could inject, but I'm wondering if there is some functionality to disable mocking for certain urls. It seems like httpx_mock mocks everything.

Thanks in advance.
Phil

For those using aioresponses to mock aiohttp

I started using httpx_mock and tried to mock multiple requests to one resource.
Then I expected httpx_mock.reset() to reset the state of HTTPXMock. However, it resets only callbacks, self._requests are still left unchanged.

Generally:

def test_something(httpx_mock: HTTPXMock):
    # custom setup that has to be here, but includes mocked requests
    httpx_mock.reset() 
    for i in range(5):
        httpx.request(...)
    assert len(httpx_mock.get_requests()) == 5 # fails, because of the mocked requests in "setup" part. There is no way to reset list of requests

Is there a reason why .reset() is not resetting requests list?

For now, it's behaving as the connection is still open

Since version 0.16.0 method add_callback has been requiring callback with httpx.Response return value. If we create callback to only rise exception then return value is None. mypy won't pass with such expected return (I had to ignore triggering add_callback). I know that #10 would probably fix it but maybe You'll find some way to patch it sooner. I could write -> httpx.Response but it just doesn't seem True if all I do in function is raising an exception.

I have a method where if a call received a 401 response, it then calls the auth API to refresh its auth token, and then re-issue the call with the new token. When done synchronously with requests, I can mock out the call with multiple responses in the side_effect parameter.

Moving to httpx and async calls, I'm stumped how to test this case. I tried adding multiple httpx_mock.add_reponse() calls, but of course that didn't work. Any advice?

Hi, I have a use-case where I'm querying over 80 different endpoints.
Currently the outputs are verified using pytest-snapshot.
But the tests won't run fast as the HTTP calls have to be made.

What I'm trying to achieve is a system where if I add a new module and run the test, it's HTTP calls will be run once and written to a file. For future runs, that response will be provided by the mock, so we can skip the HTTP calls.

Is such an approach feasible with enough modifications in pytest_httpx, or are there any architectural limitations which would stop me from adding such a feature?

import httpx
import pytest
import pytest_httpx


@pytest.mark.asyncio
async def test_something(httpx_mock):
    httpx_mock.add_response(json={"abc": "def"})
    with httpx.Client() as client:
        # response.stream will be a BoundSyncStream
        client.get("https://example.com/")
        # response.stream will be a BoundSyncStream, referencing a BoundSyncStream
        client.get("https://example.com/")
        # response.stream will be a BoundSyncStream, referencing a BoundSyncStream, referencing a BoundSyncStream
        client.get("https://example.com/")

The issue is essentially that a single response instance is being returned by the httpx_mock transport, and being reused.

Originally posted by @tomchristie in encode/httpx#2777 (comment)

I have a FastAPI application that's using httpx, and I'm having trouble getting the pytests to run successfully in the GitHub environment when I push the code. The tests run fine when run in a local Docker container.

Here's my Dockerfile

FROM python:3.9.7-slim
WORKDIR /app/
ENV PYTHONPATH "${PYTHONPATH}:/"
COPY requirements.txt .
RUN pip install -r requirements.txt
CMD ["uvicorn", "main:app", "--reload", "--host", "0.0.0.0"]

And here's the requirements.txt file it uses to get the modules the application needs:

fastapi==0.82.0
uvicorn==0.18.3
pydantic==1.10.2
pylint==2.15.2
pytest==7.1.3
boto3==1.24.68
python-dotenv==0.21.0
pytest-mock==3.8.2
sseclient==0.0.27
pytest-asyncio==0.19.0
wheel==0.37.1
httpx==0.23.0
pytest-httpx==0.21.0
pytest-trio==0.7.0

Here's the .github/workflows/unit-tests.yml file that runs the tests in the GitHub environment:

name: unit-tests

on: [push]
jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
        with:
          fetch-depth: 1
          ref: ${{ github.event.inputs.branch_name }}

      - name: Set up Python 3.9.x
        uses: actions/setup-python@v1
        with:
          python-version: 3.9.x

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install fastapi==0.82.0
          pip install pydantic==1.10.2
          pip install pytest==7.1.3
          pip install boto3==1.24.68
          pip install python-dotenv==0.21.0
          pip install pytest-mock==3.8.2
          pip install sseclient==0.0.27
          pip install pytest-asyncio==0.19.0
          pip install httpx==0.23.0
          pip install pytest-httpx==0.21.0
          pip install pytest-cov==4.0.0
          pip install pytest-trio==0.7.0

      - name: Running unit tests
        run: |
          pytest -v --cov=app tests/unit/ --asyncio-mode=strict

And here's part of the errors generated when run in the GitHub environment:

Run pytest --cov=app tests/unit/ --asyncio-mode=strict
============================= test session starts ==============================
platform linux -- Python 3.9.14, pytest-7.1.3, pluggy-1.0.0
rootdir: /home/runner/work/integration-engine/integration-engine
plugins: httpx-0.21.0, anyio-3.[6](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:7).1, cov-4.0.0, asyncio-0.19.0, mock-3.8.2, trio-0.[7](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:8).0
asyncio: mode=strict
collected 69 items

tests/unit/test_main.py .                                                [  1%]
tests/unit/dependencies/test_validate_requests.py ..                     [  4%]
tests/unit/helpers/test_agent_command_helper.py .......                  [ 14%]
tests/unit/helpers/test_agent_message_helper.py ....                     [ 20%]
tests/unit/helpers/test_cognito_helper.py ssFF                           [ 26%]
tests/unit/helpers/test_mercure_helper.py .....                          [ 33%]
tests/unit/helpers/test_pmb_helper.py ....                               [ 39%]
tests/unit/helpers/test_repository_helper.py .......                     [ 49%]
tests/unit/repositories/test_agent_message_repository.py ...             [ 53%]
tests/unit/repositories/test_agent_repository.py .....                   [ 60%]
tests/unit/routers/test_agent_communication.py ..............            [ [8](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:9)1%]
tests/unit/routers/test_agent_search.py ....                             [ 86%]
tests/unit/routers/test_authentication.py ..                             [ 8[9](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:10)%]
tests/unit/routers/test_diagnostics.py ..                                [ 92%]
tests/unit/routers/test_healthz.py ..                                    [ 95%]
tests/unit/routers/test_registration.py ...                              [[10](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:11)0%]

=================================== FAILURES ===================================
____ test_get_token_with_invalid_credentials_returns_error_message[asyncio] ____

httpx_mock = <pytest_httpx._httpx_mock.HTTPXMock object at 0x7f9346b296d0>

    @pytest.mark.anyio
    async def test_get_token_with_invalid_credentials_returns_error_message(httpx_mock: HTTPXMock):
        """Test that get token with invalid credentials returns error message"""
        mock_valid_response = {
            "errorMessage": "test_error_message"
        }
        mock_user_auth_request = UserAuthenticationRequest(
            username="test_invalid_username",
            ***
        )
        httpx_mock.add_response(json=mock_valid_response)
>       response = await get_token(mock_user_auth_request)

tests/unit/helpers/test_cognito_helper.py:50: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
app/helpers/cognito_helper.py:37: in get_token
    data = await call_cognito(url=url, body=body)
app/helpers/cognito_helper.py:59: in call_cognito
    request = httpx.Request("GET", url=url, json=body)
/opt/hostedtoolcache/Python/3.9.[14](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:15)/x64/lib/python3.9/site-packages/httpx/_models.py:326: in __init__
    self.url = URL(url)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <[AttributeError("'URL' object has no attribute '_uri_reference'") raised in repr()] URL object at 0x7f9346aa[23](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:24)70>
url = None, kwargs = {}

    def __init__(
        self, url: typing.Union["URL", str, RawURL] = "", **kwargs: typing.Any
    ) -> None:
        if isinstance(url, (str, tuple)):
            if isinstance(url, tuple):
                raw_scheme, raw_host, port, raw_path = url
                scheme = raw_scheme.decode("ascii")
                host = raw_host.decode("ascii")
                if host and ":" in host and host[0] != "[":
                    # it's an IPv6 address, so it should be enclosed in "[" and "]"
                    # ref: https://tools.ietf.org/html/rfc[27](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:28)[32](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:33)#section-2
                    # ref: https://tools.ietf.org/html/rfc[39](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:40)86#section-3.2.2
                    host = f"[{host}]"
                port_str = "" if port is None else f":{port}"
                path = raw_path.decode("ascii")
                url = f"{scheme}://{host}{port_str}{path}"
    
            try:
                self._uri_reference = rfc3986.iri_reference(url).encode()
            except rfc3986.exceptions.InvalidAuthority as exc:
                raise InvalidURL(message=str(exc)) from None
    
            if self.is_absolute_url:
                # We don't want to normalize relative URLs, since doing so
                # removes any leading `../` portion.
                self._uri_reference = self._uri_reference.normalize()
        elif isinstance(url, URL):
            self._uri_reference = url._uri_reference
        else:
>           raise TypeError(
                f"Invalid type for url.  Expected str or httpx.URL, got {type(url)}: {url!r}"
            )
E           TypeError: Invalid type for url.  Expected str or httpx.URL, got <class 'NoneType'>: None

/opt/hostedtoolcache/Python/3.9.14/x[64](https://github.com/rectanglehealth/integration-engine/actions/runs/3207285033/jobs/5242003697#step:5:65)/lib/python3.9/site-packages/httpx/_urls.py:102: TypeError

My apologies if this isn't the right forum to be asking this, but I'm at my wits end trying to resolve this. Any ideas, pointers or suggestions are welcome.

with pytest_httpx_bug_demo.py

from pytest_httpx import HTTPXMock
import httpx


def test_something(httpx_mock: HTTPXMock):
    httpx_mock.add_response(method="GET", url="http://www.google.com")
    with httpx.Client() as client:
        client.get("http://www.google.com")

running pyright in stict mode with the following pyproject.toml

[tool.pyright]
typeCheckingMode = "strict"

results in the following error

pyright pytest_httpx_bug_demo.py
/home/[redacted]/pytest_httpx_bug_demo.py
  /home/[redacted]/pytest_httpx_bug_demo.py:6:5 - error: Type of "add_response" is partially unknown
    Type of "add_response" is "(status_code: int = 200, http_version: str = "HTTP/1.1", headers: Headers | Dict[str, str] | Dict[bytes, bytes] | Sequence[Tuple[str, str]] | Sequence[Tuple[bytes, bytes]] = None, content: bytes | None = None, text: str | None = None, html: str | None = None, stream: Any = None, json: Any = None, **matchers: Unknown) -> None" (reportUnknownMemberType)
1 error, 0 warnings, 0 informations

This is because the **marchers arguments don't have any type declaration. Changing that into **matchers: Any would addresses that.

When I register an async function as a callback to httpx_mock I get this error:
TypeError: cannot unpack non-iterable coroutine object

I suppose it's not awaited here:
https://github.com/Colin-b/pytest_httpx/blob/develop/pytest_httpx/_httpx_mock.py#L179

Is this a bug or am I using the library wrong?

Thanks!

httpx released a new version. Currently the httpx is limited to 0.17.*.

It would be nice if pytest-httpx is updated.

Thanks

It should avoid issues such as #19

Hello,

thanks a lot for this project 🙏

The only thing I miss until now, is an option to define a response list, like it exists for requests-mock.

I need it to simulate a retry. Unlike in #75 my requests are identical, so unfortunately I have no header or something like that, which I can use to distinguish the requests.

fin swimmer

This will allow to replay #113 and have the side effect of letting users have responses in files instead of in python tests (especially useful in case of heavy amount of responses or even huge payloads).
Argument parsing should be intelligent enough to allow every existing parameter to date (and be generic in some way?)

Add a shortcut to add_callback only requiring to provide an exception instance / class

httpx released version 0.23.0 this morning, which corrects a significant security issue. It also removes support for Python 3.6.

I'd like to update my projects to use the new version alongside pytest_httpx. Could the project update its dependency, or would you welcome a PR to make that change?

With the following test, which correctly matches the Authorization header, but with an incorrect url in this case, the non-matching request causes a very confusing error message that seems to suggest the headers are wrong (besides the url also being wrong):

from pytest_httpx import HTTPXMock
import httpx


def test_something(httpx_mock: HTTPXMock):
    httpx_mock.add_response(
        method="GET",
        url="http://www.google.com?q=b",
        match_headers={"Authorization": "Bearer: Something"},
    )
    with httpx.Client() as client:
        client.get(
            "http://www.google.com", headers={"Authorization": "Bearer: Something"}
        )

Running this with pytest results in the following error:

E       httpx.TimeoutException: No response can be found for GET request on http://www.google.com with {} headers amongst:
E       Match GET requests on http://www.google.com?q=b with {'Authorization': 'Bearer: Something'} headers

I was expecting it to fail, but it says with {} header as if no headers were passed in, while they were.

I found out what the issue is:
In _explain_that_no_response_was_found it includes if name in expect_headers}. The issue is that in the request, the Authorization header is turned into lowercase. So this ends up on that line as "authorization" not in {"Authorization": "Bearer: Something"}, so it won't display the header.

This code is based on https://github.com/Colin-b/pytest_httpx#reply-with-custom-body :

def test_str_body(httpx_mock: HTTPXMock):
    httpx_mock.add_response(text="This is my UTF-8 content", status_code=httpx.codes.BAD_REQUEST)

    with httpx.Client(
        event_hooks={
            "response": [lambda response: response.raise_for_status()]
        }
    ) as client:
        try:
            client.get("https://test_url")
        except httpx.HTTPStatusError as exc:
            assert exc.response.text == "This is my UTF-8 content"

It fails with this error:

httpx.ResponseNotRead: Attempted to access streaming response content, without having called `read()`.

Document differences between this module and respx

good job!👍🏻
I realy like the project and it dost help me a lot.
in some test I use pytest_httpx to mock some requests but I do not need to ensure all requests will be sent
so , perhaps it's a good feature to add a toggle assert_all_requests_are_fired just like what responses do?

Some functions I'm testing are caching results. I'd like to assert that subsequent calls do not hit the network.

I'm thinking of something like this:

def test_cached_results(httpx_mock):
    response = httpx_mock.add_response()
    my_function(...)
    
    # in two statements:
    assert not response.fired
    httpx_mock.remove(response)

    # in one statement: pop and assert not fired
    assert not httpx_mock.pop().fired

    # one statement, variant: if the response exists, it is removed, otherwise it fails
    assert httpx_mock.remove(response)

I have a potential use case for an async callback which was originally mentioned in #38. I have the following (pseudo-)code which I want to test:

async def foo():
    async with httpx.AsyncClient() as c:
        with trio.move_on_after(timeout) as cs:
            await c.post(endpoint, json=payload)
        if cs.cancelled_caught:
            ...  # do stuff
        else:
            ...  # do other stuff

This is the test function I want to write

async def test_foo_timeout(httpx_mock, autojump_clock):
    async def cb(*args):
        await trio.sleep(some_long_time)
    
    httpx_mock.add_callback(cb)

    foo()

    ...  # assertions about what happens if the request times out

Importantly, autojump_clock virtualizes the clock so that my test executes immediately. It requires use of trio.sleep instead of other sleep functions, e.g. time.sleep.

Alternatively, I could inject a post function to foo and avoid mocking altogether, i.e.

async def foo(post):
    with trio.move_on_after(timeout) as cs:
        await post(endpoint, payload)
    if cs.cancelled_caught:
        ...  # do stuff
    else:
        ...  # do other stuff

In which case my test would become

async def test_foo_timeout(autojump_clock):
    async def post(*args):
        await trio.sleep(some_long_time)

    foo(post)

    ...  # assertions about what happens if the request times out

Let me know if this makes sense or there is another alternative which I didn't consider.

When mocking a request body with match_content, the value passed in gets converted to bytes. However, when making the actual request, the body gets converted to json, even if I use content and pass an array of bytes. Therefore, I'm unable to match any requests when using the match_content parameter.

I would like to be able to send json in the request and set a matcher on the mocker to only pass when a request with that json body is sent.

Request:

async with AsyncClient() as client:
    response = await client.post("api.test.com", json={"userId": 1, "data": {"names": ["Test1", "Test2"]}})

Test:

@pytest.mark.asyncio
async def test_send_request(httpx_mock: HTTPXMock):
    httpx_mock.add_response(url="api.test.com", method="POST", match_content={"userId": 1, "data": {"names": ["Test1", "Test2"]}})
    await send_request()

Test failure message:

httpx.TimeoutException: No response can be found for POST request on /api.test.com with b'{"userId": 1, "data": {"names": ["Test1", "Test2"]}}' body amongst:
Match POST requests on api.test.com with {'userId': 1, 'data': {'names': ['Test1', 'Test2']}} body

httpx_mock.add_response(
      url='test_url?query_type=数据',
      method='GET',
      json={'result': 'ok'}
 )

Executing the above code, It wil cause an encoding error:

obj = '数据', encoding = 'ascii', errors = 'strict'

def _encode_result(obj, encoding=_implicit_encoding,
                        errors=_implicit_errors):
   return obj.encode(encoding, errors)

E UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)

/usr/local/Cellar/[email protected]/3.9.13_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/parse.py:108: UnicodeEncodeError

I have logging that reads request time from .elapsed attribute of httpx.Request. Everything works with 0.13, but after upgrading to 0.15 I'm getting exceptions:

'.elapsed' may only be accessed after the response has been read or closed.

My versions:
httpx==0.21.1
pytest-httpx==0.15.0

Hi !!

It is possible to match request attributes? These attributes are timeout, auth, and others.
I'm trying to use this to ensure that the request has these attributes.

At the documentation has headers, methods, and content. Can we develop new feature?

Problem
Response.elapsed isn't set after a mocked POST if add_response is given json=... or content=b....

Reproduce
This snippet replicates the problem for me:

def test_elapsed(httpx_mock: HTTPXMock):
    httpx_mock.add_response(json={})
    r = httpx.post(
        "https://jsonplaceholder.typicode.com/posts",
        data={"title": "foo", "body": "bar", "userId": 1,},
        headers={"Content-type": "application/json; charset=UTF-8"},
    )
    r.raise_for_status()
    r.close()
    print(r.elapsed.total_seconds())

Versions
httpx=0.21.3
pytest_httpx=0.15.0

Hi,

For a project that makes a lot of async requests I would like to emulate the effect of slow http responses. AFAIK that is currently not supported in this library. To me it would be useful to have an option to specify a delay in (micro)seconds.

I would be happy to provide a PR if this is something you would consider merging.

Cheers

I don't think that feature exists yet?

I need to set a cookie in a response 🙂

I found the link https://colin-b.github.io/pytest_httpx/ on pypi but it is not show up on github.

Please it could be add section about of repository?

Hi,

Thanks for the nice library.
Currently, pytest-httpx depends on pytest ==6.*

But there is a fix to an annoying async issue only available in pytest 7.2+
pytest-dev/pytest#3747

Could you please increment the dependency version?
Thank you