magicstack / httptools Goto Github PK
View Code? Open in Web Editor NEWFast HTTP parser
License: MIT License
Fast HTTP parser
License: MIT License
I see no announcement in the changelog that support for 3.5 has been dropped. Was this intentional or not?
Could you please add tags matching packages released on pypi?
Hello anyone,
What's about llhttp? As says a developers, it's more fastest than http_parser. Also, http_parser unmaintainable.
The readme and pypi page starts with the statement:
It's still in a very early development stage, expect APIs to break.
However the package has the "Development Status :: 5 - Production/Stable" classifier, in addition httptools seems to be used by servers/components like uvicorn which appear to be production ready.
Is it reasonable to assume that httptools is production ready? If so perhaps worth removing that statement?
In your uvloop docs you talk about running aiohttp with this parser instead of the built in one. Do you have any documentation on how that would work?
From time to time, my server output an error that let me think that Request.on_url
is not called for the current stream.
Here is a small script to reproduce the behaviour:
import asyncio
from httptools import HttpRequestParser
class Request:
def __init__(self):
self.EOF = False
def on_url(self, url: bytes):
self.on_url_called = True
def on_message_complete(self):
self.EOF = True
async def serve(reader, writer):
chunks = 2 ** 16
req = Request()
parser = HttpRequestParser(req)
while True:
data = await reader.read(chunks)
parser.feed_data(data)
if not data or req.EOF:
break
assert req.on_url_called
writer.write(b'HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nOK')
writer.write_eof()
if __name__ == '__main__':
loop = asyncio.get_event_loop()
coro = loop.create_task(asyncio.start_server(serve, '127.0.0.1', 8080))
server = loop.run_until_complete(coro)
try:
loop.run_forever()
except KeyboardInterrupt:
print('Bye.')
finally:
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
To be run with python file.py
And then benchmark it with something like ab -n 10000 -c 8 'http://127.0.0.1:8080/'
Once every, say, 10000 requests, I get this error:
Task exception was never retrieved
future: <Task finished coro=<serve() done, defined at test_httptools.py:18> exception=AttributeError("'Request' object has no attribute 'on_url_called'",)>
Traceback (most recent call last):
File "test_httptools.py", line 27, in serve
assert req.on_url_called
AttributeError: 'Request' object has no attribute 'on_url_called'
Any hint? :)
Thanks!
I try to combine pycurl and httptools, but I found it can't deal with https, here is my code:
success with http:
import pycurl
import httptools
class HttpResponse:
def on_message_begin(self):
print('on_message_begin')
def on_url(self, url: bytes):
print(f'on_url: url={url}')
def on_header(self, name: bytes, value: bytes):
print(f'on_header: name={name}, value={value.decode("ISO-8859-1")}')
def on_headers_complete(self):
print(f'on_header_complete')
def on_body(self, body: bytes):
print(f'on_body: {len(body)}')
def on_message_complete(self):
print(f'on_message_complete')
def on_chunk_header(self):
print(f'on_chunk_header')
def on_chunk_complete(self):
print(f'on_chunk_complete')
def on_status(self, status: bytes):
print(f'on_status: status={status}')
print(pycurl.version)
print(httptools.__version__)
m = HttpResponse()
p = httptools.HttpResponseParser(m)
c = pycurl.Curl()
c.setopt(pycurl.URL, "http://uvloop.readthedocs.io/")
c.setopt(pycurl.HTTP_TRANSFER_DECODING, 0)
c.setopt(pycurl.WRITEFUNCTION, p.feed_data)
c.setopt(pycurl.HEADERFUNCTION, p.feed_data)
c.perform()
and output:
PycURL/7.45.2 libcurl/7.76.1 OpenSSL/1.1.1u zlib/1.2.11 libssh2/1.9.0 nghttp2/1.43.0
0.5.0
on_message_begin
on_status: status=b'Found'
on_header: name=b'Date', value=Thu, 08 Jun 2023 09:50:21 GMT
on_header: name=b'Content-Type', value=text/html; charset=utf-8
on_header: name=b'Transfer-Encoding', value=chunked
on_header: name=b'Connection', value=keep-alive
on_header: name=b'Location', value=https://uvloop.readthedocs.io/
on_header: name=b'CF-Ray', value=7d403ae94955cec5-SJC
on_header: name=b'CF-Cache-Status', value=EXPIRED
on_header: name=b'Cache-Control', value=max-age=1200
on_header: name=b'Content-Language', value=en
on_header: name=b'Vary', value=Accept-Language, Cookie, Accept-Encoding
on_header: name=b'CDN-Cache-Control', value=public
on_header: name=b'Referrer-Policy', value=no-referrer-when-downgrade
on_header: name=b'X-Backend', value=web-i-0854c4793bcd745a7
on_header: name=b'X-Content-Type-Options', value=nosniff
on_header: name=b'X-RTD-Domain', value=uvloop.readthedocs.io
on_header: name=b'X-RTD-Project', value=
on_header: name=b'X-RTD-Project-Method', value=public_domain
on_header: name=b'X-RTD-Redirect', value=http_to_https
on_header: name=b'X-RTD-Version-Method', value=path
on_header: name=b'X-Served', value=Django-Proxito
on_header: name=b'X-XSS-Protection', value=1; mode=block
on_header: name=b'Server', value=cloudflare
on_header: name=b'alt-svc', value=h3=":443"; ma=86400
on_header_complete
on_chunk_header
on_chunk_complete
on_message_complete
fail with https:
import certifi
import pycurl
import httptools
class HttpResponse:
def on_message_begin(self):
print('on_message_begin')
def on_url(self, url: bytes):
print(f'on_url: url={url}')
def on_header(self, name: bytes, value: bytes):
print(f'on_header: name={name}, value={value.decode("ISO-8859-1")}')
def on_headers_complete(self):
print(f'on_header_complete')
def on_body(self, body: bytes):
print(f'on_body: {len(body)}')
def on_message_complete(self):
print(f'on_message_complete')
def on_chunk_header(self):
print(f'on_chunk_header')
def on_chunk_complete(self):
print(f'on_chunk_complete')
def on_status(self, status: bytes):
print(f'on_status: status={status}')
print(pycurl.version)
print(httptools.__version__)
m = HttpResponse()
p = httptools.HttpResponseParser(m)
c = pycurl.Curl()
c.setopt(pycurl.URL, "https://uvloop.readthedocs.io/")
c.setopt(c.CAINFO, certifi.where())
c.setopt(pycurl.HTTP_TRANSFER_DECODING, 0)
c.setopt(pycurl.WRITEFUNCTION, p.feed_data)
c.setopt(pycurl.HEADERFUNCTION, p.feed_data)
c.perform()
output:
PycURL/7.45.2 libcurl/7.76.1 OpenSSL/1.1.1u zlib/1.2.11 libssh2/1.9.0 nghttp2/1.43.0
0.5.0
on_message_begin
---------------------------------------------------------------------------
HttpParserError Traceback (most recent call last)
httptools/parser/parser.pyx in httptools.parser.parser.HttpParser.feed_data()
HttpParserError: Expected dot
---------------------------------------------------------------------------
error Traceback (most recent call last)
/tmp/ipykernel_16171/3019824725.py in <module>
37 c.setopt(pycurl.WRITEFUNCTION, p.feed_data)
38 c.setopt(pycurl.HEADERFUNCTION, p.feed_data)
---> 39 c.perform()
error: (23, 'Failed writing header')
by the way, how can i let httptools auto decode with Accept-Encoding: gzip,deflate
Installing httptools v0.0.13 on Python 3.9 from PyPI (e.g. pip install httptools==0.0.13
) doesn't work.
The relevant error output is this:
httptools/parser/parser.c:8977:52: error: no member named 'tp_print' in 'struct _typeobject'
__pyx_type_9httptools_6parser_6parser_HttpParser.tp_print = 0;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
httptools/parser/parser.c:8988:59: error: no member named 'tp_print' in 'struct _typeobject'
__pyx_type_9httptools_6parser_6parser_HttpRequestParser.tp_print = 0;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
httptools/parser/parser.c:9000:60: error: no member named 'tp_print' in 'struct _typeobject'
__pyx_type_9httptools_6parser_6parser_HttpResponseParser.tp_print = 0;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
httptools/parser/parser.c:9009:45: error: no member named 'tp_print' in 'struct _typeobject'
__pyx_type_9httptools_6parser_6parser_URL.tp_print = 0;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
Since released PyPI package versions are immutable this cannot be fixed.
You can simply use httptools v0.1.0 instead. If this for some odd reason isn't possible for you, you can regenarate parser.c
from parser.pyx
using Cython version 0.29.14 or later and then install.
This isn't a bug report. Just an observation for later users, posted here for posterity. Closing immediately.
when I install httptools on windows via pip, I got error:
C:/>pip install httptools
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting httptools
Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz (104kB)
|████████████████████████████████| 112kB 731kB/s
Building wheels for collected packages: httptools
Building wheel for httptools (setup.py) ... error
ERROR: Complete output from command 'c:\program files\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\ashengtx\\AppData\\Local\\Temp\\pip-install-e390hs44\\httptools\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\ashengtx\AppData\Local\Temp\pip-wheel-0mti7e4p' --python-tag cp37:
ERROR: running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
creating build\lib.win-amd64-3.7\httptools
copying httptools\__init__.py -> build\lib.win-amd64-3.7\httptools
creating build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser\__init__.py -> build\lib.win-amd64-3.7\httptools\parser
running egg_info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing top-level names to httptools.egg-info\top_level.txt
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info\SOURCES.txt'
copying httptools\parser\parser.c -> build\lib.win-amd64-3.7\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
creating build\temp.win-amd64-3.7\Release\httptools
creating build\temp.win-amd64-3.7\Release\httptools\parser
creating build\temp.win-amd64-3.7\Release\vendor
creating build\temp.win-amd64-3.7\Release\vendor\http-parser
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-Ic:\program files\python37\include" "-Ic:\program files\python37\include" /Tchttptools/parser/parser.c /Fobuild\temp.win-amd64-3.7\Release\httptools/parser/parser.obj -O2
error: command 'cl.exe' failed: No such file or directory
----------------------------------------
ERROR: Failed building wheel for httptools
Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: httptools
Running setup.py install for httptools ... error
ERROR: Complete output from command 'c:\program files\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\ashengtx\\AppData\\Local\\Temp\\pip-install-e390hs44\\httptools\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\ashengtx\AppData\Local\Temp\pip-record-edr9eisv\install-record.txt' --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.7
creating build\lib.win-amd64-3.7\httptools
copying httptools\__init__.py -> build\lib.win-amd64-3.7\httptools
creating build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser\__init__.py -> build\lib.win-amd64-3.7\httptools\parser
running egg_info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing top-level names to httptools.egg-info\top_level.txt
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info\SOURCES.txt'
copying httptools\parser\parser.c -> build\lib.win-amd64-3.7\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
creating build\temp.win-amd64-3.7
creating build\temp.win-amd64-3.7\Release
creating build\temp.win-amd64-3.7\Release\httptools
creating build\temp.win-amd64-3.7\Release\httptools\parser
creating build\temp.win-amd64-3.7\Release\vendor
creating build\temp.win-amd64-3.7\Release\vendor\http-parser
cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD "-Ic:\program files\python37\include" "-Ic:\program files\python37\include" /Tchttptools/parser/parser.c /Fobuild\temp.win-amd64-3.7\Release\httptools/parser/parser.obj -O2
error: command 'cl.exe' failed: No such file or directory
----------------------------------------
ERROR: Command "'c:\program files\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\ashengtx\\AppData\\Local\\Temp\\pip-install-e390hs44\\httptools\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\ashengtx\AppData\Local\Temp\pip-record-edr9eisv\install-record.txt' --single-version-externally-managed --compile" failed with error code 1 in C:\Users\ashengtx\AppData\Local\Temp\pip-install-e390hs44\httptools\
however, I have cl.exe installed
C:\>cl.exe
用于 x64 的 Microsoft (R) C/C++ 优化编译器 19.16.27027.1 版
版权所有(C) Microsoft Corporation。保留所有权利。
用法: cl [ 选项... ] 文件名... [ /link 链接选项... ]
how to solve this problem?
Hi, can you add a git tag + github release for the 0.2.0 version, thanks!
The HttpParserUpgrade
exception appears to get called after the event hooks have been called into (at least with small requests that fit into a single data_received call)
This makes it not terribly useful, as we may have already called into a handler function / issued a response / etc... by the time the upgrade is raised.
Sketch of protocol class:
def __init__(self, ...):
self.request_parser = httptools.HttpRequestParser(self)
...
# The asyncio.Protocol hooks...
...
def data_received(self, data):
print('data_received()')
try:
self.request_parser.feed_data(data)
except httptools.HttpParserUpgrade:
print('upgrade exception')
# Event hooks called back into by HttpRequestParser...
...
def on_headers_complete(self):
print('on_headers_complete()')
def on_message_complete(self):
print('on_message_complete()')
Output:
data_received()
on_headers_complete()
on_message_complete()
upgrade exception
On first sight this looks awkward to resolve, so it could just be considered a constraint of the implementation, and deal with it in python-land instead (which would be acceptable from my POV)
According to the specification of HTTP/1.1 ...
Responses to the HEAD request method never include a message body because the associated response header fields (e.g., Transfer-Encoding, Content-Length, etc.), if present, indicate only what their values would have been if the request method had been GET.
Source: RFC 7230 (Section 3.3)
... responses to HEAD requests do not have a body. Even if e.g. the Content-Length
header is present.
However, the response parser cannot know based on the data it receives whether the corresponding request was using the HEAD or another method and whether to expect a body or to just ignore any header flags about the content.
Suggestion:
Add an additional boolean flag to HttpResponseParser.feed_data(self, data: bytes)
to tell the parser that it should not expect a body (or even throw a parser exception if there is one present).
Because HTTP/1.1 also allows pipelining, this additional parameter should also rather be a list of "body presence" flags. Here's why:
If you send multiple requests to a server without waiting for the responses (= pipelining), you are going to receive back a bunch of responses together. Without them being parsed yet, one cannot identify the message boundaries and therefore they cannot be fed to the parser one by one. So the parser has to support that the whole stack of responses is fed to it and it needs a list of the "expected body presence".
Example:
If the pipelined requests were something like {HEAD, GET, HEAD, GET}, the "body presence" flags need to alternate as well.
For convenience, it may be good to have a parameter that can be set using both types, a single flag and a list of flags.
======================================================================
FAIL: test_parser_response_1 (test_parser.TestResponseParser)
----------------------------------------------------------------------
httptools.parser.errors.HttpParserError: invalid constant string
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/brain/rpmbuild/BUILD/httptools-0.0.9/tests/test_parser.py", line 107, in test_parser_response_1
p.feed_data(b'12123123')
AssertionError: "data received after completed connection" does not match "invalid constant string"
----------------------------------------------------------------------
This is 0.0.9 release.
I don't see any problems (apart from test suite) to work on py2. Is it unsupported or what's the status?
Summary
Installing httptools on aarch64 via pip using command "pip3 install httptools" tries to build wheel from source code
Problem description
httptools doesn't have wheel for aarch64 on PyPI repository. So, while installing httptools via pip on aarch64, pip builds wheel for same resulting in it takes more time to install httptools. Making wheel available for aarch64 will benefit aarch64 users by minimizing httptools installation time.
Expected Output
Pip should be able to download httptools wheel from PyPI repository rather than building it from source code.
@httptools-team, please let me know if I can help you building wheel/uploading to PyPI repository. I am curious to make httptools wheel available for aarch64. It will be a great opportunity for me to work with you.
Hi! (Again),
Thanks for fixing the previous CVE's which i reported here last year.
Today i noticed 3 CVE's (9.1 score) which are a bit obscurely reported by the nodeJS release notes, and which have been fixed in llhttp 6.0.7.
[https://www.opencve.io/cve?vendor=llhttp]
This morning I tried to prepare a merge-request to bump llhttp to 6.0.9, but I came to the conclusion it's best to get some advice.
It seems there's some controversie about llhttp dropping support for LF delimiting headers and requiring CR delimiters.
I'm not sure how httptools and other projects using httptools potentially could get impacted by such a change, and if there are better ways to get arround this.. or if it's not an issue.
So i stopped the process of refactoring the test-cases, to get them to just pass, and ask for help..
(I'd hoped to be of use, but this is a too big of a question for me / I'll likely hold up the process of getting this fixed properly and timely)
notes:
Add the release branch to .gitmodules:
[submodule "vendor/llhttp"] path = vendor/llhttp url = https://github.com/nodejs/llhttp.git branch = release
Use a recent git version to run:
git submodule update --remote
(or use some other methods if you prefer not to update your git client).
test_parser.py contains 'HTTP/1.2' in some of the validations causing tests to fail with an exception about invalid version
.. change the version to HTTP/1.1 everywhere
then all the remaining exceptions are about the CR 's that are required
File "httptools\parser\parser.pyx", line 212, in httptools.parser.parser.HttpParser.feed_data
httptools.parser.errors.HttpParserError: Missing expected CR after header value
Thanks again!
We have issues when passing really long URL through Sanic. It uses httptools.parse_url for parsing URL in Request object.
It seems that problem is in C++ integer limitations (values from 0 to 65535 (2**16-1)).
So, If URL is longer 65535 symbols - it will be trimmed.
https://github.com/MagicStack/httptools/blob/master/httptools/parser/parser.pyx#L406
Is it possible to implement bigint instead of int variable in httptools.parse_url or smth like that for such cases? :)
Thanks
Would it be possible to add wheels for python3.9 with a new patch release? :)
Currently it is only possible to compile the package on the fly in a virtualized environment on python3.9.
It has been noticed that while using edgedb/action-release/merge@master your gpg_key_id 5C468778062D87BF! is present in plaintext. Please ensure that secrets are encrypted or not passed as plain text in github workflows.
if any broken data chunk is fed to feed_data, it will be parsed in incomplete form.
Hello,
I am getting the following error when installing uvicorn
, with the following command:
pip install "uvicorn[standard]"
Error:
Using cached uvicorn-0.23.2-py3-none-any.whl (59 kB)
Building wheels for collected packages: httptools
Building wheel for httptools (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for httptools (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [66 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-cpython-312
creating build\lib.win-amd64-cpython-312\httptools
copying httptools\_version.py -> build\lib.win-amd64-cpython-312\httptools
copying httptools\__init__.py -> build\lib.win-amd64-cpython-312\httptools
creating build\lib.win-amd64-cpython-312\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-cpython-312\httptools\parser
copying httptools\parser\__init__.py -> build\lib.win-amd64-cpython-312\httptools\parser
running egg_info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing requirements to httptools.egg-info\requires.txt
writing top-level names to httptools.egg-info\top_level.txt
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'httptools.egg-info\SOURCES.txt'
C:\Users\asegura\AppData\Local\Temp\pip-build-env-thzd1pw8\overlay\Lib\site-packages\setuptools\command\build_py.py:204: _Warning: Package 'httptools.parser' is absent from the `packages` configuration.
!!
********************************************************************************
############################
# Package would be ignored #
############################
Python recognizes 'httptools.parser' as an importable package[^1],
but it is absent from setuptools' `packages` configuration.
This leads to an ambiguous overall configuration. If you want to distribute this
package, please make sure that 'httptools.parser' is explicitly added
to the `packages` configuration field.
Alternatively, you can also rely on setuptools' discovery methods
(for example by using `find_namespace_packages(...)`/`find_namespace:`
instead of `find_packages(...)`/`find:`).
You can read more about "package discovery" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
If you don't want 'httptools.parser' to be distributed and are
already explicitly excluding 'httptools.parser' via
`find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
you can try to use `exclude_package_data`, or `include-package-data=False` in
combination with a more fine grained `package-data` configuration.
You can read more about "package data files" on setuptools documentation page:
- https://setuptools.pypa.io/en/latest/userguide/datafiles.html
[^1]: For Python, any directory (with suitable naming) can be imported,
even if it does not contain any `.py` files.
On the other hand, currently there is no concept of package data
directory, all directories are treated like packages.
********************************************************************************
!!
check.warn(importable)
copying httptools\parser\parser.c -> build\lib.win-amd64-cpython-312\httptools\parser
copying httptools\parser\url_parser.c -> build\lib.win-amd64-cpython-312\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
error: Microsoft Visual C++ 14.0 or greater is required. Get it with "Microsoft C++ Build Tools": https://visualstudio.microsoft.com/visual-cpp-build-tools/
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for httptools
Failed to build httptools
ERROR: Could not build wheels for httptools, which is required to install pyproject.toml-based projects
And I need the httptools
package to be able to install the uvicorn
server.
Any suggestions?
By the way, I'm using Python 3.12.0 x64.
Thanks in advance.
i am working on httptools
integration with aiohttp
, some tests are failing with chunking and compression. if i modify httptools tests, it does not pass as well.
CHUNKED_REQUEST1_3 = b'''POST /test.php?a=b+c HTTP/1.2
User-Agent: Fooo
Host: bar
Transfer-Encoding: chunked
b\r\n+\xce\xcfM\xb5MI,I\x04\x00\r\n0\r\n\r\n'''
For the sake of compatibility with urlparse
and urlsplit
from the urllib.parse
standard lib module should the URL
class
schema
to scheme
host
to netloc
or that was on purpose?
I am not sure if its due to the deprecation warnings but there are failures while installing it on Python 3.10. Error log as below :
Probably related to python/cpython#20878
Running setup.py install for httptools ... error
ERROR: Command errored out with exit status 1:
command: /root/dev-denv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3qm507hv/httptools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3qm507hv/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ldf_awgo/install-record.txt --single-version-externally-managed --compile --install-headers /root/dev-denv/include/site/python3.10/httptools
cwd: /tmp/pip-install-3qm507hv/httptools/
Complete output (155 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.10
creating build/lib.linux-x86_64-3.10/httptools
copying httptools/_version.py -> build/lib.linux-x86_64-3.10/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.10/httptools
creating build/lib.linux-x86_64-3.10/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.10/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.10/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing requirements to httptools.egg-info/requires.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.10/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.10
creating build/temp.linux-x86_64-3.10/httptools
creating build/temp.linux-x86_64-3.10/httptools/parser
creating build/temp.linux-x86_64-3.10/vendor
creating build/temp.linux-x86_64-3.10/vendor/http-parser
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/root/dev-denv/include -I/root/cpython/Include -I/root/cpython -I/tmp/pip-install-3qm507hv/httptools/vendor/http-parser -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.10/httptools/parser/parser.o -O2
httptools/parser/parser.c: In function ‘__pyx_tp_dealloc_9httptools_6parser_6parser_HttpParser’:
httptools/parser/parser.c:8187:5: error: lvalue required as increment operand
++Py_REFCNT(o);
^~
httptools/parser/parser.c:8189:5: error: lvalue required as decrement operand
--Py_REFCNT(o);
^~
httptools/parser/parser.c: In function ‘__Pyx_ParseOptionalKeywords’:
httptools/parser/parser.c:10178:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10178:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10178:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10178:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10178:21: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10178:21: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**name) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘PyUnicode_AsUnicode’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:580:45: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode(
^~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c:10194:25: warning: ‘_PyUnicode_get_wstr_length’ is deprecated [-Wdeprecated-declarations]
(PyUnicode_GET_SIZE(**argname) != PyUnicode_GET_SIZE(key)) ? 1 :
^
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:446:26: note: declared here
static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
^~~~~~~~~~~~~~~~~~~~~~~~~~
httptools/parser/parser.c: In function ‘__Pyx_decode_c_bytes’:
httptools/parser/parser.c:10379:9: warning: ‘PyUnicode_FromUnicode’ is deprecated [-Wdeprecated-declarations]
return PyUnicode_FromUnicode(NULL, 0);
^~~~~~
In file included from /root/cpython/Include/unicodeobject.h:1026:0,
from /root/cpython/Include/Python.h:97,
from httptools/parser/parser.c:20:
/root/cpython/Include/cpython/unicodeobject.h:551:42: note: declared here
Py_DEPRECATED(3.3) PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode(
^~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/gcc' failed with exit code 1
----------------------------------------
ERROR: Command errored out with exit status 1: /root/dev-denv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-3qm507hv/httptools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-3qm507hv/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ldf_awgo/install-record.txt --single-version-externally-managed --compile --install-headers /root/dev-denv/include/site/python3.10/httptools Check the logs for full command output.
I got this stack trace
Collecting sanic
Using cached sanic-0.5.4-py3-none-any.whl
Requirement already satisfied: aiofiles>=0.3.0 in ./.pypyenv/site-packages (from sanic)
Collecting httptools>=0.0.9 (from sanic)
Using cached httptools-0.0.9.tar.gz
Collecting ujson>=1.35 (from sanic)
Using cached ujson-1.35.tar.gz
Collecting uvloop>=0.5.3 (from sanic)
Using cached uvloop-0.8.0.tar.gz
Collecting websockets>=3.2 (from sanic)
Using cached websockets-3.3.tar.gz
Installing collected packages: httptools, ujson, uvloop, websockets, sanic
Running setup.py install for httptools ... error
Complete output from command /Users/mrkaspa/code/py/pylangs/.pypyenv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/c1/m21ng1bd1yl83fcrjcf6qpxw0000gq/T/pip-build-rw46_3s6/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/c1/m21ng1bd1yl83fcrjcf6qpxw0000gq/T/pip-g3c9_py8-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/mrkaspa/code/py/pylangs/.pypyenv/include/site/python3.5/httptools:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.12-x86_64-3.5
creating build/lib.macosx-10.12-x86_64-3.5/httptools
copying httptools/__init__.py -> build/lib.macosx-10.12-x86_64-3.5/httptools
creating build/lib.macosx-10.12-x86_64-3.5/httptools/parser
copying httptools/parser/__init__.py -> build/lib.macosx-10.12-x86_64-3.5/httptools/parser
copying httptools/parser/errors.py -> build/lib.macosx-10.12-x86_64-3.5/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
warning: manifest_maker: standard file '-c' not found
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.macosx-10.12-x86_64-3.5/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.macosx-10.12-x86_64-3.5
creating build/temp.macosx-10.12-x86_64-3.5/httptools
creating build/temp.macosx-10.12-x86_64-3.5/httptools/parser
creating build/temp.macosx-10.12-x86_64-3.5/vendor
creating build/temp.macosx-10.12-x86_64-3.5/vendor/http-parser
gcc -pthread -DNDEBUG -O2 -I/usr/local/opt/openssl/include -fPIC -I/Users/mrkaspa/code/py/pylangs/.pypyenv/include -I/usr/local/Cellar/pypy3/5.7.1/libexec/include -c httptools/parser/parser.c -o build/temp.macosx-10.12-x86_64-3.5/httptools/parser/parser.o -O2
gcc -pthread -DNDEBUG -O2 -I/usr/local/opt/openssl/include -fPIC -I/Users/mrkaspa/code/py/pylangs/.pypyenv/include -I/usr/local/Cellar/pypy3/5.7.1/libexec/include -c vendor/http-parser/http_parser.c -o build/temp.macosx-10.12-x86_64-3.5/vendor/http-parser/http_parser.o -O2
gcc -pthread -shared -L/usr/local/opt/openssl/lib -I/usr/local/opt/openssl/include build/temp.macosx-10.12-x86_64-3.5/httptools/parser/parser.o build/temp.macosx-10.12-x86_64-3.5/vendor/http-parser/http_parser.o -o build/lib.macosx-10.12-x86_64-3.5/httptools/parser/parser.pypy3-57-x86_64-darwin.so
clang: warning: argument unused during compilation: '-pthread'
Undefined symbols for architecture x86_64:
"_PyPyBaseObject_Type", referenced from:
___pyx_tp_new_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_tp_new_9httptools_6parser_6parser_URL in parser.o
"_PyPyBuffer_Release", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyBytes_AS_STRING", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyBytes_FromString", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_3get_method in parser.o
"_PyPyBytes_FromStringAndSize", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_url in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_status in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
...
"_PyPyBytes_Size", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyBytes_Type", referenced from:
___pyx_tp_new_9httptools_6parser_6parser_URL in parser.o
"_PyPyCFunction_NewEx", referenced from:
_PyInit_parser in parser.o
"_PyPyCapsule_New", referenced from:
_PyInit_parser in parser.o
"_PyPyCode_New", referenced from:
_PyInit_parser in parser.o
___Pyx_AddTraceback in parser.o
"_PyPyDict_GetItem", referenced from:
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_1__init__ in parser.o
___pyx_tp_new_9httptools_6parser_6parser_URL in parser.o
"_PyPyDict_GetItemString", referenced from:
_PyInit_parser in parser.o
"_PyPyDict_New", referenced from:
_PyInit_parser in parser.o
"_PyPyDict_Next", referenced from:
___Pyx_ParseOptionalKeywords in parser.o
"_PyPyDict_SetItem", referenced from:
_PyInit_parser in parser.o
"_PyPyDict_SetItemString", referenced from:
_PyInit_parser in parser.o
"_PyPyDict_Size", referenced from:
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_1__init__ in parser.o
___pyx_tp_new_9httptools_6parser_6parser_URL in parser.o
"_PyPyErr_Clear", referenced from:
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_1__init__ in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyErr_ExceptionMatches", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_begin in parser.o
...
"_PyPyErr_Fetch", referenced from:
___Pyx_GetException in parser.o
___pyx_tp_dealloc_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyErr_Format", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___Pyx_ImportType in parser.o
___Pyx_Raise in parser.o
___pyx_tp_new_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
...
"_PyPyErr_GetExcInfo", referenced from:
___pyx_f_9httptools_6parser_6parser_cb_on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_begin in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_chunk_header in parser.o
...
"_PyPyErr_NoMemory", referenced from:
___pyx_tp_new_9httptools_6parser_6parser_HttpParser in parser.o
"_PyPyErr_NormalizeException", referenced from:
___Pyx_GetException in parser.o
"_PyPyErr_Occurred", referenced from:
_PyInit_parser in parser.o
___Pyx_GetException in parser.o
___Pyx_ParseOptionalKeywords in parser.o
"_PyPyErr_Restore", referenced from:
___pyx_tp_dealloc_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyErr_SetExcInfo", referenced from:
___pyx_f_9httptools_6parser_6parser_cb_on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_begin in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_chunk_header in parser.o
...
"_PyPyErr_SetObject", referenced from:
___Pyx_Raise in parser.o
"_PyPyErr_SetString", referenced from:
_PyInit_parser in parser.o
___Pyx_Raise in parser.o
"_PyPyErr_WarnEx", referenced from:
_PyInit_parser in parser.o
___Pyx_ImportType in parser.o
"_PyPyExc_AttributeError", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_1__init__ in parser.o
"_PyPyExc_BaseException", referenced from:
___Pyx_Raise in parser.o
"_PyPyExc_ImportError", referenced from:
_PyInit_parser in parser.o
"_PyPyExc_NameError", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyExc_TypeError", referenced from:
___Pyx_ImportType in parser.o
___Pyx_Raise in parser.o
___pyx_tp_new_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
___Pyx_ParseOptionalKeywords in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_1__init__ in parser.o
___pyx_tp_new_9httptools_6parser_6parser_URL in parser.o
...
"_PyPyExc_ValueError", referenced from:
___Pyx_ImportType in parser.o
"_PyPyException_SetTraceback", referenced from:
___Pyx_GetException in parser.o
"_PyPyFrame_New", referenced from:
___Pyx_AddTraceback in parser.o
"_PyPyImport_AddModule", referenced from:
_PyInit_parser in parser.o
"_PyPyImport_GetModuleDict", referenced from:
_PyInit_parser in parser.o
"_PyPyImport_Import", referenced from:
___Pyx_ImportType in parser.o
"_PyPyImport_ImportModuleLevelObject", referenced from:
_PyInit_parser in parser.o
"_PyPyList_New", referenced from:
_PyInit_parser in parser.o
"_PyPyList_SET_ITEM", referenced from:
_PyInit_parser in parser.o
"_PyPyLong_FromLong", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_5get_http_version in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_7should_keep_alive in parser.o
___pyx_pw_9httptools_6parser_6parser_18HttpResponseParser_3get_status_code in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyLong_FromSize_t", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyMem_Free", referenced from:
___pyx_tp_dealloc_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyMem_Malloc", referenced from:
___Pyx_AddTraceback in parser.o
___pyx_tp_new_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyMem_Realloc", referenced from:
___Pyx_AddTraceback in parser.o
"_PyPyModule_Create2", referenced from:
_PyInit_parser in parser.o
"_PyPyModule_GetDict", referenced from:
_PyInit_parser in parser.o
"_PyPyNumber_InPlaceAdd", referenced from:
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_header_value in parser.o
"_PyPyOS_snprintf", referenced from:
_PyInit_parser in parser.o
___Pyx_ImportType in parser.o
"_PyPyObject_Call", referenced from:
___pyx_f_9httptools_6parser_6parser_10HttpParser__maybe_call_on_header in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_complete in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_begin in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_message_complete in parser.o
...
"_PyPyObject_GC_UnTrack", referenced from:
___pyx_tp_dealloc_9httptools_6parser_6parser_HttpParser in parser.o
___pyx_tp_dealloc_9httptools_6parser_6parser_URL in parser.o
"_PyPyObject_GetAttr", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___Pyx_ImportType in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_5get_http_version in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_17HttpRequestParser_1__init__ in parser.o
...
"_PyPyObject_GetBuffer", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyObject_GetItem", referenced from:
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyObject_IsInstance", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyObject_IsSubclass", referenced from:
___Pyx_Raise in parser.o
"_PyPyObject_IsTrue", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_7should_keep_alive in parser.o
"_PyPyObject_SetAttr", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyObject_SetAttrString", referenced from:
_PyInit_parser in parser.o
"_PyPyThreadState_Get", referenced from:
___Pyx_AddTraceback in parser.o
"_PyPyTraceBack_Here", referenced from:
___Pyx_AddTraceback in parser.o
"_PyPyTuple_New", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__maybe_call_on_header in parser.o
___Pyx_Raise in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_5get_http_version in parser.o
___pyx_pw_9httptools_6parser_6parser_3URL_3__repr__ in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyTuple_Pack", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_body in parser.o
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_url in parser.o
___pyx_f_9httptools_6parser_6parser_cb_on_status in parser.o
___pyx_pw_9httptools_6parser_6parser_1parse_url in parser.o
"_PyPyType_Check", referenced from:
___Pyx_ImportType in parser.o
___Pyx_Raise in parser.o
"_PyPyType_Ready", referenced from:
_PyInit_parser in parser.o
"_PyPyUnicode_Check", referenced from:
___Pyx_ParseOptionalKeywords in parser.o
"_PyPyUnicode_Compare", referenced from:
___Pyx_ParseOptionalKeywords in parser.o
"_PyPyUnicode_Decode", referenced from:
_PyInit_parser in parser.o
"_PyPyUnicode_DecodeLatin1", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyUnicode_FromFormat", referenced from:
___Pyx_AddTraceback in parser.o
"_PyPyUnicode_FromString", referenced from:
___Pyx_ImportType in parser.o
___Pyx_AddTraceback in parser.o
"_PyPyUnicode_FromStringAndSize", referenced from:
_PyInit_parser in parser.o
"_PyPyUnicode_FromUnicode", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_9feed_data in parser.o
"_PyPyUnicode_InternFromString", referenced from:
_PyInit_parser in parser.o
"_PyPy_GetVersion", referenced from:
_PyInit_parser in parser.o
"__PyPy_Dealloc", referenced from:
_PyInit_parser in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__maybe_call_on_header in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
...
"__PyPy_NoneStruct", referenced from:
___pyx_f_9httptools_6parser_6parser_10HttpParser__init in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__maybe_call_on_header in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_header_field in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_header_value in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_headers_complete in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_header in parser.o
___pyx_f_9httptools_6parser_6parser_10HttpParser__on_chunk_complete in parser.o
...
"__PyPy_TrueStruct", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_7should_keep_alive in parser.o
"__PyPy_ZeroStruct", referenced from:
___pyx_pw_9httptools_6parser_6parser_10HttpParser_7should_keep_alive in parser.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/Users/mrkaspa/code/py/pylangs/.pypyenv/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/c1/m21ng1bd1yl83fcrjcf6qpxw0000gq/T/pip-build-rw46_3s6/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /var/folders/c1/m21ng1bd1yl83fcrjcf6qpxw0000gq/T/pip-g3c9_py8-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/mrkaspa/code/py/pylangs/.pypyenv/include/site/python3.5/httptools" failed with error code 1 in /private/var/folders/c1/m21ng1bd1yl83fcrjcf6qpxw0000gq/T/pip-build-rw46_3s6/httptools/
While installing the module i get the following error
Collecting httptools
Using cached https://files.pythonhosted.org/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz
Building wheels for collected packages: httptools
Building wheel for httptools (setup.py) ... error
Complete output from command c:\python\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\MOHAMM~1\\AppData\\Local\\Temp\\pip-install-3uy2x1yx\\httptools\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d C:\Users\MOHAMM~1\AppData\Local\Temp\pip-wheel-ha0o16_d --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\httptools
copying httptools\__init__.py -> build\lib.win-amd64-3.6\httptools
creating build\lib.win-amd64-3.6\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-3.6\httptools\parser
copying httptools\parser\__init__.py -> build\lib.win-amd64-3.6\httptools\parser
running egg_info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing top-level names to httptools.egg-info\top_level.txt
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info\SOURCES.txt'
copying httptools\parser\parser.c -> build\lib.win-amd64-3.6\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/
----------------------------------------
Failed building wheel for httptools
Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: httptools
Running setup.py install for httptools ... error
Complete output from command c:\python\python.exe -u -c "import setuptools, tokenize;__file__='C:\\Users\\MOHAMM~1\\AppData\\Local\\Temp\\pip-install-3uy2x1yx\\httptools\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record C:\Users\MOHAMM~1\AppData\Local\Temp\pip-record-schzesxc\install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build\lib.win-amd64-3.6
creating build\lib.win-amd64-3.6\httptools
copying httptools\__init__.py -> build\lib.win-amd64-3.6\httptools
creating build\lib.win-amd64-3.6\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-3.6\httptools\parser
copying httptools\parser\__init__.py -> build\lib.win-amd64-3.6\httptools\parser
running egg_info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing top-level names to httptools.egg-info\top_level.txt
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info\SOURCES.txt'
copying httptools\parser\parser.c -> build\lib.win-amd64-3.6\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/
I am on windows and I have installed Visual C++ Build Tools. But the problem isnt resolved yet
Simply requesting the ability to download and install through the whl file
this happened when I try to install httptools, one error shows up :
Microsoft Visual C++ 14.0 is required.
But I can not find where to download and install it.
I need help.
httptools-master>python setup.py install
running install
running bdist_egg
running egg_info
creating httptools.egg-info
writing httptools.egg-info\PKG-INFO
writing dependency_links to httptools.egg-info\dependency_links.txt
writing top-level names to httptools.egg-info\top_level.txt
writing manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest file 'httptools.egg-info\SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '.c' under directory 'vendor'
warning: no files found matching '.h' under directory 'vendor'
warning: no files found matching 'LICENSE*' under directory 'vendor'
warning: no files found matching 'README*' under directory 'vendor'
writing manifest file 'httptools.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
creating build
creating build\lib.win-amd64-3.7
creating build\lib.win-amd64-3.7\httptools
copying httptools_init_.py -> build\lib.win-amd64-3.7\httptools
creating build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser\errors.py -> build\lib.win-amd64-3.7\httptools\parser
copying httptools\parser_init_.py -> build\lib.win-amd64-3.7\httptools\parser
running build_ext
building 'httptools.parser.parser' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": https://visualstudio.microsoft.com/downloads/
Would you accept a Python 2 support PR?
It looks like not a lot of code changes are required.
I played around a little, here are my findings:
the Cython code just compiles and seems to work on Python 2 unchanged
the __all__
in the __init__.py
fails, because I think __all__
in Cython is a unicode object, something Python 2 doesn't like. I'd advocate changing the __init__.py
to explicitly import everything needed instead of doing a from .. import *
. This way you are in full control of what's exposed in the public API.
the test suite relies on some features from Python 3:
unitest.mock
, which is available as mock
on PyPI so that's easyassertRaisesRegex
which is missing. It'd need to be backported, or the test rewrittensubTest
, either backported or the test rewritten.I'm a fan of pytest myself so alternatively we can port the tests over over to that; it's compatible
with Python 2 and Python 3. I'd be happier doing that instead.
I recommend a tool like tox so we can easily run the tests on both Pythons.
the Makefile is geared towards Python 3, but doesn't do all that much. I just called the build_ext command myself with Python 2. We could cruftify the makefile to support Python 2. Alternatively we could also just simply document the few standard Python commands needed; tox should care of building stuff and running stuff anyway and that's a simple command.
Hello!
Based on tradition settled in #59 :)
Could you please provide Python 3.10 wheels for 0.2.0? uvicorn[standard] (0.15.0)
tries to install this version.
Now it seems to be impossible to install httptools
in pipenv environment on Windows without Microsoft C++ Build Tools
which are huge to have on dev machine just for httptools installation :)
The same problem arises in Linux with Python containers without gcc
New node js parser: nodejs/http-parser#285 (comment)
The httptools 1.0.0 release notes say that there should be no breaking change, but upgrading httptools from version 0.0.13 breaks uvicorn tests:
PR: encode/uvicorn#569
Test: https://travis-ci.org/encode/uvicorn/jobs/647886597
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.11
creating build/temp.linux-x86_64-3.11/httptools
creating build/temp.linux-x86_64-3.11/httptools/parser
creating build/temp.linux-x86_64-3.11/vendor
creating build/temp.linux-x86_64-3.11/vendor/llhttp
creating build/temp.linux-x86_64-3.11/vendor/llhttp/src
x86_64-linux-gnu-gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/tmp/pip-req-build-cyh4gi6y/vendor/llhttp/include -I/tmp/pip-req-build-cyh4gi6y/vendor/llhttp/src -I/home/dima/.cache/pypoetry/virtualenvs/silly-chat-9D3wOO4G-py3.11/include -I/usr/include/python3.11 -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.11/httptools/parser/parser.o -O2
httptools/parser/parser.c: In function ‘__pyx_pf_9httptools_6parser_6parser_10HttpParser_10feed_data’:
httptools/parser/parser.c:3452:23: warning: assignment discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
3452 | __pyx_v_err_pos = llhttp_get_error_pos(__pyx_v_self->_cparser);
| ^
httptools/parser/parser.c: In function ‘__Pyx_AddTraceback’:
httptools/parser/parser.c:454:62: error: dereferencing pointer to incomplete type ‘PyFrameObject’ {aka ‘struct _frame’}
454 | #define __Pyx_PyFrame_SetLineNumber(frame, lineno) (frame)->f_lineno = (lineno)
| ^~
httptools/parser/parser.c:9421:5: note: in expansion of macro ‘__Pyx_PyFrame_SetLineNumber’
9421 | __Pyx_PyFrame_SetLineNumber(py_frame, py_line);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
[end of output]
httptools support gevent modules?
pip install httptools
with gcc-8.3 fails, with an error about unrecognized file formats when linking (see below). This is fair enough, maybe this recent version is unsupported.
Collecting httptools
Downloading https://files.pythonhosted.org/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz (104kB)
|████████████████████████████████| 112kB 7.4MB/s
Building wheels for collected packages: httptools
Building wheel for httptools (setup.py) ... error
ERROR: Complete output from command /home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-xrfgwi79/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-8yjnvozz --python-tag cp36:
ERROR: running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
gcc -pthread -B /home/matthew/.local/conda/envs/optimade/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
gcc -pthread -B /home/matthew/.local/conda/envs/optimade/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c vendor/http-parser/http_parser.c -o build/temp.linux-x86_64-3.6/vendor/http-parser/http_parser.o -O2
gcc -pthread -shared -B /home/matthew/.local/conda/envs/optimade/compiler_compat -L/home/matthew/.local/conda/envs/optimade/lib -Wl,-rpath=/home/matthew/.local/conda/envs/optimade/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/httptools/parser/parser.o build/temp.linux-x86_64-3.6/vendor/http-parser/http_parser.o -o build/lib.linux-x86_64-3.6/httptools/parser/parser.cpython-36m-x86_64-linux-gnu.so
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-3.6/httptools/parser/parser.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for httptools
Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: httptools
Running setup.py install for httptools ... error
ERROR: Complete output from command /home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-xrfgwi79/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_hgd3hzc/install-record.txt --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
gcc -pthread -B /home/matthew/.local/conda/envs/optimade/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
gcc -pthread -B /home/matthew/.local/conda/envs/optimade/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c vendor/http-parser/http_parser.c -o build/temp.linux-x86_64-3.6/vendor/http-parser/http_parser.o -O2
gcc -pthread -shared -B /home/matthew/.local/conda/envs/optimade/compiler_compat -L/home/matthew/.local/conda/envs/optimade/lib -Wl,-rpath=/home/matthew/.local/conda/envs/optimade/lib -Wl,--no-as-needed -Wl,--sysroot=/ build/temp.linux-x86_64-3.6/httptools/parser/parser.o build/temp.linux-x86_64-3.6/vendor/http-parser/http_parser.o -o build/lib.linux-x86_64-3.6/httptools/parser/parser.cpython-36m-x86_64-linux-gnu.so
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
/home/matthew/.local/conda/envs/optimade/compiler_compat/ld: build/temp.linux-x86_64-3.6/httptools/parser/parser.o: unable to initialize decompress status for section .debug_info
build/temp.linux-x86_64-3.6/httptools/parser/parser.o: file not recognized: file format not recognized
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-xrfgwi79/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_hgd3hzc/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-xrfgwi79/httptools/
I tried again with gcc-4.9 (CC=gcc-4.9 CXX=gcc-4.9
), and I receive a different error about a missing library (libmpfr.so.4
; I have libmpfr.so.6
). Symlinking libmpfr.so.6
to libmpfr.so.4
gets me back to the same error as gcc-8.3:
$ CC=gcc-4.9 CXX=gcc-4.9 pip --no-cache-dir install httptools
Collecting httptools
Downloading https://files.pythonhosted.org/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz (104kB)
|████████████████████████████████| 112kB 7.5MB/s
Building wheels for collected packages: httptools
Building wheel for httptools (setup.py) ... error
ERROR: Complete output from command /home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-bso7iy81/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-pvaohfeg --python-tag cp36:
ERROR: running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
gcc-4.9 -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/cc1: error while loading shared libraries: libmpfr.so.4: cannot open shared object file: No such file or directory
error: command 'gcc-4.9' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for httptools
Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: httptools
Running setup.py install for httptools ... error
ERROR: Complete output from command /home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-bso7iy81/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-t5796_ix/install-record.txt --single-version-externally-managed --compile:
ERROR: running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
gcc-4.9 -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/home/matthew/.local/conda/envs/optimade/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.9.3/cc1: error while loading shared libraries: libmpfr.so.4: cannot open shared object file: No such file or directory
error: command 'gcc-4.9' failed with exit status 1
----------------------------------------
ERROR: Command "/home/matthew/.local/conda/envs/optimade/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-bso7iy81/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-t5796_ix/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-bso7iy81/httptools/
Notice that the third call to the C compiler is to `gcc`, not `gcc-4.9` as set by my environment variables. Running `CC=clang CXX=clang pip --no-cache-dir install httptools` works fine. As GCC is the only compiler guaranteed to be installed in almost every Linux distro, do you know how this could be rectified? Thanks.
Behind a reverse-proxy or a load balancer, you often need to get the original client IP.
A lot of load balancer provide these information with the PROXY protocol - HAproxy for exemple.
I have a load balancer that provide only this method to get the client info, and I would not be the only one.
Here a doc of the protocol: http://www.haproxy.org/download/2.4/doc/proxy-protocol.txt
In a nutshell, it's one line added on top of the body, here for the v1 protocol:
PROXY TCP4 192.168.0.1 192.168.0.11 56324 443\r\n
GET / HTTP/1.1\r\n
Host: 192.168.0.11\r\n
\r\n
This protocol is implemented in Gunicorn
, and can be activated with --proxy-protocol
: https://docs.gunicorn.org/en/stable/settings.html#proxy-protocol
edit: A NodeJS parser: https://github.com/racker/node-proxy-protocol/blob/master/index.js
Any idea on possible workaround:
After pip3 install httptools
:
Collecting httptools
Using cached https://files.pythonhosted.org/packages/1b/03/215969db11abe8741e9c266a4cbe803a372bd86dd35fa0084c4df6d4bd00/httptools-0.0.13.tar.gz
Building wheels for collected packages: httptools
Building wheel for httptools (setup.py) ... error
Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ffzualnz/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-lq_cgux2 --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
httptools/parser/parser.c:4:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Failed building wheel for httptools
Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: httptools
Running setup.py install for httptools ... error
Complete output from command /usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ffzualnz/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-xaxh1zu8/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/httptools
copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
creating build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
running egg_info
writing httptools.egg-info/PKG-INFO
writing dependency_links to httptools.egg-info/dependency_links.txt
writing top-level names to httptools.egg-info/top_level.txt
reading manifest file 'httptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'httptools.egg-info/SOURCES.txt'
copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
running build_ext
building 'httptools.parser.parser' extension
creating build/temp.linux-x86_64-3.6
creating build/temp.linux-x86_64-3.6/httptools
creating build/temp.linux-x86_64-3.6/httptools/parser
creating build/temp.linux-x86_64-3.6/vendor
creating build/temp.linux-x86_64-3.6/vendor/http-parser
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
httptools/parser/parser.c:4:20: fatal error: Python.h: No such file or directory
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ffzualnz/httptools/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-xaxh1zu8/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-ffzualnz/httptools/
Ubuntu 16.04 LTS
Got error:
↳ python3 setup.py build_ext --inplace
running build_ext
building 'httptools.parser.parser' extension
/usr/local/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Users/guozixing/Idea/newpy/env/include -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c httptools/parser/parser.c -o build/temp.macosx-10.6-intel-3.6/httptools/parser/parser.o -O2
/usr/local/bin/clang -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Users/guozixing/Idea/newpy/env/include -I/Library/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c vendor/http-parser/http_parser.c -o build/temp.macosx-10.6-intel-3.6/vendor/http-parser/http_parser.o -O2
/usr/local/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-3.6/httptools/parser/parser.o build/temp.macosx-10.6-intel-3.6/vendor/http-parser/http_parser.o -o build/lib.macosx-10.6-intel-3.6/httptools/parser/parser.cpython-36m-darwin.so
ld: in '/usr/local/lib/libunwind.dylib', file was built for x86_64 which is not the architecture being linked (i386): /usr/local/lib/libunwind.dylib for architecture i386
clang-3.9: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/local/bin/clang' failed with exit status 1
The below command
/usr/local/bin/clang -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-3.6/httptools/parser/parser.o build/temp.macosx-10.6-intel-3.6/vendor/http-parser/http_parser.o -o build/lib.macosx-10.6-intel-3.6/httptools/parser/parser.cpython-36m-darwin.so
should be
/usr/local/bin/clang -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.6-intel-3.6/httptools/parser/parser.o build/temp.macosx-10.6-intel-3.6/vendor/http-parser/http_parser.o -o build/lib.macosx-10.6-intel-3.6/httptools/parser/parser.cpython-36m-darwin.so
remove the -arch i386
param and all passed.
My system info:
↳ uname -a
Darwin Mac.local 16.1.0 Darwin Kernel Version 16.1.0: Thu Oct 13 21:26:57 PDT 2016; root:xnu-3789.21.3~60/RELEASE_X86_64 x86_64
I see #59 has provide a wheel for python 3.9,but pip install always Using a httptools-0.4.0.tar.gz and try to build it:
ERROR: Failed building wheel for httptools
How to get access to request body? Something like this: httptools.HttpRequestParser.get_body()
Currently the wheel for Python 3.12 does not exist.
Due to this, we have to manually build it, which is a pain if you don't have all the dev dependencies installed. see issue #93
On the pip install
compilation:
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -Qunused-arguments� -Wno-unused-function -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.5m -I/home/cacilhas/tmp/test/__/include/python3.5m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.5/httptools/parser/parser.o -O2
x86_64-linux-gnu-gcc: error: unrecognized command line option ‘-Qunused-arguments� -Wno-unused-function’
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
The -Qunused-arguments�
parameter has a LF and a bizarre 001E characters at the end as part of the parameter, making GCC to fail.
Python 2.7.13 (5.8.0+dfsg-2~ppa2~ubuntu14.04, Jun 17 2017, 18:51:40)
[PyPy 5.8.0 with GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>> import httptools
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/chengs/playground/pypy5.8env/.env/site-packages/httptools/__init__.py", line 1, in <module>
from .parser import *
TypeError: 'Item in ``fromlist'' must be str, not unicode
httptools support gevent with python3.x
I hope I'm not mistaking,
I notice the build/ PyPI package needs a update due to CVE-2021-22959 & CVE-2021-22960 in the NodeJS llhttp parser.
https://nodejs.org/fr/blog/vulnerability/oct-2021-security-releases/
thanks!
here test code. also i am not sure how to get data after end of message back. i am about \x89\x04data
import httptools
import unittest
from unittest import mock
UPGRADE_RESPONSE = b'''HTTP/1.1 101 Switching Protocols
UPGRADE: websocket
SEC-WEBSOCKET-ACCEPT: rVg+XakFNFOxk3ZH0lzrZBmg0aU=
TRANSFER-ENCODING: chunked
CONNECTION: upgrade
DATE: Sat, 07 May 2016 23:44:32 GMT
SERVER: Python/3.4 aiohttp/1.0.3
\r\n\x89\x04data'''
class TestResponseParser(unittest.TestCase):
def test_parser_upgrade_response(self):
m = mock.Mock()
headers = {}
m.on_header.side_effect = headers.__setitem__
p = httptools.HttpResponseParser(m)
try:
p.feed_data(UPGRADE_RESPONSE)
except:
pass
self.assertEqual(p.get_http_version(), '1.1')
self.assertEqual(p.get_status_code(), 101)
m.on_status.assert_called_once_with(b'Switching Protocols')
m.on_headers_complete.assert_called_once_with()
self.assertEqual(m.on_header.call_count, 6)
self.assertEqual(len(headers), 6)
m.on_message_complete.assert_called_once_with()
Hi there 👋
I'm trying to implement a flag --limit-request-header-count
on uvicorn
, which I implemented changing the value of an attribute too_many_headers
and raising an exception from on_header
callback, see https://github.com/encode/uvicorn/pull/1683/files.
My question is: is my implementation as intended? Should the HttpParserCallbackError
send more information when an exception happens from one of the callbacks?
greetings, while I see the following issue: #43 has been closed I wondered since the parser was changed if this could be reevaluated or if we should handle that downstream in uvicorn,
this added test will fail if the url us 2^16-1 bytes long and the returned url is b''
def test_parser_url_11(self):
LARGE_URL = b'/' + b'a' * (2**16-1)
self.assertEqual(self.parse(LARGE_URL), (None, None, None, LARGE_URL, None, None, None))
Failure
Traceback (most recent call last):
File "/home/lotso/PycharmProjects/httptools/tests/test_parser.py", line 630, in test_parser_url_11
self.assertEqual(self.parse(LARGE_URL), (None, None, None, LARGE_URL, None, None, None))
AssertionError: Tuples differ: (None, None, None, b'', None, None, None) != (None, None, None, b'/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[65510 chars]None)
First differing element 3:
b''
b'/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[65491 chars]aaaa'
Diff is 71676 characters long. Set self.maxDiff to None to see it.
note the issue is slightly different here we get b''
while in old issue the url was trimmed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.