Comments (20)
@nateprewitt this is a different side-effect of #6655. I think to enable this particular use case we need to expose a way for sub-classes to muck with the pool kwargs, something like
class HTTPAdapter(BaseAdapter):
def build_pool_kwargs(self, request, verify, cert=None):
return _urllib3_request_context(request, verify, cert)
That way a subclass like this could do something like:
class CustomSSLAdapter(HTTPAdapter):
def build_pool_kwargs(self, request, verify, cert=None):
params, kwargs = super().build_pool_kwargs(request, verify, cert)
params["ssl_context"] = self.context
Thoughts? I'm not particularly fond of this but it seems like the least intrusive way to preserve the fix for the CVE while giving people the ability to use custom SSLContexts the way they already were
from requests.
This is running on databricks using python 3.10.12.
from requests.
We have a different error, but probably the same issue. We inject a ssl_context
with a custom cipher selection. If there is a more "correct" way to do this I can also update our code, but a fix in the library would be appreciated.
Anyway, thanks for the quick response.
from requests.
You will need to fix your code but right now it's not easily doable
from requests.
when do you expect releasing v2.32.3 which is suggested at #6716?
from requests.
@DanSIntel See the conversation in the MR: #6716 (review)
from requests.
On requests 2.32.x with truststore we are seeing a maximum recursion error with verify=True on windows Jupyter labs and jupyter notebook environments.
import requests
with requests.Session() as session:
print(session.get("https://www.arcgis.com/sharing/rest/portals/self?f=json").text)
from requests.
@achapkowski I don't see trust store being used there or any kind of traceback
from requests.
I also can't reproduce your claim @achapkowski but I don't have windows available:
>>> import truststore; truststore.inject_into_ssl()
>>> import requests
>>> requests.get('https://google.com')
<Response [200]>
>>> requests.get('https://www.arcgis.com/sharing/rest/portals/self?f=json')
<Response [200]>
from requests.
It recursively occurs here.
full stack attached:
from requests.
Dug in a bit more:
import requests
print(requests.get("https://www.arcgis.com/sharing/rest/portals/self?f=json").text)
with requests.Session() as session:
print(session.get("https://www.arcgis.com/sharing/rest/portals/self?f=json").text)
The above works on windows
Below will fails on windows. Works on Mac/Linux
import truststore
truststore.inject_into_ssl()
with requests.Session() as session:
print(session.get("https://www.arcgis.com/sharing/rest/portals/self?f=json").text)
from requests.
I posted a response to the recursion issue in #6716 (comment). This is already known behavior that was reported to be broken in arcgis previously due to misuse of truststore (Esri/arcgis-python-api#1698). It will need to be fixed there. That's unrelated to the topic of this issue.
For the minimal repro, you're producing this because your repro is also misusing truststore in the same way as arcgis
. truststore
MUST be imported before any networking code for either urllib3 or Requests. inject_into_ssl()
is not an intended entry point for any library or package code. You'll find a large warning at the top of the user guide directing users to avoid this.
You can swap the order of your imports and the above repro works fine on Windows:
# Must be done first
import truststore
truststore.inject_into_ssl()
import requests
with requests.Session() as session:
print(session.get("https://www.arcgis.com/sharing/rest/portals/self?f=json").text)
from requests.
I had a similar issue. Try setting the REQUESTS_CA_BUNDLE=
environment variable if possible to point to your CA bundle
from requests.
We used pip-system-certs to trust the os bundle which has custom CAs in. I dont fully understand why but since 2.32.0 we get what I think is the same issue. Someone has raised an issue over at gitab https://gitlab.com/alelec/pip-system-certs/-/issues/27
from requests.
#6721 is staged for release on Tuesday. We're going to hold releasing over the weekend to avoid any unnecessary pain for ops teams and will release after the holiday in the US on Monday. You can follow along for the merge of that PR to track the release time.
from requests.
For the adapter issue with 2.32.x do developers need to modify their logic? Is that documented anywhere if so?
Also thank you for the quick fix!
from requests.
@achapkowski If you're using the init_poolmanager
setup now like what's linked in the example, you shouldn't need to make any additional changes. 2.32.3 should work the same as 2.31.0 and earlier. If you want to do anything more than what's described in the first post of this issue, the new build_connection_pool_key_attributes
will be the escape hatch to do any additional customization.
from requests.
Alright, 2.32.3 is out. Thanks everyone for your patience, please let us know if you're still hitting this failure case after upgrading.
from requests.
Alright, 2.32.3 is out. Thanks everyone for your patience, please let us know if you're still hitting this failure case after upgrading.
With 2.32.0-2.32.2 I've been seeing the same issue when using requests_pkcs12; now with 2.32.3, the issue did unfortunately not disappear but rather move to HTTPSConnectionPool(host='some.system', port=443): Max retries exceeded with url: /foo/bar (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)')))
.
I've opened an issue with requests_pkcs12, but since you specifically asked to point out problems where things do not behave like in 2.31.0...
from requests.
We also use requests_pkcs12 and I can confirm that 2.32.3 doesn't work with it. Stack trace I saw:
Traceback (most recent call last): File "/workspace/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( ^^^^^^^^^^^^^^^^^^^ File "/workspace/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/workspace/.venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/workspace/.venv/lib/python3.11/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrap_socket( ^^^^^^^^^^^^^^^^ File "/workspace/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl( ^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/.venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket return self.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/ssl.py", line 1108, in _create self.do_handshake() File "/usr/lib/python3.11/ssl.py", line 1379, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)
from requests.
Related Issues (20)
- requests library seems to ignore "Transfer-Encoding" header HOT 1
- ValueError: Timeout value connect was <object object at 0x7c6b5e484a80>, but it must be an int, float or None. HOT 1
- Requests v2.32.0 caused the error `Segmentation fault` when including the `cert` parameter HOT 2
- Check for codes HOT 1
- Unclear file handling in documentation examples HOT 2
- catch FileNotFoundError if there aren't any certificates HOT 2
- requests > 2.31.0 - Unhashable type 'list' for 'cert' parameter HOT 1
- RFC9113 - HTTP/2.0 HOT 1
- Possible .get bug in version 2.32.3 HOT 2
- PDF download is distorted using requests HOT 1
- Frequently hangs or times out when trying to fetch https://openssl.org/source/ HOT 3
- I am having error on requests.post HOT 1
- Error when headers return 'Location' as an app://xxxx although I have set disable redirects HOT 3
- requests library seems to ignore "Transfer-Encoding" header HOT 1
- Request can't get berkshirehathaway letters.html correctly HOT 1
- "Transfer-Encoding" header is ignored HOT 1
- Body with Special Characters Gets Cut HOT 9
- permission denied regression reading extracted certs with multiple users HOT 1
- 一个while True requests运行的爬虫程序,会报错:HTTPSConnectionPool(host='***', port=443): Max retries exceeded with url: /aa (Caused by SSLError(OSError(24, 'Too many open files'))) HOT 1
- Same code works with Python 3.10 but not with 3.11
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from requests.