sethmlarson / truststore Goto Github PK
View Code? Open in Web Editor NEWVerify certificates using OS trust stores
Home Page: https://truststore.readthedocs.io
License: MIT License
Verify certificates using OS trust stores
Home Page: https://truststore.readthedocs.io
License: MIT License
Shouldn't be receiving any warnings when using urllib3 and aiohttp.
Have different messages for revoked certificates, weak certificates, etc.
I suspect this will be possible in both macOS and Windows trust store APIs.
Hi, I'm generally quite in favour of keeping up with latest stable python however there are many common OS / distros which still ship with Putin 3.8 / 3.9.
Are there particular features of 3.10 that this library depends on that aren't available in older versions?
As per #75 I'm interested in ways to use this everywhere by default, however the 3.10 minimum feels a bit restrictive still.
When running the test suite on pypy3.10 7.3.12, every test fails with the following error:
_____________________________________________________ test_verify_mode_cert_none ______________________________________________________
def test_verify_mode_cert_none():
> ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
tests/test_sslcontext.py:39:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/pypy3.10/ssl.py:496: in __new__
self = _SSLContext.__new__(cls, protocol)
/usr/lib/pypy3.10/_cffi_ssl/_stdssl/__init__.py:1221: in __new__
lib.SSL_CTX_set_post_handshake_auth(self.ctx, self.post_handshake_auth)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <truststore._api.SSLContext object at 0x00005599f6700138>
@property
def post_handshake_auth(self) -> bool:
> return self._ctx.post_handshake_auth
E AttributeError: 'SSLContext' object has no attribute '_ctx'. Did you mean: 'ctx'?
.nox/test/lib/pypy3.10/site-packages/truststore/_api.py:232: AttributeError
First: Awesome work. Certificate issue in enterprise is the main cause of at least 10% of my grey hair.
Second: I watched your talk from 2022 and read the documentation and release log and still not fully confident. Is this ready for production now? Can I already import this magic and have it fix my problem?
SSLContext has a _load_windows_store_certs
method which is called by SSLContext.load_default_certs
to load CA certs on Windows. We could replace it with a method that does nothing to avoid unnecessary work.
Failing consistently on only macOS with the following error:
______________________ test_async_httpx_works_with_inject ______________________
cls = <class '_pytest.runner.CallInfo'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x11222e320>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)
@classmethod
def from_call(
cls,
func: "Callable[[], TResult]",
when: "Literal['collect', 'setup', 'call', 'teardown']",
reraise: Optional[
Union[Type[BaseException], Tuple[Type[BaseException], ...]]
] = None,
) -> "CallInfo[TResult]":
"""Call func, wrapping the result in a CallInfo.
:param func:
The function to call. Called without arguments.
:param when:
The phase in which the function is called.
:param reraise:
Exception or exceptions that shall propagate if raised by the
function, instead of being wrapped in the CallInfo.
"""
excinfo = None
start = timing.time()
precise_start = timing.perf_counter()
try:
> result: Optional[TResult] = func()
.nox/test/lib/python3.10/site-packages/_pytest/runner.py:339:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.nox/test/lib/python3.10/site-packages/_pytest/runner.py:260: in <lambda>
lambda: ihook(item=item, **kwds), when=when, reraise=reraise
.nox/test/lib/python3.10/site-packages/pluggy/_hooks.py:265: in __call__
return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
.nox/test/lib/python3.10/site-packages/pluggy/_manager.py:80: in _hookexec
return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
.nox/test/lib/python3.10/site-packages/_pytest/unraisableexception.py:88: in pytest_runtest_call
yield from unraisable_exception_runtest_hook()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
with catch_unraisable_exception() as cm:
yield
if cm.unraisable:
if cm.unraisable.err_msg is not None:
err_msg = cm.unraisable.err_msg
else:
err_msg = "Exception ignored in"
msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
msg += "".join(
traceback.format_exception(
cm.unraisable.exc_type,
cm.unraisable.exc_value,
cm.unraisable.exc_traceback,
)
)
> warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E pytest.PytestUnraisableExceptionWarning: Exception ignored in: <socket.socket fd=-1, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0>
E
E Traceback (most recent call last):
E File "/Users/runner/work/truststore/truststore/.nox/test/lib/python3.10/site-packages/anyio/_core/_eventloop.py", line 153, in get_asynclib
E return sys.modules[modulename]
E KeyError: 'anyio._backends._asyncio'
E
E During handling of the above exception, another exception occurred:
E
E Traceback (most recent call last):
E File "/Users/runner/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/ast.py", line 50, in parse
E return compile(source, filename, mode, flags,
E ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=0, laddr=('::1', 9999, 0, 0)>
.nox/test/lib/python3.10/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning
We recently encountered an interesting interaction with boto3 and arcgis due to arcgis' adoption of truststore
(boto/boto3#3912). Boto3 has existing usage of the SSLContext from urllib3.utils.ssl_
to work around historical issues with pyopenssl. Most of that no longer exists, but the imports/behaviors are left in place for backwards compatibility.
When truststore.inject_into_ssl()
is called, it's currently patching the SSLContext referenced in urllib3 resulting in a RecusionError
depending on the order these operations are performed. I've created a minimal repro with only Truststore and urllib3.
from urllib3.util.ssl_ import SSLContext, PROTOCOL_TLS_CLIENT
import truststore
truststore.inject_into_ssl()
s = SSLContext(PROTOCOL_TLS_CLIENT)
s.options |= 0 # Any arbitrary options int
This issue is avoided if the truststore
injection is done first. This is fine for code you control, but becomes more complicated to root cause when a module using truststore
is imported later in user code.
Original report is for Windows and Python 3.11.5, I've also repro'd on macOS 13.5.2 with Python 3.11.5.
urllib3==2.0.7
truststore==0.8.0
After generating a certificate with mkcert
(using the example in README) and starting a server on localhost, https://localhost
correctly runs in the browser, but using socket
with truststore
fails with the following:
Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> import ssl
>>> import truststore
>>> sock = socket.create_connection(("127.0.0.1", 443))
>>> ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
>>> sock = ctx.wrap_socket(sock, server_hostname="127.0.0.1")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "site-packages\truststore\_api.py", line 54, in wrap_socket
_verify_peercerts(ssl_sock, server_hostname=server_hostname)
File "site-packages\truststore\_api.py", line 106, in _verify_peercerts
_verify_peercerts_impl(
File "site-packages\truststore\_windows.py", line 318, in _verify_peercerts_impl
_get_and_verify_cert_chain(
File "site-packages\truststore\_windows.py", line 410, in _get_and_verify_cert_chain
raise err from None
ssl.SSLCertVerificationError: ('Certificate chain policy error 0x80092012 [0]',)
To reproduce:
Install the following on a Windows machine:
Then run:
import requests
The error pops up stating that the file is already open by another program.
@jacobian pointed out that it would be nice if truststore
could also use client certs from the system trust store, particularly on devices that provide device identity by creating certs using a private key stored in a Trusted Platform Module. I suspect this may require actually using system APIs for encrypting the data stream and not just for cert validation though.
Current PyPI page showing: "Verify certificates using OS trust stores. This is useful when your system contains". Apparently we only get one line for the module-level short description?
mypy --strict SOURCE_FILES
to a linting jobHi,
I use functionality that is unique to PyOpenSSL contexts (OpenSSL.SSL.Context
) to verify certificate chains without having an SSL connection open. PyOpenSSL's trust store support is very limited (see notes in https://www.pyopenssl.org/en/stable/api/ssl.html#OpenSSL.SSL.Context.set_default_verify_paths). Because of this, I've been falling back to certifi, but would prefer to make use of the functionality supported by truststore instead. Do you have any thoughts on the feasibility of this?
Depends on pypa/pip#11082
We should ensure we don't break any of pip's users trying to use our experimental feature flag --use-feature=truststore
, at least for a basic installation from PyPI. It's tougher for us to simulate installing from corporate proxies/package indices but maybe we can do something with mkcert and warehouse?
This is more a question/discussion than an issue.
I'm using a lot of Python libraries, which build on urllib3
, aiohttp
or requests
. In seldom cases, I am able to get the pool manager or SSL context to inject truststore's context into it.
Is there an option to inject truststore more globally, so that dependent libraries use it?
Here are some examples:
urllib3
and requests
for testsurllib3
Hi Guys,
I tested my service with truststore library in MacOS to fetch missing certs and it is working fine. However, when the service is deployed to AWS ECS with linux OS, it doesn't seem to work and still throws SSL CERT VERIFY FAILED error.
Do you know if there's other setting I need to enable?
I am trying to use truststore on Mac OS X 10.11 (El Capitan). Both Python 3.10 and truststore would seem to work with this combination, but it is failing to load a symbol, which seems to have been introduced in 10.14 when verifying a certificate in a HTTPS request.
Are older macOS versions really tested with the current truststore?
Please let me know what additional debug info I can provide, but here is output from a Terminal with some useful steps to check if truststore is working properly:
Last login: Fri Oct 13 14:03:12 on ttys004
Mac-mini-ElCapitan-VM:~ jorik$ uname -a
Darwin Mac-mini-ElCapitan-VM.local 15.6.0 Darwin Kernel Version 15.6.0: Thu Jun 21 20:07:40 PDT 2018; root:xnu-3248.73.11~1/RELEASE_X86_64 x86_64
Mac-mini-ElCapitan-VM:~ jorik$ source ~/src/trust_test/venv/bin/activate
(venv) Mac-mini-ElCapitan-VM:~ jorik$ pip list
Package Version
------------------ ---------
certifi 2023.7.22
charset-normalizer 3.3.0
idna 3.4
pip 23.2.1
requests 2.31.0
setuptools 68.1.2
truststore 0.8.0
urllib3 2.0.6
(venv) Mac-mini-ElCapitan-VM:~ jorik$ python
Python 3.10.13 (main, Oct 12 2023, 14:16:20) [Clang 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> R = requests.get('https://www.python.org/')
>>>
(venv) Mac-mini-ElCapitan-VM:~ jorik$ python
Python 3.10.13 (main, Oct 12 2023, 14:16:20) [Clang 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import truststore
>>> truststore.inject_into_ssl()
>>> import requests
>>> R = requests.get('https://www.python.org/')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/api.py", line 73, in get
return request("get", url, params=params, **kwargs)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
resp = conn.urlopen(
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
response = self._make_request(
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
self._validate_conn(conn)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1092, in _validate_conn
conn.connect()
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connection.py", line 642, in connect
sock_and_verified = _ssl_wrap_socket_and_match_hostname(
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/connection.py", line 783, in _ssl_wrap_socket_and_match_hostname
ssl_sock = ssl_wrap_socket(
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 469, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 513, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_api.py", line 104, in wrap_socket
_verify_peercerts(ssl_sock, server_hostname=server_hostname)
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_api.py", line 300, in _verify_peercerts
_verify_peercerts_impl(
File "/Users/jorik/src/trust_test/venv/lib/python3.10/site-packages/truststore/_macos.py", line 437, in _verify_peercerts_impl
sec_trust_eval_result = Security.SecTrustEvaluateWithError(
File "/usr/local/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
func = self.__getitem__(name)
File "/usr/local/Cellar/[email protected]/3.10.13/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: dlsym(0x7fc3d9462db0, SecTrustEvaluateWithError): symbol not found
>>>
From this page, it seems SecTrustEvaluateWithError
is available since 10.14? Or maybe I'm misreading the documentation?
I expect this will be a FAQ
python -m pip install openai --use-feature=truststore
Collecting openai
Using cached openai-0.26.5.tar.gz (55 kB)
Installing build dependencies: started
Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [8 lines of output]
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/setuptools/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/setuptools/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/setuptools/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/setuptools/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))': /simple/setuptools/
Could not fetch URL https://pypi.org/simple/setuptools/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/setuptools/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)'))) - skipping
ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
ERROR: No matching distribution found for setuptools
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
I would like to fix packaging for OpenSUSE, but your release tarball does not contain the tests directory. Could you please distribute them as well?
Hi Seth,
Thank you for your work on this - all of us who are using local HTTPS proxies, or who are working in corporate environments with TLS inspection, or have otherwise had to confront the 'certificate verify failed' error, are immensely grateful for it.
I'm working with @bnoordhuis on a Node version of this (https://github.com/bnoordhuis/node-native-certs), which takes the approach of using the Rust rustls-native-certs library to do the hard work. This also means we don't have to touch C for the native parts of the module - our glue code is all Rust - which makes cross platform packaging a lot easier. If you're interested, I have put together a little prototype of what using rustls-native-certs from Python could look like.
Secondly, with your work in Python, our work in Node, and the Rust team's foundational work on this in Rust, it's now clear that a pattern is emerging. People are waking up to the idea that even if a language is cross-platform, it should still be a good citizen of each platform, and get its certificates from the native trust store. To this end I've started making a list of languages and CLI tools which have native trust store integration here: https://native-certs.github.io. (Naturally truststore
is featured in that list.) If you know of any others that I haven't thought of yet, please let me know!
Since we're using ctypes we should run memray to ensure we're not leaking/regressing on memory.
I'm hitting a RecursionError: maximum recursion depth exceeded
while running https://github.com/pradyunsg/did with a patch to add truststore.inject_ssl()
for use with httpx (i.e. not the requests/urllib3 stack). Patch below...
diff --git a/src/did.py b/src/did.py
index 31ebad3..a6eaf2b 100644
--- a/src/did.py
+++ b/src/did.py
@@ -6,6 +6,10 @@
OSS update blog posts.
"""
+import truststore
+
+truststore.inject_into_ssl()
+
__version__ = "0.1.0"
import asyncio
I'm not quite sure why the recursion error is happening, but filing an issue eagerly to flag this to y'all. :)
❯ pip freeze
aiorwlock==1.3.0
anyio==3.6.2
attrs==22.2.0
certifi==2022.12.7
cffi==1.15.1
click==8.1.3
cryptography==40.0.1
fasteners==0.17.3
gidgethub==5.2.1
h11==0.14.0
httpcore==0.16.3
httpx==0.23.3
httpx-cache==0.7.0
idna==3.4
markdown-it-py==2.2.0
mdurl==0.1.2
msgpack==1.0.5
-e git+https://github.com/pradyunsg/did.git@60816da46caf348e8262698e21043bcd7e6d0118#egg=pradyunsg_did
pycparser==2.21
Pygments==2.14.0
PyJWT==2.6.0
rfc3986==1.5.0
rich==13.3.2
sniffio==1.3.0
truststore==0.6.0
uritemplate==4.1.1
This is a traceback printed with rich.traceback
with_locals=True
, trimmed. You should see this if you run did this quarter
or did this month
after installing that repository.
[manual note: trimmed for brevity]
│ /Users/pgedam/Developer/github/did/src/did.py:481 in this │
│ │
│ 478 │ """stats for current [period header time]""" │
│ 479 │ period_ref, since, until = get_this_period(today=date.today(), period=period) │
│ 480 │ print(f"# Status update for {period_ref} ({since} to {until}*)") │
│ ❱ 481 │ main(since=since, until=until) │
│ 482 │
│ 483 │
│ 484 @did.command("yesterday") │
│ │
│ ╭──────────────── locals ─────────────────╮ │
│ │ period = 'quarter' │ │
│ │ period_ref = 'Q1 2023' │ │
│ │ since = datetime.date(2023, 1, 1) │ │
│ │ until = datetime.date(2023, 3, 30) │ │
│ ╰─────────────────────────────────────────╯ │
│ │
│ /Users/pgedam/Developer/github/did/src/did.py:435 in main │
│ │
│ 432 def main(*, since: date, until: date) -> None: │
│ 433 │ print() │
│ 434 │ try: │
│ ❱ 435 │ │ asyncio.run(github(since, until)) │
│ 436 │ except gidgethub.BadRequest: │
│ 437 │ │ rich.print(rich.traceback.Traceback(suppress=[asyncio]), file=sys.stderr) │
│ 438 │ │ gh_token_issue("Maybe the token expired? https://github.com/settings/tokens/") │
│ │
│ ╭────────────── locals ──────────────╮ │
│ │ since = datetime.date(2023, 1, 1) │ │
│ │ until = datetime.date(2023, 3, 30) │ │
│ ╰────────────────────────────────────╯ │
│ │
│ /Users/pgedam/.asdf/installs/python/3.10.4/lib/python3.10/asyncio/runners.py:44 in run │
│ │
│ /Users/pgedam/.asdf/installs/python/3.10.4/lib/python3.10/asyncio/base_events.py:646 in │
│ run_until_complete │
│ │
│ /Users/pgedam/Developer/github/did/src/did.py:305 in github │
│ │
│ 302 │ # "SponsorshipEvent", │
│ 303 │ # } │
│ 304 │ │
│ ❱ 305 │ async with httpx_cache.AsyncClient(cache=CACHE) as client: │
│ 306 │ │ gh = gidgethub.httpx.GitHubAPI( │
│ 307 │ │ │ client, │
│ 308 │ │ │ "pradyunsg", │
│ │
│ ╭─────────────────────────────────────────── locals ───────────────────────────────────────────╮ │
│ │ searches = { │ │
│ │ │ 'Issues created': 'author:pradyunsg type:issue │ │
│ │ created:2023-01-01..2023-03-30', │ │
│ │ │ 'Assigned issues closed': 'assignee:pradyunsg type:issue │ │
│ │ closed:2023-01-01..2023-03-30', │ │
│ │ │ 'PRs created': 'author:pradyunsg type:pr created:2023-01-01..2023-03-30', │ │
│ │ │ 'PRs reviewed': 'reviewed-by:pradyunsg type:pr │ │
│ │ reviewed:2023-01-01..2023-03-30', │ │
│ │ │ 'Assigned PRs closed': 'assignee:pradyunsg type:pr │ │
│ │ closed:2023-01-01..2023-03-30' │ │
│ │ } │ │
│ │ since = datetime.date(2023, 1, 1) │ │
│ │ time_term = '2023-01-01..2023-03-30' │ │
│ │ until = datetime.date(2023, 3, 30) │ │
│ ╰──────────────────────────────────────────────────────────────────────────────────────────────╯ │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx_cache/client.py:167 │
│ in __init__ │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_client.py:1400 in │
│ __init__ │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx_cache/client.py:200 │
│ in _init_transport │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_client.py:1448 in │
│ _init_transport │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_transports/default. │
│ py:261 in __init__ │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_config.py:51 in │
│ create_ssl_context │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_config.py:75 in │
│ __init__ │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_config.py:87 in │
│ load_ssl_context │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_config.py:124 in │
│ load_ssl_context_verify │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_config.py:160 in │
│ _create_default_ssl_context │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/httpx/_compat.py:30 in │
│ set_minimum_tls_version_1_2 │
│ │
│ /Users/pgedam/Developer/github/did/.venv/lib/python3.10/site-packages/truststore/_api.py:226 in │
│ minimum_version │
│ │
│ 223 │ │
│ 224 │ @minimum_version.setter │
│ 225 │ def minimum_version(self, value: ssl.TLSVersion) -> None: │
│ ❱ 226 │ │ self._ctx.minimum_version = value │
│ 227 │ │
│ 228 │ @property │
│ 229 │ def options(self) -> ssl.Options: │
│ │
│ ╭────────────────────────── locals ──────────────────────────╮ │
│ │ self = <truststore._api.SSLContext object at 0x105e2f8c0> │ │
│ │ value = <TLSVersion.TLSv1_2: 771> │ │
│ ╰────────────────────────────────────────────────────────────╯ │
│ │
│ /Users/pgedam/.asdf/installs/python/3.10.4/lib/python3.10/ssl.py:603 in minimum_version │
│ │
│ 600 │ │ def minimum_version(self, value): │
│ 601 │ │ │ if value == TLSVersion.SSLv3: │
│ 602 │ │ │ │ self.options &= ~Options.OP_NO_SSLv3 │
│ ❱ 603 │ │ │ super(SSLContext, SSLContext).minimum_version.__set__(self, value) │
│ 604 │ │ │
│ 605 │ │ @property │
│ 606 │ │ def maximum_version(self): │
│ │
[manual note: this stack entry is repeated a _lot_ of times, trimmed for brevity]
│ ╭──────────────────── locals ────────────────────╮ │
│ │ self = <ssl.SSLContext object at 0x105edc7c0> │ │
│ │ value = <TLSVersion.TLSv1_2: 771> │ │
│ ╰────────────────────────────────────────────────╯ │
│ │
│ /Users/pgedam/.asdf/installs/python/3.10.4/lib/python3.10/ssl.py:603 in minimum_version │
│ │
│ 600 │ │ def minimum_version(self, value): │
│ 601 │ │ │ if value == TLSVersion.SSLv3: │
│ 602 │ │ │ │ self.options &= ~Options.OP_NO_SSLv3 │
│ ❱ 603 │ │ │ super(SSLContext, SSLContext).minimum_version.__set__(self, value) │
│ 604 │ │ │
│ 605 │ │ @property │
│ 606 │ │ def maximum_version(self): │
│ │
│ ╭──────────────────── locals ────────────────────╮ │
│ │ self = <ssl.SSLContext object at 0x105edc7c0> │ │
│ │ value = <TLSVersion.TLSv1_2: 771> │ │
│ ╰────────────────────────────────────────────────╯ │
│ │
│ /Users/pgedam/.asdf/installs/python/3.10.4/lib/python3.10/ssl.py:603 in minimum_version │
│ │
│ 600 │ │ def minimum_version(self, value): │
│ 601 │ │ │ if value == TLSVersion.SSLv3: │
│ 602 │ │ │ │ self.options &= ~Options.OP_NO_SSLv3 │
│ ❱ 603 │ │ │ super(SSLContext, SSLContext).minimum_version.__set__(self, value) │
│ 604 │ │ │
│ 605 │ │ @property │
│ 606 │ │ def maximum_version(self): │
│ │
│ ╭──────────────────── locals ────────────────────╮ │
│ │ self = <ssl.SSLContext object at 0x105edc7c0> │ │
│ │ value = <TLSVersion.TLSv1_2: 771> │ │
│ ╰────────────────────────────────────────────────╯ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
RecursionError: maximum recursion depth exceeded
I am currently using Python for performing client authentication on Windows as part of a HTTP request with code like this:
import http.client, ssl
# load clientAuth cert (with private key) from file
context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
context.load_default_certs()
context.load_cert_chain(certfile="MyClientCert.pem")
# submit HTTP request
conn = http.client.HTTPSConnection(hostname, port=443, context=context)
conn.request('GET', '/')
# print response
r = conn.getresponse()
This works fine but has the downside of requiring the client certificate (including private key) to be provided through a PEM file. I would like to avoid this and instead use a certificate directly from the Windows certificate store, so that the private key can be stored non-exportable in the TPM chip for improved security.
Is it possible to use this library for client authentication on Windows?
This is roughly the type of code that I want to write to enable TPM-based client authentication:
import socket, ssl, truststore
# Load clientAuth cert named "MyClientCert" from "CurrentUser/My" store
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
ctx.load_cert_chain("MyClientCert", "My", StoreLocation.CurrentUser)
# Connect and initiate TLS handshake with client auth.
sock = socket.create_connection((hostname, 443))
sock = ctx.wrap_socket(sock, server_hostname=hostname)
...
This will then utilize the following certificate from the Windows certificate store:
If you're here from the article and running pip with --use-feature=truststore
worked on your machine then leave a reaction below based on your operating system. To avoid spam, please do not comment unless your operating system isn't represented below.
If your operating system isn't represented in this list then you can comment below.
If truststore didn't work then please open an issue describing your environment and operating system.
When installing a package from a zip file, like this:
pip install https://github.com/goldmann/docker-squash/archive/4a7fc2c3a2175d868ff60eefdbab53240a7641d5.zip --use-
feature=truststore
I received the following error:
Collecting https://github.com/goldmann/docker-squash/archive/4a7fc2c3a2175d868ff60eefdbab53240a7641d5.zip
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000)'))': /goldmann/docker-squash/archive/4a7fc2c3a2175d868ff60eefdbab53240a7641d5.zip
[...]
ERROR: Could not install packages due to an OSError: HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /goldmann/docker-squash/archive/4a7fc2c3a2175d868ff60eefdbab53240a7641d5.zip (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1000)')))
I'm not sure if installing via a zip file specifically is the issue, because AFAIK our pypi certificates are not being substituted by corporate.
I can confirm that curl https://github.com
works without any cert problems.
Also, using the pip-system-certs
package instead of the truststore option worked in this case.
From my limited understanding, it seems that in our corporate environment github returns a certificate signed by our corporate root CA with no intermediates. This root CA was manually installed using ca-certificates
package on ubuntu 22.04.
Hi! 👋
I'm currently trying to package this project for Arch Linux and ran into one failing test (test_trustme_cert_loaded_via_capath
).
Maybe you have any ideas what might be going wrong here. I think I have all required (development) dependencies installed.
test_trustme_cert_loaded_via_capath failed (1 runs remaining out of 2).
<class 'urllib3.exceptions.MaxRetryError'>
HTTPSConnectionPool(host='localhost', port=32851): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
unable to get local issuer certificate (_ssl.c:1006)')))
[<TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:341>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:262>, <TracebackEntry /usr/lib/python3.11/site-pa
ckages/pluggy/_hooks.py:493>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_manager.py:115>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:152>, <TracebackEntry /usr
/lib/python3.11/site-packages/pluggy/_result.py:114>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:77>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:177>
, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:169>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/python.py:1792>, <TracebackEntry /usr/lib/python3.11/site-packages
/pluggy/_hooks.py:493>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_manager.py:115>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:152>, <TracebackEntry /usr/lib/p
ython3.11/site-packages/pluggy/_result.py:114>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:77>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/python.py:194>, <Tra
cebackEntry /build/python-truststore/src/truststore-0.8.0/tests/test_api.py:316>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/request.py:77>, <TracebackEntry /usr/lib/python3.11/site-pac
kages/urllib3/request.py:99>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/poolmanager.py:376>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <Tracebac
kEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <TracebackEntry /usr/lib/python3.11/site-pack
ages/urllib3/connectionpool.py:799>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/util/retry.py:592>]
test_trustme_cert_loaded_via_capath failed; it passed 0 out of the required 1 times.
<class 'urllib3.exceptions.MaxRetryError'>
HTTPSConnectionPool(host='localhost', port=32851): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
unable to get local issuer certificate (_ssl.c:1006)')))
[<TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:341>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:262>, <TracebackEntry /usr/lib/python3.11/site-pa
ckages/pluggy/_hooks.py:493>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_manager.py:115>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:152>, <TracebackEntry /usr
/lib/python3.11/site-packages/pluggy/_result.py:114>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:77>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:177>
, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/runner.py:169>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/python.py:1792>, <TracebackEntry /usr/lib/python3.11/site-packages
/pluggy/_hooks.py:493>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_manager.py:115>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:152>, <TracebackEntry /usr/lib/p
ython3.11/site-packages/pluggy/_result.py:114>, <TracebackEntry /usr/lib/python3.11/site-packages/pluggy/_callers.py:77>, <TracebackEntry /usr/lib/python3.11/site-packages/_pytest/python.py:194>, <Tra
cebackEntry /build/python-truststore/src/truststore-0.8.0/tests/test_api.py:316>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/request.py:77>, <TracebackEntry /usr/lib/python3.11/site-pac
kages/urllib3/request.py:99>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/poolmanager.py:376>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <Tracebac
kEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/connectionpool.py:827>, <TracebackEntry /usr/lib/python3.11/site-pack
ages/urllib3/connectionpool.py:799>, <TracebackEntry /usr/lib/python3.11/site-packages/urllib3/util/retry.py:592>]
test_trustme_cert_still_uses_system_certs passed 1 out of the required 1 times. Success!
test_macos_10_7_import_error passed 1 out of the required 1 times. Success!
===End Flaky Test Report===
=========================== short test summary info ============================
FAILED tests/test_api.py::test_trustme_cert_loaded_via_capath - urllib3.excep...
================== 1 failed, 41 passed, 10 skipped in 17.85s ===================
Full build and test logs:
python-truststore-0.8.0-1-x86_64-check.log
python-truststore-0.8.0-1-x86_64-build.log
Requests is quite a popular library for http requests. It uses urllib3 as the powerhouse and it would be great if I could use truststore also with requests.
I am not sure if this something for requests to implement or needs instruction here.
I have not found a direct injection point where can I access the SSL context in urllib3 from a requests Session
.
Would it be possible to use truststore with requests somehow?
Truststore version : 0.8.0
Win 10 (64bit)
Python version: 3.11.5
Hi (completely newby here)
Coming here after reading :
https://pip.pypa.io/en/stable/topics/https-certificates/#using-system-certificate-stores
I have used:
python.exe -m pip install requests --use-feature=truststore
I used this approach because validationissues while placing request to corporate network server. When I developed the validation issues code had been workaurond using pip-system-certs, But for a last few weeks I am getting errors (I do not know what could have changed).
The SSL errors are of the form
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)
While googling to find some fix I read the entry in:
https://pip.pypa.io/en/stable/topics/https-certificates/#using-system-certificate-stores
where they wrote:
"If you encounter a TLS/SSL error when using the truststore feature you should open an issue on the truststore GitHub issue tracker instead of pip’s issue tracker. The maintainers of truststore will help diagnose and fix the issue."
See below the dup I am getting after going the trusstore route on requests library.
File "C:\Users\AB5720\PycharmProjects\ReportCertificates\venv\Lib\site-packages\urllib3\util\ssl_.py", line 469, in ssl_wrap_socket
ssl_sock = ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\AB5720\PycharmProjects\ReportCertificates\venv\Lib\site-packages\urllib3\util\ssl.py", line 513, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\AB5720\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\AB5720\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1108, in _create
self.do_handshake()
File "C:\Users\AB5720\AppData\Local\Programs\Python\Python311\Lib\ssl.py", line 1379, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006)
This is the log while i installed truststore and applied it to requests
(venv) PS C:\Users\AB5720\PycharmProjects\ReportCertificates\venv\Scripts> pip install truststore
Requirement already satisfied: truststore in c:\users\ab5720\pycharmprojects\reportcertificates\venv\lib\site-packages (0.8.0)
(venv) PS C:\Users\AB5720\PycharmProjects\ReportCertificates\venv\Scripts> python.exe -m pip install requests --use-feature=truststore
Collecting requests
Obtaining dependency information for requests from https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Collecting charset-normalizer<4,>=2 (from requests)
Obtaining dependency information for charset-normalizer<4,>=2 from https://files.pythonhosted.org/packages/91/6e/db0e545302bf93b6dbbdc496dd192c7f8e8c3bb1584acba069256d8b51d4/charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl.metadata
Downloading charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl.metadata (31 kB)
Collecting idna<4,>=2.5 (from requests)
Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting urllib3<3,>=1.21.1 (from requests)
Obtaining dependency information for urllib3<3,>=1.21.1 from https://files.pythonhosted.org/packages/37/dc/399e63f5d1d96bb643404ee830657f4dfcf8503f5ba8fa3c6d465d0c57fe/urllib3-2.0.5-py3-none-any.whl.metadata
Downloading urllib3-2.0.5-py3-none-any.whl.metadata (6.6 kB)
Collecting certifi>=2017.4.17 (from requests)
Obtaining dependency information for certifi>=2017.4.17 from https://files.pythonhosted.org/packages/4c/dd/2234eab22353ffc7d94e8d13177aaa050113286e93e7b40eae01fbf7c3d9/certifi-2023.7.22-py3-none-any.whl.metadata
Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Downloading charset_normalizer-3.2.0-cp311-cp311-win_amd64.whl (96 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 kB 2.8 MB/s eta 0:00:00
Using cached urllib3-2.0.5-py3-none-any.whl (123 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2023.7.22 charset-normalizer-3.2.0 idna-3.4 requests-2.31.0 urllib3-2.0.5
Hello, could the source of this problem be that the python system does not use the certification repository? I am very new to this topic, I may be asking a silly question. Actually what I want to do is to login users to the 2nd website using GitLab as an OAuth between two websites but I get the following error. I created a certificate in Gitlab. I have Python 3.8 and pip 19.3 in my environment. How can I solve this problem?
thankyou for support.
An error occurred while attempting to login via your social network account.
HTTPSConnectionPool(host='gitlab.tutel', port=443): Max retries exceeded with url: /oauth/token
(Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed:
self-signed certificate (_ssl.c:997)')))
Maybe a manually triggered github workflow with required review? https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments
Hi,
In my org, we have our own PyPI server with self-signed certs.
I have managed to use truststore successfully through pip on both Windows and Ubuntu through the command line option --use-feature truststore
, like this :
pip install --extra-index-url https://[address-of-our-pypiserver] --use-feature truststore [some_package]
However, when I try the exact same thing using a requirements file like this:
requirements.txt
--extra-index-url https://[address-of-our-pypiserver]
--use-feature truststore
[some_package]
and: pip install -r requirements.txt
It does not work, and I get the unable to get local issuer certificate
error.
pip seems to recognizes the feature because otherwise it raises an error when I type an unknown feature by pip.
I am not sure whether all this is a pip issue or a trustore issue, so I am posting this here.
Thanks a lot for your help !
Extra info:
python version: 3.10
pip version: 23.2.1
truststore version: 0.8.0
Windows 10, Ubuntu 20.04
I suggest that you reconsider the load order for system trust store on Linux and BSD. OpenSSL prefers SSL_CERT_FILE
/ SSL_CERT_DIR
env vars and then falls back to compile-time locations for cafile and capath. If you deviate from the default, then it may not only break applications that rely on the order. Linux vendors will patch and modify your code to re-establish default settings, too. You can get the defaults from Python's ssl module:
>>> ssl.get_default_verify_paths()
DefaultVerifyPaths(cafile='/etc/pki/tls/cert.pem', capath='/etc/pki/tls/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/pki/tls/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/pki/tls/certs')
The cafile
and capath
attributes resolve the env vars.
For cafile, do
SSL_CERT_FILE
, use it if the env var is set and don't search for additional locations. An empty or invalid SSL_CERT_FILE
isFor capath, do
SSL_CERT_DIR
r"[0-9a-fA-F]{8}\.[0-9]"
You may even want to be more strict. If either SSL_CERT_FILE
is set, SSL_CERT_DIR
is set, default cafile exists, or default capath contains certs, then call load_default_certs()
or set_default_verify_paths()
. Only fall back to alternative search paths if the defaults don't work.
I also noticed that you are using an old, outdated list of cert file locations. I have seen the list before in other projects. It contains wrong paths (FreeBSD) or discontinued distros (OpenELEC). You can find an up to date list of paths for Linux at https://github.com/tiran/certifi-system-store/blob/main/src/certifi/core.py and some research at https://github.com/tiran/distro-truststore/.
Currently we're testing many cases using ssl.PROTOCOL_TLS_CLIENT
and not many with the server. We ran into an issue with specifically macOS that was tough to debug w/ CI alone. Need an actual macOS machine to see what's going on easier: https://github.com/sethmlarson/truststore/actions/runs/4258995744/jobs/7410790041
Avoids a DeprecationWarning.
Things like:
Maybe more?
Hello! 🙂
We normally use pip_system_certs, which makes Python automatically use the Windows cert store. My understanding is that truststore
is supposed to provide the same functionality. If not, then maybe this bug report is bogus 😄
Anyway, I installed truststore
and some packages, like so:
python -m venv ".venv"
.\.venv\Scripts\activate\
python -m pip install --upgrade pip
python -m pip install truststore --upgrade
python -m pip install robotframework --upgrade --use-feature=truststore
python -m pip install robotframework-requests --upgrade --use-feature=truststore
python -m pip install urllib3 --upgrade --use-feature=truststore
python -m pip install requests --upgrade --use-feature=truststore
python -m pip install RESTinstance --upgrade --use-feature=truststore
...
When we then make a call using robot-framework-requests
, I get this error:
SSLError: HTTPSConnectionPool(host='localhost', port=44327): Max retries exceeded with url: /example (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:997)')))
OS: Windows
Python: 3.10.5
Let me know if there's any more info that I can provide! 🙂
Testing out requests
and truststore
with some corporate CA yields errors, depending on how the certificate is set up.
urllib3\connection.py:458: SubjectAltNameWarning: Certificate for xxx.intranet.cnb has no `subjectAltName`, falling back to check for a `commonName` for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/urllib3/urllib3/issues/497 for details.)
warnings.warn(
ERROR empty or no certificate, match_hostname needs a SSL socket or SSL context with either CERT_OPTIONAL or CERT_REQUIRED
Trying the same servers with urllib3
and truststore
:
>>> import urllib3
>>> import truststore
>>> ctx = truststore.SSLContext()
<stdin>:1: DeprecationWarning: ssl.SSLContext() without protocol argument is deprecated.
<stdin>:1: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated
>>> http = urllib3.PoolManager(ssl_context=ctx)
>>> resp = http.request('GET', 'https://xxx.intranet.cnb')
C:\Users\wdwni\.virtualenvs\sdr-api-example-WgBfLP6C\lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'xxx.intranet.cnb'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
>>> resp = http.request('GET', 'https://yyy.intranet.cnb')
C:\Users\wdwni\.virtualenvs\sdr-api-example-WgBfLP6C\lib\site-packages\urllib3\connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host 'yyy.intranet.cnb'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
>>> truststore.__version__
'0.4.0'
>>> urllib3.__version__
'1.26.11'
Originally discovered in #71
I just noticed that nox is skipping the tests on Python 3.11 because of
Line 30 in f7d492d
Allow passing additional certificates via SSLContext.load_verify_locations()
. Only load system certificates when SSLContext.load_default_certs()
is used.
Currently on all 3 platforms, truststore is setting TrustStoreSSLContext.check_hostname
to False to suppress the built-in hostname check, and then doing its own check at the bottom of _verify_peercerts
On Mac, this isn't necessary because the hostname is passed to Security.SecPolicyCreateSSL -- the tests still pass if I comment out the call to _match_hostname
. On Windows the situation is similar with the hostname passed to CertVerifyCertificateChainPolicy.
On Linux I think we should be able to set check_hostname = True
and rely on the built-in check, since verify_mode is not set to None. But, in fact this fails the test with urllib3 when the hostname is an IP address because urllib3 is not passing the hostname along to wrap_socket
in that case.
So the question is a) am I missing something, and b) can we get rid of _match_hostname?
(And one more minor question: is there any problem that _verify_peercerts is using sslobj.get_verified_chain() to get the hostname, given that the ssl module doesn't know how to verify the chain on its own?)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.