HTTPS Cert Checking about requests

[kennethreitz]

there are some patches/workarounds: http://stackoverflow.com/questions/1875052/using-paired-certificates-with-urllib2

[issackelly]

Maybe we could have an empty list of PEMs as the default, and then just some methods like:

verisign_pem = """-----BEGIN CERTIFICATE-----
MIICPDCCAaUCEHC65B0Q2Sk0tjjKewPMur8wDQYJKoZIhvcNAQECBQAwXzELMAkG
A1UEBhMCVVMxFzAVBgNVBAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFz
cyAzIFB1YmxpYyBQcmltYXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MB4XDTk2
MDEyOTAwMDAwMFoXDTI4MDgwMTIzNTk1OVowXzELMAkGA1UEBhMCVVMxFzAVBgNV
BAoTDlZlcmlTaWduLCBJbmMuMTcwNQYDVQQLEy5DbGFzcyAzIFB1YmxpYyBQcmlt
YXJ5IENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIGfMA0GCSqGSIb3DQEBAQUAA4GN
ADCBiQKBgQDJXFme8huKARS0EN8EQNvjV69qRUCPhAwL0TPZ2RHP7gJYHyX3KqhE
BarsAx94f56TuZoAqiN91qyFomNFx3InzPRMxnVx0jnvT0Lwdd8KkMaOIG+YD/is
I19wKTakyYbnsZogy1Olhec9vn2a/iRFM9x2Fe0PonFkTGUugWhFpwIDAQABMA0G
CSqGSIb3DQEBAgUAA4GBALtMEivPLCYATxQT3ab7/AoRhIzzKBxnki98tsX63/Do
lbwdj2wsqFHMc9ikwFPwTtYmwHYBV4GSXiHx0bH/59AhWM1pF+NEHJwZRDmJXNyc
AA9WjQKZ7aKQRUzkuxCkPfAyAw7xzvjoyVGM5mKf5p/AfbdynMk2OmufTqj/ZA1k
-----END CERTIFICATE-----
"""

requests.add_ca(verisign_pem)

It also probably wouldn't be too hard to keep a small list of acceptable PEMs on file, and then do

requests.accept_ca(["verisign", "geotrust"]) and pull them from an internal dict.

Just some ideas to get the ball rolling.

[kennethreitz]

I'm thinking of having a default list of PEMs that the major browsers use and accept. The list can be extended/modified via requests.settings.certs. By default, everything will behave as it does now.

Response.raise_for_cert() will raise an exception if the cert isn't accepted. I'm thinking an optional custom PEM argument could be passed into raise_for_cert() in the case that there's a private cert shared between the server and the client.

Will this suit everyone's needs?

[issackelly]

Yes, that would be perfect.

On Tue, May 24, 2011 at 7:42 PM, kennethreitz <
[email protected]>wrote:

I'm thinking of having a default list of PEMs that the major browsers use
and accept. The list can be extended/modified via requests.settings.certs.
By default, everything will behave as it does now.

Response.raise_for_cert() will raise an exception if the cert isn't
accepted. I'm thinking an optional custom PEM could be passed into
raise_for_cert() in the case that there's a private cert shared between
the server and the client.

Will this suit everyone's needs?

Reply to this email directly or view it on GitHub:
https://github.com/kennethreitz/requests/issues/30#comment_1232227

[cjw296]

Please can you make sure the default option is to use modssl's list of accepted certificates?
The trend in python request libraries to have to specify all the accepted certs rather than using the "system standards" is pretty annoying. If I have my own CA, I just want to install the certificate in /etc/ssl/certs/ rather than having to pass it in every piece of python client software I write ;-)

[merwok]

The standard ssl module has something to help here, but only for Python 3.2+: http://docs.python.org/dev/library/ssl#ssl.SSLContext.set_default_verify_paths

[kennethreitz]

The more I think about this, the more I think it's a bad idea.

If you look at httplib2, almost all of its issues are ssl-related.

I'm not ruling it out yet, but maybe it's a bit out of scope for requests itself. There could easily be a requests-ssl package that uses the hooks system to force checks and such.

Like I said, I'm just thinking out loud at this point.

[issackelly]

I wouldn't mind this as an external library, I just think it needs to exist in one form or another. I wouldn't mind you closing this with a "#wontfix" for being out of scope.

[kennethreitz]

eldarion/braintree_python@634f1f3#comments

[kennethreitz]

http://pypi.python.org/pypi/backports.ssl_match_hostname/

[kennethreitz]

http://whatschrisdoing.com/blog/2011/10/30/making-https-requests-secure-in-python/

[kennethreitz]

Definitely want to add this in, but it will certainly be considered experimental for a while.

[johncwheeler]

In the meantime, is there a reasonable way to implement this in requests manually? I have a requirement that might force me to drop the whole library, i would hate that - and it would be a ton of work.

(and thanks for the response, very glad to know it is coming! :)

[kennethreitz]

@JCWDev see eldarion/braintree_python@634f1f3#comments.

I honestly don't think it will take much effort. I'll accept a working pull requests that fits this workflow as soon as it happens :)

[johncwheeler]

Perhaps I'm just tired, but reading the code linked above it appears to be validating the server certificate, but perhaps it is validating the server cert and host name against the client cert?

I am just looking for a way to pass a client cert along with my request so that the receiving server can use it for identity check.

Hopefully I'm making sense :)

eg.
http://www.osmonov.com/2009/04/client-certificates-with-urllib2.html

[sciyoshi]

I don't think the Response.raise_for_cert() idea will work, since by the time the Response has been returned to the caller, the HTTP headers will already have been sent (and if it's a MITM attack, that leaks information).

[kennethreitz]

urllib3/urllib3#25

[kennethreitz]

This just landed in urllib3.

[kennethreitz]

I'm planning on bundling a CA bundle. Unfortunately, Mozilla's is GPL.

[kennethreitz]

Requests v0.8.8 was just released that includes ssl verification!

http://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification

[kennethreitz]

v0.9.0 adds this by default. Enjoy.

✨ 🍰 ✨

[ojii]

<3

[sciyoshi]

Awesome, thanks!

[dpwiz]

Is there a way to provide client certificate (key file + cert file) for authorization purposes?

[kennethreitz]

@wlz you can provide any CA Bundle.

[dpwiz]

@kennethreitz that is to check server cert against a CA bundle or i'm missing something?

I've found traces of key/cert pair in packages, but not a mention of this in requests itself. Looks like there's some non-obvious way to actually pass client stuff to server as an auth token.

[cjw296]

How can I unsubscribe from this issue?!

[kennethreitz]

@cjw296: this pretty link: http://cl.ly/2U003k373O1L0b400y08

[cjw296]

thanks :-)

[dpwiz]

There, i fixed it™: https://gist.github.com/1710121