kmip.server.session0000001[n] - WARNING - Failure parsing request message about pykmip

[PeterHamilton]

Hi @JRobinson333, I can help. A recent server update added in some debug logging that allows the server admin to see what TLS ciphers are shared between the client and server when new connections are made. This feature uses the shared_ciphers attribute of the SSLSocket object, which was added in Python 3.5. If you're running the server with an older version of Python you'd probably hit this error.

I'm working on a quick bug fix to address this but for additional clarity can you provide any other Python traceback or information from your log to help me verify that this is the problem? Thanks in advance!

[JRobinson333]

Sure, here's some extra info from the log:

2017-11-09 07:26:24,805 - kmip.server.config - INFO - Loading server configuration settings from: /etc/pykmip/server.conf
2017-11-09 07:26:24,814 - kmip.server.engine - INFO - Loading user-defined operation policy files from: /etc/pykmip/policies
2017-11-09 07:26:24,814 - kmip.server.engine - INFO - Loading user-defined operation policies from file: /etc/pykmip/policies/policy.json
2017-11-09 07:26:24,815 - kmip.server - INFO - Starting server socket handler.
2017-11-09 07:26:24,815 - kmip.server - INFO - Server successfully bound socket handler to <>:5696
2017-11-09 07:26:24,816 - kmip.server - INFO - Starting connection service...
2017-11-09 07:26:48,111 - kmip.server - INFO - Receiving incoming connection from: <>:59652
2017-11-09 07:26:48,112 - kmip.server - INFO - Dedicating session 00000001 to <>:59652
2017-11-09 07:26:48,112 - kmip.server.session.00000001 - INFO - Starting session: 00000001
2017-11-09 07:26:48,113 - kmip.server.session.00000001 - INFO - Stopping session: 00000001
2017-11-09 07:26:48,169 - kmip.server - INFO - Receiving incoming connection from: <>:59654
2017-11-09 07:26:48,169 - kmip.server - INFO - Dedicating session 00000002 to <>:59654
2017-11-09 07:26:48,170 - kmip.server.session.00000002 - INFO - Starting session: 00000002
2017-11-09 07:26:48,170 - kmip.server.session.00000002 - INFO - Stopping session: 00000002
2017-11-09 07:26:48,231 - kmip.server - INFO - Receiving incoming connection from: <>:59656
2017-11-09 07:26:48,231 - kmip.server - INFO - Dedicating session 00000003 to <>:59656
2017-11-09 07:26:48,231 - kmip.server.session.00000003 - INFO - Starting session: 00000003
2017-11-09 07:26:48,232 - kmip.server.session.00000003 - INFO - Stopping session: 00000003
2017-11-09 07:26:48,254 - kmip.server - INFO - Receiving incoming connection from: <>:59658
2017-11-09 07:26:48,254 - kmip.server - INFO - Dedicating session 00000004 to <>:59658
2017-11-09 07:26:48,255 - kmip.server.session.00000004 - INFO - Starting session: 00000004
2017-11-09 07:26:48,255 - kmip.server.session.00000004 - WARNING - Failure parsing request message.
2017-11-09 07:26:48,255 - kmip.server.session.00000004 - ERROR - 'SSLSocket' object has no attribute 'shared_ciphers'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/PyKMIP-0.6.0-py2.7.egg/kmip/services/server/session.py", line 161, in _handle_message_loop
shared_ciphers = self._connection.shared_ciphers()
AttributeError: 'SSLSocket' object has no attribute 'shared_ciphers'
2017-11-09 07:26:48,257 - kmip.server.session.00000004 - WARNING - Failure parsing request message.
2017-11-09 07:26:48,257 - kmip.server.session.00000004 - ERROR - 'SSLSocket' object has no attribute 'shared_ciphers'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/PyKMIP-0.6.0-py2.7.egg/kmip/services/server/session.py", line 161, in _handle_message_loop
shared_ciphers = self._connection.shared_ciphers()
AttributeError: 'SSLSocket' object has no attribute 'shared_ciphers'
2017-11-09 07:26:48,258 - kmip.server.session.00000004 - INFO - Stopping session: 00000004

[JRobinson333]

Worth mentioning i'm currently running python 2.7.11-1.

In a separate test, vSphere web client shows the connection status as 'Normal' when trust is established between docker container version of PyKMIP (lamw/vmwkmip) and vmware as the client.

[JRobinson333]

Applied that patch, an updated log file is attached - guess it should be getting beyond DiscoverVersions however:

2017-11-10 03:03:45,366 - kmip.server.config - INFO - Loading server configuration settings from: /etc/pykmip/server.conf
2017-11-10 03:03:45,375 - kmip.server.engine - INFO - Loading user-defined operation policy files from: /etc/pykmip/policies
2017-11-10 03:03:45,375 - kmip.server.engine - INFO - Loading user-defined operation policies from file: /etc/pykmip/policies/policy.json
2017-11-10 03:03:45,376 - kmip.server - INFO - Starting server socket handler.
2017-11-10 03:03:45,377 - kmip.server - INFO - Server successfully bound socket handler to <>:5696
2017-11-10 03:03:45,377 - kmip.server - INFO - Starting connection service...
2017-11-10 03:04:05,763 - kmip.server - INFO - Receiving incoming connection from: <>:47508
2017-11-10 03:04:05,763 - kmip.server - INFO - Dedicating session 00000001 to <>:47508
2017-11-10 03:04:05,764 - kmip.server.session.00000001 - INFO - Starting session: 00000001
2017-11-10 03:04:05,765 - kmip.server.session.00000001 - INFO - Stopping session: 00000001
2017-11-10 03:04:05,819 - kmip.server - INFO - Receiving incoming connection from: <>:47510
2017-11-10 03:04:05,820 - kmip.server - INFO - Dedicating session 00000002 to <>:47510
2017-11-10 03:04:05,820 - kmip.server.session.00000002 - INFO - Starting session: 00000002
2017-11-10 03:04:05,820 - kmip.server.session.00000002 - INFO - Stopping session: 00000002
2017-11-10 03:04:05,876 - kmip.server - INFO - Receiving incoming connection from: <>:47512
2017-11-10 03:04:05,876 - kmip.server - INFO - Dedicating session 00000003 to <>:47512
2017-11-10 03:04:05,877 - kmip.server.session.00000003 - INFO - Starting session: 00000003
2017-11-10 03:04:05,877 - kmip.server.session.00000003 - INFO - Stopping session: 00000003
2017-11-10 03:04:05,905 - kmip.server - INFO - Receiving incoming connection from: <>:47514
2017-11-10 03:04:05,905 - kmip.server - INFO - Dedicating session 00000004 to <>:47514
2017-11-10 03:04:05,905 - kmip.server.session.00000004 - INFO - Starting session: 00000004
2017-11-10 03:04:05,941 - kmip.server.session.00000004 - INFO - Session client identity: <>
2017-11-10 03:04:05,972 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:05
2017-11-10 03:04:05,972 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:05,975 - kmip.server.session.00000004 - INFO - Session client identity: <>
2017-11-10 03:04:05,977 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:05
2017-11-10 03:04:05,977 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:05,978 - kmip.server.session.00000004 - INFO - Stopping session: 00000004
2017-11-10 03:04:08,202 - kmip.server - INFO - Receiving incoming connection from: <>:47520
2017-11-10 03:04:08,203 - kmip.server - INFO - Dedicating session 00000005 to <>:47520
2017-11-10 03:04:08,203 - kmip.server.session.00000005 - INFO - Starting session: 00000005
2017-11-10 03:04:08,203 - kmip.server.session.00000005 - INFO - Stopping session: 00000005
2017-11-10 03:04:08,256 - kmip.server - INFO - Receiving incoming connection from: <>:47522
2017-11-10 03:04:08,256 - kmip.server - INFO - Dedicating session 00000006 to <>:47522
2017-11-10 03:04:08,256 - kmip.server.session.00000006 - INFO - Starting session: 00000006
2017-11-10 03:04:08,257 - kmip.server.session.00000006 - INFO - Stopping session: 00000006
2017-11-10 03:04:08,312 - kmip.server - INFO - Receiving incoming connection from: <>:47524
2017-11-10 03:04:08,312 - kmip.server - INFO - Dedicating session 00000007 to <>:47524
2017-11-10 03:04:08,313 - kmip.server.session.00000007 - INFO - Starting session: 00000007
2017-11-10 03:04:08,315 - kmip.server.session.00000007 - INFO - Stopping session: 00000007
2017-11-10 03:04:08,343 - kmip.server - INFO - Receiving incoming connection from: <>:47526
2017-11-10 03:04:08,343 - kmip.server - INFO - Dedicating session 00000008 to <>:47526
2017-11-10 03:04:08,344 - kmip.server.session.00000008 - INFO - Starting session: 00000008
2017-11-10 03:04:08,345 - kmip.server.session.00000008 - INFO - Session client identity: <>
2017-11-10 03:04:08,347 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:08
2017-11-10 03:04:08,347 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:08,349 - kmip.server.session.00000008 - INFO - Session client identity: <>
2017-11-10 03:04:08,351 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:08
2017-11-10 03:04:08,351 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:08,352 - kmip.server.session.00000008 - INFO - Stopping session: 00000008
2017-11-10 03:04:10,582 - kmip.server - INFO - Receiving incoming connection from: <>:47532
2017-11-10 03:04:10,582 - kmip.server - INFO - Dedicating session 00000009 to <>:47532
2017-11-10 03:04:10,583 - kmip.server.session.00000009 - INFO - Starting session: 00000009
2017-11-10 03:04:10,583 - kmip.server.session.00000009 - INFO - Stopping session: 00000009
2017-11-10 03:04:10,634 - kmip.server - INFO - Receiving incoming connection from: <>:47534
2017-11-10 03:04:10,634 - kmip.server - INFO - Dedicating session 00000010 to <>:47534
2017-11-10 03:04:10,634 - kmip.server.session.00000010 - INFO - Starting session: 00000010
2017-11-10 03:04:10,635 - kmip.server.session.00000010 - INFO - Stopping session: 00000010
2017-11-10 03:04:10,687 - kmip.server - INFO - Receiving incoming connection from: <>:47536
2017-11-10 03:04:10,688 - kmip.server - INFO - Dedicating session 00000011 to <>:47536
2017-11-10 03:04:10,688 - kmip.server.session.00000011 - INFO - Starting session: 00000011
2017-11-10 03:04:10,688 - kmip.server.session.00000011 - INFO - Stopping session: 00000011
2017-11-10 03:04:10,716 - kmip.server - INFO - Receiving incoming connection from: <>:47538
2017-11-10 03:04:10,716 - kmip.server - INFO - Dedicating session 00000012 to <>:47538
2017-11-10 03:04:10,716 - kmip.server.session.00000012 - INFO - Starting session: 00000012
2017-11-10 03:04:10,717 - kmip.server.session.00000012 - INFO - Session client identity: <>
2017-11-10 03:04:10,719 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:10
2017-11-10 03:04:10,719 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:10,721 - kmip.server.session.00000012 - INFO - Session client identity: <>
2017-11-10 03:04:10,723 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:04:10
2017-11-10 03:04:10,724 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:04:10,725 - kmip.server.session.00000012 - INFO - Stopping session: 00000012
2017-11-10 03:05:31,645 - kmip.server - INFO - Receiving incoming connection from: <>:47816
2017-11-10 03:05:31,645 - kmip.server - INFO - Dedicating session 00000013 to <>:47816
2017-11-10 03:05:31,646 - kmip.server.session.00000013 - INFO - Starting session: 00000013
2017-11-10 03:05:31,646 - kmip.server.session.00000013 - INFO - Stopping session: 00000013
2017-11-10 03:05:31,697 - kmip.server - INFO - Receiving incoming connection from: <>:47818
2017-11-10 03:05:31,698 - kmip.server - INFO - Dedicating session 00000014 to <>:47818
2017-11-10 03:05:31,698 - kmip.server.session.00000014 - INFO - Starting session: 00000014
2017-11-10 03:05:31,698 - kmip.server.session.00000014 - INFO - Stopping session: 00000014
2017-11-10 03:05:31,749 - kmip.server - INFO - Receiving incoming connection from: <>:47820
2017-11-10 03:05:31,749 - kmip.server - INFO - Dedicating session 00000015 to <>:47820
2017-11-10 03:05:31,750 - kmip.server.session.00000015 - INFO - Starting session: 00000015
2017-11-10 03:05:31,750 - kmip.server.session.00000015 - INFO - Stopping session: 00000015
2017-11-10 03:05:31,773 - kmip.server - INFO - Receiving incoming connection from: <>:47822
2017-11-10 03:05:31,773 - kmip.server - INFO - Dedicating session 00000016 to <>:47822
2017-11-10 03:05:31,774 - kmip.server.session.00000016 - INFO - Starting session: 00000016
2017-11-10 03:05:31,775 - kmip.server.session.00000016 - INFO - Session client identity: <>
2017-11-10 03:05:31,778 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:05:31
2017-11-10 03:05:31,778 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:05:31,780 - kmip.server.session.00000016 - INFO - Session client identity: <>
2017-11-10 03:05:31,782 - kmip.server.engine - INFO - Received request at time: 2017-11-10 11:05:31
2017-11-10 03:05:31,783 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2017-11-10 03:05:31,784 - kmip.server.session.00000016 - INFO - Stopping session: 00000016

[JRobinson333]

As far as I can see the CNs between the client .crt and server .pem certificates match as per documentation.

I know also from the documentation that the client certificate must have Extended Key Usage clientAuth set. It looks to me like it is missing when revealing the information with openssl. There seems to be a problem with uploading images on this post, but below is a comparison of x509v3 extensions for both client and server.

CLIENT

X509v3 extensions:
X509v3 Key Usage:
Digital Signature, Key Encipherment, Key Agreement

SERVER

X509v3 extensions:
X509v3 Extended Key Usage:
TLS Web Client Authentication, TLS Web Server Authentication

Correct me if i'm wrong but wouldn't vmware also fail to link with the older PyKMIP version running in the docker container if new certs needed to be generated on the vmware side with the clientAuth value specified for Extended Key Usage during key configuration?

[PeterHamilton]

Thanks for the additional logs.

I believe the docker container is running the base PyKMIP 0.6 (released last December), which does not have the newer certificate and shared_ciphers features. It should work out of the box with a basic set of certificates, which from the logs you provided appears to be the case. Unfortunately I am not familiar with the order of operations used by the vSphere web client, so I do not know if the repeated occurrence of DiscoverVersions is normal behavior or not. When you use the docker container for the PyKMIP server, are you able to successful create and use encrypted VMs? Do the log files look the same when you compare using the docker container with the patched version of master?

If you're running the master branch from Github (which you were originally), you'll see the shared_ciphers error and also hit certificate issues if your certificate does not have the extended key usage extension. You can set enable_tls_client_auth to False in the server.conf configuration file, which will turn off the strict check for the extended key usage extension. The certificates you use with the docker container should work then.

I'll merge in the shared_ciphers bug fix today and then master should work for you regardless of which Python version you're using. That should hopefully resolve all of the errors you were running into.