openkmip / pykmip

A Python implementation of the KMIP specification.

PyKMIP

PyKMIP is a Python implementation of the Key Management Interoperability Protocol (KMIP), an OASIS communication standard for the management of objects stored and maintained by key management systems. KMIP defines how key management operations and operation data should be encoded and communicated between client and server applications. Supported operations include the full CRUD key management lifecycle, including operations for managing object metadata and for conducting cryptographic operations. Supported object types include:

• symmetric/asymmetric encryption keys
• passwords/passphrases
• certificates
• opaque data blobs, and more

For more information on KMIP, check out the OASIS KMIP Technical Committee and the OASIS KMIP Documentation.

For more information on PyKMIP, check out the project Documentation.

Installation

You can install PyKMIP via pip:

$ pip install pykmip

See Installation for more information.

Community

The PyKMIP community has various forums and resources you can use:

• Source code
• Issue tracker
• IRC: #pykmip on irc.freenode.net
• Twitter: @pykmip

pykmip's Issues

error in integration test using tox

We are getting issue using open ssl . We are trying to test open source KMIP server PYKMIP. For the same we have used tox utility which runs unit and integration test cases. First We got below error message while testing the same.

“SSLError: [Errno 1] _ssl.c:510: error:140890C7:SSL routines:SSL3_GET_CLIENT_CERT
IFICATE:peer did not return a certificate”

After searching through contents on internet ,We tried to use latest security certificates as recommended by below link:
ask/python-github2#58

But after doing recommended changes we are facing below issue:

“ERROR - [Errno 0] _ssl.c:344: error:00000000:lib(0):func(0):reason(0)”

Steps taken till now:

1. Updated security certificate with latest certificate file.
2. Installed httplib2.
3. Updated tox utility.
4. Generated self-signed security certificate.

But same problem persists. Please help on this issue.

Key name should not have space in it

SafeNet is a popular KMIP server. However, it does not allow any spaces in the name of the key. Therefore, the create script should not use "Test Key" as name in order to support SafeNet. The patch below works on SafeNet 8.1.0:

diff --git a/kmip/demos/units/create.py b/kmip/demos/units/create.py
index 8a08cc1..22d6f95 100644
--- a/kmip/demos/units/create.py
+++ b/kmip/demos/units/create.py
@@ -107,7 +107,7 @@ if name == 'main':
length_obj = attribute_factory.create_attribute(attribute_type,
length)
name = Attribute.AttributeName('Name')

• name_value = Name.NameValue('Test Key')
• name_value = Name.NameValue('TestKey')
  name_type = Name.NameType(NameType.UNINTERPRETED_TEXT_STRING)
  value = Name(name_value=name_value, name_type=name_type)
  name = Attribute(attribute_name=name, attribute_value=value)

Server Certificates are expired.

The 2 following sample certificates are expired:
PyKMIP/kmip/demos/certs/server.crt
PyKMIP//kmip/tests/integration/utils/certs/server.crt

I checked the expiration date on both and got the following output:

notAfter=Aug 19 04:20:33 2015 GMT

Add Activate operation support to the server

The KMIP specification defines a basic state machine for managed objects that maps to their natural and security lifecycle. Several KMIP operations directly impact object state, allowing subsequent operations to occur in a fully featured server application. Add support for the Activate operation, which transitions managed objects from the Pre-Active state to the Active state, officially earmarking them as usable entities.

AUTHORS.txt is out of date

AUTHORS.txt has not been updated in 10 months. It should be updated or removed (as authors may be easily determined from Git's log).

query_server_information fails on SafeNet server

I got the following problem when calling this command on SafeNet. This happened only on the QUERY_SERVER_INFORMATION command, but worked on other QUERY_* commands. Does it mean SafeNet sent 24 extra bytes?

Traceback (most recent call last):
File "query.py", line 114, in
username=opts.username, password=opts.password)
File "query.py", line 59, in query
result = client.query(query_functions=query_functions)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/services/kmip_client.py", line 355, in query
response = self._send_and_receive_message(request)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/services/kmip_client.py", line 838, in _send_and_receive_message
response.read(data)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/core/messages/messages.py", line 387, in read
batch_item.read(istream)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/core/messages/messages.py", line 296, in read
self.response_payload.read(tstream)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/core/messages/payloads/query.py", line 225, in read
self.is_oversized(tstream)
File "/media/Data/wenguangw/work/encryption/pykmip/kmip/core/primitives.py", line 47, in is_oversized
raise errors.StreamNotEmptyError(Base.name, extra)
kmip.core.errors.StreamNotEmptyError: Invalid length used to read Base, bytes remaining: 24

If I comment out the error checking which raised the exception, I got this:

2015-10-10 09:54:51,993 - main - INFO - query() result status: ResultStatus.SUCCESS
2015-10-10 09:54:51,993 - main - INFO - number of operations supported: 0
2015-10-10 09:54:51,994 - main - INFO - number of object types supported: 0
2015-10-10 09:54:51,994 - main - INFO - vendor identification: None
2015-10-10 09:54:51,994 - main - INFO - server information: 541000070000000e4b657953656375726520313530760000541001070000000d4b4e554357374b4358594432450000005410020700000006382e312e30200000
2015-10-10 09:54:51,994 - main - INFO - number of application namespaces supported: 0
2015-10-10 09:54:51,994 - main - INFO - number of extensions supported: 0

The weird server information can be converted to (using http://www.rapidtables.com/convert/number/hex-to-ascii.htm):

T���KeySecure 150vT���
KNUCW7KCXYD2ET����8.1.0

In what way might the SafeNet server be violating the KMIP protocol? Is there an easy way to work around this from PyKMIP?

Server ignores hostname setting in /etc/pykmip/server.conf

All hostname values in /etc/pykmip/server.conf results in server listening only on local port.

cat /etc/pykmip/server.conf
[server]
hostname=0.0.0.0
port=5696
certificate_path=/etc/pykmip/certs/server.cert
key_path=/etc/pykmip/certs/server.key
ca_path=/etc/pykmip/certs/ca-chain.cert
auth_suite=Basic
policy_path=/etc/pykmip/policies

netstat -atp
tcp        0      0 localhost:5696          *:*                     LISTEN      18047/python3

As expected removing the hostname setting results in the following error: "Setting 'hostname' is missing from the configuration file"

As expected setting hostname on the command line succeeds.

python3 ./bin/run_server.py --hostname=0.0.0.0

netstat -atp
tcp        0      0 *:5696                  *:*                     LISTEN      18059/python3

create/create_key_pair operation in pie

Hi Peter,

Maybe I have some misunderstanding of your philosophy of pie, but I think for create/create_key_pair, it's not vey reasonable to only have "name" and "policy" as the attributes inputs.

In real scenario, creating a key with other attributes probably is very common. For example, for MAC, I need to set the crypto usage mask for the key in the create() operation. Or maybe I want to tag a "activate date" with it. Or a "custom attribute" with it. Of course I can add the attributes later but that's not very neat and also make the operation not atomic.

So do you think it's more reasonable to give them a list of attributes as the input instead of just "name" and "policy".

I know this will make the user to build all the attributes. But I think that's necessary step to make the pie interface really useful in real scenario.

add "Add Attribute"

As now the create/create_key_pair pie client interfaces only accept specific inputs, "Add Attribute" will be a pretty useful operation.

I have seen some attempts to add "Add Attribute" in the pull request and they haven't been merged. Not sure why.

I will try to work on it if you do not oppose~

Add support for Python 3.6

Python3.6 was released in December 2016. Update PyKMIP to support Python3.6.

Add fixtures for better byte string management

Frequent usage of long byte strings (see #339) bloats the codebase and makes data management and deduplication difficult. Create a fixtures package/folder/section and relocate all/most byte strings there. Also consider excluding fixtures from the main library install, since users don't need the extra bloat accrued from these byte strings.

Code is not compatible with python2.7

It'd be nice if the code would work with python 2.7. As it is, some of the validation checking fails. For example, enum is not a default package in python2.7, and even after installing it, the enum.Enum values result in an integer type, not an enum.Enum.

For example:

>>> CryptographicAlgorithm.AES.__class__ 
<type 'int'>

How do I configure the username and password credentials on the server?

It seems that I have to specify the username and password (I provide bogus/bogus) to the client config, but I don't see any provision to do so for the server. And although the auth credentials are not configured on the server, the client requests somehow succeed. That looks like a bug.

Also, it seems that the code in from kmip.services.server.server is unable to parse username and password info. Is there a way to configure these when running the KMIP server?

Time to string test fails

I'm not sure I understand how far this issue goes, but the date_time test_str test fails in some locations.

_StringException: Traceback (most recent call last):
  File "/home/viraptor/Projects/PyKMIP/kmip/tests/unit/core/primitives/test_date_time.py", line 97, in test_str
    self.assertEqual(s, str(date_time))
  File "/home/viraptor/Projects/PyKMIP/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 362, in assertEqual
    self.assertThat(observed, matcher, message)
  File "/home/viraptor/Projects/PyKMIP/.tox/py27/local/lib/python2.7/site-packages/testtools/testcase.py", line 447, in assertThat
    raise mismatch_error
testtools.matchers._impl.MismatchError: 'Tue Aug 11 09:18:55 2015' != 'Tue Aug 11 08:18:55 2015'

Due to different dates the summer time is active, it looks like the test will fail in some locations (Australia, VIC) but not others. I'm not sure what the solution is here, but I think this code shouldn't try to use/interpret local times at all.

For comparison: Australia/VIC:

>>> datetime.datetime.fromtimestamp(time.mktime((2015, 8, 11, 9, 18, 55, 1, 223, 1)))
datetime.datetime(2015, 8, 11, 8, 18, 55)
>>> datetime.datetime.fromtimestamp(time.mktime((2015, 8, 11, 9, 18, 55, 1, 223, 0)))
datetime.datetime(2015, 8, 11, 9, 18, 55)

UTC:

>>> datetime.datetime.fromtimestamp(time.mktime((2015, 8, 11, 9, 18, 55, 1, 223, 1)))
datetime.datetime(2015, 8, 11, 9, 18, 55)
>>> datetime.datetime.fromtimestamp(time.mktime((2015, 8, 11, 9, 18, 55, 1, 223, 0)))
datetime.datetime(2015, 8, 11, 9, 18, 55)

pykmip-server shuts down on client connect (vCenter Server 6.5)

I am having issues with pykmip-server shutting down on client connect, client is VMware vCenter Server 6.5.

Below is server.log contents from pykmip-server start to vCenter Server 6.5 connect. I have full certificate trusts configured for client and server.

2016-11-27 01:33:54,959 - kmip.server.config - INFO - Loading server configuration settings from: /etc/pykmip/server.conf
2016-11-27 01:33:54,974 - kmip.server.engine - INFO - Loading user-defined operation policy files from: /etc/pykmip/policies
2016-11-27 01:33:54,975 - kmip.server.engine - INFO - Loading user_defined operation policies from file: /etc/pykmip/policies/policy.json
2016-11-27 01:33:54,976 - kmip.server - INFO - Starting server socket handler.
2016-11-27 01:33:54,978 - kmip.server - INFO - Server successfully bound socket handler to 0.0.0.0:5696
2016-11-27 01:33:54,978 - kmip.server - INFO - Starting connection service...
2016-11-27 01:34:08,006 - kmip.server - WARNING - Error detected while establishing new connection.
2016-11-27 01:34:08,007 - kmip.server - ERROR - [SSL: NO_CERTIFICATE_RETURNED] no certificate returned (_ssl.c:765)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/services/server/server.py", line 297, in serve connection, address = self._socket.accept()
File "/usr/lib64/python2.7/ssl.py", line 862, in accept server_side=True)
File "/usr/lib64/python2.7/ssl.py", line 350, in wrap_socket _context=self)
File "/usr/lib64/python2.7/ssl.py", line 588, in init self.do_handshake()
File "/usr/lib64/python2.7/ssl.py", line 810, in do_handshake self._sslobj.do_handshake()
SSLError: [SSL: NO_CERTIFICATE_RETURNED] no certificate returned (_ssl.c:765)
2016-11-27 01:34:08,008 - kmip.server - INFO - Stopping connection service.
2016-11-27 01:34:08,008 - kmip.server - INFO - Cleaning up remaining connection threads.
2016-11-27 01:34:08,009 - kmip.server - INFO - Shutting down server socket handler.

license indications inconsistent

The trove classifiers on https://pypi.python.org/pypi/PyKMIP say that the library is released under the BSD license, but there is an Apache license in LICENSE.txt. Which is correct?

about locate operaton

Hi Peter,

Can I ask you two questions?

1. Do you have any ETA for the next release?

2. Currently I need an operation to "list" all the existing keys on the server. Should be "locate". Seems that currently it's not supported by server.
   I checked the SPEC, maximum items/storage status mask/object group member are all non-trivial features to support, I think.
   So do you think currently it makes sense to implement a simple locate operation which will just return uuids of all the existing managed objects of the server without any filtering? Later we can add the filtering piece by piece.

add key wrapping support encrypt decrypt signature and signature verify

Hello everybody
For one project I have to implement KMIP functionalities (client side) missing from this project
key wrapping support
encrypt and decrypt (executed on server)
signature and signature verify (executed on server)
Those functionalities exist in the KMIP server I have to connect
Before beginning the development,
1/ I would like to know if somebody has already done one of these
2/ I saw that in pull request key wrapping support was proposed long time ago but developments were never integrated in the project? Can you explain me why ? Those developments are out of sync from the master branch but part of the code can be reused, so should I start with it or restart from start ?

Thanks in advance

Bruno

ETA for my commits?

Hi Peter,
I have seen the commits are pending there for a long time.
Probably you are busy:) But can I ask an ETA for merging? No hurry~
Let me know if there is any issue. Thanks.

Newer Protocols

The README.md states that v1.2 is currently supported. Is there a roadmap for supporting newer versions and their features or is this just not on the horizon?

Are there plans for supporting more profiles than just the Basic Discover?

0.4.1 breaks openstack/barbican

Openstack consumes PyKMIP and in it's older kilo branch

Good: http://logs.openstack.org/44/246644/1/check/gate-barbican-python27/8824961/console.html.gz#_2015-11-17_23_20_38_489
Bad: http://logs.openstack.org/44/246644/1/check/gate-barbican-python27/f560247/console.html#_2015-12-22_00_58_13_829

There are many difference between the good and bad runs BUT if I re-test locally with PyKMIP 0.4.0 the tests pass. I believe that the following:
20dbad5

Is a backwards incompatible change and should have bumped the major version,

Add optional 'name' argument to Create and Create Key Pair functions

It would be helpful if we could add an optional 'name' keyword to some of the operations in ProxyKmipClient and possibly KMIPProxy. Notably, these operations:

• Create
• Create Key Pair

I don't think this would be a large change (I've got it working already for Create) and wouldn't break backwards compatibility (as it would default to not specifying the name).

If I went ahead and added this, would it be something you would consider accepting in a pull request?

Fails pep8 test

kmip/services/kmip_client.py:228:15: F821 undefined name 'e'

        for host in self.host_list:
            self.host = host
            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self._create_socket(sock)
            self.protocol = KMIPProtocol(self.socket)
            try:
                self.socket.connect((self.host, self.port))
            except Exception as e:
                self.logger.error("An error occurred while connecting to "
                                  "appliance " + self.host)
                self.socket.close()
            else:
                return

        self.socket = None
        raise e

Obviously on the last line "raise e", e is not in scope. I assume the last two lines should be moved into the expect block.

ssl.SSLError: [SSL] PEM lib (_ssl.c:2964)

KMIP client is failing to connect with ssl error, I have proper certificates and verified connection using openSSL s_client connect. Not able to figure out why connection is failing with KMIP client. It would be great if someone can help me on this.

`5268 21284 10/06 20:08:58 DEBUG KMIPProxy keyfile: C:\Encryption\SafeNetCerts\clientkey
5268 21284 10/06 20:08:58 DEBUG KMIPProxy certfile: C:\Encryption\SafeNetCerts\signed.crt
5268 21284 10/06 20:08:58 DEBUG KMIPProxy cert_reqs: 2 (CERT_REQUIRED: 2)
5268 21284 10/06 20:08:58 DEBUG KMIPProxy ssl_version: 2 (PROTOCOL_SSLv23: 2)
5268 21284 10/06 20:08:58 DEBUG KMIPProxy ca_certs: C:\Encryption\SafeNetCerts\MyCA.crt
5268 21284 10/06 20:08:58 DEBUG KMIPProxy do_handshake_on_connect: True
5268 21284 10/06 20:08:58 DEBUG KMIPProxy suppress_ragged_eofs: True

ssl.SSLError: [SSL] PEM lib (_ssl.c:2964)`

a bug in kmip/demos/units/create_key_pair.py

name = Attribute(attribute_name=name, attribute_value=value)
name = Attribute.AttributeName('Cryptographic Usage Mask')
value = CryptographicUsageMask(
    UsageMaskEnum.ENCRYPT.value | UsageMaskEnum.DECRYPT.value)
usage_mask = Attribute(attribute_name=name, attribute_value=value)
attribute_type = AttributeType.CRYPTOGRAPHIC_LENGTH
length_obj = attribute_factory.create_attribute(attribute_type,
                                                length)
attributes = [algorithm_obj, length_obj, name, usage_mask]

This will not work. Two 'name' assignment? I think it should be something like:

name = Attribute(attribute_name=name, attribute_value=value)
name_mask = Attribute.AttributeName('Cryptographic Usage Mask')
value = CryptographicUsageMask(
    UsageMaskEnum.ENCRYPT.value | UsageMaskEnum.DECRYPT.value)
usage_mask = Attribute(attribute_name=name_mask, attribute_value=value)

attribute_type = AttributeType.CRYPTOGRAPHIC_LENGTH
length_obj = attribute_factory.create_attribute(attribute_type,
                                                length)
attributes = [algorithm_obj, length_obj, name, usage_mask]

pykmip server resetting connection request from kmip4j client

I am trying to connected to pykmip server from kmip4j client but I am getting below message in server logs.

"kmip.server - ERROR - [Errno 1] _ssl.c:510: error:1408
F10B:SSL routines:SSL3_GET_RECORD:wrong version number"

I have created server certificates using below openssl commands.

openssl genrsa -des3 -out server.orig.key 2048
openssl rsa -in server.orig.key -out server.key
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

On client side I am using kmip4j taken from github link.I use below command to bring up gui interface of client working on ubuntu.

java -cp "./gui/target/classes:./core/target/classes:/usr/share/maven-repo/org/slf4j/slf4j-api/1.6.4/:/usr/share/maven-repo/org/slf4j/slf4j-api/debian:/usr/share/maven-repo/org/slf4j/slf4j-api-1.7.21.jar:/usr/share/maven-repo/org/slf4j/slf4j-simple-1.7.2.jar:/usr/share/maven-repo/org/slf4j/impl/slf4j-simple-1.7.21.jar:./utils/target/kmip4j-utils-1.0.jar:./core/target/kmip4j-core-1.0.jar:./kmls/kmls/target/kmip4j-kmls-1.0.jar:./kmls/kmls-db/target/kmip4j-kmls-db-1.0.jar" ch.ntb.inf.kmip.client.gui.KMIPClientGUI

Solutions: Solution provided on different forums suggest disabling specific sslv version.
But I am getting a way of disabling this version with kmip4j client.
Please help

Testing vSphere 6.5 with PyKMIP - Get_Attributes operation is not supported by the server

vCenter Side log
2016-12-01T03:42:21.118+05:30 error vpxd[06808] [Originator@6876 sub=CryptoManagerKmipWrapper opID=EditEncryptionModeViewMediator-apply-22497-ngc-99] Get, Attributes, Id=%s, Names=State failed on key 44, Get_Attributes operation is not supported by the server.
2016-12-01T03:42:21.118+05:30 warning vpxd[06808] [Originator@6876 sub=CryptoManager opID=EditEncryptionModeViewMediator-apply-22497-ngc-99] Cannot get key 44 from KMS server kms, reason: Get_Attributes operation is not supported by the server.

PyKMIP side server log

2016-12-01 03:42:20,850 - kmip.server.engine - INFO - Processing operation: Create
2016-12-01 03:42:20,850 - kmip.server.engine.cryptography - INFO - Generating a AES symmetric key with length: 256
2016-12-01 03:42:20,891 - kmip.server.engine - INFO - Created a SymmetricKey with ID: 44
2016-12-01 03:42:20,893 - kmip.server.session.00000001 - INFO - Stopping session: 00000001
2016-12-01 03:42:20,970 - kmip.server - INFO - Receiving incoming connection from: 192.168.100.10:51904
2016-12-01 03:42:20,970 - kmip.server - INFO - Dedicating session 00000002 to 192.168.100.10:51904
2016-12-01 03:42:20,991 - kmip.server.session.00000002 - INFO - Starting session: 00000002
2016-12-01 03:42:20,992 - kmip.server.session.00000002 - INFO - Session client identity: kmipClient20161130113042
2016-12-01 03:42:20,994 - kmip.server.engine - INFO - Received request at time: 2016-11-30 22:12:20
2016-12-01 03:42:20,994 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2016-12-01 03:42:21,001 - kmip.server.session.00000002 - INFO - Session client identity: kmipClient20161130113042
2016-12-01 03:42:21,005 - kmip.server.engine - INFO - Received request at time: 2016-11-30 22:12:21
2016-12-01 03:42:21,006 - kmip.server.engine - INFO - Processing operation: Get
2016-12-01 03:42:21,011 - kmip.server.engine - INFO - Getting a SymmetricKey with ID: 44
2016-12-01 03:42:21,020 - kmip.server.session.00000002 - INFO - Session client identity: kmipClient20161130113042
2016-12-01 03:42:21,025 - kmip.server.engine - INFO - Received request at time: 2016-11-30 22:12:21
2016-12-01 03:42:21,027 - kmip.server.session.00000002 - INFO - Stopping session: 00000002

client.py argument checking

Hi Peter,

When I am working on adding the pie support for locate operation, I do notice that there are some issues about the arguments checking in client.py. It's too strict, I think. For example, for get() operation, from the spec, uid can be ignored while it must be a string in the codes. This means that using the pie interface, I can not issue an get request without any uid?

Should we refactor this some time? I think your vision is to put all the argument validation inside the payload module in the future?

Add support for Python 3.5

With the recent release of Python 3.5, many Python developers are upgrading to a version of Python not officially supported by PyKMIP. Since PyKMIP already supports Python 3.3 and 3.4, adding official support for 3.5 should be trivial.

kmip.server NotImplementedError with VMware vCenter Server 6.5

I am having trouble issuing certificates to VMware vCenter Server 6.5. Certificate requests trigger an "NotImplementedError" on pykmip-server.

Below is contents of server.log during certificate request

2016-11-27 13:03:47,353 - kmip.server - INFO - Receiving incoming connection from: 192.168.78.10:40412
2016-11-27 13:03:47,354 - kmip.server - INFO - Dedicating session 00000007 to 192.168.x.x:40412
2016-11-27 13:03:47,355 - kmip.server.session.00000007 - INFO - Starting session: 00000007
2016-11-27 13:03:47,358 - kmip.server.session.00000007 - INFO - Session client identity: kmipClient20161127094708
2016-11-27 13:03:47,360 - kmip.server.engine - INFO - Received request at time: 2016-11-27 11:03:47
2016-11-27 13:03:47,361 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2016-11-27 13:03:47,364 - kmip.server.session.00000007 - INFO - Session client identity: kmipClient20161127094708
2016-11-27 13:03:47,367 - kmip.server.engine - INFO - Received request at time: 2016-11-27 11:03:47
2016-11-27 13:03:47,368 - kmip.server.engine - INFO - Processing operation: Create
2016-11-27 13:03:47,368 - kmip.server.engine.cryptography - INFO - Generating a AES symmetric key with length: 256
2016-11-27 13:03:47,379 - kmip.server.engine - INFO - Created a SymmetricKey with ID: 13
2016-11-27 13:03:47,381 - kmip.server.session.00000007 - INFO - Stopping session: 00000007
2016-11-27 13:03:47,454 - kmip.server - INFO - Receiving incoming connection from: 192.168.78.10:40416
2016-11-27 13:03:47,454 - kmip.server - INFO - Dedicating session 00000008 to 192.168.x.x:40416
2016-11-27 13:03:47,455 - kmip.server.session.00000008 - INFO - Starting session: 00000008
2016-11-27 13:03:47,457 - kmip.server.session.00000008 - INFO - Session client identity: kmipClient20161127094708
2016-11-27 13:03:47,459 - kmip.server.engine - INFO - Received request at time: 2016-11-27 11:03:47
2016-11-27 13:03:47,460 - kmip.server.engine - INFO - Processing operation: DiscoverVersions
2016-11-27 13:03:47,463 - kmip.server.session.00000008 - INFO - Session client identity: kmipClient20161127094708
2016-11-27 13:03:47,465 - kmip.server.engine - INFO - Received request at time: 2016-11-27 11:03:47
2016-11-27 13:03:47,466 - kmip.server.engine - INFO - Processing operation: Get
2016-11-27 13:03:47,471 - kmip.server.engine - INFO - Getting a SymmetricKey with ID: 13
2016-11-27 13:03:47,474 - kmip.server.session.00000008 - INFO - Session client identity: kmipClient20161127094708
2016-11-27 13:03:47,475 - kmip.server.session.00000008 - WARNING - Failure parsing request message.
2016-11-27 13:03:47,475 - kmip.server.session.00000008 - ERROR -
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/services/server/session.py", line 150, in _handle_message_loop
request.read(request_data)
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/core/messages/messages.py", line 354, in read
batch_item.read(istream)
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/core/messages/messages.py", line 203, in read
self.operation.value)
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/core/factories/payloads/init.py", line 44, in create
return self._create_get_attributes_payload()
File "/usr/lib/python2.7/site-packages/PyKMIP-0.5.0-py2.7.egg/kmip/core/factories/payloads/init.py", line 117, in _create_get_attributes_payload
raise NotImplementedError()
NotImplementedError
2016-11-27 13:03:47,477 - kmip.server.session.00000008 - INFO - Stopping session: 00000008

Below is logs from vCenter Server

2016-11-27T11:08:28.434Z info vpxd[7F2212164700] [Originator@6876 sub=CryptoManager opID=EditEncryptionModeViewMediator-apply-14823-ngc-93] Generated new cryptoKeyId 14/KMS for the host [vim.HostSystem:host-109,esxi.lan.local]
2016-11-27T11:08:28.525Z error vpxd[7F2212164700] [Originator@6876 sub=CryptoManagerKmipWrapper opID=EditEncryptionModeViewMediator-apply-14823-ngc-93] Get, Attributes, Id=%s, Names=State failed on key 14, Error parsing request message. See server logs for more information.
2016-11-27T11:08:28.525Z warning vpxd[7F2212164700] [Originator@6876 sub=CryptoManager opID=EditEncryptionModeViewMediator-apply-14823-ngc-93] Cannot get key 14 from KMS server kms1, reason: Error parsing request message. See server logs for more information.
2016-11-27T11:08:28.526Z info vpxd[7F2212164700] [Originator@6876 sub=vpxLro opID=EditEncryptionModeViewMediator-apply-14823-ngc-93] [VpxLRO] -- FINISH task-536
2016-11-27T11:08:28.526Z info vpxd[7F2212164700] [Originator@6876 sub=Default opID=EditEncryptionModeViewMediator-apply-14823-ngc-93] [VpxLRO] -- ERROR task-536 -- host-109 -- vim.HostSystem.configureCryptoKey: vmodl.RuntimeFault:
--> Result:
--> (vmodl.RuntimeFault) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = (vmodl.LocalizableMessage) [
--> (vmodl.LocalizableMessage) {
--> key = "com.vmware.vim.vpxd.encryption.keyNotFound",
--> arg = (vmodl.KeyAnyValue) [
--> (vmodl.KeyAnyValue) {
--> key = "key",
--> value = "14/KMS"
--> }
--> ],
--> message =
--> }
--> ]
--> msg = ""
--> }
--> Args:
-->
--> Arg keyId:
-->

policy_path error when running server

I'm receiving the following error when attempting to run the PyKMIP server 0.5.0:
"kmip.core.exceptions.ConfigurationError: Setting 'policy_path' is missing from the configuration file."

I cannot find policy_path documented anywhere and have not found any guidance online. I have attempted to insert a path statement in the config file the error then changes to:
"kmip.core.exceptions.ConfigurationError: The policy path value, if specified, must be a valid string path to a filesystem directory." The path I used is a valid path with appropriate permissions but not knowing the format of the expected policy file I am unable to provide a suitable target.

Any guidance would be appreciated.

CRYPTOGRAPHIC_USAGE_MASK for pie client

Hi Peter,

Currently I am trying to add the MAC operation(client and server parts). I want to use the pie client.

I found for the create/create_key_pair operations(in pie/client.py), you make the CRYPTOGRAPHIC_USAGE_MASK to be encrypt/decrypt. According to KMIP protocol, the key object can be used for MAC only if the MAC usage mask is set. So shouldn't we make the usage mask as a input argument of the create/create_key_pair functions?

SSL: CERTIFICATE_VERIFY_FAILED : PyKmip client debugging options

Hello There,

Quick Summary : I'm trying to use the Pykmip client to talk to a kmip server. I run into the "certificate verify failed" error. How can I debug this ? How can I enable logging ?

Background : I have a kmip server running on a remote host which stores the client-credentials in a *.pfx format (It follows pkcs12).

I generated 3 files from the server end:

1. A certfile
   extracted using the openssl command :
   openssl pkcs12 -in certname.pfx -nokeys -out cert.pem
2. A keyfile
   extracted using the openssl command :
   openssl pkcs12 -in certname.pfx -nocerts -out key.pem -nodes
3. ca_certs (root-ca certificates for the server) : stored in a server directory

python test_client.py
WARNING:kmip.core.config_helper:Config file ['~/.pykmip/pykmip.conf', '/etc/pykmip/pykmip.conf', '/Library/Python/2.7/site-packages/PyKMIP-0.6.0-py2.7.egg/kmip/pykmip.conf', '/Library/Python/2.7/site-packages/PyKMIP-0.6.0-py2.7.egg/kmip/kmipconfig.ini'] not found

username: None, conformance_clauses: [<ConformanceClause.DISCOVER_VERSIONS: 1>], certfile: ~/testcert.pem, timeout: 30, host_list_str: 10.129.130.65, keyfile: ~/server.key, do_handshake_on_connect: True, host_list: ['10.129.130.65'], ssl_version: 2, authentication_suites: [<AuthenticationSuite.BASIC: 1>, <AuthenticationSuite.TLS12: 2>], host: 10.129.130.65, batch_items: [], credential_factory: <kmip.core.factories.credentials.CredentialFactory object at 0x1098dfc50>, cert_reqs: 2, ca_certs: ~/server-ca.cer, logger: <logging.Logger object at 0x1098dfb90>, password: None, config: client, port: 5696, suppress_ragged_eofs: True, socket: None

ERROR:kmip.services.kmip_client:An error occurred while connecting to appliance 10.129.130.65
Traceback (most recent call last):
File " ", line 76, in
client.open()
File "/Library/Python/2.7/site-packages/PyKMIP-0.6.0-py2.7.egg/kmip/services/kmip_client.py", line 239, in open
raise last_error
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

Bug: No accessible keys in DeriveKey support

The implementation of DeriveKey in the server does not handle the case where none of the provided base objects are accessible to the user. When this happens, an indexing error occurs, causing the operation to fail.

Implement a fix for this bug and add a unit test case explicitly checking for this condition.

statement about "auth_suite" is not accurate

"
Note: TLS1.2 can only be used with versions of Python that support TLS 1.2 (e.g,. Python 2.7.9+ or Python 3.4+). If you are running on an older version of Python, you will only be able to use basic TLS 1.0 authentication.
"
probably is not very accurate.

I did some investigation. For example, package "ssl" in python 2.7.6 doesn't have the constant "PROTOCOL_TLSv1_2" defined. But actually "ssl" is just a very thin wrapper of openssl. PROTOCOL_TLSv1_2 is 5 in openssl. As long as we pass 5 to it, it should work.

So probably we can come up with some workarounds instead of saying "only be able to use basic TLS 1.0 authentication"?

SQLAlchemy DetachedInstanceError intermittently breaking unit tests

PR #296 is failing unit tests with the following stacktrace:

________ TestKmipEngine.test_derive_key_oversized_cryptographic_length _________

NOTE: Incompatible Exception Representation, displaying natively:

testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/home/travis/build/OpenKMIP/PyKMIP/kmip/tests/unit/services/server/test_engine.py", line
4085, in test_derive_key_oversized_cryptographic_length
    unique_identifiers=[str(base_key.unique_identifier)],
  File "/home/travis/build/OpenKMIP/PyKMIP/.tox/py36/lib/python3.6/site-packages/sqlalchemy
/orm/attributes.py", line 237, in __get__
    return self.impl.get(instance_state(instance), dict_)
  File "/home/travis/build/OpenKMIP/PyKMIP/.tox/py36/lib/python3.6/site-packages/sqlalchemy
/orm/attributes.py", line 579, in get
    value = state._load_expired(state, passive)
  File "/home/travis/build/OpenKMIP/PyKMIP/.tox/py36/lib/python3.6/site-packages/sqlalchemy
/orm/state.py", line 592, in _load_expired
    self.manager.deferred_scalar_loader(self, toload)
  File "/home/travis/build/OpenKMIP/PyKMIP/.tox/py36/lib/python3.6/site-packages/sqlalchemy
/orm/loading.py", line 644, in load_scalar_attributes
    (state_str(state)))
sqlalchemy.orm.exc.DetachedInstanceError: Instance <SymmetricKey at 0x7f04e5dae358> is not 
bound to a Session; attribute refresh operation cannot proceed

----------------------------- Captured stderr call -----------------------------

The specified operation policy directory is not valid. No user-defined policies will be loaded.

Interesting things to note:

• The unit test that fails, test_derive_key_oversized_cryptographic_length, is always the same test and is not modified at all by #296.
• The sqlalchemy-related code in the unit test is identical to dozens of other tests and yet seemingly only fails in this one test.
• This failure consistently shows up in the TravisCI test runs but has only shown up once during local testing. Reruns of the local unit test suite cannot reproduce the error.
• This failure only occurs under Python3.6.

Bug: socket.error during test_kmip_client.py test execution

Running the unit test suite will sometimes generate the following error when executing kmip/tests/unit/services/test_kmip_client.py:

Exception socket.error: error(107, 'Transport endpoint is not connected') in <bound method
KMIPProxy.__del__ of <kmip.services.kmip_client.KMIPProxy object at 0x7f495986fed0>> ignored

Note that this is an intermittent error and shows up in roughly 1 in 20 test runs.

Identify the test causing this error and fix it.

Support to VM Encryption in vSphere 6.5

Hello All, happy to see vmware now support to VM encryption with KMIP protocol, only PyKMIP is free solution at this moment, hope to see more article to show to setup and manage PyKMIP in this topic and see vmware contribute more code to pykimp

Drop support for Python 2.6

The ssl module in Python 2.6 does not support TLS 1.2. Given that other prominent open source projects, like OpenStack and pyca/cryptography, are moving away from supporting 2.6, PyKMIP should move to deprecate 2.6 support in the next release, ultimately dropping 2.6 support down the road.

runserver.py prints warning at startup

/home/toothwalker/.local/lib/python2.7/site-packages/sqlalchemy/orm/mapper.py:1034: SAWarning: Reassigning polymorphic association for identity 'Certificate' from <Mapper at 0x7fb3e7980410; Certificate> to <Mapper at 0x7fb3e7980910; X509Certificate>: Check for duplicate use of 'Certificate' as value for polymorphic_identity.
self, self.polymorphic_identity)

regarding KmipOperationFailure

Hi Peter,

In pie client, KmipOperationFailure will be raised if the operation failed.
I have seen in exceptions.py, you combined status, reason and message into one message string for KmipOperationFailure. Then how the caller is supposed to handle the exception based on different reasons? By parsing the message string?
Should it be more reasonable to define it like this:

  class KmipOperationFailure(Exception):
    def __init__(self, status, reason, message):
        super(KmipOperationFailure, self).__init__(message)
        self.status = status
        self.reason = reason

So that the caller can directly access 'reason' and maybe have different handlers for different reason?

Install/Run Instructions and SQLAlchemy Issue

I have installed the latest version of pykmip-server (0.5.0) but have some questions.

I noticed that the instructions on the Wiki mention to run the "run_server.py" script but it appears as though the newer versions of pykmip support installation of Pykmip-server using the easy-install/pip method (python server.py build/install). Is the easy-install/pip the preferred method of installing now?

When I start the server (0.5.0) using the pykmip-server script in /bin, I am receiving the following message:

/usr/lib/python3.4/site-packages/SQLAlchemy-1.1.0b3-py3.4-linux-x86_64.egg/sqlalchemy/orm/mapper.py:1034: SAWarning: Reassigning polymorphic association for identity 'Certificate' from <Mapper at 0x7f2fa55a99b0; Certificate> to <Mapper at 0x7f2fa55a9eb8; X509Certificate>: Check for duplicate use of 'Certificate' as value for polymorphic_identity

I have generated certs and they are signed by a CA. I even tried using the demo certs that were included. It appears as though the server still runs with after this message is displayed but wasn't sure if it is a warning or is causing other issues. I confirmed that I am binding to port 5696 on a public interface and enabled logging. I'm not currently able to connect from my client.

The message I receive in the log when a client attempts to connect is:

2016-09-21 16:08:35,948 - kmip.server.session.00000001 - WARNING - An unexpected error occurred while processing request.
2016-09-21 16:08:35,948 - kmip.server.session.00000001 - ERROR - 'NoneType' object has no attribute 'credential'
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/PyKMIP-0.5.0-py3.4.egg/kmip/services/server/session.py", line 106, in _handle_message_loop
request
File "/usr/lib/python3.4/site-packages/PyKMIP-0.5.0-py3.4.egg/kmip/services/server/engine.py", line 142, in decorator
return function(self, _args, *_kwargs)
File "/usr/lib/python3.4/site-packages/PyKMIP-0.5.0-py3.4.egg/kmip/services/server/engine.py", line 252, in process_request
auth_credentials = header.authentication.credential
AttributeError: 'NoneType' object has no attribute 'credential'

Last question. I tried going back to a previous version of PyKmip (such as 0.4.1) since this is prior to using sqlalchemy. When I install this, there is no pykmip-server in /bin and I see no run_server.py. I tried running the kmip/services/server/server.py script but it runs and then exits right away. What is the preferred way to run in version 0.4.1?

Sorry for all of the questions and thanks for your hard work. I would greatly appreciate any help you can provide.

cryptography>=1.1 ?

I have found in your requirements.txt you specify cryptography>=1.1.
However, I found at least in version 1.2.3, 'ANSIX923' doesn't exist. So should we update the info?

`

import cryptography as c
c.version
'1.2.3'
from cryptography.hazmat.primitives import padding as symmetric_padding
dir(symmetric_padding)
['AlreadyFinalized', 'PKCS7', 'PaddingContext', '_PKCS7PaddingContext', '_PKCS7UnpaddingContext', 'builtins', 'doc', 'file', 'name', 'package', 'abc', 'absolute_import', 'division', 'lib', 'print_function', 'six', 'utils']`

`

import cryptography as c
c.version
'1.7.2'
from cryptography.hazmat.primitives import padding as symmetric_padding
dir(symmetric_padding)
['ANSIX923', 'AlreadyFinalized', 'PKCS7', 'PaddingContext', '_ANSIX923PaddingContext', '_ANSIX923UnpaddingContext', '_PKCS7PaddingContext', '_PKCS7UnpaddingContext', 'builtins', 'doc', 'file', 'name', 'package', '_byte_padding_check', '_byte_padding_pad', '_byte_padding_update', '_byte_unpadding_check', '_byte_unpadding_update', 'abc', 'absolute_import', 'division', 'lib', 'print_function', 'six', 'utils']`

default CLI args not over-written by config settings

I have a config file that contains

port=7000

However, the server still uses the default port value (5696). I see that is because

1. the OptionParser has the "port" option using a default of 5696. This causes KmipServer._setup_configuration() to have its 'port' parameter set to 5696
2. KmipServer._setup_configuration() calls self.config.load_settings() before handling the settings passed in its input arguments.

Fixing this may be as simple as moving the call to config.load_settings() down a few lines (after the handling of the input parameters).

Use pbr to handle version management

The PyKMIP version number is currently stored in kmip/version.py and is dynamically loaded wherever needed. Investigate the use of pbr to do this instead by dynamically pulling version information from the Git repository tags.

Failures in client integration tests

Steps I took

1. Created certificates
2. Created /etc/pykmip/server.conf
3. Modified kmip/pykmip.conf to use new certs
4. ./bin/run_server.py
5. tox -e integration -- --config client

4 tests deselected
7 failed, 12 passed, 4 deselected in 2.59 seconds
ERROR: InvocationError: '/home/toothwalker/PyKMIP/.tox/integration/bin/py.test --strict kmip/tests/integration -m not ignore --config client'

Files of interest: (note: added .txt for github to accept them)
pykmip.conf.txt
server.conf.txt
server.log.txt
test_output.txt

kmip.server.session0000001[n] - WARNING - Failure parsing request message

I'm evaluating the server functionality of the PyKMIP-master branch with vCenter 6.5 as a client so I can use the encryption feature provided by VMWare.

In /var/log/pykmip/server.log I get the above message when trying to establish trust with KMS from vCenter.

It follows with kmip.server.session.000000[n] - ERROR - 'SSLSocket' object has no attribute 'shared_ciphers'.

Can anybody point me in the right direction with this?

Add GetAttributes operation support to the server

The setting of most managed object attributes is currently handled by the server when creating new managed objects, but there is no way to retrieve arbitrary attributes, like object state, from the server. Add support for the GetAttributes operation to allow this retrieval to occur.

Need support for Locate operation

The Locate operation would be useful for finding keys by name attribute rather than UUID.

General question on testing client functions

Can anyone point me to examples using the PyKMIP client for CreateKeyPair in either the original client or the pie client. I've looked in kmip/demos and see the various py files but I was hoping to find a listing of example runs with sample arguments provided. Thanks in advance for any guidance.