canonical / pylxd

Python module for LXD

pylxd

A Python library for interacting with the LXD REST API.

Installation

pip install pylxd

Bug reports

Bug reports can be filed on the GitHub repository.

Support and discussions

We use the LXD category on Ubuntu's Discourse.

If you prefer live discussions, some of us also hang out in #lxd on irc.libera.chat.

LXD Documentation: https://documentation.ubuntu.com/lxd/en/latest/

PyLXD API Documentation: https://pylxd.readthedocs.io/en/latest/

Contributing to pyLXD

If you wish to contribute to pyLXD please read the Contributing Guide, particularly around unit tests, integration tests and signing commits.

pylxd's Issues

pypi pylxd is out-of-date

https://pypi.python.org/pypi/pylxd/0.14.0

Please can we get the current release (0.18.x) published

Create operations can fail silently

Here is a case in which a create operation will appear to return successfully, but the container has not been created. The client has no idea why.

from pylxd import api
CONTAINER_NAME="nettest"

config = {
  'source': {
    'alias': '16.04', 'type': 'image', 'mode': 'pull', 'protocol': 'lxd', 'server': 'https://images.linuxcontainers.org'},
  'name': CONTAINER_NAME,
  'profiles': ['default']
}

lxd = api.API()
lxd.container_init(config)
if lxd.container_defined(CONTAINER_NAME):
    print("Container is running")
else:
    print("Whoops - please report a bug!")

Prints: Whoops - please report a bug!

Of course, almost certainly I'm sending the wrong REST parameters here. The issue is that presumably pylxd is getting some sort of error response over HTTP, but it is not catching it and turning it into an exception.

pylxd.Client() raises ValueError if e.g. http_proxy environment variable is set

pylxd appears to pass any proxy related environment variables onto Requests, even in the default case when connecting to a Unix domain socket. This results in an an unhandled exception

$ python
Python 2.7.11+ (default, Apr 17 2016, 14:00:29) 
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pylxd
>>> pylxd.__version__
'2.0.2'
>>> pylxd.Client()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/willmerae/.local/lib/python2.7/site-packages/pylxd/client.py", line 183, in __init__
    response = self.api.get()
  File "/home/willmerae/.local/lib/python2.7/site-packages/pylxd/client.py", line 87, in get
    response = self.session.get(self._api_endpoint, *args, **kwargs)
  File "/home/willmerae/.local/lib/python2.7/site-packages/requests/sessions.py", line 487, in get
    return self.request('GET', url, **kwargs)
  File "/home/willmerae/.local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/willmerae/.local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/home/willmerae/.local/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/willmerae/.local/lib/python2.7/site-packages/requests_unixsocket/adapters.py", line 53, in get_connection
    % self.__class__.__name__)
ValueError: UnixAdapter does not support specifying proxies

If I unset every proxy related environment variable, then the call succeeds

$ unset http_proxy https_proxy no_proxy HTTP_PROXY HTTPS_PROXY NO_PROXY 
$ python
Python 2.7.11+ (default, Apr 17 2016, 14:00:29) 
[GCC 5.3.1 20160413] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pylxd
>>> pylxd.Client()
<pylxd.client.Client object at 0x7f6d65fd5e10>

Example API usage specifies config for container_init, but takes container argument

examples/api_test.py has the following:

config = {'name': CONTAINER_NAME,
          'source': {'type': 'none'}}
lxd.container_init(config)

But pylxd/container.py has:

def container_init(self, container):
        return self.connection.get_object('POST', '/1.0/containers',
                                          json.dumps(container))

It looks like you're passing a config, not a container, and it's not documented what the options to that config are.

Image upload fails with multi part file (http upload)

pylxd.exceptions.ClientConnectionFailed
>>> c = Client()
>>> image_data = open('lxd_post_220970285').read()
>>> i = c.images.create(image_data, wait=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/stack/pylxd/pylxd/image.py", line 94, in create
    Operation.wait_for_operation(client, response.json()['operation'])
  File "/opt/stack/pylxd/pylxd/operation.py", line 29, in wait_for_operation
    operation.wait()
  File "/opt/stack/pylxd/pylxd/operation.py", line 51, in wait
    raise exceptions.LXDAPIException(response)
pylxd.exceptions.LXDAPIException: detectCompression failed, err='Unsupported compression.', tarfile='/var/lib/lxd/images/lxd_build_064453356/lxd_tar_482497790'

'module' object has no attribute 'PY34' - pylxd

I ran the example provided below and got the error
'module' object has no attribute 'PY34'
What is this about ? Running Ubuntu.

import uuid
from pylxd import api

# Let's pick a random name, avoiding clashes
CONTAINER_NAME = str(uuid.uuid1())

lxd = api.API()

try:
    lxd.container_defined(CONTAINER_NAME)
except Exception as e:
    print("Container does not exist: %s" % e)

config = {'name': CONTAINER_NAME,
          'source': {'type': 'none'}}
lxd.container_init(config)
if lxd.container_defined(CONTAINER_NAME):
    print("Container is running")
else:
    print("Whoops - please report a bug!")
containers = lxd.container_list()
for x in containers:
    lxd.container_destroy(x)

execute don't work with environement variable

using lxc bash the following works :

 lxc exec container -- echo $HOME

But using pylxd, it doesn't :

  container.execute(["echo", "$HOME"]) 

What's wrong ?

Remote Authentication Fail - Response 400

Hello, I had a problem when I tried to do a client authentication as described is here:
https://github.com/lxc/pylxd/blob/master/doc/source/authentication.rst

The problem was that I was getting http response 400.
Then, I tried to do it manually and it was successful (sending the post request with "curl").
I fugure out that the problem was in the python dictionary that is send in the request.

However, when I change the dictionary to simple string, works:
data = '{"type": "client", "password": "******"}'

Container.update needs better documentation

Currently we can specify config for a container only when we create a container: pylxd/container.py at 865ec541bc44b7915502a63a3764ada98a7465cf · lxc/pylxd
Could you add a method to Container for updating configuration?
With the LXD REST API PUT /1.0/containers/, you can update configuration of a container.

Cannot import pylxd from terminal

I installed pylxd using pip install pylxd and when I try to load it in the terminal I get...

>>> import pylxd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/akahan/Work/containers_js/env/lib/python2.7/site-packages/pylxd/__init__.py", line 19, in <module>
    from pylxd.deprecated import api  # NOQA
  File "/Users/akahan/Work/containers_js/env/lib/python2.7/site-packages/pylxd/deprecated/api.py", line 17, in <module>
    from pylxd.deprecated import certificate
  File "/Users/akahan/Work/containers_js/env/lib/python2.7/site-packages/pylxd/deprecated/certificate.py", line 17, in <module>
    from pylxd.deprecated import base
  File "/Users/akahan/Work/containers_js/env/lib/python2.7/site-packages/pylxd/deprecated/base.py", line 16, in <module>
    from pylxd.deprecated import connection
  File "/Users/akahan/Work/containers_js/env/lib/python2.7/site-packages/pylxd/deprecated/connection.py", line 46, in <module>
    DEFAULT_TLS_VERSION = OpenSSL.SSL.TLSv1_2_METHOD
NameError: name 'OpenSSL' is not defined
>>> import OpenSSL
>>> OpenSSL.SSL.TLSv1_2_METHOD
6

Not sure what's going on here and why it's trying to import code from a path with deprecated in it? Are there other deps that are needed to install this?

Add git tag for 2.0.0b2

hi,

Can you add 2.0.0b2 tag on this commit:
b96a4bd

Thanks,

Make something about `python -m pylxd`

Would be nice to have something like diagnostic info when executed standalone. Maybe some commands.

$ python -m pylxd
...
No module named pylxd.__main__; 'pylxd' is a package and cannot be directly executed

I personally think to try pylxd for controlling remote container, so inspecting if pyxld can connect it from command line would be nice.

host_info() is broken

Hi,

given that little python script:

#!/usr/bin/env python

from pylxd import api

if __name__ == '__main__':
        lxd = api.API()
        lxd.host_info()

I get the following traceback:

Traceback (most recent call last):
File "./test_pylxd.py", line 7, in
lxd.host_info()
File "/home/sadig/Projects/git.sadig/pylxd/pylxd/api.py", line 44, in host_info
return self.hosts.host_info()
File "/home/sadig/Projects/git.sadig/pylxd/pylxd/hosts.py", line 38, in host_info
self.get_lxd_backing_fs(data.get('metadata')),
File "/home/sadig/Projects/git.sadig/pylxd/pylxd/hosts.py", line 73, in get_lxd_backing_fs
return data['environment']['backing_fs']
KeyError: 'backing_fs'

checking the 'data' hash, it seems, that the result of 'metadata' doesn't contain 'backing_fs' anymore.
The same issue is also with the 'version' key from 'metadata'

oslo_utils is used but not in the requirements.txt

Used by https://github.com/lxc/pylxd/blob/master/pylxd/image.py#L19 but not installed from the requirements.txt file

container.profiles are not updated after updating profiles

In my environment, I created the profile named "bridged".

I tried the following script.

from pylxd.client import Client

client = Client()
container = client.containers.get('myubuntu2')
print(container.profiles)

container.profiles = ['default', 'bridged']
container.update()

container = client.containers.get('myubuntu2')
print(container.profiles)

I got the following output.

[u'default']
[u'default']

Actually the profiles are applied to the container.

$ lxc config show --expanded myubuntu2
name: myubuntu2
profiles:
- default
- bridged
config:
  limits.cpu: "2"
  volatile.base_image: ed0fb49ea8c3698c96c14157dff05b8c55eab2db438d3b043af1037836f1fa2b
  volatile.eth0.hwaddr: 00:16:3e:ef:d4:f6
  volatile.eth0.name: eth1
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
devices:
  eth0:
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    type: disk
ephemeral: false

So I think the output should become like the following:

[u'default']
[u'default', u'bridged']

I installed pylxd with pip install git+https://github.com/lxd/pylxd

$ pip list | grep pylxd
pylxd (2.0.3.dev20)

AttributeError: 'Image' object has no attribute 'update_source'

First, thanks for this very nice written piece of Software. It got a very nice API!

LXC Info:

driver: lxc
driverversion: 2.0.3
kernel: Linux
kernelarchitecture: x86_64
kernelversion: 4.4.0-31-generic
server: lxd
serverpid: 1291
serverversion: 2.0.3
storage: btrfs
storageversion: "4.4"

PyLXD: 2.0.3 from pip
Python: 2.7

I've created a image with lxc:

$ lxc image copy ubuntu:x local: --copy-aliases --auto-update

Now when i run:

import pylxd
client = pylxd.Client()
images = client.images.all()
[i.marshall() for i in images]

I get the stacktrace:

root@srv02:/home/pcdummy# salt-call lxd.image_list --out=json
[ERROR   ] An un-handled exception was caught by salt's global exception handler:
AttributeError: 'Image' object has no attribute 'update_source'
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    salt_call()
  File "/usr/lib/python2.7/dist-packages/salt/scripts.py", line 345, in salt_call
    client.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/call.py", line 58, in run
    caller.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 134, in run
    ret = self.call()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 197, in call
    ret['return'] = func(*args, **kwargs)
  File "/var/cache/salt/minion/extmods/modules/lxd.py", line 1069, in image_list
    return [i.marshall() for i in images]
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 183, in marshall
    marshalled[key] = getattr(self, key)
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 112, in __getattribute__
    self.sync()
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 144, in sync
    setattr(self, key, val)
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 125, in __setattr__
    return super(Model, self).__setattr__(name, value)
AttributeError: 'Image' object has no attribute 'update_source'
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    salt_call()
  File "/usr/lib/python2.7/dist-packages/salt/scripts.py", line 345, in salt_call
    client.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/call.py", line 58, in run
    caller.run()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 134, in run
    ret = self.call()
  File "/usr/lib/python2.7/dist-packages/salt/cli/caller.py", line 197, in call
    ret['return'] = func(*args, **kwargs)
  File "/var/cache/salt/minion/extmods/modules/lxd.py", line 1069, in image_list
    return [i.marshall() for i in images]
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 183, in marshall
    marshalled[key] = getattr(self, key)
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 112, in __getattribute__
    self.sync()
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 144, in sync
    setattr(self, key, val)
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 125, in __setattr__
    return super(Model, self).__setattr__(name, value)
AttributeError: 'Image' object has no attribute 'update_source'

Auto-Update is pretty new feature:
lxc/incus@c71a218

Feature: operations websocket

Hi,

i will wish use this API : /1.0/operations//websocket

Thanks,

Poor handling of error responses during create operation

In pylxd/container.py:

    @classmethod
    def create(cls, client, config, wait=False):
        """Create a new container config."""
        response = client.api.containers.post(json=config)

        if wait:
            Operation.wait_for_operation(client, response.json()['operation'])
        return cls(name=config['name'])

However when there is an error response, it has no 'operation' member, as shown in the API

As a result, the exception generated is not helpful in diagnosing the problem:

  File \"/usr/lib/python2.7/dist-packages/pylxd/container.py\", line 68, in create
    Operation.wait_for_operation(client, response.json()['operation'])
KeyError: 'operation'

An example of this is in the new ansible lxd module

No way to create a new, stopped container

We have no way to create a new container that isn't immediately put in the running state.

Call to execute() is slow

Using pylxd 2.1, it seems the execute() call is quite slow and takes at least 5 seconds to finish. Not sure if its due to ws4py or something else.

Migration fails

DBUG[07-24|19:06:46] ContainerCreate driver=storage/zfs name=chuck isPrivileged=false
DBUG[07-24|19:06:46] ContainerStart driver=storage/zfs container=chuck
DBUG[07-24|19:06:46] ContainerStop driver=storage/zfs container=chuck
EROR[07-24|19:06:46] Error during migration sink err="lookup http+unix: invalid domain name"
DBUG[07-24|19:06:46] containerDeleteSnapshots container=chuck
DBUG[07-24|19:06:46] ContainerDelete driver=storage/zfs container=chuck
DBUG[07-24|19:06:47] Failure for task operation: 4b69f75b-e63c-4626-82b7-7c4573f68d63: Error transferring container data: lookup http+unix: invalid domain name
INFO[07-24|19:09:30] handling method=GET url=/1.0 ip=@
DBUG[07-24|19:09:30]

Contributing points to nonexisting repo

https://github.com/lxc/pylxd/blob/master/CONTRIBUTING.rst points to https://bugs.launchpad.net/pylxd as the master repo and where the patches should go but the repo is nonexisting.

Profile.update() broken

Note: I haven't really used github before so let me know if I'm doing something wrong.

Profile.update() is broken due to the 'config' slot not being filled out by the profile.all() method, which causes getattr() in the 'Marshallable' mixin to break. Using git clone of master from [05/27/16] on Ubuntu 16.04 x64.

Sample code:

from pylxd.client import Client

API = Client()

for x in API.profiles.all():
    print(">> Name: " + x.name)
    x.update()

Results:

(env) pjones@ThinkPad:~/Development/Node-Z/src$ python nzhd.py
>> Name: default
>> NAME: _client
>> NAME: config
Traceback (most recent call last):
  File "nzhd.py", line 11, in <module>
    x.update()
  File "/home/pjones/Development/Node-Z/env/lib/python3.5/site-packages/pylxd/profile.py", line 68, in update
    marshalled = self.marshall()
  File "/home/pjones/Development/Node-Z/env/lib/python3.5/site-packages/pylxd/mixin.py", line 37, in marshall
    marshalled[name] = getattr(self, name)
AttributeError: config

Strangely enough I don't seem to have a profile called '_client' or 'config'.

(env) pjones@ThinkPad:~/Development/Node-Z/src$ lxc profile list
default
docker
Jail
nz-01

And when I modify the code to remove the 'x.update()' line, the profile list is accurate:

from pylxd.client import Client

API = Client()

for x in API.profiles.all():
    print(">> Name: " + x.name)

(env) pjones@ThinkPad:~/Development/Node-Z/src$ python nzhd.py
>> Name: default
>> Name: docker
>> Name: Jail
>> Name: nz-01

test_images failing due to dir misconstruction

# itxaka at zeus in ~/projects/pylxd on git:master x [15:20:21]
$ python -V
Python 2.7.9
(.venv)
# itxaka at zeus in ~/projects/pylxd on git:master x [15:20:23]
$ which python
/home/itxaka/projects/pylxd/.venv/bin/python
(.venv)
# itxaka at zeus in ~/projects/pylxd on git:master x [15:20:27]
$ pwd
/home/itxaka/projects/pylxd

Running tox from the root dir
Some of the tests wil fail because they are trying to get the worng local image:
IOError: [Errno 2] No such file or directory: '/home/itxaka/projects/images/lxd.tar.xz'

Probably due to this 3 lines at https://github.com/lxc/pylxd/blob/master/pylxd/tests/test_image.py#L22-L24

TESTDIR = os.path.dirname(os.path.abspath(__file__))
ROOTDIR = os.path.normpath(os.path.join(TESTDIR, '..', '..', '..'))
IMAGE = os.path.join(ROOTDIR, 'images', 'lxd.tar.xz')

Not sure why this is there. On the repo, the images dir is under the test dir, so the first TESTDIR should already contain the proper path to the test dir.

Busybox image question

Hi !

This is what the metadata.yaml generated by busybox.py looks like:

{
    "architecture": "x86_64",
    "creation_date": 1463647974,
    "properties": {
        "architecture": "x86_64",
        "description": "Busybox x86_64",
        "name": "busybox-x86_64",
        "obfuscate": "<function uuid4 at 0x7f7e75833488>",
        "os": "Busybox"
    }
}

Is that the right value for obfuscate ?

Some container methods on master are incorrect

freeze, unfreeze and restart all point to the stop API endpoint

AttributeError: pre_run in py26, py27 and py34 jobs

When runnning tox tests if pbr==1.0.0 is installed they wont run du to this bug:
https://bugs.launchpad.net/oslo-incubator/+bug/1456663

Upgrading pbr to 1.0.1 fixes this issue.

Improvement for container creation failure

Currently, a runtime exception is raised when lxd responds with a failure to create a container. In user code, this makes it impossible to route exception handling ie.:

try:
    client.containers.create(dict(name='foo'))
except ContainerCreationFailure:
    print "Something went creating the container"
except:
    print "Something unknown went wrong, but the container was created"

Also, the runtime error doesn't carry the response, making it impossible for user code to use the HTTP response, ie. to display it:

try:
    client.containers.create(dict(name='foo'))
except ContainerCreationError as e:
    print "Server did not create the container, details: ", e.response.json()

Please let me know your thought about this, thanks again for the amazing job you're doing hosting this sprint !

Cannot authenticate client against LXD

Hello, not sure if bug or intended:

Following this procedure I've tried to setup a simple client:

from pylxd import Client
c = Client(endpoint='http://192.168.33.10:8443', cert=('client.crt', 'client.key'), verify=False)

Theorically it should create a Client instance that later I can authenticate with the authenticate() method and the trust password (already set in LXD) but every time I get the exception:

ClientAuthenticationFailed: LXD client certificates are not trusted.

Seems that in pylxd==2.0.2 (currently the latest available in PyPI) the client raise an exception every time is not authenticated, so it's impossible to register new certificates.

I noted also that in the current code available on GitHub this should be fixed.

LXD 2.0.0beta3 : create container : unexpected non-sync response

Hi,

with pylxd 2.0.0b1 and lxd 2.0.0beta3:

Create request:

{
    "architecture": 2,
    "config": {
        "boot.autostart": "true",
        "limits.cpu": "1-2",
        "limits.memory": "256MB",
        "security.privileged": "true"
    },
    "devices": {
        "tun": {
            "path": "/dev/net/tun",
            "type": "unix-char"
        }
    },
    "name": "maestro-test-container-lxd4",
    "profiles": [
        "default"
    ],
    "source": {
        "alias": "ubuntu/trusty/amd64",
        "mode": "pull",
        "server": "https://images.linuxcontainers.org",
        "type": "image"
    }
}

Operation info

[
    200,
    {
        "metadata": {
            "class": "task",
            "created_at": "2016-02-19T14:06:52.772322197+01:00",
            "err": "unexpected non-sync response",
            "id": "7fdc8322-7d6c-4873-94fd-13c5252dd407",
            "may_cancel": false,
            "metadata": null,
            "resources": {
                "containers": [
                    "/1.0/containers/maestro-test-container-lxd4"
                ]
            },
            "status": "Failure",
            "status_code": 400,
            "updated_at": "2016-02-19T14:06:52.772322197+01:00"
        },
        "status": "Success",
        "status_code": 200,
        "type": "sync"
    }
]

On LXD with debug mode:

DBUG[02-19|14:13:19] Responding to container create 
DBUG[02-19|14:13:19] Raw response: {"type": "sync", "status": "Success", "status_code": 200, "metadata": {"target": "7c0b8de47a5b4353281b8224e9b3f092eb71de64d9014cb70114c408e6912b9d", "name": "ubuntu/trusty/amd64"}}

DBUG[02-19|14:13:19] New task operation: 4f9b578c-472e-426c-9214-ff395578ea52 
DBUG[02-19|14:13:19] Started task operation: 4f9b578c-472e-426c-9214-ff395578ea52 
DBUG[02-19|14:13:19] 
    {
        "type": "async",
        "status": "Operation created",
        "status_code": 100,
        "metadata": {
            "id": "4f9b578c-472e-426c-9214-ff395578ea52",
            "class": "task",
            "created_at": "2016-02-19T14:13:19.665477733+01:00",
            "updated_at": "2016-02-19T14:13:19.665477733+01:00",
            "status": "Running",
            "status_code": 103,
            "resources": {
                "containers": [
                    "/1.0/containers/maestro-test-container-lxd4"
                ]
            },
            "metadata": null,
            "may_cancel": false,
            "err": ""
        },
        "operation": "/1.0/operations/4f9b578c-472e-426c-9214-ff395578ea52"
    } 
INFO[02-19|14:13:19] Image not in the db, downloading it      image=ubuntu/trusty/amd64 server=https://images.linuxcontainers.org
INFO[02-19|14:13:19] Downloading the image                    image=ubuntu/trusty/amd64
INFO[02-19|14:13:19] handling                                 method=GET url="/1.0/operations/4f9b578c-472e-426c-9214-ff395578ea52/wait?status_code=200&timeout=240" ip=@
DBUG[02-19|14:13:20] Raw response: {
    "error": "Only full and short (12 chars) hashes are supported by this server",
    "error_code": 400,
    "type": "error"
}

EROR[02-19|14:13:20] Failed to download image metadata        image=ubuntu/trusty/amd64 err="unexpected non-sync response"
DBUG[02-19|14:13:20] Failure for task operation: 4f9b578c-472e-426c-9214-ff395578ea52: unexpected non-sync response 
DBUG[02-19|14:13:20] 
    {
        "type": "sync",
        "status": "Success",
        "status_code": 200,
        "metadata": {
            "id": "4f9b578c-472e-426c-9214-ff395578ea52",
            "class": "task",
            "created_at": "2016-02-19T14:13:19.665477733+01:00",
            "updated_at": "2016-02-19T14:13:19.665477733+01:00",
            "status": "Failure",
            "status_code": 400,
            "resources": {
                "containers": [
                    "/1.0/containers/maestro-test-container-lxd4"
                ]
            },
            "metadata": null,
            "may_cancel": false,
            "err": "unexpected non-sync response"
        }
    } 

No way to query container resources

No way to query Disk Usage, Memory Usage, network usage, etc.

Run integration tests on CI

Currently, master passes unit tests with a an lxd REST API mock, but the code is not completely compatible with actual LXD REST API, and CI isn't warning commiters. We have great integration tests, which we can run both on a local and remote lxd instance. They help greatly in finding bugs, ie. lxd http rest api has been updated upstream.

Can we have CI for integration tests ?

This would help finding bugs earlier in the software.

Thanks !

AttributeError: last_used_at on images.linuxcontainers.org

Doing the following:

import pylxd
c1 = pylxd.Client(endpoint='https://images.linuxcontainers.org:8443', cert=('/root/.config/lxc/client.crt', '/root/.config/lxc/client.key',), verify=False)
c1.images.get_by_alias('ubuntu/xenial/amd64').marshall()

results in

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 183, in marshall
    marshalled[key] = getattr(self, key)
  File "/usr/local/lib/python2.7/dist-packages/pylxd/model.py", line 113, in __getattribute__
    return super(Model, self).__getattribute__(name)
AttributeError: last_used_at

Integration tests require an existing image

Right now, the integration tests require a busybox image. That's not good. We've taken a few swings at fixing it, though the create_container class probably needs to just start using create_image and using the image created there, deleting it at the end of the test.

Add usage documentation

There's currently no useful usage documentation for anyone stumbling upon pylxd which means they usually revert back to using the LXD API via the requests module, also meaning we're missing out on potential contributors. Simple usage documentation would resolve this, at least enough for other contributors to build upon without having to dig through tons of code.

Edit: I see there's some generated docs in doc/source but it's not referenced from the README.

`Image.export` reads the entire image into memory

This is not good, particularly in cases where images might be large (I started noticing it working with container snapshots I had published that were >1GB).

The solution is that a file-like object should be returned. Unfortunately, this breaks backwards compatibility (something we should be trying hard to maintain).

ws4py should be optional

In most cases, no one needs the websocket functionality of pylxd. It should be optional.

Also, it makes backporting patches to xenial easier (ws4py is not in main).

Integration tests should not be run in production

Integration tests should make sure their environment is clean of existing images and containers before trying to run. We can't guarantee they won't delete everything, or manipulate true containers in a way that makes them lose data, etc.

Integration tests should bomb out if there's any pre-existing data.

Python 3.4: The strict parameter was removed.

Hi,

With pyhton 3.4

  File "/usr/local/lib/python3.4/dist-packages/pylxd/api.py", line 128, in container_list
    return self.container.container_list()
  File "/usr/local/lib/python3.4/dist-packages/pylxd/container.py", line 25, in container_list
    (state, data) = self.connection.get_object('GET', '/1.0/containers')
  File "/usr/local/lib/python3.4/dist-packages/pylxd/connection.py", line 172, in get_object
    response = self._request(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/pylxd/connection.py", line 153, in _request
    self.connection = self.get_connection()
  File "/usr/local/lib/python3.4/dist-packages/pylxd/connection.py", line 169, in get_connection
    return UnixHTTPConnection(self.unix_socket)
  File "/usr/local/lib/python3.4/dist-packages/pylxd/connection.py", line 54, in __init__
    timeout=timeout)
TypeError: __init__() got an unexpected keyword argument 'strict'

https://docs.python.org/3/library/http.client.html

Changed in version 3.4: The strict parameter was removed. HTTP 0.9-style “Simple Responses” are not longer supported.

https://github.com/lxc/pylxd/blob/master/pylxd/connection.py#L52-L54

Host documentation in HTML format

Currently, documentation is maintained with sphinx in RST format in the doc directory.

It would be great if we could host an HTML version.

Just in case, note that readthedocs.org provides a free service with a github hook to update the documentation when changes are merged. It also allows to switch versions allowing to read the docs for the current master or latest stable release or even older releases.

Test failures on Ubuntu 14.04

==============================
Failed 2 tests - output below:
==============================

pylxd.tests.test_connection.LXDInitConnectionTest.test_https_connection
-----------------------------------------------------------------------

Captured traceback:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "/��PKGBUILDDIR��/pylxd/tests/test_connection.py", line 48, in test_https_connection
        conn.connect()
      File "/��PKGBUILDDIR��/pylxd/connection.py", line 57, in connect
        ssl_version=ssl.PROTOCOL_TLSv1_2)
    AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'

pylxd.tests.test_connection.LXDInitConnectionTest.test_https_proxy_connection
-----------------------------------------------------------------------------
Captured traceback:
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/mock/mock.py", line 1305, in patched
        return func(*args, **keywargs)
      File "/��PKGBUILDDIR��/pylxd/tests/test_connection.py", line 66, in test_https_proxy_connection
        conn.connect()
      File "/��PKGBUILDDIR��/pylxd/connection.py", line 57, in connect
        ssl_version=ssl.PROTOCOL_TLSv1_2)
    AttributeError: 'module' object has no attribute 'PROTOCOL_TLSv1_2'

Issue with autodoc module on hosted documentation

Currently on the hosted documentation, python elements are not hyperlinked. For autodoc module to work, it needs to import the modules which are referenced. Could you update RTFD to install pylxd in the advanced settings please ? Then, references to python classes/methods/attrs/etc will link to the corresponding section in the api page.

Please provide Image.get_by_alias

Hi, the alias functionality is deprecated and is missing from the new Client.

examples/api_test.py needs fixing

In 5b7895d examples/api_test.py got pretty substantial changes, which turned it from a Python 2.x example to some kind of a py2/py3 combination; the shebang is set to #!/usr/bin/python, which refers to Python 2 (at least on 15.04 and earlier - granted, this might depend on the virtualenv usage) and, as far as I can tell, it seems that the 'strict' parameter from the http_client call in pylxd/connection.py has been removed in Python 3.4, so running the example on 15.04 fails with:

TypeError: __init__() got an unexpected keyword argument 'strict'

Note that I'm omitting the full output here.

This seem to be easy things to fix, I can try to clean this up a bit if needed.

package a python3 deb

It would be nice to have a py3 deb for lxd.

container operations and wait_operation not in sync

Container operations return an operation url that looks like /1.0/operations/c212109b-2690-4c4e-a3a0-80c7bf1bd3bc. When this value is passed to wait_operation, /1.0/operations/ is again prepended, resulting in a request of /1.0/operations/1.0/operations/c212109b-2690-4c4e-a3a0-80c7bf1bd3bc.

LXD supports Profile rename now

When pylxd 2.0 api was originally created, LXD's profile rename wasn't implemented yet. It is now, but pylxd still raises NotImplementedError.

Client.container.all() returns state payload instead of container

It's confusing for the all() method to return a list of containers filled with the state information.

To really get the container (including config and other info) you have to subsequently use:

container = Client.containers.get(container.name)

which results in a fuller container object.

The container object should always have the same attributes, this two-step move to get to the detail is confusing.

No way to export images

We can't currently export images stored in lxd.

Documentation lacking for accessing remote hosts

pylxd documentation doesn't say how to access remote hosts. Poking around, I find a number of issues.

1. The front page intro document says to use pylxd.api.API(), however in the code pylxd.api is imported from pylxd.deprecated. Should anyone even be using this in the first place??

   Looking at ansible lxd_container, they use pylxd.client.Client() instead. I haven't found any documentation for that entry point into pylxd. Indeed, if there is any documentation for pylxd, I've been unable to find it: only a skeleton at https://github.com/lxc/pylxd/tree/master/doc/source

2. From the source, pylxd.api.API() takes an optional host and port. I found that I can use pylxd.api.API(host="nuc1.ws.nsrc.org").container_list() and this works just fine. It doesn't matter which variation of the host's name I use.

3. From the source, pylxd.client.Client() takes an optional URL. However if I try this code:

   pylxd.client.Client('https://nuc1w.ws.nsrc.org:8443').containers.all()
   

   then I get:

   requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
   

   I have tried all the variations of the host name, and also connecting by IP address. If I connect using openssl s_client then I see the certificate has:

           Subject: O=linuxcontainers.org, [email protected]
   ...
               X509v3 Subject Alternative Name:
                   DNS:nuc1.ws.nsrc.org, DNS:fe80::c23f:d5ff:fe63:6411/64, DNS:10.10.0.238/24, DNS:10.10.0.237/32, DNS:2001:db8:100::238/64, DNS:fe80::c23f:d5ff:fe63:6411/64, DNS:fe80::fcd5:94ff:fe50:96ca/64, DNS:fe80::8461:ddff:fe6f:3aa8/64, DNS:fe80::1/64
   

   i.e. all hostname variations are in SAN (except I don't know why the certificate would include the prefix length, but then I've never seen certificates with IP addresses in them)

So anyway: for code which uses pylxd.client.Client() then I'd like to see an example of how to make it work without certificate errors. I'd also like it to be clear whether pylxd.api or pylxd.client is the recommended interface to use.