digitalocean / go-qemu Goto Github PK

godoc.org shows some irrelevant example code because example_test.go is treated as a "whole file example". Fix this.

This way, the build will fail if the copyright header is missing.

Create a guide to contributing to this project. Explain the code quality standards that are expected in a submission, adding a person's name to AUTHORS, etc.

hypervisor.Hypervisor will become a concrete type which uses a hypervisor.Driver to communicate with its monitors, similar to database/sql.DB.

This will reduce code duplication and allow easily swapping new drivers in.

The libvirt-go cgo bindings will likely provide the best stability and error handling capabilities for interacting with libvirt, so we should add the ability to use them. They must be guarded by a build tag.

If built without the tag, all cgo operations will return error stubs.

Hello,

I was testing out these binding in a environment running libvirtd 1.3.1 and QEMU 2.5.0 and observed the following behavior:

vagrant@localhost:/kvmhost$ ./main --domainName smoketestvm2

Connecting to unix:///var/run/libvirt/libvirt-sock
2016/10/31 13:50:28 failed to create domain object: xdr:DecodeUint: EOF while decoding 4 bytes - read: '[]'
vagrant@localhost:/kvmhost$ virsh list
 Id    Name                           State
----------------------------------------------------
 4     smoketestvm2                   running

Has anything changed significantly between libvirtd 1.2.{1,2} and/or QEMU versions that has broken the bindings?

Would appreciate any insight into this.

Need an example or two of how to use the hypervisor.Hypervisor type with a hypervisor.Driver.

Many of the QMP JSON structures are messy and confusing. We can simplify them for our end users and then implement json.{Unm,M}arshaler interfaces to convert between the raw QMP format and our simplified one.

Same idea as #29 , but for package hypervisor. Should be implemented once #29 is complete.

Hey @robertojrojas , my apologies, I forgot to ask you to add your name and email address to AUTHORS. Would you mind doing that?

Thanks again for your contribution.

Gocyclo calculates cyclomatic complexities of functions in Go source code. The cyclomatic complexity of a function is calculated according to the following rules: 1 is the base complexity of a function +1 for each 'if', 'for', 'case', '&&' or '||'

go-qemu/mock_test.go
Line 53: warning: cyclomatic complexity 18 of function (mockMonitor).Run() is high (> 15) (gocyclo)

golint has a flag that causes it to exit with non-zero status, rendering our CI build script for this purpose obsolete.

Replace with golint -set_exit_status ./... in CI.

This is an easy Hacktoberfest contribution, so I'll leave this unfixed for a few days. Feel free to submit a PR to fix this and work toward that t-shirt!

It would be great to be able to verify the last git commit author's email against the emails in AUTHORS, to ensure the two are kept in sync.

For my own uses, I'm in the middle of writing a go generate codegen to transform the QMP API spec (qapi-schema.json) into reasonably idiomatic Go code. Would you be interested in a contribution of this autogen and resulting API?

Depending on how idiomatic I'm able to make it (the API is pretty rough in places), we could make it a subpackage and allow the main qmp library to use either higher level types that wrap the raw ones, or the raw ones directly if the user needs to use a feature that hasn't been exposed. My aspirational goal is to make the autogenerated bindings good enough that this isn't necessary, but of course that's hard :).

Attempt to communicate using libvirt's RPC interface when libvirt is used with qemu.

Add a notice that this should be considered pre-production software with an unstable API.

Show how to perform common actions with these packages.

This protocol needs much more explanation than what's currently available on the web.

Hey @danderson , we forgot (again) to ask you to add your name and email to AUTHORS. Mind sending us a PR?

Thanks!

Some older versions of QEMU or libvirt may not support the asynchronous events API. Gracefully handle this by returning an error from qemu.Domain.Events.

Hi,

Thanks for the great library. Btw, is there any examples of how to start a new machine? Is starting new machine is the same API with NewDomain? What about image configuration, etc?

Thank you

Virsh output:

[zsh|matt@nerr-2]:~ 0 % virsh list
 Id    Name                           State
----------------------------------------------------

[zsh|matt@nerr-2]:~ 0 % virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0

Test program: https://gist.github.com/mdlayher/30e4b344f46a5627ae923bb2fc49248d.
Program output: https://gist.github.com/mdlayher/bdcc40bd09f16ea3d4c4d8e53634cf5d

Things that implement Monitor should probably have a *Monitor suffix.

The Command interface is a little clunky and could be better by splitting out the Prepare method into a Preparer interface.

It got us started on this project, but at this point, I think that the RPC monitor is stable enough to totally replace the shell monitor when using libvirt.

Thoughts @digitalocean/go-qemu ?

• libvirtrpc
• socket

It's possible to speak QMP directly with QEMU when it isn't launched by libvirt. In addition, asynchronous events arrive automatically when the socket is being used for communication.

[zsh|matt@servnerr-2]:~ 1 % sudo socat - UNIX-CONNECT:/var/lib/libvirt/qemu/ubuntuvm0.monitor
{"QMP": {"version": {"qemu": {"micro": 0, "minor": 0, "major": 2}, "package": " (Debian 2.0.0+dfsg-2ubuntu1.22)"}, "capabilities": []}}
{"execute":"qmp_capabilities"}
{"return": {}}
// SIGINT sent in another terminal
{"timestamp": {"seconds": 1461880188, "microseconds": 107251}, "event": "SHUTDOWN"}
{"timestamp": {"seconds": 1461880188, "microseconds": 107550}, "event": "DEVICE_TRAY_MOVED", "data": {"device": "drive-ide0-1-0", "tray-open": true}}

Now that the majority of libvirtrpc has been moved into go-libvirt, let's remove this package and meld its types into the relevant go-qemu packages.

[zsh|matt@nerr-2]:~/src/github.com/digitalocean/go-qemu 0 (master) ± go test ./...
ok      github.com/digitalocean/go-qemu 4.004s
ok      github.com/digitalocean/go-qemu/hypervisor      0.002s
ok      github.com/digitalocean/go-qemu/internal/shellexec      0.002s
ok      github.com/digitalocean/go-qemu/internal/virsh  0.014s
ok      github.com/digitalocean/go-qemu/qmp     0.002s

Let's do what we can to make qemu tests as fast as the other packages; or at least, sub 1-second if possible.

mock_monitor is too complex, we need to simplify testing. Something like:

func TestBlockStats(t *testing.T) {
    dom := testDomain(t, func(t *testing.T, cmd qmp.Cmd) interface{} {
        if qmp.Cmd != "query-blockstats" // error

        return BlockStats{
            Device: "ide0-0",
        }
    })

    if err := dom.BlockStats(); err != nil {
        //
    }
}

Test program:

package main

import (
    "log"
    "net"
    "time"

    "github.com/digitalocean/go-qemu"
    "github.com/digitalocean/go-qemu/qmp/libvirtrpc"
)

func main() {
    c, err := net.DialTimeout("unix", "/var/run/libvirt/libvirt-sock", 2*time.Second)
    if err != nil {
        log.Fatalf("failed to dial libvirt: %v", err)
    }

    mon := libvirtrpc.New("ubuntuvm0", c)
    if err := mon.Connect(); err != nil {
        log.Fatalf("failed to connect monitor: %v", err)
    }

    dom, err := qemu.NewDomain(mon, "ubuntuvm0")
    if err != nil {
        log.Fatalf("failed to create domain: %v", err)
    }

    _, err = dom.Version()
    if err != nil {
        log.Fatalf("failed to retrieve version: %v", err)
    }
}

Libvirt and QEMU versions:

[zsh|matt@servnerr-2]:~ 0 % virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0

Error output:

2016/05/03 16:44:55 failed to retrieve version: invalid character '\x00' after top-level value

Test program:

package main

import (
    "log"
    "net"
    "time"

    "github.com/digitalocean/go-qemu"
    "github.com/digitalocean/go-qemu/qmp/libvirtrpc"
)

func main() {
    c, err := net.DialTimeout("unix", "/var/run/libvirt/libvirt-sock", 2*time.Second)
    if err != nil {
        log.Fatalf("failed to dial libvirt: %v", err)
    }

    mon := libvirtrpc.New("ubuntuvm0", c)
    if err := mon.Connect(); err != nil {
        log.Fatalf("failed to connect monitor: %v", err)
    }

    dom, err := qemu.NewDomain(mon, "ubuntuvm0")
    if err != nil {
        log.Fatalf("failed to create domain: %v", err)
    }

    _, err = dom.Version()
    if err != nil {
        log.Fatalf("failed to retrieve version: %v", err)
    }
}

Libvirt and QEMU versions:

[zsh|matt@servnerr-2]:~ 0 % virsh version
Compiled against library: libvirt 1.2.2
Using library: libvirt 1.2.2
Using API: QEMU 1.2.2
Running hypervisor: QEMU 2.0.0

Error output:

2016/05/03 16:33:58 failed to retrieve version: 
7Requested operation is not valid: domain is

qemu.Domain is the type which enables high-level interactions with QEMU. Adding functionality to it generally requires the following steps:

• issue a qmp.Cmd with appropriate parameters
• unmarshal response JSON into anonymous struct
• process anonymous struct into useful data
• return to caller

As an example, see the source for https://godoc.org/github.com/digitalocean/go-qemu#Domain.Version.

Over time, we will ideally be able to implement methods for each supported QMP command from this document: https://github.com/qemu/qemu/blob/master/qmp-commands.hx.

This will be a long-standing issue and additional issues should be created for each new method to be added. Contributions are very welcome!

=== RUN   TestEvents
--- FAIL: TestEvents (1.13s)
    rpc_test.go:472: expected event, received timeout
    rpc_test.go:477: expected device "drive-ide0-0-0", got %!q(<nil>)

Can be reproduced in a shell:

while true; do go test -v -run=TestEvents github.com/digitalocean/go-qemu/qmp/libvirtrpc ; done

Work in progress here: https://github.com/digitalocean/go-qemu/tree/screendump.

Add binaries which demonstrate usage of these packages, and make them easy to build and run to figure out how everything fits together.

This package could be useful in more than just our own tests.

@benlemasurier should have a patch for this issue shortly.

If go generate ./... in CI produces a dirty repository, the build should fail.

It doesn't belong in package qmp, and was just a temporary measure to enable access from package hypervisor.

In order to avoid exposing too many options for qmp.Monitor types that interact with libvirt, we may be able to mask them all behind a new parameter in the qmp.NewLibvirtMonitor constructor.

Currently, the constructor looks like:

func NewLibvirtMonitor(uri, domain string) (*Libvirt, error)

The proposed constructor could be something like:

func NewLibvirtMonitor(driver LibvirtDriver, uri, domain string) (*Libvirt, error) {

Where driver could be one of the following:

type LibvirtDriver int

const (
    LibvirtDriverShell LibvirtDriver = iota
    LibvirtDriverRPC   LibvirtDriver
    LibvirtDriverCgo   LibvirtDriver
)

The shell driver is already implemented, and the native RPC driver is well underway. The cgo driver could be added in a future PR, and guarded with build tags to make it a no-op unless compiled with the correct tags.

For a long time, I didn't like the pattern I mentioned in the title.

Now, I find it pretty aggravating when repo and package names differ, because it's a surprise that you won't find out unless you explicitly check the code or docs.

Moving package qemu to a "subpackage" would also help clarify that this repository is a collection of packages for interacting with QEMU.

On the other hand, right now, it's front and center, and the first thing users see. That's arguably something we want to preserve as well.

Thoughts?