docker-archive / classicswarm Goto Github PK

Should the core swarmd code be an importable package the I can import in my applications?

i tried running libswarm using following command

$ swarmd 'dockerserver tcp://localhost:4243'  'dockerclient tcp://localhost:4243'

Then tried to connect to the running server using following

$ docker -H localhost:4243 ps

which throws follwing error

https://gist.github.com/ric03uec/999f82756285d6657991
Tried restarting the server(as suggested by @cpuguy83 on irc) but didn't help.

System info

$ uname -a
Linux crossbow 3.8.0-35-generic #50~precise1-Ubuntu SMP Wed Dec 4 17:25:51 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

$ docker version
Client version: 1.2.0
Client API version: 1.14
Go version (client): go1.3.1
Git commit (client): fa7b24f
OS/Arch (client): linux/amd64
Server version: 1.2.0
Server API version: 1.14
Go version (server): go1.3.1
Git commit (server): fa7b24f

$ docker -D info
Containers: 12
Images: 359
Storage Driver: aufs
 Root Dir: /data/docker/aufs
 Dirs: 385
Execution Driver: native-0.2
Kernel Version: 3.8.0-35-generic
Operating System: Ubuntu precise (12.04.4 LTS)
Debug mode (server): true
Debug mode (client): true
Fds: 61
Goroutines: 52
EventsListeners: 0
Init Path: /usr/bin/docker
Username: ric03uec
Registry: [https://index.docker.io/v1/]
WARNING: No swap limit support

Using @vieux's recommendation in #145, I'm attempting to use swarm manage --addr=<swarm listener> <dockerport1> <dockerport2>, but it always fails with:

ERRO[0000] node was already added to the cluster        

Tracking this error message down to here, I see it's using the Node's ID property to check if it's been added already. Tracing that to samalba/dockerclient/types.go, it looks like it populates the Node object by parsing the JSON response of issuing /info... which has no ID field. This leads me to believe it is a blank string, and therefore you can never have more than one trying to use swarm this way.

I am not a Go developer, so I'm probably missing things here, but figured I'd bring this up just in case.

Hi,

I'm trying to use swarm for the first time, and I followed the README but there is one error when I do "swarm create":

FATA[0000] Post /clusters: unsupported protocol scheme ""

uname -a :

Linux 3.14.1-031401-generic #201404141220 SMP Mon Apr 14 16:21:48 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

docker version:

Client version: 1.4.0
Client API version: 1.16
Go version (client): go1.3.3
Git commit (client): 4595d4f
OS/Arch (client): linux/amd64
Server version: 1.4.0
Server API version: 1.16
Go version (server): go1.3.3
Git commit (server): 4595d4f

docker -D info:

Containers: 10
Images: 87
Storage Driver: devicemapper
 Pool Name: docker-8:5-6161142-pool
 Pool Blocksize: 65.54 kB
 Data file: /var/lib/docker/devicemapper/devicemapper/data
 Metadata file: /var/lib/docker/devicemapper/devicemapper/metadata
 Data Space Used: 9.783 GB
 Data Space Total: 107.4 GB
 Metadata Space Used: 9.712 MB
 Metadata Space Total: 2.147 GB
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Kernel Version: 3.14.1-031401-generic
Operating System: Ubuntu 14.04.1 LTS
CPUs: 8
Total Memory: 7.731 GiB
Name: Totote
ID: LPHH:4727:NJ63:DSPC:USZI:WPMJ:NPGU:IW7D:66II:OHDS:2SPP:FACK
Debug mode (server): false
Debug mode (client): true
Fds: 11
Goroutines: 10
EventsListeners: 0
Init Path: /usr/bin/docker
Docker Root Dir: /var/lib/docker
WARNING: No swap limit support

Any idea ?

Thank you,

Emeline

Looks like TLS cert authentication is missing for both front and back end connections.

I presume this would be added with flags mirroring those of docker?

Trying to build swarm. Seems to fail on Dockerserver + Rax

[sg@whatup ~]# go get github.com/docker/libswarm/...
# github.com/docker/libswarm/backends
mygo/src/github.com/docker/libswarm/backends/dockerserver.go:429: cannot use "github.com/dotcloud/docker/api".APIVERSION (type "github.com/docker/docker/pkg/version".Version) as type "github.com/dotcloud/docker/pkg/version".Version in assignment
mygo/src/github.com/docker/libswarm/backends/rax.go:290: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:291: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:292: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:293: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:294: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:295: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:296: too few values in struct initializer
mygo/src/github.com/docker/libswarm/backends/rax.go:376: cannot use job (type *"github.com/dotcloud/docker/engine".Job) as type *"github.com/docker/docker/engine".Job in function argument
mygo/src/github.com/docker/libswarm/backends/rax.go:477: cannot use job (type *"github.com/dotcloud/docker/engine".Job) as type *"github.com/docker/docker/engine".Job in function argument
mygo/src/github.com/docker/libswarm/backends/rax.go:477: too many errors

Using Go version 1.2.2

and docker version ?

swarmd(initial-play) $ ./swarmd 'dockerserver unix://tmp/docker' 'dockerclient tcp://localhost:2375'

...

libswarm(initial-play) $ docker -H unix:///tmp/docker info
2014/06/22 22:47:41 Error: 404 page not found

When I try to install libswarm I get:

$ go install github.com/docker/libswarm/swarmd
# github.com/docker/libswarm/backends
go/src/github.com/docker/libswarm/backends/ec2.go:345: c.instance.IPAddress undefined (type *ec2.Instance has no field or method IPAddress)
go/src/github.com/docker/libswarm/backends/ec2.go:436: c.instance.IPAddress undefined (type *ec2.Instance has no field or method IPAddress)

I wrote the "Installation" section of the README a little while back and it might still work in some cases, but with the introduction of godeps etc. I'm noting here that things should be rewritten to cover the more "modern" way of building libswarm e.g. you have to run make deps

It looks like a key part of swarm is hosted on docker.com. The API to that service is looks to be closed source and undocumented. Is that code/spec available anywhere.

I've followed these steps:

• host:
  • /usr/bin/docker -d -p /var/run/docker.pid -H 0.0.0.0:2375
  • swarm create
• container:
  • swarm join --discovery token://**** --addr=172.17.0.2:2375

without error. But on

• host:
  • swarm manage --discovery token://**** -H=172.17.42.1:4243

I am seeing

• INFO[0000] Listening for HTTP on tcp (172.17.42.1:4243)
• ERRO[0000] Get http:///v1.15/info: http: no Host in request URL. Are you trying to connect to a TLS-enabled daemon without TLS?
• ERRO[0000] Get http://:9999/v1.15/info: dial tcp :9999: connection refused
• ERRO[0000] Get http://127.0.0.1:4243/v1.15/info: dial tcp 127.0.0.1:4243: connection refused
• ERRO[0000] Get http://localhost:4243/v1.15/info: dial tcp 127.0.0.1:4243: connection refused
• ERRO[0000] Get http://docker.hknochi.de:2376/v1.15/info: malformed HTTP response "\x15\x03\x01\x00\x02\x02". Are you trying to connect to a TLS-enabled daemon without TLS?

(As you can see, I'm running the docker daemon without "--tlsverify". This is a test launch.)

I guess this is because I am making bad choices for swarm_ip and swarm_port (and maybe node_ip and node_port also). Googling for "no Host in request URL" doesn't look pretty.

Can anyone tell me what I need to change?

Thanks

ec2 backend stucks at this message after that there is no activity

waiting for docker daemon on remote machine to be available.

Hi,

Running the following installation procedures, I am running on error:

Building my vagrant machine

~/dev/docker_swarm> vagrant init ubuntu/trusty64
~/dev/docker_swarm> wget https://master.dockerproject.com/linux/amd64/docker-1.3.2-dev
~/dev/docker_swarm> vagrant up
~/dev/docker_swarm> vagrant ssh

From inside the vagrant machine

vagrant@vagrant-ubuntu-trusty-64:$ sudo apt-get install golang-go git -y && sudo -E go get github.com/docker/swarm && sudo -E go install github.com/docker/swarm
vagrant@vagrant-ubuntu-trusty-64:$ sudo /vagrant/docker-1.3.2-dev -H 0.0.0.0:2375 -d &
vagrant@vagrant-ubuntu-trusty-64:$ swarm create
6f1104155948d4163cae6d316502af55
vagrant@vagrant-ubuntu-trusty-64:$ swarm join --token=6f1104155948d4163cae6d316502af55 --addr=127.0.0.1:4243
vagrant@vagrant-ubuntu-trusty-64:~$ swarm manage --token=6f1104155948d4163cae6d316502af55

And now the error:

INFO[0008] GET /v1.15/info
ERRO[0008] Node http://127.0.0.1:4243 is running an unsupported version of Docker Engine. Please upgrade.

I have been thinking about cluster configuration lately.

My question is; does it make sense to keep some mutable state inside a container? If so, does it make sense to design a standard interface to state controls?

Example:

If we look at containers as lego bricks. You can either build a house solely out of stateless bricks. Whenever you want to switch on the lights, you swap out the dark lightbulb bricks with glowing lightbulb bricks. On the other hand, those lightbulb bricks might expose a switch that lets you turn them on and off.

What does the community think? Just wondering 🐱

The repository doesn't have a travis build.
We should add one.
What are the steps to build and test libswarm?

I searched the code but I couldn't find any references to the links feature. What is swarm's networking model? This should be clearly defined to be able to build a distributed system.

Perhaps this is just a documentation bug.

When using swarm with TLS I get the error: " x509: certificate signed by unknown authority " when using "docker start" and it also happens with "docker logs".

The problem is that proxy function in utils.go doesn't use the tlsConfig.

This is what is missing is to configure the tls transport in the proxy function:

httpTransport := &http.Transport{
TLSClientConfig: tlsConfig,
}
client := &http.Client{Transport: httpTransport}

After making these changes "docker start" worked correctly.

here is full-list of changes I made to get it work:

api/api.go

first I needed to relay cluster variable to proxy function

    func proxyContainer(c *context, w http.ResponseWriter, r *http.Request) {
        container, err := getContainerFromVars(c, mux.Vars(r))
        if err != nil {
                httpError(w, err.Error(), http.StatusNotFound)
                return
        }

        if err := proxy(c.cluster, container, w, r); err != nil {
                httpError(w, err.Error(), http.StatusInternalServerError)
        }
    }

api/utils.go

from the cluster variable I read the TlsConfig

 func proxy(c *cluster.Cluster, container *cluster.Container, w http.ResponseWriter, r *http.Request)   error {

    httpTransport := &http.Transport{
            TLSClientConfig: c.TlsConfig,
    }

    // Use a new client for each request
    client := &http.Client{Transport: httpTransport}

    // RequestURI may not be sent to client
    r.RequestURI = ""

    parts := strings.SplitN(container.Node().Addr, "://", 2)
    if len(parts) == 2 {
            r.URL.Scheme = parts[0]
            r.URL.Host = parts[1]
    } else {
            r.URL.Scheme = "http"
            r.URL.Host = parts[0]
    }

    log.Debugf("[PROXY] --> %s %s", r.Method, r.URL)
    resp, err := client.Do(r)
    if err != nil {
            return err
    }
    w.WriteHeader(resp.StatusCode)
    io.Copy(w, resp.Body)

    return nil
}

cluster/cluster.go

TlsConfig needed to be public for proxy function to be able to access it.

    type Cluster struct {
        sync.RWMutex
        TlsConfig     *tls.Config
        eventHandlers []EventHandler
        nodes         map[string]*Node
    }

....

  if err := n.Connect(c.TlsConfig); err != nil {

The Watch method of the discovery backend calls cluster.UpdateNodes method to update the node list according the return value of the Fetch method of the discovery service.
But it only inserts the nodes that aren't exist in the node list and doesn't touch the nodes that are "removed" from the discovery service.

If a node is removed from the discovery service, the swarm list command returns the correct result because it calls the Fetch method of the discovery service directly. But in fact the removed node is still in the node list of the cluster.

Hi, i have 3 machines running docker and docker-swarm, 2 with ubuntu server 14.04 and 1 with ubuntu desktop 14.04. The docker and swarm have the same version in all machines, docker 1.4.1 and docker-swarm 0.0.1.

I create a cluster with "swarm create" and start swarm join in 2 machines running ubuntu server, all without errors. When i start swarm manage in ubuntu desktop it's running ok, but when i try execute any command(ps, run, info...) i receive this error message:

"ERRO[0014] unable to find a node with port available"

Using the docker client i can access all machines from your ip:port and running commands directly on docker client works fine too. Someone know what is happening?

Is there any particular reason that DISCOVERY_URL is not configurable? It would be nice if it was configurable similar to DOCKER_HOST. Would you accept a pull request for this?

Currently many endpoints have a notImplementedHandler with a message that the endpoint is not supported in clustering mode. Found this out trying to do a swarmed image pull, which would have been much nicer than pull in a loop.

Is this a case of "not implemented yet" or "swarm will not implement these APIs"?

If the latter, is the intended way to (for example) update images across the cluster to manually hit all the daemons?

$ go version
go version go1.3 darwin/amd64

$ uname -a
Darwin alis-macbook-air.fritz.box 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64

$ go get github.com/docker/libswarm
package github.com/docker/libchan/data
    imports github.com/docker/libchan/data
    imports github.com/docker/libchan/data: cannot find package "github.com/docker/libchan/data" in any of:
    /usr/local/Cellar/go/1.3/libexec/src/pkg/github.com/docker/libchan/data (from $GOROOT)
    /Users/ali/Work/golang/src/github.com/docker/libchan/data (from $GOPATH)

Looks like these changes are covered by moby/swarmkit#115 -- or should I be doing something different to pull in an old version of libchan?

In order to consume the swarm manage API using javascript, we need to enable CORS. It looks like it's already there but api.go has a hard coded false for enableCors on line 360. Could this be made into a flag?

On Ubuntu 14.04 immediately after installing:

root@docker-swarm-1:~# swarm create
unexpected fault address 0x7f2800000011
fatal error: fault
[signal 0xb code=0x1 addr=0x7f2800000011]

goroutine 3 [running]:

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

    :0

goroutine 1 [select]:
github.com_docker_swarm_discovery_token.CreateCluster.pN61_github.com_docker_swarm_discovery_token.TokenDiscoveryService
    /root/go/src/github.com/docker/swarm/discovery/token/token.go:85
main.$nested1
    /root/go/src/github.com/docker/swarm/main.go:47
github.com_codegangsta_cli.Run.N34_github.com_codegangsta_cli.Command
    /root/go/src/github.com/codegangsta/cli/command.go:101
github.com_codegangsta_cli.Run.pN30_github.com_codegangsta_cli.App
    /root/go/src/github.com/codegangsta/cli/app.go:154
main.main
    /root/go/src/github.com/docker/swarm/main.go:98

in some instances its just a "Segmentation fault (core dumped)"

I spun up a new VM on Digital Ocean via applications > docker 1.4.0, ssh'd in and ran this before hand so its a clean install, plenty of cpu and ram:

apt-get install gccgo-go
export GOPATH=/root/go/
go get -u github.com/docker/swarm
cp /root/go/bin/swarm /usr/bin/

The strangest thing about this is that if I give it some time it doesn't happen any more. I gave it a few minutes and the command successfully executed.

Executing the go get -u github.com/docker/swarm command again causes the problem to occur again for some small but significant amount of time. Using either the new binary or the one in /usr/bin which isn't too surprising.

It could be cool to have a cloudinit example in the readme, as that would help to understand on how to use Swarm.

I'm trying to aggregate backends on a cluster of coreos VMs. If I run swarmd like this, it works as expected:

./swarmd 'dockerserver tcp://0.0.0.0:3333' 'dockerclient tcp://node-1:2375'

However, if I try to aggregate instead, running docker -H tcp://0.0.0.0:3333 ps just hangs and never returns:

./swarmd 'dockerserver tcp://0.0.0.0:3333' "aggregate 'dockerclient tcp://node-1:2375' 'dockerclient tcp://node-2:2375' 'dockerclient tcp://node-3:2375' 'dockerclient tcp://node-4:2375' 'dockerclient tcp://node:2375'"

Interestingly enough, if I use the simulator, things work as expected as well. If I try to aggregate two simulators, I see the same exact hang.

This is on master of libswarm. Here's the other debug output listed in the contributing docs:

bmorton@node-1 ~/swarmd $ uname -a
Linux node-1 3.17.2+ #2 SMP Sat Nov 8 00:22:33 UTC 2014 x86_64 Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz GenuineIntel GNU/Linux
bmorton@node-1 ~/swarmd $ docker version
Client version: 1.3.0
Client API version: 1.15
Go version (client): go1.3.2
Git commit (client): c78088f
OS/Arch (client): linux/amd64
Server version: 1.3.0
Server API version: 1.15
Go version (server): go1.3.2
Git commit (server): c78088f
bmorton@node-1 ~/swarmd $ docker -D info
Containers: 42
Images: 114
Storage Driver: btrfs
Execution Driver: native-0.2
Kernel Version: 3.17.2+
Operating System: CoreOS 494.0.0
Debug mode (server): false
Debug mode (client): true
Fds: 41
Goroutines: 55
EventsListeners: 0
Init SHA1: 25a6fdb579aa99537fd0d9488c553700693e73df
Init Path: /usr/libexec/docker/dockerinit

Do we have a roadmap?

If I can see that I will have:

1. A better understanding of where to focus my time.
2. Confidence that the work Im doing in meaningful and will have a greater chance of being accepted.

From what I can tell so far the discovery seems to be based on a similar process to etcd bootstrapping in CoreOS. (The swarm discovery looks like it might even be using etcd as its backend?)

I've had a lot of problems with that in the past when experimenting with CoreOS and would therefore like to have some way to statically define clusters, so that I can template that configuration myself.

The other day I sat down with @aanand and learned some core libswarm stuff that I couldn't figure out on my own. It would be very helpful to have some documentation that covered what is the purpose of verbs, etc., and what does the overlying architecture of the system look like. I'm interested in giving this a go, just filing an issue so it's kept track of somewhere other than in my head.

Additionally, if #121 gets merged in, I would be happy to write up a detail section on how to use aggregate backend to do the "connect to dockerserver, run docker ps from one host that is actually multiple hosts" thing.

╰─> ./build/bin/swarmd 'dockerserver' 'dockerclient tcp://x.x.x.x:4243'
copying task output from &beam.receiverWrapper{Receiver:(*libchan.PipeReceiver)(0x12a00e78)} to &beam.senderWrapper{Sender:(*libchan.PipeSender)(0x12aa3198)}
Starting Docker server...
Calling POST /containers/create
Error: expected status code 201, got 404:
No such image: busybox (tag: latest)

╰─> DOCKER_HOST="tcp://localhost:4243" docker run -d -t busybox
2014/06/25 12:24:27 EOF

Once the image is on the machine (manually run the container on the host first), docker run work as expected.

It seems like some of this code needs to be in the libswarm dockerclient too:

https://github.com/dotcloud/docker/blob/master/api/client/commands.go#L1933-L1969

• Does swarm use a Docker plugin system? Or does it wrap Docker?
• Will swarm itself have a plugin system, such as networking (e.g. links across hosts) or storage?

Every time I run swarmd it prints the following line to standard output:

2014/06/17 17:22:12 Couldn't automatically load credentials

It seems to come from the Tutum backend, specifically from a library pulled in as dependency by that backend. @fermayo could you fix it to avoid the pollution please?

Thanks.

see moby/moby#6962

/cc @SvenDowideit

/containers/json prefixes Name with the node name, but /containers/:id/json doesn't, so currently it looks like there is no way to know with/containers/:id/json which node you are running on.

This case is a trivial fix, but according to TODOs in api.go the whole Name thing is unresolved anyway. So, is there an official word on the thinking behind how host info will ultimately be reported?

Just having a consistent Name prefixing everywhere seems sensible to me -- but that's not what's currently in the code.

Hi,

I am trying to create "delete" operation for docker in libswarm. I made following changes:

1. add new verb to verbs.go
2. add new dockerserver handler (dockerserver.go)
3. add new method in client.go
4. Added handler in my backend service

Still, i am not able to get this new verb operational.

How easy or hard would it be to add a strategy where

1. Selection favors data locality on host (image cache)
2. Selection favors least weighted host (each task .. building vs running, assigns different weights to a host)

Here is something we wrote internally.
https://github.com/CodeNow/mavis

We don't have any Go programmers but will try to contribute to this

#148/#160 work fine for the frontend connection, but this change seems to assume all of the TLS backend servers use the same TLS config as the swarm manage listener... This is not always the desired behavior, especially with heterogeneous backends that don't all use the same certificate set (in which case you basically can't use swarm?).

There seems to be no way to override this behavior; correct me if I'm wrong.

This problem is solved if there was a way specify the TLS key/cert/ca for docker daemons on a per-server basis, but it's not completely straightforward how that would work with the various discovery APIs...

It would be very useful for my use case to run the swarm manage server on a unix socket, without binding any ports at all (e.g. the case of running swarm in a networkless container).

This also would also bring swarm closer to parity with the docker daemon functionality.

I couldn't figure out how to use a TLS connection with the dockerclient backend, although some TLS stuff is mentioned in the source code. Is it possible?

It would be nice to have an adapter which exposes a tree of objects over HTTP, in a rest-ish interface which can easily be navigated by eg. curl.

POST /containers/create in particular seems to return 200 OK, but the correct code is 201 Created (https://docs.docker.com/reference/api/docker_remote_api_v1.16/#create-a-container).

From a quick look this seems like a broader problem since none of the API status codes are being proxied at all.

This breaks drivers like dockerode that expect the API spec to be followed.

See postContainersCreate(), https://github.com/docker/swarm/blob/master/api/api.go#L139:

if config.AttachStdout || config.AttachStdin || config.AttachStderr {
    http.Error(w, "Attach is not supported in clustering mode, use -d.", http.StatusInternalServerError)
    return
}

This is preventing swarm from creating attachable containers, period. I know swarm doesn't support the attach endpoint, but I don't think erroring out on the create config is the correct place to fail: It's reasonable to create an attachable container via swarm and attach to it later some other way. At least, it's reasonable enough for my tests to hit this!

I realize this is probably just a hack to dump a clean early error message for the CLI, but it's disruptive to API clients with different flows.

It's probably a bit to late now, but it should have been named "Shoal" which is a large number of fish swimming together :-)

The addressing of servers is rather inconsistent, especially between docker & swarm.

• The Docker documentation says to use the format tcp://1.2.3.4:2375 (here)
• The swarm documentation says to use the format 1.2.3.4:2375 (here).
• The swarm list and docker info commands output the format `http://1.2.3.4:2375
• The docker client does not accept http://protocol prefixes, only tcp://.
• The format tcp://1.2.3.4:2375 (and http://...) is not a host (is named as such in $SWARM_HOST and $DOCKER_HOST). The format 1.2.3.4:2375 is not a host either. A host, or hostname, does not include protocol or port.
• The swarm join command uses --addr for specifying the IP & PORT
• The swarm manage command uses --host for specifying the IP & PORT

As you can see, the naming is all over the place (and this is just from a brief look as well). Half these are URLs & URIs, the other half might be considered addresses, and none of them are hosts, but most are labeled as such.
While I personally would prefer we not call things 'hosts' that aren't hosts, it's a little late to fix this naming issue. However I do think all this needs to be made consistent. The documentation can be fixed at any time, and the swarm code can be fixed before it is labeled as 'stable'.

When creating and managing multiple machines, it is confusing to identify which machine you have logged into after using machine ssh. Setting the hostname to match the machine name would be useful in this scenario.

• What messages and verbs are
• How to write a service