binocarlos / powerstrip-weave Goto Github PK

A Powerstrip plugin that runs weave inside a container and ensures that containers are connected to the weave network before running their entrypoints.

Makefile 0.39% JavaScript 73.76% Shell 25.10% Ruby 0.75%

powerstrip-weave's People

Contributors

Stargazers

Watchers

Forkers

laynepeng psfales abdul

powerstrip-weave's Issues

Support for "weave attach"

We have 3 different subnets in our environment. dmz, app and database. We were able to create the environment with a sequence of "weave run" and "weave attach" commands. It will be ideal, if "weave attach" commands can be avoided by enhancing "weave run" to support multiple CIDRS. But, I am filing this issue, because it was suggested on weavenetwork irc channel that it can be solved in powerstrip-weave.

@rade also created an issue on weave.
weaveworks/weave#467

Reduce size of container image

At the minute the Dockerfile bases its image on node:0.10, which is notionally reasonable since that's what it needs. However, that base image contains a huge amount of extra guff that's not needed at runtime (it's based itself on debian), and ends up being 700-odd MB.

There are node:*-slim images that are a bit smaller, but I think better still would be to use a container to get the dependencies, then copy those and the runtime out into another container. @errordeveloper has done this elsewhere for IO.js (https://registry.hub.docker.com/u/errordeveloper/iojs-minimal-runtime/)

Problem getting powerstrip container to run

My woefully bad (i.e. non-existent) knowledge of python is preventing me from working out why I can't get the powerstrip container to run.

I am building the container using this command from the master branch of powerstrip:

$ docker build -t clusterhq/powerstrip .

At first the error was saying that the yaml and treq packages could not be found so I added them to the Dockerfile as follows:

RUN         pip install service_identity pycrypto pyyaml treq && \

However - now I'm getting this error:

Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/app.py", line 642, in run
    runApp(config)
  File "/usr/local/lib/python2.7/dist-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/app.py", line 376, in run
    self.application = self.createOrGetApplication()
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/app.py", line 441, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/app.py", line 452, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/usr/local/lib/python2.7/dist-packages/twisted/application/service.py", line 405, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
  File "/usr/local/lib/python2.7/dist-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile
    exec fileObj in d, d
  File "powerstrip.tac", line 11, in <module>
    if "://" not in DOCKER_HOST:
exceptions.TypeError: argument of type 'NoneType' is not iterable

Failed to load application: argument of type 'NoneType' is not iterable

I'm am most probably doing it wrong but am a bit stuck so was wandering if you could teach me to suck python eggs (to coin a phrase) and tell me what is going wrong here.

Handling of content type

After setting it all up according to the blog post, I ran this:

CID=$(docker run -e "WEAVE_CIDR=10.255.0.51/8" -d busybox)

and got:

server listening on port: 80

undefined:1
No such image: busybox
^
SyntaxError: Unexpected token N
    at Object.parse (native)
    at /srv/app/actions/create.js:47:26
    at /srv/app/dockerclient.js:15:7
    at ConcatStream.<anonymous> (/srv/app/node_modules/concat-stream/index.js:32:43)
    at ConcatStream.emit (events.js:117:20)
    at finishMaybe (/srv/app/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:460:14)
    at afterWrite (/srv/app/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:340:3)
    at /srv/app/node_modules/concat-stream/node_modules/readable-stream/lib/_stream_writable.js:327:9
    at process._tickCallback (node.js:442:13)

Faster weave command access :)

Use this shell alias and you can use weave commands faster...

_weave() {
  docker exec -ti powerstrip-weave /srv/app/run.sh weave $@
}
alias weave _weave

usage

$ weave status
$ weave expose 10.255.0.1/8

/usr/bin/docker requires libdevmapper.so on CoreOS and other versions of Linux

There is an additional issue, on a number of distros /usr/bin/docker requires libdevmapper.so, as the storage backends differ.

Resolve having to mount the docker binary

The Readme and run.sh both do this trick a lot:

$ docker run -d --name powerstrip-weave \
    --expose 80 \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /usr/bin/docker:/usr/bin/docker \
    binocarlos/powerstrip-weave launch

Where we should only need to mount the docker socket not the binary itself.

A solution is to download the docker binary inside the container but I then had a version mis-match between the newer docker binary inside the container and the older docker server I was running on my host.

I'm not sure which approach is better and have gone for mounting the docker binary from the host to ensure the same version for the moment - issue pending feedback!

Getting the adapter to trigger

I'm having some trouble getting an adapter to be triggered from the powerstrip container.

Would love some help working out what I'm doing wrong or what the issue is as powerstrip-weave is now very close to done :-)

Here are the steps to replicate what is happening:

1. build the powerstrip container

this is from the 46d6a0eb79583e1e78e3ab0d8d0cf6102f47fe5b commit of the socket-support-protocol-tweaks-and-versioning branch

$ docker build -t clusterhq/powerstrip .

2. start a debug webserver

This is the simplest HTTP server that writes the request url to stdout.

var http = require('http')
var server = http.createServer(function(req, res){
  console.log('-------------------------------------------');
  console.log('req: ' + req.url)
})
server.listen(80, function(){
  console.log('server listening on port: 80')
})

This is listening not in a container but on my host (which is 192.168.8.120)

$ sudo node simpleserver.js

I run this in a seperate shell so can see what it logs.

3. create `~/powerstrip-demo/adapters.yml`

The config will point at the debug web server running above on 192.168.8.120 - I've checked this IP is accesible from inside containers.

$ cat > ~/powerstrip-demo/adapters.yml <<EOF
version: 1
endpoints:
  "POST /*/containers/create":
    pre: [debug]
  "POST /*/containers/*/start":
    post: [debug]
adapters:
  debug: http://192.168.8.120/extension
EOF

4. start the powerstrip container

$ docker run -ti --rm --name powerstrip \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/powerstrip-demo/adapters.yml:/etc/powerstrip/adapters.yml \
  -p 2375:2375 \
  clusterhq/powerstrip

I run this in attached mode in another seperate shell so can see what it logs.

5. change DOCKER_HOST

$ export DOCKER_HOST=localhost:2375

6. check docker ps works

$ docker ps -a

I get back results and see the powerstrip container logging - this works great.

7. docker run

This is where I cannot for the life of me get the adapter to trigger i.e. there is no output from the simple webserver where I would expect /v1.16/containers/create

$ docker run ubuntu echo hello

The full log from the powerstrip container is below - it is getting the requests for the following sequence (as expected):

POST /v1.16/containers/create
POST /v1.16/containers/33bd32dba637d4fb0ea21658426ce01ccf4bc0ee0f3eb08942ca2254464422d3/start
POST /v1.16/containers/33bd32dba637d4fb0ea21658426ce01ccf4bc0ee0f3eb08942ca2254464422d3/wait

However - the adapter itself (which should be printing these urls or at least the first one) never triggers.

Sorry if this is something dumb I'm doing - I've being racking my brains for the past few hours!

export DOCKER_HOST=tcp://localhost:2375
2015-01-22 01:47:49+0000 [-] Log opened.
2015-01-22 01:47:49+0000 [-] twistd 14.0.0 (/usr/bin/python 2.7.6) starting up.
2015-01-22 01:47:49+0000 [-] reactor class: twisted.internet.epollreactor.EPollReactor.
2015-01-22 01:47:49+0000 [-] ServerProtocolFactory starting on 2375
2015-01-22 01:47:49+0000 [-] Starting factory <powerstrip.powerstrip.ServerProtocolFactory instance at 0x7fefae119098>
2015-01-22 01:48:05+0000 [HTTPChannel,0,172.17.42.1] Starting factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac046830>
2015-01-22 01:48:05+0000 [DockerProxyClient,client] "172.17.42.1" - - [22/Jan/2015:01:48:05 +0000] "GET /v1.16/containers/json?all=1 HTTP/1.1" 200 300 "-" "Docker-Client/1.4.1"
2015-01-22 01:48:05+0000 [DockerProxyClient,client] Stopping factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac046830>
2015-01-22 01:48:30+0000 [HTTPChannel,1,172.17.42.1] Starting factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac062a70>
2015-01-22 01:48:30+0000 [DockerProxyClient,client] "172.17.42.1" - - [22/Jan/2015:01:48:30 +0000] "POST /v1.16/containers/create HTTP/1.1" 201 90 "-" "Docker-Client/1.4.1"
2015-01-22 01:48:30+0000 [DockerProxyClient,client] Stopping factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac062a70>
2015-01-22 01:48:30+0000 [HTTPChannel,2,172.17.42.1] Starting factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefabdef518>
2015-01-22 01:48:30+0000 [HTTPChannel,1,172.17.42.1] Starting factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac035d40>
2015-01-22 01:48:31+0000 [DockerProxyClient,client] "172.17.42.1" - - [22/Jan/2015:01:48:30 +0000] "POST /v1.16/containers/33bd32dba637d4fb0ea21658426ce01ccf4bc0ee0f3eb08942ca2254464422d3/start HTTP/1.1" 204 - "-" "Docker-Client/1.4.1"
2015-01-22 01:48:31+0000 [DockerProxyClient,client] Stopping factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefac035d40>
2015-01-22 01:48:31+0000 [HTTPChannel,1,172.17.42.1] Starting factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefabdefdd0>
2015-01-22 01:48:31+0000 [DockerProxyClient,client] Stopping factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefabdef518>
2015-01-22 01:48:31+0000 [DockerProxyClient,client] "172.17.42.1" - - [22/Jan/2015:01:48:30 +0000] "POST /v1.16/containers/33bd32dba637d4fb0ea21658426ce01ccf4bc0ee0f3eb08942ca2254464422d3/wait HTTP/1.1" 200 17 "-" "Docker-Client/1.4.1"
2015-01-22 01:48:31+0000 [DockerProxyClient,client] Stopping factory <powerstrip.powerstrip.DockerProxyClientFactory instance at 0x7fefabdefdd0>

Unable to run powerstrip-weave demo

I'm unable to run this script completely because of the error that gives me

#!/bin/bash
echo "Starting powerstrip-weave"
CID=$(sudo docker run -d --name powerstrip-weave \
    --expose 80 \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v /usr/bin/docker:/usr/bin/docker \
    binocarlos/powerstrip-weave launch)
sudo docker logs $CID
echo "Starting powerstrip"
CID=$(sudo docker run -d --name powerstrip \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v $HOME/powerstrip-demo/adapters.yml:/etc/powerstrip/adapters.yml \
  --link powerstrip-weave:weave \
  -p 2375:2375 \
  clusterhq/powerstrip)
sudo docker logs $CID
echo "Starting powerstrip-weave-example"
DOCKER_HOST=tcp://127.0.0.1:2375 CID=$(sudo docker run -e "WEAVE_CIDR=10.255.0.51/24" -d binocarlos/powerstrip-weave-example hello world)
sleep 30
sudo docker logs $CID
echo "docker ps -a"
sudo docker ps -a
echo "Finish"

[aanm@localhost powerstrip-demo]$ ./demopowerstripandweave 
Starting powerstrip-weave
Starting powerstrip
Starting powerstrip-weave-example
100 ms elapsed
200 ms elapsed
300 ms elapsed
400 ms elapsed
500 ms elapsed
600 ms elapsed
700 ms elapsed
800 ms elapsed
900 ms elapsed
1000 ms elapsed
1100 ms elapsed
1200 ms elapsed
1300 ms elapsed
1400 ms elapsed
1500 ms elapsed
1600 ms elapsed
1700 ms elapsed
1800 ms elapsed
1900 ms elapsed
2000 ms elapsed
2100 ms elapsed
2200 ms elapsed
2300 ms elapsed
2400 ms elapsed
2500 ms elapsed
2600 ms elapsed
2700 ms elapsed
2800 ms elapsed
2900 ms elapsed
3000 ms elapsed
3100 ms elapsed
3200 ms elapsed
3300 ms elapsed
3400 ms elapsed
3500 ms elapsed
3600 ms elapsed
3700 ms elapsed
3800 ms elapsed
3900 ms elapsed
4000 ms elapsed
4100 ms elapsed
4200 ms elapsed
4300 ms elapsed
4400 ms elapsed
4500 ms elapsed
4600 ms elapsed
4700 ms elapsed
4800 ms elapsed
4900 ms elapsed
5000 ms elapsed
5100 ms elapsed
5200 ms elapsed
5300 ms elapsed
5400 ms elapsed
5500 ms elapsed
5600 ms elapsed
5700 ms elapsed
5800 ms elapsed
5900 ms elapsed
6000 ms elapsed
6100 ms elapsed
6200 ms elapsed
6300 ms elapsed
6400 ms elapsed
6500 ms elapsed
6600 ms elapsed
6700 ms elapsed
6800 ms elapsed
6900 ms elapsed
7000 ms elapsed
7100 ms elapsed
7200 ms elapsed
7300 ms elapsed
7400 ms elapsed
7500 ms elapsed
7600 ms elapsed
7700 ms elapsed
7800 ms elapsed
7900 ms elapsed
8000 ms elapsed
8100 ms elapsed
8200 ms elapsed
8300 ms elapsed
8400 ms elapsed
8500 ms elapsed
8600 ms elapsed
8700 ms elapsed
8800 ms elapsed
8900 ms elapsed
9000 ms elapsed
9100 ms elapsed
9200 ms elapsed
9300 ms elapsed
9400 ms elapsed
9500 ms elapsed
9600 ms elapsed
9700 ms elapsed
9800 ms elapsed
9900 ms elapsed
10000 ms elapsed
Network did not connect after 10000 ms
eth0      Link encap:Ethernet  HWaddr 02:42:ac:11:00:10  
          inet addr:172.17.0.16  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:acff:fe11:10/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:18 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1436 (1.4 KB)  TX bytes:648 (648.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

docker ps -a
CONTAINER ID        IMAGE                                        COMMAND                CREATED              STATUS                          PORTS                                            NAMES
f7aaaf811569        zettio/weave:git-d1400bc23377                "/home/weave/weaver    About a minute ago   Up About a minute               0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp   weave               
1cff0cceae02        binocarlos/powerstrip-weave-example:latest   "bash /srv/app/run.s   About a minute ago   Exited (1) About a minute ago                                                    sad_perlman         
8e619e8e19c8        clusterhq/powerstrip:latest                  "twistd -noy powerst   About a minute ago   Up About a minute               0.0.0.0:2375->2375/tcp                           powerstrip          
a731df47c2ef        binocarlos/wait-for-weave:latest             "/home/weavewait/wai   About a minute ago   Exited (0) About a minute ago                                                    weavewait           
98e501a07d99        binocarlos/powerstrip-weave:latest           "/srv/app/run.sh lau   About a minute ago   Up About a minute               80/tcp                                           powerstrip-weave  
Finish

@binocarlos I think here we can solve it properly ;-)

transport closed before response received error

Hey, I'm having a little trouble connecting via powerstrip - here is what I did:

built the powerstrip container from the latest socket-support-protocol-tweaks-and-versioning branch
run a powerstrip-debug container which simply logs any requests to stdout before returning
create the following powerstrip config:

endpoints:
  "/*/containers/create":
    pre: [debug]
  "/*/containers/*/start":
    post: [debug]
plugins:
  debug: http://debug/v1/extension

start powerstrip:

$ docker run -d --name powerstrip \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v ~/powerstrip-demo/plugins.yml:/etc/powerstrip/plugins.yml \
  --link powerstrip-debug:debug \
  -p 2375:2375 \
  clusterhq/powerstrip

edit DOCKER_HOST

$ export DOCKER_HOST=localhost:2375

do a docker run against powerstrip

$ docker run --rm ubuntu bash -c "echo hello"

I get the following error when doing this:

FATA[0000] Post http://127.0.0.1:2375/v1.16/containers/create: net/http: transport closed before response was received. Are you trying to connect to a TLS-enabled daemon without TLS?

There is nothing logged from the debugger container so I don't think the request is making it that far.

Am a bit stuck - sorry if this is something obvious - any clues?

Thanks

binocarlos/wait-for-weave is exiting without any logs

docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e1d06951ebf3 zettio/weave:git-8e65c7127d82 "/home/weave/weaver 12 minutes ago Up 12 minutes 0.0.0.0:6783->6783/tcp, 0.0.0.0:6783->6783/udp weave

c6fc0d9f544e binocarlos/wait-for-weave:latest "/home/weavewait/wai 12 minutes ago Exited (0) 12 minutes ago weavew
ait
21f2ceedf2b2 binocarlos/powerstrip-weave:latest "/srv/app/run.sh lau 12 minutes ago Up 12 minutes 80/tcp powers
trip-weave

I am trying to run powerstrip-weave on latest CoreOs.

In older versions its complaining "clinet and sever version did not match"
can you please guide me how to rectify this?