appc / docker2aci Goto Github PK

View Code? Open in Web Editor NEW

186.0 186.0 60.0 1.78 MB

library and CLI tool to convert Docker images to ACIs (archived, see https://github.com/rkt/rkt/issues/4024)

License: Apache License 2.0

Go 94.92% Shell 5.08%

docker2aci's People

Contributors

Stargazers

Watchers

docker2aci's Issues

Allow outputting an image layout instead of an image

I'm working on aproject where I'm going to use docker2aci only to immediately extract the ACI into an image layout. It would by handy to enable docker2aci to output an image layout directly and save a bunch of useless disk IO.

I'm not sure if this fits docerk2aci's mission includes this, but it does seem like a good place to put that functionality.

Make docker2aci compatible with the new Docker Registry 2.0.0

The new Docker Registry 2.0.0 is getting ready.

We should check whether docker2aci works fine with it and investigate what changes in docker2aci are necessary to support it.

Use libsquash?

https://github.com/winchman/libsquash

No app section in image

$ docker2aci docker://gcr.io/google_containers/etcd:2.0.9docker://gcr.io/google_containers/etcd:2.0.9
...
Generated ACI(s):
google_containers-etcd-2.0.9.aci

$ actool cat-manifest google_containers-etcd-2.0.9.aci 
{"acKind":"ImageManifest","acVersion":"0.5.1","name":"gcr.io/google_containers/etcd","labels":[{"name":"version","value":"2.0.9"},{"name":"os","value":"linux"},{"name":"arch","value":"amd64"}],"annotations":[{"name":"authors","value":"Dawn Chen \[email protected]\u003e"},{"name":"created","value":"2015-04-07T22:51:57Z"}]}

Add a bump-release script and document the release process

Now that we do releases, that would help.

Build fails (incompatible type as argument)

Getting this when installing

go get github.com/appc/docker2aci

# github.com/appc/docker2aci/lib/common
go/src/github.com/appc/docker2aci/lib/common/common.go:96: cannot use reader (type *aci.TarReadCloser) as type *tar.Reader in argument to aci.ValidateArchive
go/src/github.com/appc/docker2aci/lib/common/common.go:358: cannot use *reader (type aci.TarReadCloser) as type tar.Reader in argument to tarball.Walk

docker2aci should work with local docker images

Currently docker2aci can only take a docker image from a registry (through the network).

After building a Docker image locally from a Dockerfile, it is inconvenient to have to upload it to a registry before being able to convert it to an aci with docker2aci.

Instead, it would be great if docker2aci could work on local files produced by docker-save.

docker2aci: output aci(s) on disk

The basic invocation of docker2aci should create an ACI for each layer of the input image and write it to a file on disk.

Volumes are ignored during conversion

The Volumes mount points are ignored so I need to patch the resulting ACI with correct mount points.
The problem is that docker doesn't name the volumes so I don't know if docker2aci should set arbitrarily a name or interactively asking for the name of each volume.

Docker images with PaxHeaders generate ACI images with unequal hashes

When fetching the same Docker image one should get an ACI image with identical hashes. With docker://busybox for example this is the case.

~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://busybox
Downloading cf2616975b4a: [====================================] 32 B/32 B
Downloading 6ce2e90b0bc7: [====================================] 1.15 MB/1.15 MB
Downloading 8c2e06607696: [====================================] 32 B/32 B
sha512-9d710100ce6769569b12a39100318bfe
~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://busybox
Downloading cf2616975b4a: [====================================] 32 B/32 B
Downloading 6ce2e90b0bc7: [====================================] 1.15 MB/1.15 MB
Downloading 8c2e06607696: [====================================] 32 B/32 B
sha512-9d710100ce6769569b12a39100318bfe

However, with the redis image this is sadly not the case.

~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://redis
Downloading ba249489d0b6: [====================================] 37.2 MB/37.2 MB
Downloading 19de96c112fc: [====================================] 32 B/32 B
Downloading d990a769a35e: [====================================] 1.7 KB/1.7 KB
Downloading 8656a511ce9c: [====================================] 5.94 MB/5.94 MB
Downloading f7022ac152fb: [====================================] 93.6 KB/93.6 KB
Downloading 8e84d9ce7554: [====================================] 611 KB/611 KB 
Downloading c9e5dd2a9302: [====================================] 32 B/32 B
Downloading 27b967cdd519: [====================================] 32 B/32 B
Downloading 3024bf5093a1: [====================================] 32 B/32 B
Downloading e6a9eb403efb: [====================================] 3.04 MB/3.04 MB
Downloading c3532a4c89bc: [====================================] 98 B/98 B
Downloading 35fc08946add: [====================================] 32 B/32 B
Downloading d586de7d17cd: [====================================] 32 B/32 B
Downloading 1f677d77a8fa: [====================================] 196 B/196 B
Downloading ed09b32b8ab1: [====================================] 32 B/32 B
Downloading 54647d88bc19: [====================================] 32 B/32 B
Downloading 2f2578ff984f: [====================================] 32 B/32 B
sha512-039afdb1311dc35a802a48352b6b29f2
~/W/s/rkt: sudo rkt --insecure-skip-verify fetch docker://redis
Downloading ba249489d0b6: [====================================] 37.2 MB/37.2 MB
Downloading 19de96c112fc: [====================================] 32 B/32 B
Downloading d990a769a35e: [====================================] 1.7 KB/1.7 KB
Downloading 8656a511ce9c: [====================================] 5.94 MB/5.94 MB
Downloading f7022ac152fb: [====================================] 93.6 KB/93.6 KB
Downloading 8e84d9ce7554: [====================================] 611 KB/611 KB
Downloading c9e5dd2a9302: [====================================] 32 B/32 B
Downloading 27b967cdd519: [====================================] 32 B/32 B
Downloading 3024bf5093a1: [====================================] 32 B/32 B
Downloading e6a9eb403efb: [====================================] 3.04 MB/3.04 MB
Downloading c3532a4c89bc: [====================================] 98 B/98 B
Downloading 35fc08946add: [====================================] 32 B/32 B
Downloading d586de7d17cd: [====================================] 32 B/32 B
Downloading 1f677d77a8fa: [====================================] 196 B/196 B
Downloading ed09b32b8ab1: [====================================] 32 B/32 B
Downloading 54647d88bc19: [====================================] 32 B/32 B
Downloading 2f2578ff984f: [====================================] 32 B/32 B
sha512-4a60165ef194356a7a3070f5af0f1601

Inspecting the images with xxd we can see that the difference is the PaxHeader.

diff  -u <(xxd redis-latest.aci) <(xxd redis-latest1.aci)
...
@@ -1271809,13 +1271809,13 @@
 01368000: 726f 6f74 6673 2f75 7372 2f73 6861 7265  rootfs/usr/share
 01368010: 2f63 612d 6365 7274 6966 6963 6174 6573  /ca-certificates
 01368020: 2f6d 6f7a 696c 6c61 2f50 6178 4865 6164  /mozilla/PaxHead
-01368030: 6572 732e 3233 3837 302f 5442 5441 4b5f  ers.23870/TBTAK_
+01368030: 6572 732e 3233 3831 302f 5442 5441 4b5f  ers.23810/TBTAK_
 01368040: 5545 4b41 455f 4b6b 5f53 6572 7469 6669  UEKAE_Kk_Sertifi
 01368050: 6b61 5f48 697a 6d65 745f 5361 6c61 7963  ka_Hizmet_Salayc
 01368060: 735f 2d5f 3030 3030 3030 3000 3030 3030  s_-_0000000.0000
 01368070: 3030 3000 3030 3030 3030 3000 3030 3030  000.0000000.0000
 01368080: 3030 3030 3137 3000 3132 3431 3036 3034  0000170.12410604
-01368090: 3530 3000 3033 3134 3630 0020 7800 0000  500.031460. x...
+01368090: 3530 3000 3033 3134 3532 0020 7800 0000  500.031452. x...
 013680a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 013680b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 013680c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

The result for the rkt user is that she sees lots of redis images listed when running rkt image list

KEY                                 NAME                        IMPORTTIME              LATEST
sha512-b6fd68c419b8b42aed109ca0e18d31b3010d6daa784b71f94dc660e00d0496a5 coreos.com/rkt/stage1:0.8.0+gitb18cf28      2015-09-10 16:03:33.805 +0200 CEST  false   
sha512-b843248e28fa9132e23e1e763142049d17a61bab7873dff1e1ff105f9ddb2708 redis:latest                    2015-09-17 13:27:04.24 +0200 CEST   true    
sha512-44b881600b101eb45c7ea626a2bef9cac97e8ae2c194258fd06f5dd97bcf463c redis:latest                    2015-09-17 14:55:31.19 +0200 CEST   true    
sha512-fb6b47cc9e7ee29f67422c6585c8f517c16e0e0ee9f4cf8b8cafd8e1c1d29233 redis:latest                    2015-09-17 14:57:36.779 +0200 CEST  true    
sha512-cbe0ddf1053e9c0141744b3929bbdde2938976c3dfee9a8976b05cf3cb22c365 coreos.com/rkt/stage1:0.8.0+git68b56a6-dirty    2015-09-17 14:58:07.649 +0200 CEST  false   
sha512-1d318506c1f328e2d58f29db710889001a631063d148973f86a784d316d3e91b redis:latest                    2015-09-17 14:59:34.478 +0200 CEST  true    
sha512-91e98d7f1679a097c878203c9659f2a26ae394656b3147963324c61fa3832f15 coreos.com/etcd:v2.0.9              2015-09-17 14:59:41.335 +0200 CEST  false   
sha512-039afdb1311dc35a802a48352b6b29f2e120b804fd0362fd2866ba1cb6ab301e redis:latest                    2015-09-17 16:03:51.421 +0200 CEST  true    
sha512-4a60165ef194356a7a3070f5af0f1601427e05c8fc65c5c4eec3b34cbd9ec662 redis:latest                    2015-09-17 16:05:33.074 +0200 CEST  true    
sha512-9d710100ce6769569b12a39100318bfed5b6b98115ee6315b724c11658db3751 busybox:latest                  2015-09-17 16:07:30.582 +0200 CEST  true

handle hard links and whiteouts in different layers

Bug initially reported on rkt/rkt#1653: if a Docker image has the hard links /LINK-A and /LINK-B in the first layer but /LINK-A gets deleted in the second layer, docker2aci generates a squashed tarball that contains only /LINK-B as a dangling reference, so the tarball cannot be extracted.

To produce the image docker://albanc/busybox-hardlinks, I followed the steps:

# docker run -ti busybox
(container)# touch /LINK-A
(container)# ln /LINK-A /LINK-Z
(container)# exit
# docker commit $container_id
# docker run -ti $container_id
(container)# rm -f /LINK-A
(container)# exit
# docker commit $container_id_2 albanc/busybox-hardlinks
$ docker2aci docker://albanc/busybox-hardlinks
$ tar xf albanc-busybox-hardlinks-latest.aci
tar: rootfs/LINK-Z: Cannot hard link to ‘rootfs/LINK-A’: No such file or directory

support configurable tmpdir

As a library, docker2aci is used by tools like rkt to transparently retrieve + convert Docker images for users. In some cases these tools have strict requirements about their filesystem writes. Currently docker2aci makes heavy use of temporary files when it retrieves images. To integrate more cleanly with tools like rkt, it should support being passed a configuration value controlling where it writes these files.

Resume fetch after failure

Initially reported on rkt/rkt#1280:

If a fetch is interrupted by a crash or by power loss, it currently must completely start over the download. It would be great if it at least stored layers, if not resumeable bytes like a basic program like wget.

Allow selecting the compression method

The ACI spec says

Image archives MAY be compressed with gzip, bzip2, or xz.

It would be handy if the docker2aci CLI could let the user choose the compression method.

some docker2aci generated images doesn't have an arch label.

docker2aci docker://busybox
Downloading cf2616975b4a: [====================================] 32 B/32 B
Downloading 6ce2e90b0bc7: [====================================] 1.15 MB/1.15 MB
Downloading 8c2e06607696: [====================================] 32 B/32 B

Generated ACI(s):
busybox-latest.aci

tar xvfO busybox-latest.aci manifest 
manifest
{"acKind":"ImageManifest","acVersion":"0.5.1","name":"busybox","labels":[{"name":"version","value":"latest"},{"name":"os","value":"linux"}],"app":{"exec":["/bin/sh"],"user":"0","group":"0"},"annotations":[{"name":"authors","value":"Jérôme Petazzoni \[email protected]\u003e"},{"name":"created","value":"2015-04-17T22:01:13Z"}]}

This is problably due to an empty layerData.Architecture (probably before docker supported archs different than amd64).

In future with rkt/rkt#1167 stage0 will refuse to execute it due to missing arch.

One possible solution will be to set os=Linux if os is empty and arch=amd64 if arch is empty.

whiteout files are not handled correctly

When walking the Docker layer tarballs, docker2aci currently ignores whiteout files completely. This is not correct since these files are then not blacklisted from the produced ACI.

Simple example:

; docker run quay.io/jonboulle/blah ls /abc
ls: /abc: No such file or directory

; ./docker2aci quay.io/jonboulle/blah
Downloading layer: 420c12abb770a1549fb67c8cde9b9545a5a6ff5f06d03100d925e79592e5c5ef
Downloading layer: e677c0c2711e10193aec7a6932a7844deae84f4731370b4e1112ad14393d128c
Downloading layer: 4986bf8c15363d1c5d15512d5266f8777bfba4974ac56e3270e7760f6f0a8125
Downloading layer: ea13149945cb6b1e746bf28032f02e9b5a793523481a0a18645fc77ad53c4ea2
Downloading layer: df7546f9f060a2268024c8a230d8639878585defcc1bc6f79d2728a13957871b
Downloading layer: 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158

Generated ACI(s):
jonboulle-blah-latest.aci

 ; tar tvf jonboulle-blah-latest.aci rootfs/abc
-rw-r--r-- 0/0               0 2015-03-25 18:00 rootfs/abc

tag a release

it would be nice to have a tagged release with a version number for packaging.

Add a license to the repo

Could you please add an explicit LICENSE file to the repo so that it's clear under what terms you're providing it, and under what license you expect the contributions to be?

Thanks!

Converting big images is slow

I did some benchmarks with big images after #33 and the conversion is around 40% slower. This is so because we're compressing each layer and decompressing it when we squash to compress the squashed image again.

I think generating one ACI for each layer is a rather uncommon use case so compressing only the squashed image would be acceptable.

We could also consider using https://github.com/klauspost/pgzip to make it even faster.

Allow setting socketActivated: true on converted ports

When converting a Docker image to ACI, the converted ports are always set to "socketActivated: false".

It would be handy to have a CLI flag to control the socketActivated property on the converted ACI image or have the ability to change something on the input Docker image to hint the tool that this is a SocketActivated port.

This is following a discussion here: https://groups.google.com/forum/#!topic/rkt-dev/WZrAGMOVZrY

docker2aci: create a library that can be imported

As well as providing a command line tool, docker2aci should be usable as a library that other projects can leverage to work with docker images.

For example, rocket might import docker2aci and use it to be able to fetch/run directly from Docker registry URLs.

(The subtext here is that there should be nothing specific to rocket or any other implementation in this project itself)

docker2aci should compress the acis it produces

It would save some disk space.

404 when fetching docker image from artifactory hosted docker registry

It looks like repository.go uses version 1 of the docker registry api [1], but none of the endpoints are prefixed with /v1/. Here's an example line from repository.go:

req, err := http.NewRequest("GET", "https://"+path.Join(registry, "repositories", appName, "tags"), nil)

The artifactory docker registry [2] requires that v1 be in the registry url just like the api spec says, but there is no way to specify it to docker2aci.

[1] https://docs.docker.com/v1.6/reference/api/registry_api/
[2] http://www.jfrog.com/confluence/display/RTF/Docker+Repositories

docker2aci fails to retrieve images from Google Container Registry

As noted in rkt/rkt#726:

$ ./docker2aci docker://gcr.io/google_containers/kibana:1.2


Error: conversion error: error getting ImageID from tag 1.2: HTTP code: 405. URL: https://gcr.io/v1/repositories/google_containers/kibana/tags/1.2

From my very brief look, it seems like gcr.io doesn't properly implement the Docker registry API? Hence the command above failing - the tags endpoints (e.g. https://gcr.io/v1/repositories/google_containers/kibana/tags, https://gcr.io/v1/repositories/google_containers/kibana/tags/1.2) are 405ing for no good reason I can tell...

But Docker works just fine:

$ docker pull gcr.io/google_containers/kibana:1.2
Pulling repository gcr.io/google_containers/kibana
46724d0d1108: Pulling dependent layers 
511136ea3c5a: Download complete 
fa4fd76b09ce: Downloading [==================================================>] 66.26 MB/66.26 MB

docker2aci: create ACI from Dockerfile

It would be nice to be able to take a Dockerfile and convert it to an ACI. This should be semantically equivalent to running docker build and then using docker2aci on the resulting image.

Hopefully this could leverage the work done by @mcuadros in appc/spec#51 (originally https://github.com/mcuadros/rocketizer) - this issue should obviate that one when implemented.

dockerfile builder aci

As we have the build-repository docker ACI and this library we should build an ACI that can take a Dockerfile in a volume and then outputs the ACI into another volume.

The basic invocation for rkt would be something like:

$ git clone https://github.com/coreos/etcd
$ rkt run -volume dockerfile,kind=host,source=`pwd`/etcd -volume output,kind=host,source=path-to-output dockerfile2aci.aci

use ContainerApplicationGenericLabels for VCS information

The ContainerApplicationGenericLabels effort is looking to standardise labels for containers - we should sync up wherever possible, e.g. https://github.com/projectatomic/ContainerApplicationGenericLabels/pull/12/files

Provide warnings/errors during conversions

As noted in #20 (comment) , it would be good to be able to warn people about cases when docker2aci has to perform suboptimal translations (for example when converting an exec statement to use /bin/sh)

User/group is missing when translating some docker images

$ docker2aci docker://gcr.io/google_containers/etcd:2.0.9
Downloading 511136ea3c5a: [====================================] 32 B/32 B
Downloading 4ec7f790b564: [====================================] 32 B/32 B
Downloading cfeffad3cf16: [====================================] 1.95 MB/1.95 MB
Downloading b6b9a86dc06a: [====================================] 1.81 MB/1.81 MB

Generated ACI(s):
google_containers-etcd-2.0.9.aci

$ actool cat-manifest --pretty-print google_containers-etcd-2.0.9.aci 
{
    "acKind": "ImageManifest",
    "acVersion": "0.7.0",
    "name": "gcr.io/google_containers/etcd",
    "labels": [
        {
            "name": "version",
            "value": "2.0.9"
        },
        {
            "name": "os",
            "value": "linux"
        },
        {
            "name": "arch",
            "value": "amd64"
        }
    ],
    "annotations": [
        {
            "name": "authors",
            "value": "Dawn Chen \[email protected]\u003e"
        },
        {
            "name": "created",
            "value": "2015-04-07T22:51:57Z"
        },
        {
            "name": "appc.io/docker/registryurl",
            "value": "gcr.io"
        },
        {
            "name": "appc.io/docker/repository",
            "value": "google_containers/etcd"
        },
        {
            "name": "appc.io/docker/imageid",
            "value": "b6b9a86dc06aa1361357ca1b105feba961f6a4145adca6c54e142c0be0fe87b0"
        },
        {
            "name": "appc.io/docker/parentimageid",
            "value": "cfeffad3cf16e8e0055bfbd7f5af5470d4772f9433c12b4d25f010eb41f96ccc"
        }
    ]
}

Mapping between docker tags and appc manifest labels

I think that a docker image tags should be used to detect the version label in the manifest. Now latest is used by I have some doubts about the latest pattern in appc/spec (see appc/spec#60 and appc/spec#73) and the usage of latest as a valid version.

Some questions and notes.

Docker images can have multiple tags. Which one to use for the version label?
Given the previous point, what to do with the remaining tags?
As docker tags can come and go from an image with the same Id (for example the latest tag can be moved to another image), while aci manifest labels are tied to the ACI imageID (as changing a label, so the manifest will change its sha512), if the tags converted to manifest labels (previous points) changes in the docker registry a new aci with different imageID will be created. I don't see this as a problem but as a different behavior. The cas, should handle this. (for example the implementation done in rkt/rkt#394 handles the image import time and the latest pattern).

failed to convert docker://kubernetes/redis-slave:v2

$ sudo /home/yifan/rkt/build-rkt-0.7.0+git/bin/rkt --debug=false --insecure-skip-verify=true fetch docker://kubernetes/redis-slave:v2
[sudo] password for yifan: 
Downloading 511136ea3c5a: [====================================] 32 B/32 B
Downloading f0dde87450ec: [====================================] 65.8 MB/65.8 MB
Downloading 76b658ecb564: [====================================] 71.5 KB/71.5 KB
Downloading 4faa69f72743: [====================================] 679 B/679 B
Downloading 2103b00b3fdf: [====================================] 32 B/32 B
Downloading 4303edc5e986: [====================================] 85.4 MB/85.4 MB
Downloading 14dfde0f38c5: [====================================] 643 B/643 B
Downloading d489fc3e869f: [====================================] 438 B/438 B
Downloading a8253f7d9cc1: [====================================] 22.7 KB/22.7 KB
Downloading 38cf7a363b14: [====================================] 32 B/32 B
Downloading d9343f29a1c4: [====================================] 32 B/32 B
Downloading 9cb3dca21a1f: [====================================] 32 B/32 B
Downloading c9eda0ec97c0: [====================================] 2.93 MB/2.93 MB
Downloading a44f8ddf5fb8: [====================================] 32 B/32 B
Downloading cf0fc1a9cd2b: [====================================] 32 B/32 B
Downloading 8567350addd7: [====================================] 32 B/32 B
Downloading dff9dfa9220d: [====================================] 32 B/32 B
Downloading 3fd8168c2f23: [====================================] 511 B/511 B
Downloading 85b1106c3ae7: [====================================] 544 B/544 B
Downloading 48a1e2b8f859: [====================================] 32 B/32 B
error converting docker image to ACI: error squashing image: error validating image: flate: corrupt input before offset 105339330

Add ability to stream output

docker2aci can take in a stream input from docker save, but doesn't then allow you to stream the resulting aci anywhere afterwards.

Streaming would allow one pass signing and upload to storage area.

Generated ACI contains several entries for "rootfs"

In this example, 3 entries for "rootfs":

$ docker2aci docker://busybox
$ tar tvzf busybox-latest.aci|grep rootfs$
drwxr-xr-x 0/0               0 1970-01-01 01:00 rootfs
drwxr-xr-x 0/0               0 1970-01-01 01:00 rootfs
drwxr-xr-x 0/0               0 1970-01-01 01:00 rootfs
$

I discovered this when I wanted to check the permissions of /. It does not seem to be causing any visible problems.

docker2aci creates an invalid manifest when entrypoint, user or group aren't set

the appc spec requires that if app is set in the manifest, exec, user and group are required (see https://github.com/appc/spec/blob/master/spec/aci.md)

When docker2aci encounters a docker image that does not have entrypoint, user or group set it generates a manifest that has an app section, but doesn't set these fields. This is invalid and rkt run fails with these images.

Let user specify the stream to print messages

Currently whether the message is printed to stdout or stderr is hardcoded in the lib, we want to have user be able to specify the stream.
Also see #60 .

support authentication to a registry

Reading a .dockercfg
Taking in configuration via env variables and command line
Support being passed credentials when used as a library

vendor dependencies with godep

see https://github.com/tools/godep

go get failed (Dependency field 'App' in struct literal)

When I try to go get, command line said that

$go get github.com/appc/docker2aci
# github.com/appc/docker2aci/lib/common
src/github.com/appc/docker2aci/lib/common/common.go:199: unknown "github.com/appc/spec/schema/types".Dependency field 'App' in struct literal

parallelise layer retrieval

Fetching and converting an image is really slow when every layer is downloaded in serial.

Provide /dev/stdout, /dev/stderr

Docker images expect to have /dev/stdout and /dev/stderr according to:
https://github.com/opencontainers/runc/blob/master/libcontainer/SPEC.md#filesystem

But ACI images should not expect it according to:
https://github.com/appc/spec/blob/master/spec/OS-SPEC.md#devices-and-file-systems

This bug was first reported on rkt/rkt#1617. Fixes in docker2aci should be tested against the image mentioned in that issue.

Since stdout/stderr are commonly unix sockets to systemd-journal and that unix sockets cannot be opened, /dev/stdout and /dev/stderr should be symlinks to /dev/console instead.

docker2aci fails to convert the quay.io/kelseyhightower/kubelet:0.19.0 image

docker2aci does not generate the right set of volumes when converting the quay.io/kelseyhightower/kubelet:0.19.0 Docker image. Seems to have trouble translating volumes defined in an JSON array:

docker2aci docker://quay.io/kelseyhightower/kubelet:0.19.0
Downloading c6b09d8961e4: [====================================] 32 B/32 B
Downloading d988518ba01b: [====================================] 7.92 MB/7.92 MB
Downloading 51fa9c9fc1bb: [====================================] 32 B/32 B
Downloading c6d0abb02258: [====================================] 32 B/32 B

Converted volumes:
    name: "volume-rootfs", path: "/rootfs,", readOnly: false
    name: "volume-sys", path: "/sys,", readOnly: false
    name: "volume-var-lib-kubelet", path: "/var/lib/kubelet,", readOnly: false
    name: "volume-var-run", path: "/var/run,", readOnly: false
    name: "volume-etc-kubernetes", path: "/etc/kubernetes,", readOnly: false
    name: "volume-etc-ssl-certs", path: "/etc/ssl/certs", readOnly: false
    name: "volume-lib64", path: "/lib64]", readOnly: false
    name: "volume-nsenter", path: "/nsenter,", readOnly: false
    name: "volume-usr", path: "/usr,", readOnly: false
    name: "volume---var-lib-docker", path: "[/var/lib/docker,", readOnly: false

Generated ACI(s):
kelseyhightower-kubelet-0.19.0.aci

The resulting manifest looks like this:

{
   "acKind":"ImageManifest",
   "acVersion":"0.5.1",
   "name":"quay.io/kelseyhightower/kubelet",
   "labels":[
      {
         "name":"version",
         "value":"0.19.0"
      },
      {
         "name":"os",
         "value":"linux"
      }
   ],
   "app":{
      "exec":[
         "/kubelet"
      ],
      "user":"0",
      "group":"0",
      "mountPoints":[
         {
            "name":"volume-rootfs",
            "path":"/rootfs,"
         },
         {
            "name":"volume-sys",
            "path":"/sys,"
         },
         {
            "name":"volume-var-lib-kubelet",
            "path":"/var/lib/kubelet,"
         },
         {
            "name":"volume-var-run",
            "path":"/var/run,"
         },
         {
            "name":"volume-etc-kubernetes",
            "path":"/etc/kubernetes,"
         },
         {
            "name":"volume-etc-ssl-certs",
            "path":"/etc/ssl/certs"
         },
         {
            "name":"volume-lib64",
            "path":"/lib64]"
         },
         {
            "name":"volume-nsenter",
            "path":"/nsenter,"
         },
         {
            "name":"volume-usr",
            "path":"/usr,"
         },
         {
            "name":"volume---var-lib-docker",
            "path":"[/var/lib/docker,"
         }
      ]
   },
   "annotations":[
      {
         "name":"authors",
         "value":"Kelsey Hightower \[email protected]\u003e"
      },
      {
         "name":"created",
         "value":"2015-06-18T01:24:39Z"
      }
   ]
}

Error if the url contains custom port

$ ./docker2aci internal-registry.com:5000/demo:v0.5
Downloading layer: 511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158
Conversion error: error building layer: error generating the manifest: invalid char in ACName: :

https://github.com/appc/docker2aci/blob/master/lib/docker2aci.go#L421
https://github.com/appc/docker2aci/blob/master/lib/docker2aci.go#L472

please tag formal releases

Please consider assigning version numbers and tagging releases. Tags/releases
are quite useful for downstream package maintainers (in Debian and other distributions) to export source tarballs, automatically track new releases and to declare dependencies between packages. Read more in the Debian Upstream Guide.

Thank you.

docker2aci: support squashing images

In the case of an image with multiple layers, it would be nice if docker2aci --squash worked; rather than generating ACIs for each layer and turning them into dependencies, it would simply merge the filesystems of all layers into a single ACI.

Fails to build with latest version of appc/spec

# github.com/appc/docker2aci/lib/common
lib/common/common.go:95: tr.Close undefined (type *tar.Reader has no field or method Close)
lib/common/common.go:97: tr.Reader undefined (type *tar.Reader has no field or method Reader)
lib/common/common.go:358: tr.Close undefined (type *tar.Reader has no field or method Close)
lib/common/common.go:360: tr.Reader undefined (type *tar.Reader has no field or method Reader)

Where tr comes from the NewCompressedTarReader function in github.com/appc/spec/aci

use acirenderer?

rkt/rkt#464 introduces a library for rendering ACIs from their embedded rootfs and dependencies. Following the discussion in #6 , we should be able to split out this library and leverage it in docker2aci itself. This will probably be blocked on the library moving out of Rocket and upstream into the appc org somewhere (appc/spec? appc/tools?), since it is a general-purpose and not Rocket specific.

remove rocket-specific import/logic

docker2aci should not have any dependency on code in coreos/rocket; instead we should see the inverse, where tools like Rocket leverage docker2aci to fetch/run Docker images (e.g. rkt/rkt#145). So this ticket is to track the removal of the current dependency within the docker2aci CLI on a Rocket package.

It's possible that rather than removing the import feature of docker2aci entirely, it could instead move to a more generic model - for example, the ACI Registry proposed in rkt/rkt#464. I will leave that to the implementer to explore; the main point here is that the Rocket dep goes away :-)

use docker image ID as value for version label

Per OOB discussion, it would be helpful for people to correlate between Docker images and ACIs created through docker2aci by using the docker image's Image ID as the version label in the produced ACI.

ENTRYPOINT CMD info is lost after conversion

As ENTRYPOINT and CMD are appended to construct Exec [1]. So after conversion, we cannot tell which part is the ENTRYPOINT, and which is the CMD. This creates a problem when users expect to substitute only the CMD in k8s [2].

Some solutions came out of my mind:

Change the spec to match docker. (I am kinda reluctant to do this until we really feel this is a valid usecase).
Write down the ENTRYPOINT and CMD during conversion in annotations, so image consumers can use the annotation to reconstruct the ENTRYPOINT and CMD.

[1]

docker2aci/lib/common/common.go

Lines 448 to 449 in 783b6b1

 func getExecCommand(entrypoint []string, cmd []string) appctypes.Exec { 

 return append(entrypoint, cmd...)

[2] https://github.com/kubernetes/kubernetes/blob/master/docs/getting-started-guides/rkt/notes.md#doesnt-support-entrypoint--cmd-feature

cc @iaguis @alban

set default exec from ENTRYPOINT and CMD

Dockerfiles have two parameters controlling what gets executed: ENTRYPOINT and CMD. There is a somewhat complicated relationship between the two as they both support exec and shell forms and CMD has a third form which just results in being passed as arguments to ENTRYPOINT.

We should just follow the same logic that Docker does for resolving these into the actual default command line to be executed, and then set that as the app.exec.cmd in our created image manifest.

	func getExecCommand(entrypoint []string, cmd []string) appctypes.Exec {
	return append(entrypoint, cmd...)

appc / docker2aci Goto Github PK

docker2aci's People

Contributors

Stargazers

Watchers

Forkers

docker2aci's Issues

Recommend Projects

Recommend Topics

Recommend Org

Jobs