dnephin / dobi

A build automation tool for Docker applications

dobi

A build automation tool for Docker applications.

Keep your project tasks organized, portable, repeatable, and fast with dobi. Define the resources and tasks required to build, test, and release your project in a dobi.yaml and run them with dobi TASK.

See Getting Started

Features

Key features of dobi:

• optimal - tasks are only run when the resource is stale. If a resource hasn't changed the task is skipped.
• portable - all tasks run in a container, so developers are free to use different operating systems and environments.
• repeatable - tasks are defined in a dobi.yaml so new contributers can get started quickly, and a task will always produce the same results. Variables are supported, but must be explicitly defined, so there's no hidden environment variables that could change the behaviour of a task.
• organized - tasks can be chained together using an alias resource to produce entire workflows like test or release, which may involve multiple independent tasks.
• dependencies - tasks can depend on other tasks using depends. When a task is run, its dependencies are checked first, and run if they are stale.

Install

The one liner:

curl -L -o /usr/local/bin/dobi "https://github.com/dnephin/dobi/releases/download/v0.15.0/dobi-$(uname -s)"; chmod +x /usr/local/bin/dobi

For a Windows binary, and more install options, see Install

Examples

See Examples

Documentation

See Documentation

Contributing

dobi is still in early development. If you'd like to contribute, please open an issue, or find an existing issue, and leave a comment saying you're working on a feature.

dobi's Issues

VCS Integration

Hello!
Just pushed this WIP PR #39. Mainly to talk about how can dobi better integrate with git.

As I see things, there are two flows I would like dobi to allow. The first one is what #39 aims to achive.
Forbidding a job to run if HEAD is not the same as the dobi job branch parameter. This would allow a workflow similar to the below.

• Its not quite clear in my mind yet if we need to provide similar behavior with the image resource. Since if a image depends on a job that is bound to a branch, the image task would not run anyway.

job=release:
    use: releaser
    mounts: [dist]
    branch: "master"
    env:
     - "VERSION={env.VERSION}"
    depends: [binary]

job=canary:
    use: releaser
    branch: "canary"
    env:
     - "VERSION={env.CANARY_VERSION}"
    depends: [binary]

The second would allow a workflow similar to Captain where dobi build,dobi push and dobi pull would run/push/pull a specific image task depending where HEAD is. This would be great as a base for a future deploy resource. Maybe a git resource that would let the user link branch to an image

git:
    branches:
     - canary=canary
     - master=release

Additional resource types

Current resources are: Image, Run, Volume, Alias

Some more possible resources are:

Push

Tag an image with one or more tags and push them to a registry

push=stage:
  image: <image resource name>
  tags: [...]

Pull Image

Pull an image from a registry, which can be used in place of an Image resource.

pull=golang:
  image: <image with a tag>
  when: <always | missing | time.duration>

Compose

Start an environment defined by a Compose file

compose=acceptance:
  file: [docker-compose.yml]
  project: "acceptance-test-{USER}"

Service

Start a service which will continue to run for the entire duration of the execution, and will be shutdown before dobi exits.

service=mysql:
  use: <image resource name>
  ... # service options

Task Alias Implemented in ecae1cd

`image:push` to private registries is broken when using hostname/repo

I believe you should be able to push to a private registry by using the full name of the image that includes the host/port/path to the registry.

dobi supports the same auth config as the docker client, so authentication should work as well.

I need to test it out to confirm.

cc @jmmills (let's move the discussion here, out of docker/compose#1126)

Proposal: new `service` resource type

Sometimes a build task requires some "service" to be run in the "background" (aka detached container). Currently the only resource type that runs this way is a compose resource. So to provide a service to another build task you have to create a Compose file.

Docker 1.12 added the concept of a Swarm mode service. It would be good to add a new resource type which starts these services, so that it's not necessary to create a Compose file just to get a single service.

Default to interactive

Instead of an interactive: field on jobs, maybe all jobs should receive tty: true, interactive: true if dobi is run from a tty.

Job abstraction

These are the current dobi's resources:

// Build tool specific
- env
- alias
// You are nice
- compose
// What the end user actually run/expects
- image 
- job
// you cant use a mount on a image. Its strictly job related
- mount

But images are a boring abstraction. Compose deals with them super nice already. You just run from a file or ou pull it from the hub. Its how you actually run them is what makes dobi special. We should make the job abstraction stronger for itself. I think that ultimately dobi should support a single manifest abstraction for running a job, in the same sense of #55, alowing the end user to have a single file with all the information about the job.

job=watch:
    image:  
        // I dont like this. currently `image=refactorer:    image: dnephin/go-refactor` is a stutterer
        // I would rather container, container-name or repo or name. IDK
        container: dnephin/dobi-dev  
        steps:
          - FROM: golang:1.7-alpine
          - RUN: apk add -U git bash curl tree
          - RUN: go get github.com/dnephin/filewatcher
          - GLIDE: 0.12.3
    command: filewatcher x '.git' -- go run main.go
    interactive: true
    working-dir: /go/src/github.com/dnephin/dobi
    description: "Watch for code changes and run"
    env:
      - 'PS1=# '
      - 'CGO_ENABLED=0'
    local-mounts:
        - bind: .
          path: /go/src/github.com/dnephin/dobi

The main intent here is to make dobi more concise, avoiding the file changing dizzyness as well as saving some lines from the dockerfile itself, once that we dont need to set things like WORKDIR and ENV because we enforced them on the way the job run. So when you are actually hands on writing a dobi resource, you either decide if you need a image or a job. You might use the other stuff, but only if you want/need. You would also be able to keep a 1:1 ratio of the tasks you will actually be able to run with the number of resource you wrote.

To do that we need to

1. Implement local-mount field on a job, seem easy.
2. Implement a image field tha would embed a image type inside a job to run instead of use field, which sounds way trickier.

Proposal: Add "browser" resource

The browser resource would open a tab in the default browser with the given url. This would only be useful if there was some way to pass along the url with container port from a previous job step.

Code cleanup/refactor

I'd like to make the following large changes:

• replace https://github.com/dnephin/configtf with https://github.com/mitchellh/mapstructure for struct transforms. dnephin/configtf would still be used for documentation
• use github.com/docker/docker/client instead of github.com/fsouza/go-dockerclient
• expand the Task interface:
  • include IsFresh() bool so that freshness check is separate from Run(). This will allow support for #73.
  • create an emptyTask to implement no-op defaults for each step
  • better code re-use between tasks in different packages

Cannot remove artifact of job

When running the examples/minimal-docker-image project, I wasn't able to follow the instructions as currently written in the README.md. I submitted a quick fix/workaround PR there, but it seems there are a couple of underlying bugs I'll also raise.

In that example, there is a job task with the name "binary". It uses the builder image and produces the artifact dist/bin/hello.

If you try to run dobi binary:rm, it fails with the following warning:

[WARN] [run:rm binary] ./dist/bin/hello container=example-hello-dre-binary Failed to remove container: No such container: example-hello-dre-binary
(where "dre" is my $HOST_USER)

Looks like it is mistakenly trying to target a container rather than the artifact. If I look into this and fix it, I'm assuming that the binary:rm shouldn't cascade down to the builder image and rm that as well, right?

[Windows] Can't bind "/var/run/docker.sock"

https://github.com/dnephin/dobi/blob/master/tasks/job/run.go#L301

This file doesn't exist on Windows. Should there be a meta.provide_docker: option to disable?

Proposal: a config file to restrict actions/paths used by dobi

dobi can be used for CI with systems like circle/travis that already take care of providing a secured environment to run the tests.

If dobi is used with something like a public jenkins instance a PR could alter the dobi.yaml to do malicious things. The purpose of this "restrictions" config file would be to prevent any malicious actions from a PR.

Supported restrictions would include:

• prevent the use of some job options (ex: privileged, provide-docker, etc)
• restrict mounts to certain bind paths (ex: only the workspace directory)
• restrict image names to some predefined name or a specific repo (so that a user couldn't image:rm a predefined image, or override an image). Alternatively, maybe just force all the images to have a specific repo/ prefix to prevent conflicts.
• restrict the fields in any Compose files (similar to the restrictions on mounts and jobs above)
• prevent meta.include from outside the workspace directory (so that users can't fish for files)
• possibly restrict some config variables (ex: some environment variables)

Format:

meta:
  include_paths:
    allow: [...]
    disallow: [...]

variables:
  allow: [...]
  disallow: [...]

image:
  allow: [...]
  disallow: [...]
...

The allow/disallow lists should support:

• using {fs.workspace} (or maybe just use env?)
• prefix matching for image names

dobi --restrictions dobi-restrictions.yaml

[Windows] domain users create syntax error during image creation

[ERROR] Failed to execute task "build-img:build": API error (500): {"message":"Error parsing reference: \"<project-name>-build:<project-name>-<ad_domain>\\\\<username>\" is not a valid repository/tag: invalid reference format"}

Since my employer uses AD, my username causes a syntax error in the image name.

Versioning Golang app

The dobi repo itself holds its version inside cmd/dobi.go just like most go apps.
How can I use that specific version as an environment variable so I can push images accordingly.
With a Makefile is quite easy.

export VERSION:=$(shell go run src/backend/api.go version)

deploy:
    @echo Building version $(VERSION)
    docker-compose push production
    ...

I want to avoid setting it manually like VERSION=0.1.4 dobi production:push

job=binary:
    use: builder
    artifact: dist/
    mounts: [source,dist]
    command: go install src/backend/api.go
    working-dir: /go/src/bitbucket.org/cescoferraro/api
    description: "Build the static binary"

image=production:
    image: cescoferraro/api
    tags: ["v{env.VERSION}"]
    dockerfile: docker/Dockerfile.prod
    depends: [binary]
    description: "Production Image"

Feature: allow for inclusion of subproject/dobi.yaml

dobi.yaml allows for includes, but those includes are handled as "config fragments". All the relative paths in the included file are relative to the dobi.yaml which includes the file.

In a subproject setup, the paths need to be adjusted so that they are relative to the directory of the file that references them.

I'd like to support both use cases.

My proposal is to add a new field to meta: called subproject: which would act like include: but would handle rewriting all the "known" paths to be relative to file which references them. The fields which include known paths are:

• image.context
• job.artifact
• job.sources
• mount.bind
• compose.files
• meta.include
• env.files

Other paths (like image.dockerfile, job.command, etc) are already relative to a context directory, so don't need to be modified.

Need a way to specify a port mapping

If a job is created to be a "run" type task such as starting up a development web server, I don't believe there is currently any way to actually map that port to the host so you could do anything with the container.

Maybe rather than having a specific declaration just for ports, it would make sense to add a more generic "docker_args" declaration for jobs that would let the developer specify any particular extra args that should be passed through?

Should the `run` resource be renamed?

Most resources are nouns (mount, image, etc).

The run resource could be called container, but that's a bit long, and it's also not really relevant. The resource is about performing some task, not about starting a container. task is already overloaded (swarm task, and also resources provide tasks).

These resources are often about creating some file or files (artifact), so something related to that might be a good name.

Some options, in no particular order:

• process (still kind of long)
• job (fairly common term, but still my top choice right now)
• exec
• action (would need to rename the "action" part of tasks)

Document how to use dobi on common CI/CD system

• travis - https://github.com/aanand/compose-file/pull/18/files
• circle - https://github.com/dnephin/configtf/blob/master/circle.yml
• gitlab ci - https://gitlab.com/d.nephin/dobi/blob/try-gitlab-ci/.gitlab-ci.yml
• jenkins
• drone?

Add elapsed time of task to verbose log

Elapsed time for each task might be useful. If it can be parsed out of the log easily that would be good.

[Windows] Image build fails due to vendor bug

fsouza/go-dockerclient#589

Proposal: compose hooks

This is less of a proposal and more of a request for comments.

From what I can tell, dobi is the answer to a lot of questions I had about using Docker with compiled languages. I couldn't figure out a clean way to create builder Dockerfiles and still use Compose the way I wanted to.

What do y'all think about making a PR to docker compose to include build hooks to run dobi tasks before up?

Ex (in v1 for brevity):

# docker-compose.yml
app:
   image: example/hello
   hooks:
       before_up: dobi dist-img

# dobi.yml
image=builder:
    image: minimal-dev
    context: .
    dockerfile: Dockerfile.build

job=binary:
    use: builder
    artifact: ./dist/bin/hello
    mounts: [source, dist]
    description: "Build the static binary"

image=dist-img:
    image: example/hello
    dockerfile: Dockerfile.dist
    tags: ['{user.name}']
    depends: [binary]
    description: "Build the distribution image"

With the above, all would be required of the developer would be a docker-compose up. DC would call out to dobi for a build, and then run the appropriate image.

Run task as non-root

Is there any way to get a Run task to execute as non-root? Currently, there doesn't appear to be a way to set the user for the run task like you can with docker via the -u command. I have tried setting the USER value in my Dockerfile but the run task still seems to execute as root.

Support git-describe as well as git sha Config Variable

git-describe returns a more descriptive short reference to the current git commit than a plain SHA. If the commit has a tag, it will return the just the tags name. If not, it will return the nearest tag from the history, followed by the number of commits since that tag, followed by the short sha.

These make nicer tags than plain SHAs as you can guess the git-tagged ones (i.e. released versions) directly, and the others will generally sort in numerical order behind the release they are based on.

Errors when bootstrapping dobi

Following the instructions on https://dnephin.github.io/dobi/install.html to clone the repo and run the docker command to generate binaries, I get a bunch of errors. I'm running with Docker for Mac v 1.12.1-beta24.1.

The very first sign of trouble is a warn:

Status: Downloaded newer image for dnephin/dobi:0.5
[WARN] Failed to load auth config: open /.dockercfg: no such file or directory
[image:build builder] dobi-dev Building

I'm not sure why it would be looking for a .dockercfg at all, especially in the root file system. My auth config is in the standard location, ~/.docker/config.json

It then plugs along for a while seemingly okay until:

Step 9 : ENV CGO_ENABLED 0

It downloads about 20 dependencies fine, up to:

[INFO] --> Fetching updates for github.com/docker/go-units.

and then it starts erroring for most the rest of them:

[WARN] Unable to checkout github.com/inconshreveable/mousetrap
[ERROR] Update failed for github.com/inconshreveable/mousetrap: Unable to get repository
[WARN] Unable to checkout github.com/docker/engine-api
[ERROR] Update failed for github.com/docker/engine-api: Unable to get repository
[WARN] Unable to checkout github.com/Sirupsen/logrus
[ERROR] Update failed for github.com/Sirupsen/logrus: Unable to get repository
[WARN] Unable to checkout github.com/fsouza/go-dockerclient
[INFO] --> Fetching updates for gopkg.in/yaml.v2.
[INFO] --> Fetching updates for golang.org/x/sys.
[ERROR] Update failed for github.com/fsouza/go-dockerclient: Unable to get repository
[WARN] Unable to checkout github.com/spf13/pflag
[ERROR] Update failed for github.com/spf13/pflag: Unable to get repository
[WARN] Unable to checkout github.com/kballard/go-shellquote
[ERROR] Update failed for github.com/kballard/go-shellquote: Unable to get repository
[WARN] Unable to checkout github.com/hashicorp/go-cleanhttp
[ERROR] Update failed for github.com/hashicorp/go-cleanhttp: Unable to get repository

Finally, it fails all together with:

An Error has occurred
[ERROR] Failed to execute task "deps:run": Exited with non-zero status code 2

Pulling images doesn't seem to work

dobi.yaml:

...

image=go-swagger:
    image: "quay.io/goswagger/swagger:0.6.0"
    pull: once

job=go-swagger-validate:
    use: go-swagger
    command: "validate /code/swagger.yaml"
    ...

$ dobi go-swagger-validate
[image:build go-swagger] quay.io/goswagger/swagger:0.6.0 Building
[ERROR] Failed to execute task "go-swagger:build": missing remote repository e.g. 'github.com/user/repo'

There's no example of how to do this, so I'm guessing this is correct. Either I've got it wrong and an example is needed, or it's actually broken.

Proposal: "Context"-aware resources

There are two conditions which result in sub-optimal behaviour:

• an env resource can modify the behaviour of a task, which requires all subsequent tasks be marked as "required" (they can't be skipped)
• compose resources (and the proposed service and job:serve resources) create long running processes, so they must always run. There's no way to skip these tasks without doing some kind of look-ahead, but there isn't current enough information in the config format to determine how far to look ahead. It's not clear which of the following steps use the service.

I think both of these issues could be addressed by adding some form of a "context" (inspired by python context managers).

This config construct would provide a scope for the above resources, making it possible to skip long-running tasks (if all the tasks inside the scope are fresh), and only "require" tasks within the environment scope.

Watch Resource

I havent used that many build tools. But tools like gulp/webpack/grunt all have the concept of watching the source code for changes and reacting to it. I have been noticing this patterns in all dobi projects I have been working on lately.

I think dobi could offer a way for us to have better control over this process and avoid unnecessary files.
I have been using https://github.com/cespare/reflex which it has proven to be the fastest cross platform go implementation and on top of that you could spun multiple watchs task with different reaction commands by simply creating a text file. If I could make a PR to make reflex read from a stdout pipe instead of an actual file this should be easy to implement.

We would also need a way to download the right reflex binary into whatever image the user want to use, or maybe make it a requirement, then check if the image contains reflex on the path at runtime before proceeding watching the files.

// This resource would run  `go run` whernever a .go file changes, 
// as well as running `node server.js` whenever any .tsx  or 
// ends with .config.js change
watch=dev:
    use: cescoferraro/project:development
    reflex:
      - global: '*.config.js'
        regex: '\.tsx'
        cmd: node server.js
      - regex: '\.go'
        cmd: go run main.go

Support new `docker build` features

It would be great if the image resource could support:

• cache-from
• network
• target

Run unit-tests for windows on Appveyor CI

It would be good to run the unit tests on windows.

Mount Devices

Trying to mount an arduino to a container runned by dobi Fails

mount=device:
    bind: /dev/ttyACM0
    path: /dev/ttyACM0

job=watch:
    use: watch-builder
    interactive: true
    net-mode: host
    ports: ["5000:5000"]
    mounts: [source,machine-id,device]
    env:
      - "POWER_JWT={env.POWER_JWT:}"
    description: "Watch for code changes and run the unit tests"

The same container created by dobi, run with plain docker using the --device flag:

cesco@cesco-desktop: ~/code/go/src/github.com/cescoferraro/power on master [+!?]
$ docker run -p 5000:5000 -v /var/lib/dbus/machine-id:/var/lib/dbus/machine-id  -v $PWD:/go/src/github.com/cescoferraro/power --device /dev/ttyACM0 --net=host  cescoferraro/power:watch 
[00] Starting service
[00] 2016/10/22 00:38:52 Failed to read config Unsupported Config Type ""
[00] 2016/10/22 00:38:52 verbose enabled
[00] 2016/10/22 00:38:52 ***********VIPER*************
[00] 2016/10/22 00:38:52 ping: true
[00] 2016/10/22 00:38:52 channels: 8
[00] 2016/10/22 00:38:52 owner: [email protected]
[00] 2016/10/22 00:38:52 api: https://api.cescoferraro.xyz/iot/devices
[00] 2016/10/22 00:38:52 dev-api: http://localhost:9000/iot/devices
[00] 2016/10/22 00:38:52 serial_port: /dev/ttyACM0

I am trying to set a single device on job/run.go and it works like this. I was wondering how should we implement this. ["host:container:cgroup"] or ["same:cgroup"] or ["same"] with full permissions?

job=binary:
    use: builder
    artifact: ./dist/bin/
    devices: "/dev/ttyACM0"


    device := docker.Device{
        PathInContainer:t.config.Devices,
        PathOnHost:t.config.Devices,
        CgroupPermissions:"rwm"}

    portBinds, exposedPorts := asPortBindings(t.config.Ports)
    // TODO: only set Tty if running in a tty
    opts := docker.CreateContainerOptions{
        Name: name,
        Config: &docker.Config{
            Cmd:          t.config.Command.Value(),
            Image:        imageName,
            User:         t.config.User,
            OpenStdin:    interactive,
            Tty:          interactive,
            AttachStdin:  interactive,
            StdinOnce:    interactive,
            AttachStderr: true,
            AttachStdout: true,
            Env:          t.config.Env,
            Entrypoint:   t.config.Entrypoint.Value(),
            WorkingDir:   t.config.WorkingDir,
            ExposedPorts: exposedPorts,
        },
        HostConfig: &docker.HostConfig{
            Binds:        t.bindMounts(ctx),
            Privileged:   t.config.Privileged,
            NetworkMode:  t.config.NetMode,
            Devices:  []docker.Device{device},
            PortBindings: portBinds,
        },
    }

CI Linter

My box

cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/dobifile [?]
$ go run main.go lint
Step 1 : FROM golang:1.7-alpine
....
[check-merge-conflict] Check for merge conflicts.........................Passed
[end-of-file-fixer] Fix End of Files.....................................Passed
[trailing-whitespace] Trim Trailing Whitespace...........................Passed
[check-yaml] Check Yaml..................................................Passed
[check-json] Check JSON..............................(no files to check)Skipped
[go-fmt] go fmt..........................................................Passed
[go-lint] go lint........................................................Passed
[go-vet] go vet..........................................................Passed
[job: lint] Done

But my last builds have been sequentially failing to lint for those reasons.

tasks/task/stack.go:32:29: error strings should not be capitalized or end with punctuation or a newline
utils/fs/expanduser.go:25:27: error strings should not be capitalized or end with punctuation or a newline

A quick search shows that dobi has got quite a few of them.I have fixed the first one on the photo

Additional config variables

This issue is a tracker for potential additions to the config variables provided by dobi.

Mercurial variables

• hg.sha
• hg.branch
• hg.short-sha

Filesystem variables

• fs.cwd - current working directory (0.6)
• fs.projectdir - the directory which contains the dobi.yaml (0.6)

User

• user.name, user.group - the username and group of the current user (0.8)
• user.uid, user.gid - the uid and gid of the current user (0.8)
• user.home - the home directory of the user (0.8)

System

• sys.os - the operating system name (linux, windows, darwin, etc)
• sys.kernel - the kernel version
• sys.arch - the system architecture (amd64, arm, 386, etc)

Resource

• res.self.<field> - use the value of one field in another, ex: the name of the resource could be the artifact path as well
• res.<name>.<field> - reference fields of other resources. This would create an implicit dependency from one resource to another, forcing the default action for that resource to run first (unless it's configured as a dependency).

env var on a job command

maybe I am missing a way to skip those chars

image=busybox:
    image: busybox
    tags: ["latest"]
    pull: "once"

job=play:
    use: busybox
    interactive: true
    command: env
    depends: [production-vars]
    env:
      - "VERSION={env.VERSION}"
    description: "bla"


job=play:
    use: busybox
    interactive: true
    command: echo $VERSION
    depends: [production-vars]
    env:
      - "VERSION={env.VERSION}"
    description: "bla"

cescoferraro@desktop: ~/code/go/src/github.com/cescoferraro/api on develop [+!?$]
$ dobi play
[env:set production-vars] Done
[job: play] env Start
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=1b44e5e5e282
TERM=xterm
VERSION=0.2.5
HOME=/root
[job: play] env Done
cescoferraro@desktop: ~/code/go/src/github.com/cescoferraro/api on develop [+!?$]
$ dobi play
[env:set production-vars] Done
[job: play] echo ${VERSION} Start
${VERSION}
[job: play] echo ${VERSION} Done

Autoclean bugs & design

• Autoclean command should not require a variable.
• Autoclean command should force remove a container to avoid things like You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

If autoclean is meant to be project specific, maybe there is room for another command.
Maybe dobi format that should

• not be project specific
• stop/remove all running containers created by dobi?
• remove exited containers.

cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ docker ps -a | grep cescoferraro-
d8c9d51c44f7        cescoferraro/mongo:latest   "/run.sh"                 8 hours ago         Up 8 hours                27017/tcp, 28017/tcp   dobi-cescoferraro-serve
d43f786a2c7c        cescoferraro/api:tools      "/usr/bin/reflex -r \\"   41 hours ago        Exited (0) 14 hours ago                          api-cescoferraro-watch
850158394fbc        cescoferraro/power:watch    "docker/script/watch"     42 hours ago        Exited (0) 14 hours ago                          power-cescoferraro-watch
6e77373d6526        cescoferraro/iot:base       "ionic serve --all"       2 days ago          Exited (0) 14 hours ago                          iot-cescoferraro-watch


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ dobi autoclean
[job:rm watch]  Removed
[job:rm test-unit]  Removed
[job:rm test-examples]  Removed
[job:rm test-examples]  Removed
[job:rm test-unit]  Removed
[alias:rm test] Done
[WARN] [mount:rm source] .:/go/src/github.com/dnephin/dobi Bind mounts are not removable
[job:rm shell]  Removed
[WARN] container=dobi-cescoferraro-serve [job:rm serve]  Failed to remove container: API error (409): You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

[job:rm serve]  Removed
[WARN] [image:rm releaser] dobi-release failed to remove "dobi-release:dobi-cescoferraro": no such image
[image:rm releaser] dobi-release Removed
[job:rm release-version]  Removed
[ERROR] A value is required for variable "env.DOBI_VERSION"


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ VERSION=22 go run main.go autoclean
[job:rm watch] {[]} Removed
[job:rm test-unit] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-unit] {[]} Removed
[alias:rm test] Done
[WARN] [mount:rm source] .:/go/src/github.com/dnephin/dobi Bind mounts are not removable
[job:rm shell] {[]} Removed
[WARN] [job:rm serve] {[]} container=dobi-cescoferraro-serve Failed to remove container: API error (409): You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

[job:rm serve] {[]} Removed
[WARN] [image:rm releaser] dobi-release failed to remove "dobi-release:dobi-cescoferraro": no such image
[image:rm releaser] dobi-release Removed
[job:rm release-version] {[]} Removed
[ERROR] A value is required for variable "env.DOBI_VERSION"
exit status 1


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ DOBI_VERSION=234 VERSION=22 go run main.go autoclean
[job:rm watch] {[]} Removed
[job:rm test-unit] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-unit] {[]} Removed
[alias:rm test] Done
[WARN] [mount:rm source] .:/go/src/github.com/dnephin/dobi Bind mounts are not removable
[job:rm shell] {[]} Removed
[WARN] [job:rm serve] {[]} container=dobi-cescoferraro-serve Failed to remove container: API error (409): You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

[job:rm serve] {[]} Removed
[WARN] [image:rm releaser] dobi-release failed to remove "dobi-release:dobi-cescoferraro": no such image
[image:rm releaser] dobi-release Removed
[job:rm release-version] {[]} Removed
[WARN] [image:rm dist-img] dnephin/dobi failed to remove "dnephin/dobi:234": no such image
[image:rm dist-img] dnephin/dobi Removed
[job:rm release-version] {[]} Removed
[ERROR] A value is required for variable "env.GITHUB_TOKEN"
exit status 1


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ GITHUB_TOKEN=2 DOBI_VERSION=234 VERSION=22 go run main.go autoclean
[job:rm watch] {[]} Removed
[job:rm test-unit] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-unit] {[]} Removed
[alias:rm test] Done
[WARN] [mount:rm source] .:/go/src/github.com/dnephin/dobi Bind mounts are not removable
[job:rm shell] {[]} Removed
[WARN] [job:rm serve] {[]} container=dobi-cescoferraro-serve Failed to remove container: API error (409): You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

[job:rm serve] {[]} Removed
[WARN] [image:rm releaser] dobi-release failed to remove "dobi-release:dobi-cescoferraro": no such image
[image:rm releaser] dobi-release Removed
[job:rm release-version] {[]} Removed
[WARN] [image:rm dist-img] dnephin/dobi failed to remove "dnephin/dobi:234": no such image
[image:rm dist-img] dnephin/dobi Removed
[job:rm release-version] {[]} Removed
[ERROR] A value is required for variable "env.CIRCLE_TAG"
exit status 1


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ GITHUB_TOKEN=2 DOBI_VERSION=234 VERSION=22 CIRCLE_TAG=22 go run main.go autoclean
[job:rm watch] {[]} Removed
[job:rm test-unit] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-unit] {[]} Removed
[alias:rm test] Done
[WARN] [mount:rm source] .:/go/src/github.com/dnephin/dobi Bind mounts are not removable
[job:rm shell] {[]} Removed
[WARN] [job:rm serve] {[]} container=dobi-cescoferraro-serve Failed to remove container: API error (409): You cannot remove a running container d8c9d51c44f7323114923f0c0dcaa85b8f14b08e5f49450a35aa63b1d11c3a6d. Stop the container before attempting removal or use -f

[job:rm serve] {[]} Removed
[WARN] [image:rm releaser] dobi-release failed to remove "dobi-release:dobi-cescoferraro": no such image
[image:rm releaser] dobi-release Removed
[job:rm release-version] {[]} Removed
[WARN] [image:rm dist-img] dnephin/dobi failed to remove "dnephin/dobi:234": no such image
[image:rm dist-img] dnephin/dobi Removed
[job:rm release-version] {[]} Removed
[job:rm github-release] {[]} Removed
[job:rm binary] {[./dist/bin/]} Removed
[job:rm deps] {[vendor/]} Removed
[alias:rm release] Done
[WARN] [image:rm refactorer] dnephin/go-refactor failed to remove "dnephin/go-refactor:latest": no such image
[image:rm refactorer] dnephin/go-refactor Removed
[job:rm refactor-shell] {[]} Removed
[WARN] [mount:rm projectdir] .:/home/cescoferraro/code/go/src/github.com/dnephin/dobi Bind mounts are not removable
[image:rm mongo] cescoferraro/mongo Removed
[job:rm mocks] {[]} Removed
[image:rm linter] dobi-linter Removed
[job:rm lint] {[]} Removed
[job:rm github-release] {[]} Removed
[WARN] [image:rm example-tester] dobi-test-examples failed to remove "dobi-test-examples:dobi-cescoferraro": no such image
[image:rm example-tester] dobi-test-examples Removed
[job:rm docs-watch] {[]} Removed
[job:rm docs-shell] {[]} Removed
[WARN] [image:rm docs-img] dobi-docs-dev failed to remove "dobi-docs-dev:dobi-cescoferraro": no such image
[image:rm docs-img] dobi-docs-dev Removed
[job:rm docs-build] {[./docs/build/html]} Removed
[job:rm docs] {[]} Removed
[WARN] [image:rm dist-img] dnephin/dobi failed to remove "dnephin/dobi:234": no such image
[image:rm dist-img] dnephin/dobi Removed
[WARN] [mount:rm dist] ./dist/bin/:/go/bin/ Bind mounts are not removable
[job:rm deps] {[vendor/]} Removed
[job:rm binary] {[./dist/bin/]} Removed
[job:rm docs-build] {[./docs/build/html]} Removed
[job:rm test-unit] {[]} Removed
[job:rm lint] {[]} Removed
[alias:rm ci-circle-main] Done
[job:rm test-examples] {[]} Removed
[alias:rm ci-circle-examples] Done
[image:rm builder] dobi-dev Removed
[job:rm binary] {[./dist/bin/]} Removed
[alias:rm binary-linux] Done
[job:rm binary] {[./dist/bin/]} Removed
[job:rm binary] {[./dist/bin/]} Removed
[job:rm docs-build] {[./docs/build/html]} Removed
[job:rm test-examples] {[]} Removed
[job:rm test-unit] {[]} Removed
[alias:rm test] Done
[job:rm lint] {[]} Removed
[alias:rm all] Done


cescoferraro@desktop: ~/code/go/src/github.com/dnephin/dobi on feature/serve [?]
$ docker ps -a | grep cescoferraro-
d8c9d51c44f7        c8cd8c23fe0e                "/run.sh"                 8 hours ago         Up 8 hours                27017/tcp, 28017/tcp   dobi-cescoferraro-serve
d43f786a2c7c        cescoferraro/api:tools      "/usr/bin/reflex -r \\"   41 hours ago        Exited (0) 15 hours ago                          api-cescoferraro-watch
850158394fbc        cescoferraro/power:watch    "docker/script/watch"     42 hours ago        Exited (0) 15 hours ago                          power-cescoferraro-watch
6e77373d6526        cescoferraro/iot:base       "ionic serve --all"       2 days ago          Exited (0) 15 hours ago                          iot-cescoferraro-watch

Add a flag that turns `mounts` into an image layer

Some environments do not support bind mounts (ex: a daemon on a different host).
Some use cases benefit from having an artifact that captures the state of files used by a task.

To support both of these, dobi should have a flag that tells it to replace mounts with an added COPY . . layer on the image used by the job. The mount becomes a no-op because the code is already in the image.

In the future hopefully there will be better docker support for toggling between these cases.

Provide additional install options

Some options:

• rpm
• deb
• snap
• flatpak

Support named volumes

In addition to host bind mounts

Proposal: improve multi-arch builds

Projects that want to support multiple architectures often end up with many copies of a very similar Dockerfile , but with minor variations for the platform (notably the base image will be different). docker/docker is an example of such a project.

It would be nice to provide a way to remove that boilerplate and make it easy to build a binary for different architectures.

One option would be to use a job to render a Dockerfile from a template, but that won't work unless you already have some way to pull a multi-arch image.

Another option might be to provide templating for the Dockerfile from dobi itself.

Allow tagging with a repository name after the build

In an image, you must give the repository name in the image: attribute, and any entries in the tags: attribute are only appended to the repository name as "version tags".

In non-dobi builds, we build with an initial simple short name:
docker build -t myco/myapp .
and then afterward, we tag it for the registry:
docker tag myco/myapp 123532365.aws.com/ecr/myco/myapp:latest

but I'm not able to do this in Dobi, I have to give the long tag name as the image name.

Add "env" resource

An env resource is a list of environment variables.

By itself it doesn't do much, but as a step in an alias, it can be used to fulfill variables required by build steps later in the list. That would allow for a single build step to work differently based on the env resource that appears before it in the alias task list.

Possibly also allow reading values from a file.

How is this going to work with resolving variables at config time?

Authentication does not seem to be working quite right

First Off, awesome tool...the amount of scripting that I can replace with this tool is amazing!

When I try to build an image that requires one of our private images as a base image the authentication is not working:

$ docker login -u <my-user> -p <my-pass> <my-url>
Login Succeeded
$ dobi
Step 1/10 : FROM <company>/<project>/alpine-docker:17.03.0-ce
[ERROR] failed to execute task "<task>:build": Get <image-url>: unknown: Authentication is required
$ docker pull <company>/<project>/alpine-docker:17.03.0-ce
17.03.0-ce: Pulling from <project>/alpine-docker
ec37562cf8fa: Already exists 
371cc900ff27: Pull complete 
51ad1472b975: Pull complete 
76c902807740: Pull complete 
Digest: sha256:c5ca1b30d6731ac39df368936b03a83824be99d98fb202a12beb55ad7b4ac541
Status: Downloaded newer image for <company>/<project>/alpine-docker:17.03.0-ce

dobi.yaml

meta:
  project: <project name>
  default: build

image=<image name>:
  image: <company>/<project>/<image name>
  context: <directory>
  tags:
    - 'latest'
    - '{git.short-sha}'
  description: "Build the <image name> image"

alias=build:
  tasks: ['<image name>:build']
  description: "Build images"

alias=tag:
  tasks: ['<image name>:tag']
  description: "Build and tag images"

alias=push:
  tasks: ['<image name>:push']
  description: "Build, tag, and push images"

alias=clean:
  tasks: ['<image name>:rm']
  description: "Remove all the images"

My question is why is dobi not using the same auth mechanism as docker? And if it is then it doesn't seem to be exactly the same.

Provide arm binaries

I think it would be nice if we could provide arm binaries for the next release.
Gox does not support all arm arch, as of right now
It think you know that. I have used https://github.com/franciscocpg/gox lately and it works great.

We just need to change this dep

https://github.com/dnephin/dobi/blob/master/dockerfiles/Dockerfile.build#L16

But the build script need a little change

https://github.com/dnephin/dobi/blob/master/script/build#L7

Maybe

gox \
    -ldflags "-X github.com/dnephin/dobi/cmd.gitsha=$gitsha \
              -X \"github.com/dnephin/dobi/cmd.buildDate=$builddate\"" \
    -output="/go/bin/dobi-{{.OS}}/{{.Arch}}" \
    -arch="amd64 armv5 armv6 armv7" -os="$oslist"   \
    .

image:push authentication required

cesco@cesco-desktop: ~/code/go/src/bitbucket.org/cescoferraro/api on develop [+$]
$ VERSION=0.1.2 dobi  production:push
[WARN] Missing auth config for "https://golang/v1"
1.7.1-alpine: Pulling from library/golang
Digest: sha256:03ca8be2e13e3f83af9ce3fa181f88bddd39e83f1d2714ae7153620e9f051069
Status: Image is up to date for golang:1.7.1-alpine
[image:pull base] golang Pulled
[job:run binary] go build src/backend/api.go Start
[job:run binary] go build src/backend/api.go Done
Step 1 : FROM alpine:3.4
 ---> ee4603260daa
Step 2 : RUN apk add --no-cache ca-certificates apache2-utils
 ---> Using cache
 ---> 45118353fd77
Step 3 : ADD ./api /srv/app/api
 ---> Using cache
 ---> b53fd7b4f556
Step 4 : WORKDIR /srv/app
 ---> Using cache
 ---> a4d7f6a52822
Step 5 : CMD ./api runserver
 ---> Using cache
 ---> 94d98e1b5743
Step 6 : EXPOSE 9000
 ---> Using cache
 ---> 11fddc5aa441
Successfully built 11fddc5aa441
[image:build production] cescoferraro/api Created
[image:tag production] cescoferraro/api Tagged
[WARN] Missing auth config for "https://cescoferraro/v1"
The push refers to a repository [docker.io/cescoferraro/api]
328ac6e90ae9: Preparing 
b4b945d9e06c: Preparing 
9007f5987db3: Preparing 
[ERROR] Failed to execute task "production:push": unauthorized: authentication required
cesco@cesco-desktop: ~/code/go/src/bitbucket.org/cescoferraro/api on develop [+$]
$ docker push cescoferraro/api:v0.1.2
328ac6e90ae9: Pushed 
328ac6e90ae9: Pushing [===================================>               ] 11.14 MB/15.65 MB
b4b945d9e06c: Layer already exists 
v0.1.2: digest: sha256:7f551ac2ac64ba3535eaef411c0c4fd6ba729351ee83ff1b886ad8b48f23aa36 size: 949

dobi.yaml

meta:
    project: api
    default: production

mount=source:
    bind: .
    path: /go/src/bitbucket.org/cescoferraro/api

image=base:
    image: golang
    tags: [1.7.1-alpine]
    pull: "always"

job=binary:
    use: base
    mounts: [source]
    command: go build src/backend/api.go
    working-dir: /go/src/bitbucket.org/cescoferraro/api
    description: "Build the static binary"

image=production:
    image: cescoferraro/api
    tags: ["v{env.VERSION}"]
    dockerfile: docker/Dockerfile.prod
    depends: [binary]
    description: "Production Image"

image record should use the same modified time for both build and pull

Right now the image record has a separate modified time for build and pull. By using the same field we can support a workflow where image:pull is used to skip the build on CI.

Fail to push image from CI job

I try to use dobi from CircleCI.
So far, I can create required images, run tests, and other tasks. Now I want to tag and push image into DockerHub
I run:

$ dobi -v dist-img:tag
# all images tagged as expected, I can see them with `docker images` command
$ dobi -v dist-img:push
# push fails with error bellow ...

[ERROR] Failed to execute task "dist-img:push": API error (500): repository name component must match "[a-z0-9](?:-*[a-z0-9])*(?:[._][a-z0-9](?:-*[a-z0-9])*)*"

Regular docker push command works just fine from CI ssh prompt. And running dobi dist-img:push locally also works without any problem.

You can take a look at my project https://github.com/gaia-adm/pumba develop branch

Did you see this error before?

`image:push` runs the build multiple times when used with an `env` resource

env=vars:
    variables:
      - FRONTEND_VERSION=0.1.7
      - API_VERSION=0.1.17

job=api:
    description: "Builds API docker image"
    use: dev-image
    artifact: dist/spotify
    user: "{user.uid}"
    mounts: [source]
    interactive: true
    command:   bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go"
    depends: [vars]
    env:
      - "API_VERSION={env.API_VERSION}"
      - "GODEBUG=http2client=0"

image=api-image:
    image: cescoferraro/spotify
    tags: ["api-{env.API_VERSION}"]
    dockerfile: build/docker/Dockerfile.api
    depends: [vars, api]
    description: "Build the api-image"

Build

cescoferraro@mint-pro: ~/code/go/src/github.com/cescoferraro/spotify on master [+!$]
$ dobi api-image
[env:set vars] Done
[image:build dev-image] cescoferraro/spotify is fresh
[env:set vars] Done
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Start
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Done
Step 1 : FROM golang
latest: Pulling from library/golang
Digest: sha256:5787421a0314390ca8da11b26885502b58837ebdffda0f557521790c13ddb55f
Status: Downloaded newer image for golang:latest
 ---> 6639f812dbc7
Step 2 : MAINTAINER Francesco Ferraro <[email protected]>
 ---> Running in fcfbc3a6a97c
 ---> 889dd4c3c51b
Removing intermediate container fcfbc3a6a97c
Step 3 : ADD www/ www
 ---> d76564cf30e4
Removing intermediate container 52a6c19a1574
Step 4 : ADD dist/spotify spotify
 ---> bf693f142553
Removing intermediate container 4d65fd8dd41e
Step 5 : ENV PORT 8080
 ---> Running in d8fda0735f3c
 ---> 7e6ddde0cb70
Removing intermediate container d8fda0735f3c
Step 6 : EXPOSE 8080
 ---> Running in 89a83d11b3f2
 ---> c949bc3da0b7
Removing intermediate container 89a83d11b3f2
Step 7 : ENTRYPOINT /spotify
 ---> Running in 8c99cc99d277
 ---> 16c0a85747ca
Removing intermediate container 8c99cc99d277
Successfully built 16c0a85747ca
[image:build api-image] cescoferraro/spotify Created

Push

cescoferraro@mint-pro: ~/code/go/src/github.com/cescoferraro/spotify on master [+!$]
$ dobi api-image:push
[env:set vars] Done
Step 1 : FROM golang:1.7.4
 ---> 6639f812dbc7
Step 2 : RUN apt-get install -y curl
 ---> Using cache
 ---> cbb7f2130c36
Step 3 : RUN curl -sL https://deb.nodesource.com/setup_6.x -o nodesource_setup.sh
 ---> Using cache
 ---> c99cfa54bfb2
Step 4 : RUN bash nodesource_setup.sh
 ---> Using cache
 ---> 9e5fb9e5d712
Step 5 : RUN apt-get install -y nodejs
 ---> Using cache
 ---> beb37f2c6cb4
Step 6 : RUN apt-get install -y build-essential
 ---> Using cache
 ---> d7fa22c4dd89
Step 7 : RUN go get github.com/cespare/reflex &&         cp /go/bin/reflex /usr/bin/ &&         rm -rf /go/src/* /go/pkg/* /go/bin/*
 ---> Using cache
 ---> 937d6028e42e
Step 8 : ENV PS1 "# "
 ---> Using cache
 ---> b7f4d2ff45c1
Step 9 : ENV GOBIN /go/binw
 ---> Using cache
 ---> d59cce53944b
Step 10 : ADD package.json package.json
 ---> Using cache
 ---> 8b373f48ed53
Step 11 : ENV CGO_ENABLED 0
 ---> Using cache
 ---> 4ec6c4327572
Step 12 : WORKDIR /go/src/github.com/cescoferraro/spotify
 ---> Using cache
 ---> a418964be539
Successfully built a418964be539
[image:build dev-image] cescoferraro/spotify Created
[env:set vars] Done
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Start
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Done
Step 1 : FROM golang
 ---> 6639f812dbc7
Step 2 : MAINTAINER Francesco Ferraro <[email protected]>
 ---> Using cache
 ---> 889dd4c3c51b
Step 3 : ADD www/ www
 ---> Using cache
 ---> d76564cf30e4
Step 4 : ADD dist/spotify spotify
 ---> Using cache
 ---> bf693f142553
Step 5 : ENV PORT 8080
 ---> Using cache
 ---> 7e6ddde0cb70
Step 6 : EXPOSE 8080
 ---> Using cache
 ---> c949bc3da0b7
Step 7 : ENTRYPOINT /spotify
 ---> Using cache
 ---> 16c0a85747ca
Successfully built 16c0a85747ca
[image:build api-image] cescoferraro/spotify Created
[env:set vars] Done
Step 1 : FROM golang:1.7.4
 ---> 6639f812dbc7
Step 2 : RUN apt-get install -y curl
 ---> Using cache
 ---> cbb7f2130c36
Step 3 : RUN curl -sL https://deb.nodesource.com/setup_6.x -o nodesource_setup.sh
 ---> Using cache
 ---> c99cfa54bfb2
Step 4 : RUN bash nodesource_setup.sh
 ---> Using cache
 ---> 9e5fb9e5d712
Step 5 : RUN apt-get install -y nodejs
 ---> Using cache
 ---> beb37f2c6cb4
Step 6 : RUN apt-get install -y build-essential
 ---> Using cache
 ---> d7fa22c4dd89
Step 7 : RUN go get github.com/cespare/reflex &&         cp /go/bin/reflex /usr/bin/ &&         rm -rf /go/src/* /go/pkg/* /go/bin/*
 ---> Using cache
 ---> 937d6028e42e
Step 8 : ENV PS1 "# "
 ---> Using cache
 ---> b7f4d2ff45c1
Step 9 : ENV GOBIN /go/binw
 ---> Using cache
 ---> d59cce53944b
Step 10 : ADD package.json package.json
 ---> Using cache
 ---> 8b373f48ed53
Step 11 : ENV CGO_ENABLED 0
 ---> Using cache
 ---> 4ec6c4327572
Step 12 : WORKDIR /go/src/github.com/cescoferraro/spotify
 ---> Using cache
 ---> a418964be539
Successfully built a418964be539
[image:build dev-image] cescoferraro/spotify Created
[env:set vars] Done
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Start
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Done
[image:tag api-image] cescoferraro/spotify Tagged
[env:set vars] Done
Step 1 : FROM golang:1.7.4
 ---> 6639f812dbc7
Step 2 : RUN apt-get install -y curl
 ---> Using cache
 ---> cbb7f2130c36
Step 3 : RUN curl -sL https://deb.nodesource.com/setup_6.x -o nodesource_setup.sh
 ---> Using cache
 ---> c99cfa54bfb2
Step 4 : RUN bash nodesource_setup.sh
 ---> Using cache
 ---> 9e5fb9e5d712
Step 5 : RUN apt-get install -y nodejs
 ---> Using cache
 ---> beb37f2c6cb4
Step 6 : RUN apt-get install -y build-essential
 ---> Using cache
 ---> d7fa22c4dd89
Step 7 : RUN go get github.com/cespare/reflex &&         cp /go/bin/reflex /usr/bin/ &&         rm -rf /go/src/* /go/pkg/* /go/bin/*
 ---> Using cache
 ---> 937d6028e42e
Step 8 : ENV PS1 "# "
 ---> Using cache
 ---> b7f4d2ff45c1
Step 9 : ENV GOBIN /go/binw
 ---> Using cache
 ---> d59cce53944b
Step 10 : ADD package.json package.json
 ---> Using cache
 ---> 8b373f48ed53
Step 11 : ENV CGO_ENABLED 0
 ---> Using cache
 ---> 4ec6c4327572
Step 12 : WORKDIR /go/src/github.com/cescoferraro/spotify
 ---> Using cache
 ---> a418964be539
Successfully built a418964be539
[image:build dev-image] cescoferraro/spotify Created
[env:set vars] Done
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Start
[job: api] bash -c "go build -o dist/spotify -ldflags \"-X main.VERSION=$API_VERSION\" src/api/*.go" -> dist/spotify Done
The push refers to a repository [docker.io/cescoferraro/spotify]
58b328818efb: Pushed 
540ea697ec3f: Pushed 
8b77aafa7afb: Mounted from library/golang 
988a4cf7bfad: Mounted from library/golang 
e6769ace411a: Mounted from library/golang 
779900daf86c: Mounted from library/golang 
53c779688d06: Mounted from library/golang 
60a0858edcd5: Mounted from library/golang 
b6ca02dfe5e6: Mounted from library/golang 
api-0.1.16: digest: sha256:066b1f33224cda57b2a645a8751ef32b80b0806e909c547397a3ca60ee54aea1 size: 2214
[image:push api-image] cescoferraro/spotify Pushed

Bulk `:rm` tasks

From this discussion

It might be valuable to remove resources in batches. The current options are:

• create an alias to remove things

  alias=clean:
      tasks: ['builder:rm', 'binary:rm', 'dist-img:rm']
  

  dobi clean
  

• use the :rm action of an existing alias

  alias=build:
      tasks: ['builder', 'binary', 'dist-img']
  

  dobi build:rm
  

Some potential additions could be:

• add a built-in autoclean task that runs the :rm of every resource Done in #29 (dobi 0.7)
• a new action on a job resource to run remove on the image (and possibly any dependencies) of the job. Maybe job:clean ?

Support "default mounts"

Many projects only have a single mount for the source code. Instead of repeating that mount on every job it would be nice to provide a way to specify a default that is applied to all jobs (unless the job has defined different mounts:).

I think the syntax would look like a regular mount, with the name default:

mount=default:
   ...

Subprojects cannot use mounts from the parent.

They seem to silently fail to do the mount.

Cannot install via go get

Trying to install dobi via go get, I run into the error:

$ go get github.com/dnephin/dobi
# github.com/dnephin/dobi/cmd
/srv/go/src/github.com/dnephin/dobi/cmd/dobi.go:44: unknown cobra.Command field 'TraverseChildCommands' in struct literal
/srv/go/src/github.com/dnephin/dobi/cmd/dobi.go:45: unknown cobra.Command field 'Args' in struct literal

I'm using go 1.6:

$ go version
go version go1.6.2 linux/amd64

I am running Ubuntu16.04.

Support a list of artifacts in the job resource

Instead of just a single artifact.