vube / depman

depman

--

Depman was deprecated on 25 February 2015

We recommend using godep instead: https://github.com/tools/godep

Dependency management helper for Golang packages. Supports versioned dependencies using standard Golang imports.

Installation

Run:

go get github.com/vube/depman
depman help

Author

Nicholas Capo [email protected]

Copyright

Copyright 2013-2014 Vubeology, Inc. Released under the MIT License (see LICENSE).

Features

• Simple json configuration file

• Recursive installs

• Uses go get when possible

• Supports Git, Mercurial, and Bazaar

• Automatic installation of dependencies from git repositories that do not support go get

• Time based "cache" to increase speed

• Handles Multi-Part $GOPATH

Usage

Run depman in the directory with your deps.json, or use the --path argument to specify the directory where deps.json is located.

Commands

• init Create an empty deps.json

• add Add a dependency (interactive)

• install Install all the dependencies listed in deps.json (default)

• update [nickname] [branch] Update [nickname] to use the latest commit in [branch]

• show-frozen Show dependencies as resolved to commit IDs. Use the --recursive flag to descend into dependencies depth-first.

• help Display help message

Options

• -clean=false: Remove changes to code in dependencies

• -clear-cache=false: Delete the time based cache

• -debug=false: Display debug messages. Implies --verbose

• -help=false: Display help

• -no-colors=false: Disable colors

• -path=".": Directory or full path to deps.json

• -silent=false: Don't display normal output. Overrides --debug and --verbose

• -skip-cache=false: Skip the time based cache for this run only

• -verbose=false: Display commands as they are run, and other informative

messages

• -version=false: Display version number

Basic Algorithm

For each dependency:

1. Use go get to download the dependency

2. Checkout the specified version

3. Look in the dependency's directory for a deps.json and recursively install those dependencies

4. If the dependency type is git-clone then manually run git clone, git fetch, etc as needed

Duplicates

Duplicated dependencies (anywhere in the tree) with identical versions will be skipped (this just saves some time and prevents infinite recursion). Duplicated dependencies with different versions cause a fatal error and must be fixed by the developer.

Non Go-Getable Repos

Some repositories (private bitbucket repositories for example), are not supported by go get. To include those repositories in depman:

1. Change the type to git-clone (hg and bzr are not yet supported)

2. Change the repo to a full git url (include everything necessary for git clone)

3. Add an alias field to specify a directory in which to clone, the path is rooted at $GOPATH/src/

See the example below or the included deps.json file.

Multi-Part $GOPATH Support

Depman since version 2.8.0 supports multi-part $GOPATH. When installing dependencies, depman will search the parts of $GOPATH for the appropriate directory, if not found, depman will install the dependency to the first part of $GOPATH.

Cache

Depman uses a time based cache to speed up normal operations.

A global list of dependencies and timestamps is kept at $GOPATH/.depman.cache. When depman is run it looks at the timestamp to decide whether to update the repo or not (go get -u, git clone, git fetch, etc).

If the dependency is more that 1 hour old, depman will fetch updates from the network, otherwise depman uses the repo as is.

If the cache was stale or unused, a '*' will be printed at the end of the installation line.

You can clear the cache by deleting the cache file, or running depman with the --clear-cache flag. The cache can be skipped for the current run by using the --skip-cache flag.

Additional information about the cache (including the time spent while installing) can been seen by running depman with the --verbose flag.

The code for this feature is in the timelock package.

Implementation Requirements

• Depman shall not require any external dependencies (beyond the standard library) for normal operation

• Testing dependencies shall be installed through depman

• go vet shall not indicate any problems

• golint (https://github.com/golang/lint) shall not indicate any problems

Testing

Depman can be tested by running make in the source directory.

This does the following:

1. Installs depman

2. Installs testing dependencies

3. Runs depman with a few common flags to confirm basic operation

4. Run go vet, and golint

5. Executes the unit tests

6. Generate Readme.md

See Makefile for more information.

JSON Structure

NOTE: The file must be named deps.json

{
	"shortname":{
		"repo":"url/to/package, just like in import",
		"version":"commit, tag, or branch",
		"type": "one of 'git', 'bzr', 'hg'"
		"skip-cache":"optional, set to 'true' to always ignore the cache"
	},
	"not go getable":{
		"repo":"full git repo url, just like git clone"
		"version":"commit, tag, or branch",
		"type": "must be 'git-clone'",
		"alias": "target directory to clone into, (only supported for type 'git-clone')"
	}
}

Example

{
	"gocheck": {
		"repo": "launchpad.net/gocheck",
		"version": "87",
		"type": "bzr"
	},
	"gocov": {
		"repo": "github.com/vube/gocov/gocov",
		"version": "93371a7ae85bec1c4afe9b9f3281c062ab106e6d",
		"type": "git",
		"skip-cache": true
	},
	"gocov-html": {
		"repo": "https://github.com/matm/gocov-html.git",
		"version": "1512341d22ab06788fc5ad63925fd07979a9ef39",
		"type": "git-clone",
		"alias": "github.com/matm/gocov-html"
	},
	"godocdown": {
		"repo": "github.com/robertkrimen/godocdown",
		"version": "0bfa0490548148882a54c15fbc52a621a9f50cbe",
		"type": "git"
	},
	"golint": {
		"repo": "github.com/golang/lint/golint",
		"version": "7e9cdc93310598b5c6cd9ce4d0d0a37c0f5b9e4c",
		"type": "git"
	}
}

Todo

• Support manually cloning hg or bzr repositories (add types hg-clone, and bzr-clone)

depman's Issues

Cannot find .depman.cache

On Windows 7, I keep receiving the following message for the init and install commands.

"open C.depman.cache: The system cannot find the path specified."

Invalid sub-dependency deps.json is not fatal

Invalid json in deps.json is fatal, but invalid json in a dependencies deps.json is not.

Steps to reproduce:

1. Assume project A
2. Assume project B
3. A depends on B using depman (deps.json)
4. B/deps.json is invalid json
5. Running depman in A outputs the following message but does not exit non-zero
   a. "Error reading deps from [...]"

Invalid deps.json from a dependency should be an error that results in a non-zero exit status.

Issues: run go fmt on code, fix imports to use local instances of code, handle blank version

I am about to submit a pull request for the following three issues:

• Code needs to have go-fmt run on it
• Imports in files refer to "github.com/vube/depman" instead of "vube/depman". This means that changes made locally get ignored.
• Code currently requires version to be specified in deps.json. A change where it assumes "master" as the default for a non-specified value would be helpful.

refactor color functions for ease of reading and use

As is, the color functions merely take a string. Thus, some sort of string composition must be done to build a string to be colored. This can be changed so the color functions take arguments as printf, for readability.

complains about missing deps.json in non-applicable actions

Reproduction steps:

1. cd to a directory with no deps.json
2. make sure GOPATH is set to something
3. run depman self-upgrade
4. it will complain about a missing deps.json

I suspect that this will occur for other actions as well.

Needs a way to depend on configs.

Projects will depend on a revision of a set of configs, but depman doesn't support this.

Acceptance Criteria

• Depman allows a new attribute of a dependency that indicates where the repo is to reside, which also indicates that go get shouldn't be used.

support multi-value GOPATH

when GOPATH contains more than one path (separated by :), depman will not work as it treats that as a single path:

[...]
 | | github.com/ActiveState/httpapi (6a46f16c055) *
 | | $ cd /s/go/src/myproject/vendor:/s/go/src/github.com/ActiveState/httpapi
 | | chdir /s/go/src/myproject/vendor:/s/go/src/github.com/ActiveState/httpapi: no such file or directory

Default output should include version number

Output from a default run should include the version number being installed.

Example:

$ depman
Depman Version 2.5.1
Installing:
 | gocheck (85)
 | gocov (90c9390)
 | gocov-html (f83448b)
 | golint (22f4d5e)
Success

Mercurial errors when installing dependencies

Depman Version 2.9.4 [19/1939]
Installing:
| lrucache (2c00de6812e8a48d37dacba0b0494e5f2bc5e7e3)
| robotstxt-go (961a)
| goquery (a3fdc62e311a44c5199b0ef323ee5e056c527ef1) *
| $ go get -u github.com/PuerkitoBio/goquery
| cd /home/user/gopath/src/code.google.com/p/cascadia; hg pull
abort: requirement 'dotencode' not supported!
package github.com/PuerkitoBio/goquery
imports code.google.com/p/cascadia: exit status 255
cd /home/user/gopath/src/code.google.com/p/go.net; hg pull
abort: requirement 'dotencode' not supported!
package github.com/PuerkitoBio/goquery
imports code.google.com/p/go.net/html: exit status 255

Add skip cache option to a dependency

Depman 2.6.0: Git pulls before checkout

Depman 2.6.0

For git branch dependencies, depman does a pull before a checkout resulting in a possibly incorrect merge into an unrelated branch.

Need to refactor the VersionControl interface to distinguish between fetching from the remote and merging remote changes.

update action inserts only the short commit hash

As is, when running depman update, only a short commit hash will be inserted, which has a much higher chance of eventual collision than a full hash.

Acceptance Criteria

• change code to insert full commit hash

will recurse into the current project if there is a circular dependency

Say you have two projects, A and B. A depends on B and B depends on A. You are in A and run depman. depman will get B, see that it depends on A, get A, and stop when it sees B again. Put simply, depman has no concept of the identity of the project it's being run for.

Acceptance Criteria

• When depman is run for A and there is a circular dependency to A in a dependency, it should not recurse.

stat github.com/docopt/docopt.go: no such file or directory

with this deps.json:

{
    "github.com/codegangsta/inject": {
        "repo": "github.com/codegangsta/inject",
        "version": "9aea7a2fa5b79ef7fc00f63a575e72df33b4e886",
        "type": "git"
    },
    "github.com/codegangsta/martini": {
        "repo": "github.com/codegangsta/martini",
        "version": "bbe60ba962de9b5627873af719f1c98133d43afb",
        "type": "git"
    },
    "github.com/docopt/docopt.go": {
        "repo": "github.com/docopt/docopt.go",
        "version": "b74119e4500ceb6a80187775e1b081a64231c5ad",
        "type": "git"
    }
}

i get:

Depman Version 2.7.0
Installing:
 | github.com/codegangsta/inject (9aea7a2fa5b79ef7fc00f63a575e72df33b4e886) *
 | | github.com/codegangsta/martini (bbe60ba962de9b5627873af719f1c98133d43afb) *
 | | | github.com/docopt/docopt.go (b74119e4500ceb6a80187775e1b081a64231c5ad) *
 | | | $ go get -u github.com/docopt/docopt.go
 | | | stat github.com/docopt/docopt.go: no such file or directory

failure getting a dependency still returns exit code 0

As is, depman will continue with the next dependency if it encounters a problem getting a dependency, but it will report success at the end and return exit code 0.

Desired functionality is that it gloss over the failed dependency, continue with the other dependencies, and return a non-zero exit code.

go get -u equivalent

I might have missed something, but there seems to be no way to force a dependency to be rebuilt/reinstalled if upstream updates - running depman install a second time fetches the latest from git, but doesn't seem to actually rebuild the package.

Basically, the equivalent of go get -u for the deps in deps.json would be very helpful.