dropseed / deps-git

Deps component for updating git dependencies

deps-git

This component allows you to track remote repositories (on GitHub or elsewhere) and do a find-and-replace in your repo when new tags are pushed.

This is especially useful for dependencies that don't use a package manager.

Example deps.yml

version: 3
dependencies:
- type: git
  settings:
    remotes:
      https://github.com/kubernetes/minikube.git:
        replace_in_files:
        - filename: dev/install.go
          # pattern is a regex that must have 1 capture group
          pattern: minikube version (\S+)
          # you can optionally disable semver parsing on the tags
          # (which means latest version will be the last tag)
          semver: false

      https://github.com/dropseed/deps-git.git:
        replace_in_files:
        - filename: file.txt
          pattern: deps-git (\S+)
          # use a semver range to limit updates
          # https://github.com/blang/semver#ranges
          range: "< 1.0.0"

      https://github.com/getsentry/sentry-javascript.git:
        replace_in_files:
        - filename: file.txt
          pattern: raven==(\S+)
          # only use tags with this prefix (and remove the prefix so we just get the version number)
          tag_prefix: raven-js@
          # include semver pre-releases
          prereleases: true

      https://github.com/libevent/libevent.git:
        replace_in_files:
        - filename: file.txt
          pattern: libevent (\S+)
          # filter tags to those that match a specific pattern, and use the captured
          # group as the version name (i.e. you'll get "2.1.10" instead of "release-2.1.10")
          tag_filter:
            matching: 'release-(\S+)-stable'
            output_as: '$1'

      https://github.com/libevent/libevent.git:
        replace_in_files:
        - filename: file.txt
          pattern: libevent (\S+)
          # filter tags to those that match a specific pattern, and use the
          # full tag name as the version
          tag_filter:
            matching: 'release-\S+-stable'

      https://github.com/curl/curl.git:
        replace_in_files:
        - filename: file.txt
          pattern: curl==(\S+)

          tag_filter:
            matching: 'curl-(\d+)_(\d+)_(\d+)'
            sort_as: '$1.$2.$3'  # sort as a semver-compatible version, without affecting output

Support

Any questions or issues with this specific component should be discussed in GitHub issues.

If there is private information which needs to be shared then please use the private support channels in dependencies.io.

deps-git's Issues

Update install script to have range setting

The install script in https://github.com/dropseed/deps-git/blob/master/deps_component.yml is still installing 0.1.1 which doesn't have support for range config.

array index out of range error when pattern doesn't match

If you use a pattern that doesn't match anything in the file, you get an unclear error .

panic: runtime error: index out of range

goroutine 1 [running]:
main.collect(0x7ffea4e3bd4a, 0x1, 0x7ffea4e3bd4c, 0x13, 0x7)
	/home/travis/gopath/src/github.com/dropseed/deps-git/collect.go:28 +0x883
main.main()
	/home/travis/gopath/src/github.com/dropseed/deps-git/main.go:24 +0x20a

deps-git/collect.go

Line 28 in 87a51bd

currentVersion := submatches[1]

arm64 release

Currently deps fails on the m1/m2 macs when developers are running locally with git dependencies being checked.

deps -v upgrade

> Using component from https://github.com/dropseed/deps-git
> Storing component in /Users/<username>/Library/Caches/deps/components/deps-git
> git pull
Already up to date.
> Skipping install of /Users/<username>/Library/Caches/deps/components/deps-git
> Collecting with /Users/<username>/Library/Caches/deps/components/deps-git...
> Input path: .
> $DEPS_COMPONENT_PATH/bin/deps-git --collect . /var/folders/<key>/deps-958642410
sh: /Users/<username>/Library/Caches/deps/components/deps-git/bin/deps-git: No such file or directory
exit status 127

Any plans for releasing an arm64 version for this component?

Change how version is set/updated in deps_component.yml

Sorting semver-ish tags

The semver parsing is strict, and tags that are almost semver can be out of order.

Examples:

• curl-7_59_0 - https://github.com/curl/curl
• release-57-1 - https://github.com/unicode-org/icu
• OpenSSL_1_0_2o - https://github.com/openssl/openssl.git

We want to be able to sort these kinds of tags in their semver order.

This settings API is evolving a little bit... here is one idea for how we could give some control over how tags are sorted and what string you actually end up using when doing the replacement in a file:

https://github.com/curl/curl.git:
  replace_in_files:
  - filename: file.txt
    pattern: curl==(\S+)

    tag_filter:
      from: 'curl-(\d+)_(\d+)_(\d+)'
      to: '$1'  # the original full tag name
      sort_as: '$2.$3.$4'  # this format will only be used for sorting (semver)

    semver:
      enabled: true
      range: '< 8.0.0'
      prereleases: false

More flexible filtering and sorting when not using semver

Given the following example:

version: 3
dependencies:
- type: git
  settings:
    remotes:
      https://github.com/minio/mc.git:
        replace_in_files:
        - filename: charts/minio/values.yaml
          pattern: 'tag: "(\S+)"'
          semver: false

I see the following output, which is trying to replace the tag with the very first tag in the repo:

> Collecting with deps-git

> 1 new updates to be made
> [b2f436a] Update minio/mc from RELEASE.2018-07-13T00-53-22Z to RELEASE.Wed-Jun-24-21-56-09-GMT-2015

There appears to be some sorting still going on. And changing the pattern to be more specific doesn't seem to change the result at all?

tag_prefix: 'RELEASE.'
pattern: 'tag: "RELEASE.(\d\d\d\d-\d\d-\d\dT\d\d-\d\d-\d\dZ)"'