julialang / pkgdev.jl

Tools for Julia package developers

PkgDev

Overview

PkgDev provides tools for Julia package developers. The package is currently being rewritten for Julia 1.x and only for brave early adopters.

Usage

PkgDev.tag(package_name, version=nothing; registry=nothing, release_notes=nothing)

Tag a new release for package package_name. The package you want to tag must be deved in the current Julia environment. You pass the package name package_name as a String. The git commit that is the HEAD in the package folder will form the basis for the version to be tagged.

If you don't specify a version, then the version field in the Project.toml must have the format x.y.z-DEV, and the command will tag version x.y.z as the next release. Alternatively you can specify one of :major, :minor or :patch for the version parameter. In that case PkgDev.tag will increase that part of the version number by 1 and tag that version. Finally, you can also specify a full VersionNumber as the value for the version parameter, in which case that version will be tagged.

The only situation where you would specify a value for registry is when you want to register a new package for the first time in a registry that is not General. In all other situations, PkgDev.tag will automatically figure out in which registry your package is registered. When you do pass a value for registry, it should simply be the short name of a registry that is one of the registries your local system is connected with.

If you want to add custom release notes for TagBot, do so with the release_notes keyword.

PkgDev.tag runs through the following process when it tags a new version:

1. Create a new release branch called release-x.y.z.
2. Change the version field in Project.toml and commit that change on the release branch.
3. Change the version field in Project.toml to x.y.z+1-DEV and commit that change also to the release branch. 4a. For packages in the General registry: add a comment that triggers Registrator. 4b. For packages in other registries: Open a pull request against the registry that tags the first new commit on the release branch as a new version x.y.z.
4. Open a pull request against the package repository to merge the release branch into master.

If you have TagBot installed for your package with the branches: true setting, it will automatically merge the release-x.y.z branch into master once the pull request for the registry has been merged. If you use the package butler (desribed below) it auto-configures your repository for this workflow.

PkgDev.enable_pkgbutler(package_name; channel=:auto, template=:auto)

Enables the Julia Package Butler for package package_name. The package must be deved in the current Julia environment. The command will make various modifications to the files in the deved folder of the package. You then then need to commit these changes and push them to GitHub. The command will also add a deploy key to the GitHub repository of the package and show instructions on how to add two GitHub Actions secrets to the repository.

The channel argument can be :auto, :stable or :dev. There are two channels of updates: stable and dev. The dev channel will run the Julia Package Butler workflow every 5 minutes and it will use the master branch of the Julia Packge Butler engine, i.e. it will get new features more quickly. The stable branch runs the Julia Package Butler workflow every hour, and new features in the Julia Package Butler engine are only pushed out to the stable channel once they have been tested for a while on the dev channel. If you specify :auto as the argument, any existing channel choice you have previously made for the package will be retained, otherwise the package will be configure for the stable channel.

The template argument can be :auto, :default or :bach. Different templates will configure different aspects of your package. At this point everyone should use the :default template (or :auto template), everything else is considered experimental.

PkgDev.switch_pkgbutler_channel(package_name, channel)

Switch the Julia Package Butler channel for package package_name. The package you want to tag must be deved in the current Julia environment and the Julia Package Butler must already be enabled for the package. The channel argument can be :auto, :stable or :dev, see the documentation for PkgDev.enable_pkgbutler for an explanation of the different channels.

PkgDev.switch_pkgbutler_template(package_name, template)

Switch the Julia Package Butler template for package package_name. The package you want to tag must be deved in the current Julia environment and the Julia Package Butler must already be enabled for the package. The template argument can be :auto, :default or :bach.

PkgDev.format(package_name)

Format all the Julia source code files for the package with name package_name. The package you want to format must be deved in the current Julia environment. This function uses DocumentFormat.jl.

pkgdev.jl's Issues

RFC: Julia Package Manifest File

Following discussion in on extended package manifest file julia/#11955. I started to work on specifying content of such manifest file in the manifest branch.

Package manifest documentation file, MANIFEST.md, includes description of the fields and some thoughts on changes in work of the package manager.

@StefanKarpinski @IainNZ

Pkg.rm removes the package, but keeps it in the session cache

Example:

julia> Pkg.clone("https://github.com/johnmyleswhite/MySQL.jl.git")
INFO: Cloning MySQL from https://github.com/johnmyleswhite/MySQL.jl.git
INFO: Computing changes...

julia> using MySQL
ERROR: DBI not found
 in require at loading.jl:47
 in include at /Applications/Juno.app/Contents/Resources/app/julia/lib/julia/sys.dylib
 in include_from_node1 at /Applications/Juno.app/Contents/Resources/app/julia/lib/julia/sys.dylib
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
while loading /Users/naresh/.julia/v0.3/MySQL/src/MySQL.jl, in expression starting on line 2

julia> Pkg.clone("https://github.com/JuliaDB/DBI.jl.git")
INFO: Cloning DBI from https://github.com/JuliaDB/DBI.jl.git
INFO: Computing changes...

julia> using MySQL

---

Now

julia> Pkg.rm("DBI")
INFO: Removing DBI (unregistered)

julia> using MySQL

Should have not worked...

julia> Pkg.resolve()
INFO: No packages to install, update or remove

julia> using MySQL

Makes no difference

julia> exit(0)

Exited the session and after coming back...

julia> using MySQL
ERROR: DBI not found
 in require at loading.jl:47
 in include at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in include_from_node1 at /Applications/Julia-0.3.11.app/Contents/Resources/julia/lib/julia/sys.dylib
 in reload_path at loading.jl:152
 in _require at loading.jl:67
 in require at loading.jl:51
while loading /Users/naresh/.julia/v0.3/MySQL/src/MySQL.jl, in expression starting on line 2

Unclear screen message "Enter host password"

While publishing changes:

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing FlexibleArrays permanent tags: v1.1.0
Password for 'eschnett@https://github.com/eschnett/FlexibleArrays.jl':
INFO: Submitting METADATA changes
INFO: Forking JuliaLang/METADATA.jl to eschnett
Enter host password for user 'eschnett':
WARNING: readall is deprecated, use readstring instead.

Julia asked me for my "host password". It is unclear what a "host password" is. It turns out that this is my Github password. The message should be improved, or -- if this is not possible -- Julia could output an explanation before asking this question.

Pkg.generate inside a pre-existing repo

If I have a git repo that I want to turn into a package, Pkg.generate should generate the appropriate files inside that repo, and copy everything over to .julia; currently this needs to be done by hand.
Related to #8

Improvement opportunity: error message for Pkg.checkout() with outdated METADATA

I just got the following error message:

julia> Pkg.checkout("Contour")
INFO: Checking out Contour master...
INFO: Pulling Contour latest master...
ERROR: fixed packages introduce conflicting requirements for Compat: 
         JSON requires versions [0.7.1,∞)
         Contour requires versions [0.8.0,∞) [none of the available versions can satisfy this requirement]
         WoodburyMatrices requires versions [0.0.0-,∞)
       available versions are 0.0.1, 0.0.2, 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.2.2, 0.2.3, 0.2.4, 0.2.5, 0.2.6, 0.2.7, 0.2.8, 0.2.9, 0.2.10, 0.2.11, 0.2.12, 0.2.13, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.5.0, 0.5.1, 0.6.0, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.7.4, 0.7.5, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.7.10, 0.7.11 and 0.7.12
# stack trace omitted

It turns out the problem was that I had to run Pkg.update() first, to get a complete list of available versions for the problematic package. It would be nice if the error message could indicate that, and/or if a warning would be printed before attempting to check out that makes the user aware that local METADATA is outdated.

more granular Pkg.publish() #7749

Original issue: JuliaLang/julia#7749

Documentation hosting

We need to host a long form of the documentation (not just docstrings) somewhere, Read the Docs?

cc @tkelman

crash on PkgDev.register

My folder layout should be all standard.

julia> PkgDev.register("SuiteSparse")
INFO: Registering SuiteSparse at https://github.com/JuliaSparse/SuiteSparse.jl.git
INFO: Tagging SuiteSparse v0.0.1
ERROR: chdir METADATA: no such file or directory (ENOENT)
 [inlined code] from ./libuv.jl:68
 in cd(::String) at ./file.jl:39
 in cd(::PkgDev.Entry.##11#14{Base.LibGit2.GitRepo,String,VersionNumber,String,Bool,Array{Base.Pkg.Reqs.Line,1}}, ::String) at ./file.jl:47
 in write_tag_metadata(::Base.LibGit2.GitRepo, ::String, ::VersionNumber, ::String, ::Bool) at /home/kristoffer/.julia/v0.5/PkgDev/src/entry.jl:115
 in write_tag_metadata(::Base.LibGit2.GitRepo, ::String, ::VersionNumber, ::String) at /home/kristoffer/.julia/v0.5/PkgDev/src/entry.jl:110
 [inlined code] from ./dict.jl:780
 in (::PkgDev.Entry.##16#23{String,String,String})(::Base.LibGit2.GitRepo) at /home/kristoffer/.julia/v0.5/PkgDev/src/entry.jl:165
 in transact(::PkgDev.Entry.##16#23{String,String,String}, ::Base.LibGit2.GitRepo) at ./libgit2.jl:499
 in register(::String, ::String) at /home/kristoffer/.julia/v0.5/PkgDev/src/entry.jl:143
 in register(::String) at /home/kristoffer/.julia/v0.5/PkgDev/src/entry.jl:192
 in register(::String) at /home/kristoffer/.julia/v0.5/PkgDev/src/PkgDev.jl:32
 in eval(::Module, ::Any) at ./boot.jl:226

Missing common license types?

I can only imagine that not having GPL{2, 3} as one of the options is a not so subtle way of discouraging these licenses, but a fair number of packages will want to use this license (being based on R lets say, or many other common sources). Should this not be a supported option? Having code report the wrong license because the tooling is opinionated seems odd to me.

Add tests for PkgDev.register

and any other auth-required tests if we can figure out the right way to do them

GitError(Code:ERROR, Class:Net, Unsupported URL protocol)

I don't understand what the resolution of the discussion about git versus https protocols was.
I'm getting the following (newly pulled and compiled Julia master; v0.1.0 of PkgDev):

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing CRlibm permanent tags: v0.2.0
ERROR: GitError(Code:ERROR, Class:Net, Unsupported URL protocol)
 [inlined code] from libgit2/error.jl:96
 in __push#333__ at libgit2/remote.jl:84
 [inlined code] from essentials.jl:116
 in __push#353__ at libgit2.jl:178
 [inlined code] from essentials.jl:116
 in anonymous at /Users/davidsanders/.julia/v0.5/PkgDev/src/entry.jl:99
 in with at ./libgit2/types.jl:481
 [inlined code] from libgit2/types.jl:487
 in publish at /Users/davidsanders/.julia/v0.5/PkgDev/src/entry.jl:84
 in publish at /Users/davidsanders/.julia/v0.5/PkgDev/src/PkgDev.jl:55

The package is a fresh copy with Pkg.add("CRlibm") and then Pkg.checkout(), which gives:

19:03 $ git remote -v
origin  git://github.com/dpsanders/CRlibm.jl.git (fetch)
origin  [email protected]:dpsanders/CRlibm.jl.git (push)

`Pkg.publish()` unable to validate METADATA

On 0.5.0-rc0 and PkgDev master.

I kept getting this:

julia> PkgDev.publish()
INFO: Validating METADATA
ERROR: packages with unsatisfiable requirements found:
    BayesNets v0.0.2 – no valid versions exist for package DataFrames
    BayesNets v0.0.3 – no valid versions exist for package DataFrames
    BayesNets v0.0.4 – no valid versions exist for package DataFrames
    BayesNets v0.0.5 – no valid versions exist for package DataFrames
    BayesNets v0.0.6 – no valid versions exist for package DataFrames
    BayesNets v0.0.7 – no valid versions exist for package DataFrames
    BayesNets v0.0.8 – no valid versions exist for package DataFrames
    BayesNets v0.0.9 – no valid versions exist for package DataFrames
    BayesNets v0.0.10 – no valid versions exist for package DataFrames
    BayesNets v0.1.0 – no valid versions exist for package DataFrames

 in check_metadata(::Set{String}) at /Users/ericdavies/.julia/v0.5/PkgDev/src/entry.jl:286
 in (::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String})(::Base.LibGit2.GitRepo) at /Users/ericdavies/.julia/v0.5/PkgDev/src/entry.jl:82
 in with(::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:658
 in publish(::String, ::String) at /Users/ericdavies/.julia/v0.5/PkgDev/src/entry.jl:53
 in publish() at /Users/ericdavies/.julia/v0.5/PkgDev/src/PkgDev.jl:70

despite the only change I've made being:

commit 9626a54d9d34e6d19aabbbff863a5a07cc7833bd
Author: Eric Davies <[email protected]>
Date:   Thu Jul 28 11:46:03 2016 -0500

    Tag AutoHashEquals v0.0.10 [https://github.com/andrewcooke/AutoHashEquals.jl]

diff --git a/AutoHashEquals/versions/0.0.10/requires b/AutoHashEquals/versions/0.0.10/requires
new file mode 100644
index 0000000..2c4ef82
--- /dev/null
+++ b/AutoHashEquals/versions/0.0.10/requires
@@ -0,0 +1 @@
+julia 0.3
diff --git a/AutoHashEquals/versions/0.0.10/sha1 b/AutoHashEquals/versions/0.0.10/sha1
new file mode 100644
index 0000000..c232f02
--- /dev/null
+++ b/AutoHashEquals/versions/0.0.10/sha1
@@ -0,0 +1 @@
+3542065f889473ac22d0f58dc0ab57aa3c7ff32b

DataFrames is on v"0.7.6".

Pkg.untag()?

Would it be possible to untag? While one needs to be careful about tagging, but sometimes makes mistakes in tagging.

When tagging, check that commit has changed

When creating a new tag using tag, something should happen—ideally an error, or at least a warning—when the commit SHA used for the tag matches that of another tag. I was bitten by this recently when tagging a couple of JuliaStats packages.

Apparently this is already a to-do item, since it's mentioned in a comment, but I figured it would be worth submitting an issue to track it.

Disable Codecov Github comments in new packages

By default, Codecov adds Github comments each time it runs. This usually just adds noise to the discussion. See e.g. JuliaLinearAlgebra/IterativeSolvers.jl#80.

This can be avoided by adding a .codecov.yml file to newly generated packages. See the pull request above for such a configuration.

With this change, Codecov's results continue to be available in pull requests as a single coloured icon that leads to the respective Codecov page with the detailed results.

`Pkg.publish()` push permanent tags [even if it exists] to source-code-server

INFO: Validating METADATA
INFO: Pushing Box0 permanent tags: v0.1.0
Username for 'https://gitlab.com': kuldeepdhaka
Password for 'https://[email protected]': 
INFO: Submitting METADATA changes
INFO: Forking JuliaLang/METADATA.jl to kuldeepdhaka
INFO: Recompiling stale cache file /home/kuldeep/.julia/lib/v0.4/Compat.ji for module Compat.
INFO: Recompiling stale cache file /home/kuldeep/.julia/lib/v0.4/JSON.ji for module JSON.
INFO: Pushing changes as branch pull-request/1a333930
INFO: To create a pull-request, open:

  https://github.com/kuldeepdhaka/METADATA.jl/compare/pull-request/1a333930

but, the v0.1.0 annotated tag existed on the source-code-server (so, no push is required).
i was required to provide credentials or it won't proceed.

Provide a system name in a GitHub 2FA access token

Currently a GitHub 2FA token is generated with name - Julia Package Manager. If there are more then one system used for development, it is hard to distinguish where which token was generated.

A possible solution is to provide a system name or an IP address of the system where token was generated.

PkgDev.publish hangs after entering Github user/pass

I was running into the issues listed at the end of #48, but I saw that PkgDev master had a fix. So I checked it out, but now I run into this. Any ideas?

julia> PkgDev.publish("QuartzImageIO")
INFO: Validating METADATA
INFO: Pushing QuartzImageIO permanent tags: v0.1.3, v0.1.4
Username for 'https://github.com':rsrock
Password for 'https://[email protected]':

[Hangs here, I banged on ^C after about 5 minutes of waiting]

^C^C^C^C^C^C^C^C^CWARNING: Force throwing a SIGINT
^CERROR: InterruptException:
 in macro expansion at ./libgit2/error.jl:96 [inlined]
 in #push#53(::Bool, ::Base.LibGit2.PushOptions, ::Function, ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./libgit2/remote.jl:84
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./<missing>:0
 in #push#94(::String, ::String, ::Array{String,1}, ::Bool, ::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Function, ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:179
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, ::Base.LibGit2.GitRepo) at ./<missing>:0
 in (::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo) at /Users/rrock/.julia/v0.5/PkgDev/src/entry.jl:114
 in with(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:645
 in publish(::String, ::String) at /Users/rrock/.julia/v0.5/PkgDev/src/entry.jl:97
 in publish(::String) at /Users/rrock/.julia/v0.5/PkgDev/src/PkgDev.jl:70

unable to do `PkgDev.publish()`

Hello, I've got the following error message when I tried to do Pkg.publish():

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing OffsetArrays permanent tags: v0.2.0
ERROR: MethodError: no method matching Base.LibGit2.UserPasswordCredentials(::Void, ::String)
Closest candidates are:
  Base.LibGit2.UserPasswordCredentials(::AbstractString, ::AbstractString)
  Base.LibGit2.UserPasswordCredentials{T}(::Any)
 in credentials() at .../.julia/v0.5/PkgDev/src/github.jl:126
 in (::PkgDev.Entry.##6#9{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo) at .../.julia/v0.5/PkgDev/src/entry.jl:99
 in with(::PkgDev.Entry.##6#9{Dict{String,Array{String,1}}}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:660
 in publish(::String, ::String) at .../.julia/v0.5/PkgDev/src/entry.jl:84
 in publish() at .../.julia/v0.5/PkgDev/src/PkgDev.jl:55
 in eval(::Module, ::Any) at ./boot.jl:231
 in macro expansion at ./REPL.jl:92 [inlined]
 in (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:46

julia
               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.5.0-dev+4819 (2016-06-16 20:13 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit d634100* (3 days old master)
|__/                   |  x86_64-linux-gnu

Thanks!

Remove dependency on `curl`

Currently, interaction with GitHub API requires curl to be installed. It would be nice to remove this dependency an use Requests.jl.

Pkg.publish() UI issue when tagging multiple packages

Original issue: JuliaLang/julia#6263

GitError on `PkgDev.publish()`

I have no idea what is causing this. I just set up a new machine (Ubuntu 16.04) and I'm getting this on Julia 0.5-rc2:

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing TypedTables permanent tags: v0.1.2
ERROR: GitError(Code:EAUTH, Class:OS, The ProgramData file 'config' doesn't exist: )
 in macro expansion at ./libgit2/error.jl:99 [inlined]
 in #push#53(::Bool, ::Base.LibGit2.PushOptions, ::Function, ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./libgit2/remote.jl:84
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, ::Base.LibGit2.GitRemote, ::Array{String,1}) at ./<missing>:0
 in #push#94(::String, ::String, ::Array{String,1}, ::Bool, ::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Function, ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:185
 in (::Base.LibGit2.#kw##push)(::Array{Any,1}, ::Base.LibGit2.#push, ::Base.LibGit2.GitRepo) at ./<missing>:0
 in (::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}})(::Base.LibGit2.GitRepo) at /home/ferris/.julia/v0.5/PkgDev/src/entry.jl:114
 in with(::PkgDev.Entry.##6#11{Dict{String,Array{String,1}}}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:638
 in publish(::String, ::String) at /home/ferris/.julia/v0.5/PkgDev/src/entry.jl:97
 in publish() at /home/ferris/.julia/v0.5/PkgDev/src/PkgDev.jl:70

Document PkgDev.config call

Provide documentation for PkgDev.config which initializes git config and generates a GitHub user token.

Pkg.publish(): protection against non-pushed commits?

Original Julia issue: JuliaLang/julia#6555

Provide a way to change a repository protocol

At this point, many packages were generated using git(+ssh) protocol in their repositories. This will become a problem as publish is going to fail with "unsupported protocol" (related to libgit2 build).

Possible solutions:

• provide a tool that would change a push remote url for repository to use https protocol
• try push a repository with remote url converted to https protocol (if possible without changing configuration of the repository).

CONFIG_LEVEL_GLOBAL not defined

I'm getting the following error when I try to use PkgDev.

julia> Pkg.clone("https://github.com/JuliaLang/PkgDev.jl")
INFO: Cloning PkgDev from https://github.com/JuliaLang/PkgDev.jl
INFO: Computing changes...
INFO: No packages to install, update or remove

julia> using PkgDev
ERROR: LoadError: InitError: UndefVarError: CONFIG_LEVEL_GLOBAL not defined
 in __init__ at /Users/dsanders/.julia/v0.5/PkgDev/src/PkgDev.jl:111
 in include at ./boot.jl:261
 in include_from_node1 at ./loading.jl:304
 in require at ./loading.jl:243
during initialization of module PkgDev
while loading /Users/dsanders/.julia/v0.5/PkgDev/src/PkgDev.jl, in expression starting on line 109

issues with Pkg.submit()

On windows and using v0.5rc2 and I checkout out the latest PkdDev.

I'm following the steps as directed here http://docs.julialang.org/en/latest/manual/packages/#man-post-hoc-branching.

The problem: I do PkgDev.submit("Foo") it ignores the current branch name fixdepwarn that I created and creates an arbitrary branch name pull-request/fb12215b. I go the the url it says to open for the pull request. Thus far no problem other than the fact that it ignored my branch name.

Now I make additional changes to my local repo on my computer to update the PR and then do git push this gives the error

fatal: The current branch fixdepwarn has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin fixdepwarn

I look at git remote -v and I see origin set at the package owner's git address. Despite the fact that it forked the repo to my github page, I do not see my git address as a remote. Thus I have to force it to my repo via git push [email protected]:musm/Foo.jl.git fixdepwarn:pull-request/fb12215b

suggest Pkg.unregister(), Pkg.unmaintain()

There are a number of packages listed at http://pkg.julialang.org/ that are no longer maintained written by people who would like to mark them as unmaintained or remove them from the list.
Pkg.unmaintain(), Pkg.unregister() where those commands were only executable by the package author/maintainer would allow contributors to offer a cleaner package ecosystem.
They could accept a version number as an optional second argument.

The current system does not serve the a package author who prefers not to change METADATA. Nor does it support an author who no longer feels their old code should be available even for an older version of Julia. If setting the upper Julia version number to 0.0.0 or 0.0.1 would have the effect of erasing a Package, then there should be command automating that for the author/maintainer.

This suggestions stems from wanting to give people with the background and desire to offer a package to the community an eventual UNDO or NoMoreDoing, should that become their preference. It seems to me this would add to the welcome Julia extends to package contributors.

Generate a benchmarks directory

For performance tests to run with Benchmarks.pkg or similar

cc @jrevels JuliaLang/julia#13893 (comment)

Interactive package generation

Provide interactive package generation as generate list of parameters is growing larger. If calling generate without any parameter then

• (Optional) Configure developer environment (minimal git configuration through PkgDev.config)
• Ask for the package name
  • Check if the name exists in METADATA and inform user
• Select license for the package (provide preview the license or its description or both)
  • Use bundled licenses
  • Use choosealicense.com licenses from https://github.com/github/choosealicense.com/tree/gh-pages/_licenses. These license files provide along with the license text, a description of the license as it is displayed on choosealicense.com/licenses.
• Ask if various developer services (CI & coverage) are going to be used in the package development.
• Ask a location where to put the generated package.

All questions should provide a default answer except the package name.

PkgDev.publish(): `no auth sock variable`

I've got the following error when trying to push a tag.

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing Compat permanent tags: v0.7.7
ERROR: GitError(Code:ERROR, Class:SSH, error authenticating: no auth sock variable)
 [inlined code] from ./libgit2/error.jl:96
 in push at ./libgit2/remote.jl:84

The METADATA commit is created (not pushed) and tag is not pushed (to Compat.jl).

versioninfo() (yesterday master)

Julia Version 0.5.0-dev+848                                                       
Commit 0d1ea71* (2015-10-18 15:28 UTC)                                            
Platform Info:                                                                    
  System: Linux (x86_64-unknown-linux-gnu)                                        
  CPU: Intel(R) Core(TM) i7-4702HQ CPU @ 2.20GHz                                  
  WORD_SIZE: 64                                                                   
  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Haswell)                            
  LAPACK: libopenblas                                                             
  LIBM: libopenlibm                                                               
  LLVM: libLLVM-3.7.0                                                             

On PkgDev master.

C.c @jiahao who have seen the same problem.

Warn package developers to specify version for dependencies

I propose we adopt as a standard to specify version for dependencies when publishing a tag for a package.

When publishing, PkgDev should warn pkg developer that there's some package at REQUIRE file without a version tag specified.

JuliaData/CSV.jl#28

write_tag_metadata does not add files on Windows

This might be a bug in base's LibGit2.add!, not sure. Will have to come back and look into it in more detail later. I end up getting "No METADATA changes to commit" at the end of a Pkg.tag because no sha1 or requires files have been added (they've been created, but not added to the metadata git repo).

Malformed HTTP header

julia> using PkgDev

julia> PkgDev.publish()
INFO: Validating METADATA
INFO: Pushing ColorVectorSpace permanent tags: v0.1.2
ERROR: GitError(Code:ERROR, Class:Invalid, custom HTTP header 'p�{��' is malformed)
 [inlined code] from ./libgit2/error.jl:98
 in #push#44(::Bool, ::Base.LibGit2.PushOptions, ::Any, ::Base.LibGit2.GitRemote, ::Array{ASCIIString,1}) at ./libgit2/remote.jl:84
 [inlined code] from ./boot.jl:304
 in #push#85(::ASCIIString, ::ASCIIString, ::Array{ASCIIString,1}, ::Bool, ::Nullable{Base.LibGit2.UserPasswordCredentials}, ::Any, ::Base.LibGit2.GitRepo) at ./libgit2.jl:181
 [inlined code] from ./boot.jl:304
 in (::PkgDev.Entry.##6#9)(::Base.LibGit2.GitRepo) at /home/tim/.julia/v0.5/PkgDev/src/entry.jl:99
 in with(::PkgDev.Entry.##6#9, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:565
 in with(::Any, ::Type{Base.LibGit2.GitRepo}, ::ASCIIString, ::Vararg{ASCIIString}) at ./libgit2/types.jl:571
 in publish(::ASCIIString, ::ASCIIString) at /home/tim/.julia/v0.5/PkgDev/src/entry.jl:84
 in publish() at /home/tim/.julia/v0.5/PkgDev/src/PkgDev.jl:55
 in eval(::Module, ::Any) at ./boot.jl:237

I was able to push the tag manually with git push --follow-tags.

Use `http` protocol in tag url

URL in a tag should always be http(s), thus clickable, regardless of what URL stored in METADATA.

PkgDev has no versions that support 0.4.5 of julia

I'm not sure its an issue or if I am misunderstanding the way the package should work. When I call
Pkg.add("PkgDev") (just after calling Pkg.update()) I obtain

ERROR: PkgDev can't be installed because it has no versions that support 0.4.5 of julia.
You may need to update METADATA by running Pkg.update()
in error at error.jl:22
in resolve at ./pkg/entry.jl:410
in edit at pkg/entry.jl:26
in anonymous at task.jl:447
in sync_end at ./task.jl:413
[inlined code] from task.jl:422
in add at pkg/entry.jl:64
in add at pkg/entry.jl:73
in anonymous at pkg/dir.jl:31
in cd at file.jl:22
in cd at pkg/dir.jl:31
in add at pkg.jl:23

Pkg.generate fails if `origin` remote already exists

Original issue:
JuliaLang/julia#13527

tag() doesn't change anything in METADATA when package isn't registered

When calling PkgDev.tag() on a package that isn't registered in METADATA (neither locally or remotely), the tag is correctly created, but nothing else happens. This goes completely against the docs, and is really disturbing. Better raise an error.

The messages look like this:

julia> PkgDev.tag("FreqTables")
INFO: Tagging FreqTables v0.0.1

Remove Compat usage?

Given that this package is targeting v0.5 should the uses of Compat be removed?

Pkg.register on clean install fails with ENOENT

I download a nightly binary of v0.6.0-dev for the first time (no previous 0.6 julia has been installed on this machine)

I run

Pkg.update()

This successfully creates a v0.6 directory in ~/.julia and initializes it. I then clone an unregistered package

Pkg.clone("http://github.com/rprechelt/Vectorize.jl")

which successfully clones the package. I then try to register it

Pkg.add("PkgDev")
using PkgDev
PkgDev.register("Vectorize")

this last step fails with

INFO: Registering Vectorize at http://github.com/rprechelt/Vectorize.jl
INFO: Tagging Vectorize v0.0.1
ERROR: chdir METADATA: no such file or directory (ENOENT)
 in uv_error at ./libuv.jl:68 [inlined]
 in cd(::String) at ./file.jl:39
 in cd(::PkgDev.Entry.##15#18{Base.LibGit2.GitRepo,String,VersionNumber,String,Bool,Array{Base.Pkg.Reqs.Line,1}}, ::String) at ./file.jl:58
 in write_tag_metadata(::Base.LibGit2.GitRepo, ::String, ::VersionNumber, ::String, ::Bool) at /Users/rprechelt/.julia/v0.6/PkgDev/src/entry.jl:117
 in write_tag_metadata(::Base.LibGit2.GitRepo, ::String, ::VersionNumber, ::String) at /Users/rprechelt/.julia/v0.6/PkgDev/src/entry.jl:112
 in (::PkgDev.Entry.##20#28{String,String,String})(::Base.LibGit2.GitRepo) at /Users/rprechelt/.julia/v0.6/PkgDev/src/entry.jl:167
 in transact(::PkgDev.Entry.##20#28{String,String,String}, ::Base.LibGit2.GitRepo) at ./libgit2/libgit2.jl:513
 in register(::String, ::String) at /Users/rprechelt/.julia/v0.6/PkgDev/src/entry.jl:145
 in register(::String) at /Users/rprechelt/.julia/v0.6/PkgDev/src/entry.jl:194
 in register(::String) at /Users/rprechelt/.julia/v0.6/PkgDev/src/PkgDev.jl:36

Pkg.autoload("Package") ?

I was wondering if this had been discussed somewhere. I know some people are concerned that a lot of "core" (in a scientific not programming language way) functionality is being moved out of base. Prime number functionality could be such an example. Some people seem to be annoyed that they have to load stuff like Plots.jl or Primes.jl every time they run Julia. Would it make sense for Pkg to have a function that added the proper lines to the proper files on the different platforms, such that juliain the terminal would using A, B, C, ... where A, B and C had been passed to Pkg.autoload()?

Not really sure if this is the right place to ask, but Pkg discussions seem to happen here.

Pkg.submit not working on Windows?

I made some changes to a package (on my own branch, as suggested in the documentation). I did a commit, filled out a commit message in vim and after saving that and attempting to submit I get the following:

julia> Pkg.submit("PDMats")
INFO: Forking JuliaStats/PDMats.jl to KrisDM
ERROR: could not spawn `curl --version`: no such file or directory (ENOENT)
 in _jl_spawn at process.jl:262

There was a similar problem mentioned here:
JuliaLang/julia#5937

But that solution did not work for me (I don't seem to have .julia/v0.4/.github/token).

Append the package url to the tag/register commit message

This would make navigating from METADATA repo much easier.

Crash on submit

I tried to make a small change to a package and submit a pull request; I get a segmentation fault on the latest julia (Version 0.5.0-dev+3971 (2016-05-07 18:01 UTC))

julia> using PkgDev
WARNING: Base.ByteString is deprecated, use String instead.
  likely near /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:268

julia> PkgDev.submit("Nemo")
INFO: Forking wbhart/Nemo.jl to laurentbartholdi
WARNING: readall is deprecated, use readstring instead.
 [inlined code] from ./error.jl:26
 in depwarn(::String, ::Symbol) at ./deprecated.jl:64
 [inlined code] from ./strings/io.jl:51
 in readall(::Pipe, ::Vararg{Pipe,N}) at ./deprecated.jl:30
 in curl(::String, ::Cmd) at /Users/laurent/.julia/v0.5/PkgDev/src/github.jl:39
 [inlined code] from /Users/laurent/.julia/v0.5/PkgDev/src/github.jl:43
 in req(::String, ::Void, ::Cmd) at /Users/laurent/.julia/v0.5/PkgDev/src/github.jl:88
 [inlined code] from /Users/laurent/.julia/v0.5/PkgDev/src/github.jl:101
 in fork(::SubString{String}, ::SubString{String}) at /Users/laurent/.julia/v0.5/PkgDev/src/github.jl:114
 in (::PkgDev.Entry.##2#3)(::Base.LibGit2.GitRepo) at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:32
 in with(::PkgDev.Entry.##2#3, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:565
 in #pull_request#1(::String, ::String, ::String, ::Function, ::String) at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:13
 in submit(::String, ::String) at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:45
 [inlined code] from ./promotion.jl:248
 in (::Base.Pkg.Dir.##2#3{Array{Any,1},PkgDev.Entry.#submit,Tuple{String}})() at ./pkg/dir.jl:31
 in cd(::Base.Pkg.Dir.##2#3{Array{Any,1},PkgDev.Entry.#submit,Tuple{String}}, ::String) at ./file.jl:48
 in #cd#1(::Array{Any,1}, ::Function, ::Function, ::String, ::Vararg{String,N}) at ./pkg/dir.jl:31
 [inlined code] from ./pkg/dir.jl:25
 in submit(::String) at /Users/laurent/.julia/v0.5/PkgDev/src/PkgDev.jl:44
 in eval(::Module, ::Any) at ./boot.jl:230
while loading no file, in expression starting on line 0
INFO: Pushing changes as branch pull-request/ad61e0c0

signal (11): Segmentation fault: 11
while loading no file, in expression starting on line 0
git_smart__set_custom_headers at /usr/local/src/julia/usr/lib/libgit2.dylib (unknown line)
git_remote_connect at /usr/local/src/julia/usr/lib/libgit2.dylib (unknown line)
git_remote_push at /usr/local/src/julia/usr/lib/libgit2.dylib (unknown line)
[inline] at ./pointer.jl:17
push at ./libgit2/remote.jl:84
unknown function (ip: 0x32451f786)
push at ./libgit2.jl:181
unknown function (ip: 0x32451e6bb)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
unknown function (ip: 0x322dbe63d)
#2 at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:36
with at ./libgit2/types.jl:565
pull_request at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:13
unknown function (ip: 0x322dbd246)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
unknown function (ip: 0x322dba827)
submit at /Users/laurent/.julia/v0.5/PkgDev/src/entry.jl:45
[inline] at ./promotion.jl:248
#2 at ./pkg/dir.jl:31
cd at ./file.jl:48
unknown function (ip: 0x322dba25d)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
cd at ./pkg/dir.jl:31
[inline] at ./pkg/dir.jl:25
submit at /Users/laurent/.julia/v0.5/PkgDev/src/PkgDev.jl:44
unknown function (ip: 0x322db9119)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
do_call at /usr/local/src/julia/src/interpreter.c:58
eval at /usr/local/src/julia/src/interpreter.c:181
jl_toplevel_eval_flex at /usr/local/src/julia/src/toplevel.c:535
jl_toplevel_eval_in_warn at /usr/local/src/julia/src/builtins.c:547
eval at ./boot.jl:230
jlcall_eval_17829 at /usr/local/src/julia/usr/lib/julia/sys.dylib (unknown line)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
[inline] at ./REPL.jl:3
eval_user_input at ./REPL.jl:62
unknown function (ip: 0x322d83556)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
[inline] at ./REPL.jl:92
#1 at ./event.jl:46
unknown function (ip: 0x3154294ff)
[inline] at /usr/local/src/julia/src/./julia_internal.h:91
jl_call_unspecialized at /usr/local/src/julia/src/gf.c:1558
[inline] at /usr/local/src/julia/src/./julia.h:1381
jl_apply at /usr/local/src/julia/src/task.c:253
Allocations: 3348208 (Pool: 3346919; Big: 1289); GC: 6
Segmentation fault: 11

PkgDev.register failed with UndefVarError

Looks like it is refering to a function normalize_url that doesn't exist in Base.LibGit2.

julia> PkgDev.register("LibArchive")
ERROR: UndefVarError: normalize_url not defined
 in register at /usr/share/julia/site/v0.5/PkgDev/src/entry.jl:192

Ability to publish different branches?

This is admittedly likely to affect only a small number of people, but I wonder about whether it's reasonable to support publishing branches that are not metadata-v2. Now that I'm back working on packages, I often have 3 or more simultaneous PRs in to METADATA, and to avoid duplication I create separate branches. Even if we get to the point where Travis isn't the bottleneck in METADATA PRs, it's been demonstrated repeatedly that these PRs are still worth reviewing, so it seems likely that many PRs will be out there for at least several hours.

PkgDev.publish() seems to be broken

On both latest Julia and PkgDev:

julia> PkgDev.publish()
ERROR: MethodError: no method matching find_in_path(::SubString{UTF8String}, ::ASCIIString)
Closest candidates are:
  find_in_path(::Union{ASCIIString,UTF8String}, ::Any)
 in dir(::SubString{UTF8String}) at /home/jdlangs/.julia/v0.5/PkgDev/src/PkgDev.jl:21
 in (::PkgDev.Entry.##4#7{ASCIIString})(::Base.LibGit2.GitRepo) at /home/jdlangs/.julia/v0.5/PkgDev/src/entry.jl:69
 in with(::PkgDev.Entry.##4#7{ASCIIString}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:565
 in publish(::ASCIIString, ::ASCIIString) at /home/jdlangs/.julia/v0.5/PkgDev/src/entry.jl:51
 in publish() at /home/jdlangs/.julia/v0.5/PkgDev/src/PkgDev.jl:58
 in eval(::Module, ::Any) at ./boot.jl:236

Cannot access Github via ssh

I like to use an ssh url for Github repos (such as [email protected]:eschnett/FlexibleArrays.jl.git) because this allows me to store my password in a keychain, so that I don't have to type it in every time. This used to work fine in the past.

I now find that Julia cannot access Github via ssh any more, and I have to use the https protocol. This is both for reading and writing, even for read-only access to a public repository.

If this is a configuration error on my side, I apologize, but I don't recall configuring things in a particular way. Alternatively, if there is a way to set up a keychain for https, that would also be fine.

Pkg.submit suggestions #7820

Original issue: JuliaLang/julia#7820

PkgDev.publish() :: ERROR: GitError(Code:ERROR, Class:Net, Unsupported URL protocol)

INFO: Validating METADATA
INFO: Pushing OpenGene permanent tags: v0.1.0
ERROR: GitError(Code:ERROR, Class:Net, Unsupported URL protocol)
 [inlined code] from libgit2/error.jl:98
 in __push#345__ at libgit2/remote.jl:84
 in push at libgit2.jl:181
 in anonymous at /Users/shifu/.julia/v0.5/PkgDev/src/entry.jl:99
 in with at /usr/lib/julia/sys.dylib
 [inlined code] from libgit2/types.jl:487
 in publish at /Users/shifu/.julia/v0.5/PkgDev/src/entry.jl:84
 in publish at /Users/shifu/.julia/v0.5/PkgDev/src/PkgDev.jl:55
 in eval at /usr/lib/julia/sys.dylib