GithubHelp home page GithubHelp logo

tad-lispy / npm-git-install Goto Github PK

View Code? Open in Web Editor NEW
51.0 4.0 21.0 51 KB

Clones and (re)installs packages from remote git repos. See npm/npm#3055

CoffeeScript 100.00%
package package-manager package-json npm nodejs

npm-git-install's Introduction

This project was a crutch to work around a limitation in old versions of NPM. Since the introduction of the prepare hook, it should not be needed anymore.

NPM Git Install

Clones and (re)installs packages from remote git repos. It is meant as a temporary solution until npm/npm#3055 is resolved.

Install

npm install --save npm-git-install

Use

In your package.json add:

{
  "scripts": {
    "install": "npm-git install"
  }
  "gitDependencies": {
    "private-package-name": "[email protected]:user/repo.git#revision",
    "public-package-name": "https://github.com/user/repo.git#revision"
  }
}

Obviously replace *-package-name and git URLs with values relevant to your project. URLs has to be in canonical form (i.e. one that you would provide to git clone on command line) - no fancy NPM shortcuts like user/repo or bitbucket:user/repo. If you want this, we are open for a PRs.

Then install your dependencies as usual:

npm install

If you want to lock versions of git dependencies, use:

./node_modules/.bin/npm-git install --save

This will reinstall all git dependencies, but also write last matching commit's sha to package.json, effectively locking the versions.

You can also add a dependency and lock it's sha in one go:

./node_modules/.bin/npm-git install --save [email protected]:me/my-awesome-thing.git

This is probably the safest option, as it guarantees the same revision to be installed every time.

Now it should be easy to deploy, as long as the git executable is available in the environment.

Why

IMO there is a serious defect in current versions of NPM regarding installation process of dependencies from git repositories. It basically prevents us from installing anything that needs a build step directly from git repos. Because of that some authors are keeping build artifacts in the repos, which I would consider a hurdle at best. Here is relevant issue with ongoing discussion.

TL/DR:

If you npm install ../some-local-directory/my-package then npm will run prepublish script of the my-package and then install it in current project. This is fine.

One would expect that running npm install git@remote-git-server:me/my-package.git would also run prepublish before installing. Unfortunately it won't. Further more, it will apply .npmignore, which will most likely remove all your source files and make it hard to recover. Boo...

How

From command line

npm-git install

This simple script will do the following for every <url> of gitDependencies section of package.json:

  1. Clone the repo it into temporary directory

    using git clone <url>.

  2. Run npm install in this directory

    which will trigger prepublish hook of the package being installed.

  3. then run npm install <temporary directory> in your project path.

In effect you will get your dependency properly installed.

You can optionally specify different paths for package.json:

npm-git install -c git-dependencies.json

You may want to do this if you find it offensive to put non-standard section in your package.json.

Also try --help for more options.

Just like with plain NPM, on the command line you can specify a space separated list of packages to be installed:

npm-git install https://github.com/someone/awesome.git [email protected]/me/is-also-awesome.git#experimental-branch

After hash you can specify a branch name, tag or a specific commit's sha. By default master branch is used.

With --save option it will write the sha of tha HEAD (i.e. last matching commit) to the package.json, effectively locking the version of the dependency.

API

You can also use it programmatically. Just require npm-git-install. It exposes four methods:

  • discover (path)

    Reads list of packages from file at given path (e.g. a package.json) and returns array of {url, revision} objects. You can supply this to reinstall_all method.

  • reinstall_all (options, packages)

    Executes reinstall in series for each package in packages. Options are also passed to each reinstall call.

    This function is curried, so if you provide just options argument you will get a new function that takes only one argument - packages array.

    Options are the same as for reinstall.

    Returns a Promise that resolves to report, i.e. an array of metadata objects that you can pass to save. See below.

  • reinstall (options, package)

    Most of the heavy lifting happens here:

    1. Clones the repo at package.url,

    2. Checks out package.revision

    3. runs npm install at cloned repos directory

    4. installs the package from there.

    Options are:

    • silent: Suppress child processes standard output. Boolean. Default is false.
    • verbose: Print debug messages. Boolean. Default is false.

    Returns a Promise that will resolve to a metadata object:

    {
      name: "my-awesome-thing"
      sha: "ef88c40"
      url: "[email protected]/me/my-awesome-thing.git"
    }

    You probably don't want to use it directly. Just call reinstall_all with relevant options.

  • save (file, report)

    Takes a path to a package.json and an array of metadata (e.g. a report promised by reinstall_all). Updates the contents of the package.json file according to the report.

    Returns undefined.

If you are a Gulp user, then it should be easy enough to integrate it with your gulpfile. See [./src/cli.coffee][] for example use of the API.

Why not use dependencies and devDependencies

I tried and it's hard, because NPM supports fancy things as Git URLs. See messy-auto-discovery branch. You are welcome to take it from where I left.

There is also another reason. User may not want to reinstall all Git dependencies this way. For example I use gulp version 4, which is only available from GitHub and it is perfectly fine to install it with standard NPM. I don't want to rebuild it on my machine every time I install it. Now I can leave it in devDependencies and only use npm-git-install for stuff that needs it.

npm-git-install's People

Contributors

bfred-it avatar gagern avatar jrolfs avatar kintel avatar lambdatastic avatar mattvaughan avatar racerpeter avatar rom1504 avatar tad-lispy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

npm-git-install's Issues

Update nodegit

npm-git-install fails for me (node 5.6 on Mac) as it's using an old version of nodegit.
Any change you could upgrade to latest?

FYI: I'm using this to work around NPM issue #3055. Seems like a pretty clean workaround :)

Have a test suit

What I have in mind is to just set up a sample repo and during tests install it in a temporary directory. Successful installation is a passing test.

Broken links after git clone

Is this the intended behaviour, that when installing a npm package from my gogs git server, it's beeing cloned into the /tmp directory, then a link is created from the current project's node_modules to the directory from /tmp where everything has been cloned into, then said directory from /tmp is beeing deleted.

I don't understand what is going wrong here, nor why git repositories are beeing cloned into /tmp to stay there, as they will be gone after a quick reboot.

OS: Fedora 27
Version: [email protected]
Installed as sudo npm install -g npm-git-install

npm install succeeds but no temp directory is created for built package

Hey, this is a great package, it works for me up to a certain point.

I have a forked package from github, hosted under my own github account. I made sure node and npm were updated and followed the instructions for npm-git-install. It installed correctlyfrom my github to the point that the directory is created in node_modules, and it points to a temp directory. however the temp directory doesn't exist. I've done a --verbose install and it succeeds, but when I follow up with npm ls I get this output for that package:

├── UNMET DEPENDENCY package-name@file:../../../../../var/folders/df/gcycn6hj1w79lhy2r2575hrr0000gn/T/npm-git-11773-5388-j0u7av.tzkgfogvi

I'm guessing this might have something to do with permissions on my machine, but not sure. I wanted to ask if you've ever come across this. I'm using a mac, and it looks like the folder listed in the --verbose output, under /Users/myname/.npm is never created...

Any help or advice is appreciated. :)

Speed up install times for dependencies

I'm noticing if I switch b/w different SHAs or branches for a specified dependency in gitDependencies, it takes ~5 minutes to install. It seems like this is because of the dependencies of the source package it has to reinstall which should be able to leverage the cache since the package is mostly the same.

git clone hanging

For the last days I've been trying to use this package on a specific project but I'm facing an odd problem, the process blocks when it starts cloning the dependency and it hangs forever without any additional log.
This specific project works on other machines.

I'm using node 5.7.1, npm 3.6.0, git 2.6.4, OSX 10.11.3 on a Mac Pro Early 2015 if that helps in something.

I already tried to reinstall node, git and I also created a new user to test this with no success.

Is anyone facing the same problem? Any help is welcome, thank you.

Do not npm-git-install package if already installed

Is it possible to do the same as npm does.
So, i would expect, on npm install, if "gitDependencies" holds a package already installed,
no need to reinstall (git clone -> copy -> npm install) every time because it is time consuming.
We could simply check to see if package is already in node_modules.

add support for devDependencies

Unless I'm missing something, there doesn't seem to be a way to have gitDependencies for dev vs gitDependencies for production. Add support for gitDependencies and gitDevDependencies

Cannot find module '../build/Debug/nodegit.node'

Hi, I'm getting the following error when using your module.

Any ideas?

> npm-git-install

module.js:442
    throw err;
    ^

Error: Cannot find module '../build/Debug/nodegit.node'
    at Function.Module._resolveFilename (module.js:440:15)
    at Function.Module._load (module.js:388:25)
    at Module.require (module.js:468:17)
    at require (internal/module.js:20:19)
    at Object.<anonymous> (/xxx/node_modules/nodegit/lib/nodegit.js:17:12)
    at Module._compile (module.js:541:32)
    at Object.Module._extensions..js (module.js:550:10)
    at Module.load (module.js:458:32)
    at tryModuleLoad (module.js:417:12)
    at Function.Module._load (module.js:409:3)

npm ERR! Darwin 15.5.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "i"
npm ERR! node v6.2.1
npm ERR! npm  v3.9.3
npm ERR! code ELIFECYCLE
npm ERR! [email protected] install: `npm-git-install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'npm-git-install'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the angular2-seed package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     npm-git-install
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs angular2-seed
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls angular2-seed
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /xxx/npm-debug.log

I've tried node v5.6.0 and v6.2.1

Let me know if you want to see the npm-debug.log

Problem with [email protected]

Hi, I have updated npm to 5.0.3 and npm-git-install does not work, my git dependency is not installed. There is simlink in node_modules, but it refers to an unavailable location :(

lib directory is empty

This package does'n seem to work for my me, node_modules/package/lib folder is empty, but it compiles in temporary directory

npm version: 3.10.8

Only works with branch names, not commit hashes

If I try to put the following in gitDependencies:

    "octokat": "https://github.com/divdavem/octokat.js.git#b095a62"

then I get [Error: Reference 'refs/remotes/origin/b095a62' not found]. Same with a long SHA. Can we make this work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.