heroku / heroku-buildpack-nodejs Goto Github PK

View Code? Open in Web Editor NEW

1.3K 125.0 2.6K 124.56 MB

Heroku's buildpack for Node.js applications.

Home Page: https://devcenter.heroku.com/articles/buildpacks

License: MIT License

Shell 79.06% Makefile 0.82% Ruby 8.10% JavaScript 8.63% TypeScript 3.39%

heroku-languages buildpack heroku nodejs

heroku-buildpack-nodejs's Introduction

Heroku Buildpack for Node.js

This is the official Heroku buildpack for Node.js apps.

Documentation

For more information about using this Node.js buildpack on Heroku, see these Dev Center articles:

For more general information about buildpacks on Heroku:

Using the Heroku Node.js buildpack

It's suggested that you use the latest version of the release buildpack. You can set it using the heroku-cli.

heroku buildpacks:set heroku/nodejs

Your builds will always used the latest published release of the buildpack.

If you need to use the git url, you can use the latest tag to make sure you always have the latest release. The main branch will always have the latest buildpack updates, but it does not correspond with a numbered release.

heroku buildpacks:set https://github.com/heroku/heroku-buildpack-nodejs#latest -a my-app

Locking to a buildpack version

Even though it's suggested to use the latest release, you may want to lock dependencies - including buildpacks - to a specific version.

First, find the version you want from the list of buildpack versions. Then, specify that version with buildpacks:set:

heroku buildpacks:set https://github.com/heroku/heroku-buildpack-nodejs#v176 -a my-app

Chain Node with multiple buildpacks

This buildpack automatically exports node, npm, and any node_modules binaries into the $PATH for easy use in subsequent buildpacks.

Feedback

Having trouble? Dig it? Feature request?

Development

Prerequisites

For local development, you may need the following tools:

Deploying an app with a fork or branch

To make changes to this buildpack, fork it on GitHub. Push up changes to your fork, then create a new Heroku app to test it, or configure an existing app to use your buildpack:

# Create a new Heroku app that uses your buildpack
heroku create --buildpack <your-github-url>

# Configure an existing Heroku app to use your buildpack
heroku buildpacks:set <your-github-url>

# You can also use a git branch!
heroku buildpacks:set <your-github-url>#your-branch

Downloading Plugins

In order to download the latest plugins that have been released, run the following:

plugin/download.sh v$VERSION

Make sure the version is in the format v#, ie. v7.

Tests

The buildpack tests use Docker to simulate Heroku's stacks.

To run the test suite:

make test

Or to just test a specific stack:

make heroku-20-build
make heroku-22-build

The tests are run via the vendored shunit2 test framework.

Debugging

To display the logged build outputs to assist with debugging, use the "echo" and "cat" commands. For example:

test() {
  local log_file var

  var="testtest"
  log_file=$(mktemp)
  echo "this is the log file" > "$log_file"
  echo "test log file" >> "$log_file"

  # use `echo` and `cat` for printing variables and reading files respectively
  echo $var
  cat $log_file

  # some cases when debugging is necessary
  assertEquals "$var" "testtest"
  assertFileContains "test log file" "$log_file"
}

Running the test above would produce:

testtest
this is the log file
test log file

The test output writes to $STD_OUT, so you can use cat $STD_OUT to read output.

heroku-buildpack-nodejs's People

Contributors

Stargazers

Watchers

Forkers

metadaddy juggy andreif ddura wookiehangover goodeggs bsatrom joedevon attilagyorffy jkeyes markpundsack mies petejkim epictetus aomitayo sledorze envylabs collin bcherry cpj shopwise swbiggart claudiosdc conancat ericallam moowahaha lloyd solsys minefold rileydutton screenspeak tracksby gbakernet ricaurte slant astrails bmavity bloomtime thegoleffect polystorym audiosocket alxgsv imikushin mateodelnorte ctbarna andychu akume buzzedword imrehg prasincs jorisroling xsyn claytonbrown zbuc qurio-za bbn maccman jp-a arttu jstallings femto113 joshvermaire mypark sethbridges hokaccha dtan coolfront-technologies lezhangxyz denisjacquemin lackac joshskeen timshadel gabesoft webmillstudios denisbollinne lynns candland dtjm pmgodin-zz justinbeckwith jasherai raycmorgan mash180sx tleach skimcom timruffles mhuebert catesandrew intxi ken-okabe bpartridge val9 vieron brenthargrave matb33 tarkus nickmerwin gabrielmancini weotch haileys

heroku-buildpack-nodejs's Issues

node 0.8.x requires specifying npm as an engine in package.json

I just moved to node 0.8.x by specifying in my package.json...

  "engines": {
    "node": "0.8.x"
  }

When I did that, I got this crazy useless error message:

-----> Heroku receiving push
-----> Node.js app detected
-----> Resolving engine versions
       Using Node.js version: 0.8.11
       Using npm version: 1.0.106
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
       Error: npm doesn't work with node v0.8.11
       Required: [email protected] || 0.5 || 0.6
           at /tmp/node-npm-LvfM/bin/npm-cli.js:57:23
           at Object.<anonymous> (/tmp/node-npm-LvfM/bin/npm-cli.js:77:3)
           at Module._compile (module.js:449:26)
           at Object.Module._extensions..js (module.js:467:10)
           at Module.load (module.js:356:32)
           at Function.Module._load (module.js:312:12)
           at Module.require (module.js:362:17)
           at require (module.js:378:17)
           at Object.<anonymous> (/tmp/node-npm-LvfM/cli.js:2:1)
           at Module._compile (module.js:449:26)
       Error: npm doesn't work with node v0.8.11
       Required: [email protected] || 0.5 || 0.6
           at /tmp/node-npm-LvfM/bin/npm-cli.js:57:23
           at Object.<anonymous> (/tmp/node-npm-LvfM/bin/npm-cli.js:77:3)
           at Module._compile (module.js:449:26)
           at Object.Module._extensions..js (module.js:467:10)
           at Module.load (module.js:356:32)
           at Function.Module._load (module.js:312:12)
           at Module.require (module.js:362:17)
           at require (module.js:378:17)
           at Object.<anonymous> (/tmp/node-npm-LvfM/cli.js:2:1)
           at Module._compile (module.js:449:26)
       Dependencies installed
-----> Building runtime environment
-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size: 3.6MB
-----> Launching... done, v35

Specifying the npm version as * gives me this:

-----> Heroku receiving push
-----> Node.js app detected
-----> Resolving engine versions
       Using Node.js version: 0.8.11
 !     Requested engine npm version HEAD branches config description hooks info objects refs slugc_lock does not match available versions: 1.1.49 1.1.41 1.1.40 1.1.39 1.1.36 1.1.35 1.1.32 1.1.9 1.1.4 1.1.1 1.0.106 1.0.94 
 !     Heroku push rejected, failed to compile Node.js app

What finally worked:

  "engines": {
    "node": "0.8.x",
    "npm": "1.1.x"
  }

Ideally, the error messages would be a bit more helpful.

deploy failure

Just started happening (presumably due to the new buildpack that was just deployed):

   npm ERR! Error: EACCES, permission denied '/tmp/npm-1332887811645/1332887812002-0.4976726034656167/___package.npm/package/bin/hulk'
   npm ERR! 
   npm ERR! Please try running this command again as root/Administrator.
   npm ERR! 
   npm ERR! System Linux 2.6.32-343-ec2
   npm ERR! command "/tmp/node-node-iI3I/bin/node" "/tmp/node-npm-7Z0J/cli.js" "install" "--production"
   npm ERR! cwd /tmp/build_34bw89b8z5nvy
   npm ERR! node -v v0.6.12
   npm ERR! npm -v 1.1.4
   npm ERR! path /tmp/npm-1332887811645/1332887812002-0.4976726034656167/___package.npm/package/bin/hulk
   npm ERR! code EACCES
   npm ERR! message EACCES, permission denied '/tmp/npm-1332887811645/1332887812002-0.4976726034656167/___package.npm/package/bin/hulk'
   npm ERR! errno {}
   npm ERR! 
   npm ERR! Additional logging details can be found in:
   npm ERR!     /tmp/build_34bw89b8z5nvy/npm-debug.log

cc @ddollar

don't let nomnom break the build

Wrap the nomnom curl command in a subprocess, à la https://gist.github.com/ddollar/d5231397881ab3314cd9

Error fetching buildpack

buildpack for node.js cannot be fetched:

[ubuntu@ip-10-164-57-85:~/bitstarter]$git push staging-heroku staging:master
Counting objects: 456, done.
Compressing objects: 100% (323/323), done.
Writing objects: 100% (456/456), 259.62 KiB, done.
Total 456 (delta 122), reused 428 (delta 103)

-----> Fetching custom git buildpack... failed

! Push rejected, error fetching custom buildpack

To [email protected]:warm-oasis-1494.git
! [remote rejected] staging -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:warm-oasis-1494.git'

What do you suggest to do?

I reported the problem also in this thread where other people are encountering it:
heroku/heroku-buildpack-go#8 (comment)

Looking forward to hear from you.
Marco

Cache cleaning

If you remove a package, it is not removed from the cache and those packages are re-installed.

To reproduce: I did a wholesale change of the package.json (was actually a completely different app that I force pushed). The original package.json included mongodb 0.9.6. The second one did not. When I deployed, I saw:

-----> Installing dependencies with npm 1.0.8

       > [email protected] install /tmp/build_2olrpgftd8j1v/node_modules/mongodb
       > bash ./install.sh

       ================================================================================
       =                                                                              =
       =  To install with C++ bson parser do <npm install mongodb --mongodb:native>   =
       =  the parser only works for node 0.4.X or lower                               =
       =                                                                              =
       ================================================================================
       Not building native library for cygwin
       [email protected] ./node_modules/ejs 
       Dependencies installed

Allow to specify custom registry URL to override the default

A number of our customers would like to host all their dependencies in Gemfury or use a hierchical proxy like https://npmjs.org/package/kappa. This issue is to discuss adding a configurable option to specify a custom registry.

The options I'm thinking are:

Specify a field in package.json, like publishConfig['registry']
NPM_REGISTRY_URL environment variable and user-env-compile Heroku labs feature

Honor bundledDependencies in package.json

I'm not sure this is the right repo to post this but here goes.

It is apparent from the deployment log that npm prune is invoked to remove unused dependencies. Unused dependencies are defined as packages not listed in the dependencies list, as per npm docs.

Using npm prune creates a problem when the node_modules folder is tracked in git and the bundledDependencies key is used in the package.json file...

I'm not sure if there's any point pursuing a definition change for npm prune so i have to start here and ask if that command could possibly be removed from the building scripts?

Heroku can't find node with latest build

I just deployed with the latest build, and heroku reports 'node' cannot be found. I reverted to #legacy and it works fine.

Am I going crazy or did something just go wrong?

I am in a multipack environment. Using https://github.com/ddollar/heroku-buildpack-multi.git

.buildpack
https://github.com/heroku/heroku-buildpack-nodejs.git#legacy
https://github.com/virtualstaticvoid/heroku-buildpack-r.git

with a Procfile

worker: node server.js

Can't use commit SHA in bower

@zeke we chatted a while back regarding:

legacy node.js buildpack works just fine with commit SHAs in bower.json (a la <package>#<sha>, docs)
however, new build pack does not, throwing a bunch of verbose rebuildBundles, silly lockfiles, etc.

It's strange, since SHAs work just fine on local, heroku + legacy-buildpack, and other servers. And assuming bower uses npm under the hood for it's json dependencies handling, the fact that SHAs work fine in npm.json in heroku's new buildpack is strange/inconsistent.

Anyway. Because of this issue, I've just been using legacy up until now, no problem. Well what with the whole 2/28 npm fiasco, we need to use the latest buildpack to get the fix. So I had to upgrade by removing SHA deps. Eventually Heroku should support SHAs in bower (example of why it's important), so just a heads up on this.

Provide a safety net for semver.io failures.

Catch failures here and either fail fast or default to some stable node_version.

# Resolve node version using semver.io
semver_url=http://semver.io/node/$semver_range
node_version=$(curl --silent $semver_url)

Error compiling app with diet branch

I'm getting an error compiling my app with the diet branch. Here's the error with debug turned on:

... LOTS OF STUFF AND NPM COMPILES ...
+ status 'Caching node_modules for future builds'
+ echo '-----> Caching node_modules for future builds'
-----> Caching node_modules for future builds
+ rm -rf /app/tmp/repo.git/.cache/88e62695588485e80bf95c03af115c77
+ mkdir -p /app/tmp/repo.git/.cache/88e62695588485e80bf95c03af115c77
+ test -d /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/node_modules
+ cp -r /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/node_modules /app/tmp/repo.git/.cache/88e62695588485e80bf95c03af115c77/
+ status 'Building runtime environment'
+ echo '-----> Building runtime environment'
-----> Building runtime environment
+ mkdir -p /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/.profile.d
+ echo 'export PATH="$HOME/vendor/node/bin:$HOME/bin:$HOME/node_modules/.bin:$PATH"'
+ status 'Posting to nomnom'
+ echo '-----> Posting to nomnom'
-----> Posting to nomnom
+ cat /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/npm-debug.log
cat: /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/npm-debug.log: No such file or directory
++ cat_npm_debug_log
++ test -f /tmp/build_a1ae5e28-a146-4878-9f61-71625535bf63/npm-debug.log

 !     Push rejected, failed to compile Multipack app

To [email protected]:playlist-com-staging.git
 ! [remote rejected] fix-buildpack -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:playlist-com-staging.git'

This is the modified copy of the compile script I'm using: https://github.com/playlist-media/heroku-buildpack-nodejs/blob/diet/bin/compile - I enabled debug as well as outputting npm-debug.log if it exists.

Diet Buildpack: Caching ignores postinstall

If the package.json and the node version are unchanged, then the node_modules are restored from cache. However a npm postinstall option is ignored, too.

The postinstall can be used to compile some assets for production use (grunt tasks for example). In this case these tasks are not executed.

Update npm from 1.0.8 to...

We're seeing an increasing number of tickets regarding empty "lib" folders, which seems to have something to do with newer packages and our npm version. I'm not sure what the underlying cause is, but updating npm seems to be recommended.

Updating node might be a good idea as well.

buildpack is downloading nodejs-0.4.7 regardless of engine version?

buildpack is downloading nodejs-0.4.7.tgz even though it's not being used

npm rebuild isn't always necessary

npm rebuild should only be necessary if a node_modules/ directory already exists (otherwise it does twice as much work). If not, there's nothing to rebuild.

It might even make sense to run npm rebuild before npm install (in which case it noops on an empty node_modules/), as that will only rebuild modules that need it (and install would fill in whatever's missing).

support bundled dependencies

It looks like the bin/compile script is explicitly removing any node_modules directory that comes along with the git push. This is exactly the wrong thing for a deployment tool to do. Here's a great explanation of some of the reasons why: http://www.mikealrogers.com/posts/nodemodules-in-git.html

Here's what the thing should do:

check out the received pack from git, node_modules and all.
in the checkout dir, do npm install with no arguments, to install any missing deps locally.
Do npm rebuild for any bundled modules that need to be compiled for the different architecture.

Skip pruning and caching for apps with node_modules checked in.

From a customer ticket:

PROBLEM: Step 6, npm prune, is deleting all my npm modules, because all my modules are marked as "extraneous," because I don't use package.json.

SOLUTION: Only execute steps 6 (npm prune) and 7 (cache node_modules) if node_modules is not checked into git. The cache is not used in that case anyway.

Remove the default `ca` setting (now that npm's SSL cert is rolled back)

npm, Inc rolled out a fix to their cert on Saturday: https://twitter.com/npmjs/status/439893976867303424

If this fix works, we should be able to revert this commit to the buildpack: #98

latest node and npm version by default

The default node and npm version are:

engine_defaults["node"]="0.4.7"
engine_defaults["npm"]="1.0.106"

It would make more sense to use the latest versions.

0.4.7 is quite deprecated now, and it not going to be get better.

Now that node and npm are distributed together, it would also be great to have the matching npm version when only node is specified in the package.json.

Cheers

Romain

Problem with Git URLs as Dependencies

I have some dependencies like this: "module": "git+https://bitbucket.org/company/module.git#master".

Everytime I need to get the last commit from this repository I modify the package.json file (add some blank space or something like that) to make the buildpack reload all the dependencies again.

After this commit: #67, modifying the package.json file don't work any more, the dependency is always from cache. The only workaround I have found is to remove the dependency, commit, push, add the dependency again, commit and push.

New buildpack breaks package

There is some strange dependency error breaking on the new buildpack with bower. Running against legacy buildpack, it works fine.

Error

bower dustjs-linkedin#2.0.4                   ECMDERR Failed to execute "git checkout 07fec85e661ac38c2b73f7538d177f0ca45ef179", exit code of #128

       Additional error details:
       fatal: Not a git repository: '.'

You can see the repo that I am trying to deploy to heroku at https://github.com/kvarga/lavaca-starter-with-heroku

I'm continuing to look into this, but wanted to go ahead and open the ticket in case someone had any ideas.

always end up with a corrupted 64 kb node file

Trying to make a custom buildpack but running into some problems. I followed the steps described in this blog: http://blog.superpat.com/2011/11/15/running-your-own-node-js-version-on-heroku/ in addition to the readme.
When I run the package_node script, everything seems to be fine (= no errors) but the resulting file in the S3 bucket is only 64 KB and corrupted.
I tried updating the vulcan build server but I am still running into the same issues.

From time to time, the script also stops with the following error after the build step:

Building with: ./configure --prefix /app/vendor/node-v0.6.7 && make install
error: null/Users/matthias/.rvm/rubies/ruby-1.9.2-p136/lib/ruby/1.9.1/net/protocol.rb:135:in `read_nonblock': end of file reached (EOFError)

Any ideas what could be the problem?

gzip: stdin: not in gzip format

Hi Guys,

I am trying upping the version of nodejs in heroku
so following the steps to hack the buildpack, i made the necessary changes, but when i push the project the deploy show me this error:

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Exiting with failure status due to previous errors

 !     Push rejected, failed to compile Node.js app

any one pass thru for this?

Error installing dependencies with npm

-----> Removing .DS_Store files
-----> Node.js app detected
-----> Resolving engine versions
Using Node.js version: 0.8.14
Using npm version: 1.1.65
-----> Fetching Node.js binaries
-----> Vendoring node into slug
-----> Installing dependencies with npm
npm ERR! Error: ENOENT, chmod '/tmp/build_mbwsy9saim54/node_modules/express/bin/express'
npm ERR! If you need help, you may report this log at:
npm ERR! http://github.com/isaacs/npm/issues
npm ERR! or email it to:
npm ERR! [email protected]
npm ERR! System Linux 2.6.32-350-ec2
npm ERR! command "/tmp/node-node-RV0B/bin/node" "/tmp/node-npm-eoMu/cli.js" "rebuild"
npm ERR! cwd /tmp/build_mbwsy9saim54
npm ERR! node -v v0.8.14
npm ERR! npm -v 1.1.65
npm ERR! path /tmp/build_mbwsy9saim54/node_modules/express/bin/express
npm ERR! code ENOENT
npm ERR! errno 34
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /tmp/build_mbwsy9saim54/npm-debug.log
npm ERR! not ok code 0
! Failed to rebuild dependencies with npm
! Heroku push rejected, failed to compile Node.js app

Respect `npm-shrinkwrap.json` (analogous to Ruby Bundler's `Gemfile.lock`)

Software rot is bad. Ruby Bundler solved this problem with gemfile.lock. Explicit version contracts are good.

Npm (as of version 1.1.2) has an analogous feature, npm shrinkwrap which saves a file npm-shrinkwrap.json analogous to Gemfile.lock.

Reportedly, Heroku doesn't respect npm shrinkwrap. That post is a few months old, so maybe it does now. I'll check.

Regardless, the Heroku docs don't mentionnpm shrinkwrap https://devcenter.heroku.com/articles/nodejs . They should. Maybe shrinkwrapping should be made mandatory (like Ruby Heroku makes gemfile.lock mandatory) so slug compilation is deterministic and to make users safe from software rot.

Frequent timeouts during deploy when compiling native bindings

I've been experiencing relatively frequent failures when deploying my app to Heroku, that seem to come from timeouts while compiling NPM packages with native bindings. Right now, I'd estimate that this happens at least 10% of the time.

Here are all of the packages that have native bindings that are in my dependency tree:

time
pg
hiredis
bson (twice I think, once from mongoose, once from mongodb)
mongodb

Is anyone else experiencing this? It's pretty painful, I have to Ctrl-C the deploy and try again every time this happens. If I wait it out, I think I eventually get a timeout error.

Thanks for your help!

EDIT: As @ChrisMCC pointed out, @randypuro also is having similar issues that he cited in the follow up on issue #17: "the deploy often freezes for us in the midst of downloading all the dependencies."

Improve npm dependency performance

Right now, deploying nodejs on heroku involves a npm reinstall on every deploy, which quickly adds up as packages need to recompile themselves, etc.

Could we have a dependency caching system on par with the Ruby/Rails buildpack's, where we only pull down build artifacts with the engine or package version changes?

ENOENT /app/tmp after new buildpack

It seems PR #71 has broken my application. I am currently using node-temp for creating temp files for image processing.

Now whenever I go to open I temp file I get this error:

ENOENT, open '/app/tmp/product_type_3_front113113-2-bzxj7i.png'

What is a good solution for this? Should I just point my build-pack to an older version? Thankfully this app was in production yet, but I was planning to tomorrow. Is there some way of finding out when these types of sweeping changes to the buildpack are going to be released before hand?

Is there a way to run multiple node processes?

A Heroku dyno has 4 CPUs, is there a way to run multiple node.js processes with this buildpack? If not, can I use node cluster? How does it work regarding the auto-restarting policy, e.g. if the cluster process crashes, does Heroku auto-restart it?

Upgrade from Node 0.4 to 0.6 doesn't work when having native packages

We have run into an issue today on Heroku trying to upgrade our Node version. The problem was caused by having hiredis in our dependencies which the buildpack couldn't rebuild correctly. The build itself run without errors but we have seen a lot of errors in the logs:

2012-02-27T15:09:27+00:00 app[web.1]: FATAL ERROR: v8::HandleScope::Close() Local scope has already been closed

Looking into the issue I have found that it must have been caused by having the old node version in the path. I'm not sure about this but I suspect that it must have come from the slug cache.

Trying to solve the issue I removed all the native packages from our package.json. This didn't help as another strange thing happened: all the packages that had been removed were still being installed when deploying.

I think this is caused by the buildpack messing up cache directories. I looked into the compile script and found this:

CACHE_STORE_DIR="$CACHE_DIR/node_modules/$NPM_VERSION"
CACHE_TARGET_DIR="$BUILD_DIR/node_modules"

Here you're using $NPM_VERSION without first finding the right value with package_resolve_version. I don't know what the default value is for this variable but I suspect that it's either undefined or a fixed 1.0.94. Either way the buildpack copies old packages over to the new build even when the app is trying to use another npm version.

We temporarily solved the issue in our case by enabling the user_env_compile labs plugin and setting NPM_VERSION (and NODE_VERSION just to be safe) explicitely. With that the build process started with a clean node_modules dir and everything (even native packages) worked perfectly.

npm WARN cannot run in wd for all commands

I want to bower install here. I tried echo blah and whoami but no commands will run in this working directory.

Do you have any ideas how I can debug this further?

=====> Detected Framework: Node.js
-----> Requested node range:  0.10.24
-----> Resolved node version: 0.10.24
-----> Downloading and installing node
-----> Restoring node_modules directory from cache
-----> Pruning cached dependencies not specified in package.json
-----> Installing dependencies
       npm WARN cannot run in wd [email protected] echo blah (wd=/build/app)
-----> Caching node_modules directory for future builds
-----> Cleaning up node-gyp and npm artifacts
-----> Building runtime environment

Support a cached .heroku folder like the python buildpack

This folder is useful when you want to cache downloads between builds:
https://github.com/heroku/heroku-buildpack-python/blob/master/bin/compile#L117

Would be great to have it available in the nodejs buildpack as well to speed up build times!

Thanks!

Don't warn users when they don't specify a node/npm version

Omitting engines from package.json is normal behavior, and should not generate a warning:

 WARNING: No version of Node.js specified in package.json, see:
   https://devcenter.heroku.com/articles/nodejs-support#versions

Should `NODE_ENV` be set by the buildpack?

As @aseemk notes on the Heroku Node forum:

This might not be new to the diet buildpack, but I just noticed that the Node docs no longer say you guys set NODE_ENV=production. Is that right? If so, may I ask why the change?

Dev Center has two articles that reference NODE_ENV=production, one about Express caching and the other a sample .profile.d. script.

Dare I ask, should NODE_ENV be set automatically by the buildpack? PATH is currently the only environment variable it touches.

cc @ddollar @dpiddy @jclem @mojodna @ryanbrainard @rwdaigle @friism

New buildpack fails while executing postinstall script

My project executes bower install as a npm postinstall script in order to install server-side dependencies. In a certain configuration, i.e. when two bower packages have the same dependency the newest version of buildpack crashes with a message:

ENOENT ENOENT, lstat '/app/.cache/bower/packages/3725aca888af41d0b2de2b0b81f8307b/1.10.3/themes/blitzer/images/ui-icons_cc0000_256x240.png'

The build runs with no problem after switching to legacy branch of buildpack. If you remove one of two dependecies from bower.json the problem is also resolved. However, I think it's a buildpack issue and not a bower one, because changing the version of bower does not help, while returning to previous version of the buildpack does.

Below I attach the smallest configuration found to reproduce the bug and a log of failed build.

package.json

{
  "name": "heroku-buildpack-nodejs-app",
  "version": "0.0.1",
  "dependencies": {
    "bower": "~1.2.8"
  },
  "scripts": {
    "postinstall": "./node_modules/.bin/bower install --production"
  },
  "engines": {
    "node": "0.10.5",
    "npm": "1.2.18"
  }
}

bower.json

{
  "name": "heroku-buildpack-nodejs-app",
  "version": "0.0.1",
  "dependencies": {
    "jquery-ui": "~1.10.3",
    "jquery.drag-n-crop": "https://github.com/lukaszfiszer/drag-n-crop.git"
  }
}

.gitignore

node_modules/
bower_components/

log

$ git push heroku master --force
Fetching repository, done.
Counting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 307 bytes | 0 bytes/s, done.
Total 3 (delta 2), reused 0 (delta 0)

-----> Fetching custom git buildpack... done
-----> Node.js app detected
-----> Requested node range:  0.10.5
-----> Resolved node version: 0.10.5
-----> Downloading and installing node
-----> Restoring node_modules directory from cache
-----> Pruning cached dependencies not specified in package.json
       npm WARN package.json [email protected] No README.md file found!
-----> Installing dependencies
       npm WARN package.json [email protected] No README.md file found!

       > [email protected] postinstall /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b
       > ./node_modules/.bin/bower install --production

       bower jquery.drag-n-crop#*  not-cached https://github.com/lukaszfiszer/drag-n-crop.git#*
       bower jquery.drag-n-crop#*     resolve https://github.com/lukaszfiszer/drag-n-crop.git#*
       bower jquery.drag-n-crop#*    checkout master
       bower jquery-ui#~1.10.3     not-cached git://github.com/components/jqueryui.git#~1.10.3
       bower jquery-ui#~1.10.3        resolve git://github.com/components/jqueryui.git#~1.10.3
       bower jquery-ui#~1.10.3       download https://github.com/components/jqueryui/archive/1.10.3.tar.gz
       bower jquery-ui#~1.10.3        extract archive.tar.gz
       bower jquery.drag-n-crop#*  deprecated Package jquery.drag-n-crop is using the deprecated component.json
       bower jquery.drag-n-crop#*    resolved https://github.com/lukaszfiszer/drag-n-crop.git#fd75a871c3
       bower jquery-ui#~1.10.1     not-cached git://github.com/components/jqueryui.git#~1.10.1
       bower jquery-ui#~1.10.1        resolve git://github.com/components/jqueryui.git#~1.10.1
       bower jquery-ui#~1.10.1       download https://github.com/components/jqueryui/archive/1.10.3.tar.gz
       bower jquery-ui#~1.10.1        extract archive.tar.gz
       bower jquery#~1.9.1         not-cached git://github.com/components/jquery.git#~1.9.1
       bower jquery#~1.9.1            resolve git://github.com/components/jquery.git#~1.9.1
       bower jquery#~1.9.1           download https://github.com/components/jquery/archive/1.9.1.tar.gz
       bower jquery#~1.9.1            extract archive.tar.gz
       bower jquery#~1.9.1         deprecated Package jquery is using the deprecated component.json
       bower jquery#~1.9.1           resolved git://github.com/components/jquery.git#1.9.1
       bower jquery-ui#~1.10.1       resolved git://github.com/components/jqueryui.git#1.10.3
       bower jquery-ui#~1.10.3         ENOENT ENOENT, lstat '/app/.cache/bower/packages/3725aca888af41d0b2de2b0b81f8307b/1.10.3/themes/blitzer/images/ui-icons_cc0000_256x240.png'

       Stack trace:
       /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/fstream/lib/writer.js:284:26
       Object.oncomplete (fs.js:107:15)

       Console trace:
       Trace
           at StandardRenderer.error (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/lib/renderers/StandardRenderer.js:74:17)
           at Logger.updateNotifier.packageName (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/bin/bower:109:18)
           at Logger.EventEmitter.emit (events.js:95:17)
           at Logger.emit (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/bower-logger/lib/Logger.js:29:39)
           at /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/lib/commands/install.js:27:16
           at _rejected (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/q/q.js:808:24)
           at /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/q/q.js:834:30
           at Promise.when (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/q/q.js:1079:31)
           at Promise.promise.promiseDispatch (/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/q/q.js:752:41)
           at /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/node_modules/q/q.js:574:44

       System info:
       Bower version: 1.2.8
       Node version: 0.10.5
       OS: Linux 3.8.11-ec2 x64
       npm ERR! [email protected] postinstall: `./node_modules/.bin/bower install --production`
       npm ERR! `sh "-c" "./node_modules/.bin/bower install --production"` failed with 1
       npm ERR!
       npm ERR! Failed at the [email protected] postinstall script.
       npm ERR! This is most likely a problem with the heroku-buildpack-nodejs-app package,
       npm ERR! not with npm itself.
       npm ERR! Tell the author that this fails on your system:
       npm ERR!     ./node_modules/.bin/bower install --production
       npm ERR! You can get their info via:
       npm ERR!     npm owner ls heroku-buildpack-nodejs-app
       npm ERR! There is likely additional logging output above.

       npm ERR! System Linux 3.8.11-ec2
       npm ERR! command "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/node" "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/npm" "install" "--userconfig" "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/.npmrc" "--production"
       npm ERR! cwd /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b
       npm ERR! node -v v0.10.5
       npm ERR! npm -v 1.2.18
       npm ERR! code ELIFECYCLE
       npm ERR!
       npm ERR! Additional logging details can be found in:
       npm ERR!     /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/npm-debug.log
       npm ERR! not ok code 0
0 info it worked if it ends with ok
1 verbose cli [ '/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/node',
1 verbose cli   '/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/npm',
1 verbose cli   'install',
1 verbose cli   '--userconfig',
1 verbose cli   '/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/.npmrc',
1 verbose cli   '--production' ]
2 info using [email protected]
3 info using [email protected]
4 verbose read json /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/package.json
5 warn package.json [email protected] No README.md file found!
6 verbose readDependencies using package.json deps
7 verbose install where, deps [ '/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b',
7 verbose install   [ 'bower' ] ]
8 verbose from cache /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/package.json
9 info preinstall [email protected]
10 verbose read json /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/package.json
11 verbose from cache /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/package.json
12 verbose readDependencies using package.json deps
13 verbose from cache /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/node_modules/bower/package.json
14 verbose already installed skipping bower@~1.2.8 /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b
15 silly resolved []
16 info build /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b
17 verbose from cache /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/package.json
18 verbose linkStuff [ false, false, false, '/tmp' ]
19 info linkStuff [email protected]
20 verbose linkBins [email protected]
21 verbose linkMans [email protected]
22 verbose rebuildBundles [email protected]
23 verbose rebuildBundles [ '.bin', 'bower' ]
24 info install [email protected]
25 info postinstall [email protected]
26 verbose unsafe-perm in lifecycle true
27 silly exec sh "-c" "./node_modules/.bin/bower install --production"
28 silly sh,-c,./node_modules/.bin/bower install --production,/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b spawning
29 info [email protected] Failed to exec postinstall script
30 error [email protected] postinstall: `./node_modules/.bin/bower install --production`
30 error `sh "-c" "./node_modules/.bin/bower install --production"` failed with 1
31 error Failed at the [email protected] postinstall script.
31 error This is most likely a problem with the heroku-buildpack-nodejs-app package,
31 error not with npm itself.
31 error Tell the author that this fails on your system:
31 error     ./node_modules/.bin/bower install --production
31 error You can get their info via:
31 error     npm owner ls heroku-buildpack-nodejs-app
31 error There is likely additional logging output above.
32 error System Linux 3.8.11-ec2
33 error command "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/node" "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/vendor/node/bin/npm" "install" "--userconfig" "/tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b/.npmrc" "--production"
34 error cwd /tmp/build_d512cc52-807a-4241-b52a-2d221db55d1b
35 error node -v v0.10.5
36 error npm -v 1.2.18
37 error code ELIFECYCLE
38 verbose exit [ 1, true ]

 !     Push rejected, failed to compile Node.js app

To [email protected]:heroku-buildpack-nodejs-app.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:heroku-buildpack-nodejs-app.git'

Caching node_modules is unhelpful when also doing `npm rebuild`

The goal of the node_modules cache between builds is to save time between builds, but I think it's failing at this due to the npm rebuild that's run. This command will cause all of the compiled packages to be recompiled, even if they're no longer necessary.

Try this:

Make a new app, list "bcrypt" as a dependency.
Deploy to Heroku
--> Notice that bcrypt will be built from source and installed during the deploy
Remove "bcrypt" from package.json.
Deploy to Heroku
--> Notice that bcrypt will still be rebuilt from source.

The effect of this is that all future deploys of the application will be slowed down due to the need to rebuild bcrypt, even though it's no longer used. This applies to all packages that are in the dependency tree at some point in time, but is especially bad with the ones that need a slow compilation step.

It would be great to be able to force the cache to be flushed every once in a while.

sdsdsd

dsdsdsdsd

don't install devDependencies

Cannot install package from github

Hello,

I am unable to install a package from github. My package.js, in pertinent part:

...
"dependencies": { "git://github.com/<<repo>>": "" }
...

I receive the following error:

npm ERR! git clone git://github.com/<<repo>>@  <<repo>>@ is not a valid repository name

it appears your script appends an @ symbol to the repo, even when there is no version string. From what I can tell, there is no way to use npm with a git repo and a version string.

It is useful to be able to use git repos while waiting for a project to accept pull requests or to update their npm package. This can easily be made possible by only appending the @ when there is a version string.

Thank you.

Noise in npm log

My app generates the following log. It seems like NPM is puking on one of the modules, but the build seems to finish and succeed.
My app is a weird one, as I have an instance of "bundleDependencies". Fortunetly it's an open project and you can look at the code at: https://github.com/Empeeric/formage

npm http 200 https://registry.npmjs.org/lodash/-/lodash-2.2.0.tgz
npm http 200 https://registry.npmjs.org/async/-/async-0.2.9.tgz
npm ERR! Error: shasum check failed for /app/tmp/npm-397-At8o6uVB/1380567884708-0.24993811477907002/tmp.tgz
npm ERR! Expected: 43c5d83de5568b02d962b9aac6c1d705ffa33887
npm ERR! Actual:   20f26110dfb80c20f4afde2ede759813cca2db0d
npm ERR!     at /tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d/vendor/node/lib/node_modules/npm/node_modules/sha/index.js:38:8
npm ERR!     at ReadStream.<anonymous> (/tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d/vendor/node/lib/node_modules/npm/node_modules/sha/index.js:85:7)
npm ERR!     at ReadStream.EventEmitter.emit (events.js:125:20)
npm ERR!     at _stream_readable.js:896:16
npm ERR!     at process._tickCallback (node.js:317:11)
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Linux 3.8.11-ec2
npm ERR! command "/tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d/vendor/node/bin/node" "/tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d/vendor/node/bin/npm" "install" "--production"
npm ERR! cwd /tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d
npm ERR! node -v v0.11.7
npm ERR! npm -v 1.3.8
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /tmp/build_d83fd97c-7ca5-4ad2-8e92-53d90f64516d/npm-debug.log
npm ERR! not ok code 0
-----> Caching node and node_modules for future builds
-----> Building runtime environment
-----> Discovering process types
       Procfile declares types -> web

-----> Compiled slug size: 5.7MB
-----> Launching... done, v38
       http://formage.herokuapp.com deployed to Heroku

package.json not detected?

Awesome that I can use node.js on heroku now, but somehow it doesn't detect the package.json so I'm wondering what I'm doing wrong?

Test script:

mkdir test-heroku
cd test-heroku
git init
echo '{
    "name": "test-heroku"
   ,"version": "0.0.1"
}' > package.json
git add package.json
git commit -m "Added package.json"
heroku create --stack cedar --buildpack http://github.com/heroku/heroku-buildpack-nodejs.git
git push heroku master

test output:

Initialized empty Git repository in /private/tmp/test-heroku/.git/
[master (root-commit) b9c5cc0] Added package.json
 1 files changed, 4 insertions(+), 0 deletions(-)
 create mode 100644 package.json
Creating blooming-mountain-5427... done, stack is cedar
http://blooming-mountain-####.herokuapp.com/ | [email protected]:blooming-mountain-####.git
Git remote heroku added
Counting objects: 3, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 274 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)

-----> Heroku receiving push
-----> Fetching custom buildpack... done
 !     Heroku push rejected, no Cedar-supported app detected

To [email protected]:blooming-mountain-####.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to '[email protected]:blooming-mountain-####.git'

But if I look at detect the presence of the package.json should be enough?

So what am I doing wrong?

Node.js 0.6.10 Error

After Node.js 0.6.10 was added to manifest.nodejs I can't update my app because I get an error trying to compile Node.js

BUILDPACK_URL  => https://github.com/heroku/heroku-buildpack-nodejs.git#versions

-----> Heroku receiving push
-----> Fetching custom buildpack... done
-----> Node.js app detected
-----> Resolving engine versions
       Using Node.js version: 0.6.10
 !     Heroku push rejected, failed to compile Node.js app

! [remote rejected] master -> master (pre-receive hook declined)

Outdated npm version for node 0.10 (should be 1.2.14 but is 1.2.12)

The latest npm version on Heroku is 1.2.12 and when specifying 1.2.14 I get

Requested engine npm version 1.2.14 does not match available versions

The latest available version seems to be 1.2.12.
However, the bundled npm with node 0.10.0 is 1.2.14 and when trying to push my app using 0.10 I get

npm ERR! cb() never called!
npm ERR! not ok code 0

I am not 100% sure if the two issues are related but everything works fine locally with node 0.10 and npm 1.2.14. When specifying node 0.8.x and npm 1.2.x as my engine requirements everything works as well.

Update npm in newest node

Is there a way to update 0.10.24 tar to include newest version of npm?
Becuase bnoordhuis/node-heapdump#26 is faling with older version of node-gyp

Working with devDependencies in forks employing Grunt

This has more to do with a potential fork of this project for including things like Grunt, etc...

Since Grunt uses modules specified in devDependencies, in forks employing Grunt, it's necessary to run npm install without the --production flag. Something I'm considering is following up and running npm prune --production to remove those devDependencies after the initial build. The intention here would be to reduce the size of the slug. Would this do the trick?

Additionally, it would still be nice to have those devDependencies cached so they aren't pulled on each build. Would the current version of heroku-buildpack-nodejs allow for this kind of thing with minimal modification?

I'm only a few days green on buildpacks and Heroku deployment mechanics yet my projects require this extra functionality, so I'm trying to get a grip--any help is appreciated.

Bypassing the cache?

How would I bypass the cache? I can think of 2 cases where I'd want to:

my app isn't shrinkwrapped and I want to install updated dependencies (especially when pointing at the HEAD branch of a git repo)
the buildpack made an invalid slug (see #46, but probably for other reasons)

push to heroku git repo failing due to Error: SSL Error: SELF_SIGNED_CERT_IN_CHAIN

started seeing this today. was able to resolve the issue locally based on this: http://blog.npmjs.org/

Upgrade Node.

We're 'stuck' at 0.8.14 since it was released around Oct 25th last year. 0.8.19 is latest as of Feb 6th, 2013. How about upgrading us to something more recent?

Encourage (or mandate) apps to commit npm-shrinkwrap.json

Software rot is bad. Deployments should be explicit about dependencies.

Hence bundler for Ruby has both Gemfile (loose for publishing) and Gemfile.lock (explicit for deployment).

The Ruby buildpack for Heroku demands you commit a Gemfile.lock to https://devcenter.heroku.com/articles/bundler

Running bundle install also generates a Gemfile.lock file, which should be added to your git repository. Gemfile.lock ensures that your deployed versions of gems on Heroku match the version installed locally on your development machine.

If you don't commit a Gemfile.lock, your app won't build

Heroku receiving push
Ruby/Rack app detected
Gemfile.lock is required. Please run "bundle install" locally and commit your Gemfile.lock.

The Nodejs build pack should do the same - demand the user commit npm-shrinkwrap.json. Right now it doesn't.

node_modules/ gets wiped and reloaded

Just noticed that with the new buildpack any custom modules that I put in node_modules/, e.g. ones that I installed with npm install $module and then edited / are not listed as dependancies in package.json, were removed. I wasn't sure what the deal was until I ran heroku run ls node_modules/ and didn't see my module there.

The fix was to just move my custom modules to another folder, which I probably should have been doing from the beginning.

Not sure if this should even be an issue, but it might help anyone else who put custom modules in that folder and is wondering what just happened.