While executing testing with Mocha over a Restify server using shelljs module, appears a "possible" memory leak due tu a variable declared as global since it is not declared locally.

Script with the problem: shell.js
Line: 994
"Problematic" code: "for (letter in map)"

Var letter is not declared locally.

I assume this is because wrap() checks for the first character of the first argument being "-"

https://github.com/arturadib/shelljs/blob/b5aa580668a80f7dfa59b91fcabf2048f05a6d0b/shell.js#L1035

cd('~');

Just returns an error :(

cd: no such file or directory: ~

Hello. When I run shelljs in a CLI app I get the following message (even if I just include it):

to: wrong arguments

toEnd: wrong arguments.

So this is when it is inside a file that I just run something like ./bin/sh.

!/usr/bin/env node

Any thoughts on why this would be?

This is probably something really silly. But can someone take a look? This is the repo:

https://github.com/Jonovono/une

And you can see the occurance happening when I include shelljs in: https://github.com/Jonovono/UNE/blob/master/lib/files.js

Even if I have shell.mv commented out it still happens. If I include shelljs somewhere else it is fine.

Hi,
i want to import 20000 Document files in node.js.

after 300 exec calls I get:

shell.js: internal error
Error: spawn EMFILE
at errnoException (child_process.js:980:11)
at ChildProcess.spawn (child_process.js:927:11)
at exports.spawn (child_process.js:715:9)
at Object.exports.execFile (child_process.js:607:15)
at Object.exports.exec (child_process.js:578:18)
at execSync (/var/www/filesearch/node_modules/shelljs/shell.js:1793:9)
at _exec (/var/www/filesearch/node_modules/shelljs/shell.js:1130:12)
at /var/www/filesearch/node_modules/shelljs/shell.js:1487:23
at FileScanner.ingestDB (/var/www/filesearch/crawl/filescanner.js:41:25)
at callbacks (/var/www/filesearch/node_modules/express/lib/router/index.js:161:37)

I tried it asynchon or synchron. It seems fpr me that the file handle in exec is not closed.

ulimit -n xxx doesn't help here ..

You can reproduce this behaviour with following snippet:

    for(i=0;i<1000;i++) {
        var child = exec('node -v', {async:false});
        console.log(i);
    }

TIA
Michael

from #23:

a block device (mknod test/resources/block b 1 1) causes the following error:

Running test: mv.js
shell.js: internal error
Error: UNKNOWN, unknown error
    at Object.fs.readSync (fs.js:381:19)
    at copyFileSync (/Users/james/src/assets/shelljs/shell.js:1119:20)
    at /Users/james/src/assets/shelljs/shell.js:311:5
    at Array.forEach (native)
    at Object._cp (/Users/james/src/assets/shelljs/shell.js:270:11)
    at Object.cp (/Users/james/src/assets/shelljs/shell.js:1074:23)
    at Object.<anonymous> (/Users/james/src/assets/shelljs/test/mv.js:20:7)
    at Module._compile (module.js:449:26)
    at Object.Module._extensions..js (module.js:467:10)
    at Module.load (module.js:356:32)

a character device (mknod test/resources/character c 1 1) and fifo (mkfifo test/resources/fifo) causes the mv test to hang

symlink, file, directory work fine. haven't tested sockets.

It would be nice if shelljs gracefully handled file I/O errors due to permissions issues, as opposed to the current full error dump you get now:

    var shelljs = require('./shelljs');
    shelljs.mkdir('-p', '/var/log/myNewDir');

    $ node shelljsConsumer.js
    shell.js: internal error
    Error: EACCES, permission denied '/var/log/myNewDir'
        at Object.fs.mkdirSync (fs.js:642:18)
        at mkdirSyncRecursive (shell.js:1649:8)
        at shell.js:541:7
        at Array.forEach (native)
        at Object._mkdir (shell.js:526:8)
        at Object.mkdir (shell.js:1491:23)
        at Object.<anonymous> (shelljsConsumer.js:2:10)
        at Module._compile (module.js:456:26)
        at Object.Module._extensions..js (module.js:474:10)
        at Module.load (module.js:356:32)

It seems like, for a lot of use cases (like this one), a more user-friendly error handling path could be initiated.

Making a build script for a project. using shelljs/make in coffeescript.
when I try to call this function:
cp '-fR', build_dir+'/*', dest_dir

node kicks back this ugly chunk:

shell.js: internal error
Error: EEXIST, file already exists '/[dest_dir]/node_modules/.bin/express'
  at Object.fs.symlinkSync (fs.js:730:18)
  at cpdirSyncRecursive (/[redac]/node_modules/shelljs/shell.js:1575:10)
  at cpdirSyncRecursive (/redac]/node_modules/shelljs/shell.js:1572:7)
  at /redac]/node_modules/shelljs/shell.js:325:9
  at Array.forEach (native)
  at _cp (/redac]/node_modules/shelljs/shell.js:302:11)
  at /redac]/node_modules/shelljs/shell.js:1491:23
  at Function.target.install (/redac]/grind:65:3, <js>:63:5)
  at Object.target.(anonymous function) [as install] (/redac]/node_modules/shelljs/make.js:28:26)
  at Function.target.fast (/redac]/grind:25:3, <js>:19:12)
  at Object.target.(anonymous function) [as fast] (/redac]/node_modules/shelljs/make.js:28:26)
  at /redac]/node_modules/shelljs/make.js:38:20
  at Array.forEach (native)
  at [object Object]._onTimeout (/redac]/node_modules/shelljs/make.js:36:10)
  at Timer.listOnTimeout [as ontimeout] (timers.js:110:15)

The symlink in question from the source directory is:
../express/bin/express
Also, I'm overwriting whatever is in the destination in this instance. That file and the express file exists, I'm sure.

On the other hand, exec "cp -rf #{build_dir}/* #{dest_dir}" works perfectly as intended.

I'll take a crack at looking at the source later and seeing what it might be, just wanted to log it while I'm still thinking about it.

When executing the following command directly on the cmd line the database is dumped to the file correctly.

When running the same command via .exec. using Grunt.js (see below) the output is in the console and the file is created but has no contents.

shell.exec('mysqldump -u my_user -pmy_pass test > test.sql');

My ultimate goal is to run a dump via ssh like so:

shell.exec("ssh user@host \ mysqldump -u my_user -pmy_pass test > test.sql");

This ShellJS script will stall indefinitely, if 'nosuchfile.txt' doesn't exist.

require('shelljs/global')

exec("cat < nosuchfile.txt")

Will cause shelljs to stall

This fixes it:

  cmd += ' > '+stdoutFile+' 2>&1'; // works on both win/unix

  var script =
   "var child = require('child_process'), path=require('path'), \
        fs = require('fs'); \
    child.exec('"+escape(cmd)+"', {env: process.env}, function(err,stdout,stderr) { \
      fs.writeFileSync('"+escape(codeFile)+"', err ? err.code.toString() : '0'); \
      if(!path.existsSync('"+escape(stdoutFile)+"')){ fs.writeFileSync('"+escape(stdoutFile)+"',stdout.toString()+stderr.toString()); }; \
    }); \
    process.on('uncaughtException',function(err){ \
      if(!path.existsSync('"+escape(stdoutFile)+"')){ fs.writeFileSync('"+escape(stdoutFile)+"',err.toString()); } \
      if(!path.existsSync('"+escape(codeFile)+"')){ fs.writeFileSync('"+escape(codeFile)+"','-1'); } \
    }); \
    process.on('exit',function(){ \
      if(!path.existsSync('"+escape(stdoutFile)+"')){ fs.writeFileSync('"+escape(stdoutFile)+"','Uncaught ShellJS Error'); } \
      if(!path.existsSync('"+escape(codeFile)+"')){ fs.writeFileSync('"+escape(codeFile)+"','-1'); } \
    });"


  if (fs.existsSync(scriptFile)) _unlinkSync(scriptFile);
  if (fs.existsSync(stdoutFile)) _unlinkSync(stdoutFile);
  if (fs.existsSync(codeFile)) _unlinkSync(codeFile);

We have forked your repo, here: https://github.com/croteb/shelljs and will be keeping it current with our fixes.

Thanks!

I don't quite understand how to exit a child process using exit();

https://github.com/arturadib/shelljs#exitcode

Could you add an example to the README.md please.

Hi,

First thanks for writing ShellJS, so we've noticed in our project that somtimes shellJS consumes 100% cpu resources when something goes wrong with exec.

i.e.

exec("mysql -u testUser testDB < test.sql")

will sometimes hang, leaving us with 100% cpu utilization on the shellJS parent process.

Hi all,
is it possible to execute an ssh command with shelljs? I get the following error
Pseudo-terminal will not be allocated because stdin is not a terminal.

If it is possible to execute ssh command with shelljs, could someone point me to a link or example.
Thanks
José

protzenk@sauternes:pdfjs ((a68e11c...)) $ nodejs
> require("shelljs/global")
{}
> exec("ls")

Then, the nodejs process gets stuck and the only way I can get out of this is to run pkill nodejs in another terminal. Any ideas what might be causing this? I've installed nodejs as a Debian package, and I've also run "npm install shelljs" hoping that it would upgrade me to the latest version.

Thanks,

jonathan

I have a nodejs script that runs a lot of shell commands. In the last few days, script strarted throwing strange errors, but it continued to work.
This was the error on CentOS:
exec: Exec returned error code1

and this on Ubuntu:
exec: Exec returned error code127

When I changed the script dependencies to sue shelljs v0.2.2 the errors stopped appearing.
Note that the errors were only shown when the script was installed with npm, when it was called directly from the local repository there were no errors.

Some like sed('-n','/FOO/,/BAR/p',cat("cats.txt"));
says sed: option not recognized: n

following code snipper works using 'node' command, but not works using 'sjs' command

process.stdin.on('data', function (chunk) {
  process.stdout.write('data: ' + chunk);
}).resume();

shouldn't be hard to implement, would be awesome if someone implemented it :)

I was wondering what is the best way to do something like

export PATH=/usr/bin:$PATH

with shelljs. I guess you can use env, but I tried to change PS1 on linux and nothing happens.
I am particulary interested in changing PS1 somehow from within node.

Is there a way detect if shelljs or node is installed in the system, within the script. I know it's kina dumb but, say in bash script it's possible to get the the $SHELL

Asking this because, say one decides to write a setup script with shelljs, then node becomes a dependency to get shelljs first....so bit confused... env['NODE_PATH'] would be useful? But also, can we know if shelljs is installed or not and then install it within shelljs 0_o

I'd like to at least see the file type (file, directory, symlink, etc) and size.

I'd like to be able to do things like

compile "source" if not mtime "source" < mtime "out"

so either mtime, atime, ctime functions or else support the "-ot" and "-nt" tests or else a stat function that basically wraps fs.stat.

Or all of the above 😁

Sometimes, when deleting a directory that has sub-directories and files it throw an error witch code: ENOTEMPTY.

When executing this: rm('-rf', '/path/to/build/zip') the following error is thrown:

rm: could not remove directory (code ENOTEMPTY): build/zip/vendors/bootstrap

Note: It can happen with any sub-directory, is not related to a particular directory only. It doesn't happens with files.

• https://github.com/isaacs/node-glob It's 100% js now, and has sync and async APIs for finding files. Provides fancy-pants globs like Bash4 with extglob and globstar turned on (though these can be disabled).
• https://github.com/isaacs/node-touch Touches a file
• https://github.com/substack/node-mkdirp Battle tested efficient mkdir -p
• https://github.com/isaacs/node-which like which(1)
• https://github.com/isaacs/fstream Useful for piping between directories for cp -R style behavior or tar streams (no sync api, though, only streaming.)
• https://github.com/isaacs/node-tar For a tar command (no sync API, only streaming)
• https://github.com/isaacs/rimraf Battle-tested rm -rf
• https://github.com/isaacs/read like read(1)

Moreover things seem to behave differently on Windows vs. Unix.

This seems to be partly because of different behaviors of fs.unlink, partly because of our implementation.

I recently integrated shelljs into one of my projects and mocha started complaining about a global leak with respect to a key property being set on the global object.

I can reproduce the effect in the REPL, i.e. independently of the project in question:

$ node
> var shell = require('shelljs');
undefined
> global.key
undefined
> shell.exec('ls -lad *');
{ code: 0,
  output: ... }
> global.key
'async'

If this is an expected behavior, it might be good to document it in shelljs's README, as it took me a little while to figure out that I hadn't caused the "leak" with a statement in my own code. I imagine it may trip up other folks as well.

Is it a bug perhaps?

By the way, many thanks for creating shelljs. I ran into a situation recently where I just had to have a way to make a synchronous http request. Using shelljs to synchronously invoke another node script which makes the request asynchronously ended up being a far simpler solution than introducing fibers or hand-rolling a C addon.

With unix cp you can do

cp -Rf oneDir/ twoDir/

which copies all the files in oneDir into twoDir. When I do:

cp('-Rf', 'oneDir/', '/twoDir/');

nothing gets copied, I have to do

cp('-Rf', 'oneDir/*', '/twoDir/');

from man cp

If source_file designates a directory, cp copies the directory and the entire subtree connected
at that point. If the source_file ends in a /, the contents of the directory are copied rather
than the directory itself.

An example shell script I'm using: lsof -i :80 | grep node | awk '{ printf $2 }'

What this should print out is the PID of the node process running on port 80; instead it returns awk: line 1: no arguments in call to printf. I can recreate this error by deleting the $2 and running it on my terminal. I suspect that shelljs or node is incorrectly escaping or interpolating the $2 variable; I haven't figured out the magical number of slashes to escape it properly.

Hi,
I am trying to use the exec function in an asynchronous code as follows

shell = require 'shelljs'
shell.exec('ssh -l username host \"cd Desktop; touch example.txt\"', (code, output) ->
console.log code
)

Although the command is executed successfully, it doesn't print the code. So I have no way in my code to know what happened to the ssh command. Is there a different way of calling exec asynchronously?
Note that the synchronous case works perfectly
Thanks
José

cp command behaves differently from their shell parent.
Suppose, we have the following structure:

• dir1/
  • lib/
    • main.js

Or in shelljs code:

require('shelljs/global')
mkdir('dir1')
mkdir('dir1/lib')
echo('123').to('dir1/lib/main.js');

All these commands fail on my system:

cp('-r', 'dir1/*', 'dir2/')
shell.js: internal error
Error: ENOENT, no such file or directory 'dir2//lib'
at Object.fs.mkdirSync (fs.js:483:18)
at code/node_modules/shelljs/shell.js:306:14
at Array.forEach (native)
at _cp (code/node_modules/shelljs/shell.js:288:11)
at code/node_modules/shelljs/shell.js:1103:23
at repl:1:2
at REPLServer.self.eval (repl.js:111:21)
at Interface. (repl.js:250:12)
at Interface.EventEmitter.emit (events.js:93:17)
at Interface._onLine (readline.js:199:10)

cp('-r', 'dir1', 'dir2');
shell.js: internal error
Error: ENOENT, no such file or directory 'dir2/dir1'
at Object.fs.mkdirSync (fs.js:483:18)
at code/node_modules/shelljs/shell.js:306:14
at Array.forEach (native)
at _cp (code/node_modules/shelljs/shell.js:288:11)
at code/node_modules/shelljs/shell.js:1103:23
at repl:1:1
at REPLServer.self.eval (repl.js:111:21)
at rli.on.self.bufferedCmd (repl.js:260:20)
at REPLServer.self.eval (repl.js:118:5)
at Interface. (repl.js:250:12)

cp('-r', 'dir1/', 'dir2/')
shell.js: internal error
Error: ENOENT, no such file or directory 'dir2//lib'
at Object.fs.mkdirSync (fs.js:483:18)
at code/node_modules/shelljs/shell.js:306:14
at Array.forEach (native)
at _cp (code/node_modules/shelljs/shell.js:288:11)
at code/node_modules/shelljs/shell.js:1103:23
at repl:1:2
at REPLServer.self.eval (repl.js:111:21)
at Interface. (repl.js:250:12)
at Interface.EventEmitter.emit (events.js:93:17)
at Interface._onLine (readline.js:199:10)

cp('-R', 'dir1/', 'dir2');
shell.js: internal error
Error: ENOENT, no such file or directory 'dir2/lib'
at Object.fs.mkdirSync (fs.js:483:18)
at code/node_modules/shelljs/shell.js:306:14
at Array.forEach (native)
at _cp (code/node_modules/shelljs/shell.js:288:11)
at code/node_modules/shelljs/shell.js:1103:23
at repl:1:1
at REPLServer.self.eval (repl.js:111:21)
at rli.on.self.bufferedCmd (repl.js:260:20)
at REPLServer.self.eval (repl.js:118:5)
at Interface. (repl.js:250:12)

but all this calls successfully run if I've created the dir2 directory previously.

cp from bash always creates target directory (only one level) if necessary. So all these calls finish successfully.

What about other commands? Suppose I want to do cat "file.txt" | less, how do I do that? Or what about pacman -S awesome-package (Arch's package manager)?

From the looks of it, I don't think I can, can I?

Make has a debug flag:

-n, --just-print, --dry-run, --recon
  Print the commands that would be executed, but do not execute them.

It would be great to have this flag present to help debug a non-functional shelljs script. Maybe just have it print the command to the console rather than execute the command? or do both?

Any way to detect a binary file?

there have been way too many leaks recently...

Hi,

Shelljs has been awesome so far, except for exec. I know how it works, with the write-to-a-temporary file loop, and I know that its a hack necessary due to the lack of child_process.execSync.

Unfortunately, exec has been very buggy for me, randomly not working on various developer and server machines. Usually it gets stuck in an endless loop. I never seem to be able to satisfyingly reproduce the issue or find the cause.

So I decided to give that up and write shelljs-ffi based on execsync-ng which in turn is based on node-ffi. execsync-ng is a fork of execSync that also works on windows.

You can check out these modules at https://github.com/doxout/shelljs and https://github.com/doxout/execSync or try them out: npm install shelljs-ffi execsync-ng

I know that a native module dependency such as node-ffi is probably unacceptable for shelljs. However I decided to ask anyway... Is it possible that this fork may be accepted into shelljs?

This is something that happens only on Red Hat as far as I know.
I'm running a npm module as a different user thant the one currently logged in, for example:
sudo -u username client.js

This works normally on Ubuntu and Debian, but on Red Hat it gets stuck indefinitely on this line:
response = shell.exec('whoami', { silent: true });

(it doesn't work for any command, not just whoami)

Everything works okay when the script is kicked off normaly (without sudo).

It appears cp is skipping dot files.

var sh = require('shelljs')
sh.cp('-R', 'foo/*', 'bar')

where foo contains an .htaccess file among whatever else, bar fails to receive the file.

Tested on Windows 7, NodeJS v0.10.17

in jshint/jshint#687, there's an interest in being able to use the jshint cli interface by passing -- as a filename and having the input from stdin used as the contents of the file. since jshint uses shelljs i was wondering if it would be possible to add this support to shelljs - it's probably useful to lots of other people who use shelljs too.

i'm not very familiar with shelljs but if someone could give me some guidance on how to approach this, i might take a stab at it.

I'm sure part of this should be fixed in node core, but they're just sitting on the issue (nodejs/node-v0.x-archive#3728).

PATH uses : on *nix, and ; on Windows. It could either provide just the separator, or provide some sort of push method.

Example:

shell.PATH.push('../node_modules/.bin');
// or
shell.env.PATH += shell.sep + '../node_modules/.bin';

I see that the code allows for you to do exec(cmd, callback), and it will create an empty options object. However, this will mean async is false, so the callback is not used.

It seems like if you passed a callback, you expect it to be async?

See http://stackoverflow.com/a/14231570/201852

fs#renameSync won't work across partitions or attached storage. This is by design, as rename is paired to the C version of the same method, which isn't able to cross disks either. (See this closed issue for more discussion about rename.)

A solution could be piping a readStream to a writeStream, and then cleaning up the source file.
I'd attach a fix but the trouble is that listening for the end event on the pipe and firing a callback would break the synchronous nature of shelljs.

Ideas?

Great module, by the way. Very helpful!

seems like exec works in cmd and powershell in windows but stalls when running in mingw bash in windows.

Would be nice if there was a way to silent the ouput, for use in systems like grunt that have their own output. For example:

var hash = shell.exec('git rev-parse --short HEAD').output;

Now I have access to the git hash (yay), but that hash has been displayed in the console (boo).

run $ npm install [-g] shelljs
npm http GET https://registry.npmjs.org/%5B-g%5D
npm http GET https://registry.npmjs.org/shelljs
npm http 404 https://registry.npmjs.org/%5B-g%5D
npm ERR! 404 '%5B-g%5D' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, or http url, or git url.

It's on Mac OS. Any help? Thanks.

ls and find return objects (e.g. { 'file1':null, 'dir1/file2':null, ...}), supposedly for convenience. Sure, it's nice when you're doing a for in, but I'm not sure why you would. This is V8 we're talking about, why not use arrays since we have access to the full set of ES5 iteration and accessor methods? (e.g. forEach, filter, map, reduce, indexOf)

I'd be happy to submit a pull request if there is a desire for one :)

osx / chrome, the link points to the right place and opening it in a new tab works

i know it's not (at all) mimicking which(1), but imo, it'd be cool if shelljs.which() would check for binaries in module.paths.

thoughts?

i'd be happy to add this behavior if you like the idea.

A command like shell.exec('./node_modules/.bin/mocha' + MOCHA_OPTS); works on Linux and OSX, but on windows it complains that . isn't a command.