gulpjs / async-done Goto Github PK

Allow modify/extend options that have been passed to eos?

To bring inline with gulp repos. Use eslint-config-gulp and update code style

Add support to handle "end" of generators? e.g.

var i = 10
function * success() {
  yield [i++]
  return i
}

function * failure() {
  yield [i++]
  throw new Error('boom')
  return i
}

asyncDone(success, function(err, res) {
  expect(res).to.equal(11);
});
asyncDone(failure, function(err, res) {
  expect(err).to.be.instanceof(Error);
});

Running into some weird race condition stuff when an error is omitted from within one of our pumpify pipeline - Ref https://github.com/cristianl/testcase-vinyl-bug from gulpjs/gulp#2086 (comment) and mafintosh/end-of-stream#17

Need to write a test to capture errors higher up the chain.

Use jscs-preset-gulp

This ...

import { ChildProcess } from "child_process";
import { EventEmitter } from "events";
import { Stream } from "stream";

... errors in the log when I'm building my typescript project as these modules are not defined, as I've had no need for the types of node. When I manually added @types/node to my project, I could build without the index.d.ts of async-done writing errors in the log.

Would it be acceptable to have async-done wait on a Stream or a ChildProcess that is the result of a resolved promise?

Use case is like this:

async function myTask() {
    await someSetup();
    return gulp.src(/*...*/).pipe(/*...*/);
}

Happy to submit a PR if this is a good idea.

So, I've got impression that we need to do the same fix as in the original orchestrator as discussed here: robrich/orchestrator#48 (comment)

Here is a simple reproduce scenario:

var Readable = require('stream').Readable;

var through = require('through2');
var asyncDone = require('async-done');

var rs = new Readable({objectMode: true});
var c = 0;
rs._read = function () {
  if (c++ < 100) {
    rs.push(c);
  } else {
    rs.push(null);
  }
};

asyncDone(function(){

  return rs.pipe(through.obj(function (chunk, enc, cb) {
    console.log(chunk);
    this.push(chunk);
    cb();
  }));

}, function(err, result){
  console.log(err, result);
});

In this scenario the node process will terminate after reading 16 items.

Now that we've got stream-consume as a separate module a fix would be trivial, happy to send a PR if this makes sense.

I want to switch to lab for testing because it has a good built in code coverage option. I can't seem to get tap's to work.

Fixed in current orchestrator, possibly just use the same configuration.

As I understand it, this is some sort of converter between async model conventions, but I'm unclear when I'd use it. I suggest a small scenario section at the top of the README might help clarify.

I stumbled on this while fixing #42 - It seems that any thrown errors in the callback argument are swallowed by a promise returned from fn. I believe this just needs to be nextTicked but I'm looking into it further.

I would like to use this manually and chain multiple calls.

Now you've done it, got me interested enough to clone, build, and test. 👍

Running the command npm test gives me 7 minor issues, and exits with failure. They are all basically like this:

Invalid line break at test/streams.js :
     1 |'use strict';
---------------------^
     2 |
     3 |var expect = require('expect');


7 code style errors found.

I don't know your lint style, is it objecting to the semicolon, or something about the line break?

I'm on a Windows 10 machine if it matters, running node v6.10.3 and npm v5.0.3.

Per this line:

https://github.com/gulpjs/async-done/blob/v1.3.2/index.js#L3

No idea how to trace this with nothing in package.json, no?

Node.js: v7.4.0
async-done: 1.2.2
gulp: gulpjs/gulp#4.0
cp-file: 4.1.1

cc @sindresorhus

Event emitters aren't (always) streams. Eliding them (the documentation says "Stream or EventEmitter returned") causes surprising behaviour if the return type is a) a promise, b) an event emitter, and c) not a stream.

The cp-file module's copy function returns an object which is both a promise and an event emitter. The latter allows it to be monitored for progress updates when copying large files:

copy(src, dest)
    .on('progress', ...)
    .then(...)

async-done appears to treat it as a stream because of the on method, rather than a promise i.e this doesn't work:

gulpfile.js

import copy from 'cp-file'
import gulp from 'gulp'

gulp.task('copy', () => copy('foo.txt', 'bar.txt'))

output

$ gulp copy
Using gulpfile ~/temp/copy-file-test/gulpfile.js
Starting 'copy'...
The following tasks did not complete: copy
Did you forget to signal async completion?

- but this works:

gulpfile.js

import copy from 'cp-file'
import gulp from 'gulp'

gulp.task('copy', () => {
    let promise = copy('foo.txt', 'bar.txt')
    delete promise.on
    return promise
})

output

$ gulp copy
Using gulpfile ~/temp/copy-file-test/gulpfile.js
Starting 'copy'...
Finished 'copy' after 12 ms

An obvious solution would be to check if the return value is a promise before checking if it's an event emitter. Better still, why not use is-stream to check if it's actually a stream? (I'm not sure what the priority should be if the value is both a promise and a (real) stream.)

If the event-emitter-as-terminable (AKA endable/closeable/finishable &c.) interpretation is really needed, it seems like it should be the last resort, when all else fails. But is it actually needed? I can think of one example of an event emitter which is terminable, but which isn't a stream — i.e. a child process — but that appears to be handled separately. Are there any other examples?

(Apologies if this isn't the right place to report this. The layers of abstraction underpinning gulp are impressive and impeccable until you need to figure out where a problem lies, at which point they become a twisty and seemingly-interminable 😄 maze: cp-file -> gulp (v4) -> bach -> undertaker -> async-done -> ???)

(Moving details over from gulpjs/gulp#1642 (comment))

This is due to this code:

function onError(error) {
  return done(error);
}

If error is undefined, then this is equivalent to called done(undefined), which will be treated as a success, when it should really be a failure.

You can reproduce this via:

// Inside gulpfile.js:
const gulp = require('gulp');

gulp.task('reject-without-value', () => Promise.reject());
gulp.task('reject-with-value', () => Promise.reject('rejection reason'));

$ gulp -v
[14:52:22] CLI version 1.3.0
[14:52:22] Local version 4.0.0-alpha.2

$ gulp reject-without-value
[14:52:30] Using gulpfile /private/tmp/b/gulpfile.js
[14:52:30] Starting 'reject-without-value'...
[14:52:30] Finished 'reject-without-value' after 2.16 ms
$ echo $?
0

$ gulp reject-with-value
[14:52:37] Using gulpfile /private/tmp/b/gulpfile.js
[14:52:37] Starting 'reject-with-value'...
[14:52:37] 'reject-with-value' errored after 1.85 ms
[14:52:37] Error: rejection reason
    at formatError (/Users/jeffy/homebrew/lib/node_modules/gulp-cli/lib/versioned/^4.0.0/formatError.js:20:10)
    at Gulp.<anonymous> (/Users/jeffy/homebrew/lib/node_modules/gulp-cli/lib/versioned/^4.0.0/log/events.js:30:15)
    at emitOne (events.js:120:20)
    at Gulp.emit (events.js:210:7)
    at Object.error (/private/tmp/b/node_modules/undertaker/lib/helpers/createExtensions.js:61:10)
    at handler (/private/tmp/b/node_modules/now-and-later/lib/map.js:46:14)
    at f (/private/tmp/b/node_modules/once/once.js:25:25)
    at f (/private/tmp/b/node_modules/once/once.js:25:25)
    at done (/private/tmp/b/node_modules/async-done/index.js:24:15)
    at onError (/private/tmp/b/node_modules/async-done/index.js:32:12)
$ echo $?
1

I want all the gulp4 repos to have the same structure in package.json. It also needs things like engine that specifies we only work in node 0.10

In node 10, I believe they added stream.finished as the core implementation of the end-of-stream module. We should look into using that instead of relying on another dependency.

I don't think this is correct.

example

asyncDone(function () {
  return cp.spawn('not_existing_command', ['hello'])
}, function (err, res) {
  console.log(err, res)
  // err always `null`
})

it always pass the result of spawn to res, no matter it executes echo hello or not exising command.

Because this onSuccess in index.js#L43

See: https://gist.github.com/mattpodwysocki/ff72b3f66d871501e576

Should be domain.once('error')

(I'm assuming that the intent of async-done, when given a stream, is to detect when the stream has finished emitting data. If that's not true, this bug report is invalid.)

I've been trying to diagnose a curious problem with a combination of merged streams (using the event-stream module) and have come across a case where async-done doesn't detect that a stream has finished emitting data.

If the stream reports itself as writable, the end-of-stream module, by default, waits for both the readable side to complete (via an 'end' or 'close' event) and for the writable side to complete (via a 'finish' event). The read/write streams returned by event-stream#merge don't ever complete their write side, so async-done never detects that the stream has finished (emitting).

Changing the EOS configuration in async-done to include 'writable': false tells end-of-stream that we don't care whether the stream is still writable, i.e.

var eosConfig = {
  error: false,
  writable: false
};

Could you envisage a patch release? The fn.length === 0 check make things difficult for my bach experiments...

The argument types and expected returns could be expressed in a typescript declarations .d.ts file, which would enable editors like VSCode to provide some help with using this function.

The big restriction, re not supporting synchronous functions, might be hard to define in typescript. Though I write in TypeScript, the API to this is complex enough that writing the definitions file for this project may be over my head. But the expressive goodness of this should bubble up to packages like Gulp where mistakes over sync/async are more common.

I want to cleanup the jshint and add a linting task in the project. Probably not a good idea to depend on gulp to lint.

Originally reported at bluebird: petkaantonov/bluebird#266

domain = require 'domain'
{delay} = require 'bluebird'

describe 'run', ->
  it 'runs just fine and succeeds', (done) ->
    d = domain.create()
    d.once 'error', (event) -> console.log event # if you comment out this line you won't see the error
    domainBoundFn = d.bind ->
      done()
    domainBoundFn()

  it 'should throw an error but times out instead', (done) ->
    delay 500
      .done ->
        throw new Error "bla"

I am not sure every dev would get the satisfaction of finding out why this sample, if you ignore the error (which I did coming from async-done), times out.

As of today the async function can mark its completion by calling a callback, returning a stream or a promise etc. How about adding instances of ChildProcess (http://nodejs.org/api/child_process.html#child_process_class_childprocess) to the supported list.

Having built-in ChildProcess support would allow us to have functions that spawns processes as Gulp4 tasks. The async-done could work with ChildProcess instance as follows:

• listen to the 'exit' event and mark the task as success / error based on the exit code
• use domains as for other event emitters to handle errors

Not sure if the exact technical solution is easy / feasible but it would be kind of cool to be able to spawn a process and have a Gulp4 task dependent on the completion of this process.

WDYT?

In package.json, readme, etc

Would you be interested in supporting sync tasks, by adding onSuccess call to the end of asyncRunner? Possibly optionally (asyncDone.sync -> console.log "finished")? I'd argue for it because it cannot be done by using async-done.

Since we want to support streamx as the underlying gulp streams, we should add it to our tests.

Problem is that index.js#L26 limit us. Currently that limit us to pass only one value to the callback e.g.

asyncDone(function (done) {
  done(null, 123);
}, function(err, res) {
  console.log(res)
  //=> 123
})

but you cant

asyncDone(function (done) {
  done(null, 123, 456);
}, function(err, res) {
  console.log(res)
  //=> 123
})

One way to fix it is to be 1:1 relation

asyncDone(function (done) {
  done(null, 123, 456);
}, function(err, foo, bar) {
  console.log(foo) //=> 123
  console.log(bar) //=> 456
})

to do it we should change done fn to this

function done(){
  d.removeListener('error', onError);
  d.exit();

  return cb.apply(null, arguments);
}

Or other way is like .then do it - single array with values.

asyncDone(function (done) {
  done(null, 123, 456);
}, function(err, results) {
  console.log(results)
  //=> [123, 456]
})

Which includes an important bug fix calvinmetcalf/process-nextick-args@108596d