bjouhier / galaxy Goto Github PK

I remember that you added Interrupt functionality, so one could just stop other futures.

I have played around it back then and as far as I remember it worked back then, can you please take a look at https://github.com/txdv/galaxy-timeout/blob/master/main.js and maybe suggest an idea on why the letter '3' gets printed out?

In a function (error, value) { } callback style usually the error is a string describing the error. This is ok in good old javascript because we needed to specify all callbacks anyway separately and the exception context was well established.

But with generators/yield which try to mimic await/async this is not good enough anymore. I am using phantomjs and a lot of their methods return just a string 'fail'. So I tried to create an object constructor in order to create a class model for exceptions which hold additional information (like the function name of the failing callback).

But all I get in the catch (ex) { } block is an empty object {}

var galaxy = require('galaxy');

function Exception(value) {
    this.value = value;
}

function *test() {
    throw new Exception(42);
}

function *test2() {
    throw 'exception';
}

galaxy.main(function *() {
    try {
        yield test();
    } catch (ex) {
        console.log(ex);
    }

    try {
        throw new Exception(42);
    } catch (ex) {
        console.log(ex);
    }

    try {
        yield test2();
    } catch (ex) {
        console.log(ex);
    }
});

Output:

{}
{ value: 42 }
exception

Is this intended behaviour?

Some libraries (express, moccha) rely on the arity of async functions. galaxy.star and galaxy.unstar should preserve the arity when creating wrappers.

C# has a nice functionality where you can wait for one task in an enumerator to be finished.

I am new to galaxy and I am having trouble to figure how a decent implementation. Maybe you guys can come up with something?

See Sage/streamlinejs#231

galaxy.spin returns function with arity of 1 instead of 0.

This causes problem with the following code:

function* foo() { return "hello"; }
var future = galaxy.spin(foo());

galaxy.unstar(future)(function(e, r) {
    if (e) throw e;
    console.log("r=" + r);
});

because galaxy.unstar consider that future has already 1 parameter. So it puts the callback in second position.

This could be fixed by changing the example to galaxy.unstar(future, 0) { ... } but this should not be necessary.

The default callback signature is (exception, result), however there are many libraries which use (exception, result1, result2, ...).

Is there a method in the library to wrap these faster? Writing a wrapper for every method in my codebase gets really annoying and is time consuming(because the same mistakes are done all the time).

Something like galaxy.star(func, { results: [ "result1", "result2" ] }) would be nice, it would just wrap the callback results into one object and return a { result1: ..., result2: ... }.

Here is an example of what I mean:

var request = require('request');

var request2 = function (url, callback) {
    request(url, function (error, header, body) {
        if (error) {
            callback(error);
        } else {
            callback(null, { header: header, body: body });
        }
    });
};

var requestAsync = galaxy.star(request2);

So far I have encountered this stuff in mysql, phantomjs-node and request.

And the biggest problem is that if it has a variable amount of arguments, you have to do all that argument passing and stuff.

Hi, I am trying to write something that looks similar to threaded coding.

A producer loop produces values while another loop just deals with the produced values.

For some reason though it just hangs, I guess because I am calling a callback from one main loop in the other.

function GalaxyQueue() {
    this.objects = [];
    this.callbacks = [];
}

GalaxyQueue.prototype.enqueue = function (object) {
    if (this.callbacks.length > 0) {
        var callback = this.callbacks.shift();
        callback(null, object);
    } else {
        this.objects.push(object);
    }
}

GalaxyQueue.prototype.dequeue = function (callback) {
    if (this.objects.length > 0) {
        var object = this.objects.shift();
        callback(null, object);
    } else {
        this.callbacks.push(callback);
    }
}

GalaxyQueue.prototype.dequeueAsync = galaxy.star(GalaxyQueue.prototype.dequeue);

var queue = new GalaxyQueue();

galaxy.main(function *() {
    while (true) {
        var object = yield queue.dequeueAsync();
        console.log(object);
    };
});

galaxy.main(function *() {
    for (var i = 0; i < 50; i++) {
        queue.enqueue(i);
        yield timeout(20);
    }
});

Any idea on how to make this code work?

I understand that it's convenient but I think quite some people would consider it a bad practice to change the prototype of built-in types. It's modifying global state which would prevent me from using galaxy in any library: I wouldn't want to mess with global scope of the app using my library. I think the work-around is require('galaxy/lib/galaxy') but I'm not sure if anything in there relies on the Array prototype being patched. Just having require('galaxy/core') available and documented would be enough.

When I want to execute three functions in parallel, AFAIK I have to to use forEachStar (which I don't fully understand yet due to lack of documentation) or write something like this:

  // just a silly placeholder for some asynchroneous generator:
  var delay = galaxy.star(function (delay, callback) {
    setTimeout(function() {
      callback(null);
    }, delay);
  });

  // start three operations in parallel (think of three different database updates)
  var par1 = galaxy.spin(delay(500));
  var par2 = galaxy.spin(delay(500));
  var par3 = galaxy.spin(delay(500));

  yield par1();
  yield par2();
  yield par3();

Instead, couldn't you allow to pass an array to yield with all generators that must complete in parallel?

  yield [ delay(500), delay(500), delay(500) ];

That would be much shorter.
I'd also expect that yield would return immediately should one of the operations throw an error.

Also, if the results of these generators are required, give them back as an array:

  var results = yield [ delay(500), delay(500), delay(500) ];
   // or, ES6 style:
  [ some, foo, bar ] = yield [ delay(500), delay(500), delay(500) ];

What do you think?

PS: Compliments, great work! This is the solution I like the most (to simulate synchroneous programming).

All is great, but any suggestions how to implement this use case with galaxy/galaxy-streams:
Want to recursively readdir and get file contents with fs.createReadStream, pass to readable object mode stream, provide api on top for middlewares/plugins/pipes, then write with createWriteStream.

Reading directories and files - okey, but how to yield from createReadStream? Or some galaxy-ish way?

I read the tutorial and readme, but can't figure out how to do it..

var galaxy = require('galaxy');
var fs = galaxy.star(require('fs'));

function* doDir(dir, options) {
  yield (yield fs.readdir(dir)).forEachStar(10, function*(file) {
    var fullpath = dir + '/' + file, stat;
    if (options.stats) {
      stat = yield fs.stat(fullpath);
      if (stat.isFile()) {
        // streaming read `filepath`
        // piping, middlewares and etc
      } else if (options.recursive && stat.isDirectory()) {
        yield doDir(fullpath);
      }
    } else {
      // yield some like {file: filepath}
      // cuz readable/transform stream might be with objectMode.
    }
  });
}

galaxy.main(function * () {
  var future = galaxy.spin(doDir(__dirname+'/src'));
  // maybe yield future(), when want to start 
});

I'm new in galaxy approach. :)

I start immediately with the code and output and then will try to explain the code and what I think is happening with the output.

 var galaxy = require('galaxy');

function simpleTimeout(delay, callback) {
    setTimeout(function () {
        if (callback) {
            callback(null);
        }
    }, delay);
};

var simpleTimeoutAsync = galaxy.star(simpleTimeout);

var i = 0;
function timeout_(func, delay, callback) {
    var cb = callback;
    var j = i++;
    if (delay < 0) {
        // we are already behind, so just return a Timeout.Exception
        callback('timeout');
        return;
    }

    var timer = setTimeout(function () {
        console.log();
        console.log('timeout_(' + j + '): timeout');
        console.log(cb);
        timer = null;
        if (cb) {
            console.log('calling');
            cb('timeout');
            cb = null;
        }
        console.log();
    }, delay);

    func(function (error, value) {
        console.log();
        console.log('timeout_(' + j + '): callback');
        console.log(cb);
        if (timer) clearTimeout(timer);
        if (cb) {
            console.log('calling');
            cb(null, value);
            cb = null;
        }
        console.log();
    });
};

function timeout(future, delay, callback) {
    return timeout_(galaxy.unstar(future, 0), delay, callback);
}

var timeoutAsync = galaxy.star(timeout);

galaxy.main(function *() {
    try {
        yield timeoutAsync(function *() {
            console.log('1');
            yield simpleTimeoutAsync(200);
            console.log('2');
            yield simpleTimeoutAsync(250);
            console.log('3');
        }, 400);
    } catch (ex) {
        console.log('exception: ' + ex);
    }
});

1
2

timeout_(0): timeout
[Function]
calling
exception: timeout

3

timeout_(0): callback
null

simpleTimeoutAsync just returns an event in a specified time.

timeoutAsync tries to execute a task and has a timer itself to call the callback with an error timeout after some time. I figured that calling the callback with an exception would stop the generator from stepping further, but once the second simpleTimeoutAsync finishes, it just goes on.

Is this intended behaviour or a bug? Is there a way to implement my desired functionality another way?

[Please pardon me, if opening an issue for this is bad practice, but I don't know how to open a disussion here by other means.]

As I think your solution beats all others out there in simplicity, I hope you can extend the API documentation a bit more and explain in depth the parameters. For example, how should those Array-Functions be used?

The tutorial is great, but ends up a bit complex to explain concepts like parallelism (forEachStar). I'd love to see a small, but focused example instead.

Anotehr suggestion: For people like me that are new to generators and alike, it's often difficult to understand when to write yield generator or yield generator() and details like these. So, it helps when example code is written verbosely (like, use intermediary variables instead of declaring a generator and immediately calling it, generating a bunch of parenthesis).

Please don't get me wrong. I don't want to criticize. Instead, I hope more people will understand and use your solution. I really love it, and prefer it to all other generator-based solutions I've seen so far (I've studied nearly 10 approaches). Thumbs up!

Hi, after require('galaxy') i'm getting this error on node.js v0.10.14:

/home/kern0/Документы/DEV/GCTickets/node_modules/galaxy/lib/galaxy.js:189
        var F = function *() {
                         ^
SyntaxError: Unexpected token *
    at Module._compile (module.js:439:25)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/home/kern0/Документы/DEV/GCTickets/node_modules/galaxy/index.js:2:18)