Native Node.js Package Dependency about azure-functions-host

[mathewc]

Thanks for the feedback. I agree that it would be best if possible to have an all Node stack for Node.js script support. However, that would entail porting the entire Azure WebJobs SDK over to Node, and developing/maintaining two completely different stacks. Clearly we're not going to do that.

So the approach here is to use the tried and true existing .NET WebJobs SDK, and generate a very thin adapter layer, allowing Node.js and other languages to leverage the SDK investments we've made. This also ensures that in the future, as we expand the SDK, we'll be able to relatively easily pull those new features into the scripting world. If we had to re-implement each feature in Node.js and any other script languages we support in the future, it would be unmanageable.

[lijunle]

Another concern is you are using edge to bridge node.js and C# code. However, there are many opening issues in edge. Edge enforce end user to use a specified version node for their code.

I agree that, re-implement everything is not worth. But, how do you think about using C/C++ Addons to port Microsoft.Azure.WebJobs.Script DLL (the C# version) back to native node.js package?

[mathewc]

Regarding the many open issues in the Edge.js repo - there are many open issues in the WebJobs SDK repo as well :) The question is whether those issues impact the scenarios we're using the library for. We'll discover that in testing. However, we're using Edge.js in a pretty simple way - the contract is pretty simple. So if the Edge.js core interop protocol is solid, we're good.

As I mentioned, we're not going to rewrite/port the WebJobs SDK to Node, so our choice is to use something like Edge.js, or build our own .NET/Node interop, which would only make sense if Edge.js doesn't meet our needs and we're hard pressed.

What is your scenario for a native node.js package? In the primary scenario we're targeting initially for this library, the end user will only provide scripts/manifest files - they won't provide the host - that will be provided automatically. So the .NET host is an implementation detail that they won't be concerned with. However, we also want to consider other scenarios like yours.

[lijunle]

Well, Azure ship a very nice syntax for the scenario in C# - listen on a specified queue, and trigger a function when a message comes. The following code snippet is copied from Webjob tutorial:

public static void GenerateThumbnail(
[QueueTrigger("thumbnailrequest")] BlobInformation blobInfo,
[Blob("images/{BlobName}", FileAccess.Read)] Stream input,
[Blob("images/{BlobNameWithoutExtension}_thumbnail.jpg")] CloudBlockBlob outputBlob)
{
    // handle the blobInfo, input, outputBlob here
}

The SDK will handle get message from queue, delete message from queue, move message from queue to -poison queue, etc.

However, with node.js, the user need to handle this kind of low level operation by them. Why not provide same functionalities and user only care about their logic:

var queueListener = require('an-amazing-queue-listener-provider-by-azure');
queueListener.on('myqueue', function (message) {
    // handle my message, return a promise.
});

it is OK to wrap node.js into C# SDK just like what you are doing, but I think most of node.js developer hope wrap C# SDK inside a node.js package - do not need to care about the C# part.

[mathewc]

Yes, thus far in this Script library we've only addressed the triggering side of things. Using manifest.json you can easily define a function to run on queue messages as you did with [QueueTrigger("thumbnailrequest")].

What we haven't done yet is explore how the other output binders (e.g. [Blob]) translate to Node.js. Clearly there isn't the same attribute based declarative model available. We will be looking into this very soon (stay tuned) - it is our goal to simplify things for Node.js as we did for C# - it will likely just take a different form. In doing this, our goal will continue to be NOT to re-implement the WebJobs SDK in Node.js, but instead find a way to bridge the two as we've started doing.

For example, I can see this extended functionality accessible off of the context object passed into your function:

var inputPath = "images/{BlobName}",
    outputPath = "images/{BlobNameWithoutExtension}_thumbnail.jpg";

module.exports = function (context) {
    context.blob.read(inputPath, function (image, err) {
        var thumbNail = convert(image);
        context.blob.write(outputPath, thumbNail, function (err)  {
            context.done();
        }
    }    
}

function toThumbnail(image) {
    // convert the blob and return a converted image
}

Something along those lines is entirely doable using the WebJobs SDK and mapping via interop the Node.js functions to the SDK. Only a thin interop layer is needed - the heavy lifting continues to live in the WebJobs SDK (C#).

[lijunle]

I agree with you that, the functionalities are entirely doable using WebJobs SDK and WebJobs SDK Script.

My question is, can I do it without touching one single line C# code?

[mathewc]

Yes :) I have the above proposed programming model working, basically bringing parity with the WebJobs SDK to Node.js script. You don't have to code any .NET yourself - you just provide a manifest.json and your script files. The .NET host will bootstrap with your scripts and run.

I still have to work out the deployment model for the .NET host - there are lots of scenarios to consider/cover. As I mentioned above, the first primary scenario is where the script author doesn't own the host - it gets deployed to their WebJob automatically as a package. The scenario you're mentioning (pulling this functionality into a Node.js application) is also interesting and something we should explore.

[lijunle]

Cool, integrate this work to deployment is an awesome solution. The program model could be nice. Thank for talking. I am looking forward to your great news. ;)