Integrate Native Node.js Modules about electron-angular-webpack

[evonox]

After 16 hours of trial and error I managed to create a temporary solution upon this issue in my project. Tomorrow I start refactoring it and in case of interest I may prepare a solution that might become a candidate for a pull request for this issue.

[evonox]

Hi @vunb
I possibly did understand. When the supplied string is not a directory then it must be the module name that should be possible to discover by require method. Then it is necessary for the project module to have a separate package.json file.

[evonox]

Hi @vunb
sorry for the late answer. For the past two weeks I have been busy working on my GNU email client application so I did not have time for the plugin. I will tidy up my notes about the plugin and write here soon what I plan to implement to the next version 0.3.

The good news is that I did the automatic module resolution. You can either enter the name present in package.json of the module or one can address the module by the name of the inner most directory where binding.gyp is placed.

[w-raken]

Hey @vunb,
We are not very familiar with C++ but indeed it could be a great improvement if the boilerplate could support the building of native Node.js modules.
We'll see what we can do to make this possible !
It will be a good way to continue our learning.

[vunb]

I mean the boilerplate should support to re-build the binary for electron using electron-rebuild.

Two things we need to do:

1. Native modules are configured as externals modules in webpack.config.js
2. Install (or copy dependencies) node_modules to output dist with param --prod to exclude devDependencies

Here is my attempt: https://github.com/vunb/electron-angular-webpack/tree/lab/lis

[vunb]

The native module i want to use now is working for me. However, it has specific operations, such as: know exactly native modules which bundle with, rebuild if it present ... the boilerplate does not configure any packages. So I think the decision to use this feature should let the user decide for themselves or we have a guide help people how to configure it.

[lbassin]

Hi @vunb !

I'm sorry I am a little bit busy at the moment and I don't have time to look at this feature 😞
I'll look at this and reply to your comments as soon as I can !

[evonox]

As I have promised the solution is here. Finally I decided to write it as one WebPack plugin and two WebPack loaders, each distributed in a separate NPM module. The solution can be found at electron-native-plugin page. The NPM packages are already published with version 0.1.0.

The situation with native modules is quite complex so the setup process of these NPM plugins is a bit complicated. I would be pleased if you make it part of electron-angular-webpack at least to ease the installation and to raise the level of abstraction.
Otherwise any suggestions are welcome.

[evonox]

My plan for the upcoming version 0.2 of electron-native-plugin is to support building of user-defined native modules mainly. The planned configuration of the plugin might be as follows:

new ElectronNativePlugin({
    forceRebuild: true,
    outputPath: "./relative/path/for/node_modules/builds",
    userModules: [
        "./path/to/your/binding.gyp file",
        {
              source: "./path/to/your/binding.gyp file",
              outputPath: "./output path/relative/to/dist"
        }
    ]
});

[vunb]

Hi @evonox

It seems easy to understand! Could you make the configuration by listing module names? Then it will autodiscover the gyp file?

For example:

new ElectronNativePlugin({
    forceRebuild: true,
    outputPath: "./relative/path/for/node_modules/builds",    // defaults: ./node_modules
    userModules: [
        "<module_name>",
        "serialport",
        {
              // a specific package 
              source: "./path/to/your/binding.gyp file",
              outputPath: "./output path/relative/to/dist"
        }
    ]
});

[evonox]

Hi @vunb ,
the autodiscover of gyp files should have defined some rules we have to specify.

1. First we may have to look it up in node_modules directory
2. However I would like also to provide the ability to build even modules not present in node_modules directory that should be part of the project. I see a solution to specify the root search directory property where all project native modules will be placed. What do you think?
3. It is quite true, there is bindings NPM module but it looks AFAIK only for compiled native modules. But its functionality will come also handy to me.
4. I do not understand defaulting the output path to ./node_modules. This path is meant to be the relative path for distribution package which might be compiled into ASAR archive, for instance.

Thanks to clear me your suggestions more.

[vunb]

Hi @evonox ,

2. However I would like also to provide the ability to build even modules not present in node_modules directory that should be part of the project. I see a solution to specify the root search directory property where all project native modules will be placed. What do you think?

If so, i must to copy packages and manage them manually in somewhere. But how about the dependencies of native modules? Are they going to have managed in other node project by package.json or just lie on in some folder, such as deps?

3. It is quite true, there is bindings NPM module but it looks AFAIK only for compiled native modules. But its functionality will come also handy to me.

I also see it just looks up compiled binary file.

4. I do not understand defaulting the output path to ./node_modules. This path is meant to be the relative path for distribution package which might be compiled into ASAR archive, for instance.

Sorry for my miss reading the outputPath and not fully understand its meaning. Now it ok!

Hope you doing it well!

[evonox]

Hi @vunb ,
thanks for your reply. I have read the article you have given here in your first post and I think I got the key point. It had given me another good ideas how to enhance the development of ElectronNativePlugin.
To continue our discussion about point two, I would like to differentiate between two types of native modules. That are library native modules and project native modules.

By library native modules I mean just the ones present in node_modules directory. By project native modules I mean the ones you have C/C++ source code in your src directory.

As for library native modules, they are already supported in ElectronNativePlugin version 0.1 which is already released. The only requirements to embed them are that they have to be compiled by node-gyp and they must be mentioned in dependencies in package.json. The plugin then just does the following steps:

1. Parses package.json
2. Checks which dependencies are native
3. Rebuilds them by running electron-rebuild
4. Copies the resulting .node file to the output dir
5. Updates automatically all dependencies is JS
6. Then it can be bundled successfully by WebPack

In the version 0.2 of ElectronNativePlugin I primarily focus on project native modules. My idea is as follows:

1. The plugin will build a project native module by node-gyp
2. It will rebuild them by electron-rebuild
3. Copy the resulting .node file to the output
4. Update JS dependencies.

So the only native modules that need to be mentioned in webpack.config.js configuration are the project native modules. First we may specify the path of such a module in the configuration or we may implement the auto-discover feature by its name.
Now it depends. I can scan the whole project file system, except for node_modules directory of course, for a binding.gyp of a named module or this search can be limited to a certain project subfolder.

I have no other idea for now how to do the auto-discover feature for project native modules which might not be having even package.json file (quite true electron-rebuild requires them so I am playing with the idea to generate them automatically when not present).

[vunb]

@evonox How is your progress so far?

[vunb]

I believe you are doing well. I also busy too. But i hope will try to experiment this feature.

I hope we are doing it well!. Thank you for your effort!

[evonox]

Hi @vunb,
I forgot to mention that to experiment with the described automatic native module resolution you need to clone the official master branch. I did not publish it yet as a NPM package.

To be honest, I am a WebPack newbie and I have one thing I cannot find a way to solve.
Currently the plugin is divided into three separate packages, one is the ElectronNativePlugin and the other two are co-operating loaders. I do not consider it as a good design practice. I would like to make it one NPM package and also to ease the configuration for the user and place it into one place.
The thing is that I cannot find in the official WebPack docs if a WebPack plugin can add loaders in runtime. In the worse case I can prepare a sort of "wrapper loader" which would add the two loaders in the pitch phase.

At last I have a request. I am testing the plugin in Windows 10 and two Linux distros (Ubuntu and Debian). I cannot test if it will be running on MacOS so I would need a help with this.
I am going to prepare three samples for this issue.

[vunb]

At last I have a request. I am testing the plugin in Windows 10 and two Linux distros (Ubuntu and Debian). I cannot test if it will be running on MacOS so I would need a help with this.

Why do you not create Travis CI or Appveyor for testing? It has already Windows, Mac and Linux.

[evonox]

Quite right, I will check those services and prepare tests for the plugin. So far I have been testing the plugin only in my own project build procedure which is not sufficient. Currently I managed to run MacOS in VirtualBox on my PC, the only problem is that I am not able to install node-gyp on Mac due to some SSL certificate problems :-/.

[evonox]

Hi @vunb
I have published a new version of the ElectronNativePlugin.

Additionally I started a new repo containing samples of how to use this plugin at: https://github.com/evonox/electron-native-plugin-samples/.

The next sample into this repo I am planning is the use of Sqlite3 with TypeORM.
Any ideas for another samples are welcome.