nodejs / build-toolchain-next Goto Github PK

Native addons will be eventually affected by this change, therefore we need to devise a migration plan, including expected timelines and preferably migration tooling.

Opening since @mmarchini asked that we have a call for people who are interested and want a place where people can say they are interested. mPlease comment on the issue if you would be interested in joining.

There is no champion for the strategic initiative around next-gen toolchain for Node.js. (It was @mmarchini, but she is stepping away from the project for a while.) This could use a champion. @nodejs/tsc @nodejs/build

Refs: nodejs/node#42750

Adding a tsc-agenda label to see if anyone is interested or has ideas for who might be a good person to pick it up. But it can be removed if there is discussion and no candidate emerges.

List GN and Ninja platform support so we can compare with our existing supported platforms.

I don't know exactly what your requirements are, but in case this can be of any help, I'm maintaining a set of cross-platform standalone binary packages, including native toolchains (GCC and clang) and cross toolchains (arm-none-eabi, riscv-none-elf and aarch64-none-elf), plus other tools, like CMake, ninja, meson, qemu, all intended for reproducible builds.

The projects are available from:

• https://github.com/xpack-dev-tools/

These binary packages can be easily installed with xpm, which is an extension of npm:

• https://github.com/xpack/xpm-js

For example the toolchains and other tools can be referred as dependencies by other projects and used during build, like regular builds for native components or unit tests.

Here is an example of a project using such dependencies:

• https://github.com/micro-os-plus/utils-lists-xpack/blob/616fb03b9835d1c1c17c3048c082f7ba0e005a1e/package.json#L35-L40
• https://github.com/micro-os-plus/utils-lists-xpack/blob/616fb03b9835d1c1c17c3048c082f7ba0e005a1e/package.json#L257-L258

If you think that anything from these projects can be of any help for node, or for any of your other projects, I'd be more than happy to contribute.

Hybrid builds are builds using more than one meta build system (for example, building V8 with GN and the rest with CMake). These builds would allow us to perform any migration progressively (we could migrate one dependency at a time), and if we can build V8 with GN effectively the work needed to upgrade V8 should be greatly reduced.

Each option we have come with a list of advantages and drawbacks. We need to list and evaluate those so we can decide which trade-offs we are willing to make.

V8 and Chromium don't support it officially (they use Clang on all platforms AFAIK) and I have to investigate compiler issues on Windows almost for each V8 update.

Our current approach to have all dependencies in-tree is optimized for some use cases, like offline development. Since dependencies are tightly coupled to our meta build toolchain, this initiative presents an opportunity to re-evaluate how we handle dependencies. We should understand the benefits and drawbacks of the current approach compared to other approaches (like submodules or depot_tools) so we can decide if we are making the right trade-offs.

forking discussion from #4

Today node-gyp is bundled with npm (which is bundled with Node.js), therefore we don't provide a build toolchain for native addons on our own. I wonder if as part of the node-gyp deprecation and migration to something else, we should start bundling that something else with Node.js. Building a native module is more coupled to the Node.js version than it is to the package manager, so from a design perspective it makes sense. It also ensures that deprecations and changes are properly propagated across package managers.

Just bundling would be easy, but I think we can go one step further and provide APIs that allow package managers to identify build files (so that npm install can still run the build toolchain when those are present) as well as provide APIs that abstract the necessary build parameters (defines, includes, etc) so that community can safely write their own build toolchains (not limiting folks to whatever we decide to use). Depending on how well we can write those APIs, maybe we don't even need to bundle anything (bold idea, I know).

As for what a plan would look like, I think the first step would be to move node-gyp to be bundled with Node.js instead of package managers, that'll give us more control over the deprecation moving forward.

P.S.: I haven't looked into how yarn and pnpm handle node-gyp yet (specifically, if they bundle it themselves or not).

Changing the meta build system means we'll need to make changes to our build infrastructure (either just installing required dependencies or rethinking how we build and test nodejs/node). This issue is intended for us to discuss what options we have and to develop a plan of action.