neon-bindings / neon-cli Goto Github PK
View Code? Open in Web Editor NEW[MOVED] THIS REPOSITORY HAS BEEN MERGED INTO neon-bindings/neon
License: Apache License 2.0
[MOVED] THIS REPOSITORY HAS BEEN MERGED INTO neon-bindings/neon
License: Apache License 2.0
Right now the neon
cli serves as both the utility for starting new neon modules (the boilerplate) and the cargo build tool. I'm suggesting that we separate the cargo build functionality into it's own module neon-build
.
Separating the build functionality into a neon-build
dependency means neon modules don't have to depend on the entire neon-cli
, reducing the dependency tree for neon modules and consumers of neon modules.
Additionally this can help get us closer to a npm
scripts style workflow (rather than expecting neon
to be on the path globally). Mentioned in #34 and neon-bindings/neon#126.
We can achieve this without breaking changes to neon-cli
by using the new neon-build
module internally. We could probably just replace the require
statement for ./opts/neon_ build.js
, or something similarly simple.
Something about the latest version of the gypfile is causing the creation of a target
directory in the parent of the project root.
Currently neon build
automatically sets the Node ABI version number by extracting it from process.versions.modules
. It should be possible to pass an optional ABI version like neon build --abi=43
to support "cross-compiling" to different Node versions.
In #42 the Windows build logic for build.rs is open-coded, which exposes its implementation details to neon users. We should abstract this into a Rust crate for build-time logic, so the generated build.rs can look something like:
extern crate neon_build;
fn main() {
neon_build::main();
}
Need a command that deletes the index.node
and cargo build output.
Test and troubleshoot Windows support.
Was originally filed as neon-bindings/neon#28. The functionality is there but it needs a test.
The response to the prompts in the neon new
flow are not escaped when they are interpolated into the resulting package.json
, which can lead to an invalid JSON structure. For instance, entering the value my "cool" project
for the "description" prompt results in a package.json that looks like:
// ....
"description": "my "cool" project",
// ...
With @eddyb's work in neon-bindings/neon#29, I can see a glimpse of a more streamlined project structure for Neon projects. I'll probably get working on a branch ASAP but I thought I'd put down my thoughts here to get feedback, and also as a heads up to folks who've been helping get our infrastructure started.
There are three major components to the streamlined structure:
binding.cc
and no more binding.gyp
native/
subdirectoryneon build
command that's used by package.json
The basic project structure would look like this:
foobar/
├── package.json
├── lib/
│ └── index.js
└── native/
├── Cargo.toml
└── src/
└── lib.rs
After building, a project would look like:
foobar/
├── package.json
├── lib/
│ └── index.js
└── native/
├── Cargo.toml
├── Cargo.lock
├── src/
│ └── lib.rs
├── target/
│ └── release/
│ └── libfoobar.so
└── index.node
Within the project, loading the native module (for example from a module in the lib/
directory) would look like:
var nmod = require('../native');
The package.json
would require a postinstall hook:
"scripts": {
"postinstall": "neon build"
}
The vast majority of the required C++ code is used in node-sys to create the actual Neon library itself. But for a Neon project, the only remaining C++ code was the tiny bit of top-level boilerplate code in binding.cc
, which @eddyb has shown in neon-bindings/neon#29 we can do in pure Rust. This means we don't need node-gyp at all for building an individual project, and we can eliminate both binding.cc
and binding.gyp
entirely. (The node-sys project would continue to be built using node-gyp.)
native/
subdirectoryThis has several benefits:
src
directory: The convention for compile-to-JS (including ES6 via Babel) is to put it in src/
, which creates a collision with having the Cargo project and the NPM project share the same root directory. Moving the Cargo project into native/
eliminates the overlap.native/index.node
and not have a lib/
directory at all.neon build
back to package.json
Having the build step go through our own entry point puts us in control of the build process, and allows us to script it in JS instead of GYP (which has proven to be extremely challenging). The basic logic of the build step is:
native/index.node
I was temporarily attracted to the idea of eliminating a dependency on neon-cli
in a Neon project's package.json
. However, a) there's no net increase in boilerplate within the package.json
, since we eliminate the dependency on neon-bridge; b) it's not unreasonable to have a dependency on a build tool in a Node package's build step; c) having a permanent dependency on node-gyp for kicking off all build logic is just painful to contemplate.
In the event that config.user.name
is undefined
(even when config.user.email
is defined) nothing is returned from guessAuthor()
. I don't know if the try/catch
was supposed to be an else
for the if
or if gitconfig
actually can throw
an error requiring both an else
and a catch
.
In other news, investigating this led me to discover that I don't have user.name
in my gitconfig on this computer. Strange...
I have prepared a repo with example that illustrates my problem: https://github.com/jchlapinski/neon-debug-example
Basically when I try to run unit tests for debug profile in my rust code (see https://github.com/jchlapinski/neon-debug-example/blob/master/neon-build-debug.sh) I get linker errors when compiling tests. Everything works OK, when I use release profile (see https://github.com/jchlapinski/neon-debug-example/blob/master/neon-build-release.sh).
Is this a neon-bindings bug or am I doing something wrong here?
Regards
Neon projects should always contain a .neon
directory for internal metadata that doesn't go into source control. This is an idiomatic way to hide metadata that users shouldn't normally have to think about.
The artifacts.json
file should live in that directory instead of under native/
, and then it doesn't need an entry in .gitignore
.
This will also then become the easy way to determine whether a project is a Neon project, and electron-rebuild can use it as the heuristic to build the project with Neon.
This makes is impossible to use any sort of scripts that rely on exit codes.
I wanna use classes and modules. Set the project up to use Babel so I can.
Right now the default is to use nightly. When the following patches make their way to stable, switch the default to use the system Rust:
I believe this means 1.5 is the first stable version that should work with Neon.
As mentioned in neon-bindings/neon#159, cargo build script output is suppressed unless an error occurs. Should we add an argument or piggyback on the debug argument, so we can pass -vv
to cargo?
With 0.1.15:
$ neon build
neon info forcing rebuild for new build settings
neon info running cargo
warning: unused manifest key: workspaces
Compiling regex-syntax v0.4.0
Compiling cslice v0.2.0
... etc ...
0.1.16 is broken due to the missing ts-typed-json.
With 0.1.17:
$ neon build
neon ERR! Cannot read property 'length' of undefined
I haven't found an obvious way to get debug output out of neon-cli.
Node version 6.10.3, also tested with 7.10.0.
npm version 5.0.0
Should always print the neon-cli package version number when doing printUsage(), and maybe also have a neon version
command.
The prepublish
step is deprecated in npm5. What should we replace it with?
It would be great to have a neon upgrade
feature that helps you upgrade a Neon project to the latest project structure.
I am planning to relicense this project under a dual MIT/Apache-2.0 license. In practice this won't limit what people can do with the codebase compared with today's single MIT license, but it's a better, more robust choice than MIT alone, and it will make it easier and cleaner to put neon and neon-cli together in a single repo (I'm pretty sure it wouldn't be necessary, but it'll be easier for people to understand).
To agree to relicensing, please leave a comment on this issue with:
I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to choose either at their option.
I'll check people's names off as they leave their comments. Once we get all the signoffs I'll go ahead and do the relicensing.
I apologize for the inconvenience and really appreciate everyone's help, both for this relicensing and for your contributions to Neon!
There should be a rust-bindings binary that can be used to run eager builds with a single line addition to package.json. It should accept as command-line arguments all the configuration options accepted by the API.
Hi! I am writing an electron node addon based on https://github.com/dherman/neon-electron-hello (thank you very much for this example, I was able to get up in running in no time despite the fact that it was my first ever electron program).
The problem is, everytime I run npm run build
when there is a compile time error in my Rust code, neon prints:
neon info forcing rebuild for new build settings
neon info running cargo
and then rebuilds the world, which is kinda slow:
Compiling neon v0.1.15
Compiling neon-build v0.1.15
Compiling cslice v0.2.0
Compiling gcc v0.3.45
Compiling neon-hello v0.1.0 (file:///home/matklad/projects/neon-hello/native)
Compiling neon-runtime v0.1.15
If I change my rust code without introducing any errors, then only my code gets rebuild. Perhaps this forcing rebuild
is spurios? I didn't change package.json
, so the build settings are the same.
Or perhaps there's something wrong with my build setup?
I have projects/neon-electron-hello
and projects/neon-hello
and I've executed npm link ../neon-hello
inside of neon-electron-hello
.
Building should leave breadcrumbs in native/.build.json
with a record of the environment settings that were in place during the build. For example the values of a build done for electron would look like:
{
"npm_config_target": "1.6.2",
"npm_config_arch": "x64",
"npm_config_target_arch": "x64",
"npm_config_disturl": "https://atom.io/download/electron",
"npm_config_runtime": "electron",
"npm_config_build_from_source": true,
"npm_config_devdir": "~/.electron-gyp"
}
and it should compare the current values of those environment variables against the breadcrumbs to see if it needs to force a rebuild.
As @jedireza caught in neon-bindings/neon#165, the dependency on neon-build
should be in build-dependencies
, not dev-dependencies
.
Thank you for building Neon! I've already got a small Rust-based addon in production at work, and it was dead simple to get up and running.
For my own amusement, I've been working on getting Neon to work an Electron app. This is trickier, both because of node-gyp
issues, and because Neon always assumes it's building packages for the system code of Node.js. Here's how I worked around this.
This is for a top-level Electron app project with a native
subdirectory directly in the main package, and not under node_modules
.
node-gyp
Save the following file as .env.sh
in the top level of the project:
# Source this with `. env.sh` before running `yarn` or `native/build.sh`.
# This sets up the configuration required by `node-gyp` and `neon` to build
# native modules that will successfully link against Electron instead of
# the system copy of NodeJS.
export NEON_NODE_ABI=50
export npm_config_target=1.4.12
export npm_config_arch=x64
export npm_config_target_arch=x64
export npm_config_disturl=https://atom.io/download/electron
export npm_config_runtime=electron
export npm_config_build_from_source=true
$HOME
#!/bin/bash
#
# This script can be invoked from the root directory as `native/build.sh`.
# It will make sure we have the right Node.js headers to build a module for
# Electron and run the actual build manually.
#
# If this is giving weird resutls, try:
#
# rm -rf node_modules native/target
#
# ...and try again.
set -euo pipefail
# Create a fake home directory under ~/.electron-gyp to cache things
# related to Electron builds. We need to make symlinks to various Rust
# config directories if we want this to work.
mkdir -p ~/.electron-gyp
if [ ! -h ~/.electron-gyp/.cargo ]; then
ln -s ~/.cargo ~/.electron-gyp/
fi
if [ ! -h ~/.electron-gyp/.multirust ]; then
ln -s ~/.multirust ~/.electron-gyp/
fi
# Switch to the fake home directory.
HOME=~/.electron-gyp
# Make sure that our Electron Node headers are installed.
if [ ! -d ~/.node-gyp/iojs-$npm_config_target ]; then
node_modules/.bin/node-gyp install
fi
# Run the actual build, and copy it to where node will find it. Ideally
# we'd just call `neon build`, but that requires neon-bindings/neon#109 and
# neon-bindings/neon-cli#31 to have any chance of working.
case `uname -s` in
Darwin)
cd native
cargo rustc --release -- -C link-args=-Wl,-undefined,dynamic_lookup
cp target/release/libtestnative.dylib index.node
;;
Linux)
cd native
cargo build --release
cp target/release/libtestnative.so index.node
;;
*)
echo "Don't know how to build native extensions on this platform" 2>&1
exit 1
esac
The most important fixes would be something like:
This would allow me to replace the ugly case
statement with a normal neon build
call, which would be a clear improvement.
I'm less sure that Neon should touch any of the node-gyp
stuff. Some of this is handled by electron-rebuild for regular Node addons, and there might be some way to tie into that system. Or not.
Anyway, because everybody loves screenshots:
That shows the Neon-based addon being loaded into both the main
and renderer
process of Electron. It works! The full application is here.
With neon-bindings/neon#109 we extract the node modules ABI version number from the headers instead of pulling it from the environment variable. When we upgrade to neon 0.2 we should get rid of the environment variable entirely.
(This was done in #37 but rolled back in #41 to wait until we're ready for 0.2.)
Validation errors that are longer than the terminal width and wrap around to the next line cause inquirer to get confused about how to position the cursor back on the preceding line.
I did "neon new test" right after here's the error.
http://pastebin.com/quZQf2M3
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.