nodejs / js-native-api-test

Node.js js-native-api-test suite, externalized so that it can be used to test Node-API implementations outside of Node.js

js-native-api-test

Test suite for JavaScript Engine specific portion of Node-api.

This repository holds a copy of the js-native-api subset of the Node-api test suite so that it can be used to validate native api implementations outside of Node.js.

Currently we are still working on the process to extract the test suite and publish it to this repo on a regular basis.

js-native-api-test's Issues

Concept and approach

We had a discussion today and started to think about how we could enable the test suite to be re-used.

The baseline is that we want to sync the contents of https://github.com/nodejs/node/tree/master/test/js-native-api regularly to the tests directory in this repo.
We then need to be able to build and run the test suite. What we were talking about around the end of the discussion (I've expanded on the concept) was something like this:

• Implementation provides a library
• library provides symbols for all symbols defined in - https://github.com/nodejs/node-api-headers/blob/main/include/js_native_api.h
• library provides symbol set_entry_point(char* name, void* function)
• library provides symbol for int run_javascript(const char* filename, char* javascript, char* output)
  • the engine supporting run_javascript must support
    • require('../../common') -> no-op https://github.com/nodejs/node-api-headers/blob/main/include/js_native_api.hu
    • require('assert') -> return function that provides implementation of Node.js assert function
    • require('./build/${comon.buildType}/XXX') -> invoke the function passed into set_entry_point with the name
      matching XXX passing it an env and exports object.
    • global.gc() -> function that runs gc
• The repo builds a test_runner executable that links against the provided library (say by putting it
  into the impl subdirectory as test_impl. This executable includes a unique entry point for each of the tests.
• When the test_runner is executed it will
  • calls set_entry_point for each of the test entry points. The name will
    be the directory containing the test, and the function passed to it will be a function with the following signature:
    napi_value Init(napi_env, napi_value exports).
  • For each test directory
    • read the JavaScript test file bytes
    • call run_javascript(char* javascript, char* output) passing in the test file bytes. If the result is not true, then
      count the test as a failure, and print out the contents of output
  • After all test directories have been processed report the number of failures.

Existing tests also use the following so additional support may be needed or the tests would have to be updated

Building notes, we need to use the following for each test file:

• -DInit=Init_XXX
  To rename entry point to be unique, were XXX is the name of the test.

Discussion on bootstrapping NTS project

Hi @nodejs/n-api, before drafting the first version of the N-API test suite, I think we have the following issues to address.

Questions collection

• When the test suite is completely moved inside this repository, how to integrate it in Node.js?
• When the test suite is completely moved inside this repository, how to integrate it for other runtimes, is it consistent with Node.js?
• What's the rule to sync N-API tests before this move is complete?

My proposal

When the test suite is completely moved inside this repository, how to integrate it in Node.js?

For example, a bugfix on current implementation on Node.js itself is submitted on our main repository nodejs/node, the test suite needs to be merged on nodejs/napi-test-suite as a precondition, this seems a little verbose to contributors.

A suggestion is to keep the latest test suite for N-API still on Node.js, and archives tests when releasing a Node.js version. Archives tests means:

• pull the given version(v0.10.0) on the master branch.
• publish as a npm package, @nodejs/napi-test-suite?
• also create a tag.

When the test suite is completely moved inside this repository, how to integrate it for other runtimes, is it consistent with Node.js?

When the above proposal is accepted and complete, the others only needs to use npm install by a specific version, and run these tests by themselves. And there is no necessary to use this in Node.js, too, this is for Node.js-like runtimes only.

What's the rule to sync N-API tests before this move is complete?

When the above proposal is accepted and complete, we could skip this problem as well.

/cc @gabrielschulhof @mhdawson

Discussion on node-addon-api test suite

node-addon-api composites multiple C interface N-API to a C++ API and it is maintained officially by Node.js working groups, if test suite covers node-addon-api shall be maintained a copy in napi-test-suite?

Add boilerplate docs

We need to add standard docs including:

CODE_OF_CONDUCT.md
LICENCE.md

The can probably just be copied from https://github.com/nodejs/node-addon-api