vaticle / typedb-driver-nodejs

TypeDB Driver for Node.js

THIS REPOSITORY HAS BEEN ARCHIVED. Please visit: @vaticle/typedb-driver/nodejs

---

TypeDB Client for Node.js

Client Architecture

To learn about the mechanism that a TypeDB Client uses to set up communication with databases running on the TypeDB Server, refer to TypeDB > Client API > Overview.

API Reference

To learn about the methods available for executing queries and retrieving their answers using Client Node.js, refer to TypeDB > Client API > Node.js > API Reference.

Concept API

To learn about the methods available on the concepts retrieved as the answers to TypeQL queries, refer to TypeDB > Concept API > Overview.

Installation

Node.js version 14 or above is recommended.

npm install typedb-client

Further documentation: https://docs.vaticle.com/docs/client-api/nodejs

Using TypeScript

typedb-client is a TypeScript project and provides its own type definitions out of the box - for example:

import { EntityType } from 'typedb-client';

Build TypeDB Client for Node.js from Source

Note: You don't need to compile TypeDB Client from source if you just want to use it in your code. See the "Import TypeDB Client for Node.js" section above.

We recommend using yarn, see https://yarnpkg.com/ for information about yarn.

1. Make sure you have the following installed: Node.js (version 14 or above); npm package manager; yarn package manager.
2. Clone the project and run yarn at the root directory (containing package.json)
3. Run yarn run build
4. The JavaScript files, and their matching TypeScript type definitions are compiled to the dist directory.

Note: TypeDB Client can be run without TypeScript, however its type assertions may make development smoother.

typedb-driver-nodejs's Issues

User role management

Implement user role management API supporting the following functionalities:

1. Role CRUD
2. Assigning a role to a user

Enable BDD tests for all Graql Queries on Client Nodejs

Problem to Solve

Not all Graql query BDD tests are called in CI.

Proposed Solution

Call all Graql query BDD tests in CI, and make sure they pass.

Replica tracking simplification

Currently, the client keeps track of various aspects of the replicas of a given database: role, term, and 'is prefered secondary' status. The information is stored in the ClusterDatabase class that is quite heavy.

Keeping track of this information on the client is complicated, and we need to minimise it in order to make the client is as thin as possible.

The minimum number of information that can be stored is the IP addresses of all servers. The other information can be queried from the servers as needed.

TypeDB Cluster: Update password

A user must be able to update their password

Un-ignore test for modifying schema in a READ transaction

See the TODO comment in CommtTx.test.js.

matchAggregate queries throw an exception

Description

Currently, attempting to run match-aggregate queries with Client Nodejs throws:

(node:8074) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'getQueryRes' of null
    at /tmp/tmp.ZKQcOC0NKy/node_modules/grakn-client/query/QueryManager.js:58:167
    at QueryManager.<anonymous> (/tmp/tmp.ZKQcOC0NKy/node_modules/grakn-client/query/QueryManager.js:93:24)
    at Generator.next (<anonymous>)
    at fulfilled (/tmp/tmp.ZKQcOC0NKy/node_modules/grakn-client/query/QueryManager.js:22:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Environment

1. OS (where Grakn server runs): Ubuntu 20.04
2. Grakn version (and platform): 2.0.0-alpha
3. Grakn client-nodejs version: npm i https://repo.grakn.ai/repository/npm-snapshot-group/grakn-client/-/grakn-client-0.0.0-e6be60399a3bb79d3c980c227f23ad9fa15af327.tgz
4. Node.js version: v12.18.2
5. Other environment details: -

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Create database q; define x sub attribute, value long; and insert some: insert $x isa x; $x 12;; insert $x isa x; $x 24;
2. Run this piece of nodejs code:

const { GraknClient } = require("grakn-client/rpc/GraknClient");
const { Grakn } = require("grakn-client/Grakn");
const { SessionType, TransactionType } = Grakn;

async function xxx() {
	const client = new GraknClient()
	const session = await client.session("q", SessionType.DATA)
	const tx = await session.transaction(TransactionType.READ)
	console.log(`answer = ${tx.query().matchAggregate('match $x isa x; mean $x;')}`)
}

xxx().then(
    text => {
    },
    err => {
    }
);

Expected Output

Query returning an actual result (mean of 12 and 24)

Actual Output

TypeError: Cannot read property 'getQueryRes' of null

Transaction is not closed on the client when it fails on the server

Description

When a transaction fails with an exception (on the server), it is not closed on the client.

Environment

1. OS (where Grakn server runs): Mac OS 10.14.6
2. Grakn version (and platform): Grakn Core 1.5.9
3. Grakn client-nodejs version: client-nodejs 1.5.5
4. Node.js version: 11.15.0

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Open a client, session and transaction
2. run await transaction.query('match typo');
3. Receive an exception (expectedly)
4. run await transaction.query('match whatever');

Expected Output

Receive an exception saying that the transaction has already been closed.

Actual Output

Nothing happens

Client-side timeout

Client-nodejs does not set a client-side timeout, and will stall forever if the network gets interrupted.

Grakn version:
1.5

Reproducible step:

1. Start grakn: grakn server start
2. Run the nodejs-equivalent of the following Java queries using client-nodejs:

try (GraknClient.Session session = new GraknClient("localhost:48555").session("grakn")) {
        while (true) {
                GraknClient.Transaction tx = session.transaction(Transaction.Type.WRITE);
                tx.execute(Graql.match(var("x").sub("thing")).get());
                System.out.println("query");
                tx.commit();
        }
}

3. Pause grakn in the middle: kill -STOP <grakn process id>
4. Observe that the queries will stall and never times out

TODO

1. The client should be able to set a timeout when instantiating the client. We need to set a default, eg., 15 seconds.

client1 = new Grakn(DEFAULT_URI, 20) // timeout set to 20 seconds
client2 = new Grakn(DEFAULT_URI) // timeout set to the default timeout, eg., 15 seconds

2. The provided timeout value should be used to set the deadline of GRPC requests.

Useful references:

• grpc/grpc-node#541
• According to https://stackoverflow.com/a/44324135/1622847, the deadline needs to be specified in every requests and there's no way to set them once per connection. This would be unmaintainable and error prone

Integration tests fail locally with `undefined` errors

Description

Integration tests fail locally with undefined errors

Environment

1. OS (where Grakn server runs): Mac OS 10
2. Grakn version (and platform): 1.5.9
3. Grakn client-nodejs version: 1.5.6-SNAPSHOT (b52a974)
4. Node.js version: 6.5

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. run git checkout b52a974b223f6ffebc4740a198ca7ecc2783b601
2. run bazel test //:test-integration --test_output=streamed

Expected Output

Tests passing.

Actual Output

All tests failing with many undefined errors.

Additional information

The same tests fail within the CI of a PR, but they all pass within a commit made to master!

Upgrade client-nodejs to use the latest @graknlabs_common

We need to upgrade graknlabs_common to the latest, which constitutes upgrading all the bazel rules for BDD tests that run Grakn artifacts.

Unify NodeJS dependency declarations into Bazel

Right now, some dependencies and their respective version numbers are declared in package.json. This is not ideal and they should be declared in Bazel instead.

Typescript Definition file needed

Problem to Solve

Need Typescript definitions

Current Workaround

None

Proposed Solution

Create Typescript definition file(s)

Additional Information

See https://www.typescriptlang.org/docs/handbook/declaration-files/introduction.html

Refactor tests to use jasmine instead of jest

Problem to Solve

JS tests should use a unit test library

Current Workaround

jest is used

Proposed Solution

use jasmine instead

Additional Information

Reasons for refactoring tests:

• jasmine is natively supported in rules_nodejs
• jasmine has way less dependencies than jest

The package deployed to `npm` uses UMD modules, not CommonJS, causing incompatibility with Workbase

Problem to Solve

In tsconfig.json, we specify:

"compilerOptions": {
  "module": "commonjs"
}

This option causes the TypeScript compiler tsc to output modules in the CommonJS format which is fully supported by NodeJS.

However, this option is roundly ignored by the Bazel rule ts_library:

load("@npm//@bazel/typescript:index.bzl", "ts_library")

ts_library(
    name = "_client_nodejs",
    [...]
    tsconfig = "tsconfig.json",
)

According to Bazel (source: https://docs.bazel.build/versions/master/build-javascript.html#building-typescript):

Bazel controls parts of the tsconfig.json file that define locations of input and output files, manage dependencies on typings, and produce JavaScript output that’s readable by downstream tooling. Currently, this format is unbundled UMD modules, wrapping noth named (non-anonymous) AMD modules and commonjs modules.

UMD modules are incompatible with Webpack, the most popular front-end JS code bundler. This is particularly inconvenient for us as Grakn Workbase uses VueJS which uses Webpack.

Reproducible Steps

1. Run npm install in the root directory of client-nodejs to install all dependencies
2. Run tsc, which will produce a number of .js and .d.ts files in a folder named dist
3. Run bazel build //:_client_nodejs, which will produce corresponding .js and .d.ts files in bazel-bin
4. Compare the first few lines of (for example) dependencies_internal.js between the two:

Lines 20-30 of dist/dependencies_internal.js

var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    Object.defineProperty(o, k2, { enumerable: true, get: function() { return m[k]; } });
}) : (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    o[k2] = m[k];
}));
var __exportStar = (this && this.__exportStar) || function(m, exports) {
    for (var p in m) if (p !== "default" && !exports.hasOwnProperty(p)) __createBinding(exports, m, p);
};
Object.defineProperty(exports, "__esModule", { value: true });

Lines 19-33 of bazel-bin/dependencies_internal.js

var __createBinding = (this && this.__createBinding) || (Object.create ? (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    Object.defineProperty(o, k2, { enumerable: true, get: function() { return m[k]; } });
}) : (function(o, m, k, k2) {
    if (k2 === undefined) k2 = k;
    o[k2] = m[k];
}));
var __exportStar = (this && this.__exportStar) || function(m, exports) {
    for (var p in m) if (p !== "default" && !exports.hasOwnProperty(p)) __createBinding(exports, m, p);
};
(function (factory) {
    if (typeof module === "object" && typeof module.exports === "object") {
        var v = factory(require, exports);
        if (v !== undefined) module.exports = v;
    }

We need the version in dist to be deployed to npm, but currently, we deploy the version in bazel-bin to npm.

Current Workaround

We can run tsc on our local machines to generate a working grakn-client, and then copy that into the Workbase source code.

Proposed Solution

We need to, somehow, use tsc to compile the .js and .d.ts files, and bundle those into an npm package using Bazel. The npm package is built by the pkg_npm rule. This currently takes in :_client_nodejs as a dependency. Probably we just need to update that dependency to a group of files produced by tsc.

Is this library functional yet?

I got pointed here by the grakn website, but when I try to require the library I just get MODULE_NOT_FOUND. There's no index.js or main field in the package.json, and all the tags indicate alpha state... I guess it's not ready yet? If you guys know anyone who manages the grakn website, maybe they should update/ change their documentation?

Thanks.

'npm install typedb-client' on MacOS fails

Please replace every line in curly brackets ( { like this } ) with appropriate answers, and remove this line.

Description

'npm install typedb-client' on MacOS fails produces an installation error on the google-p12-pem package

Environment

1. OS (where TypeDB server runs): MacOS 10.14.6
2. TypeDB version (and platform): 2.1.1
3. TypeDB client-nodejs version: 2.1.0
4. Node.js version: node v12.19.0
5. Other environment details:

Reproducible Steps

type 'npm install typedb-client'

Expected Output

Installation should complete correctly

Actual Output

Installation fails with this error message :
$ npm install typedb-client
npm ERR! code ENOSYS
npm ERR! syscall symlink
npm ERR! path ../google-p12-pem/build/src/bin/gp12-pem.js
npm ERR! dest /Users/...../node_modules/.bin/gp12-pem
npm ERR! errno -78
npm ERR! ENOSYS: function not implemented, symlink '../google-p12-pem/build/src/bin/gp12-pem.js' -> '/Users/...../node_modules/.bin/gp12-pem'

npm ERR! A complete log of this run can be found in:
npm ERR! /Users/...../.npm/_logs/2021-06-01T07_08_40_182Z-debug.log

Additional information

Strangely enough, 'npm install -g typedb-client' (global install) works without errors

Detect incompatibility and raise a relevant exception

Problem to Solve

When there is an incompatibility between Client Node.js and the running Grakn Server, the exception thrown by the client seems unrelated to the real cause of the problem. In fact, the error message may even be incorrect, given that the keyspace does exist with the given name.

Current Workaround

If encountering this exception for the first time, only searching the web or asking the community can lead the user to the solution.

Proposed Solution

Have Client Node.js to identify the incompatibility and throw an exception with a helpful message. e.g. Client Node.js version 1.5.3 does not support the running Grakn Core server. Please refer to the documentation of client Node.js to ensure compatibility.

Additional Information

N/A

Trying to close a transaction that is already closed will stall forever

Environment

Grakn client-nodejs version: client-nodejs 1.5.1

Steps to create the smallest reproducible scenario:
tx.close() or tx.commit()
and then tx.close() again

Expected Output

move on with life normally

Actual Output

the execution stalls forever

Add `ConceptMap.get("x")` method

Problem to Solve

Client Python and Client Java both have a convenience method on the Answers that are ConceptMap. To be specific, both offer a shortcut method to get the concept corresponding to a variable called conceptMap.get("x"). In nodejs, the current solution is to do conceptMap.map().get("x"). Note that both java and python have .map() as well, though in java the usage of the map is more complicated because it returns a Map<Variable, Concept>, which requires lookup with a Variable instance, which comes from Graql (more involved).

Thus, to bring nodejs in line with the other clients, we should add the .get('x") method. This means all the clients also expose the same .get("x") method signature to get a concept by variable.

Current Workaround

Use conceptMap.map().get("x").

Proposed Solution

Add a method get(var) on ConceptMap answer type.

Additional Information

Should match the functionality available in client-python and client-java

Add a function to the Cluster client to return the current user

Problem to Solve

We have a .user() function for the TypeDB Cluster client in client-java, but not for client-nodejs.

Current Workaround

If you know the username, client.users().get(username)

Proposed Solution

Provide a .user() function for the Cluster client.

QueryManager's matchAggregate, matchGroup and matchGroupAggregate send queries with incorrect type

Description

Currently, match* methods all send Query.Match.Req (as if they are usual match queries) which results in server throwing this error:

 Error: 13 INTERNAL: [GQL04] Graql Error: The class 'GraqlMatch.Aggregate' cannot be casted to 'GraqlMatch'. Please check server logs for the stack trace.

Reproducible Steps

const client = new GraknClient("localhost:1729");
const session = await client.session("phone_calls", SessionType.DATA);
const transaction = await session.transaction(TransactionType.READ);

let firstQuery = [
		'match',
		'  $customer isa person, has age < 20;',
		'  $company isa company, has name "Telecom";',
		'  (customer: $customer, provider: $company) isa contract;',
		'  (caller: $customer, callee: $anyone) isa call, has duration $duration;',
		'get $duration; mean $duration;'
];
firstQuery = firstQuery.join("");
const firstIterator = await transaction.query().matchGroupAggregate(firstQuery);

Expected Output

Getting result of the query (number)

Actual Output

[GQL04] Graql Error: The class 'GraqlMatch.Aggregate' cannot be casted to 'GraqlMatch'

Closing a session, immediately after it's been opened, throws an error

Description

Closing a session, immediately after it's been opened, throws:
(node:57100) UnhandledPromiseRejectionWarning: Error: 2 UNKNOWN: null

Environment

1. OS (where Grakn server runs): Mac OS 10.14.2
2. Grakn version (and platform): 1.5.0 SNAPSHOT
3. Grakn client-nodejs version: 1.5.0 SNAPSHOT
4. Node.js version: 10.12.0

Reproducible Steps

Steps to create the smallest reproducible scenario:
index.js:

const Grakn = require("grakn-client");

openSession("social_network");

async function openSession (keyspace) {
    const client = new Grakn("localhost:48555");
    const session = await client.session(keyspace);
    await session.close();
};

run: node index

Expected Output

nothing to be logged into the terminal.

Actual Output

(node:57100) UnhandledPromiseRejectionWarning: Error: 2 UNKNOWN: null. Please check server logs for the stack trace.
    at Object.exports.createStatusError (/Users/soroushsaffari/tests/jclient/node_modules/grpc/src/common.js:91:15)
    at Object.onReceiveStatus (/Users/soroushsaffari/tests/jclient/node_modules/grpc/src/client_interceptors.js:1204:28)
    at InterceptingListener._callNext (/Users/soroushsaffari/tests/jclient/node_modules/grpc/src/client_interceptors.js:568:42)
    at InterceptingListener.onReceiveStatus (/Users/soroushsaffari/tests/jclient/node_modules/grpc/src/client_interceptors.js:618:8)
    at callback (/Users/soroushsaffari/tests/jclient/node_modules/grpc/src/client_interceptors.js:845:24)
(node:57100) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:57100) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Additional information

if the session is used to create a transaction, followed by closing the transaction, no errors get thrown:

const Grakn = require("grakn-client");

openSession("social_network");

async function openSession (keyspace) {
    const client = new Grakn("localhost:48555");
    const session = await client.session(keyspace);
    const transaction = await session.transaction(Grakn.txType.READ);
    await transaction.close();
    await session.close();
};

BDD: User management

User management scenarios must be implemented

Provide a method for checking if a transaction is open

Problem to Solve

At the moment, there is no straight forward way of checking whether a transaction is still open.

Current Workaround

Run some basic query and see if a Transaction is already closed error is returned.

Proposed Solution

Expose isOpen on transaction

Deploy latest master to our private npm registry

For our other repositories, such as docs and examples to depend on the latest master of client-nodejs in a conventional way, we need client-nodejs to be deployed to our private npm registry on every commit made on the master branch.

Update //client-nodejs:test-integration to work with RBE

@graknlabs_client_nodejs//:test-integration doesn't yet work with RBE. Follow the approach done by @graknlabs_grakn_core//test-end-to-end/client-java:client-java-e2e in order to make it work:

• The test needs to depend on //:distribution so it knows to re-run the test when the distribution is changed in any way.
• Grakn needs to be started and stopped from within the test program, instead of circleci bash commands

Once the test has been made to work with RBE, change the circleci yml such that //client-nodejs:test-integration is executed using the RBE mode:

bazel:
  command: bazel test //:test-integration

Updates based on new changes in 1.5

This issue contains the collection of changes that need to be made in Client Node.js in order to keep it in sync with changes introduced in Grakn 1.5.

• creating a transaction: session.transaction().read();and session.transaction().write();
• instantiation of the Grakn client to be consistent with client Java (@marco-scoppetta @vmax please discuss and clarify this change)
• closing a client: client.close();

Determine which of our direct dependencies pull in the most transitive deps

Problem to Solve

We have thousands of transitive dependencies in our node_modules folder. While this is pretty standard for any JS project, we strongly dislike it and hope there is a way to resolve it.

Proposed Solution

There should be some tool around that can identify which dependencies specified in a package.json lead to the most transitive deps being pulled in.

Node.js build fails on Apple Silicon

Problem to Solve

bazel build //... fails on an M1 Mac due to our use of deprecated rules_nodejs. We should use rules_js instead.

(Even getting rules_js working on an M1 Mac has been fiddly in the past, but at least it is officially supported. rules_nodejs explicitly does not have Apple chip support and will not add it.)

Current Workaround

Build in CI.

Proposed Solution

Migrate to rules_js, taking inspiration from our other projects using it.

Organise Bazel dependencies into //dependencies package like how we do in grakn-core

Defining a rule fails

Description

Adding a rule using the client method(putRule) causes an error, while adding the exact same rule via the console is fine.

Environment

1. OS (where Grakn server runs): Mac OS 10
2. Grakn version (and platform): Grakn.1.5.2
3. Grakn client-nodejs version: client-nodejs 1.5.3
4. Node.js version: v10.15.1

Reproducible Steps

define 

person sub entity,
  plays child,
  plays parent,
  plays grandp,
  plays grandc;

parentship sub relation,
  relates child,
  relates parent;

grandparentship sub relation,
  relates grandp,
  relates grandc;

test sub rule,
when {
(parent: $a, child: $b) isa parentship;
(parent: $b, child: $c) isa parentship;
}, then {
(grandp: $a, grandc: $c) isa grandparentship;
};

transaction.putRule("test", "{(parent: $a, child: $b) isa parentship;(parent: $b, child: $c) isa parentship;}", "{(grandp: $a, grandc: $c) isa grandparentship;)}"

Expected Output

rule defined

Actual Output

error: 2 UNKNOWN: syntax error at line 1: 
{(parent: $a, child: $b) isa parentof; (parent: $b, child: $c) isa parentof;}
                                                                             ^
no viable alternative at input '{(parent: $a, child: $b) isa parentof; (parent: $b, child: $c) isa parentof;}'
syntax error at line 1: 
{(parent: $a, child: $b) isa parentof; (parent: $b, child: $c) isa parentof;}
                                                                             ^
no viable alternative at input '{(parent: $a, child: $b) isa parentof; (parent: $b, child: $c) isa parentof;}'.

Additional notes

@vmax this issue is causing vaticle/typedb-studio#142

Make sure we rename every instance of Relationship to Relation

In tests, external APIs and internal methods.

node_dependencies to be a direct dependency

similar to this issue created on client-python:

At the moment node_dependencies is pulled in from Grakn Core, where they've recently been removed and are not available anymore. In fact, if client-nodejs depends on the latest commit of Grakn master, bazel build //... would break.

This dependency needs to be defined as a direct dependency of client-nodejs.

Use `yarn` instead of `npm` to install dependencies

Problem to Solve

We've experienced quite a few issues with npm already - non-deterministic builds, errors that are a nightmare to debug.

Proposed Solution

Based on my experience building Workbase with yarn, I feel like it's more stable and reliable, and more performant. We used to use yarn for client-nodejs 1.8, so the necessary Bazel tooling should already be there in version control, making this change substantially easier.

Support user token mechanism for faster authentication

Authenticating a user by verifying their password introduces a significant overhead, which comes from the fact that the password is hashed with cryptographic hash function. In effect, opening a new session or transaction becomes much slower.

We need to improve the speed of user-authentication by introducing a mechanism where the verification is only done once when a client connects to the server for the first time.