GithubHelp home page GithubHelp logo

kwil-js's Introduction

Kwil

Kwil-JS is a JavaScript/Typescript SDK for building browser and NodeJS applications to interact with Kwil networks.

Versioning

Make sure to use the correct version of the Kwil-JS SDK for the version of the Kwil-DB you are using:

Kwil-JS Version Kwil-DB Version
v0.7 v0.8
v0.6 v.0.7 only
v0.5 v0.6 & v0.7

Installation

npm i @kwilteam/kwil-js

Initialization

Configure your NodeKwil or WebKwil class by providing the required configurations and any optional configurations.

Web

import { BrowserProvider } from 'ethers';
import { WebKwil } from '@kwilteam/kwil-js';

// to be used for funding and signing transactions
const provider = new BrowserProvider(window.ethereum)

const kwil = new WebKwil({
    kwilProvider: "kwil_provider_endpoint",
    chainId: "your_kwil_chain_id"
});

NodeJS

const { Wallet } = require('ethers');
const kwiljs = require('@kwilteam/kwil-js');

// to be used for signing transactions
// instead of a provider, nodeJS requires a wallet
const wallet = new Wallet("my_ethereum_private_key")

const kwil = new kwiljs.NodeKwil({
    kwilProvider: "kwil_provider_endpoint",
    chainId: "your_kwil_chain_id"
});

Identifiers

Account Identifiers

In Kwil, accounts are identified by the signer(s) that are used on the Kwil Network. Kwil natively supports two types of signers: Secp256k1 (EVM) and ED25519.

Secp256k1 signers use Ethereum wallet addresses as identifiers. ED25519 signers use the ED25519 public key as identifiers.

Database Identifiers (DBID)

In Kwil, databases are identified by a 'database identifier' (sometime referred to as DBID), which is a hex encoded SHA224 Hash of the database name and public key, prepended with an x.

The account identifier can be passed as a hex-encoded string, or as Bytes (Uint8Array).

To get the DBID for an account identifier and database name, you can use the following helper method:

import { Utils } from "@kwilteam/kwil-js";

const dbid = Utils.generateDBID('account_identifier', 'database_name')

Signers

Certain operations in Kwil require signature authentication from the user (e.g. deploy database, drop database, execute CRUD actions, etc).

To manage signing, Kwil-JS uses a KwilSigner class. Out of the box, Kwil-JS supports signers from EthersJS (v5 and v6). You can also pass a signing callback function (see below).

The account identifier can be passed as a hex string or as bytes.

import { Utils, KwilSigner } from '@kwilteam/kwil-js';
import { BrowserProvider } from 'ethers';

// get ethers signer
const provider = new BrowserProvider(window.ethereum)
const signer = await provider.getSigner();

// get ethereum address
const identifier = await signer.getAddress();

// create kwil signer
const kwilSigner = new KwilSigner(signer, identifier);

Custom Signers

If you wish to sign with something other than an EtherJS signer, you may pass a callback function that accepts and returns a Uint8Array() and the enumerator for the signature type used.

Currently, Kwil supports two signature types:

Type Enumerator Identifier Description
Secp256k1 'secp256k1_ep' Ethereum Wallet Address The Kwil Signer will use a secp256k1 elliptic curve signature.
ED25519 'ed25519' ED25519 Public Key The Kwil Signer will use an ED25519 signature.

To use an ED25519 signature:

import nacl from 'tweetnacl';
import { KwilSigner } from '@kwilteam/kwil-js';

// create keypair and signer
const keys = nacl.sign.keyPair();
const customSigner = (msg) => nacl.sign.detached(msg, keys.secretKey);
const identifier = keys.publicKey;

const kwilSigner = new KwilSigner(customSigner, identifier, 'ed25519');

Database Queries

Create, Update, Delete (CUD) Actions

Any action that executes a CUD operation must be signed and broadcasted to the network through the kwil.execute() method.

.execute() takes an object that matches the ActionBody interface. Action body has two required fields: dbid and action. You can also optionally add an inputs field if the action requires inputs, and a description field to customize the signature message.

import { Utils } from '@kwilteam/kwil-js'

// begin constructing the values for the action
const input = new Utils.ActionInput()
    .put("input_name_1", "input_value_1")
    .put("input_name_2", "input_value_2")
    .put("input_name_3", "input_value_3")

// get database ID
const dbid = kwil.getDBID("account_identifier", "database_name")

const actionBody = {
    dbid,
    name: "your_action_name",
    inputs: [ input ],
    description: "Click sign to execute the action!"
}

// pass action body and signer to execute method
const res = await kwil.execute(actionBody, kwilSigner)

/*
    res.data = {
        tx_hash: "hash",
    }
*/

Reading Data

To read data on Kwil, you can (1) call a view action or (2) query with the .selectQuery() method.

View Action Message

View actions are read-only actions that can be used to query data without having to wait for a transaction to be mined on Kwil.

Only one input is allowed for view actions.

To execute a view action, pass an ActionBody object to the kwil.call() method.

import { Utils } from '@kwilteam/kwil-js'

// begin constructing the values for the action
const input = new Utils.ActionInput()
    .put("input_name_1", "input_value_1")
    .put("input_name_2", "input_value_2")
    .put("input_name_3", "input_value_3")

// retrieve database ID to locate action
const dbid = kwil.getDBID("account_identifier", "database_name")

const actionBody = {
    dbid,
    name: "your_action_name",
    inputs: [ input ]
}

// pass action body to execute method
const res = await kwil.call(actionBody)

/*
    res.data = {
        result: [ query results ],
    }
*/

If the view action uses a @caller contextual variable, you should also pass the kwilSigner to the kwil.call() method. This will allow the view action to access the caller's account identifier. Note that the user does not need to sign for view actions.

await kwil.call(actionBody, kwilSigner)

Select Query

You may also query any of the database data by calling the kwil.selectQuery() method. Note that this can only be used for read-only queries

const dbid = kwil.getDBID("account_identifier", "database_name")
const res = await kwil.selectQuery(dbid, "SELECT * FROM users")

/*
    res.data = [
        ...
    ]
*/

Network Info

ChainID and Status

To verify that you are using the correct chainId, as well as the latest block height and block hash on your chain, you can call the .chainInfo() method.

const res = await kwil.chainInfo()

/*
    res.data = {
        chain_id: "your_chain_id",
        height: "latest_block_height",
        hash: "latest_block_hash"
    }
*/

List Databases

To see databases that are deployed on the Kwil network, you can call the .listDatabases() method. You can optionally pass an account identifier to see only the databases that the account owns.

const res = await kwil.listDatabases("account_identifier (optional)")
// res.data = ["db1", "db2", "db3"]

Get Schema

You can retrieve database information by calling .getSchema() and passing the dbid. Note that the database owner is returned as a Uint8Array.

const dbid = kwil.getDBID("owner_wallet_address", "database_name")
const schema = await kwil.getSchema(dbid)

/*
    schema.data = {
        owner: Uint8Array,
        name: "database_name",
        tables: [ tableObject1, tableObject2, tableObject3 ],
        actions: [ action1, action2, action3 ],
        extensions: [ extension1, extension2 ]
    }
*/

Get Account

You can get the remaining balance of an account and the account's nonce by using the .getAccount() method. .getAccount() takes an account identifier, either in hex format or bytes (Uint8Array).

const res = await kwil.getAccount("account_identifier")

/*
    res.data = {
        identifier: Uint8Array,
        balance: "some_balance",
        nonce: "some_nonce"
    }
*/

Kwil Gateway Authentication

Kwil Gateway is an optional service on Kwil networks that allows for authenticating users with their signatures for read queries. If your Kwil network used a Kwil Gateway, you can use the kwil.authenticate() method to authenticate users. Note the additional step of passing the cookie back to the kwil class in NodeJS.

Web

// pass the kwilSigner to the authenticate method
const res = await kwil.authenticate(kwilSigner);

/*
    res = {
        status: 200,
        data: {
            result: "success"
        }
    }
*/

NodeJS

// pass the kwilSigner to the authenticate method
const res = await kwil.authenticate(kwilSigner);

/*
    res = {
        status: 200,
        data: {
            result: "success",
            cookie: "some_cookie"
        },
        
    }
*/

// pass the cookie to the kwil class
kwil.setCookie(res.data.cookie)

Database Building

Although you can deploy new databases with the JS-SDK, we strongly recommend using the Kwil Kuneiform IDE or Kwil CLI to manage the entire database deployment process.

To deploy a new database, first define your syntax in the Kuneiform IDE. You can learn more about the syntax rules here.

Once the syntax is ready, click "Compile". Right click your compiled files and click "Export to JSON".

Import your JSON to your Javascript project.

// import and call database JSON
import myDB from "./myDB.json";

// construct DeloyBody object
const deployBody = {
    schema: myDB,
    description: "Sign to deploy your database"
}

// send to kwil.deploy()
const res = await kwil.deploy(deployBody, kwilSigner);

/*
    res = {
        status: 200,
        data: {
            tx_hash: "some_hash"
        }
    }
*/

Kwil Gateway Authentication

Kwil Gateway is an optional service on Kwil networks that allows for authenticating users with their signatures for read queries / view actions. If your Kwil network used a Kwil Gateway, you should pass a KwilSigner to the kwil.call() method. If the user is not authenticated, the user will be prompted to sign a message to authenticate, and the SDK will automatically include the authentication cookie in each subsequent request.

// pass KwilSigner to the call method
const res = await kwil.call(actionBody, kwilSigner);

/*
    res.data = {
        result: [ query results ],
    }
*/

kwil-js's People

Contributors

kwilluke avatar brennanjl avatar randalf-sr avatar

Stargazers

 avatar chris lee avatar Jose Aguinaga avatar Gordon avatar  avatar  avatar

kwil-js's Issues

Remove ed25519_nr signature type

shae256 hashed ed25519 signatures (near variant) will not be supported out of the box in the next public release. It needs to be removed from the current SignatureTypes enum list.

Allow for custom signer names

Refactor the Signature Type enumerator to allow developer to specify custom signer names when they implement it in KwilDB.

Remove 0x prefix from most hex strings

Right now, our xToHex() functions prefix any hex value with 0x. Unless it is an ethereum public key, address, or tx hash, it should not have the 0x prefix.

Deprecate external API for DBBuilder and ActionBuilder

Overview

With #32 completed, we need to deprecate the DBBuilder and ActionBuilder external APIs.

The builders will still be used internally, but just keeping the external APIs as what is described in #32 will give us more flexibility as the Kwil tx payload continues to evolve.

Timeline

  1. Public release - +3 months: Add TSDOC deprecation notice, message partners.
  2. => 3 months from public release: Remove external API

Have a way to not produce a warning when explicitly passing in a placeholder value for `chainId` discovery

Context

In order to not burden our users with having to configure chainId explicitly (especially since they use a bunch of env), we do this little dance:

  static async init({
    nodeUrl = KwilWrapper.defaults.kwilProvider,
    dbId = KwilWrapper.defaults.dbId
  }) {
    console.warn("Discovering chainId. This might generate a warning.");
    const kwil = new WebKwil({ kwilProvider: nodeUrl, chainId: "" });
    const chainId = (await kwil.chainInfo()).data?.chain_id ?? KwilWrapper.defaults.chainId;

    // This assumes that nobody else created a db named "idos".
    // Given we intend to not let db creation open, that's a safe enough assumption.
    const dbId_ =
      (await kwil.listDatabases()).data?.filter(({ name }) => name === "idos")[0].dbid || dbId;

    return new KwilWrapper({ nodeUrl, dbId: dbId_, chainId });
  }

However, we're passing in "" for chainId on this throw-away WebKwil, and that make it (rightly so) complain. As far as I can tell the warning comes from: https://github.com/idos-network/kwil-js/blob/c12d441354af6521a8f71669e9851e01422caadb/src/client/kwil.ts#L348

Having this warning come from kwil-js makes us have to add our own warning to assuage people who look into the console. ๐Ÿ˜…

Desired behavior

Have a way to signal to kwil-js that "I don't want to be pinned to a specific chainId, and I just want to use whatever is on the provider". This removes the burden for folks who don't care about this specific assurance, and allows folks (like us) that want to discover the chainId with a first call to the provider.

Suggested implementation

Assigning the empty string this specific intention. That would lead to this code:

  public async chainInfo(): Promise<GenericResponse<ChainInfo>> {
    const info = await this.client.chainInfo();

    if (this.chainId !== "" && info.data?.chain_id !== this.chainId) {
      console.warn(
        `WARNING: Chain ID mismatch. Expected ${info.data?.chain_id}, got ${this.chainId}`
      );
    }

    return info;
  }

Managing Ethers dependencies to impact bundle size

In the Ethereum js community, there is still a divide in popular libraries not migrating to ethers v6 (example here).

This is relevant for Kwil because our signer must be an ethers signer, but the signer classes slightly changed between ethers v5 and ethers v6. Thus, we have to have support for both.

In our SDK, when a user passes a signer, we check whether it is the v5 or v6 signer and then send it to the appropriate section that accounts for the details (see here). The only issue with this implementation is that it requires us to use two versions of ethers as a dependency. Ethers is a fairly large package, so including it twice increases the cost of using Kwil as a dependency (see https://bundlephobia.com/package/[email protected]).

We need to explore if there is a way (potentially using peer dependencies) to check if the user already has ethers v6 installed, and if they do, then not require them to also download ethers v5. We use some utils functions internally with ethers v6, so if an application using our sdk also uses ethers v5, then they will need ethers v5 and v6. But right now, every node_modules folder that installs kwil has two copies of ethers (and if they use v5, an additional third copy of ethers v6 in the kwil/node_modules folder because of how npm manages conflicts). It feels like there should be a better way of approaching this.

Problem: `NodeKwil` does not respect logout

Although the .auth.logout() API exists on NodeKwil, it does not actually remove the cookie on each subsequent request.

To close this issue, we need to:

  1. Stop passing the cookie on call requests after .logout() is called.
  2. Allow the developer to pass the cookie in NodeKwil request

Add validation for builders

When calling .buildx() it should give a typescript validation for if the required methods have been provided.

Proposal: Builder Pattern Alternatives

Below is a proposed approach for how the public APIs in Kwil-JS can look to move away from the Builder Patterns and more closely resemble what is available in packages like EtherJS and Viem.

1. Create KwilSigner Class

Create a Kwil Signer Class that is used to signing transactions and call requests:

const mySigner = new KwilSigner(
     publicKey: string, 
     signer: EthersSigner | (msg: Uint8Array) => Uint8Array, 
     signatureType?: string
)

signatureType is required to be specified if it does not receive an EtherJS signer.

2. Create four new methods on the main kwil class

Currently, the entry point for KwilJS is with the WebKwil or NodeKwil class:

e.g.

const kwil = new WebKwil({
   kwilProvider: "https://provider.kwil.com",
   chainId: "your_chain_id",
});

Action Operations - kwil.call() and kwil.execute()

For action-related operations (state-changing and view actions), there will be two methods:

  • kwil.call(actionPayload: ActionInterface, signer?: KwilSigner) - Executes view actions. Signer only required if the action has must_sign.
  • kwil.execute(actionPayload: ActionInterface, signer: KwilSigner) - Executes mutative actions. Signer is required.

TheActionInterface for both methods will look like this:

interface ActionInterface {
     dbid: string,
     action: string,
     inputs?: Array<objects> | Array<ActionInput>,
     description?: string
}

Database Operations - kwil.deploy() and kwil.drop()

For deploying and dropping databases, there will be two methods:

  • kwil.deploy(deployPayload: DeployInterface, signer: KwilSigner) - Used to deploy a database.
  • kwil.drop(dropPayload: DropInterface, signer: KwilSigner) - Used to drop a database.

With the following respective interfaces:

interface DeployInterface {
     schema: object<CompiledKuneiform>,
     description?: string
}

interface DropInterface {
     dbid: string,
     description?: string
}

Add stateful unconfirmed nonce tracking to account requests

In the get account query string, the user can optionally give ?status=1 to return the uncofirmed nonce. This will resolve the invalid nonce issue that lots of SDK users receive.

In the NodeKwil or WebKwil config, there will be a new optional unconfirmedNonce parameter. If set to true, getAccount will return with the unconfirmed nonce. If set to false or not set, it will return the confirmed nonce only.

See more here: https://github.com/kwilteam/proto/blob/1011fcf71d39f980e9963fa949cbc0ff57c78447/kwil/tx/v1/account.proto#L19C14-L19C27

Replace cache with a third party library

The cache that is implemented in this SDK is too error prone and not worth maintaining.

We should implement either:

  • An isomorphic cache library (or a nodejs cache library with the appropriate polyfills, but that is not ideal because it forces polyfills onto the consumers of the SDK) - candidates include node-cache and memory-cache
  • Use a nodeJS cache in nodeKwil and use localstorage for webKwil - this may require some more refactoring, but may be the better solution if there is not a good isomorphic cache library.

Or, we can just remove the cache! It only helps make less calls to .getSchema() when building transactions (we call the schema to validate that the required inputs are present). That seems like an overoptimization given the scale at which kwil is currently being used.

Limit characters allowed in friendly sign

To prevent unnecessarily long signature messages that take up block space, user-passed signature messages should be no longer than 200 characters (applies to both tx's and msg's).

add cache for when getSchema is called

Each time getSchema is called (e.g. when actions are used), the result should be cached. When a user starts a new session, the cache should be cleared.

force action name to lowercase

in kwil, all action names are set to lowercase, regardless of casing when they are defined. when the user passes the action name to the actionBuilder, we should also set it to lowercase so it is case insensitive.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.