surrealdb / surrealdb.wasm Goto Github PK

Describe the bug

Instantiating Surreal wasm within the Reactjs part of the Nextjs project causes an error:

Module build failed: UnhandledSchemeError: Reading from "node:crypto" is not handled by plugins (Unhandled scheme).

Steps to reproduce

Create a NextJs app and instantiate the Surreal database within the page.tsx.

"use client"
import {Surreal} from 'surrealdb.wasm';
export default function NameDoesntMatter() {
  prepareSurreal();
  return(<p>demo</p>);
}

async function prepareSurreal() {
  let db  = new Surreal();
  try {
    await db.connect('memory');
    await db.use({database: "db", namespace: "ns"});
  } catch (error) {
    console.log(error)
  }
}

Expected behaviour

I expect it to instantiate the database so that I can use in-memory one, on the browser.

I expect to be able to create a new user and query that user back.

SurrealDB version

surrealdb wasm 0.8.0 ubuntu 22.04.3 LST

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Started following the docs but it seems the package published to npm is empty: https://www.npmjs.com/package/surrealdb.wasm

Is this intended?
Building and running locally works fine.

Describe the bug

Because of an issue in the jsonwebtoken issue, it is currently not possible to build the surrealdb crate to any wasm targets on main

There already is a mostly finished PR on the jsonwebtoken package which solves the issue: Keats/jsonwebtoken#318

Steps to reproduce

Pin rev to latest commit on surrealdb main, then build

Expected behaviour

for wasm targets to build

SurrealDB version

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Description

It seems there are some breaking changes between the V0.8 and V0.7 of the surrealdb.wasm lib:

• the 'use()' method now needs 'namespace' and 'database' instead of 'ns' and 'db'
• the 'query()' method now always returns the results encaspsulated within an array even if there was a single result set (before, if the query was designed as returning a single result set, it would come directly as an array of values in the javascript response)

More generally, it would be appreciated to list the features/changes between versions of this wasm library. It would also be nice to cite which version of the surrealdb engine the wasm library is built on, as some SQL language features might not be available. Last suggestion: the documentation is lacking the structure of the returned values when calling the API.

Finally, the examples are lacking on how to use the library either as clientside-memory or with indexdb persistence. I could pick-up some tricks from some youtube video tutorials, but a bit tough to reach.

I believe this product is great, keep going guys!

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

The default port we use here 8000 coincides with the default surreal start port. This causes a conflict when trying to test one of the remote engines while running the SurrealDB server on the same machine with the default port. Previously we were using 8080 in this repo, which is fine as it avoids the conflict.

Is your feature request related to a problem?

I can see that the typings are generated automagically, but there's a bunch of anys and some function arguments are marked required (as addressed here). It would've been nice to have proper typings for the library

Describe the solution

Explicitly defined TS typings

Alternative methods

Use as is or define custom typings

SurrealDB version

surreal.wasm-v0.7.0

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

I added the following code to the index.html:

const db = new Surreal();

The Chrome browser console throw the following error:

index.js:463 Uncaught TypeError: Cannot read properties of undefined (reading 'surreal_init')
    at new Surreal (index.js:463:26)
    at index.html:15:15

Appreciate the effort to fix the Development Server code as it is important to developers to learn about surrealdb.wasm.

Steps to reproduce

1. Run the development server with npm run serve, you first need to build the wasm module before this step.
2. Add const db = new Surreal(); to index.html.
3. Access http://localhost:8000 from the Chrome browser, you will see the error I reported in developer console of the chrome.

Expected behaviour

No error reported. The db should instantiate successfully.

SurrealDB version

1.2.0

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Is your feature request related to a problem?

Currently, live queries are not supported

Describe the solution

New API that would utilize async generators. It would've been awesome to have it typed

Alternative methods

Use with no live queries, just track updates out of database for local IndexDB cases

SurrealDB version

surrealdb.wasm^0.7.0

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

For a lot of tooling we need a parser.
In https://github.com/surrealdb/surrealdb/blob/main/lib/src/sql/parser.rs#L9 we have a parse function (not 100% sure about what it returns).
I think we should expose it in this package.

(replacement for surrealdb/surrealdb#1203)

I'm happy to help but 0 experience with rust...

Tooling that depends on a parser

• Language-Server
• IDE-Extensions
• ...

Issue: Big O

Something about this in a rust + wasm repo doesn't feel right :)

Solution:

Switch else if chain to match

Too early for PR's on this repo? Do not feel like you need to wait for me on updating/changing this :)

I may be jumping the gun on code audits here...

impl Convert<Value> for JsValue {
	fn convert(self) -> Value {
		if self.is_null() {
			Value::Null
		} else if self.is_undefined() {
		...

Describe the bug

The newly introduced ONLY statement indicates that the result is a single result.
The result should not be wrapped into an extra array.

Steps to reproduce

--- without ONLY
CREATE test:1 SET some='value'

returns

[
    {
        "id": "test:1",
        "some": "value"
    }
]

--- with ONLY
CREATE ONLY test:2 SET some='value'

returns

[
    {
        "id": "test:2",
        "some": "value"
    }
]

Expected behaviour

The behaviour should be the same as on the regular server

--- with ONLY
CREATE ONLY test:2 SET some='value'

returns

{
    "id": "test:2",
    "some": "value"
}

Same for SELECT statement.

SurrealDB version

local build from current master - surrealdb to beta.11 - on macOS

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

I wanted to try and play around with the crate from within another WASM rust crate, but I can't import anything because the crate isn't being compiled as a rust library.

Currently:

[lib]
crate-type = ["cdylib"]

Could be changed to:

[lib]
crate-type = ["cdylib", "rlib"]

The typings for functions like .connect(), .query() and more currently indicate that the second argument is required, whilst it technically isn't.

Is your feature request related to a problem?

The original issue at surrealdb/surrealdb#2332

Describe the solution

Refer to the original issue.

Alternative methods

Refer to the original issue.

SurrealDB version

1.1.1

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

I'd like to help out with this library as WASM, and Rust in general is my native tongue. Perhaps adding a list of tasks that need to be done, or throwing up issues, even internal ones, so people like myself can take a swing at them, would be great!

Is your feature request related to a problem?

make it easy for non rust users to try this out.

Describe the solution

Can we have a .wasm release out of GitHub workflow

https://github.com/surrealdb/surrealdb.wasm/releases

then we can build up usage examples for JS or typescript devs using it.

Alternative methods

not applicable....

SurrealDB version

all

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

The db.select() and db.delete() throw the error above when the given id is not found.

Steps to reproduce

1. Add the following code to the development server, index.html:

     const db = new Surreal();
     await db.connect("indxdb://test");
     await db.use({ namespace: "test", database: "test" });
     await db.select("person:1");

2. Run the development server with command npm run serve.
3. Access the http://localhost:8000 and you should see the error in developer console of Chrome browser.

Expected behaviour

It should return null or [].

SurrealDB version

1.2.0 and 1.3.0

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

Error: There was a problem with a datastore transaction: An IndexedDB error occured: failed to
execute indexed db request: TransactionInactiveError: Failed to execute 'get' on 'IDBObjectStore':
The transaction is not active.

The error being threw when executing the following statement:

await db.query('UPDATE ONLY main_chats:0000danjtr1rdkh233rjgrk08s MERGE {"name":"This virtual assistant","created":"2024-01-18T03:43:37.285Z","updated":"2024-01-18T03:43:49.715Z"}');

Steps to reproduce

It is hard to re-produce as the issue is coming out irregularly.

Expected behaviour

The SurrealQL statement should be executed successfully

SurrealDB version

surreal 1.0.0

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

After running compilation on rust level, that can be ivoced also by npm run build command after downloading sources from git, I get:
`...
Compiling cipher v0.4.4
error: failed to build archive: 'wasm32.o': section too large

The following warnings were emitted during compilation:

warning: warning: /Library/Developer/CommandLineTools/usr/bin/ranlib: archive library: /Users/(...)/surrealdb.wasm/target/wasm32-unknown-unknown/release/build/psm-e7fd6e39c2c90a79/out/libpsm_s.a the table of contents is empty (no object file members in the library define global symbols)

error: could not compile psm (lib) due to previous error
warning: build failed, waiting for other jobs to finish...
Error: Compiling your crate to WebAssembly failed
Caused by: Compiling your crate to WebAssembly failed
Caused by: failed to execute cargo build: exited with exit status: 101
full command: cd "/Users/pedropuchalski/Documents/praca/FirmaNext/sceneMgm/surrealdb.wasm" && "cargo" "build" "--lib" "--release" "--target" "wasm32-unknown-unknown" "--no-default-features" "--features" "protocol-ws,protocol-http,kv-indxdb,kv-mem,rustls"`

Steps to reproduce

Try to compile on Apple MacBook Air with M1, Ventura.

Expected behaviour

Compiled *.wasm file.

SurrealDB version

"1.0.0", default-features = false

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Is your feature request related to a problem?

The surrealdb.js uses this code to connect:

	// Signin as a namespace, database, or root user
	await db.signin({
		user: "root",
		pass: "root",
	});

	// Select a specific namespace / database
	await db.use({ ns: "test", db: "test" });

but the surrealdb.wasm uses this code to connect:

	// Signin as a namespace, database, or root user
	await db.signin({
		username: "root",
		password: "root",
	});

	// Select a specific namespace / database
	await db.use({ namespace: "test", database: "test" });

Why not use the same params for closely matches the API of the Javascript library ?

Describe the solution

Closely matches the API of the Javascript library using the same params to connect.

Alternative methods

Use the same params for closely matches the API of the Javascript library ?

SurrealDB version

1.0.0+20231002.920cdfc for linux on x86_64

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Example

True for both statements:

CREATE chat CONTENT { customer: "guest:ht28nge5czf39lrs39ox" };
CREATE chat SET customer = "guest:ht28nge5czf39lrs39ox"

The current querying result return a MAP objects.
It's better to return the results in JSON format to match the API of the Javascript library.

I have run into several issues trying to use this package in an Electron environment, so I have now created a minimal Vue.js app with Vite to see if the issues are Electron related.

I am now encountering a different issue.

I have the following file, based on the example in the README:

import { Surreal } from 'surrealdb.wasm';

const db = new Surreal();

try {
    // Connect to the database
    await db.connect("indxdb");
} catch (e) {
    console.error("ERROR", e);
}

export default db

In my entry point file I have:

import { createApp } from 'vue'
import db from './db'
import './style.css'
import App from './App.vue'

createApp(App).mount('#app')

Running yarn dev I get a number of errors relating to top level await being used in surrealdb.wasm/dist/full/index.js

I'm trying to import the WS module, however the imports are listed properly not sure if this is a package issue with the files it exports, tsconfig or something on my end

But I expect it to show: "/ws" and "/http" and not dist and lib

on version 0.6.0

Is your feature request related to a problem?

/

Describe the solution

/

Alternative methods

/

SurrealDB version

0.6.0 for Linux

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Is your feature request related to a problem?

Indexeddb is known to be slow and limited. And, when surreal is an abstraction on top of it, it has to go through many levels of abstractions - surreal.wasm > js > indexeddb > leveldb/sqlite.

(see here for info about how chromium currently uses leveldb - which rocksdb evolved out of - as the backing store for indexeddb, but are changing it to sqlite https://issues.chromium.org/issues/40253999)

Describe the solution

The Origin Private File System (OPFS) is a new-ish browser api that allows for high performance file system access. https://developer.mozilla.org/en-US/docs/Web/API/File_System_API/Origin_private_file_system

It has support of all major browsers (look at the opfs-specific footnotes) https://caniuse.com/native-filesystem-api

A high profile example of it being used is SQLite Wasm, which seemingly allows for full sqlite usage in the browser. https://developer.chrome.com/blog/sqlite-wasm-in-the-browser-backed-by-the-origin-private-file-system/

Presumably surreal's rocksdb, speedb, and tikv backends could be used in a similar manner, which would surely simplify and improve performance for your wasm efforts. Likewise surrealkv when that is ready.

Alternative methods

SQLite wasm is getting a lot of attention and would probably be the easiest and most stable option. But it doesn't appear that Surreal uses SQLite as a storage backend, so there's probably no point in looking into it.

SurrealDB version

N/a

Contact Details

Reply to this issue

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct