hirofa / quickjs_es_runtime Goto Github PK

this is a wrapper library for the javascript runtime quickjs written in rust which works with typescript, modules, promises, async, await and much more

Home Page: https://github.com/HiRoFa/quickjs_es_runtime

License: MIT License

Rust 99.88% Shell 0.12%

async ecmascript es6 javascript modules promises quickjs rust typescript

quickjs_es_runtime's Issues

compile test on 32 bits

if thats really thre problem, running on raspberry pi gives segfaults

test cases, not just in doc

a lot of methods are now just tested in docs and not in a mod tests, in order to test them all wait valgrind we need them also in a test mod

unit tests for every method in utils

fix tests (free runtime issue)

32bit tests in github actions

if thats really the problem, (running on raspberry pi gives segfaults)

JSValueRef revamp

revert back to the consume_value method for calling quickjs methods which take ownership insterad of just doing *borrow_value()

EsValueFacade::invoke_function_batch(_sync)(Vec<Vec<EsValueFacade>>)

performance thing when using stuff like mysql feature in esseslib-q

Leaked promise inside a dropped context causes panic!

Hi there!,
Please check below full code of main.rs with inline comments to reproduce the issue.

In summary, when a js code, executed using add_rt_task_to_event_loop to run within a context, leaks a promise. quickjs_runtime panics when the leaked promise resolves as the context was already dropped.

main.rs

use futures::executor::block_on;
use hirofa_utils::js_utils::{JsError, Script};
use log::LevelFilter;
use quickjs_runtime::esruntime::EsRuntime;
use quickjs_runtime::esruntimebuilder::EsRuntimeBuilder;
use quickjs_runtime::esvalue::EsValueFacade;
use std::sync::Arc;
use std::thread;
use std::time::Duration;

async fn test(rt: Arc<EsRuntime>) -> anyhow::Result<(), JsError> {
    // create context
    rt.create_context("abc")?;
    let result = rt
        .add_rt_task_to_event_loop(move |q_js_rt| {
            // use the above created context to run the eval
            let q_ctx = q_js_rt.get_context("abc");
            let res = q_ctx.eval(Script::new(
                "basics.es",
                r#"
            // the below promise is leaked
            new Promise((resolve) => {
                setTimeout(() => {
                    console.log("leaked promise resolved...");
                    resolve("done done");
                }, 1000);
            });
            // the below promise is returned
            new Promise((resolve) => {
                setTimeout(() => {
                    console.log("promise resolved...");
                    resolve("done");
                }, 200); // increase this time to 2s to see the leaked promise resolved first and hence doesn't crash
            });
        "#,
            ));

            match res {
                Ok(res) => EsValueFacade::from_jsval(q_ctx, &res),
                Err(e) => {
                    log::error!("{}", e);
                    Err(e)
                }
            }
        })
        .await;

    match result {
        Ok(res) => {
            // resolve the returned promise
            let fut = res.get_promise_result();
            let val = fut.await;
            log::info!("{:?}", val);
        }
        Err(e) => {
            log::error!("{}", e);
        }
    };

    // drop the context as the js execution is complete
    rt.drop_context("abc");
    log::info!("context dropped...");

    // wait or do something else...
    // at this point... the leaked promise get's resolved and panics with 'no such context'

    thread::sleep(Duration::from_secs(5)); // if this line is commented then it doesn't panic as the thread get's dropped here

    Ok(())
}

fn main() {
    simple_logging::log_to_stderr(LevelFilter::Info);
    let rt = EsRuntimeBuilder::new().build();
    match block_on(test(rt)) {
        Ok(_) => {}
        Err(e) => {
            log::error!("{}", e);
        }
    }
    thread::sleep(Duration::from_secs(5));
}

Error

thread '<unnamed>' panicked at 'no such context', /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjsruntime.rs:308:31
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:515:5
   1: core::panicking::panic_fmt
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/panicking.rs:92:14
   2: core::option::expect_failed
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/option.rs:1243:5
   3: core::option::Option<T>::expect
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/option.rs:351:21
   4: quickjs_runtime::quickjsruntime::QuickJsRuntime::get_context
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjsruntime.rs:308:9
   5: quickjs_runtime::features::set_timeout::set_timeout::{{closure}}::{{closure}}::{{closure}}
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/features/set_timeout.rs:90:33
   6: quickjs_runtime::quickjsruntime::QuickJsRuntime::do_with::{{closure}}
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjsruntime.rs:424:13
   7: std::thread::local::LocalKey<T>::try_with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:400:16
   8: std::thread::local::LocalKey<T>::with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:376:9
   9: quickjs_runtime::quickjsruntime::QuickJsRuntime::do_with
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjsruntime.rs:422:9
  10: quickjs_runtime::features::set_timeout::set_timeout::{{closure}}::{{closure}}
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/features/set_timeout.rs:87:17
  11: core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ops/function.rs:227:5
  12: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/alloc/src/boxed.rs:1575:9
  13: hirofa_utils::eventloop::EventLoop::run_timeouts_and_intervals
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/hirofa_utils-0.2.1/src/eventloop.rs:134:13
  14: hirofa_utils::eventloop::EventLoop::new::{{closure}}::{{closure}}
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/hirofa_utils-0.2.1/src/eventloop.rs:97:37
  15: std::thread::local::LocalKey<T>::try_with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:400:16
  16: std::thread::local::LocalKey<T>::with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:376:9
  17: hirofa_utils::eventloop::EventLoop::new::{{closure}}
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/hirofa_utils-0.2.1/src/eventloop.rs:71:13
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread '<unnamed>' panicked at 'cannot access a Thread Local Storage value during or after destruction: AccessError', /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:376:26
stack backtrace:
   0: rust_begin_unwind
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/panicking.rs:515:5
   1: core::panicking::panic_fmt
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/panicking.rs:92:14
   2: core::result::unwrap_failed
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/result.rs:1355:5
   3: core::result::Result<T,E>::expect
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/result.rs:997:23
   4: std::thread::local::LocalKey<T>::with
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:376:9
   5: <quickjs_runtime::quickjscontext::QuickJsContext as core::ops::drop::Drop>::drop
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjscontext.rs:298:13
   6: core::ptr::drop_in_place<quickjs_runtime::quickjscontext::QuickJsContext>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
   7: core::ptr::drop_in_place<(alloc::string::String,quickjs_runtime::quickjscontext::QuickJsContext)>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
   8: core::ptr::mut_ptr::<impl *mut T>::drop_in_place
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mut_ptr.rs:995:18
   9: hashbrown::raw::Bucket<T>::drop
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.11.0/src/raw/mod.rs:344:9
  10: hashbrown::raw::RawTable<T,A>::drop_elements
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.11.0/src/raw/mod.rs:598:17
  11: hashbrown::raw::RawTable<T,A>::clear
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.11.0/src/raw/mod.rs:591:13
  12: hashbrown::map::HashMap<K,V,S,A>::clear
             at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/hashbrown-0.11.0/src/map.rs:774:9
  13: std::collections::hash::map::HashMap<K,V,S>::clear
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/collections/hash/map.rs:566:9
  14: <quickjs_runtime::quickjsruntime::QuickJsRuntime as core::ops::drop::Drop>::drop
             at /Users/x/.cargo/registry/src/github.com-1ecc6299db9ec823/quickjs_runtime-0.5.1/src/quickjsruntime.rs:507:9
  15: core::ptr::drop_in_place<quickjs_runtime::quickjsruntime::QuickJsRuntime>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
  16: core::ptr::drop_in_place<core::option::Option<quickjs_runtime::quickjsruntime::QuickJsRuntime>>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
  17: core::ptr::drop_in_place<core::cell::UnsafeCell<core::option::Option<quickjs_runtime::quickjsruntime::QuickJsRuntime>>>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
  18: core::ptr::drop_in_place<core::cell::RefCell<core::option::Option<quickjs_runtime::quickjsruntime::QuickJsRuntime>>>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
  19: core::ptr::drop_in_place<core::option::Option<core::cell::RefCell<core::option::Option<quickjs_runtime::quickjsruntime::QuickJsRuntime>>>>
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/ptr/mod.rs:192:1
  20: core::mem::drop
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/core/src/mem/mod.rs:889:24
  21: std::thread::local::fast::destroy_value
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/thread/local.rs:656:13
  22: std::sys::unix::thread_local_dtor::register_dtor::run_dtors
             at /rustc/a178d0322ce20e33eac124758e837cbd80a6f633/library/std/src/sys/unix/thread_local_dtor.rs:89:17
  23: <unknown>
  24: __pthread_tsd_cleanup
  25: __pthread_exit
  26: __pthread_start

script preprocessor support

For

Macro's
Ifdef
Ts
Etc

rename EsRuntime(Builder) to QuickjsRuntimeFacade/Builder

and QuickjsContext/Runtime to *Adapter

rebuild EsValueFacade to enum of different types

Instead of is_str type functions

add InteruptHandler to EsRuntimeBuilder

uses JS_SetInteruptHandler

32bit support / windows

async/await support

EsValueFacade.promise_future()
EsValueFacade.invoke_function_future()

start from EsRuntime.add_task_future()

these are all added as extra func so current fire_and_forget and /sync version may still exist

todo:

EventQueue.asyncTask
- EventQueue.asyncTask -> impl with waker()
EsRuntime methods
EsValueFacade.invoke_function
- EsValueFacadeFuture -> impl with waker()
EsValueFacade.get_promise_result() (also rename get promise_result_blocking to get promise_result_sync

Proxy+ProxyInstance .toString

refactor obj handling in EsValueFacade

make it a 'live' link just like Function and Promise, and only serialize to Map if get_object is called, then add methods like get_property / has_property etc

revamp all utils to use *mut JSContext instead of q_js_rt

How about event loop?

console

full support as in es_runtime

Ability to load pre-compiled modules

Hi there!
I'm able to pre-compile a JS/TS function and save as bytecode and later execute using run_compiled_function.

I'm looking for a similar option for pre-compiling some modules (e.g. lodash, date-fns etc) to avoid loading and compiling on every invocation.

May be ByteCodeModuleLoaderAdapter that will be almost similar to ScriptModuleLoaderAdapter except for the below line.

quickjs_es_runtime/src/quickjsruntimeadapter.rs

Line 83 in 0225d27

let compiled_module = unsafe { compile_module(q_ctx.context, script)? };

Any thoughts?

timeouts on invocations like call_function and eval

for theduke/quickjs-rs#4

i think i want to impl this a bit broader (basicly in all public methods of JSRealmAdapter) so you can allways recover from endless loops and such...

Also i want to reimpl timeouts for get_promise_result (which was removed because it was difficult to impl with Futures)

EsFunction struct for creating functions as EsValueFacade

use code from EsRuntime::set_function

and replace that with EsFunction::new()

also needed for simple proxy creation for fs module

native module utils

be able to add native modules on demand

e.g.

import("hirofa.http").then((http) => {
    // http is a native module here
    return http.get("www.google.com");
}).then((result) => {
    console.log("got %1", result.textContent);
});

EventLoop revamp

use https://docs.rs/futures/0.3.13/futures/executor/struct.LocalPool.html

use an extra scheduling thread for setTimeout/setInterval

revamp EsValueFacade

based on utils::JsValueFacade just like StarlightRuntime

replace helper_thread_pool in EsRuntime with tokio

so we can do .await in promise resolvers and such

bigint and date support

mostryly to get rid of the copypasted cfg stuff

module loading gc workings unclear

currently in modules.rs we have LOADED_MODULE_REGISTRY in which we check to see if a module is loaded..

can this be replaced by using q::JS_DetectModule ?

when are modules garbage collected?

e.g. does this even work?

eval: {import("test.mes").then((module) => {
    console.log(module.foo);
})}
gc();
eval: {import("test.mes").then((module) => {
    console.log(module.foo);
})}

addEventListener from rust
removeEventListener from rust
dispatchEvent from rust

test todos

removeEventListener
dispatch from javascript
addListener from rust

panicked at 'already borrowed: BorrowMutError'

Code to reproduce the issue

[dependencies]
quickjs_runtime = "0.4.2"
futures = "0.3"
log = "0.4.11"
simple-logging = "2.0.2"

use futures::executor::block_on;
use log::LevelFilter;
use quickjs_runtime::esruntime::EsRuntime;
use quickjs_runtime::esruntimebuilder::EsRuntimeBuilder;
use quickjs_runtime::esscript::EsScript;
use std::sync::Arc;

async fn test(rt: Arc<EsRuntime>) {
    let res = rt
        .eval(EsScript::new(
            "basics.es",
            r#"
            new Promise((resolve) => {
                setTimeout(() => {
                    console.log('in timeout 1');
                    setTimeout(() => {  // panic
                        console.log('in timeout 2');
                        resolve(true);
                    }, 1000)
                }, 1000);
            });
        "#,
        ))
        .await
        .ok()
        .unwrap();

    let fut = res.get_promise_result();
    let val = fut.await;
    println!("{:?}", val);
}

fn main() {
    simple_logging::log_to_stderr(LevelFilter::Info);
    let rt = EsRuntimeBuilder::new().build();

    block_on(test(rt));
}

Other example js that causes panic

new Promise((resolve) => {
    const timeout = setTimeout(() => {
        console.log('in timeout 1');
        clearTimeout(timeout);  // panic
        resolve(true);
    }, 1000);
});

new Promise((resolve) => {
    const timeout = setTimeout(() => {
        console.log('in timeout 1');
    }, 10000);
    setTimeout(() => {
        console.log('in timeout 2');
        clearTimeout(timeout); // panic
        resolve(true);
    }, 1000);
});

use JS_UpdateStackTop

after init of the runtime use this set the correct stack top

and maybe set the max size as a large number by default

ESRuntime refactor

get rid of inner, use the Arc around ESRuntime
move all publics to a Trait which we can reuse in ES_Runtime

all types

hirofa / quickjs_es_runtime Goto Github PK

quickjs_es_runtime's People

Stargazers

Watchers

Forkers

quickjs_es_runtime's Issues

Error

Recommend Projects

Recommend Topics

Recommend Org

Jobs