Seems like the updated getrandom dependency breaks WASM compilation

To finally support plugins in the browser, the runtime in the is missing serializing the response on the host side correctly(as far as I can remember since its been a while).

Related to #8 and possibly #13, there should be an easy way to have some state that is shared across requests(e.g. a DB connection). One way is attaching data to context that is passed to requests, middlewares and possible plugin life-cycle hooks.

#[hook]
async fn on_created(cx: &mut Context) {...}

#[vlugin]
async fn handler(req: Request, cx: &Context) -> Response {...}

Clients of a Valor API will be HTTP-only initially as it is the only protocol interceptable in Service Workers. To provide an event subscription mechanism there is luckily SSE that works on top of plain HTTP unlike WebSockets that follows a different protocol upgrade and is not interceptable from the SW.
From the plugin implementation side it could be that if the return type of the on_request handler is a Stream then we assume the protocol used is SSE. Other option is for the plugin to block and loop indefinitely publishing messages via the Context object and the runtime decides what to do with the events, like send it to other plugins that are subscribed or a special plugin that handles all the SSE requests.

The main goal of Valor being decentralization, when registering a plugin, instead of providing a url it should be possible to provide a hash of a file or directory that exists on IPFS so the runtime can fetch it with an embedded client, from a local IPFS node or a well known gateway.

Loading a plugin should be decoupled form adding an instance to the registry. This would allow having the same plugin being instantiated multiple times with different configuration.
The following definition should load the plugin once and then create different instances:

[
  {"type":"native","name":"foo"},
  {"type":"native","name":"foo", "prefix": "bar"},
  {"type":"native","name":"foo", "prefix": "baz", "cfg": {"prop": "value"}}
]

Plugins could also be middlewares that modify and pass along the current request or response.
For Rust plugins I imagine the #[vlugin] macro being smart enough to detect the that the annotated function should be a middleware based on the method signature.
i.e.

#[vlugin]
pub fn check_auth(&mut Request) { ... }

Plugins need to run in a browser context where requests that are received asynchronously are passed to the handler function that returns a response that is sent back to the caller.

Plugin writers should be able to add more than one handler that would process messages coming from an event bus or a different source of data. The one handler could also be just the custom message handler instead of an HTTP one.

// V1 HTTP request handler 
#[vlugin]
async fn http_handler(req: Request) -> Response {}

#[vlugin]
async fn my_data_handler(req: MyData) {}

Currently a plugin is only a function that handles a request. Plugins should be allowed to hold some state that is initialized at registration time.

Writing a JS plugin should be as easy as exporting an async function that receives a request and returns a response.

This issue tracks the required bits to make writing JS plugins easy like any helper libraries or code in the host that loads the scripts and runs the right function. For JavaScript support in the server see #12

To use the ? operator handlers should be able to return a Result(probably a re-export of http_types::Result). The macro would still allow to return a plain response.

// this works
#[vlugin]
async fn handler(req: Request) -> Response {} 
// this as well
#[vlugin]
async fn handler(req: Request) -> Result {} 

Common request, specially when existing services haven't or can't be migrated to be run as a plugin is to allow for simple forwarding of requests to external services. Some extra points homework(probably in a separate issue) is to design it in a way that the upstream backends can be other instances of Valor running on separate threads/workers and more points if there can be multiple backends for the same path prefix with a simple round-robin or randomized load balancing.

The server should accept reading a configuration file that lists the plugins that should be pre-loaded at start up.

A more suitable WASM runtime for the server side could be Wasmer as it supports AOT compilation and comes with a smaller footprint. It might also easy the browser implementation as it provides shims for WASI or BitInt support.

Currently a panic in a plugin crashes the whole program. We need some sort of supervisor that can handle a panic in a macro and return an error response instead.

I'm not sure if one can wrap a function that panics and handle the panic or if it has to be done for the whole thread which would mean there would need to be a thread for request handling that should be restarted reloading the relevant plugins.

With the help of deno-core we can bring a V8 powered JS runtime to the server side to be able to write and run plugins in JavaScript and WASM. Ideally the runtime should support all the APIs available to Web Workers in the browser but we can start simple and support some basic web APIs and fetch, sadly it's still not straight forward to extract the Worker environment out of Deno so the initial implementation will have a reduced API surface and possibly not even import other modules.

With valor_bin when a plugin is not matched or apparently with any other runtime error the connection gets closed instead of returning an HTTP response with an error code and message.

As native plugins are the current focus of the project, documentation to go along with their loading and routing allows for some experimentation with features already available.

cargo docs could be added to be served as a GitHub page too.

When a handler in a plugin errors the response doesn't include the same headers as the non error response.

Badges are useful like linking to the documentation, CI, etc.

When running Valor in the web specially, it will be very useful to swap one plugin with another that has the same API but changes its implementation in some way, for example one could quickly load the runtime and create an instance of the Sube plugin that uses the lightweight HTTP backend to connect to a remote node and in parallel download the heavier version of the plugin that uses a light node as a backend and once it's downloaded and ready one can swap the HTTP version for the lightnode without the client application realizing there was a change, if the application wants to be more transparent with the user it could install the heavier version only after prompting the user to install the application.

Still missing a lot of documentation and tests to showcase how Valor works. We need to at least cover the basics.

With the added Context type we can expose an API to send requests to other plugins. The requester should know in advance what kind of messages the receiver is able to handle(e.g. HTTP).

Question to self about API design ... Should this be a simpler "synchronous" kind of API or should it be part of a more generic async one that publishes messages with the option to wait for a response? maybe a sync helper built on top of the async one? should there be a bult-in message queue that dispatches messages to potentially multiple subscribers or leave it to the user to configure a default "message processor" 🤔

The Valor cli besides running the HTTP server, it can provide a plugin sub-command that allows creating plugins from a template, take care of the building and publishing.

If we want to keep the server lightweight, this tool that would be used for easier development can be also released as a separate companion application or behind a feature flag.

valor.js(the runtime, not the plugins) can be loaded and run in the main thread of the browser but should also be able to run when loaded as a Worker.

It should be possible to decorate a struct that implements the request handler interface to be loaded as a native library or WASM Worker.

#[vlugin]
struct MyHandler;

impl RequestHandler for MyHandler {...}

The Runtime is itself a plugin that is not needed when valor is used as dependency of a plugin.

The runtime needs to be as tiny as possible with no enabled features by default. Anything that looks like it can be a feature behind a flag should be. We need to pay attention to what gets exported for plugin authors and not make assumptions like the plugin being an HTTP handler.

• API Docs
• CI Status
• ...

The current #[vlugin] interface is simple enough but needs some rework.

First, we won't have http messages, so on_request and the http dependency is deprecated. We will only have an on_message/on_msg that accepts a well known user defined message that implements Deserialize(on_create remains).
Current vlugin example:

#[vlugin]
pub async fn on_create(cx: &mut Context) {
    let someone = cx.config::<String>().unwrap();
    cx.set(someone + " says hello");
}

// DEPRECATED
pub async fn on_request(cx: &Context, req: http::Request) -> http::Result<http::Response> {
    // ...
    Ok("answer".into())
}

Second, the macro is too magic, it requires a build.rs script to parse the source file as "temporary" workaround for custom inner attributes not being available but it adds friction and the feature doesn't look like it will land anytime soon. Instead we can be a bit more verbose and create a derive macro for the message that the user needs to define

#[derive(Message, Deserialize)]
enum MyMsg {
    DoThing,
}

fn on_msg(cx: &Context, msg: MyMsg) {
    // ...
}

Third, the WASM support means we trash what's currently being generated to support dynamic libraries and replace it with a wasm_bindgen interface that can be called by the host JS environment.

Last, to compensate for the new added verbosity we can add an optional cqrs feature that provides macros that generate the boilerplate necessary for this recommended pattern of separating commands and queries.

#[cmd]
mod cmd {
    fn do_thing(cx: &Context, some_param: String) { ... }

    fn save_something(cx: &Context, some_param: String) { ... }
}

#[query]
mod query {
    fn get_stuff(cx: &Context, some_filter: MyFilter) { ... }
}

Nothing set in stone but good to start the conversation. @mrkaspa let me know what you think ;)

Valor: A Multi-Target, Transport-Agnostic Runtime for Modular Code Execution

Valor is a Rust runtime designed to run arbitrary code organized in a modular way, independent of the transport layer. It can be compiled for multiple targets, including server-side applications, WebAssembly (WASM/WASI), and embedded systems.

Design

Module Registry

Valor maintains a static, thread-safe registry of modules. Each module contains a collection of methods, along with arbitrary metadata in the form of extensions. This design allows for organized and manageable execution of code in different environments.

Attribute Macros

Valor provides three attribute macros: valor::module, valor::method, and valor::extensions. These macros are used to define modules and their associated methods and extensions.

• valor::module: Defines a module. The module contains a collection of methods and metadata.
• valor::method: Marks functions as methods of the containing module.
• valor::extensions: Provides arbitrary metadata for methods or modules.

Module and Method Structures

Modules and methods in Valor are represented by the Module and Method structs, respectively. Both structs contain a name and a set of extensions. The Method struct also contains a function that can be called to execute the method.

Example Usage

Here's an example of how to define a module and its methods using Valor:

#[valor::module(a_module)]
pub mod a_module {
    use valor::structures::{Request, Response};

    #[valor::method(a_method)]
    #[valor::extensions(http_verb = "GET", http_path = "/")]
    pub fn a_method (request: Request) -> Response {
        // Implement the function
    }
}

In this test, we create a Module, add it to the registry, and then verify that it was correctly added.

Future Work

The design of Valor allows for further expansion and adaptation to different use cases. Future plans include additional attribute macros for more granular control over methods, an expanded registry that can manage modules across multiple servers and applications, and potential integrations with different transport layers. Stay tuned for updates!

Trying to build the application using make command. Facing some core rust installation issues.
Can you provide docker image with which the application can be replicated, built and deployed locally easily.

Create function in macro to export Wasm code to JavaScript through Wasm Bindgen if the target architecture is Wasm.