38 / plumber Goto Github PK
View Code? Open in Web Editor NEWThe General-Purpose Cross-Language Dataflow Programming
Home Page: http://plumberserver.com
License: BSD 2-Clause "Simplified" License
The General-Purpose Cross-Language Dataflow Programming
Home Page: http://plumberserver.com
License: BSD 2-Clause "Simplified" License
We are planning to implement an asnyc task, which is the task do not need to hold the worker thread during running. This is meaningful when our servlet is supposed to perform some slow operation, like call other service and etc. Because the asyc task do not occupy the worker thread, so the same thread will be able to serve another request since then.
We need to limit what a async task can do, because if we don't have any limit on this, it probably introduce some race condition and break the lockless design. One resonable limit is, the asnyc task will only able to read data from the given buffer and do not have any acess to the framework related APIs, for example, pipe_read, etc.
So the async servlet should be defined as
SERVLET_DEF = {
.desc = "xxxxxx";
.init = _init,
.exec_init = exec_init, /* The function will be called by worker thread to setup the async task */
.exec_async = exec_async, /* The function will be called by async worker to do some slow ops */
.exec_finalize = exec_final, /* Clean up function called by worker thread */
.unload = _unload
}
Currently in the sched_step_next function, we mark the pipe ready right after the upstream output has been created. But this is no longer ture, and we should prevent the downstream task from starting until the async task is finished.
This prbably can be done by not calling the sched_task_input_pipe right after the pipe pair has been created. Instead, we call the sched_task_input_pipe when we are going to run exec_finalize.
I will create pull request about these.
RLS Stream might be very computing intensive, for example, a RLS with pending operation for compression, encryption and transcoding. And this will make the framework unable to reuse the result of the computational intensive operations on some immutable objects.
For example, for the static content server, it actually loss 50% of the performance if we enable compression.
For the design details about this Cache, we actually need an RLS object that abstracts the cache.
Thus we also need a callback for the object hash code and if the function is not defined, indicates we just got a volatile object (For example an object that generates a random number)
Additionally, this hashing infrastructure can be used as ETag in the HTTP server.
Currently, we have merged the container RLS types, but is far away from finished.
pstd_blob_* is the function that manages a typed memory region. It also caches the RLS object for future use.
pstd_array_* is the implementation for the array type. (Needs the blob model to create)
pstd_type_model_field_t served as the accessor in a memory blob
For the generic servlet with type "Array $T", we need add a type assertion and in the type assertion
we will be able to create the blob model, since $T is known.
For each time we want to read/write a RLS token using the cache, we need to use the blob model.
But for now, it's better to add some laziness to the HTTP parsere. (For exmaple, some unused header never gets parsed, etc.) So it seems we need to implement a seperate library so that we can create some RLS object specific for HTTP senario rather than use the generic one
It seems we actually don't want use the filesystem as a RESTful storage, because the data consistancy issues.
However, once we create a storage servlet wraps another data storage program, like mogodb, etc. Then we actually need the async task model to make it has high troughput.
This makes the task not feasible at this point, because the async task is not supported by the framework.
At the same time, it seems the async task support should have the highest priority, because we need this for:
So we need to do the RESTful after we finished the Async Task Support
Spliting too many servlet actually makes the server runs slow and graph be more complicated.
So we need to some of them, here's the candidate:
filesystem/mimetype and filesystem/readfile: Actually we could have one servlet which read the file and guess the mime-type, since there's no reason to split them.
network/http/encoder and network/http/render(Purposed): Since encoding a HTTP body has no other use than render a HTTP response.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.