includeos / acorn

Acorn Web Server Appliance, built with IncludeOS

WARNING: Deprecated. Repo moved to demo-examples

acorn

Acorn Web Server Appliance, built with IncludeOS.

Live Demo: acorn2.unofficial.includeos.io (sporadically unavailable)

Features

• Easily build RESTful APIs
• Use middleware to browse and serve static content with only a few lines of code
• Optionally show real-time information about your server with an interactive dashboard

Acorn is a simple web server using a collection of libraries and extensions:

• Mana - IncludeOS Web Framework
• Bucket - Simplified in-memory database
• Butler - Middleware for serving static content
• Cookie - Cookie support
• Dashboard - Back-end support for serving IncludeOS statistics
• Director - Middleware for listing static content
• Json - JSON support

Requirements

• IncludeOS installed (together with its dependencies)
• git

Setup

To setup the service:

$ ./setup.sh

It will do the following:

• Pull the required git submodules
• Create a 2 MB FAT16 disk called memdisk.fat - which will be included into service
• Mount it on mnt/ and copy the content from disk1/ onto the disk

Take it for a spin

To build and run the service:

$ ./run.sh

It will do the following:

• Mount and update memdisk.fat with the latest content from disk1/
• make the service
• Run the service with qemu

acorn's Issues

Dasboard: Scroll bars for panels containing tables

Currently panels containing tables doesn't scale well to mobile. Scrollbars inside the panel sounds like the easiest fix and should work fine. Logger has scrollbar and that works well.

Add timestamps

Current server stops responding now and then. Since it's impossible to see the whole log, its hard to track if the VM stopped working, or if the timer keeps printing the same data over and over.

Unit tests!

We need tests of Acorn as well as IncludeOS. Rico is testing http "off-line", which we should try to do with most things in Acorn as well.

Access server name via API

The server name should contain the IncludeOS version info (git describe) as well as Acorn git-info. This should be available via the API a well.

Crash report 25.06.16

This is the output. Have to dig around some more. The assertion needs to be handled, and a 404/500 needs to be sent. Not sure why it's looking for public, need to try to replicate. Could be that the memdisk has been corrupt over time.

[@GET:*] Fallback route - try to serve index.html
[@GET:*] (Fallback) Responding with index.html. 
<Waitress> Found file: public (0 B)
assertion "ent.is_file()" failed: file "./server/response.hpp", line 16, function: File::File(fs::Disk_ptr, const fs::Dirent &)
!!! Kill PID: 1, SIG: 6 - IOT trap / ABORT

     **** PANIC: ****
    Killing a process doesn't make sense in IncludeOS. Panic.
    Heap end: 0xcb1000

        [ ELF ] symtab or strtab is empty, indicating image may be stripped

[0] 0x20e166 + 0x0000: 0x20e166
[1] 0x20e3fe + 0x0000: 0x20e3fe
[2] 0x27e90a + 0x0000: 0x27e90a
[3] 0x29e269 + 0x0000: 0x29e269
[4] 0x29df86 + 0x0000: 0x29df86
[5] 0x22e1e2 + 0x0000: 0x22e1e2
[6] 0x22c9ab + 0x0000: 0x22c9ab

Add terminal to Acorn

Could be useful to have an alternate route in, so that we can verify if the whole thing has crashed or not. This should not be the default for a production version, but it might be useful at this stage.

Malformed GET request makes Acorn non-responsive

This request reliably makes Acorn stop responding:

"GET / HTTP/1.0\r\nContent-length:18\r\nUser-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)\r\n\r\n"

Add License file

GitHub now prominently shows the license in the repo overview, but since Acorn does not have a License file, nothing is displayed, making potential users having to go in and manually examine individual file header texts.

Add more debug information

We have currently no idea whats happening when thing goes wrong. Make Acorn pull more information from IncludeOS (currently thinking about logging more TCP events).

Cookie::set_max_age() marked as noexcept, but throws CookieException

cookie::Cookie c {"USER_TOKEN"s, "1mbda"s};
c.set_max_age(-666);
// => terminating with uncaught exception of type cookie::CookieException: Invalid max-age attribute (-666) of cookie! Negative number of seconds not allowed.
// => Abort trap: 6

Workaround: remove noexcept, but I haven't read the cookie spec, so ¯_(ツ)_/¯.

Exception trown after a few requests

Acorn terminates after throwing uncaught exception, releated to stoul.

Service output:

Listening to port 8081
================================================================================
 IncludeOS v0.8.0-rc.1-380-g54ea625
 +--> Running [ My Acorn Service ]
\~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
<Server> Responding with index.html.
     [ DHCPv4 ] Negotiation timed out
     [ DHCPv4 ] Negotiation timed out
<TCP::Connection::reduce_ssthresh> Slow start threshold reduced: 2147482188
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<Server> Responding with index.html.
<TCP::Connection::reduce_ssthresh> Slow start threshold reduced: 2147482188
<Server> Responding with index.html.
terminating with uncaught exception of type std::invalid_argument: stoul: no conversion

         !!! abort_ex. Why: terminating with uncaught exception of type std::invalid_argument: stoul: no conversion
         **** PANIC: ****
 terminating with uncaught exception of type std::invalid_argument: stoul: no conversion
        Heap end: 0x2c58000

Unfortunately there was no more detailed debug output from the service. Anyone knows what the last request was?

Error on Mana request/response

I'm getting an error on make:

lib/butler/butler.o: In function `butler::Butler::process(std::__1::shared_ptr<mana::Request>, std::__1::shared_ptr<mana::Response>, std::__1::shared_ptr<delegate<void ()> >)':
lib/butler/butler.cpp:(.text+0x2bd): undefined reference to `std::__1::shared_ptr<mana::Request>::shared_ptr(std::__1::shared_ptr<mana::Request> const&)'
lib/butler/butler.cpp:(.text+0x2d0): undefined reference to `std::__1::shared_ptr<mana::Response>::shared_ptr(std::__1::shared_ptr<mana::Response> const&)'
lib/butler/butler.cpp:(.text+0x2e3): undefined reference to `std::__1::shared_ptr<delegate<void ()> >::shared_ptr(std::__1::shared_ptr<delegate<void ()> > const&)'
lib/butler/butler.cpp:(.text+0x515): undefined reference to `std::__1::shared_ptr<mana::Request>::shared_ptr(std::__1::shared_ptr<mana::Request> const&)'
lib/butler/butler.cpp:(.text+0x528): undefined reference to `std::__1::shared_ptr<mana::Response>::shared_ptr(std::__1::shared_ptr<mana::Response> const&)'
lib/butler/butler.cpp:(.text+0x53b): undefined reference to `std::__1::shared_ptr<delegate<void ()> >::shared_ptr(std::__1::shared_ptr<delegate<void ()> > const&)'
lib/director/director.o: In function `director::Director::process(std::__1::shared_ptr<mana::Request>, std::__1::shared_ptr<mana::Response>, std::__1::shared_ptr<delegate<void ()> >)':
lib/director/director.cpp:(.text+0x138): undefined reference to `std::__1::shared_ptr<mana::Request>::shared_ptr(std::__1::shared_ptr<mana::Request> const&)'
lib/director/director.cpp:(.text+0x148): undefined reference to `std::__1::shared_ptr<mana::Response>::shared_ptr(std::__1::shared_ptr<mana::Response> const&)'
lib/director/director.cpp:(.text+0x15b): undefined reference to `std::__1::shared_ptr<delegate<void ()> >::shared_ptr(std::__1::shared_ptr<delegate<void ()> > const&)'
ld: Acorn: hidden symbol `_ZNSt3__110shared_ptrIN4mana7RequestEEC1ERKS3_' isn't defined

Is there some dependency I'm not aware of for request/resopnse?

Dasboard/stats: parent nodes should be highlighted

A collapsable tree would probably be even nicer, but I think just highlighting all parent nodes with light gray background and bold fonts or something should go a long way.

Limit number of squirrels stored

Bucket/Acorn need to limit how many objects it can store, to avoid running out of memory.

Best solution is probably to have Bucket take a limit, and throw an Exception if full - this will be work nicely in the front end with the new improvements (#17)

Add timestamp created to Squirrel model

Add a member timestamp to Squirrel to register when it was registered.

Guess it has to wait until clock is not that CPU intensive.

User identification / Admin authentication

Let's see if we can make some simple login stuff. I guess it requires Cookies?

Connections stuck in state LAST-ACK

From the Openstack output, some TCP connections seems to not disappear, and gets stuck in state LAST-ACK. This is the last state before a Connection gets shut down, and not sure why it's happening (on a local instance I never get this error). Not sure yet if it's because the connections doesn't gets correctly shutdown - if the issue is with IncludeOS or Acorn - yet.

To not worry or cause panic to the masses which can result in riots, i'll hide this issue here for now and keep debugging the TCP.

Display when server was created (booted)

Would be nice to display some information for the user when the instance was created.
It's no problem getting the value back end - the question is how to display it on the web page. It's not possible to inject the timestamp inside the HTML (until @fwsGonzo finished the template engine Kappa), so the best bet is to make a route and return the date as JSON.
Feels like a lot of work for so little, so maybe add a lot more information here?

Cookie constructor does not handle long Max-Age setting, throws unexpected exception

When calling the Cookie constructor I would expect to get a valid Cookie or having a CookieException thrown.

This works:

  using namespace std::string_literals;
  try {
    std::vector<std::string> options {"Max-Age"s, "-1"s};
    cookie::Cookie c {"USER_TOKEN"s, "1mbda"s, options};
    std::cout << c << std::endl;
  }
  catch (const cookie::CookieException& ce) {
    std::cout << "bad cookie!" << std::endl;
  }

This does not:

    using namespace std::string_literals;
    try {
      std::vector<std::string> options {"Max-Age"s, "9999999999"s};
      cookie::Cookie c {"USER_TOKEN"s, "1mbda"s, options};
      std::cout << c << std::endl;
    }
    catch (const cookie::CookieException& ce) {
      std::cout << "bad cookie!" << std::endl;
    }
    // => terminating with uncaught exception of type std::out_of_range: stoi: out of range

IncludeOS goes unresponsive when request spamming for a file

To replicate the error:

• Start acorn
• Go to http://10.0.0.42/static/books/borkman.txt (or any other book) in your browser
• Spam refresh

Then it won't take long for the OS to go unresponsive (like 10 requests maybe).
Timers still working.

My first guess something is overwriting memory.
The heap is growing, so I guess there is a leak somewhere.
... or maybe not. It's between 3.5MB and 3.6MB

Integration tests

We need integration tests for acorn, Including, but not limited to:

• Verify the file content of all text files
• Verify that all routes provide valid json and that the ones where you expect frequent updates actually do update
• Stress: verify that the VM survives things like ping- and http-flooding, and that memory usage is as expected after stress
• Verify that all kinds of invalid http requests produce suitable status codes (e.g. index.asdf, /non/existing/file`
• Verify that requests with a method not subscribed to for a route gets dropped / does the right thing

...etc.

Regex route match should preserve matched groups

Eventually, we want e.g. /api/users/:id/profile to result in capturing id into a key-value data structure,
so we can e.g. do req.route("id") => 52 for the URI /api/users/52/profile.

While we don't have the regex-from-route-path-generator we know it's going to use regex capture groups, so we can still preserve these. We can now do e.g.: /api/users/(\d+)/profile, and provide the resulting matches in a datastructure. We don't have key names yet, so it will be
req.route_part(0) => 52 for the URI /api/users/52/profile.

Route-syntax to Regex

We want to be able to express route paths in the same format as in express.js. We should port path to regex

Migration of Squirrels

Acorn is currently not stable enough to not crash, but good enough to run on OpenStack.
It's fun to see users capturing new Squirrels 😸 , but sad to see them escape when the VM dies 😿 .

My current idea is something like this:

• Create a route to something like /api/squirrels/migrate/:ip (or ?ip= until parsing is implemented)
• When booting a new instance with a clean Bucket, the "admin" just enters http://<current instance>/api/squirrels/migrate/<old instance>, and all the squirrels will clone and jump into the new bucket.
• To avoid being exploited, we just add an bool if it's been a migration, if true, just return 404 (or a more suited status code).

This will not help if the old instance has already crashed tho (which it shouldn't soon™).

Any better ideas?