riolet / wafer

WAFer is a C language-based software platform for scalable server-side and networking applications. Think node.js for C programmers.

WAFer

WAFer is a C language-based ultra-light scalable server-side web applications framework. Think node.js for C programmers. Because it's written in C for the C eco system, WAFer is wafer-thins with a memory footprint that is only a fraction of that of node.js and other bulky frameworks.

Just copy server.c (say, as myserver.c), put your code inside the function void server(Request request) in myserver.c and, make with make SERVER=myserver, and you are good to go.

WAFer can operate in many different configurations, all selected at compile time. They include:

1. Single-threaded (Default) or multi-threaded (make with THREADS=n where n>0)

2. Select(Default) or epoll (make with LOOP=epoll) based event loop

3. C10K mode (make with LOOP=epoll MAX_CON_CONS=n where n>10,000)

Default port is 4242. Set environment variable 'PORT' to change it.

That's really it. The source comes with a simple example example.c to get you started.

Note to Contributors

Thank you for making this a wonderful project!

Here's our preferred formatting style:

find . \( -name '*.c' -o -name '*.h' \) -exec indent --no-tabs  --linux-style --line-length 90 --indent-level 4 -bli0 \{\} \;

Acknowledgements

1. J. David Blackstone and Feng Shen, whose web servers have been repurposed to build this platform.

2. Mark Karpeles for the incredible number of bug fixes!

3. Fine folks at /r/programming for the honest and constructive feedback.

wafer's Issues

Buffer overflow in nprintf()

nprintf does not check bounds of its buffer.

Example code:

char *testbuf = calloc(1, sizeof(char) * 1000001);
memset(testbuf, 'A', 1000000);
printf("Testbuf len: %ld\n", strlen(testbuf));
nprintf(client, testbuf);

Mallocs are not checked for failure

Malloc can fail.

http://pubs.opengroup.org/onlinepubs/009695399/functions/malloc.html

Static sized arrays without perform bounds checking on inputs

As informed by /u/jmhodges

Building on OS X fails

On OS X, building nope.c fails for me with:

> make
gcc -W -Wall -O2 -c -o nope.o nope.c
nope.c:13:10: fatal error: 'sys/sendfile.h' file not found
#include <sys/sendfile.h>
         ^
1 error generated.
make: *** [nope.o] Error 1

gcc --version output:

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
Target: x86_64-apple-darwin13.3.0
Thread model: posix

Buffer overflow in client_error

After having a quick look at the code I think you should put a big warning on the README that this code is not at all ready for anything that listens on a public port. Besides the stack overflow that was already reported by MagicalTux there's also this code which will cause a bufferoverflow if msg & longmsg are too long:

void client_error(int fd, int status, char *msg, char *longmsg){
    char buf[MAXLINE];
    sprintf(buf, "HTTP/1.1 %d %s\r\n", status, msg);
    sprintf(buf + strlen(buf),
            "Content-length: %lu\r\n\r\n", strlen(longmsg));
    sprintf(buf + strlen(buf), "%s", longmsg);
    writen(fd, buf, strlen(buf));
}

Pretty sure this is not the only problem remaining.

Method to set response Mimetype

Perhaps not an issue, may be my short sightedness. Is there any way to set the mimetype of the response. I would like to use nope.c to produce small rest servers serving simple dynamic json. i would like to be able to set response mimetype as "application/json", based on the response.

support other multiplexing I/O methods?

Hi, I read the code of the whole project, it's now only supporting the select method. And there are someother multiplexing I/O methods, such as epoll, kqueue. Will nope.c support these methods? If so, I think the project needs a simple event driven library which packages multiple multiplexing I/O methods. And I'm writing a simple event driven library libse, I hope I could make some contributions to nope.c. Looking forward to your reply!
thank you very much!

Why not use dprintf instead of nprintf?

It's in POSIX ≥ 2008.

Better non-blocking socket reading

As @MagicalTux explained in issue #19, our select() is not very effective since "While this new method allows a single worker to process multiple requests at the same time, it is quite unlikely to happen - especially since the process() method will not return the processing of that query is done."

Is there a standard function to handle 404?

an alternative solution might be

void server(Request request)
{
    int is_404 = 0;
    is_404 |= nope_route(request, "/factor", &factor, true);
    is_404 |= something else...
    if(!is_404)
    {
        handler_404(request);
    }
}

void handler_404(Request request)
{
    not_found(request.client);
}

AF-Unix

I was a bit confused while going through your code particularly when I noticed that socket used "AF-Unix" . Why using it? I also noticed that now you have epoll, thread, and select. I fcntl and thread, but what do you again with epoll

Some clean up

notify_parent() is not used in your curent version and timespec is not used too. Could you clean this up? The way you called getnameinfo without checking its failure mode is a bit confusing to me.

Really a use for select() ?

The way nope.c works, each worker thread will wait for an incoming connection, then process it.

While this new method allows a single worker to process multiple requests at the same time, it is quite unlikely to happen - especially since the process() method will not return the processing of that query is done.

I would guess the purpose is to have multiple open sockets and wait for incoming data, but if one sender is very slow and sends fragmented request elements (ie. only "GET " at first), it will block that child while the child may have accepted other connections.

Anyway I believe it would make sense to either keep nope.c's original simple design (listen, process), or if we really want to do this right we could:

• listen is only done in parent process
• each child process has a unix pipe to the parent process and keeps waiting for data to come
• incoming connections are accepted and data is received async in a temporary buffer (until empty newline or buffer full which would cause an error to be sent)
• once full buffer received, it is passed to an available child process through the pipe, alongside the client's fd (UNIX domain sockets can pass file descriptor between processes)
• parent closes the file descriptor, free memory
• child processes the request, responds directly to the client

From there, the parent process could automatically increase the number of child processes should - for example - all children be busy, and do a lot of other interesting things.

Pros:

• If someone opens multiple connections but doesn't send anything, it doesn't cause a denial of service
• Parent process has more control on the distribution of work
• If a child dies for some reason (bug in the server for example) parent will notice and will be able to respawn some children

Cons:

• More complex to implement
• Could potentially cause parent process to block if a child is frozen
• Things like OpenSSL will have troubles dealing with the fd moving to a different process (could be solved by having the parent act as a proxy rather than sending over the fd - also would improve security as child code will not have access to certificates/etc)

README needs info on size of WAFer

WAFer is: ultra-light ... wafer-thin

Can you be more specific in the README?
We would like to see approximate LOC, size in memory, size on disk. (compared with node.js etc.)

Add logging option

Is there any interest in supporting a simple logging option for requests? I would be willing to help implement it.

<p> tag never closed in unimplemented()

nope.c, line 67, the <p> tag is never closed.

Long urls will fail

Line:
https://github.com/riolet/nope.c/blob/master/nope.c#L224
getLine(client, buf, sizeof(buf));

All requests with a first line longer than 1024 bytes will fail.
As the server stops ignores the rest of the line.

Who needs a web framework written under GPL terms anyway

Writing web applications in C is a great idea. Still, people who want to write their own app using C can't legally do it without opening their sources. That's because you distribute it under GPL terms as a set of source files which should be extended by users to get their functionality:

2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:

    a) You must cause the modified files to carry prominent notices
    stating that you changed the files and the date of any change.

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

    c) If the modified program normally reads commands interactively
    when run, you must cause it, when started running for such
    interactive use in the most ordinary way, to print or display an
    announcement including an appropriate copyright notice and a
    notice that there is no warranty (or else, saying that you provide
    a warranty) and that users may redistribute the program under
    these conditions, and telling the user how to view a copy of this
    License.  (Exception: if the Program itself is interactive but
    does not normally print such an announcement, your work based 

It would be more appropriate to license it under dual MIT/GPL license so people could use your framework without opening the application source code.

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

• pthread_mutex_init ⇒ initialize_threads
• strdup ⇒ getQueryPath

repeat code

file nope.c:
line 90 and line 92 repeated?
line 108 and line 110 repeated?

Potential buffer overflow at nope.c:269

strcpy(req.filename,url);
url can be 1024 chars
req.filename can be 512

nope.c:77 - Result of 'malloc' is converted to a pointer of type 'char', which is incompatible with sizeof operand type 'char *'

nope.c architecture

Let's have a brief discussion about nope.c architecture. What I have in mind for nope.c is a layered architecture that embodies the Unix philosophy with three main strata:

1. The core (nope.c): A minimalist HTTP server, mostly focussing on the event loop. We are using select() currently, but we could provide additional event notification systems (poll(),epoll() etc) as drop in replacements.
2. The shell (nopeutils.c - optional): Provides a programming interface to the core, abstracting over most of the cores operations.
3. The modules (optional): Provides various enhancements to make it easier to write useful web applications. nope.c does not need many specific nope.c modules as there are thousands of c libraries that nope.c can use natively. That said, we need modules for ssl, multi-part parsing and web sockets.

What are your thoughts?

TCP_CORK is Linux-specific, need to #ifdef it as such

Load Test Failed

I carried out a load test for nope example.

./wrk -t1 -c1 -d10s http://localhost:4242/factor

The config is: NOPE_EPOLL, NOPE_THREAD 2
The bug looks like this:

GET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET /factorGET .... /factorGET /factorGEclose: Bad file descriptor

check send buf before sending data?

Now in nope.c, send function is called without checking whether the kernel send buffer is enough using select. For example, there are 100 active clients, we need to send 1k data to each client, but the total kernel buffer is 10k (just as an example), we know that when send function is finished, it only ensures the data of process is copied into kernel send buffer, maybe when dealing with the 50th client, the kernel buffer is full, then the process will blocked until the kernel buffer has enough space for data.

any chance to relicense post-GPL, like LGPL, MIT, BSD?

would make a nice addendum to what we use in machinekit.io - unfortunately we're unable to incorporate GPL-only code

any chance to relicense with a 'postwar' license ;-?

Buffer overflow in nope.c:278

Incriminated code:
http_request req;
strcpy(req.filename,url);

"url" is 1024 bytes, req.filename is 512 bytes. Buffer overflow ensues by passing an URL between 512 bytes and 1024 bytes (plus "req" is helpfully allocated on the stack).

Suggestion: use strncpy().

need a automatically expandable buffer?

In nope.c, buffer is used everywhere, I think we may need a automatically expandable buffer, which could automatically expand the memory space when add or copy data into the buffer. I wrote an simple one, see this .

Integer overflow in array allocations

malloc(n * size) is an antipattern as n * size can overflow (especially from an attacker). You'll want to use calloc or create your own malloc_array function. Something like the following could work:

static inline void * malloc_array(size_t nmemb, size_t size)
{
    /* Make the check so that a constant size deletes the zero check
     * and the division.
     */
    if (size != 0u && SIZE_MAX / size < nmemb) {
        errno = ENOMEM;
        return NULL;
    }

    return malloc(size * nmemb);
}