tiny-http / tiny-http

Low level HTTP server library in Rust

tiny-http

Documentation

Tiny but strong HTTP server in Rust. Its main objectives are to be 100% compliant with the HTTP standard and to provide an easy way to create an HTTP server.

What does tiny-http handle?

• Accepting and managing connections to the clients
• Parsing requests
• Requests pipelining
• HTTPS (using either OpenSSL, Rustls or native-tls)
• Transfer-Encoding and Content-Encoding
• Turning user input (eg. POST input) into a contiguous UTF-8 string (not implemented yet)
• Ranges (not implemented yet)
• Connection: upgrade (used by websockets)

Tiny-http handles everything that is related to client connections and data transfers and encoding.

Everything else (parsing the values of the headers, multipart data, routing, etags, cache-control, HTML templates, etc.) must be handled by your code. If you want to create a website in Rust, I strongly recommend using a framework instead of this library.

Installation

Add this to the Cargo.toml file of your project:

[dependencies]
tiny_http = "0.11"

Usage

use tiny_http::{Server, Response};

let server = Server::http("0.0.0.0:8000").unwrap();

for request in server.incoming_requests() {
    println!("received request! method: {:?}, url: {:?}, headers: {:?}",
        request.method(),
        request.url(),
        request.headers()
    );

    let response = Response::from_string("hello world");
    request.respond(response);
}

Speed

Tiny-http was designed with speed in mind:

• Each client connection will be dispatched to a thread pool. Each thread will handle one client. If there is no thread available when a client connects, a new one is created. Threads that are idle for a long time (currently 5 seconds) will automatically die.
• If multiple requests from the same client are being pipelined (ie. multiple requests are sent without waiting for the answer), tiny-http will read them all at once and they will all be available via server.recv(). Tiny-http will automatically rearrange the responses so that they are sent in the right order.
• One exception to the previous statement exists when a request has a large body (currently > 1kB), in which case the request handler will read the body directly from the stream and tiny-http will wait for it to be read before processing the next request. Tiny-http will never wait for a request to be answered to read the next one.
• When a client connection has sent its last request (by sending Connection: close header), the thread will immediately stop reading from this client and can be reclaimed, even when the request has not yet been answered. The reading part of the socket will also be immediately closed.
• Decoding the client's request is done lazily. If you don't read the request's body, it will not be decoded.

Examples

Examples of tiny-http in use:

• heroku-tiny-http-hello-world - A simple web application demonstrating how to deploy tiny-http to Heroku
• crate-deps - A web service that generates images of dependency graphs for crates hosted on crates.io
• rouille - Web framework built on tiny-http

License

This project is licensed under either of

• Apache License, Version 2.0, (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in tiny-http by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

tiny-http's Issues

Server root

......

break OR else in task_pool.rs

Just a little thing, to be fixed one way or the other:
task_pool.rs lines 110 - 116:

let task;
loop {
    if let Some(poped_task) = todo.pop_front() {
        task = poped_task;
        break;
    } else {

Support the `expect` header

Add limit to size of headers

If a client sends 20 GB of data without any new line, tiny-http will end up panicking because of lack of memory.
It should instead return 413 Entity Too Large.

Other servers usually have this limit between 4 KB and 16 KB all headers combined.

Improve efficiency with multiple calls to `recv()`

Currently it is not very efficient to call recv() multiple times simultaneously

RFC: strong headers typing

Use strong typing for headers. But unlike hyper, use struct fields instead of that confusing type-checked thing.

Something like:

pub struct ResponseHeaders {
    pub content_length: Option<u64>,
    pub content_disposition: Option<Cow<'static, str>>,
    pub redirect: Option<Cow<'static, str>>,
    ...
    pub custom_headers: Vec<(Cow<'static, str>, Cow<'static, str>)>,
}

It would be forbidden to put standard headers in custom_headers. Only things such as X-MyCustomWebAppHeader would be allowed.

The main motivation behind this change is that some headers are too "system-y" like Connection, Trailer or Transfer-Encoding. We really don't want a user to be able to manually write these headers.

It would also prevent typos.

Parse the request body so it's easily usable by the user

Support deflate transfer-encoding

What might a tiny-http server using a thread-pool for workers look like?

Does anyone have an example of a simple complete tiny-http server using a thread-pool of workers where the thread-pool is managed by tiny-http? Have I missed such? I only see an example with a fixed vector of 4 threads managed by fn main(). I've also not seen the API for using a thread-pool in the documentation.

I'd be happy to help refine such an example and such documentation and I need something to start with. I'm new to Rust and slogging through the tiny-http source files trying to figure these things out. It's difficult to tell from the implementation what is supposed to be the API and what is temporary scaffolding.

Thanks, _Greg

Rewrite headers parsing to use only Ascii

Encodings other than Ascii are not allowed in headers. By handling only Ascii we could avoid all the unicode-related performance hit.

Allow responses without a Content-Length header

According to https://tools.ietf.org/html/rfc7230#section-3.3.2 , the Content-Length header shouldn't be set when returning certain status codes:

"A server MUST NOT send a Content-Length header field in any response
with a status code of 1xx (Informational) or 204 (No Content)."

I haven't found a way of doing this with tiny-http yet, I've tried constructing a response with:

Response::empty(204)

this results in a response which still contains the Content-Length header:

HTTP/1.1 204 No Content
Server: tiny-http (Rust)
Date: Fri, 13 Oct 2017 09:46:34 GMT
Content-Length: 0

Websockets connections should not time out

Test for panic recovery

Rust has some ways to recover from panics, with thread::spawn and thread::catch_panic.

When this happens, there are two possibilities: either the Server can't handle this and it will panic at the next call, or it handles this properly and will not. The second option is preferable.

Fix the travis benchmarks

For some reason, ab fails to connection to the local servers on travis.

Parse and analyse the request's TE header

Change thread model

Currently one thread is spawned per client connection (like Apache does).
However this is the only possible solution today, since Rust doesn't allow waiting on a Reader.

Another possibility would be to use green threads, but there are issues when passing I/O objects between green threads and native threads.

Support for HTTP 2

The resource on the request line is not necessarly an URL

Client errors should not always close the connection

Some errors in the client request may be recoverable (eg. a wrong Expect header), and should not close the connection.

Need public access to Method::as_str()

Is it intentional that Method::as_str() be private?

impl Method {
    fn as_str(&self) -> &AsciiStr {
        match self { &Method(ref s) => s }
    }
... }

I'm attempting to send a Request to the database
and the ONLY part of the Request I can't seem to
get access to is the Method (except by writing it
to a buffer which is awfully inefficient). Getting
it as an AsciiStr would be ideal.

_Greg

Support for HTTPS

This is a long-term issue. Don't expect it to be implemented within a few weeks or months.

Add timeout to client connections

Idle connections should close after a certain time

spins burning CPU

On my mac I see tiny-http is constantly using 10% CPU.

While idle, tiny-http loops through these syscalls:

78284/0xe23d5: psynch_mutexdrop(0x109C250F0, 0x318C03, 0x318A00) = 0 0
78284/0xe23d4: psynch_mutexwait(0x109C250F0, 0x318B03, 0x318900) = 3247363 0
78284/0xe23d4: psynch_mutexdrop(0x109C250F0, 0x318D03, 0x318B00) = 0 0
78284/0xe23d6: psynch_mutexwait(0x109C250F0, 0x318C03, 0x318900) = 3247363 0
78284/0xe23d6: psynch_mutexdrop(0x109C250F0, 0x318D03, 0x318C00) = 0 0
78284/0xe23d3: psynch_mutexwait(0x109C250F0, 0x318D03, 0x318A00) = 3247363 0
78284/0xe23d0: __semwait_signal(0x903, 0x0, 0x1) = -1 Err#60
78284/0xe23d5: __semwait_signal(0x903, 0x0, 0x1) = -1 Err#60
78284/0xe23d4: __semwait_signal(0x903, 0x0, 0x1) = -1 Err#60
78284/0xe23d6: __semwait_signal(0x903, 0x0, 0x1) = -1 Err#60
78284/0xe23d3: __semwait_signal(0x903, 0x0, 0x1) = -1 Err#60
...

It would be nice if tiny-http was using zero CPU while idle.

Support for websocket

basic auth?

I cant find any reference to HTTP basic auth being supported, is there a way we can support it?

Add optional mio support

It should be possible to do it without changing the API.
The event handler would add requests to a list, and recv/try_recv would pick from the list or run event_loop.run_once.

The biggest blocker is that this requires changing the parsing model from pull to push.

spins using 100% CPU

The default example provided in the README.md file uses 100% CPU after serving a single request.
It shouldn't be using any CPU as I only sent it a single request.

Support trailer headers in case of chunked request data

I'm not even sure if this is really something

support for ipv6

Seems like it might be really simple to add it here. I'll play around a bit, but I'm pretty new to Rust (at least since 1.0).

how to more easily send response data stored in memory

This is parallel to #48 but for response data which is stored in a string or byte array. As issue $48 points out, the existing API for constructing Response objects seems to be biased towards serving files and I don't know of anything like istrstreams for Rust yet. The best I've come up with to work around this is

use std::io::{BufReader,Read};
fn str_reader<'a>(s: &'a str)->BufReader<&'a[u8]> { BufReader::new(s.as_bytes()) }
...
let headers: Vec<tiny_http::Header> = Vec::new();
...
req.respond(tiny_http::Response::new(
  tiny_http::StatusCode::from(200), headers,
  str_reader("Hello World"),
  None, None
));

Is there a better way? I'm guessing that people will mostly NOT choose tiny-http if they merely want to hand back files as that's what the more common heavyweight (fancier yet slower and less flexible) http server frameworks provide. Some adapters wrapping strings, byte-arrays & functions into Readers might be all that's needed, as issue #48 suggested.

tiny-http comes closer than any other Rust http-server kit to my needs, although of course there are areas where the fit is not perfect. I'll try to turn each area where I'm having trouble using tiny-http into an issue so that either I can learn how I can do what I want within the existing framework or perhaps tiny-http can be improved for additional use cases.

Headers as HashMap rather than Vec<Header>?

Might it make more sense to implement Headers as a std::collections::HashMap rather than a Vec of Headers? Since HTTP headers are always key-value pairs, and often we want to do lookups on them (see, for example, this iteration, it seems to me a HashMap of Strings might ultimately be a better choice. The pros and cons that stand out to me are below:

Pros

• Faster lookup
• Potential feature expansion

Cons

• Refactor of an existing and working feature
• Lose control over header format without additional refactor on functions that build Headers

Ensure that responses are in sequential order

It is currently possible to produce two Request objects from the same connection and two have the second request answered before the first one. This should not be possible.

Use ToSocketAddrs when building the server

This requires a breaking change in the API.

The example panics

The sample README.md code panics:
let server = tiny_http::ServerBuilder::new().build().unwrap();

fix:
let server = tiny_http::ServerBuilder::new().with_random_port().build().unwrap();

or with_port(8080) ...

Support for ranges

Allow override of Content-Type header when already specified

Currently, if tiny-http automatically inserts a Content-Type header into a Response, and then the developer inserts an explicit response, the Content-Type header is sent twice.

As an example:

let resp = Response::from_string("hello")
    .with_header(Header::from_bytes("Content-Type".as_bytes(), "application/json".as_bytes()).unwrap());

This results in the following response:

HTTP/1.1 200 OK                                                                                                                        
Content-Length: 17                                                                                                                     
Content-Type: text/plain; charset=UTF-8                                                                                                
Content-Type: application/json                                                                                                         
Date: Thu, 17 Aug 2017 18:01:10 GMT                                                                                                    
Server: tiny-http (Rust)  

Following the API from an intuitive perspective:

1. Craft a response from this string hello
2. with the header Content-Type: application/json

It's rather inconvenient to need to use the Response::new(..) method to create a response from a string with a custom content type.

Thanks!

visibility issues

in src/lib.rs:

mod client;
mod common;
mod request;
mod response;

should be

pub mod client;
pub mod common;
pub mod request;
pub mod response;

Rationale: so I can define a Request param like this:

...
for request in server.incoming_requests() {
test(&request)
}
...

fn test (request: &tiny_http::request::Request) {
...
}

Rework the Response to make the copy optional

The idea behind the design of the Response is that you can build a Response object without having to tie it to the request.
This allows the user to build multiple responses and choose the right one afterwards, makes testing easier because you don't need a mock request, etc.

Therefore the Request object holds ownership of a reader that implements the trait Read and contains the data. When you answer the request, it calls something like std::io::copy(&mut socket, self.data).

I suggest to make things more direct and allow the user to bypass this copy and write directly to the socket. Instead of taking an object that implements Read, the Response would take an object that implements Render:

pub trait Render {
    type Err;
    fn render<W: Write>(&self, &mut W) -> Result<(), Err>;
}

If you call Response::from_file or Response::from_string, tiny-http would build the appropriate object that implements Render and that copies data from the file or string to the socket.

This change is not trivial, as it requires more changes like adding a EqualWriter type, plus the error handling thing needs some thinking.

There's a memory leak somewhere

When running the hello-world example, the memory consumption slowly increases over time.

Add the possibility to configure the server

Should be configurable:

• Connections timeout
• Maximum number of connections before 503

Events when recv data.

Can i use events when data recv?
Example:
let srv = Server::http("127.0.0.1:8080");
srv.handle(foo);

fn foo(req: Request, res: Responce) {
// This method call when srv recv data.
}

what's the best way to test/match Method objects

I'm finding it awkward to construct and test Method objects. I've found myself doing clumsy things like:

let get_ = tiny_http::Method::from_str("GET").unwrap();
let put_ = tiny_http::Method::from_str("PUT").unwrap();
if req.get_method().eq(&get_) || req.get_method().eq(&put_) { ... }

is there a simpler way to construct and/or test values of type Method?

It seems to me that modelling type Method as an enum might be simpler, easier and more efficient. Am I missing something?

Still loving tiny-http! _Greg

Allow creating servers listening to a range of ports

I'm not sure if this really is an issue or if I just misunderstood the documentation. It does not seem to be possible to have a single server listen to a range of ports (i.e. 80 and 443), only one port per server seems possible.

Example unused result must be used

extern crate tiny_http;

fn main() {
    use tiny_http::{Server, Response};

    let server = Server::http("0.0.0.0:8000").unwrap();

    for request in server.incoming_requests() {
        println!("received request! method: {:?}, url: {:?}, headers: {:?}",
                 request.method(),
                 request.url(),
                 request.headers()
        );

        let response = Response::from_string("hello world");
        request.respond(response);
    }
}

From the example has this warning:

warning: unused result which must be used, #[warn(unused_must_use)] on by default
  --> src/main.rs:16:9
   |
16 |         request.respond(response);
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^

Support chunked transfer

Support multipart/byteranges for requests?

Support for new version Rust

I use rustc 1.0.0-nightly (4874ca36f 2015-01-23 00:18:57 +0000)

The closure syntax has changed. So cargo build fails.

I try to fix these failures, but the following patch still not work:

diff --git a/src/util/task_pool.rs b/src/util/task_pool.rs
index a5e2473..b2917c3 100644
--- a/src/util/task_pool.rs
+++ b/src/util/task_pool.rs
@@ -8,7 +8,7 @@ use std::time::duration::Duration;
 /// A new thread is created every time all the existing threads are full.
 /// Any idle thread will automatically die after 5 seconds.
 pub struct TaskPool {
-    free_tasks: Arc<Queue<Sender<proc():Send>>>,
+    free_tasks: Arc<Queue<Sender<<T: Fn>:Send>>>,
     active_tasks: Arc<AtomicUint>,
 }

which give the following error message:

src/util/task_pool.rs:11:36: 11:37 error: expected `as`, found `:`
src/util/task_pool.rs:11     free_tasks: Arc<Queue<Sender<<T: Fn>:Send>>>,
                                                            ^

Return appropriate error codes to the client in case of an issue with its request

Implement max number of clients before 503

This is not implemented yet

Both links in README to examples are now dead!

Bottom of page:

• https://github.com/tiny-http/tiny-http
  Dead links:
• https://github.com/tiny-http/heroku-tiny-http-hello-world
• https://github.com/tiny-http/crate-deps

Hey everyone, do you have any other examples of tiny-http in action? Especially small examples and demos? The more the merrier!