tokio-rs / website

Website for the Tokio project

Tokio Website

The website for the Tokio project. Lives at https://tokio.rs.

Besides containing the content for the website, it also includes crates containing the example code used in the tutorial. These crates can be compiled and run.

• hello-tokio
• spawning
• shared-state
• channels
• io
  • echo-server-copy
  • echo-server
• mini-tokio

Contributing

Thinking about contributing? Great! This should help you get the website running locally.

Getting Started

The website is built using Next.js paired with the Bulma CSS framework. You'll need NPM to install the required packages with:

npm install

Next, start the development server:

npm run dev

Then, open http://localhost:3000.

Resources

To learn more about Next.js, take a look at the following resources:

• Next.js Documentation - learn about Next.js features and API.
• Learn Next.js - an interactive Next.js tutorial.
• Bulma documentation - learn about Bulma.

License

This project is licensed under the MIT license.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in Tokio by you, shall be licensed as MIT, without any additional terms or conditions.

You can run our tests by running the commands:

# in doc-test (unstable APIs are needed by some tests)
RUSTFLAGS="--cfg tokio_unstable" cargo +nightly test

# in tutorial-code
cargo test --all

The doc tests verify that all code blocks are valid Rust, and the tutorial-code folder contains the full code examples from the tutorial.

website's Issues

Document how to store TcpClient in a struct

The generic params make it a bit tricky.

Recenter top nav bar

The top navigation bar is supposed to be centered over the content (on desktop at least). It's possible that this was broken in the fixed for mobile. We should fix it.

cc @jefflembeck

Should the "getting started" link go directly to `simple-server`?

I think that could simplify the "call to action" step to get directly into building something. The various parts of the stack could come in the footer of that tutorial?

cc/ @aturon @alexcrichton

Docs: "why not green threads?"

A lot of people wonder why tokio is made the way that it is. Especially with regards to other languages. It'd be worth having some sort of page that lays out the details here, so that someone can understand the basic design in comparison to others.

For example, there are some things that M:N threads do better than tokio, but those fall afoul of core Rust principles. Discussing this in an even-handed way would be very useful.

Bug in example for docs/going-deeper/transports/

Under
Implementing a transport
There is a comma where there should be a semicolon.

let mut buf = [0, 1];
Should be
let mut buf = [0; 1];

Echo server example uses deprecated BoxFuture

After copy-pasting all the code from the example into a file, when compiling I get:

warning: use of deprecated item: removed without replacement, recommended to use a local extension trait or function if needed, more details in #228
  --> src/main.rs:11:31
   |
11 | use futures::{future, Future, BoxFuture};
   |                               ^^^^^^^^^
   |
   = note: #[warn(deprecated)] on by default

warning: use of deprecated item: removed without replacement, recommended to use a local extension trait or function if needed, more details in #228
  --> src/main.rs:83:19
   |
83 |     type Future = BoxFuture<Self::Response, Self::Error>;
   |                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(deprecated)] on by default

warning: use of deprecated item: removed without replacement, recommended to use a local extension trait or function if needed, more details in #228
  --> src/main.rs:88:25
   |
88 |         future::ok(req).boxed()
   |                         ^^^^^
   |
   = note: #[warn(deprecated)] on by default

Favicon not visible with dark browser theme

Because the favicon is purely black and transparent, it's not visible when the tab background is also black. To switch Firefox to a dark theme, select customize in the burger menu. Perhaps a white border could be added, to make the icon visible with any background

Logo in SVG

The logo is provided in two sizes as a PNG. SVG would be more space-efficient and easier to extend for other purposes. Can we please have the source SVG?

Improve story for mobile

We should test and improve the site for display on mobile devices.

Overselling in landing page copy

On the landing page, the first bit of copy reads:

Fast

Tokio's zero-cost abstractions give you bare-metal performance.

I think this is a tad bit misleading: the abstractions aren't entirely zero-cost (e.g., there is a thread pool, that is spawned into, which fundamentally has a cost) and it's certainly not bare-metal performance (because there is an OS under the hood, and it's still performing system calls, copying buffers to the NIC driver, etc).

I'm not suggesting this points to anything wrong with Tokio. Just that as the first thing people read on the site, it slightly oversells. And it doesn't need to. Because Tokio is good enough without doing the impossible!

It looks like that copy is from the very 5th ever commit on the website, nearly two years ago---waaaaaaaaay before 0.1---so it makes sense it wouldn't be entirely prophetic of the state of Tokio today.

I'm proposing a simple update to that particular copy. Maybe something along the lines of "Tokio's low-cost abstractions give you comparable performance to hand-rolled event-driven I/O code in most workloads," but way spiffier.

Perhaps we could brainstorm copy that is the best of: a bold claim; not so bold as to be incorrect; brief and to the point.

IO-patterns reference links to a 403 Forbidden

In the section https://tokio.rs/docs/going-deeper-tokio/core-low-level/#io-patterns the section on tasks link to an erroneous page: https://tokio.rs/docs/going-deeper-tokio/tasks.
It seems the right link must be: https://tokio.rs/docs/going-deeper-futures/tasks/

Add `runtime` doc examples showing how to use from a library

Provide examples showing how a library can encapsulate the Tokio runtime and provide a blocking API.

This requires the library to hold on to the Runtime reference and use fns like block_on to interact with the runtime from outside the runtime context.

This depends somewhat on #328.

Reference Documentation

I think it would be helpful to link directly to the reference documentation for the official Tokio crates. Right now you have to do a search on docs.rs or crates.io and potentially filter through many third-party crates to find what you're looking for.

For example, look at the "Documentation" page https://tokio.rs/docs/getting-started/tokio/ . Somewhere in each of the tokio-proto, tokio-core and futures list items there could be links to https://docs.rs/tokio-proto, https://docs.rs/tokio-core, and https://docs.rs/futures respectively.

Another option might be to add links on the left side of the page under a new "Reference" heading.

I'm willing to work on this if there is any interest.

Please make the website font readable

Currently used sans-serif font is very difficult for eyes to read. Additionally, because futures is not so simple subject, this is an obstacle for me as a learner.

From my point of view, the "Source Serif Pro" used in rust api docs would be ok. It can be added by linking to google fonts.

tokio.rs domain does not resolve

https://tokio.rs is failing to resolve :(

Clarify multiplexing behavior

From IRC:

I'm learning tokio and i wanted to know something, i've seen this line on the Tokio docs : "The current_thread executor multiplexes all spawned tasks on the current thread." What does that means ? There is only one thread, how things can be multiplexed ?

Unify `codec` / `transport` terminology

I think that there are a bunch of places in the website that refer to codec, but we should unify that terminology (transport, or something else).

Add a glossary

I think that https://tokio.rs/docs/getting-started/tokio/ would benefit from a glossary of terms covering tokio-core, tokio-proto, tokio-service, and futures. I think a lot of the terms are already explained well in isolation on various pages, but a single page that brings all the definitions together would be helpful.

Copied from: tokio-rs/tokio-proto#127

Document how to use futures-cpupool with tokio

I'm not 100% sure if there is a section discussing that tokio-core futures must not perform heavy CPU computation... if there isn't, there should be :)

Add threading section

Discuss how the various components work w/ threading.

Add a section about Service middleware

One page for all API docs

We've got a whole bunch of crates and rustdoc has lots of support for supporting groups of crates. We should have a hosted version of the documentation with a sidebar that has all the tokio/futures related crates. This should help facilitate exploration of the project, for example.

Document Multiplex Streaming Protocols

Maybe just an example, as of now there is little documentation on Multiplex Streaming protocols.

Just a few questions myself:

• What are they used for? (Examples).
• Can different 'Message's be sent before a previous 'Body' stream is complete? (I'd assume yes, but there's no clarification that I could find).
• How much overhead is created per new RequestId? (So far it looks like a Body 'Stream' is created every time a new RequestId is created).

I suppose the only necessary thing is an example / a type of protocol Streaming Multiplex would be good for.

Thanks!

Add Read-Timeout example

Hello, based on the example code of this page, I struggled to implement a read-timeout. Finally I looked into the trust-dns project repository to understand how the "select2" combinator should be used.

https://gist.github.com/frehberg/1ab4cc3cf98a3188766d20461f6f87bd

I would like to add the code as example, demonstrating read-timeouts and usage of "select2" combinator.

What do you think?

Add error handling section to futures-mechanics

We forgot to fill it out by accident so I had to remove it.

Mention `Either` in the section about returning futures

https://tokio.rs/docs/going-deeper/returning/

the Either type is helpful in this case.

No license for tokio-rs/website repository.

Not sure if I'm being thick or if there isn't a license file for this repo yet.

I have a small patch I'd like to make to fix a few incorrect links: some of the links in https://tokio.rs/docs/going-deeper-futures/tasks/ which are supposed to link to futures::executor::spawn are instead linking to futures::executor::Spawn but I need the repository to add a license before I can open a PR.

Cannot build Example: serving database content using proto

I cannot build the example in the documentation no matter where I look for the crates. The example doesn't say where to get the crates from (some are in crates.io and some are not), nor what versions the dependencies should have.

Please include Cargo.toml snippets with the example code, to save a lot of time for the new user.

Audit the guides discussing protocols (pipelined / multiplexed) and place emphasis on request / response

I believe that currently most guides discussing protocols take the fact that they are request / response oriented for granted. We should audit the docs and make sure that this is clarified w/ links to how to work with stream oriented protocols.

That being said "streaming pipelined protocols" vs. "stream oriented" is confusing... maybe we should come up w/ better terminology.

Documentation refers to Task::park rather than Task::current

https://tokio.rs/docs/going-deeper-futures/futures-model/ needs to be updated to use the latest terminology, rather than the deprecated and hidden park / unpark.

Broken example in guides

This example isn't correct (do_map needs to be cast)

https://tokio.rs/docs/going-deeper-futures/returning/#named-types

Update API URLs to point to types in the facade instead of sub crates.

Going Deeper Futures Mechanics has broken Streams link

content/docs/going-deeper/futures-mechanics.md:10 has a "streams" link that's attempting to point to "../../getting-started/streams-and-sinks", but there's no such file.

The repo does have a "./content/legacy/getting-started/streams-and-sinks.md", but I'm not sure whether teh link should be updated, or if that's so out of date that the link should be removed?

Direct links are getting responded with 403

For example, https://docs.rs/tokio-io/0.1.6/tokio_io/ has a link to https://tokio.rs/docs/getting-started/core/ which responds with 403.

So does https://tokio.rs/docs/getting-started/ and
https://tokio.rs/docs/

Menu pushes down content on mobile

Every page in the getting started docs requires scrolling way down. Attached a composite screenshot of what https://tokio.rs/docs/getting-started/tokio/ looks like on my phone with Chrome:

Add a section talking about pipelined protocols

Currently, there is no section that is dedicated to discussing what a pipelined protocol is and how it fits in with tokio-proto.

Async database: there is a driver

Hi! While teaching a friend about tokio, I read that there was no async database driver yet.
Turns out I wrote one, called pleingres, and used it successfuly for a few months now on nest.pijul.com. May I write a pull request to replace that page with examples using postgres?

Blog should have Atom or RSS feed

I just discovered the new Tokio website today and saw that there's an associated blog. I'd like to keep up with developments in Tokio, but it seems there's no Atom or RSS feed for the blog that would make that easy to do.

I don't know what that would take in Hugo to implement, so this is more of a "nice to have" than anything.

License under dual MIT/Apache-2.0

The repository currently does not have an associated license, preventing some from contributing.

In order to fully relicense this repository, all contributors must sign-off.

You are receiving this notification because you have contributed to this repo.

While smaller changes can't be copyrighted by law, its non-trivial to find out with certainty whether a given change falls under copyright or not, due to the nature of the matter. Therefore I'm asking you to agree to the new terms even if you consider your contributions to be not copyrightable.

Checkoff

To agree to the licensing terms, please comment with:

I license past and future contributions under the dual MIT/Apache-2.0 license, allowing licensees to chose either at their option.

Thank you!

• @alexcrichton
• @aturon
• @carllerche
• @nelsonjchen
• @hockeybuggy
• @rbalicki2
• @shepmaster
• @RadicalZephyr
• @munckymagik
• @khaledkbadr
• @funkill
• @Rufflewind
• @vi
• @mbrubeck
• @radupopescu
• @irtefa
• @SuperFluffy
• @theduke
• @phrohdoh
• @tafia
• @sheeldotme
• @kanerogers
• @braddunbar
• @lazyval
• @toddWannaCode
• @oconnor663
• @mohammadsamir
• @hngnaig
• @DirkyJerky
• @mayhewj
• @jasondavies
• @DmitryBochkarev
• @zmanian
• @baloo
• @schleumer
• @jbendig
• @mre
• @dfockler
• @ineol
• @daschl
• @srijs
• @Freyskeyd
• @danobi
• @kw217
• @AaronStGeorge
• @jedisct1
• @CrockAgile
• @tanimoto
• @awelkie
• @matematikaadit
• @rofrol
• @jefflembeck
• @mquandalle
• @iamcodemaker
• @zklapow
• @schuster
• @acobster
• @cgwalters
• @P-E-Meunier
• @mgattozzi
• @steveklabnik
• @dwrensha
• @redaready

Document back pressure strategies

There is a little snippet in the future doc, but we could elaborate a lot on it.

Unclear how to handle multiple calls to poll in a Future impl

I'm trying to implement the Future trait for another type (as it happens, one which implements its own flavour of futures in C). I've followed the documentation in https://tokio.rs/docs/going-deeper-futures/futures-model/ but it's unclear on one point: how exactly should I store the task(s) I receive from task::current()?

The documentation currently says

Blocking a future is a matter of using park to get a handle to its task, putting the resulting Task in some wakeup queue for the event of interest, and returning NotReady.

"In a queue" suggests to me that there might be more than one task in that queue. But just above it says

Crucially, though, the task instance stays fixed for the lifetime of the future it is executing—so no allocation is needed to create or install this callback.

So when my poll wants to return NotReady, should I push the task onto a queue, and arrange to notify all those tasks when I become ready? Or should I keep at most one task, dropping any existing one? I see there's even a method Task::will_notify_current - should I check if any of the tasks already on the queue are already current, and only push the current task onto the queue if this is a new one?

Looking at tokio-core's ScheduledIo and TimeoutState it looks like the intention is that I should simply keep one, i.e., I should store an Option<Task> within my object; when my poll wants to return NotReady I set it to Some(task::current()), and when it becomes ready I notify the task if present. Is that right?

It would be great if you could clarify this area of the docs to make it clear how tasks should be handled by futures. Thanks!

Document implementing custom future.

I think it's good to have a documentation on implementing a custom future. Especially because the current Future trait API is not exactly intuitive rust-lang/futures-rs#129. Let's come up with a good example usecase and document it.

complete is deprecated: renamed to `send`

Hi, I copied the code from the oneshot example, and I tried to compile it.

This is the code I have in main:

extern crate futures;

use std::thread;
use futures::Future;
use futures::sync::oneshot;

fn expensive_computation() -> u32 {
    200
}

fn main() {
    let (tx, rx) = oneshot::channel();

    thread::spawn(move || {
        tx.complete(expensive_computation());
    });

    let rx = rx.map(|x| x + 3);
    let result = rx.wait().unwrap();
    assert_eq!(result, 203);
}

This is what I get from the compiler:

$ cargo check
   Compiling oneshot v0.1.0 (file:///home/real/projects/learn_tokio/oneshot)
warning: use of deprecated item: renamed to `send`
  --> src/main.rs:15:12
   |
15 |         tx.complete(expensive_computation());
   |            ^^^^^^^^
   |
   = note: #[warn(deprecated)] on by default

If I change complete to send I get a different warning:

$ cargo check
   Compiling oneshot v0.1.0 (file:///home/real/projects/learn_tokio/oneshot)
warning: unused `std::result::Result` which must be used
  --> src/main.rs:15:9
   |
15 |         tx.send(expensive_computation());
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: #[warn(unused_must_use)] on by default

We should probably change the code there to compile nicely and without warning. What do you suggest?

Edit:

$ rustc --version
rustc 1.21.0-nightly (df511d554 2017-08-14)

Enable cron job build for Tokio website?

How about master branch, weekly, and do not run if there has been a build of master recently?

The changes I put forth in #105 would be much stronger with this option enabled.

https://docs.travis-ci.com/user/cron-jobs/

This is something I can't make a PR for.

TLS Error

Since today is 01/10 I figured I would mention this issue. When I click on https://tokio.rs I get an insecure site warning and then upon proceeding get redirected to a random LinkendIn page.

rustc complains on sample code

https://github.com/tokio-rs/website/blame/master/content/docs/getting-started/pipeline-server.md#L158

error[E0053]: method `call` has an incompatible type for trait
   --> src/main.rs:148:5
    |
148 |       fn call(&mut self, input: String) -> Self::Future {
    |  _____^ starting here...
149 | |         future::ok(input).boxed()
150 | |     }
    | |_____^ ...ending here: types differ in mutability
    |
    = note: expected type `fn(&EchoService, std::string::String) -> Box<futures::Future<Error=std::io::Error, Item=std::string::String> + std::marker::Send + 'static>`
    = note:    found type `fn(&mut EchoService, std::string::String) -> Box<futures::Future<Error=std::io::Error, Item=std::string::String> + std::marker::Send + 'static>`

error: aborting due to previous error

I fixed it by removing mut for self.

Streams link on Essential Combinators links to a 403

There's a link on the first paragraph of https://tokio.rs/docs/going-deeper/futures-mechanics/, it points to a 403: https://tokio.rs/docs/getting-started/streams-and-sinks which returns a 403.

I can't figure out if the page doesn't exist, or if the link points to the wrong place.

Googling "Rust Tokio tutorial" leads to wrong place

Expecting it to get at least to the Getting Started documentation pages, but actually the first link leads to a related, but clearly insufficient repository.

Document stateful Codec implementations

I don't think that the current documentation shows how to manage state in the codec. This has come up a few times in Gitter.

tasks section: "on the current thread" doesn't explain that notify is never called

This section doesn't explain that running a task that blocks (and thus returns NotReady) will deadlock when called with .wait(). This seems perfectly "obvious" and yet the absence of the explicit callout lead me to spend a bunch of time looking through the source and writing a test to convince myself that indeed this deadlocks:

#[test]
fn deadlock() {
    use std::time::{Duration,Instant};

    let start = Instant::now();
    println!("starting at {:?}!", &start);
    let future = future::poll_fn::<_, u32, _>(move || {
        let now = Instant::now();
        println!("being polled at {:?}!", &now);
        if now.duration_since(start) < Duration::from_secs(1) {
            Ok(Async::NotReady)
        } else {
            Ok(Async::Ready(1))
        }
    });
    assert_eq!(future.wait().unwrap(), 1)
}

It might be worth calling this out explicitly.