Introduction

Problem

Understanding async Rust gives you a hard time.

Future

What’s a Future

A future is an object which represents a value, a value that becomes ready “some time later.”

Future as function result

Therefore, futures can be used to represent function results when those results can only become ready “some time later.” This usage of future separates results from functions.

Diagram 1: Represented by a future, the result bar is separated from its function foo.

This separation provides new ways to process ready-some-time-later results, making async Rust possible.

Diagram 2: Obtaining bar through a future.

Scenarios

Futures, while they can represent any type of values, are often used in specific scenarios.

One of the scenarios is I/O operation, such as reading from the Internet, or writing to a local file system, etc.

Diagram: Function foo obtains data through the BSD socket interface read in nonblocking mode.

std::future::Future

Rust futures are std::future::Futures. Types which implement the trait expose the represented values through the method poll. The method returns a Enum Poll: Ready(T) when the value is ready, or Pending if not.

Poll definition:

pub enum Poll<T> {
    Ready(T),
    Pending,
}

Diagram: Obtaining value of a std::future::Future.

Polling in a loop

To obtain the value of a Future, one obvious approach is to call the method poll in a loop until it returns Poll::Ready(T).

Example

“UDP Hello” examples

The “UDP Hello” examples, which run on UDP protocol, send data b"hello"(repeatedly) to a given echo service and receive the response(s).

The examples include “loop-poll-hello” and “loop-poll-many-hello”.

`Recv`

Recv, as a major part of the “UDP Hello” examples, is a Future which receives one response from a given service. Its poll implementation reads data from a given UDP socket. Recv requires and assumes that the UDP socket is in nonblocking mode.

Source: examples/src/recv.rs


use std::future::Future;
use std::io;
use std::net::UdpSocket;
use std::task::Poll;

/// Future which receives one response from a given service.
pub struct Recv {
    socket: UdpSocket,
}

impl Recv {
    /// Creates a new Recv.
    ///
    /// # Safety
    ///
    /// Caller must ensure that `socket` has been moved into nonblocking mode.
    #[must_use]
    pub unsafe fn new(socket: UdpSocket) -> Self {
        Self { socket }
    }
}

impl Future for Recv {
    type Output = Vec<u8>;

    fn poll(
        self: std::pin::Pin<&mut Self>,
        _cx: &mut std::task::Context<'_>,
    ) -> Poll<Self::Output> {
        let mut buf = [0; 1024];
        match self.socket.recv(&mut buf) {
            Ok(n) => Poll::Ready(buf[..n].to_vec()),
            Err(ref e) if e.kind() == io::ErrorKind::WouldBlock => std::task::Poll::Pending,
            Err(e) => panic!("IO error: {e}"),
        }
    }
}

Example: “loop-poll-hello”

“loop-poll-hello” sends b"hello" to a given echo service, creates a Recv, and calls poll of the Recv until Poll::Ready(Vec<u8>) is returned.

Source: examples/src/bin/loop-poll-hello.rs

Example: “loop-poll-many-hello”

“loop-poll-many-hello” repeatedly sends b"hello" to a given echo service for 1000 times, creates a vector of 1000 Recvs, and calls poll of each Recv until all returns Poll::Ready(Vec<u8>).

Source: examples/src/bin/loop-poll-many-hello.rs

The echo service: “Lazy Echo”

“Lazy Echo” is an UDP echo service: the service sends received data back to where it comes from. The service waits one second before any sending, hence lazy.

“Lazy Echo” has two implementations: “lazy-echo-udp-smol” and “lazy-echo-udp-tokio” – same behavior but with different async runtimes.

“lazy-echo-udp-smol” source: examples/src/bin/lazy-echo-udp-smol.rs
“lazy-echo-udp-tokio” source: examples/src/bin/lazy-echo-udp-tokio.rs

Run the examples

Start the echo service, either “lazy-echo-udp-smol” or “lazy-echo-udp-tokio”
- In the “examples” directory, execute:
```
cargo run --bin lazy-echo-udp-smol
```
  OR
```
cargo run --bin lazy-echo-udp-tokio
```
- Keep the service running
Run “loop-poll-hello”. In the “examples” directory, execute:
```
cargo run --bin loop-poll-hello
```
Run “loop-poll-many-hello”. In the “examples” directory, execute:
```
cargo run --bin loop-poll-many-hello
```

NOTE

“loop-poll-hello” and “loop-poll-many-hello” should both finish in about 1 second.

Problems

Wastes CPU time

As a Future has to be polled repeatedly and aggressively in a loop, CPU time is wasted when data isn’t ready.

Extreme example: when the data of a Future never becomes ready, the polling loop is infinite.

Doesn’t scale

As all Futures have to be “polled” one after another, when a Future becomes ready, its data may not be polled promptly.

Extreme example: when there are 1000 Futures and only the last one is ready, a program has to poll 999 times before the last one can be accessed.

Waker

A waker is an object which associates to a given future. The waker can be used to have its associated future “polled”.

Diagram: A waker which leads to its associated future being “polled” when the data is ready.

The “Waker” pattern provides a solution to the problems that “polling in a loop” raises.

Problem: “polling in a loop” wastes CPU time.

Solution: instead of polling the future repeatedly, we can rely on a waker to have the future polled when the data is ready.
Problem: “polling in a loop” doesn’t scale.

Solution: for a given future, when its data is ready, the waker associates to the future will have the exact future polled “soon enough”, if not immediately.

Questions

How can a waker effectively knows when its future’s data is ready?
How can a waker have its future “polled”?
How to interact with data obtained through the “Waker” pattern?

Reactor, handling I/O event notifications

Question

How can a waker efficiently knows when its data is ready?

Answer

“efficiently” means no loop. In other words, a waker should be notified when its data is ready. Such effectiveness is often achieved by using the notification mechanisms provided by the operating systems.

Case

For futures which represent results of I/O operations, the “I/O event notification mechanisms” provided by the operating systems can be used to notify the wakers.

In this case, the wakers interact with some specific code that waits for the I/O event notifications. This specific code is often called “reactor”.

Diagram: A waker driven by a reactor.

Waking, pushing futures to a polling queue

Question

How can a waker have its future “polled”?

Answer

A waker calls specific “waking” code when the waker is notified that its data is ready. The “waking” code locates the future that the waker associates to, and pushes the future to a “polling queue”. A waker’s job is finished after “waking”.

With a “polling queue”, wakers are decoupled from “external code” which pops futures from the queue, polls the futures and dispatches data.

Diagram: a waker wakes.

Channel, passing results

Question

How to interact with the “Waker” pattern?

Blocking

Execution of “normal Rust code” blocks. It blocks until data is returned, and continues with the obtained data.

Diagram 1: “Polling in a loop” blocks.

Nonblocking

Execution of the “Waker” pattern does not block.

Diagram 2: Wakers put code execution in nonblocking mode.

The question, again

How to interact with the “Waker” pattern, as the code execution is nonblocking?

Answer

Solution 1: blocks the current thread until data is obtained

Parks the current thread, runs wakers and futures in dedicated threads, and unpark the suspended thread when data is ready.

Solution 2: interacts with data using the “channel” mechanism

“Channel” is a communication mechanism, allowing data to be sent and received across threads. The mechanism can often work in both blocking and nonblocking modes.

Using the “channel” mechanism, data can be sent across threads when it’s ready and obtained, and will be received in a specific thread, where the data processing starts.

Lifecycle

A waker’s life begins before a future is polled, and ends when the waker wakes. Details:

Creating.

Before a future can be polled, a waker which associates to the future is created.
Passing.

When the future is polled, the waker is passed to it.
Registering.

During the polling, if the data is returned (as its ready), nothing is done to the waker and its life ends.

If the data isn’t ready, the waker is “registered”. Once registered:
- The waker is stored.
- The waker starts waiting for data through a given reactor.
NOTE

The registering repeats for each poll of the future, meaning that:
- The stored waker will be replaced.
- Multiple polls lead to a single waking of the “current waker”.
Waking.

When the reactor notifies the waker that its data is ready, the waker wakes.

std::task::Waker

Rust wakers are std::task::Wakers. A Waker wraps a std::task::RawWaker which provides the waking functionality.

Reasons behind the “delegation”:

RawWakers are unsafe to use. By wrapping RawWakers in Wakers, it possible to avoid the unsafe part.
std::task::LocalWaker (nightly) is the thread unsafe version of Waker, meaning that a LocalWaker have to be accessed from the same thread where it was created. The wrapping enables code sharing between Waker and LocalWaker.

std::task::Context

Polling a Future is to call its method poll, passing a Waker which is wrapped in a std::task::Context. While “currently, Context only serves to provide access to a &Waker”, it’s possible to add more fields to Context when there is need to pass additional data to poll(). The API was designed this way for future expanding in a backwards-compatible way.

std::task::RawWaker

RawWakers function like Trait Objects. The RawWakerVTable provides a common behavior, and the *const () pointer stores arbitrary data that the common behavior applies on.

While inconvenient, this “vtable strategy” has its benefits:

Breaks away from the requirements that dyn compatibility (formerly “object safety”) enforces.
Enables reducing allocations.

Examples

Example: “reactor-hello”

Source: examples/src/bin/reactor-hello.rs

Example: “waker-many-hello”

Source: examples/src/bin/waker-many-hello.rs

Run the examples

Start the echo service, either “lazy-echo-udp-smol” or “lazy-echo-udp-tokio”
- In the “examples” directory, execute:
```
cargo run --bin lazy-echo-udp-smol
```
  OR
```
cargo run --bin lazy-echo-udp-tokio
```
- Keep the service running
Run “reactor-hello”. In the “examples” directory, execute:
```
cargo run --bin reactor-hello
```
Run “waker-many-hello”. In the “examples” directory, execute:
```
cargo run --bin waker-many-hello
```

NOTE

“reactor-hello” and “waker-many-hello” should both finish in about 1 second.

std::pin::Pin

The Future method poll involves std::pin::Pin. A Pin is a struct which wraps a pointer to a given value.

When a pointer is wrapped in a Pin, its access is restricted. The restriction prevents the value, which the pointer refers to, from being “moved” (change of memory location).

The problem `Pin` solves

A Future may define fields which refer to the future itself. The references will become invalidated when the containing future moves.

To allow such “self-reference”, Futures must not move, hence the requirement of Pin for poll.

Runtime

A runtime provides what Rust left. Only with a runtime, async Rust can actually be used to program.

Features

Runtimes are all third-party. Some of the major features they provide:

Future implementations.
Reactor implementations.
Managing Futures.
Managing Wakers.
Scheduling Future polling.
Utilities.

Exercises

Write function waker_fn which “converts a closure into a Waker”.
Write function waker_unpark which creates a waker. The waker unparks a given thread when waking.
Write function block_on which “blocks the current thread on a future”.
Rewrite Recv to support the Waker pattern. Name the new struct RecvWithWaker.
Rewrite “reactor-hello” using RecvWithWaker and block_on.

Solutions

Composing Futures

A Future may compose with other Futures in order to be used. In other words, before such a future returns its data, it needs to access and process data of other given futures.

Diagram: Future A composes with Future B, and Future B composes with Future C and D.

Example: continue

Future A waits for Future B, and continues with data obtained from B.

Code

See examples/src/bin/continue-hello.rs

Example: join

Future B joins futures C and D, waiting for both to complete.

Code

See examples/src/bin/join-hello.rs

NOTE

C and D shares the same Waker.

Example: select

Future D selects future E, F, and G, waiting for any of the three to complete.

Code

See examples/src/bin/select-hello.rs

Future Tree

When a Future compose with other Futures, these “other Futures” may also be composed with more Futures, hense constructing a Future “tree”.

Diagram: a Future tree that puts A, B and D together.

Procedural

A Future (tree) is procedural.

Externally, as a whole, a Future is “single-threaded” – at any given time, a Future should be polled in one thread.

Internally, as a tree of Futures, the Futures involved in the “current external poll” should be polled one by one, in the given thread.

`async` and `await`

async turns a function into a Future. await, which is “valid only within an async context”, composes Futures.

Examples

`async`

An async-prefixed function foo as defined in its original form:
```
async fn foo(v: usize) -> usize {
    // ...
}
```

The pseudocode mimics what the compiler generates:

fn foo(v: usize) -> impl Future<Output = usize> {
    // ...
}

`await`

A Future “awaits” in its original form:

// ...
// within an async context
// ...
let v = future.await;
// ...

The pseudocode mimics what the compiler generates (the keyword yield ties to a Coroutine):

// ...
let mut future = pin!(future);
let v = loop {
    match future.as_mut().poll(cx) {
        Poll::Ready(output) => break output,
        Poll::Pending => yield,
    }
};
// ...

Closure and block

async and await can also be applied to a closure or a block.

Size

An async generated Future (tree) is “the perfectly sized stack”, with everything it needs across yields.

The lint large_futures checks for such sizes, as they can become unexpectedly large. You’ll have to enable the lint manually as it’s in the group clippy::pedantic.

See examples/src/bin/large-future.rs for an example. To examine the warning, in the “examples” directory, execute:

cargo clippy -- -W clippy::pedantic

Exercises

Rewrite “continue-hello” using async and await.

Solutions

See examples/src/bin/async-continue-hello.rs

Task

Internally

Runtimes use Tasks internally to manage Futures. A Task is an object which contains a Future and “everything else” needed to manage the Future.

Externally

While “Task” is internal to a runtime, knowing that it exists can help us understand the runtimes better, and thus use them well.

Like a Future, a Task is also procedural. To have Futures polled in a “multi-threaded” way, they must be spawned (see smol::spawn or tokio::task::spawn).

Hello async Rust