When backporting patches to LTS branches, we often run into CI failures due to
changes in rust. Newer rust versions add more lints, which break CI. We really
don't want to also have to backport patches that fix CI, so instead, LTS branches
should pin the stable rust version in CI (e.g. #4434).
This PR restructures the CI config files to make it a bit easier to set a specific rust
version in CI.
* Several of tokio's features (e.g. the channel implementation) do not
need a runtime to work, and can be compiled and used for
wasm32-unknown-unknown targets
* This change enables running tests for the `sync` and `macros` features
so that we can note any regressions there
This patch reduces the number of times worker threads wake up without having
work to do in the multi-threaded scheduler. Unnecessary wake-ups are expensive
and slow down the scheduler. I have observed this change reduce no-op wakes
by up to 50%.
The multi-threaded scheduler is work-stealing. When a worker has tasks to process,
and other workers are idle (parked), these idle workers must be unparked so that
they can steal work from the busy worker. However, unparking threads is expensive,
so there is an optimization that avoids unparking a worker if there already exists
workers in a "searching" state (the worker is unparked and looking for work). This
works pretty well, but transitioning from 1 "searching" worker to 0 searching workers
introduces a race condition where a thread unpark can be lost:
* thread 1: last searching worker about to exit searching state
* thread 2: needs to unpark a thread, but skip because there is a searching worker.
* thread 1: exits searching state w/o seeing thread 2's work.
Because this should be a rare condition, Tokio solves this by always unparking a
new worker when the current worker:
* is the last searching worker
* is transitioning out of searching
* has work to process.
When the newly unparked worker wakes, if the race condition described above
happened, "thread 2"'s work will be found. Otherwise, it will just go back to sleep.
Now we come to the issue at hand. A bug incorrectly set a worker to "searching"
when the I/O driver unparked the thread. In a situation where the scheduler was
only partially under load and is able to operate with 1 active worker, the I/O driver
would unpark the thread when new I/O events are received, incorrectly transition
it to "searching", find new work generated by inbound I/O events, incorrectly
transition itself from the last searcher -> no searchers, and unpark a new thread.
This new thread would wake, find no work and go back to sleep.
Note that, when the scheduler is fully saturated, this change will make no impact
as most workers are always unparked and the optimization to avoid unparking
threads described at the top apply.
Re-applies #4377 and fixes the bug resulting in Hyper's double panic.
Revert: #4394
Original PR:
This PR does some refactoring to the current-thread scheduler bringing it closer to the structure of the
multi-threaded scheduler. More specifically, the core scheduler data is stored in a Core struct and that
struct is passed around as a "token" indicating permission to do work. The Core structure is also stored
in the thread-local context.
This refactor is intended to support #4373, making it easier to track counters in more locations in the
current-thread scheduler.
I tried to keep commits small, but the "set Core in thread-local context" is both the biggest commit and
the key one.
Under the hood, the watch channel uses a RwLock to implement reading
(borrow) and writing (send). This may cause a deadlock if a user has
concurrent borrows on the same thread. This is most likely to occur due
to a recursive borrow.
This PR adds documentation to describe the deadlock so that future users
of the watch channel will be aware.
This patch does some refactoring to the current-thread scheduler bringing it closer to the
structure of the multi-threaded scheduler. More specifically, the core scheduler data is stored
in a Core struct and that struct is passed around as a "token" indicating permission to do
work. The Core structure is also stored in the thread-local context.
This refactor is intended to support #4373, making it easier to track counters in more locations
in the current-thread scheduler.
This adds the following methods:
- UdpSocket::set_send_buffer_size
- UdpSocket::send_buffer_size
- UdpSocket::set_recv_buffer_size
- UdpSocket::recv_buffer_size
Currently, the docs.rs documentation for tokio is built without
--cfg tokio_unstable set. This means that unstable features are not shown in
the API docs, making them difficutl to discover. Clearly, we do want to
document the existence of unstable APIs, given that there's a section in
the lib.rs documentation listing them, so it would be better if it was
also possible to determine what APIs an unstable feature enables when
reading the RustDoc documentation.
This branch changes the docs.rs metadata to also pass --cfg tokio_unstable
when building the documentation. It turns out that it's
necessary to separately pass the cfg flag to both RustDoc and rustc,
or else the tracing dependency, which is only enabled in
target.cfg(tokio_unstable).dependencies, will be missing and the build
will fail.
In addition, I made some minor improvements to the docs for unstable
features. Some links in the task::Builder docs were broken, and the
required tokio_unstable cfg was missing from the doc(cfg(...))
attributes. Furthermore, I added a note in the top-level docs for
unstable APIs, stating that they are unstable and linking back to the
section in the crate-level docs that explains how to enable unstable
features.
Fixes#4328
Carl Lerche