The `pipe_mode` function would erase any previously set configuration
option that is specified using the pipe_mode fit field. This patch fixes
the builder to maintain the bit field when changing the pipe mode.
The global flag remains and is used to prevent duplicated shutdowns and
new io operations after shutdown.
On shutdown, the driver flips the shutdown flag of every pending io
operations and wake them to fail with a shutdown error.
Fixes: #5227
- Use dtolnay/rust-toolchain instead of actions-rs/toolchain
- Use cargo/cross directly instead of actions-rs/cargo
- Use rustsec/audit-check instead of actions-rs/audit-check
This patch simplifies the implementation of `task::Id` by moving
conditional compilation into the `AtomicU64` definition. To handle
platforms that do not include `const fn Mutex::new()`, `StaticAtomicU64`
is defined as always having a `const fn new()`. `StaticAtomicU64` is
implemented with `OnceCell` when needed.
This adds CI coverage for a couple of code paths that are not currently
hit in CI:
* no `const fn Mutex::new`
* no `AtomicU64`
This is done by adding some new CFG flags used only for tests in order
to force those code paths.
Adds a method `Builder::max_io_events_per_tick()` to the runtime builder. This can be used to configure the capacity of events that may be processed per OS poll.
Raise the mio-aio dev dependency, which transitively brings in Nix, to
ensure that the tests will continue to compile if libc switches from a
FreeBSD 11 ABI to a FreeBSD 12 one.
Readiness futures inconsistently return the current readiness of an I/O resource if it is immediately available, or all readiness relevant for the given `Interest`, if a future needs to wait. In particular, it always returns `read_closed` for `Interest::READABLE` and `write_closed` for `Interest::WRITABLE`, which often is not true. Tokio should not tolerate false positives for `*_closed` events because they are considered final states and are not cleared internally.
In the case of an `io_resource.ready(Interest::READABLE | Interest::WRITABLE)` call, this behavior may also lead to false positives of other events.
## Solution
Follow the same strategy as `poll_ready` and return the current resource's readiness.
Closes: #5098
There's already an existing warning about this combo in the
documentation for `Handle::block_on`. This commit adds a summarized
version in `Runtime::handle`.
PR #5223 changed the behavior of `yield_now()` to store yielded tasks
and notify them *after* polling the resource drivers. This PR fixes a
couple of bugs with this new behavior when combined with
`block_in_place()`.
First, we need to avoid freeing the deferred task queue when exiting a
runtime if it is *not* the root runtime. Because `block_in_place()`
allows a user to start a new runtime from within an existing task, this
check is necessary.
Second, when a worker core is stolen from a thread during a
`block_in_place()` call, we need to ensure that deferred tasks are
notified anyway.
Previously, calling `task::yield_now().await` would yield the current
task to the scheduler, but the scheduler would poll it again before
polling the resource drivers. This behavior can result in starving the
resource drivers.
This patch creates a queue tracking yielded tasks. The scheduler
notifies those tasks **after** polling the resource drivers.
Refs: #5209
This PR should hopefully reduce the amount of code generated per
future-type spawned on the runtime. The following methods are no longer generic:
* `try_set_join_waker`
* `remote_abort`
* `clone_waker`
* `drop_waker`
* `wake_by_ref`
* `wake_by_val`
A new method is added to the vtable called schedule, which is used when a task
should be scheduled on the runtime. E.g. wake_by_ref will call it if the state change
says that the task needs to be scheduled. However, this method is only generic over
the scheduler, and not the future type, so it also isn't generated for every task.
Additionally, one of the changes involved in the above makes it possible to remove
the id field from JoinHandle and AbortHandle.
This patch updates CI to use `cross` to run Tokio tests on virtualized
ARM and i686 VMs. Because ipv6 doesn't work on Github action running in
a docker instance, those tests are disabled
`LocalSet` cleans up any tasks that have not yet been completed when it is
dropped. Previously, this cleanup process required access to a thread-local.
Suppose a `LocalSet` is stored in a thread-local itself. In that case, when it is
dropped, there is no guarantee the drop implementation will be able to
access the internal `LocalSet` thread-local as it may already have been
destroyed.
The internal `LocalSet` thread local is mainly used to avoid writing unsafe
code. All `LocalState` that cannot be moved across threads is stored in the
thread-local and accessed on demand.
This patch moves this local-only state into the `LocalSet`'s "shared" struct.
Because this struct *is* `Send`, the local-only state is stored in `UnsafeCell`,
and callers must ensure not to touch it from other threads.
A debug assertion is added to enforce this requirement in tests.
Fixes#5162
An earlier change updated `enter_runtime` to also set the current
handle. However, the change did not store the `SetCurrentGuard`, so the
"current handle" was immediately unset. This patch stores the
`SetCurrentGuard` in the `EnterRuntimeGuard`.
No existing test exposed this bug because all tests went via `Runtime`
instead of `Handle`. Currently, `Runtime` is still explicitly setting
the handle before entering runtime, so all tests still passed. A new
test is added that covers the case of calling `Handle::block_on` and
accessing the current handle.
A previous patch moved code related to entering a runtime into the
context module but did not change anything. This patch combines both
thread-local variables.
This moves the functions, types, and thread-local related to entering a
runtime into the context module. This does not yet unify the thread-local
variables, as that, will be done in a follow-up PR.
Instead of each scheduler flavor holding a reference to the scheduler
handle, the scheduler handle is passed in as needed. This removes a
duplicate handle reference in the `Runtime` struct and lays the
groundwork for further handle struct tweaks.
Publish the blocking thread pool metrics as thread-safe values, written
under the blocking thread pool's lock and able to be read in a lock-free
fashion by any reader.
Fixes#5156
This is a first step towards unifying the concepts of "entering a
runtime" and setting `Handle::current`.
Previously, these two operations were performed separately at each call
site (runtime block_on, ...). This is error-prone and also requires
multiple accesses to the thread-local variable. Additionally, "entering
the runtime" conflated the concept of entering a blocking region. For
example, calling `mpsc::Receiver::recv_blocking` performed the "enter
the runtime" step. This was done to prevent blocking a runtime, as the
operation will panic when called from an existing runtime.
To untangle these concepts, the patch splits out each logical operation
into functions. In total, there are three "enter" operations:
* `set_current_handle`
* `enter_runtime`
* `enter_blocking_region`
There are some behavior changes with each function, but they
should not translate to public behavior changes. The most significant is
`enter_blocking_region` does not change the value of the thread-local
variable, which means the function can be re-entered. Since
`enter_blocking_region` is an internal-only function and we do not
re-enter, this has no public-facing impact.
Because `enter_runtime` takes a `&Handle` to combine the
`set_current_handle` operation with entering a runtime, the patch
exposes an annoyance with the current `scheduler::Handle` struct layout.
A new instance of `scheduler::Handle` must be constructed at each call
to `enter_runtime`. We can explore cleaning this up later.
This patch also does not combine the "entered runtime" thread-local
variable with the "context" thread-local variable. To keep the patch
smaller, this has been punted to a follow-up change.
This patch consolidates the budget thread-local state with the
`runtime::context` thread local. This reduces the number of thread-local
variables used by Tokio by one.
The `LocalSet::run_until` future is just a "plain" future that should
run on a runtime that already has a coop budget. In other words, the
`run_until` future should not get its own budget but should inherit the
calling task's budget. Getting this behavior is done by removing the
call to `budget` in `run_until`.
This is a step towards unifying thread-local variables. In the future,
`coop` will be updated to use the runtime context thread-local to store
its state.
This patch does some internal renames to remove some confusion.
* `allow_blocking` is renamed to `allow_block_in_place` to indicate that
the variable only impacts the `block_in_place()` function.
* `context::try_enter` is renamed to `context::try_set_current` to
disambiguate between the various "enter" functions. This function only
sets the runtime handle used by Tokio's public APIs. Entering a runtime
is a separate operation. # Please enter the commit message for your
changes.
* `scheduler::Handle::enter()` is removed to consolidate methods that
set the current context.
This patch is the first step towards unifying all the thread-local
variables spread out across Tokio. A new `Context` struct is added which
will be used to replace the various thread-locals that exist today.
Initially, `Context` only holds the current runtime handle and the
random number generator. Further PRs will add other thread-local state.
A previous PR removed `runtime::context`. At that time,
`runtime::context` was used as an extra layer to access the various
runtime driver handles. This version of `runtime::context` serves a
different purpose (unifying all the thread-locals).
While setting the random number generator seed with the multi-threaded
scheduler does result in deterministic behavior related to the random
number generator, threads still introduce non-determinism, making it
hard (impossible?) to test this. There also is little value in doing so.
This patch also updates the docs to remove mention of work stealing.
The signal driver still uses an `Arc` internally to track if the driver
is still running, however, using the `scheduler::Handle` to access the
signal driver handle lets us delete some code.
The improvement to the `rng_seed` tests added in #5075 missed a case in
the `rt_threaded` tests which was still checking for a specific value.
As described in that PR, this makes the tests fragile and changing tokio
internals may require updating the test.
This change fixes that half-implemented improvement so that the tests no
longer depend on the exact internal ordering, but rather compare two
runs of separate runtimes built with the same seed to check that the
results are the same.
This is the start of applying a similar treatment to the I/O driver as
the time driver. The I/O driver will no longer hold its own reference to
the I/O handle. Instead, the handle is passed in when needed.
This patch also moves the process driver to the `runtime` module.
The signal driver uses a `UnixStream` to receive signal events.
Previously, the signal driver used `PollEvented` internally to receive
events on the `UnixStream`. However, using `PollEvented` from
within a runtime driver created a circular link between the runtime and
the `PollEvented` instance.
This patch replaces `PollEvented` usage in favor of accessing the I/O
driver directly. The I/O driver now reserves a token for signal-related
events and tracks signal readiness internally. The signal driver queries
the I/O driver to check for signal-related readiness.
The signal feature only requires a driver with unix platforms. The
unix signal driver uses the I/O driver. A future refactor would like to
make the signal driver use internal APIs of the I/O driver. To do this,
the signal driver must be moved to the runtime module.
This patch removes a handle to the internal runtime driver handle held
by the runtime. This is another step towards reducing the number of Arc
refs across the runtime internals. Specifically, this change is part of
an effort to remove an Arc in the time driver itself.
This excludes the initial let statement of the proc-macro expansion
from receiving the last statement spans to aid completions in rust-analyzer.
The current span confuses rust-analyzer as it will map the tail expression
tokens to the let keyword (as this is the first token it finds with the
same span) which currently breaks completions. This commit should not
degrade the initial intent of the span reusages, as the type mismatch
parts are still spanned appropriately.
The `LocalSet` implementation includes a conditional compilation clause
that removes the `const` statement from the `thread_local` definition.
However, there already is an internal macro that does this:
`tokio_thread_local`.
This patch removes the conditional compilation in favor of using the
`tokio_thread_local` macro. This also fixes a conditional compilation
issue with an internal utility (`RcCell`).
Tokio maintains an internal thread_local macro that abstracts some
conditional build logic. Before this patch, the macro was named the same
as the `std` macro (`thread_local`). This resulted in confusion as to
whether or not the internal macro or the std macro was being called.
This patch renames the internal macro to `tokio_thread_local` making it
more obvious.
Motivation
Currently, when a task spawned on a `LocalSet` is woken by an I/O driver
or time driver running on the same thread as the `LocalSet`, the task is
pushed to the `LocalSet`'s locked remote run queue rather than to its
unsynchronized local run queue. This is unfortunate, as it
negates some of the performance benefits of having an unsynchronized
local run queue. Instead, tasks are only woken to the local queue when
they are woken by other tasks also running on the local set.
This occurs because the local queue is only used when the `CONTEXT`
thread-local contains a Context that's the same as the task's
`Schedule` instance (an `Arc<Shared>`)'s Context. When the `LocalSet`
is not being polled, the thread-local is unset, and the local run queue
cannot be accessed by the `Schedule` implementation for `Arc<Shared>`.
Solution
This branch fixes this by moving the local run queue into Shared along
with the remote run queue. When an `Arc<Shared>`'s Schedule impl wakes
a task and the `CONTEXT` thread-local is None (indicating we are not
currently polling the LocalSet on this thread), we now check if the
current thread's `ThreadId` matches that of the thread the `LocalSet`
was created on, and push the woken task to the local queue if it was.
Moving the local run queue into `Shared` is somewhat unfortunate, as it
means we now have a single field on the `Shared` type, which must not be
accessed from other threads and must add an unsafe impl `Sync` for `Shared`.
However, it's the only viable way to wake to the local queue
from the Schedule impl for `Arc<Shared>`, so I figured it was worth
the additional unsafe code. I added a debug assertion to check that the
local queue is only accessed from the thread that owns the `LocalSet`.
## Motivation
It turns out that `LocalSet` has...never actually passed Miri. This is
because all of the tests for `LocalSet` are defined as integration tests
in `tokio/tests/task_local_set.rs`, rather than lib tests in
`tokio/src`, and we never run Miri for integration tests.
PR #5095 added a new test reproducing an unrelated bug in `LocalSet`,
which had to be implemented as a lib test, as it must make internal
assertions about the state of the `LocalSet` that cannot be tested with
just the public API. This test failed under Miri, and it turns out that
the reason for this is unrelated to the change in #5095 --- `LocalSet`
uses the `LocalOwnedTasks` type, which turns out to contain a stacked
borrows violation due to converting an `&Header` into an
`NonNull<Header>` when removing a task from the `LocalOwnedTasks` list.
## Solution
Fortunately, this was actually quite easy to fix. The non-`Local`
version of `OwnedTasks` already uses `Task::header_ptr()` to get a
`NonNull<Header>` in its version of `remove`, which avoids this issue.
Therefore, the fix was as simple as updating `LocalOwnedTasks` to do the
same.
I've also added a very simple lib test in `src/task/local_set.rs` that
just creates a `LocalSet` and spawns a single task on it under a
current-thread runtime. This test fails under Miri without the fix in
this PR: https://github.com/tokio-rs/tokio/actions/runs/3237072694/jobs/5303682530
We should probably keep the test so that we're always testing at least
the most trivial `LocalSet` usage under Miri...
The original tests for the `Builder::rng_seed` added in #4910 were a bit
fragile. There have already been a couple of instances where internal
refactoring caused the tests to fail and need to be modified. While it
is expected that internal refactoring may cause the random values to
change, this shouldn't cause the tests to break.
The tests should be more robust and not be affected by internal
refactoring or changes in the Rust compiler version. The tests are
changed to perform the same operation in 2 runtimes created with the
same seed, the expectation is that the values that result from each
runtime are the same.
When 64-bit atomics are supported, use 32-bit queue indices. This
greatly improves resilience to ABA and has no impact on performance on
64-bit platforms.
Fixes: #5041
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds `#[track_caller]` to two public APIs in tokio task
module which weren't added in #4848.
* `tokio::task::block_in_place`
* `tokio::task::spawn_local`
These APIs had call stacks that went through closures, which is a use
case not supported by `#[track_caller]`. These two cases have been
refactored so that the `panic!` call is no longer inside a closure.
Tests have been added for these two new cases.
Refs: #4413
This patch removes all internal `Unpark` runtime types. The runtime now
uses the `Handle` types (`runtime::io::Handle` and
`runtime::time::Handle`) to signal threads to unpark.
Without separate `Unpark` types, future patches will be able to remove
more `Arc`s used inside various drivers.
This patch removes the driver Unpark handle out of the multi-thread
parker and passes a reference in when it is needed. This is a first step
towards getting rid of the separate driver unpark handle in favor of
just using the regular driver handle. Because the regular driver handle
is owned at a higher level (at the top of the worker struct).
This is the equivalent as tokio-rs/tokio#5023, but for the MultiThread
scheduler. This patch updates `MultiThread` to use `Arc<Handle>` for all
of its cross-thread needs instead of `Arc<Shared>`.
The effect of this change is that the multi-thread scheduler only has a
single `Arc` type, which includes the driver handles, keeping all
"shared state" together.
This is the next step in the internal runtime refactoring. This patch
updates `CurrentThread` to use `Arc<Handle>` for all of its cross-thread
needs instead of `Arc<Shared>`.
The effect of this change is that the current-thread scheduler only has
a single `Arc` type, which includes the driver handles, keeping all
"shared state" together.
The `runtime::Handle` struct is part of the public API but is also used
internally. This has created a bit of tension. An earlier patch made
defined Handle as a private struct in some cases when `rt` is not
enabled.
This patch splits out internal handle concerns into a new
`scheduler::Handle` type, which will only be internal. This also defines
a `Handle` type for each scheduler variant. Eventually, the
per-scheduler `Handle` types will replace the per-scheduler `Spawner`
types, but more work is needed before we can make that change.
The `tokio::select!` macro polls branches in a random order. While this
is desirable in production, for testing purposes a more deterministic
approach can be useul.
This change adds an additional parameter `rng_seed` to the runtime
`Builder` to set the random number generator seed. This value is then
used to reset the seed on the current thread when the runtime is entered
into (restoring the previous value when the thread leaves the runtime). All
threads created explicitly by the runtime also have a seed set as the
runtime is built. Each thread is set with a seed from a deterministic
sequence.
This guarantees that calls to the `tokio::select!` macro which are
performed in the same order on the same thread will poll branches in the
same order.
Additionally, the peer chosen to attempt to steal work from also uses a
deterministic sequence if `rng_seed` is set.
Both the builder parameter as well as the `RngSeed` struct are marked
unstable initially.
The `runtime::Kind` enum doesn't really represent the runtime flavor,
but is an enumeration of the different scheduler types. This patch
renames the enum to reflect this.
At a later time, it is likely the `Scheduler` enum will be moved to
`tokio::runtime::scheduler`, but that is punted to a later PR.
This rename is to make space for other enums
This patch removes println statements from tests. Some of these were
used to test that types implement `fmt::Debug`. These println statements
have been replaced with an equivalent test that does not output to
STDOUT. The rest of the println statements are most likely left over
from debugging sessions.
Currently, drivers are responsible for clean up when they are dropped.
This is possible today because each driver struct holds a reference to
its internal handle.
As part of an effort to decouple drivers from their handles, this patch
removes the drop implementations for the time and IO driver in favor of
having the scheduler explicitly call `shutdown()` as part of its
shutdown process. The scheduler will hold a reference to both the driver
and the driver handles, so in the future, it will be able to pass in the
driver handles as part of the shutdown process.
Currently, the various resource drivers use a layered approach where
each driver contains the next one. When the scheduler calls park on the
top-most driver, it does work then calls park on the inner driver it
holds. The handles do the same with unparking.
This patch is a step towards refactoring the runtime to move away from
the nested approach towards keeping all drivers and handles together in
a single runtime driver/handle. The unparker is removed from the time
handle and placed in the runtime handle and is passed into the time
driver as needed.
The `TimerEntry` struct is the internal integration point for public
time APIs (`sleep`, `interval`, ...) with the time driver. Currently,
`TimerEntry` holds an ref-counted reference to the time driver handle.
This patch replaces the reference to the time driver handle with a
reference to the runtime handle. This is part of a larger effort to
consolate internal handles across the runtime.
In an earlier PR (#4629), driver handles were moved into the scheduler
handle (`Spawner`). This was done to let the multi-threaded scheduler
have direct access to the thread pool spawner.
However, we are now working on a greater decoupling of the runtime
internals. All drivers and schedulers will be peers, stored in a single
thread-local variable, and the scheduler will be passed the full
runtime::Handle. This will achieve the original goal of giving the
scheduler access to the thread-pool while also (hopefully) simplifying
other aspects of the code.
As has been pointed out a few times in the past (e.g., #4036 (comment)), each-feature
is not sufficient for features check.
Ideally, we'd like to check all combinations of features, but there are too many
combinations. So limit the max number of simultaneous feature flags to 2 by --depth
option. I think this should be sufficient in most cases as @carllerche said in
taiki-e/cargo-hack#58.
The docs for `Registration::new_with_interest_and_handle` (internal
function) included a reference to two old functions `new_with_interest`
and `new` on the same struct. These functions no longer exist, which
could be confusing.
Since there is currently only a single constructor function on
`Registration`, the misleading text has simply been removed.
A few spawn_blocking methods were implemented on a Handle struct that is
intended to hold all runtime driver handles. These methods make more
sense in the blocking module.
A step towards the goal of consolidating thread-local variables, this
patch removes the `time::Handle` reference from the time driver struct.
The reference is moved a level up and passed into methods that need it.
The end goal is to have a single `Driver` struct that holds each
sub-driver and a single `Handle` ref-counted struct that holds all the
misc handles. Further work on the time driver is blocked by some other
refactoring that will happen in follow-up patches.
Years ago, Tokio was organized as a cluster of separate crates. This
architecture required a trait to represent "park the thread and do
work.". When all crates were combined into a single crate, the `Park`
and `Unpark` traits remained as internal details.
This patch removes these traits as they are no longer needed. This is in
service of a future refactor that will decouple the various resource
drivers and store them in a single struct instead of nesting them.
However, in the mean time, removing the Park/Unpark traits adds a bit of
messy code to make the conditional compilation work. This code will
(hopefully) be short-lived.
This patch makes some minor refactors. It renames `ClockTime` to
`TimeSource` since that is how all variables refer to it. It also moves
the type into a new file.
Finally, it moves the `unpark` handle out of the mutex as it does not
need to be there. Note, the call to `unpark` is still called while the
mutex is held, so there is no functional change. Moving it out of the
mutex is in preparation for moving the unpark handle completely out of
the time driver.
This patch moves the time driver into the runtime module. The time driver
is a runtime concern and is only used by the runtime. Moving the drivers
is the first step to cleaning up some Tokio internals. There will be
follow-up patches that integrate the drivers and other runtime concerns
more closely.
This is an internal refactor and should not impact any public APIs.
Current documentation of `sync::watch::Sender::send` may be intepreted
as the method failing if at some point in past the channel has been
closed because every receiver has been dropped. This isn't true,
however, as the channel could have been reopened by using
`sync::watch::Sender::subscribe`.
This fix clarifies the behavior. Moreover, it is noted that on failure,
the value isn't made available to future subscribers (but returned as
part of the `SendError`).
Fixes#4957.
For historical reasons, the current-thread runtime's scheduler was named
BasicScheduler and the multi-thread runtime's scheduler were named
ThreadPool. This patch renames the schedulers to mirror the runtime
names to increase consistency. This patch also moves the scheduler
implementations into a new `runtime::scheduler` module.
This patch moves the I/O driver into the runtime module. The I/O driver
is a runtime concern and is only used by the runtime. Moving the driver
is the first step to cleaning up some Tokio internals. There will be
follow-up patches that integrate the I/O driver and other runtime concerns
more closely.
Because `Interest` and `Ready` are public APIs, they were moved to the
top-level `io` module instead of moving the types to `runtime`.
This is an internal refactor and should not impact any public APIs.
The multi-threaded scheduler includes a per-worker LIFO slot to
store the last scheduled task. This can improve certain usage patterns,
especially message passing between tasks. However, this LIFO slot is not
currently stealable.
Eventually, the LIFO slot **will** become stealable. However, as a
stop-gap, this unstable option lets users disable the LIFO task when
doing so improves their application's overall performance.
Refs: #4941
The command to build the documentation locally provided in the
Contribution Guide did not work for all crates in the project workspace.
Specifically, to build the docs for `tokio-stream` the flag `--cfg
docsrs` needs to be in the environment variable `RUSTFLAGS` in addition
to being in `RUSTDOCFLAGS`.
Additionally, there was text describing that the docs cannot be built
from the root of the workspace with a link to rust-lang/cargo#9274. That
issue has since been closed as complete and the listed commands do now
work from the root of the workspace. As such, that text has been
removed.
The documentation for`PollEvented` had a single remaining reference to
`Registration::clear_read_ready`, which no longer exists. This change
updates the documentation to refer to `Registration::clear_readiness`
instead.
Upgrade the nightly to a version that works with all of miri,
cargo-hack, minimal-versions, and cargo-check-external-types.
This is a prerequisite for #4902.
The broadcast channel allows multiple senders to send messages to
multiple receivers, where each receiver receives messages starting from
when it subscribes. After all senders are dropped, the receivers will
continue to receive all waiting messages in the buffer and then receive
a `Closed` error.
To mark that a channel has closed, it stores two closed flags, one on
the channel level and another in the buffer slot *after* the last used
slot (this may also be the earliest entry being kept for lagged
receivers, see #2425).
However, we don't need both closed flags, keeping the channel level
closed flag is sufficient.
Without the slot level closed flag, each receiver receives each message
until it is up to date and for that receiver the channel is empty. Then,
the actual return message is chosen depending on the channel level
closed flag; if the channel is NOT closed, then `Empty` is returned, if
the channel is closed then `Closed` is returned instead.
With the modified logic, there is no longer a need to append a closed
token to the internal buffer (by setting the slot level closed flag on
the next slot). This fixes the off by one error described in #4814,
which caused a receiver which was created after the channel was already
closed to get `Empty` from `try_recv` (or hang forever when calling
`recv`) instead of receiving `Closed`.
As a bonus, we save a single `bool` on each buffer slot.
Refs: #4814
The contribution guide describes the Tokio code of conduct and gives a
list of steps to follow when contributing in different ways. In the
guide to contributing a pull request, commands are listed to be run by
the contributor locally to help ensure that the PR will pass on CI.
The command to run clippy isn't the same as the one run on CI,
specifically the command doesn't check the tests. This commit changes
the command to match the one on CI, so that tests are checked by clippy
as well.
As the [epoll documentation points out](https://man7.org/linux/man-pages/man7/epoll.7.html), a read that only partially fills a buffer is sufficient to show that the socket buffer has been drained.
We can take advantage of this by clearing readiness in this case, which seems to significantly improve performance under certain conditions.
Co-authored-by: Carl Lerche <[email protected]>
The documentation for the runtime `Builder::worker_threads` function
incorrectly stated that it would panic if used when constructing a
`current_thread` runtime. In truth, the call to the function has no
effect.
Since adding the described panic to the code could cause new panics in
existing code using tokio, the documentation has been modified to
describe the existing behavior.
Refs: #4773
This change adds a case that was missing from the original PR, #4793.
The `io::driver::Handle::current` function was only covered by
`#[track_caller]` in the case that the `rt` feature is enabled, however
it was missing in the case that the `rt` feture isn't enabled (in which
case a panic would be more common).
This particular case cannot be tested in the tokio tests as they always
run with all features enabled.
Refs: #4413
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds `#[track_caller]` to all the public APIs in the task
module of the tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs.
In cases where `#[track_caller]` does not work, it has been left out.
For example, it currently does not work on async functions, blocks, or
closures. So any call stack that passes through one of these before
reaching the actual panic is not able to show the calling site outside
of tokio as the panic location.
The following functions have call stacks that pass through closures:
* `task::block_in_place`
* `task::local::spawn_local`
Tests are included to cover each potentially panicking function.
The following functions already had `#[track_caller]` applied everywhere
it was needed and only tests have been added:
* `task::spawn`
* `task::LocalKey::sync_scope`
Refs: #4413
Functions that may panic can be annotated with #[track_caller] so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds #[track_caller] to all the non-unstable public net APIs
in the main tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs. Not all panic
cases can have #[track_caller] applied fully as the callstack passes
through a closure which isn't yet supported by the annotation (e.g. net
functions called from outside a tokio runtime).
Additionally, the documentation was updated to indicate additional cases
in which the public net functions may panic (the same as the io
functions).
Tests are included to cover each potentially panicking function.
Refs: #4413
This adds initial, unstable, support for the wasm32-wasi target. Not all of Tokio's
features are supported yet as WASI's non-blocking APIs are still limited.
Refs: tokio-rs/tokio#4827
Currently, we only have WASM regression tests that run without WASI.
However, rust provides a WASI specific target which enables code to
special case WASI. This PR adds a basic test to cover that case.
This is an initial addition to help land tokio-rs/tokio#4716.
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds `#[track_caller]` to all the public APIs in the sync
module of the tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs.
In cases where `#[track_caller]` does not work, it has been left out.
For example, it currently does not work on async functions, blocks, or
closures. So any call stack that passes through one of these before
reaching the actual panic is not able to show the calling site outside
of tokio as the panic location.
The following functions have call stacks that pass through closures:
* `sync::watch::Sender::send_modify`
* `sync::watch::Sender::send_if_modified`
Additionally, in the above functions it is a panic inside the supplied
closure which causes the function to panic, and so showing the location
of the panic itself is desirable.
The following functions are async:
* `sync::mpsc::bounded::Sender::send_timeout`
Tests are included to cover each potentially panicking function.
Refs: #4413
Functions that may panic can be annotated with #[track_caller] so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds #[track_caller] to the signal() function which is the
only function in the signal public API which can panic. Documentation
was added to this function to indicate that it may panic.
Not all panic cases can have #[track_caller] applied fully as the
callstack passes through a closure which isn't yet supported by the
annotation (e.g. signal() called from outside a tokio runtime).
Tests are included to cover the case where signal() is called from a
runtime without IO enabled.
Refs: #4413
This fixes#4801, where, as a result of https://github.com/rust-lang/rust/pull/95469, our implementation of cat used for this test no longer works, as stdio functions on windows now can abort the process if the pipe is set to nonblocking mode.
Unfortunately in windows, setting one end of the pipe to be nonblocking makes the whole thing nonblocking, so when, in tokio::process we set the child pipes to nonblocking mode, it causes serious problems for any rust program at the other end.
Fixing this issue is for another day, but fixing the tests is for today.
Functions that may panic can be annotated with #[track_caller] so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds #[track_caller] to all the non-unstable public io APIs
in the main tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs.
Additionally, the documentation for `AsyncFd` was updated to indicate
that the functions `new` and `with_intent` can panic.
Tests are included to cover each potentially panicking function. The
logic to test the location of a panic (which is a little complex), has
been moved to a test support module.
Refs: #4413
Functions that may panic can be annotated with #[track_caller] so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds #[track_caller] to all the non-unstable public APIs in
tokio-util where the documentation describes how the function may panic
due to incorrect context or inputs.
In cases where `#[track_caller]` does not work, it has been left out. For
example, it currently does not work on async functions, blocks, or
closures. So any call stack that passes through one of these before
reaching the actual panic is not able to show the calling site outside
of tokio as the panic location.
The public functions that could not have `#[track_caller]` added for
this reason are:
* `time::advance`
Tests are included to cover each potentially panicking function. In the
following cases, `#[track_caller]` had already been added, and only
tests have been added:
* `time::interval`
* `time::interval_at`
Refs: #4413
* util: add track_caller to public APIs
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds `#[track_caller]` to all the non-unstable public APIs in
tokio-util where the documentation describes how the function may panic
due to incorrect context or inputs.
In one place, an assert was added where the described behavior appeared
not to be implemented. The documentation for `DelayQueue::reserve`
states that the function will panic if the new capacity exceeds the
maximum number of entries the queue can contain. However, the function
didn't panic until a higher number caused by an allocation failure. This
is inconsistent with `DelayQueue::insert_at` which will panic if the
number of entries were to go over MAX_ENTRIES.
Tests are included to cover each potentially panicking function.
Refs: #4413
* fix tests on FreeBSD 32-bit (I hope)
Some tests were failing on FreeBSD 32-bit because the "times too far in
the future" for DelayQueue were also too far in the future for the OS.
Fixed by copying the MAX_DURATION value from where it's defined and
using it to create a duration that is just 1 more than the maximum. This
will start to break once we get close (within 2 and a bit years) of the
Epochalypse (19 Jan, 2038) - but a lot of other things are going to be
breaking on FreeBSD 32-bit by then anyway.
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds `#[track_caller]` to all the non-unstable public APIs
in tokio-stream (only `chunks_timeout` in `StreamExt`) where the
documentation describes how this function may panic due to incorrect
input.
A test has been included to cover the panic.
Refs: #4413
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.
This change adds track caller to all the non-unstable public APIs in
Tokio core where the documentation describes how the function may panic
due to incorrect context or inputs. Since each internal function needs
to be annotated down to the actual panic, it makes sense to start in
Tokio core functionality.
Tests are needed to ensure that all the annotations remain in place in
case internal refactoring occurs.
The test installs a panic hook to extract the file location from the
`PanicInfo` struct and clone it up to the outer scope to check that the
panic was indeed reported from within the test file. The downside to
this approach is that the panic hook is global while set and so we need
a lot of extra functionality to effectively serialize the tests so that
only a single panic can occur at a time.
The annotation of `block_on` was removed as it did not work. It appears
to be impossible to correctly chain track caller when the call stack to
the panic passes through clojures, as the track caller annotation can
only be applied to functions. Also, the panic message itself is very
descriptive.
Allows the user to configure the runtime's behavior when a spawned task
panics. Currently, the panic is propagated to the JoinHandle and the
runtime resumes. This patch lets the user set the runtime to shutdown on
unhandled panic.
So far, this is only implemented for the current-thread runtime.
Refs: #4516
Extracts the refactor from #4518.
The basic scheduler takes many configuration options as arguments to the
constructor. This cleans it up a bit by defining a `Config` struct and
using that to pass arguments to the constructor.
Found some `if let` statements that don't actually bind any variables.
This can make it somewhat confusing as to whether the pattern is binding
the value or being matched against. Switching to `matches!` and equality
comparisons allow these to be removed.
Co-authored-by: David Koloski <[email protected]>
This doesn't cause any issues in practice because this is a private API
that is only used in ways that cannot trigger UB. Indexing into `slots`
is not sound until after we've asserted that the page is allocated,
since that aliases the first slot which may not be allocated. This PR
also switches to using `as_ptr` to obtain the base pointer for clarity.
Co-authored-by: David Koloski <[email protected]>
* task: add consume_budget for cooperative scheduling
For cpu-only computations that do not use any Tokio resources,
budgeting does not really kick in in order to yield and prevent
other tasks from starvation. The new mechanism - consume_budget,
performs a budget check, consumes a unit of it, and yields only
if the task exceeded the budget. That allows cpu-intenstive
computations to define points in the program which indicate that
some significant work was performed. It will yield only if the budget
is gone, which is a much better alternative to unconditional yielding,
which is a potentially heavy operation.
* tests: add a test case for task::consume_budget
The test case ensures that the task::consume_budget utility
actually burns budget and makes the task yield once the whole
budget is gone.
Updating an atomic variable with a load followed by a store is racy. It
defeats the entire purpose of using atomic variables, and can result in
"lost" updates. Instead, use fetch_add .
Adds knobs to the runtime builder to control the number of ticks
between polling the global task queue (fairness) and the event driver
(I/O prioritization).
Both varieties of scheduler already supported these intervals, but they
were defined by private constants. Some workloads benefit from
customizing these values.
Closes#4651
## Motivation
Currently, the various spawning methods on `tokio::task::JoinSet` and
`tokio_util::task::JoinMap` lack `#[track_caller]` attributes, so the
`tracing` spans generated for tasks spawned on `JoinSet`s/`JoinMap`s
will have the `JoinSet` spawning method as their spawn location, rather
than the user code that called that method.
## Solution
This PR fixes that by...adding a bunch of `#[track_caller]` attributes.
## Future Work
In the future, we may also want to consider adding additional data to
the task span indicating that the task is spawned on a `JoinSet`...
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
`task::JoinSet` currently has both `spawn`/`spawn_local` methods,
and `spawn_on`/`spawn_local_on` variants of these methods that take a
reference to a runtime `Handle` or to a `LocalSet`, and spawn tasks on
the provided runtime/`LocalSet`, rather than the current one. The
`task::Builder` type is _also_ an API type that can spawn tasks, but it
doesn't have `spawn_on` variants of its methods. It occurred to me that
it would be nice to have similar APIs on `task::Builder`.
## Solution
This branch adds `task::Builder::spawn_on`,
`task::Builder::spawn_local_on`, and `task::Builder::spawn_blocking_on`
methods, similar to those on `JoinSet`. In addition, I did some
refactoring of the internal spawning APIs --- there was a bit of
duplicated code that this PR reduces.
Signed-off-by: Eliza Weisman <[email protected]>
This documents why it is safe to convert `bytes::UninitSlice` to `&mut
[MaybeUninit<u8>]`, and shrinks one of the unsafe blocks to make these
functions easier to audit.
Add a function that is more versatile than send_modify(). The result of
the passed closure indicates if the mutably borrowed value has actually
been modified or not. Receivers are only notified if the value has been
modified as indicated by the sender.
Signed-off-by: Uwe Klotz <[email protected]>
## Motivation
In many cases, it is desirable to spawn a set of tasks associated with
keys, with the ability to cancel them by key. As an example use case for
this sort of thing, see Tower's [`ReadyCache` type][1].
Now that PR #4530 adds a way of cancelling tasks in a
`tokio::task::JoinSet`, we can implement a map-like API based on the
same `IdleNotifiedSet` primitive.
## Solution
This PR adds an implementation of a `JoinMap` type to
`tokio_util::task`, using the `JoinSet` type from `tokio::task`, the
`AbortHandle` type added in #4530, and the new task IDs added in #4630.
Individual tasks can be aborted by key using the `JoinMap::abort`
method, and a set of tasks whose key match a given predicate can be
aborted using `JoinMap::abort_matching`.
When tasks complete, `JoinMap::join_one` returns their associated key
alongside the output from the spawned future, or the key and the
`JoinError` if the task did not complete successfully.
Overall, I think the way this works is pretty straightforward; much of
this PR is just API boilerplate to implement the union of applicable
APIs from `JoinSet` and `HashMap`. Unlike previous iterations on the
`JoinMap` API (e.g. #4538), this version is implemented entirely in
`tokio_util`, using only public APIs from the `tokio` crate. Currently,
the required `tokio` APIs are unstable, but implementing `JoinMap` in
`tokio-util` means we will never have to make stability commitments for
the `JoinMap` API itself.
[1]: https://github.com/tower-rs/tower/blob/master/tower/src/ready_cache/cache.rs
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
PR #4538 adds a prototype implementation of a `JoinMap` API in
`tokio::task`. In [this comment][1] on that PR, @carllerche pointed out
that a much simpler `JoinMap` type could be implemented outside of
`tokio` (either in `tokio-util` or in user code) if we just modified
`JoinSet` to return a task ID type when spawning new tasks, and when
tasks complete. This seems like a better approach for the following
reasons:
* A `JoinMap`-like type need not become a permanent part of `tokio`'s
stable API
* Task IDs seem like something that could be generally useful outside of
a `JoinMap` implementation
## Solution
This branch adds a `tokio::task::Id` type that uniquely identifies a
task relative to all other spawned tasks. Task IDs are assigned
sequentially based on an atomic `usize` counter of spawned tasks.
In addition, I modified `JoinSet` to add a `join_with_id` method that
behaves identically to `join_one` but also returns an ID. This can be
used to implement a `JoinMap` type.
Note that because `join_with_id` must return a task ID regardless of
whether the task completes successfully or returns a `JoinError`, I've
also changed `JoinError` to carry the ID of the task that errored, and
added a `JoinError::id` method for accessing it. Alternatively, we could
have done one of the following:
* have `join_with_id` return `Option<(Id, Result<T, JoinError>)>`, which
would be inconsistent with the return type of `join_one` (which we've
[already bikeshedded over once][2]...)
* have `join_with_id` return `Result<Option<(Id, T)>, (Id, JoinError)>>`,
which just feels gross.
I thought adding the task ID to `JoinError` was the nicest option, and
is potentially useful for other stuff as well, so it's probably a good API to
have anyway.
[1]: https://github.com/tokio-rs/tokio/pull/4538#issuecomment-1065614755
[2]: https://github.com/tokio-rs/tokio/pull/4335#discussion_r773377901Closes#4538
Signed-off-by: Eliza Weisman <[email protected]>
Previously, `runtime::Handle` was a single struct composed of the
internal handles for each runtime component. This patch splits the
`Handle` struct into a `HandleInner` which contains everything
**except** the task scheduler handle. Now, `HandleInner` is passed to
the task scheduler during creation and the task scheduler is responsible
for storing it. `Handle` only needs to hold the scheduler handle and
can access the rest of the component handles by querying the task
scheduler.
The motivation for this change is it now enables the multi-threaded
scheduler to have direct access to the blocking spawner handle.
Previously, when spawning a new thread, the multi-threaded scheduler had
to access the blocking spawner by accessing a thread-local variable.
Now, in theory, the multi-threaded scheduler can use `HandleInner`
directly. However, this change hasn't been done in this PR yet.
Also, now the `Handle` struct is much smaller.
This change is intended to make it easier for the multi-threaded
scheduler to shutdown idle threads and respawn them on demand.
Handle::current docs say it's not possible to call it on any non-runtime thread, but you can call it from a runtime context created by an EnterGuard. This updates the docs to mention EnterGuard as a way to avoid this panic.
It turns out that the CI job for testing `tokio_unstable` features isn't
actually running doctests for `tokio_unstable`, just lib and integration
tests. This is because RustDoc is responsible for running doctests, and
it needs the unstable cfg passed to it separately from `RUSTFLAGS`.
This means that if the examples for unstable APIs are broken, CI won't
catch this, which is not great!
This commit changes the `test-unstable` CI job to pass `--cfg
tokio_unstable` in `RUSTDOCFLAGS` as well as `RUSTFLAGS`. This way,
doctests for unstable APIs should actually run.
I also fixed a typo in one of the runtime metrics doctests that was
causing a compilation error, which was caught as a result of actually
testing the unstable API docs on CI. :)
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Recent Clippy releases have added some new lints that trigger on some
code in Tokio. These aren't a big deal, but seeing them in my editor is
mildly annoying.
## Solution
This branch fixes the following issues flagged by Clippy:
* manual `Option::map` implementation
* use of `.map(...).flatten(...)` that could be replaced with
`.and_then(...)`
* manual implementation of saturating arithmetic on `Duration`s
* simplify some boolean expressions in assertions (`!res.is_ok()` can be
`res.is_err()`)
* fix redundant field names in initializers
* replace an unnecessary cast to `usize` with an explicitly typed
integer literal
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Before we stabilize the `JoinSet` API, we intend to add a method for
individual tasks in the `JoinSet` to be aborted. Because the
`JoinHandle`s for the tasks spawned on a `JoinSet` are owned by the
`JoinSet`, the user can no longer use them to abort tasks on the
`JoinSet`. Therefore, we need another way to cause a remote abort of a
task on a `JoinSet` without holding its `JoinHandle`.
## Solution
This branch adds a new `AbortHandle` type in `tokio::task`, which
represents the owned permission to remotely cancel a task, but _not_ to
await its output. The `AbortHandle` type holds an additional reference
to the task cell.
A crate-private method is added to `JoinHandle` that returns an
`AbortHandle` for the same task, incrementing its ref count.
`AbortHandle` provides a single method, `AbortHandle::abort(self)`, that
remotely cancels the task. Dropping an `AbortHandle` decrements the
task's ref count but does not cancel it. The `AbortHandle` type is
currently marked as unstable.
The spawning methods on `JoinSet` are modified to return an
`AbortHandle` that can be used to cancel the spawned task.
## Future Work
- Currently, the `AbortHandle` type is _only_ available in the public
API through a `JoinSet`. We could also make the
`JoinHandle::abort_handle` method public, to allow users to use the
`AbortHandle` type in other contexts. I didn't do that in this PR,
because I wanted to make the API addition as minimal as possible, but we
could make this method public later.
- Currently, `AbortHandle` is not `Clone`. We could easily make it
`Clone` by incrementing the task's ref count. Since this adds more trait
impls to the API, we may want to be cautious about this, but I see no
obvious reason we would need to remove a `Clone` implementation if one
was added...
- There's been some discussion of adding a `JoinMap` type that allows
aborting tasks by key, and manages a hash map of keys to `AbortHandle`s,
and removes the tasks from the map when they complete. This would make
aborting by key much easier, since the user wouldn't have to worry about
keeping the state of the map of abort handles and the tasks actually
active on the `JoinSet` in sync. After thinking about it a bit, I
thought this is probably best as a `tokio-util` API --- it can currently
be implemented in `tokio-util` with the APIs added in `tokio` in this
PR.
- I noticed while working on this that `JoinSet::join_one` and
`JoinSet::poll_join_one` return a cancelled `JoinError` when a task is
cancelled. I'm not sure if I love this behavior --- it seems like it
would be nicer to just skip cancelled tasks and continue polling. But,
there are currently tests that expect a cancelled `JoinError` to be
returned for each cancelled task, so I didn't want to change it in
_this_ PR. I think this is worth revisiting before stabilizing the API,
though?
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Rust error messages seem to have changed a bit in 1.59.0
## Solution
Update the `trybuild` stderr output.
This should unbreak the tests on the latest Rust (and fix CI).
## Motivation
The `JoinSet` type is currently missing the `tokio_unstable` and
`feature = "rt"` `doc(cfg(...))` attributes, making it erroneously
appear to be available without the required feature and without unstable
features enabled. This is incorrect.
I believe this is because `doc(cfg(...))` on a re-export doesn't
actually add the required cfgs to the type itself, and the
`cfg_unstable!` is currently only guarding a re-export and module.
## Solution
This PR fixes the missing attributes.
## Motivation
PR #4499 made the `JoinSet` API unstable, but did not add a
documentation note explaining unstable features. In general, since the
docs.rs build includes unstable APIs, it's probably worth including
these notes so that users understand what it means for an API to be
unstable.
## Solution
This branch adds a note on unstable APIs to the `JoinSet` type-level
documentation, similar to the notes for `task::Builder` and the runtime
metrics APIs.
Also, I noticed that there was a broken link to the top-level
documentation on unstable APIs in the docs for `task::Builder`, so I
fixed that as well.
time: resolve TODO
## Motivation
Existing `TODO` comment in `src/time/driver/wheel/level.rs`.
## Solution
`level_range()` always return a strictly positive power of 2. If `b` is a
strictly positive power of 2, `a - (a % b)` is equal to `a & !(b - 1)`.
The major breaking change in Mio v0.8 is TcpSocket type being removed.
Replacing Mio's TcpSocket we switch to the socket2 library which
provides a similar type Socket, as well as SockRef, which provide all
options TcpSocket provided (and more!).
Tokio's TcpSocket type is now backed by Socket2 instead of Mio's
TcpSocket. The main pitfall here is that socket2 isn't non-blocking by
default, which Mio obviously is. As a result we have to do potentially
blocking calls more carefully, specifically we need to handle
would-block-like errors when connecting the TcpSocket ourselves.
One benefit for this change is that adding more socket options to
TcpSocket is now merely a single function call away (in most cases
anyway).
PR #3881 factored out the spawning of local tasks on a `LocalSet` into a
function `spawn_local_inner`, so that the implementation could be shared
with the `task::Builder` API. But, that PR neglected to add a
`#[track_caller]` attribute to `spawn_local_inner`, so the `tracing`
spans for local tasks are all generated with `spawn_local_inner` as
their spawn location, rather than forwarding the actual spawn location
from the calling function.
This causes pretty useless results when using `tokio-console` with code
that spawns a number of local tasks, such as Actix
(https://reddit.com/r/rust/comments/snt5fq/can_tokioconsole_profile_actixrt/)
This commit fixes the issue by adding the missing `#[track_caller]`
attribute.
`Instant::duration_since`, `Instant::elapsed`, and `Instant::sub` may
panic. This is especially dangerous when `Instant::now` travels back in
time. While this isn't supposed to happen, this behavior is highly
platform-dependent (e.g., rust-lang/rust#86470).
This change modifies the behavior of `tokio::time::Instant` to prevent
this class of panic, as proposed for `std::time::Instant` in
rust-lang/rust#89926.
When backporting patches to LTS branches, we often run into CI failures due to
changes in rust. Newer rust versions add more lints, which break CI. We really
don't want to also have to backport patches that fix CI, so instead, LTS branches
should pin the stable rust version in CI (e.g. #4434).
This PR restructures the CI config files to make it a bit easier to set a specific rust
version in CI.
* Several of tokio's features (e.g. the channel implementation) do not
need a runtime to work, and can be compiled and used for
wasm32-unknown-unknown targets
* This change enables running tests for the `sync` and `macros` features
so that we can note any regressions there
This patch reduces the number of times worker threads wake up without having
work to do in the multi-threaded scheduler. Unnecessary wake-ups are expensive
and slow down the scheduler. I have observed this change reduce no-op wakes
by up to 50%.
The multi-threaded scheduler is work-stealing. When a worker has tasks to process,
and other workers are idle (parked), these idle workers must be unparked so that
they can steal work from the busy worker. However, unparking threads is expensive,
so there is an optimization that avoids unparking a worker if there already exists
workers in a "searching" state (the worker is unparked and looking for work). This
works pretty well, but transitioning from 1 "searching" worker to 0 searching workers
introduces a race condition where a thread unpark can be lost:
* thread 1: last searching worker about to exit searching state
* thread 2: needs to unpark a thread, but skip because there is a searching worker.
* thread 1: exits searching state w/o seeing thread 2's work.
Because this should be a rare condition, Tokio solves this by always unparking a
new worker when the current worker:
* is the last searching worker
* is transitioning out of searching
* has work to process.
When the newly unparked worker wakes, if the race condition described above
happened, "thread 2"'s work will be found. Otherwise, it will just go back to sleep.
Now we come to the issue at hand. A bug incorrectly set a worker to "searching"
when the I/O driver unparked the thread. In a situation where the scheduler was
only partially under load and is able to operate with 1 active worker, the I/O driver
would unpark the thread when new I/O events are received, incorrectly transition
it to "searching", find new work generated by inbound I/O events, incorrectly
transition itself from the last searcher -> no searchers, and unpark a new thread.
This new thread would wake, find no work and go back to sleep.
Note that, when the scheduler is fully saturated, this change will make no impact
as most workers are always unparked and the optimization to avoid unparking
threads described at the top apply.
Re-applies #4377 and fixes the bug resulting in Hyper's double panic.
Revert: #4394
Original PR:
This PR does some refactoring to the current-thread scheduler bringing it closer to the structure of the
multi-threaded scheduler. More specifically, the core scheduler data is stored in a Core struct and that
struct is passed around as a "token" indicating permission to do work. The Core structure is also stored
in the thread-local context.
This refactor is intended to support #4373, making it easier to track counters in more locations in the
current-thread scheduler.
I tried to keep commits small, but the "set Core in thread-local context" is both the biggest commit and
the key one.
Under the hood, the watch channel uses a RwLock to implement reading
(borrow) and writing (send). This may cause a deadlock if a user has
concurrent borrows on the same thread. This is most likely to occur due
to a recursive borrow.
This PR adds documentation to describe the deadlock so that future users
of the watch channel will be aware.
This patch does some refactoring to the current-thread scheduler bringing it closer to the
structure of the multi-threaded scheduler. More specifically, the core scheduler data is stored
in a Core struct and that struct is passed around as a "token" indicating permission to do
work. The Core structure is also stored in the thread-local context.
This refactor is intended to support #4373, making it easier to track counters in more locations
in the current-thread scheduler.
This adds the following methods:
- UdpSocket::set_send_buffer_size
- UdpSocket::send_buffer_size
- UdpSocket::set_recv_buffer_size
- UdpSocket::recv_buffer_size
Currently, the docs.rs documentation for tokio is built without
--cfg tokio_unstable set. This means that unstable features are not shown in
the API docs, making them difficutl to discover. Clearly, we do want to
document the existence of unstable APIs, given that there's a section in
the lib.rs documentation listing them, so it would be better if it was
also possible to determine what APIs an unstable feature enables when
reading the RustDoc documentation.
This branch changes the docs.rs metadata to also pass --cfg tokio_unstable
when building the documentation. It turns out that it's
necessary to separately pass the cfg flag to both RustDoc and rustc,
or else the tracing dependency, which is only enabled in
target.cfg(tokio_unstable).dependencies, will be missing and the build
will fail.
In addition, I made some minor improvements to the docs for unstable
features. Some links in the task::Builder docs were broken, and the
required tokio_unstable cfg was missing from the doc(cfg(...))
attributes. Furthermore, I added a note in the top-level docs for
unstable APIs, stating that they are unstable and linking back to the
section in the crate-level docs that explains how to enable unstable
features.
Fixes#4328
For now, this is only allowed on TcpStream. This is a problem when one
want to disable lingering (i.e. set it to Duration(0, 0)). Without being
able to set it prior to the connect call, if the connect future is
dropped it would leave sockets in a TIME_WAIT state.
Co-authored-by: Fabien Gaud <[email protected]>
Reads and buffered reads from a `tokio::io::empty` were always marked
as ready. That makes sense, given that there is nothing to wait for.
However, doing repeated reads on the `empty` could stall the event
loop and prevent other tasks from making progress.
This change uses tokio's coop system to yield control back to the
executor when appropriate.
Note that the issue that originally triggered this PR is not fixed
yet, because the `timeout` function will not poll the timer after
empty::read runs out of budget. A different change will be needed to
address that.
Refs: #4291
The implicit elided lifetimes of the `AsyncFd` references in return
types of methods on `AsyncFdReadyGuard` resolved to that of `&self`.
However that lifetime is smaller than `'a` since `self` contains an `&'a
AsyncFd` reference. This will not change so the change also does not
lessen future proofing.
## Motivation
This allows `StreamMap` to be used with [`futures::stream::StreamExt::collect`][collect].
My use case is something like this:
```rust
let stream_map: StreamMap<_, _> = things
.into_iter()
.map(|thing| make_stream(thing)) // iterator of futures
.collect::<FuturesUnordered<_>>() // stream of streams
.collect::<StreamMap<_, _>>() // combine all the inner streams into one
.await;
async fn make_stream(thing: Thing) -> impl Stream { ... }
```
[collect]: https://docs.rs/futures/0.3.17/futures/stream/trait.StreamExt.html#method.collect
## Solution
Add `Extend` impl that delegates to the inner `Vec`.
Depends on #4226
## Motivation
Currently, the safety invariants and synchronization strategy used in
`tokio::sync::oneshot` are not particularly obvious, especially to a new
reader. It would be nice to better document this code to make these
invariants clearer.
## Solution
This branch adds `SAFETY:` comments to the `oneshot` channel
implementation. In particular, I've focused on documenting the
invariants around when the inner `UnsafeCell` that stores the value can
be accessed by the sender and receiver sides of the channel.
I still want to take a closer look at when the waker cells can be set,
and I'd like to add more documentation there in a follow-up branch.
Signed-off-by: Eliza Weisman <[email protected]>
Since a waker can trigger arbitrary code, such as with a custom waker,
and even more so now that it can emit a tracing event that could do
respond, we must be careful about the internal state when that code is
triggered. The clone method of a waker is one of those instances.
This changes the internals of `Notify` so that the waker is cloned
*before* locking the waiter list. While this does mean that in some
contended cases, we'll have made an optimistic clone, it makes `Notify`
more robust and correct.
Note that the included test case is built from an instance that did
happen naturally in another project, see
https://github.com/tokio-rs/console/issues/133.
This change enables `tokio-console` to parse the location information
for a spawned task into a structured object rather than simply displaying
this info as a field on the task.
Signed-off-by: Zahari Dichev <[email protected]>
This branch instruments the `Sleep` resource to allow the tokio-console
to consume data about resources usage. The corresponding console branch
is here: https://github.com/tokio-rs/console/pull/77
Signed-off-by: Zahari Dichev <[email protected]>
## Motivation
PR #4055 added a new `WakeList` type, to manage a potentially
uninitialized array when waking batches of wakers. This has the
advantage of not initializing a bunch of empty `Option`s when only a
small number of tasks are being woken, potentially improving performance
in these cases.
Currently, `WakeList` is used only in the IO driver. However,
`tokio::sync` contains some code that's almost identical to the code in
the IO driver that was replaced with `WakeList`, so we can apply the
same optimizations there.
## Solution
This branch changes `tokio::sync::Notify` and
`tokio::sync::batch_semaphore::Semaphore` to use `WakeList` when waking
batches of wakers. This was a pretty straightforward drop-in
replacement.
Signed-off-by: Eliza Weisman <[email protected]>
Currently, the per-task spans generated by Tokio's `tracing` feature
have the span name "task" and the target "tokio::task". This is because
the console subscriber identified tasks by looking specifically for the
"tokio::task" target.
In tokio-rs/console#41, it was proposed that the console change to a
more generic system for identifying the spans that correspond to tasks,
to allow recording tasks belonging to multiple runtime crates (e.g. an
application that uses Tokio for async tasks and Rayon for CPU-bound
tasks). PR tokio-rs/console#68 changed the console to track any spans
"runtime.spawn", regardless of target (so that the target can be used to
identify the runtime a task came from), with "tokio::task/task" tracked
for backwards-compatibility with the current release version of tokio.
This branch changes Tokio's span naming to the new convention. I also
rearranged a couple fields so that the task's kind field always comes
before the name and spawn location, since it's likely to be the
shortest, and renamed the `function` field on blocking tasks to `fn`,
for brevity's sake.
Signed-off-by: Eliza Weisman <[email protected]>
Adds a builder API for spawning tasks. Initially, this enables the caller to name the spawned
task in order to provide better visibility into all tasks in the system.
This patch documents cancellation safety. It also moves the "Avoid racy if preconditions"
section in the select! documentation since otherwise the first code block on the page
shows how not to use it, which seems counterintuitive.
Tokio 1.7.0 introduced a change intended to eagerly shutdown newly
spawned tasks if the runtime is in the process of shutting down.
However, it introduced a bug where already spawned tasks could be
shutdown too early, resulting in the potential introduction of deadlocks
if tasks acquired mutexes in drop handlers.
Fixes#3869
This builds on https://github.com/tokio-rs/mio/pull/1351 and introduces the
tokio::net::windows::named_pipe module which provides low level types for
building and communicating asynchronously over windows named pipes.
Named pipes require the `net` feature flag to be enabled on Windows.
Co-authored-by: Alice Ryhl <[email protected]>
Instead of using sleep in time::advance, this fixes the root of the issue. When futures passed
to `Runtime::block_on` are woken, it bypassed all the machinery around advancing time. By
intercepting wakes in the time driver, we know when the block_on task is woken and skip
advancing time in that case.
Fixes#3837
## Motivation
In support of tokio-rs/console#37, we want to understand when a specific task's waker has been interacted with, such as when it is awoken, or if it's forgotten (not cloned), etc.
## Solution
When the tracing feature is enabled, a super trait of Future (InstrumentedFuture) is implemented for Instrumented<F> that allows grabbing the task's ID (well, its span ID), and stores that in the raw task trailer. The waker vtable then emits events and includes that ID.
This commit fixed issue #3787 by removing [doc(inline)] from
macro `cfg_macros` and added proper #[doc(inline)] attributes
to `pub use` items inside `cfg_macros` calls.
It's probably not `cfg_macros`s responsibility to inlining public
macros, though it's conveninent to do so. Notice that in lib.rs:
cfg_macros! {
/// Implementation detail of the `select!` macro. This macro is **not**
/// intended to be used as part of the public API and is permitted to
/// change.
#[doc(hidden)]
pub use tokio_macros::select_priv_declare_output_enum;
...
}
`#[doc(hidden)]` and `#[doc(inline)]` are conflict with each other
in the sense of correctness.
Fixes: #3787
Previously, `time::advance` would set the mocked clock forward the
requested amount, then yield. However, if there was no work ready to
perform immediately, this would result in advancing to the next expiring
sleep.
Now, `time::advance(...)` will unblock at the requested time. The
difference between `time::advance(...)` and `time::sleep(...)` is a bit
fuzzy. The main difference is `time::sleep(...)` operates on the current
task and `time::advance(...)` operates at the runtime level.
Fixes#3710
* sync: add `mpsc::Sender::{reserve_owned, try_reserve_owned}`
## Motivation
The `mpsc::Sender::reserve` method currently returns a permit that borrows
from the `Sender`. It would be nice to have a version of it that returns
an owned permit.
## Solution
This branch adds an `OwnedPermit` type and `Sender::{reserve_owned,
try_reserve_owned}` methods. Unlike the comparable methods on
`Semaphore`, these methods do *not* require an `Arc<Sender>` as the
receiver; this is because the sender internally reference counts the
channel and is already cheap to clone. Requiring an `Arc` would simply
add an unnecessary second layer of reference counting, which is not
ideal; instead, the documentation encourages the user to clone the
sender prior to calling `reserve_owned` when necessary.
Since these methods take the `Sender` by value, they also have the ability
to _return_ the `Sender` from a successful `OwnedPermit::send`. This
allows them to be used without additional clones. Essentially, this is a
very simple type-level encoding of the sender's state, with the
transitions
```
┌──────┐
┌───►│Sender├───┐
│ └──────┘ │
send reserve
│ ┌───────────┐ │
└─┤OwnedPermit│◄┘
└───────────┘
```
Additionally, I added an `OwnedPermit::release`, which returns the
`Sender` and releases the permit *without* sending a message.
Closes#3688
Signed-off-by: Eliza Weisman <[email protected]>
Simply exposes the number of available permits of the semaphore.
This makes some kinds of bookkeeping easier without having to manually keep counts using atomics.
Fixes#2642
Fixes#2388
Previously `#[tokio::test]` would error on functions that took
arguments. That meant other attribute macros couldn't do further
transformations on them. This changes that so arguments are forwarded as
is.
Whatever else might be included on the function is forwarded as well.
For example return type, generics, etc.
Worth noting that this is only for compatibility with other macros.
`#[test]`s that take arguments will still fail to compile.
A bit odd that [trybuild] tests don't fail `#[test]` functions with
arguments which is why the new tests are run with `t.pass(...)`. They do
actually fail if part of a real crate.
[trybuild]: https://crates.io/crates/trybuild
When aborting a task registered with a current-thread scheduler from off runtime, the tasks may not
be immediately unlinked from the runtime. Instead, send a message to the runtime, notifying it to
remove the aborted task.
Fixes#3662
## Motivation
The `tokio::sync::Semaphore` type provides a
[`Semaphore::available_permits` method][1] which returns the current
number of permits available on the semaphore.
`tokio_util::sync::PollSemaphore` [does not expose such a method][2]. It
is possible to use `PollSemaphore::into_inner` or
`PollSemaphore::clone_inner` to unwrap the inner semaphore, but this may
require cloning/dropping the semaphore's `Arc` which shouldn't be
necessary to access the permits.
## Solution
This commit adds `PollSemaphore::available_permits`. It also adds
`PollSemaphore::add_permits` and an `AsRef<Semaphore>` impl while
we're here.
[1]: https://docs.rs/tokio/1.4.0/tokio/sync/struct.Semaphore.html#method.available_permits
[2]: https://docs.rs/tokio-util/0.6.5/tokio_util/sync/struct.PollSemaphore.html#implementationsCloses#3682
Improves a few of the error messages for `#[tokio::main]` and `#[tokio::test]`.
Also adds a note to the docs about `start_paused` requiring the `test-util` feature which wasn't mentioned previously.
Add `runtime::Handle::block_on`. The function enters the runtime context and
blocks the current thread while the future executes.
Refs: #3097Fixes#2965, #3096
I ran into this when writing the docs for `Handle::block_on`. Seems to
be caused by a bug in cargo. Until that is fixed I think it makes sense
to mention it in the contributing guide.
It is tempting to use very large `Duration` value to get a practically
infinite timeout.
Before this commit Tokio panics on checked Instant + Duration
overflow.
This commit implements very simple fix: if Instant + Duration
overflows, we use duration = 30 years. Better fix should avoid
firing a timer on duration overflow. It requires deeper understanding
how timers work, but also it is not clear, for example, what
`Sleep::deadline` function should return.
Similar fix is done for `sleep`.
The trait-level documentation accurately describes this method, but the
function-level documentation imples that an implementer should overwrite
the entire buf, rather than just the unfilled portion.
Before this commit `close` after successful `try_recv` panics.
My use case is this: on drop, I call `close` to prevent pushing a
message to the queue, and then fetch the message if any and process
it.
But if message is already processed, `close` panics. And there is
no API to know if message was already fetched or not (except for
writing a wrapped which would track that info, which would be an
overkill).
But generally `close` operation should be safe to be called any
time.
Since ed5a94eb2d, there is no longer a
limit of 32,768 threads on the worker threads and blocking thread pool,
but the documentation has not been updated. This commit removes that
part of the documentation.
The time driver stores an Instant internally used as a "base" for future
time calculations. Since this is generated as the Runtime is being
constructed, it previously always happened before the user had a chance
to pause time. The fractional-millisecond variations in the timing
around the runtime construction and time pause cause tests running
entirely in paused time to be very slightly deterministic, with the time
driver advancing time by 1 millisecond more or less depending on how the
sub-millisecond components of the `Instant`s involved compared.
To avoid this, there is now a new option on `runtime::Builder` which
will create a `Runtime` with time "instantly" paused. This, along with a
small change to have the time driver use the provided clock as the
source for its start time allow totally deterministic tests with paused
time.
* Add `TryInto<Stdio>` impls for `ChildStd{in,out,err}` for ergonomic
conversions into `std::process::Stdio` so callers can perform the
conversion without needing to manipulate raw fds/handles directly
The Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [[email protected]](mailto:[email protected]). Security issues should not be reported via the public Github Issue tracker.
The Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [[email protected]](mailto:[email protected]). Security issues should not be reported via the public GitHub Issue tracker.
## Vulnerability coordination
Remediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [Github Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.
Remediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [GitHub Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.
Downstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, Github username(s) and any other salient information to [[email protected]](mailto:[email protected]). Participation in security issue coordination processes is at the discretion of the Tokio team.
Downstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, GitHub username(s) and any other salient information to [[email protected]](mailto:[email protected]). Participation in security issue coordination processes is at the discretion of the Tokio team.
## Security advisories
The project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project Github Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).
The project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project GitHub Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Carl Lerche
dtolnay