This commit factors out the common task queue logic in the basic
scheduler runtime and the `LocalSet` struct in `tokio::task`. This is
because as more work was done on the `LocalSet`, it has gotten closer
and closer to the basic scheduler in behavior, and factoring out the
shared code reduces the risk of errors caused by `LocalSet` not doing
something that the basic scheduler does.
In particular, I noticed the basic scheduler has a flag that indicates
the remote queue has been closed, which is set when dropping the
scheduler. This prevents tasks from being added after the scheduler has
started shutting down, stopping a potential task leak. Rather than
duplicating this code in `LocalSet`, I thought it was probably better to
factor it out into a shared type.
There are a few cases where there are small differences in behavior,
though, so there is still a need for separate types implemented _using_
the new `Queues` struct. However, it should cover most of the identical
code.
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
Currently, `tokio::task::LocalSet`'s `block_on` method requires the
future to live for the 'static lifetime. However, this bound is not
required — the future is wrapped in a `LocalFuture`, and then passed
into `Runtime::block_on`, which does _not_ require a `'static` future.
This came up while updating `tokio-compat` to work with version 0.2. To
mimic the behavior of `tokio` 0.1's `current_thread::Runtime::run`, we
want to be able to have a runtime block on the `recv` future from an
mpsc channel indicating when the runtime is idle. To support `!Send`
futures, as the old `current_thread::Runtime` did, we must do so inside
of a `LocalSet`. However, with the current bounds, we cannot await an
`mpsc::Receiver`'s `recv` future inside the `LocalSet::block_on` call.
## Solution
This branch removes the unnecessary `'static` bound.
Signed-off-by: Eliza Weisman <[email protected]>
Adds `read_buf` and `write_buf` which work with `T: BufMut` and `T: Buf`
respectively. This adds an easy API for using the buffer traits provided
by `bytes.
`oneshot::Receiver::try_recv` does not provide any information as to the
reason **why** receiving failed. The two cases are that the channel is
empty or that the channel closed.
`TryRecvError` is changed to be an enum of those two cases. This is
backwards compatible as `TryRecvError` was an opaque struct.
This also expands on `oneshot` API documentation, adding details and
examples.
Closes#1872
Provide convenience methods for encoding and decoding big-endian numbers
on top of asynchronous I/O streams. Only primitive types are provided
(24 and 48 bit numbers are omitted).
In general, using these methods won't be the fastest way to do
encoding/decoding with asynchronous byte streams, but they help to get
simple things working fast.
The "global executor" thread-local is to track where to spawn new tasks,
**not** which scheduler is active on the current thread. This fixes a
bug with scheduling tasks on the basic_scheduler by tracking the
currently active basic_scheduler with a dedicated thread-local variable.
Fixes: #1851
In version 0.1 there was File::into_std method that destructured
tokio_fs::File into std::fs:File. That method was lacking in
version 0.2.
Fixes: #1852
It turns out that the `Scheduler::release` method on `LocalSet`'s
`Scheduler` *is* called, when the `Scheduler` is dropped with tasks
still running. Currently, that method is `unreachable!`, which means
that dropping a `LocalSet` with tasks running will panic.
This commit fixes the panic, by pushing released tasks to
`pending_drop`. This is the same as `BasicScheduler`.
Fixes#1842
This fixes the API docs for both `TcpListener::incoming` and
`UnixListener::incoming`. The function now takes `&mut self` instead of
`self`. Adds an example for both function.
## Motivation
In earlier versions of `tokio`, the `current_thread::Runtime` type could
be used to run `!Send` futures. However, PR #1716 merged the
current-thread and threadpool runtimes into a single type, which can no
longer run `!Send` futures. There is still a need in some cases to
support futures that don't implement `Send`, and the `tokio-compat`
crate requires this in order to provide APIs that existed in `tokio`
0.1.
## Solution
This branch implements the API described by @carllerche in
https://github.com/tokio-rs/tokio/pull/1716#issuecomment-549496309. It
adds a new `LocalSet` type and `spawn_local` function to `tokio::task`.
The `LocalSet` type is used to group together a set of tasks which must
run on the same thread and don't implement `Send`. These are available
when a new "rt-util" feature flag is enabled.
Currently, the local task set is run by passing it a reference to a
`Runtime` and a future to `block_on`. In the future, we may also want
to investigate allowing spawned futures to construct their own local
task sets, which would be executed on the worker that the future is
executing on.
In order to implement the new API, I've made some internal changes to
the `task` module and `Schedule` trait to support scheduling both `Send`
and `!Send` futures.
Signed-off-by: Eliza Weisman <[email protected]>
This provides the ability to get the raw OS handle for a `File`. The
`Into*` variant cannot be provided as `File` needs to maintain ownership
of the `File`. The actual handle may have been moved to a background
thread.
This updates the tokio `Command` and `Child` behavior to match that of
the stdlib: spawned processes will *not* be automatically killed when
the handle is dropped
Unlike the stdlib, any dropped (unix) processes may be reaped by tokio
behind-the-scenes after they exit and if new processes are awaited,
which mitigates the risks of piling up unreaped zombie unix processes
A `Command::kill_on_drop` method is added to allow the caller to
control whether the spawned child should be killed when the handle is
dropped. By default, this value is `false`.
The `Child::forget` method has been removed, as it is superseded by
`Command::kill_on_drop`
Changes the set of `default` feature flags to `[]`. By default, only
core traits are included without specifying feature flags. This makes it
easier for users to pick the components they need.
For convenience, a `full` feature flag is included that includes all
components.
Tests are configured to require the `full` feature. Testing individual
feature flags will need to be moved to a separate crate.
Closes#1791
This directory was deleted when `cargo hack` was introduced, however
there were some tests that were still useful (macro failure output).
Also, additional build tests will be added over time.
Annotates types in `tokio::io` module with their required feature flag.
This annotation is included in generated documentation.
Notes:
* The annotation must be on the type or function itself. Annotating just
the re-export is not sufficient.
* The annotation must be **inside** the `pin_project!` macro or it is
lost.
* runtime: cleanup and add config options
This patch finishes the cleanup as part of the transition to Tokio 0.2.
A number of changes were made to take advantage of having all Tokio
types in a single crate. Also, fixes using Tokio types from
`spawn_blocking`.
* Many threads, one resource driver
Previously, in the threaded scheduler, a resource driver (mio::Poll /
timer combo) was created per thread. This was more or less fine, except
it required balancing across the available drivers. When using a
resource driver from **outside** of the thread pool, balancing is
tricky. The change was original done to avoid having a dedicated driver
thread.
Now, instead of creating many resource drivers, a single resource driver
is used. Each scheduler thread will attempt to "lock" the resource
driver before parking on it. If the resource driver is already locked,
the thread uses a condition variable to park. Contention should remain
low as, under load, the scheduler avoids using the drivers.
* Add configuration options to enable I/O / time
New configuration options are added to `runtime::Builder` to allow
enabling I/O and time drivers on a runtime instance basis. This is
useful when wanting to create lightweight runtime instances to execute
compute only tasks.
* Bug fixes
The condition variable parker is updated to the same algorithm used in
`std`. This is motivated by some potential deadlock cases discovered by
`loom`.
The basic scheduler is fixed to fairly schedule tasks. `push_front` was
accidentally used instead of `push_back`.
I/O, time, and spawning now work from within `spawn_blocking` closures.
* Misc cleanup
The threaded scheduler is no longer generic over `P :Park`. Instead, it
is hard coded to a specific parker. Tests, including loom tests, are
updated to use `Runtime` directly. This provides greater coverage.
The `blocking` module is moved back into `runtime` as all usage is
within `runtime` itself.
## Motivation
Tokio's crate-level docs are currently pretty sparse, and in some cases
reference old names for APIs. Before 0.2 is released, they could use a
fresh coat of paint.
## Solution
This branch reworks and expands the `lib.rs` docs. In particular, I've
added a new "A Tour of Tokio" section, inspired by the [standard
library's similarly-named section][std]. This section lists all of
`tokio`'s public modules, and summarizes their major APIs. It also lists
the feature flags necessary to enable those APIs.
[std]: https://doc.rust-lang.org/std/index.html#a-tour-of-the-rust-standard-library
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
The `tokio::runtime` module's docs need to be updated to
track recent changes.
## Solution
This branch updates and expands the `runtime` docs.
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
The `tokio::io` module's docs are fairly sparse and not particularly up
to date. They ought to be improved before release.
## Solution
This branch adds new module-level docs to `tokio::io`. The new docs are
largely inspired by `std::io`'s documentation, and highlight the
similarities and differences between `tokio::io` and `std::io`.
Signed-off-by: Eliza Weisman <[email protected]>
## Motivation
The new `tokio::task` module is pretty lacking in API docs.
## Solution
This branch adds new API docs to the `task` module, including:
* Module-level docs with a summary of the differences between
tasks and threads
* Examples of how to use the `task` APIs in the module-level docs
* More docs for `yield_now`
* More docs and examples for `JoinHandle`, based on the
`std::thread::JoinHandle` API docs.
This branch contains commits cherry-picked from #1794
Signed-off-by: Eliza Weisman <[email protected]>
Eliza Weisman