Compare commits

...
Author SHA1 Message Date
Noah Kennedy 89a87d5836 checkpoint 2022-09-21 11:31:12 -05:00
Carl Lerche d69e5bebf1 chore: release tokio-stream 0.1.10 (#5028)
Closes #5011
2022-09-18 10:11:03 -07:00
Carl Lerche 0b92f80a65 rt: rm internal Unpark types for Handle types (#5027)
This patch removes all internal `Unpark` runtime types. The runtime now
uses the `Handle` types (`runtime::io::Handle` and
`runtime::time::Handle`) to signal threads to unpark.

Without separate `Unpark` types, future patches will be able to remove
more `Arc`s used inside various drivers.
2022-09-18 10:10:49 -07:00
Carl Lerche cba5c1009e rt: move driver unpark out of multi-thread parker (#5026)
This patch removes the driver Unpark handle out of the multi-thread
parker and passes a reference in when it is needed. This is a first step
towards getting rid of the separate driver unpark handle in favor of
just using the regular driver handle. Because the regular driver handle
is owned at a higher level (at the top of the worker struct).
2022-09-17 16:46:44 -07:00
Carl Lerche cdd6eeaf70 rt: update MultiThread to use its own Handle (#5025)
This is the equivalent as tokio-rs/tokio#5023, but for the MultiThread
scheduler. This patch updates `MultiThread` to use `Arc<Handle>` for all
of its cross-thread needs instead of `Arc<Shared>`.

The effect of this change is that the multi-thread scheduler only has a
single `Arc` type, which includes the driver handles, keeping all
"shared state" together.
2022-09-16 19:57:45 -07:00
Carl Lerche 2effa7ff8a rt: update CurrentThread to use its own Handle (#5023)
This is the next step in the internal runtime refactoring. This patch
updates `CurrentThread` to use `Arc<Handle>` for all of its cross-thread
needs instead of `Arc<Shared>`.

The effect of this change is that the current-thread scheduler only has
a single `Arc` type, which includes the driver handles, keeping all
"shared state" together.
2022-09-16 17:34:09 -07:00
Carl Lerche ebeb78ed40 rt: split internal runtime::Handle concerns (#5022)
The `runtime::Handle` struct is part of the public API but is also used
internally. This has created a bit of tension. An earlier patch made
defined Handle as a private struct in some cases when `rt` is not
enabled.

This patch splits out internal handle concerns into a new
`scheduler::Handle` type, which will only be internal. This also defines
a `Handle` type for each scheduler variant. Eventually, the
per-scheduler `Handle` types will replace the per-scheduler `Spawner`
types, but more work is needed before we can make that change.
2022-09-16 14:05:09 -07:00
Hayden Stainsby b5709baa91 rt: add rng_seed option to runtime::Builder (#4910)
The `tokio::select!` macro polls branches in a random order. While this
is desirable in production, for testing purposes a more deterministic
approach can be useul.

This change adds an additional parameter `rng_seed` to the runtime
`Builder` to set the random number generator seed. This value is then
used to reset the seed on the current thread when the runtime is entered
into (restoring the previous value when the thread leaves the runtime). All
threads created explicitly by the runtime also have a seed set as the
runtime is built. Each thread is set with a seed from a deterministic
sequence.

This guarantees that calls to the `tokio::select!` macro which are
performed in the same order on the same thread will poll branches in the
same order.

Additionally, the peer chosen to attempt to steal work from also uses a
deterministic sequence if `rng_seed` is set.

Both the builder parameter as well as the `RngSeed` struct are marked
unstable initially.
2022-09-16 17:41:09 +02:00
Carl Lerche 9e02759779 rt: create driver::Handle struct (#5018)
Move all individual driver handles into a single `driver::Handle`
struct.
2022-09-15 16:31:55 -07:00
Carl Lerche 198e2d8e79 rt: rename internal runtime::Kind to Scheduler (#5017)
The `runtime::Kind` enum doesn't really represent the runtime flavor,
but is an enumeration of the different scheduler types. This patch
renames the enum to reflect this.

At a later time, it is likely the `Scheduler` enum will be moved to
`tokio::runtime::scheduler`, but that is punted to a later PR.

This rename is to make space for other enums
2022-09-15 15:10:08 -07:00
Carl Lerche 57e08e7147 chore: remove println from tests (#5015)
This patch removes println statements from tests. Some of these were
used to test that types implement `fmt::Debug`. These println statements
have been replaced with an equivalent test that does not output to
STDOUT. The rest of the println statements are most likely left over
from debugging sessions.
2022-09-14 18:43:40 -07:00
Carl Lerche 98e642d234 rt: remove driver drop implementations (#5014)
Currently, drivers are responsible for clean up when they are dropped.
This is possible today because each driver struct holds a reference to
its internal handle.

As part of an effort to decouple drivers from their handles, this patch
removes the drop implementations for the time and IO driver in favor of
having the scheduler explicitly call `shutdown()` as part of its
shutdown process. The scheduler will hold a reference to both the driver
and the driver handles, so in the future, it will be able to pass in the
driver handles as part of the shutdown process.
2022-09-14 15:47:41 -07:00
Carl Lerche 3f379abda4 rt: remove unparker from time driver (#5013)
Currently, the various resource drivers use a layered approach where
each driver contains the next one. When the scheduler calls park on the
top-most driver, it does work then calls park on the inner driver it
holds. The handles do the same with unparking.

This patch is a step towards refactoring the runtime to move away from
the nested approach towards keeping all drivers and handles together in
a single runtime driver/handle. The unparker is removed from the time
handle and placed in the runtime handle and is passed into the time
driver as needed.
2022-09-14 15:38:42 -07:00
Carl Lerche 588408c060 rt: update TimerEntry to use runtime::Handle
The `TimerEntry` struct is the internal integration point for public
time APIs (`sleep`, `interval`, ...) with the time driver. Currently,
`TimerEntry` holds an ref-counted reference to the time driver handle.

This patch replaces the reference to the time driver handle with a
reference to the runtime handle. This is part of a larger effort to
consolate internal handles across the runtime.
2022-09-14 15:38:42 -07:00
Taiki Endo ac1ae2cfbc ci: use latest Valgrind (#4367) 2022-09-14 21:49:32 +09:00
Carl Lerche 56ffea09e5 rt: store driver handles next to scheduler handle (#5008)
In an earlier PR (#4629), driver handles were moved into the scheduler
handle (`Spawner`). This was done to let the multi-threaded scheduler
have direct access to the thread pool spawner.

However, we are now working on a greater decoupling of the runtime
internals. All drivers and schedulers will be peers, stored in a single
thread-local variable, and the scheduler will be passed the full
runtime::Handle. This will achieve the original goal of giving the
scheduler access to the thread-pool while also (hopefully) simplifying
other aspects of the code.
2022-09-13 12:40:46 -07:00
Taiki Endo b891714bdb ci: use --feature-powerset --depth 2 in features check (#5007)
As has been pointed out a few times in the past (e.g., #4036 (comment)), each-feature 
is not sufficient for features check.

Ideally, we'd like to check all combinations of features, but there are too many
combinations. So limit the max number of simultaneous feature flags to 2 by --depth
option. I think this should be sufficient in most cases as @carllerche said in
taiki-e/cargo-hack#58.
2022-09-13 11:05:38 -07:00
Jernej Kos 0fddb765d4 ci: add build test for x86_64-fortanix-unknown-sgx target (#5001) 2022-09-13 15:12:35 +02:00
Alice Ryhl 29e3584c4d ci: don't auto-cancel CI on old commits on master (#5006) 2022-09-13 13:13:27 +02:00
Alice Ryhl 0d68bef7f7 Merge 'tokio-1.21.1' into 'master' 2022-09-13 11:43:54 +02:00
Alice Ryhl dea1cd4995 chore: prepare Tokio v1.21.1 (#5003) 2022-09-13 11:20:59 +02:00
Alice Ryhl e4cbc70279 task: ignore failure to set TLS in LocalSet Drop (#4976) 2022-09-13 08:51:04 +02:00
Alice Ryhl 97e981e797 net: fix dependency resolution for socket2 (#5000) 2022-09-13 08:50:30 +02:00
Carl Lerche 3a4f18b93b rt: remove internal runtime::ToHandle trait (#5002)
The internal `runtime::ToHandle` trait is no longer needed as the
runtime handle is used everywhere now.
2022-09-12 16:18:17 -07:00
Yotam Ofek 1c823093cb codec: fix LengthDelimitedCodec buffer over-reservation (#4997) 2022-09-11 15:58:51 +02:00
Hayden Stainsby 0b1f640308 io: remove reference to missing functions from Registration docs (#4995)
The docs for `Registration::new_with_interest_and_handle` (internal
function) included a reference to two old functions `new_with_interest`
and `new` on the same struct. These functions no longer exist, which
could be confusing.

Since there is currently only a single constructor function on
`Registration`, the misleading text has simply been removed.
2022-09-09 20:36:42 +02:00
Carl Lerche d7b3b33c9c rt: move spawn_blocking methods to blocking mod (#4994)
A few spawn_blocking methods were implemented on a Handle struct that is
intended to hold all runtime driver handles. These methods make more
sense in the blocking module.
2022-09-08 13:38:20 -07:00
Carl Lerche 71b29b9409 rt: remove Handle reference from time driver (#4993)
A step towards the goal of consolidating thread-local variables, this
patch removes the `time::Handle` reference from the time driver struct.
The reference is moved a level up and passed into methods that need it.

The end goal is to have a single `Driver` struct that holds each
sub-driver and a single `Handle` ref-counted struct that holds all the
misc handles. Further work on the time driver is blocked by some other
refactoring that will happen in follow-up patches.
2022-09-08 11:42:08 -07:00
Piotr 3966acf966 io: rewrite immediate_exit_on_error test to use io::Mock (#4984) 2022-09-08 11:39:06 +02:00
Alice Ryhl dfdb550cd4 chore: prepare tokio-util 0.7.4 (#4987) 2022-09-08 10:34:23 +02:00
Carl Lerche 99aa8d12b7 rt: rm internal Park,Unpark traits (#4991)
Years ago, Tokio was organized as a cluster of separate crates. This
architecture required a trait to represent "park the thread and do
work.". When all crates were combined into a single crate, the `Park`
and `Unpark` traits remained as internal details.

This patch removes these traits as they are no longer needed. This is in
service of a future refactor that will decouple the various resource
drivers and store them in a single struct instead of nesting them.
However, in the mean time, removing the Park/Unpark traits adds a bit of
messy code to make the conditional compilation work. This code will
(hopefully) be short-lived.
2022-09-07 18:39:45 -07:00
Carl Lerche 2ad347465e rt: minor time driver refactors (#4989)
This patch makes some minor refactors. It renames `ClockTime` to
`TimeSource` since that is how all variables refer to it. It also moves
the type into a new file.

Finally, it moves the `unpark` handle out of the mutex as it does not
need to be there. Note, the call to `unpark` is still called while the
mutex is held, so there is no functional change. Moving it out of the
mutex is in preparation for moving the unpark handle completely out of
the time driver.
2022-09-07 13:24:02 -07:00
Alex Rudy 291fce8de3 io: add error handling example to StreamReader (#4975) 2022-09-07 13:58:06 +00:00
Adam Chalmers 0267516214 net: fix links in UnixStream docs (#4985) 2022-09-07 08:40:55 +00:00
Carl Lerche bbfd34f6a3 rt: move time driver into runtime module (#4983)
This patch moves the time driver into the runtime module. The time driver
is a runtime concern and is only used by the runtime. Moving the drivers
is the first step to cleaning up some Tokio internals. There will be
follow-up patches that integrate the drivers and other runtime concerns
more closely.

This is an internal refactor and should not impact any public APIs.
2022-09-06 14:12:09 -07:00
Alice Ryhl 116fa7c614 task: introduce RcCell helper (#4977) 2022-09-06 13:40:07 +02:00
Hsiang-Cheng Yang 1fd7f82468 task: fix a doc typo in LocalKey::sync_scope (#4971) 2022-09-03 13:57:18 +00:00
Noah Kennedy c2612b446f io: reduce syscalls in poll_write (#4970)
This applies the same optimization made in #4840 to writes.
2022-09-02 15:23:17 -05:00
Alice Ryhl 50795e652e chore: prepare Tokio v1.21.0 (#4967) 2022-09-02 13:51:31 +02:00
Alice Ryhl a6a95bb4a6 wasm: add documentation for wasm support (#4966) 2022-09-02 11:39:38 +02:00
Alice Ryhl ce5d2a466f task: fix some doc and test things related to stabilizing JoinSet (#4965) 2022-09-01 21:39:20 +00:00
Alice Ryhl e5467ca767 tokio: increase LTS duration to one year (#4964) 2022-09-01 21:30:51 +00:00
Jan Behrens 431ec68f93 sync: doc of watch::Sender::send improved (#4959)
Current documentation of `sync::watch::Sender::send` may be intepreted
as the method failing if at some point in past the channel has been
closed because every receiver has been dropped. This isn't true,
however, as the channel could have been reopened by using
`sync::watch::Sender::subscribe`.

This fix clarifies the behavior. Moreover, it is noted that on failure,
the value isn't made available to future subscribers (but returned as
part of the `SendError`).

Fixes #4957.
2022-09-01 21:11:01 +02:00
Alice Ryhl f207e1afe4 tokio: make 1.20.x an LTS release (#4962) 2022-09-01 21:08:31 +02:00
Alice RyhlandTaiki Endo d3cae06b5e wasm: use thread::sleep on non-wasi wasm too (#4963)
Co-authored-by: Taiki Endo <[email protected]>
2022-09-01 16:52:56 +02:00
Alice Ryhl 01ebb0aa29 task: fix warning (#4960) 2022-08-31 15:42:30 +02:00
Jon Kelley adc774bc4f Feature: Add more windows signal handlers (#4924) 2022-08-31 06:29:31 +00:00
TennyZhuang 5a2bc850af task: fix incorrect signature in Builder::spawn_on (#4953) 2022-08-30 09:23:53 +00:00
Vladimir ba93b28033 signal: make SignalKind methods const (#4956) 2022-08-29 19:47:55 +00:00
Clint Frederickson 505cc0901d time: add a comment to Interval::tick example (#4951) 2022-08-27 18:02:56 +00:00
Carl Lerche a3411a412c rt, chore: rename internal scheduler types (#4945)
For historical reasons, the current-thread runtime's scheduler was named
BasicScheduler and the multi-thread runtime's scheduler were named
ThreadPool. This patch renames the schedulers to mirror the runtime
names to increase consistency. This patch also moves the scheduler
implementations into a new `runtime::scheduler` module.
2022-08-26 09:47:19 -07:00
Carl Lerche 218f2629ff rt: move I/O driver into runtime module (#4942)
This patch moves the I/O driver into the runtime module. The I/O driver
is a runtime concern and is only used by the runtime. Moving the driver
is the first step to cleaning up some Tokio internals. There will be
follow-up patches that integrate the I/O driver and other runtime concerns
more closely.

Because `Interest` and `Ready` are public APIs, they were moved to the
top-level `io` module instead of moving the types to `runtime`.

This is an internal refactor and should not impact any public APIs.
2022-08-25 12:58:57 -07:00
Isaac Sikkema a66884a2fb net: add documentation to try_read() for zero-length buffers (#4937) 2022-08-25 09:33:25 +00:00
Carl Lerche b023522a37 rt: add unstable option to disable the LIFO slot (#4936)
The multi-threaded scheduler includes a per-worker LIFO slot to
store the last scheduled task. This can improve certain usage patterns,
especially message passing between tasks. However, this LIFO slot is not
currently stealable.

Eventually, the LIFO slot **will** become stealable. However, as a
stop-gap, this unstable option lets users disable the LIFO task when
doing so improves their application's overall performance.

Refs: #4941
2022-08-24 12:48:30 -07:00
imlk 733931d85f io: add SyncIoBridge::shutdown() (#4938) 2022-08-24 12:18:53 +00:00
Carl Lerche df28ac092f rt: extract basic_scheduler::Config (#4935) 2022-08-24 13:49:30 +02:00
John DiSanti d720770b07 ci: add cargo-check-external-types check (#4914) 2022-08-24 13:35:32 +02:00
Hayden Stainsby c9d444e8e0 doc: correct cargo doc command in contrib guide (#4933)
The command to build the documentation locally provided in the
Contribution Guide did not work for all crates in the project workspace.
Specifically, to build the docs for `tokio-stream` the flag `--cfg
docsrs` needs to be in the environment variable `RUSTFLAGS` in addition
to being in `RUSTDOCFLAGS`.

Additionally, there was text describing that the docs cannot be built
from the root of the workspace with a link to rust-lang/cargo#9274. That
issue has since been closed as complete and the listed commands do now
work from the root of the workspace. As such, that text has been
removed.
2022-08-24 13:33:23 +02:00
Hayden Stainsby e005b6c899 io: remove PollEvented docs reference to clear_read_ready (#4931)
The documentation for`PollEvented` had a single remaining reference to
`Registration::clear_read_ready`, which no longer exists. This change
updates the documentation to refer to `Registration::clear_readiness`
instead.
2022-08-23 08:47:59 +02:00
Clint Frederickson f5c1ff7599 chore: slight re-wording of unconstrained's module docs (#4928) 2022-08-21 14:15:18 -05:00
Noah Kennedy b67b8c1398 chore: stabilize JoinSet and AbortHandle (#4920)
Closes #4535.

This leaves the ID-related APIs unstable.
2022-08-19 17:16:11 +00:00
Linda_pp de81985762 docs: fix indent of tokio::sync::watch example code (#4922) 2022-08-18 09:12:58 -05:00
John DiSanti 4b1c4801b1 ci: upgrade to nightly-2022-07-26 and pin miri nightly (#4915) 2022-08-15 21:09:47 +00:00
weegee b3c7c9846b sync: add mpsc::Sender::max_capacity method (#4904) 2022-08-14 15:52:33 +00:00
John DiSanti d416b1d1d6 ci: upgrade nightly to nightly-2022-07-25 (#4903)
Upgrade the nightly to a version that works with all of miri,
cargo-hack, minimal-versions, and cargo-check-external-types.

This is a prerequisite for #4902.
2022-08-12 16:12:09 +00:00
Alice Ryhl 6e7a630243 docs: add changelog links to readme (#4907) 2022-08-12 15:57:53 +02:00
Alice Ryhl 16427f98c1 docs: link to GitHub discussions on "new issue" page (#4906) 2022-08-12 10:49:50 +00:00
Alice Ryhl 50b8ee99b7 ci: fix new error output from rustc 1.63.0 (#4905) 2022-08-12 08:57:30 +00:00
Alice Ryhl bdbb0ece10 task: add cancel safety docs to JoinHandle (#4901) 2022-08-11 11:17:03 +00:00
Sebastian Pütz 53e127205b sync: remove unused lifetime on SemaphorePermit impl (#4896) 2022-08-11 12:12:36 +02:00
Sebastian Pütz cd0ff7fbfe fs: change panic to error in File::start_seek (#4897) 2022-08-11 09:39:43 +00:00
Ivan Petkov d48c4370ce signal: don't register write interest on signal pipe (#4898) 2022-08-10 18:53:01 -07:00
Hayden Stainsby 9d9488db67 sync: remove broadcast channel slot level closed flag (#4867)
The broadcast channel allows multiple senders to send messages to
multiple receivers, where each receiver receives messages starting from
when it subscribes. After all senders are dropped, the receivers will
continue to receive all waiting messages in the buffer and then receive
a `Closed` error.

To mark that a channel has closed, it stores two closed flags, one on
the channel level and another in the buffer slot *after* the last used
slot (this may also be the earliest entry being kept for lagged
receivers, see #2425).

However, we don't need both closed flags, keeping the channel level
closed flag is sufficient.

Without the slot level closed flag, each receiver receives each message
until it is up to date and for that receiver the channel is empty. Then,
the actual return message is chosen depending on the channel level
closed flag; if the channel is NOT closed, then `Empty` is returned, if
the channel is closed then `Closed` is returned instead.

With the modified logic, there is no longer a need to append a closed
token to the internal buffer (by setting the slot level closed flag on
the next slot). This fixes the off by one error described in #4814,
which caused a receiver which was created after the channel was already
closed to get `Empty` from `try_recv` (or hang forever when calling
`recv`) instead of receiving `Closed`.

As a bonus, we save a single `bool` on each buffer slot.

Refs: #4814
2022-08-10 21:30:05 +02:00
Lioness100 53cf021b81 chore: fix word conjugations (#4894)
* chore: fix word tenses

* hyphenate
2022-08-10 11:50:25 -05:00
Tomio aea09478e1 io: fix typo in AsyncSeekExt::rewind docs (#4893) 2022-08-10 14:54:36 +00:00
Campbell He 2099d0bd87 net: add device and bind_device methods to TCP/UDP sockets (#4882) 2022-08-10 15:30:08 +02:00
Hayden Stainsby 255c1f95b7 doc: update clippy command in contribution guide (#4883)
The contribution guide describes the Tokio code of conduct and gives a
list of steps to follow when contributing in different ways. In the
guide to contributing a pull request, commands are listed to be run by
the contributor locally to help ensure that the PR will pass on CI.

The command to run clippy isn't the same as the one run on CI,
specifically the command doesn't check the tests. This commit changes
the command to match the one on CI, so that tests are checked by clippy
as well.
2022-08-09 23:46:21 +02:00
xxchan ff6fbc327d sync: add #[must_use] to lock guards (#4886) 2022-08-09 13:44:29 +02:00
199878e287 net: add tos and set_tos methods to TCP and UDP sockets (#4877)
Co-authored-by: Campbell He <[email protected]>
Co-authored-by: Taiki Endo <[email protected]>
2022-08-01 19:53:20 +02:00
Josh Soref 5ab6aaf3cd ci: scope workflows (#4857)
Signed-off-by: Josh Soref <[email protected]>
2022-07-30 13:55:53 +02:00
b-naber 1d75b57ad0 sync: implement Weak version of mpsc::Sender (#4595) 2022-07-27 15:36:52 +02:00
Alice RyhlandTaiki Endo b2ada60e70 wasm: use build script to detect wasm (#4865)
Co-authored-by: Taiki Endo <[email protected]>
2022-07-26 11:26:54 +00:00
Alice Ryhl 0dc62da21b chore: use target_family wasm instead of target_arch wasm32 (#4864) 2022-07-26 07:47:39 +00:00
Noah KennedyandCarl Lerche 28ce4eeab2 io: reduce syscalls in poll_read (#4840)
As the [epoll documentation points out](https://man7.org/linux/man-pages/man7/epoll.7.html), a read that only partially fills a buffer is sufficient to show that the socket buffer has been drained.

We can take advantage of this by clearing readiness in this case, which seems to significantly improve performance under certain conditions.

Co-authored-by: Carl Lerche <[email protected]>
2022-07-26 05:52:30 +00:00
Alice Ryhl ffff9e65fd chore: fix version detection for addr_of! (#4863) 2022-07-25 13:46:38 +00:00
Alice Ryhl 0906351daf Merge 'master' and 'tokio-1.20.1' 2022-07-25 14:21:40 +02:00
Josh Soref f3e340a35b chore: fix spelling mistakes (#4858)
Signed-off-by: Josh Soref <[email protected]>
2022-07-25 07:26:01 +00:00
Ivan Petkov 94a7eaed51 Revert "tests: alter integration tests for stdio to not use a handrolled cat implementation (#4803)" (#4854)
This reverts commit d8cad13fd9.
2022-07-23 16:27:35 -07:00
Hayden Stainsby 62593be261 doc: remove incorrect panic section for Builder::worker_threads (#4849)
The documentation for the runtime `Builder::worker_threads` function
incorrectly stated that it would panic if used when constructing a
`current_thread` runtime. In truth, the call to the function has no
effect.

Since adding the described panic to the code could cause new panics in
existing code using tokio, the documentation has been modified to
describe the existing behavior.

Refs: #4773
2022-07-23 19:44:00 +02:00
Ivan Petkov 1578575db8 process: use blocking threadpool for child stdio I/O (#4824) 2022-07-23 10:40:04 -07:00
Hayden Stainsby 8c482a5d62 io: add track_caller caller to no-rt io driver (#4852)
This change adds a case that was missing from the original PR, #4793.

The `io::driver::Handle::current` function was only covered by
`#[track_caller]` in the case that the `rt` feature  is enabled, however
it was missing in the case that the `rt` feture isn't enabled (in which
case a panic would be more common).

This particular case cannot be tested in the tokio tests as they always
run with all features enabled.

Refs: #4413
2022-07-21 15:25:42 +00:00
Hayden Stainsby 5288e1e144 task: add track_caller to public APIs (#4848)
Functions that may panic can be annotated with `#[track_caller]` so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.

This change adds `#[track_caller]` to all the public APIs in the task
module of the tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs.

In cases where `#[track_caller]` does not work, it has been left out.
For example, it currently does not work on async functions, blocks, or
closures. So any call stack that passes through one of these before
reaching the actual panic is not able to show the calling site outside
of tokio as the panic location.

The following functions have call stacks that pass through closures:
* `task::block_in_place`
* `task::local::spawn_local`

Tests are included to cover each potentially panicking function.

The following functions already had `#[track_caller]` applied everywhere
it was needed and only tests have been added:
* `task::spawn`
* `task::LocalKey::sync_scope`

Refs: #4413
2022-07-20 23:17:41 +02:00
Alice Ryhl 56be5286ee ci: update required ci steps (#4846) 2022-07-20 14:05:23 +02:00
dwattttt 21900bd42b net: add security flags to named pipe ServerOptions (#4845) 2022-07-20 11:13:39 +00:00
Alice Ryhl 228d4fce99 runtime: move owned field to Trailer (#4843) 2022-07-19 20:28:20 +02:00
Hayden Stainsby cbdba8bd32 net: add track_caller to public APIs (#4805)
Functions that may panic can be annotated with #[track_caller] so that
in the event of a panic, the function where the user called the
panicking function is shown instead of the file and line within Tokio
source.

This change adds #[track_caller] to all the non-unstable public net APIs
in the main tokio crate where the documentation describes how the
function may panic due to incorrect context or inputs.  Not all panic
cases can have #[track_caller] applied fully as the callstack passes
through a closure which isn't yet supported by the annotation (e.g. net
functions called from outside a tokio runtime).

Additionally, the documentation was updated to indicate additional cases
in which the public net functions may panic (the same as the io
functions).

Tests are included to cover each potentially panicking function.

Refs: #4413
2022-07-18 20:57:45 +00:00
Alice Ryhl 6f048ca954 tokio: avoid temporary references in linked_list::Link impls (#4841) 2022-07-18 22:47:17 +02:00
Arnav Singh 922fc91408 task: propagate attributes on task-locals (#4837) 2022-07-14 11:32:28 +02:00
Noah Kennedy 5cf22e48c5 chore: allow miri to use latest nightly once again (#4833)
This reverts [#4825], as miri seems to be working again on the latest nightly.

I had originally thought it was this issue with rustup which was to blame, but this seems to be wrong: https://github.com/rust-lang/rustup/issues/3031

[#4825]: https://github.com/tokio-rs/tokio/pull/4825
2022-07-13 11:35:28 -05:00
gftea 14fca343d5 task: add LocalSet::enter (#4736) (#4765) 2022-07-13 18:10:09 +02:00
Colin Walters 8e20cfb9ef task: expand on cancellation of spawn_blocking (#4811)
I was looking at these docs again and I was thinking the
"cannot be cancelled" is misleading; let's talk about approaches
to do so.
2022-07-13 16:41:13 +02:00
Alice Ryhl d7b074eca3 Merge branch 'tokio-1.20.x' into 'master' 2022-07-13 08:48:27 +02:00
Ivan Petkov 3b6c74a40a Make task::Builder::spawn* methods fallible (#4823) 2022-07-12 15:56:33 -07:00
Eric Zhang de686b5355 runtime: fix typo in shutdown_background docs (#4829) 2022-07-12 20:51:28 +00:00
Richard Zak 6d3f92dddc wasm: initial support for wasm32-wasi target (#4716)
This adds initial, unstable, support for the wasm32-wasi target. Not all of Tokio's
features are supported yet as WASI's non-blocking APIs are still limited.

Refs: tokio-rs/tokio#4827
2022-07-12 13:20:26 -07:00
244 changed files with 5692 additions and 3250 deletions
+3
View File
@@ -11,6 +11,7 @@ env:
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 64-bit
auto_cancellation: $CIRRUS_BRANCH != 'master' && $CIRRUS_BRANCH !=~ 'tokio-.*'
setup_script:
- pkg install -y bash curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
@@ -25,6 +26,7 @@ task:
task:
name: FreeBSD docs
auto_cancellation: $CIRRUS_BRANCH != 'master' && $CIRRUS_BRANCH !=~ 'tokio-.*'
env:
RUSTFLAGS: --cfg docsrs --cfg tokio_unstable
RUSTDOCFLAGS: --cfg docsrs --cfg tokio_unstable -Dwarnings
@@ -42,6 +44,7 @@ task:
task:
name: FreeBSD 32-bit
auto_cancellation: $CIRRUS_BRANCH != 'master' && $CIRRUS_BRANCH !=~ 'tokio-.*'
setup_script:
- pkg install -y bash curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
+4
View File
@@ -0,0 +1,4 @@
contact_links:
- name: Question
url: https://github.com/tokio-rs/tokio/discussions
about: Questions about Tokio should be posted as a GitHub discussion.
-16
View File
@@ -1,16 +0,0 @@
---
name: Question
about: Please use the discussions tab for questions
title: ''
labels: ''
assignees: ''
---
Please post your question as a discussion here:
https://github.com/tokio-rs/tokio/discussions
You may also be able to find help here:
https://discord.gg/tokio
https://users.rust-lang.org/
+75 -19
View File
@@ -11,7 +11,7 @@ env:
RUST_BACKTRACE: 1
# Change to specific Rust release to pin
rust_stable: stable
rust_nightly: nightly-2022-04-18
rust_nightly: nightly-2022-07-26
rust_clippy: 1.52.0
# When updating this, also update:
# - README.md
@@ -34,21 +34,25 @@ jobs:
runs-on: ubuntu-latest
needs:
- test
- test-unstable
- test-parking_lot
- valgrind
- test-unstable
- miri
- asan
- cross
- features
- minrust
- minimal-versions
- fmt
- clippy
- docs
- valgrind
- loom-compile
- check-readme
- test-hyper
- x86_64-fortanix-unknown-sgx
- wasm32-unknown-unknown
- wasm32-wasi
- check-external-types
steps:
- run: exit 0
@@ -72,7 +76,7 @@ jobs:
run: rustup update stable
- uses: Swatinem/rust-cache@v1
- name: Install cargo-hack
run: cargo install cargo-hack
uses: taiki-e/install-action@cargo-hack
# Run `tokio` with `full` features. This excludes testing utilities which
# can alter the runtime behavior of Tokio.
@@ -138,9 +142,7 @@ jobs:
- uses: Swatinem/rust-cache@v1
- name: Install Valgrind
run: |
sudo apt-get update -y
sudo apt-get install -y valgrind
uses: taiki-e/install-action@valgrind
# Compile tests
- name: cargo build test-mem
@@ -195,7 +197,7 @@ jobs:
- name: Install Rust ${{ env.rust_nightly }}
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-2022-07-10
toolchain: ${{ env.rust_nightly }}
components: miri
override: true
- uses: Swatinem/rust-cache@v1
@@ -275,12 +277,12 @@ jobs:
override: true
- uses: Swatinem/rust-cache@v1
- name: Install cargo-hack
run: cargo install cargo-hack
- name: check --each-feature
run: cargo hack check --all --each-feature -Z avoid-dev-deps
uses: taiki-e/install-action@cargo-hack
- name: check --feature-powerset
run: cargo hack check --all --feature-powerset --depth 2 -Z avoid-dev-deps --keep-going
# Try with unstable feature flags
- name: check --each-feature --unstable
run: cargo hack check --all --each-feature -Z avoid-dev-deps
- name: check --feature-powerset --unstable
run: cargo hack check --all --feature-powerset --depth 2 -Z avoid-dev-deps --keep-going
env:
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
@@ -310,7 +312,7 @@ jobs:
override: true
- uses: Swatinem/rust-cache@v1
- name: Install cargo-hack
run: cargo install cargo-hack
uses: taiki-e/install-action@cargo-hack
- name: "check --all-features -Z minimal-versions"
run: |
# Remove dev-dependencies from Cargo.toml to prevent the next `cargo update`
@@ -447,6 +449,23 @@ jobs:
git diff
cargo test --features full
x86_64-fortanix-unknown-sgx:
name: build tokio for x86_64-fortanix-unknown-sgx
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Rust ${{ env.rust_nightly }}
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.rust_nightly }}
target: x86_64-fortanix-unknown-sgx
override: true
- uses: Swatinem/rust-cache@v1
# NOTE: Currently the only test we can run is to build tokio with rt and sync features.
- name: build tokio
run: cargo build --target x86_64-fortanix-unknown-sgx --features rt,sync
working-directory: tokio
wasm32-unknown-unknown:
name: test tokio for wasm32-unknown-unknown
runs-on: ubuntu-latest
@@ -478,22 +497,59 @@ jobs:
# Install dependencies
- name: Install cargo-hack
run: cargo install cargo-hack
uses: taiki-e/install-action@cargo-hack
- name: Install wasm32-wasi target
run: rustup target add wasm32-wasi
- name: Install wasmtime
run: cargo install wasmtime-cli
uses: taiki-e/install-action@wasmtime
- name: Install cargo-wasi
run: cargo install cargo-wasi
# TODO: Expand this when full WASI support lands.
# Currently, this is a bare bones regression test
# for features that work today with wasi.
- name: WASI test tokio full
run: cargo test -p tokio --target wasm32-wasi --features full
env:
CARGO_TARGET_WASM32_WASI_RUNNER: "wasmtime run --"
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
- name: WASI test tokio-util full
run: cargo test -p tokio-util --target wasm32-wasi --features full
env:
CARGO_TARGET_WASM32_WASI_RUNNER: "wasmtime run --"
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
- name: WASI test tokio-stream
run: cargo test -p tokio-stream --target wasm32-wasi --features time,net,io-util,sync
env:
CARGO_TARGET_WASM32_WASI_RUNNER: "wasmtime run --"
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
- name: test tests-integration --features wasi-rt
# TODO: this should become: `cargo hack wasi test --each-feature`
run: cargo wasi test --test rt_yield --features wasi-rt
working-directory: tests-integration
check-external-types:
name: check-external-types
runs-on: ${{ matrix.os }}
strategy:
matrix:
os:
- windows-latest
- ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install Rust ${{ env.rust_nightly }}
uses: actions-rs/toolchain@v1
with:
toolchain: ${{ env.rust_nightly }}
override: true
- uses: Swatinem/rust-cache@v1
- name: check-external-types
run: |
set -x
cargo install cargo-check-external-types --locked --version 0.1.3
cargo check-external-types --all-features --config external-types.toml
working-directory: tokio
+1
View File
@@ -7,6 +7,7 @@ on:
jobs:
triage:
runs-on: ubuntu-latest
if: github.repository_owner == 'tokio-rs'
steps:
- uses: actions/labeler@v3
with:
+1 -1
View File
@@ -17,7 +17,7 @@ jobs:
loom:
name: loom
# base_ref is null when it's not a pull request
if: contains(github.event.pull_request.labels.*.name, 'R-loom') || (github.base_ref == null)
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom') || (github.base_ref == null))
runs-on: ubuntu-latest
strategy:
matrix:
+3 -5
View File
@@ -12,7 +12,7 @@ env:
rust_stable: stable
jobs:
stess-test:
stress-test:
name: Stress Test
runs-on: ubuntu-latest
strategy:
@@ -28,9 +28,7 @@ jobs:
override: true
- uses: Swatinem/rust-cache@v1
- name: Install Valgrind
run: |
sudo apt-get update -y
sudo apt-get install -y valgrind
uses: taiki-e/install-action@valgrind
# Compiles each of the stress test examples.
- name: Compile stress test examples
@@ -38,4 +36,4 @@ jobs:
# Runs each of the examples using Valgrind. Detects leaks and displays them.
- name: Run valgrind
run: valgrind --leak-check=full --show-leak-kinds=all ./target/release/examples/${{ matrix.stress-test }}
run: valgrind --error-exitcode=1 --leak-check=full --show-leak-kinds=all ./target/release/examples/${{ matrix.stress-test }}
+4 -10
View File
@@ -146,7 +146,7 @@ When updating this, also update:
-->
```
cargo +1.49.0 clippy --all-features
cargo +1.49.0 clippy --all --tests --all-features
```
When building documentation normally, the markers that list the features
@@ -154,7 +154,7 @@ required for various parts of Tokio are missing. To build the documentation
correctly, use this command:
```
RUSTDOCFLAGS="--cfg docsrs" cargo +nightly doc --all-features
RUSTDOCFLAGS="--cfg docsrs" RUSTFLAGS="--cfg docsrs" cargo +nightly doc --all-features
```
To build documentation including Tokio's unstable features, it is necessary to
@@ -162,15 +162,9 @@ pass `--cfg tokio_unstable` to both RustDoc *and* rustc. To build the
documentation for unstable features, use this command:
```
RUSTDOCFLAGS="--cfg docsrs --cfg tokio_unstable" RUSTFLAGS="--cfg tokio_unstable" cargo +nightly doc --all-features
RUSTDOCFLAGS="--cfg docsrs --cfg tokio_unstable" RUSTFLAGS="--cfg docsrs --cfg tokio_unstable" cargo +nightly doc --all-features
```
There is currently a [bug in cargo] that means documentation cannot be built
from the root of the workspace. If you `cd` into the `tokio` subdirectory the
command shown above will work.
[bug in cargo]: https://github.com/rust-lang/cargo/issues/9274
The `cargo fmt` command does not work on the Tokio codebase. You can use the
command below instead:
@@ -562,7 +556,7 @@ Tokio ≥1.0.0 comes with LTS guarantees:
The goal of these guarantees is to provide stability to the ecosystem.
## Mininum Supported Rust Version (MSRV)
## Minimum Supported Rust Version (MSRV)
* All Tokio ≥1.0.0 releases will support at least a 6-month old Rust
compiler release.
+18 -8
View File
@@ -56,7 +56,7 @@ Make sure you activated the full features of the tokio crate on Cargo.toml:
```toml
[dependencies]
tokio = { version = "1.20.1", features = ["full"] }
tokio = { version = "1.21.1", features = ["full"] }
```
Then, on your main.rs:
@@ -161,6 +161,16 @@ several other libraries, including:
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Changelog
The Tokio repository contains multiple crates. Each crate has its own changelog.
* `tokio` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio/CHANGELOG.md)
* `tokio-util` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-util/CHANGELOG.md)
* `tokio-stream` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-stream/CHANGELOG.md)
* `tokio-macros` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-macros/CHANGELOG.md)
* `tokio-test` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-test/CHANGELOG.md)
## Supported Rust Versions
<!--
@@ -192,18 +202,18 @@ warrants a patch release with a fix for the bug, it will be backported and
released as a new patch release for each LTS minor version. Our current LTS
releases are:
* `1.14.x` - LTS release until June 2022.
* `1.18.x` - LTS release until January 2023
* `1.18.x` - LTS release until June 2023
* `1.20.x` - LTS release until September 2023.
Each LTS release will continue to receive backported fixes for at least half a
year. If you wish to use a fixed minor release in your project, we recommend
that you use an LTS release.
Each LTS release will continue to receive backported fixes for at least a year.
If you wish to use a fixed minor release in your project, we recommend that you
use an LTS release.
To use a fixed minor version, you can specify the version with a tilde. For
example, to specify that you wish to use the newest `1.14.x` patch release, you
example, to specify that you wish to use the newest `1.18.x` patch release, you
can use the following dependency specification:
```text
tokio = { version = "~1.14", features = [...] }
tokio = { version = "~1.18", features = [...] }
```
## License
+4 -4
View File
@@ -1,13 +1,13 @@
## Report a security issue
The Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [[email protected]](mailto:[email protected]). Security issues should not be reported via the public Github Issue tracker.
The Tokio project team welcomes security reports and is committed to providing prompt attention to security issues. Security issues should be reported privately via [[email protected]](mailto:[email protected]). Security issues should not be reported via the public GitHub Issue tracker.
## Vulnerability coordination
Remediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [Github Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.
Remediation of security vulnerabilities is prioritized by the project team. The project team coordinates remediation with third-party project stakeholders via [GitHub Security Advisories](https://help.github.com/en/github/managing-security-vulnerabilities/about-github-security-advisories). Third-party stakeholders may include the reporter of the issue, affected direct or indirect users of Tokio, and maintainers of upstream dependencies if applicable.
Downstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, Github username(s) and any other salient information to [[email protected]](mailto:[email protected]). Participation in security issue coordination processes is at the discretion of the Tokio team.
Downstream project maintainers and Tokio users can request participation in coordination of applicable security issues by sending your contact email address, GitHub username(s) and any other salient information to [[email protected]](mailto:[email protected]). Participation in security issue coordination processes is at the discretion of the Tokio team.
## Security advisories
The project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project Github Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).
The project team is committed to transparency in the security issue disclosure process. The Tokio team announces security issues via [project GitHub Release notes](https://github.com/tokio-rs/tokio/releases) and the [RustSec advisory database](https://github.com/RustSec/advisory-db) (i.e. `cargo-audit`).
+5 -5
View File
@@ -14,7 +14,7 @@ fn read_uncontended(b: &mut Bencher) {
rt.block_on(async move {
for _ in 0..6 {
let read = lock.read().await;
black_box(read);
let _read = black_box(read);
}
})
});
@@ -28,7 +28,7 @@ fn read_concurrent_uncontended_multi(b: &mut Bencher) {
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
let _read = black_box(read);
}
let lock = Arc::new(RwLock::new(()));
@@ -55,7 +55,7 @@ fn read_concurrent_uncontended(b: &mut Bencher) {
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
let _read = black_box(read);
}
let lock = Arc::new(RwLock::new(()));
@@ -82,7 +82,7 @@ fn read_concurrent_contended_multi(b: &mut Bencher) {
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
let _read = black_box(read);
}
let lock = Arc::new(RwLock::new(()));
@@ -110,7 +110,7 @@ fn read_concurrent_contended(b: &mut Bencher) {
async fn task(lock: Arc<RwLock<()>>) {
let read = lock.read().await;
black_box(read);
let _read = black_box(read);
}
let lock = Arc::new(RwLock::new(()));
+8
View File
@@ -1,2 +1,10 @@
Tests the various combination of feature flags. This is broken out to a separate
crate to work around limitations with cargo features.
To run all of the tests in this directory, run the following commands:
```
cargo test --features full
cargo test --features rt
```
If one of the tests fail, you can pass `TRYBUILD=overwrite` to the `cargo test`
command that failed to have it regenerate the test output.
@@ -1,5 +1,5 @@
error: The default runtime flavor is `multi_thread`, but the `rt-multi-thread` feature is disabled.
--> tests/fail/macros_core_no_default.rs:3:1
--> $DIR/macros_core_no_default.rs:3:1
|
3 | #[tokio::main]
| ^^^^^^^^^^^^^^
@@ -1,4 +1,4 @@
error: function is never used: `f`
error: function `f` is never used
--> $DIR/macros_dead_code.rs:6:10
|
6 | async fn f() {}
-2
View File
@@ -7,7 +7,6 @@ publish = false
[[bin]]
name = "test-cat"
required-features = ["rt-process-io-util"]
[[bin]]
name = "test-mem"
@@ -31,7 +30,6 @@ name = "rt_yield"
required-features = ["rt", "macros", "sync"]
[features]
rt-process-io-util = ["tokio/rt", "tokio/macros", "tokio/process", "tokio/io-util", "tokio/io-std"]
# For mem check
rt-net = ["tokio/rt", "tokio/rt-multi-thread", "tokio/net"]
# For test-process-signal
+15 -9
View File
@@ -1,14 +1,20 @@
//! A cat-like utility that can be used as a subprocess to test I/O
//! stream communication.
use tokio::io::AsyncWriteExt;
use std::io;
use std::io::Write;
#[tokio::main(flavor = "current_thread")]
async fn main() {
let mut stdin = tokio::io::stdin();
let mut stdout = tokio::io::stdout();
tokio::io::copy(&mut stdin, &mut stdout).await.unwrap();
stdout.flush().await.unwrap();
fn main() {
let stdin = io::stdin();
let mut stdout = io::stdout();
let mut line = String::new();
loop {
line.clear();
stdin.read_line(&mut line).unwrap();
if line.is_empty() {
break;
}
stdout.write_all(line.as_bytes()).unwrap();
}
stdout.flush().unwrap();
}
+5 -1
View File
@@ -1,4 +1,8 @@
#![cfg(all(feature = "macros", feature = "rt-multi-thread"))]
#![cfg(all(
feature = "macros",
feature = "rt-multi-thread",
not(target_os = "wasi")
))]
#[tokio::main]
async fn basic_main() -> usize {
+1
View File
@@ -4,6 +4,7 @@ use futures::channel::oneshot;
use futures::executor::block_on;
use std::thread;
#[cfg_attr(target_os = "wasi", ignore = "WASI: std::thread::spawn not supported")]
#[test]
fn join_with_select() {
block_on(async {
+3 -13
View File
@@ -1,5 +1,5 @@
#![warn(rust_2018_idioms)]
#![cfg(feature = "full")]
#![cfg(all(feature = "full", not(target_os = "wasi")))]
use tokio::io::{AsyncBufReadExt, AsyncReadExt, AsyncWriteExt, BufReader};
use tokio::join;
@@ -8,22 +8,12 @@ use tokio_test::assert_ok;
use futures::future::{self, FutureExt};
use std::convert::TryInto;
use std::env;
use std::io;
use std::process::{ExitStatus, Stdio};
// so, we need to change this back as a test, but for now this doesn't work because of:
// https://github.com/rust-lang/rust/pull/95469
//
// undo when this is closed: https://github.com/tokio-rs/tokio/issues/4802
// fn cat() -> Command {
// let mut cmd = Command::new(std::env!("CARGO_BIN_EXE_test-cat"));
// cmd.stdin(Stdio::piped()).stdout(Stdio::piped());
// cmd
// }
fn cat() -> Command {
let mut cmd = Command::new("cat");
let mut cmd = Command::new(env!("CARGO_BIN_EXE_test-cat"));
cmd.stdin(Stdio::piped()).stdout(Stdio::piped());
cmd
}
+8
View File
@@ -1,3 +1,11 @@
# 0.1.10 (Sept 18, 2022)
- time: add `StreamExt::chunks_timeout` ([#4695])
- stream: add track_caller to public APIs ([#4786])
[#4695]: https://github.com/tokio-rs/tokio/pull/4695
[#4786]: https://github.com/tokio-rs/tokio/pull/4786
# 0.1.9 (June 4, 2022)
- deps: upgrade `tokio-util` dependency to `0.7.x` ([#3762])
+2 -1
View File
@@ -4,7 +4,7 @@ name = "tokio-stream"
# - Remove path dependencies
# - Update CHANGELOG.md.
# - Create "tokio-stream-0.1.x" git tag.
version = "0.1.9"
version = "0.1.10"
edition = "2018"
rust-version = "1.49"
authors = ["Tokio Contributors <[email protected]>"]
@@ -38,6 +38,7 @@ parking_lot = "0.12.0"
tokio-test = { path = "../tokio-test" }
futures = { version = "0.3", default-features = false }
[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
proptest = "1"
[package.metadata.docs.rs]
+1 -1
View File
@@ -1,5 +1,5 @@
#![warn(rust_2018_idioms)]
#![cfg(feature = "time")]
#![cfg(all(feature = "time", not(target_os = "wasi")))] // Wasi does not support panic recovery
use parking_lot::{const_mutex, Mutex};
use std::error::Error;
+1
View File
@@ -325,6 +325,7 @@ fn one_ready_many_none() {
}
}
#[cfg(not(target_os = "wasi"))]
proptest::proptest! {
#[test]
fn fuzz_pending_complete_mix(kinds: Vec<bool>) {
+1 -1
View File
@@ -260,7 +260,7 @@ macro_rules! assert_err {
}};
}
/// Asserts that an exact duration has elapsed since since the start instant ±1ms.
/// Asserts that an exact duration has elapsed since the start instant ±1ms.
///
/// ```rust
/// use tokio::time::{self, Instant};
+22
View File
@@ -1,3 +1,25 @@
# 0.7.4 (September 8, 2022)
### Added
- io: add `SyncIoBridge::shutdown()` ([#4938])
- task: improve `LocalPoolHandle` ([#4680])
### Fixed
- util: add `track_caller` to public APIs ([#4785])
### Unstable
- task: fix compilation errors in `JoinMap` with Tokio v1.21.0 ([#4755])
- task: remove the unstable, deprecated `JoinMap::join_one` ([#4920])
[#4680]: https://github.com/tokio-rs/tokio/pull/4680
[#4755]: https://github.com/tokio-rs/tokio/pull/4755
[#4785]: https://github.com/tokio-rs/tokio/pull/4785
[#4920]: https://github.com/tokio-rs/tokio/pull/4920
[#4938]: https://github.com/tokio-rs/tokio/pull/4938
# 0.7.3 (June 4, 2022)
### Changed
+2 -2
View File
@@ -4,7 +4,7 @@ name = "tokio-util"
# - Remove path dependencies
# - Update CHANGELOG.md.
# - Create "tokio-util-0.7.x" git tag.
version = "0.7.3"
version = "0.7.4"
edition = "2018"
rust-version = "1.49"
authors = ["Tokio Contributors <[email protected]>"]
@@ -34,7 +34,7 @@ rt = ["tokio/rt", "tokio/sync", "futures-util", "hashbrown"]
__docs_rs = ["futures-util"]
[dependencies]
tokio = { version = "1.19.0", path = "../tokio", features = ["sync"] }
tokio = { version = "1.21.0", path = "../tokio", features = ["sync"] }
bytes = "1.0.0"
futures-core = "0.3.0"
futures-sink = "0.3.0"
+3 -7
View File
@@ -522,15 +522,11 @@ impl LengthDelimitedCodec {
}
};
let num_skip = self.builder.get_num_skip();
if num_skip > 0 {
src.advance(num_skip);
}
src.advance(self.builder.get_num_skip());
// Ensure that the buffer has enough space to read the incoming
// payload
src.reserve(n);
src.reserve(n.saturating_sub(src.len()));
Ok(Some(n))
}
@@ -568,7 +564,7 @@ impl Decoder for LengthDelimitedCodec {
self.state = DecodeState::Head;
// Make sure the buffer has enough space to read the next head
src.reserve(self.builder.num_head_bytes());
src.reserve(self.builder.num_head_bytes().saturating_sub(src.len()));
Ok(Some(data))
}
+49
View File
@@ -50,9 +50,58 @@ pin_project! {
/// # }
/// ```
///
/// If the stream produces errors which are not [std::io::Error],
/// the errors can be converted using [`StreamExt`] to map each
/// element.
///
/// ```
/// use bytes::Bytes;
/// use tokio::io::AsyncReadExt;
/// use tokio_util::io::StreamReader;
/// use tokio_stream::StreamExt;
/// # #[tokio::main]
/// # async fn main() -> std::io::Result<()> {
///
/// // Create a stream from an iterator, including an error.
/// let stream = tokio_stream::iter(vec![
/// Result::Ok(Bytes::from_static(&[0, 1, 2, 3])),
/// Result::Ok(Bytes::from_static(&[4, 5, 6, 7])),
/// Result::Err("Something bad happened!")
/// ]);
///
/// // Use StreamExt to map the stream and error to a std::io::Error
/// let stream = stream.map(|result| result.map_err(|err| {
/// std::io::Error::new(std::io::ErrorKind::Other, err)
/// }));
///
/// // Convert it to an AsyncRead.
/// let mut read = StreamReader::new(stream);
///
/// // Read five bytes from the stream.
/// let mut buf = [0; 5];
/// read.read_exact(&mut buf).await?;
/// assert_eq!(buf, [0, 1, 2, 3, 4]);
///
/// // Read the rest of the current chunk.
/// assert_eq!(read.read(&mut buf).await?, 3);
/// assert_eq!(&buf[..3], [5, 6, 7]);
///
/// // Reading the next chunk will produce an error
/// let error = read.read(&mut buf).await.unwrap_err();
/// assert_eq!(error.kind(), std::io::ErrorKind::Other);
/// assert_eq!(error.into_inner().unwrap().to_string(), "Something bad happened!");
///
/// // We have now reached the end.
/// assert_eq!(read.read(&mut buf).await?, 0);
///
/// # Ok(())
/// # }
/// ```
///
/// [`AsyncRead`]: tokio::io::AsyncRead
/// [`Stream`]: futures_core::Stream
/// [`ReaderStream`]: crate::io::ReaderStream
/// [`StreamExt`]: tokio_stream::StreamExt
#[derive(Debug)]
pub struct StreamReader<S, B> {
#[pin]
+15
View File
@@ -66,6 +66,21 @@ impl<T: AsyncWrite> SyncIoBridge<T> {
}
}
impl<T: AsyncWrite + Unpin> SyncIoBridge<T> {
/// Shutdown this writer. This method provides a way to call the [`AsyncWriteExt::shutdown`]
/// function of the inner [`tokio::io::AsyncWrite`] instance.
///
/// # Errors
///
/// This method returns the same errors as [`AsyncWriteExt::shutdown`].
///
/// [`AsyncWriteExt::shutdown`]: tokio::io::AsyncWriteExt::shutdown
pub fn shutdown(&mut self) -> std::io::Result<()> {
let src = &mut self.src;
self.rt.block_on(src.shutdown())
}
}
impl<T: Unpin> SyncIoBridge<T> {
/// Use a [`tokio::io::AsyncRead`] synchronously as a [`std::io::Read`] or
/// a [`tokio::io::AsyncWrite`] as a [`std::io::Write`].
+1
View File
@@ -29,6 +29,7 @@ cfg_codec! {
}
cfg_net! {
#[cfg(not(target_arch = "wasm32"))]
pub mod udp;
pub mod net;
}
@@ -200,7 +200,7 @@ where
/// `parent` MUST have been a parent of the node when they both got locked,
/// otherwise there is a potential for a deadlock as invariant #2 would be violated.
///
/// To aquire the locks for node and parent, use [with_locked_node_and_parent].
/// To acquire the locks for node and parent, use [with_locked_node_and_parent].
fn move_children_to_parent(node: &mut Inner, parent: &mut Inner) {
// Pre-allocate in the parent, for performance
parent.children.reserve(node.children.len());
@@ -218,7 +218,7 @@ fn move_children_to_parent(node: &mut Inner, parent: &mut Inner) {
/// Removes a child from the parent.
///
/// `parent` MUST be the parent of `node`.
/// To aquire the locks for node and parent, use [with_locked_node_and_parent].
/// To acquire the locks for node and parent, use [with_locked_node_and_parent].
fn remove_child(parent: &mut Inner, mut node: MutexGuard<'_, Inner>) {
// Query the position from where to remove a node
let pos = node.parent_idx;
@@ -80,7 +80,7 @@ fn drop_token_no_child() {
}
#[test]
fn drop_token_with_childs() {
fn drop_token_with_children() {
loom::model(|| {
let token1 = CancellationToken::new();
let child_token1 = token1.child_token();
-7
View File
@@ -416,7 +416,6 @@ where
/// * `None` if the `JoinMap` is empty.
///
/// [`tokio::select!`]: tokio::select
#[doc(alias = "join_one")]
pub async fn join_next(&mut self) -> Option<(K, Result<V, JoinError>)> {
let (res, id) = match self.tasks.join_next_with_id().await {
Some(Ok((id, output))) => (Ok(output), id),
@@ -430,12 +429,6 @@ where
Some((key, res))
}
#[doc(hidden)]
#[deprecated(since = "0.7.4", note = "renamed to `JoinMap::join_next`.")]
pub async fn join_one(&mut self) -> Option<(K, Result<V, JoinError>)> {
self.join_next().await
}
/// Aborts all tasks and waits for them to finish shutting down.
///
/// Calling this method is equivalent to calling [`abort_all`] and then calling [`join_next`] in
+2
View File
@@ -2,7 +2,9 @@
#[cfg(tokio_unstable)]
mod join_map;
#[cfg(not(target_os = "wasi"))]
mod spawn_pinned;
#[cfg(not(target_os = "wasi"))]
pub use spawn_pinned::LocalPoolHandle;
#[cfg(tokio_unstable)]
+1 -1
View File
@@ -190,7 +190,7 @@ impl<T> SlabStorage<T> {
let key_contained = self.key_map.contains_key(&key.into());
if key_contained {
// It's possible that a `compact` call creates capacitiy in `self.inner` in
// It's possible that a `compact` call creates capacity in `self.inner` in
// such a way that a `self.inner.insert` call creates a `key` which was
// previously given out during an `insert` call prior to the `compact` call.
// If `key` is contained in `self.key_map`, we have encountered this exact situation,
+1
View File
@@ -1,4 +1,5 @@
#![cfg(feature = "rt")]
#![cfg(not(target_os = "wasi"))] // Wasi doesn't support threads
#![warn(rust_2018_idioms)]
use tokio::runtime::Builder;
+1 -1
View File
@@ -130,7 +130,7 @@ fn write_hits_backpressure() {
_ => unreachable!(),
}
// Push a new new chunk
// Push a new chunk
mock.calls.push_back(Ok(b[..].to_vec()));
}
// 1 'wouldblock', 4 * 2KB buffers, 1 b-byte buffer
+21 -2
View File
@@ -1,8 +1,9 @@
#![cfg(feature = "io-util")]
#![cfg(not(target_os = "wasi"))] // Wasi doesn't support threads
use std::error::Error;
use std::io::{Cursor, Read, Result as IoResult};
use tokio::io::AsyncRead;
use std::io::{Cursor, Read, Result as IoResult, Write};
use tokio::io::{AsyncRead, AsyncReadExt};
use tokio_util::io::SyncIoBridge;
async fn test_reader_len(
@@ -41,3 +42,21 @@ async fn test_async_write_to_sync() -> Result<(), Box<dyn Error>> {
assert_eq!(dest.as_slice(), src);
Ok(())
}
#[tokio::test]
async fn test_shutdown() -> Result<(), Box<dyn Error>> {
let (s1, mut s2) = tokio::io::duplex(1024);
let (_rh, wh) = tokio::io::split(s1);
tokio::task::spawn_blocking(move || -> std::io::Result<_> {
let mut wh = SyncIoBridge::new(wh);
wh.write_all(b"hello")?;
wh.shutdown()?;
assert!(wh.write_all(b" world").is_err());
Ok(())
})
.await??;
let mut buf = vec![];
s2.read_to_end(&mut buf).await?;
assert_eq!(buf, b"hello");
Ok(())
}
+1 -1
View File
@@ -1,5 +1,5 @@
#![warn(rust_2018_idioms)]
#![cfg(feature = "full")]
#![cfg(all(feature = "full", not(target_os = "wasi")))] // Wasi doesn't support panic recovery
use parking_lot::{const_mutex, Mutex};
use std::error::Error;
+1
View File
@@ -1,4 +1,5 @@
#![warn(rust_2018_idioms)]
#![cfg(not(target_os = "wasi"))] // Wasi doesn't support threads
use std::rc::Rc;
use std::sync::Arc;
+7 -7
View File
@@ -258,7 +258,7 @@ fn cancel_only_all_descendants() {
let child2_token = token.child_token();
let grandchild_token = child1_token.child_token();
let grandchild2_token = child1_token.child_token();
let grandgrandchild_token = grandchild_token.child_token();
let great_grandchild_token = grandchild_token.child_token();
assert!(!parent_token.is_cancelled());
assert!(!token.is_cancelled());
@@ -267,7 +267,7 @@ fn cancel_only_all_descendants() {
assert!(!child2_token.is_cancelled());
assert!(!grandchild_token.is_cancelled());
assert!(!grandchild2_token.is_cancelled());
assert!(!grandgrandchild_token.is_cancelled());
assert!(!great_grandchild_token.is_cancelled());
let parent_fut = parent_token.cancelled();
let fut = token.cancelled();
@@ -276,7 +276,7 @@ fn cancel_only_all_descendants() {
let child2_fut = child2_token.cancelled();
let grandchild_fut = grandchild_token.cancelled();
let grandchild2_fut = grandchild2_token.cancelled();
let grandgrandchild_fut = grandgrandchild_token.cancelled();
let great_grandchild_fut = great_grandchild_token.cancelled();
pin!(parent_fut);
pin!(fut);
@@ -285,7 +285,7 @@ fn cancel_only_all_descendants() {
pin!(child2_fut);
pin!(grandchild_fut);
pin!(grandchild2_fut);
pin!(grandgrandchild_fut);
pin!(great_grandchild_fut);
assert_eq!(
Poll::Pending,
@@ -321,7 +321,7 @@ fn cancel_only_all_descendants() {
);
assert_eq!(
Poll::Pending,
grandgrandchild_fut
great_grandchild_fut
.as_mut()
.poll(&mut Context::from_waker(&waker))
);
@@ -339,7 +339,7 @@ fn cancel_only_all_descendants() {
assert!(child2_token.is_cancelled());
assert!(grandchild_token.is_cancelled());
assert!(grandchild2_token.is_cancelled());
assert!(grandgrandchild_token.is_cancelled());
assert!(great_grandchild_token.is_cancelled());
assert_eq!(
Poll::Ready(()),
@@ -367,7 +367,7 @@ fn cancel_only_all_descendants() {
);
assert_eq!(
Poll::Ready(()),
grandgrandchild_fut
great_grandchild_fut
.as_mut()
.poll(&mut Context::from_waker(&waker))
);
+2
View File
@@ -778,6 +778,7 @@ async fn compact_change_deadline() {
assert!(entry.is_none());
}
#[cfg_attr(target_os = "wasi", ignore = "FIXME: Does not seem to work with WASI")]
#[tokio::test(start_paused = true)]
async fn remove_after_compact() {
let now = Instant::now();
@@ -794,6 +795,7 @@ async fn remove_after_compact() {
assert!(panic.is_err());
}
#[cfg_attr(target_os = "wasi", ignore = "FIXME: Does not seem to work with WASI")]
#[tokio::test(start_paused = true)]
async fn remove_after_compact_poll() {
let now = Instant::now();
+1
View File
@@ -1,4 +1,5 @@
#![warn(rust_2018_idioms)]
#![cfg(not(target_os = "wasi"))] // Wasi doesn't support UDP
use tokio::net::UdpSocket;
use tokio_stream::StreamExt;
+105
View File
@@ -1,3 +1,108 @@
# 1.21.1 (September 13, 2022)
### Fixed
- net: fix dependency resolution for socket2 ([#5000])
- task: ignore failure to set TLS in `LocalSet` Drop ([#4976])
[#4976]: https://github.com/tokio-rs/tokio/pull/4976
[#5000]: https://github.com/tokio-rs/tokio/pull/5000
# 1.21.0 (September 2, 2022)
This release is the first release of Tokio to intentionally support WASM. The
`sync,macros,io-util,rt,time` features are stabilized on WASM. Additionally the
wasm32-wasi target is given unstable support for the `net` feature.
### Added
- net: add `device` and `bind_device` methods to TCP/UDP sockets ([#4882])
- net: add `tos` and `set_tos` methods to TCP and UDP sockets ([#4877])
- net: add security flags to named pipe `ServerOptions` ([#4845])
- signal: add more windows signal handlers ([#4924])
- sync: add `mpsc::Sender::max_capacity` method ([#4904])
- sync: implement Weak version of `mpsc::Sender` ([#4595])
- task: add `LocalSet::enter` ([#4765])
- task: stabilize `JoinSet` and `AbortHandle` ([#4920])
- tokio: add `track_caller` to public APIs ([#4805], [#4848], [#4852])
- wasm: initial support for `wasm32-wasi` target ([#4716])
### Fixed
- miri: improve miri compatibility by avoiding temporary references in `linked_list::Link` impls ([#4841])
- signal: don't register write interest on signal pipe ([#4898])
- sync: add `#[must_use]` to lock guards ([#4886])
- sync: fix hang when calling `recv` on closed and reopened broadcast channel ([#4867])
- task: propagate attributes on task-locals ([#4837])
### Changed
- fs: change panic to error in `File::start_seek` ([#4897])
- io: reduce syscalls in `poll_read` ([#4840])
- process: use blocking threadpool for child stdio I/O ([#4824])
- signal: make `SignalKind` methods const ([#4956])
### Internal changes
- rt: extract `basic_scheduler::Config` ([#4935])
- rt: move I/O driver into `runtime` module ([#4942])
- rt: rename internal scheduler types ([#4945])
### Documented
- chore: fix typos and grammar ([#4858], [#4894], [#4928])
- io: fix typo in `AsyncSeekExt::rewind` docs ([#4893])
- net: add documentation to `try_read()` for zero-length buffers ([#4937])
- runtime: remove incorrect panic section for `Builder::worker_threads` ([#4849])
- sync: doc of `watch::Sender::send` improved ([#4959])
- task: add cancel safety docs to `JoinHandle` ([#4901])
- task: expand on cancellation of `spawn_blocking` ([#4811])
- time: clarify that the first tick of `Interval::tick` happens immediately ([#4951])
### Unstable
- rt: add unstable option to disable the LIFO slot ([#4936])
- task: fix incorrect signature in `Builder::spawn_on` ([#4953])
- task: make `task::Builder::spawn*` methods fallible ([#4823])
[#4595]: https://github.com/tokio-rs/tokio/pull/4595
[#4716]: https://github.com/tokio-rs/tokio/pull/4716
[#4765]: https://github.com/tokio-rs/tokio/pull/4765
[#4805]: https://github.com/tokio-rs/tokio/pull/4805
[#4811]: https://github.com/tokio-rs/tokio/pull/4811
[#4823]: https://github.com/tokio-rs/tokio/pull/4823
[#4824]: https://github.com/tokio-rs/tokio/pull/4824
[#4837]: https://github.com/tokio-rs/tokio/pull/4837
[#4840]: https://github.com/tokio-rs/tokio/pull/4840
[#4841]: https://github.com/tokio-rs/tokio/pull/4841
[#4845]: https://github.com/tokio-rs/tokio/pull/4845
[#4848]: https://github.com/tokio-rs/tokio/pull/4848
[#4849]: https://github.com/tokio-rs/tokio/pull/4849
[#4852]: https://github.com/tokio-rs/tokio/pull/4852
[#4858]: https://github.com/tokio-rs/tokio/pull/4858
[#4867]: https://github.com/tokio-rs/tokio/pull/4867
[#4877]: https://github.com/tokio-rs/tokio/pull/4877
[#4882]: https://github.com/tokio-rs/tokio/pull/4882
[#4886]: https://github.com/tokio-rs/tokio/pull/4886
[#4893]: https://github.com/tokio-rs/tokio/pull/4893
[#4894]: https://github.com/tokio-rs/tokio/pull/4894
[#4897]: https://github.com/tokio-rs/tokio/pull/4897
[#4898]: https://github.com/tokio-rs/tokio/pull/4898
[#4901]: https://github.com/tokio-rs/tokio/pull/4901
[#4904]: https://github.com/tokio-rs/tokio/pull/4904
[#4920]: https://github.com/tokio-rs/tokio/pull/4920
[#4924]: https://github.com/tokio-rs/tokio/pull/4924
[#4928]: https://github.com/tokio-rs/tokio/pull/4928
[#4935]: https://github.com/tokio-rs/tokio/pull/4935
[#4936]: https://github.com/tokio-rs/tokio/pull/4936
[#4937]: https://github.com/tokio-rs/tokio/pull/4937
[#4942]: https://github.com/tokio-rs/tokio/pull/4942
[#4945]: https://github.com/tokio-rs/tokio/pull/4945
[#4951]: https://github.com/tokio-rs/tokio/pull/4951
[#4953]: https://github.com/tokio-rs/tokio/pull/4953
[#4956]: https://github.com/tokio-rs/tokio/pull/4956
[#4959]: https://github.com/tokio-rs/tokio/pull/4959
# 1.20.1 (July 25, 2022)
### Fixed
+13 -7
View File
@@ -6,7 +6,7 @@ name = "tokio"
# - README.md
# - Update CHANGELOG.md.
# - Create "v1.0.x" git tag.
version = "1.20.1"
version = "1.21.1"
edition = "2018"
rust-version = "1.49"
authors = ["Tokio Contributors <[email protected]>"]
@@ -58,6 +58,8 @@ net = [
"winapi/winbase",
"winapi/winnt",
"winapi/minwindef",
"winapi/accctrl",
"winapi/aclapi",
]
process = [
"bytes",
@@ -68,11 +70,11 @@ process = [
"mio/net",
"signal-hook-registry",
"winapi/handleapi",
"winapi/minwindef",
"winapi/processthreadsapi",
"winapi/threadpoollegacyapiset",
"winapi/winbase",
"winapi/winnt",
"winapi/minwindef",
]
# Includes basic task execution capabilities
rt = ["once_cell"]
@@ -112,11 +114,13 @@ pin-project-lite = "0.2.0"
bytes = { version = "1.0.0", optional = true }
once_cell = { version = "1.5.2", optional = true }
memchr = { version = "2.2", optional = true }
mio = { version = "0.8.1", optional = true }
socket2 = { version = "0.4.4", optional = true, features = [ "all" ] }
mio = { version = "0.8.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
parking_lot = { version = "0.12.0", optional = true }
[target.'cfg(not(any(target_arch = "wasm32", target_arch = "wasm64")))'.dependencies]
socket2 = { version = "0.4.4", optional = true, features = [ "all" ] }
# Currently unstable. The API exposed by these features may be broken at any time.
# Requires `--cfg tokio_unstable` to enable.
[target.'cfg(tokio_unstable)'.dependencies]
@@ -147,12 +151,14 @@ mockall = "0.11.1"
tempfile = "3.1.0"
async-stream = "0.3"
[target.'cfg(not(target_arch = "wasm32"))'.dev-dependencies]
[target.'cfg(not(any(target_arch = "wasm32", target_arch = "wasm64")))'.dev-dependencies]
proptest = "1"
rand = "0.8.0"
socket2 = "0.4"
[target.'cfg(target_arch = "wasm32")'.dev-dependencies]
[target.'cfg(not(all(any(target_arch = "wasm32", target_arch = "wasm64"), target_os = "unknown")))'.dev-dependencies]
rand = "0.8.0"
[target.'cfg(all(any(target_arch = "wasm32", target_arch = "wasm64"), not(target_os = "wasi")))'.dev-dependencies]
wasm-bindgen-test = "0.3.0"
[target.'cfg(target_os = "freebsd")'.dev-dependencies]
+18 -8
View File
@@ -56,7 +56,7 @@ Make sure you activated the full features of the tokio crate on Cargo.toml:
```toml
[dependencies]
tokio = { version = "1.20.1", features = ["full"] }
tokio = { version = "1.21.1", features = ["full"] }
```
Then, on your main.rs:
@@ -161,6 +161,16 @@ several other libraries, including:
[`mio`]: https://github.com/tokio-rs/mio
[`bytes`]: https://github.com/tokio-rs/bytes
## Changelog
The Tokio repository contains multiple crates. Each crate has its own changelog.
* `tokio` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio/CHANGELOG.md)
* `tokio-util` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-util/CHANGELOG.md)
* `tokio-stream` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-stream/CHANGELOG.md)
* `tokio-macros` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-macros/CHANGELOG.md)
* `tokio-test` - [view changelog](https://github.com/tokio-rs/tokio/blob/master/tokio-test/CHANGELOG.md)
## Supported Rust Versions
<!--
@@ -192,18 +202,18 @@ warrants a patch release with a fix for the bug, it will be backported and
released as a new patch release for each LTS minor version. Our current LTS
releases are:
* `1.14.x` - LTS release until June 2022.
* `1.18.x` - LTS release until January 2023
* `1.18.x` - LTS release until June 2023
* `1.20.x` - LTS release until September 2023.
Each LTS release will continue to receive backported fixes for at least half a
year. If you wish to use a fixed minor release in your project, we recommend
that you use an LTS release.
Each LTS release will continue to receive backported fixes for at least a year.
If you wish to use a fixed minor release in your project, we recommend that you
use an LTS release.
To use a fixed minor version, you can specify the version with a tilde. For
example, to specify that you wish to use the newest `1.14.x` patch release, you
example, to specify that you wish to use the newest `1.18.x` patch release, you
can use the following dependency specification:
```text
tokio = { version = "~1.14", features = [...] }
tokio = { version = "~1.18", features = [...] }
```
## License
+44
View File
@@ -10,8 +10,16 @@ const CONST_THREAD_LOCAL_PROBE: &str = r#"
}
"#;
const ADDR_OF_PROBE: &str = r#"
{
let my_var = 10;
::std::ptr::addr_of!(my_var)
}
"#;
fn main() {
let mut enable_const_thread_local = false;
let mut enable_addr_of = false;
match AutoCfg::new() {
Ok(ac) => {
@@ -34,6 +42,21 @@ fn main() {
enable_const_thread_local = true;
}
}
// The `addr_of` and `addr_of_mut` macros were stabilized in 1.51.
if ac.probe_rustc_version(1, 52) {
enable_addr_of = true;
} else if ac.probe_rustc_version(1, 51) {
// This compiler claims to be 1.51, but there are some nightly
// compilers that claim to be 1.51 without supporting the
// feature. Explicitly probe to check if code using them
// compiles.
//
// The oldest nightly that supports the feature is 2021-01-31.
if ac.probe_expression(ADDR_OF_PROBE) {
enable_addr_of = true;
}
}
}
Err(e) => {
@@ -54,4 +77,25 @@ fn main() {
// RUSTFLAGS="--cfg tokio_no_const_thread_local"
autocfg::emit("tokio_no_const_thread_local")
}
if !enable_addr_of {
// To disable this feature on compilers that support it, you can
// explicitly pass this flag with the following environment variable:
//
// RUSTFLAGS="--cfg tokio_no_addr_of"
autocfg::emit("tokio_no_addr_of")
}
let target = ::std::env::var("TARGET").unwrap_or_default();
// We emit cfgs instead of using `target_family = "wasm"` that requires Rust 1.54.
// Note that these cfgs are unavailable in `Cargo.toml`.
if target.starts_with("wasm") {
autocfg::emit("tokio_wasm");
if target.contains("wasi") {
autocfg::emit("tokio_wasi");
} else {
autocfg::emit("tokio_wasm_not_wasi");
}
}
}
+3 -3
View File
@@ -188,9 +188,9 @@ readiness, the driver's tick is packed into the atomic `usize`.
The `ScheduledIo` readiness `AtomicUsize` is structured as:
```
| reserved | generation | driver tick | readinesss |
|----------+------------+--------------+------------|
| 1 bit | 7 bits + 8 bits + 16 bits |
| reserved | generation | driver tick | readiness |
|----------+------------+--------------+-----------|
| 1 bit | 7 bits + 8 bits + 16 bits |
```
The `reserved` and `generation` components exist today.
+16
View File
@@ -0,0 +1,16 @@
# This config file is for the `cargo-check-external-types` tool that is run in CI.
# The following are types that are allowed to be exposed in Tokio's public API.
# The standard library is allowed by default.
allowed_external_types = [
"bytes::buf::buf_impl::Buf",
"bytes::buf::buf_mut::BufMut",
"tokio_macros::*",
# TODO(https://github.com/tokio-rs/tokio/issues/4916): Remove the libc types
"libc::unix::gid_t",
"libc::unix::pid_t",
"libc::unix::uid_t",
]
+2 -2
View File
@@ -207,7 +207,7 @@ cfg_coop! {
mod test {
use super::*;
#[cfg(target_arch = "wasm32")]
#[cfg(tokio_wasm_not_wasi)]
use wasm_bindgen_test::wasm_bindgen_test as test;
fn get() -> Budget {
@@ -215,7 +215,7 @@ mod test {
}
#[test]
fn bugeting() {
fn budgeting() {
use futures::future::poll_fn;
use tokio_test::*;
+20 -19
View File
@@ -565,29 +565,30 @@ impl AsyncSeek for File {
let me = self.get_mut();
let inner = me.inner.get_mut();
loop {
match inner.state {
Busy(_) => panic!("must wait for poll_complete before calling start_seek"),
Idle(ref mut buf_cell) => {
let mut buf = buf_cell.take().unwrap();
match inner.state {
Busy(_) => Err(io::Error::new(
io::ErrorKind::Other,
"other file operation is pending, call poll_complete before start_seek",
)),
Idle(ref mut buf_cell) => {
let mut buf = buf_cell.take().unwrap();
// Factor in any unread data from the buf
if !buf.is_empty() {
let n = buf.discard_read();
// Factor in any unread data from the buf
if !buf.is_empty() {
let n = buf.discard_read();
if let SeekFrom::Current(ref mut offset) = pos {
*offset += n;
}
if let SeekFrom::Current(ref mut offset) = pos {
*offset += n;
}
let std = me.std.clone();
inner.state = Busy(spawn_blocking(move || {
let res = (&*std).seek(pos);
(Operation::Seek(res), buf)
}));
return Ok(());
}
let std = me.std.clone();
inner.state = Busy(spawn_blocking(move || {
let res = (&*std).seek(pos);
(Operation::Seek(res), buf)
}));
Ok(())
}
}
}
+21
View File
@@ -955,3 +955,24 @@ fn partial_read_set_len_ok() {
assert_eq!(n, FOO.len());
assert_eq!(&buf[..n], FOO);
}
#[test]
fn busy_file_seek_error() {
let mut file = MockFile::default();
let mut seq = Sequence::new();
file.expect_inner_write()
.once()
.in_sequence(&mut seq)
.returning(|_| Err(io::ErrorKind::Other.into()));
let mut file = crate::io::BufReader::new(File::from_std(file));
{
let mut t = task::spawn(file.write(HELLO));
assert_ready_ok!(t.poll());
}
pool::run_one();
let mut t = task::spawn(file.seek(SeekFrom::Start(0)));
assert_ready_err!(t.poll());
}
+2 -1
View File
@@ -1,4 +1,5 @@
use crate::io::driver::{Handle, Interest, ReadyEvent, Registration};
use crate::io::Interest;
use crate::runtime::io::{Handle, ReadyEvent, Registration};
use mio::unix::SourceFd;
use std::io;
+2 -1
View File
@@ -34,8 +34,9 @@ enum State<T> {
Busy(sys::Blocking<(io::Result<usize>, Buf, T)>),
}
cfg_io_std! {
cfg_io_blocking! {
impl<T> Blocking<T> {
#[cfg_attr(feature = "fs", allow(dead_code))]
pub(crate) fn new(inner: T) -> Blocking<T> {
Blocking {
inner: Some(inner),
+2 -1
View File
@@ -1,6 +1,7 @@
//! Use POSIX AIO futures with Tokio.
use crate::io::driver::{Handle, Interest, ReadyEvent, Registration};
use crate::io::interest::Interest;
use crate::runtime::io::{Handle, ReadyEvent, Registration};
use mio::event::Source;
use mio::Registry;
use mio::Token;
@@ -1,6 +1,6 @@
#![cfg_attr(not(feature = "net"), allow(dead_code, unreachable_pub))]
use crate::io::driver::Ready;
use crate::io::ready::Ready;
use std::fmt;
use std::ops;
@@ -100,7 +100,7 @@ impl Interest {
self.0
}
pub(super) fn mask(self) -> Ready {
pub(crate) fn mask(self) -> Ready {
match self {
Interest::READABLE => Ready::READABLE | Ready::READ_CLOSED,
Interest::WRITABLE => Ready::WRITABLE | Ready::WRITE_CLOSED,
+6 -4
View File
@@ -1,5 +1,3 @@
#![cfg_attr(loom, allow(dead_code, unreachable_pub))]
//! Traits, helpers, and type definitions for asynchronous I/O functionality.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
@@ -205,15 +203,19 @@ pub use self::read_buf::ReadBuf;
pub use std::io::{Error, ErrorKind, Result, SeekFrom};
cfg_io_driver_impl! {
pub(crate) mod driver;
pub(crate) mod interest;
pub(crate) mod ready;
cfg_net! {
pub use driver::{Interest, Ready};
pub use interest::Interest;
pub use ready::Ready;
}
#[cfg_attr(tokio_wasi, allow(unused_imports))]
mod poll_evented;
#[cfg(not(loom))]
#[cfg_attr(tokio_wasi, allow(unused_imports))]
pub(crate) use poll_evented::PollEvented;
}
+54 -15
View File
@@ -1,4 +1,5 @@
use crate::io::driver::{Handle, Interest, Registration};
use crate::io::interest::Interest;
use crate::runtime::io::{Handle, Registration};
use mio::event::Source;
use std::fmt;
@@ -11,7 +12,7 @@ cfg_io_driver! {
/// [`std::io::Write`] traits with the reactor that drives it.
///
/// `PollEvented` uses [`Registration`] internally to take a type that
/// implements [`mio::event::Source`] as well as [`std::io::Read`] and or
/// implements [`mio::event::Source`] as well as [`std::io::Read`] and/or
/// [`std::io::Write`] and associate it with a reactor that will drive it.
///
/// Once the [`mio::event::Source`] type is wrapped by `PollEvented`, it can be
@@ -41,12 +42,12 @@ cfg_io_driver! {
/// [`poll_read_ready`] again will also indicate read readiness.
///
/// When the operation is attempted and is unable to succeed due to the I/O
/// resource not being ready, the caller must call `clear_readiness`.
/// resource not being ready, the caller must call [`clear_readiness`].
/// This clears the readiness state until a new readiness event is received.
///
/// This allows the caller to implement additional functions. For example,
/// [`TcpListener`] implements poll_accept by using [`poll_read_ready`] and
/// `clear_read_ready`.
/// [`clear_readiness`].
///
/// ## Platform-specific events
///
@@ -57,6 +58,7 @@ cfg_io_driver! {
/// [`AsyncRead`]: crate::io::AsyncRead
/// [`AsyncWrite`]: crate::io::AsyncWrite
/// [`TcpListener`]: crate::net::TcpListener
/// [`clear_readiness`]: Registration::clear_readiness
/// [`poll_read_ready`]: Registration::poll_read_ready
/// [`poll_write_ready`]: Registration::poll_write_ready
pub(crate) struct PollEvented<E: Source> {
@@ -77,6 +79,7 @@ impl<E: Source> PollEvented<E> {
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
#[cfg_attr(feature = "signal", allow(unused))]
pub(crate) fn new(io: E) -> io::Result<Self> {
PollEvented::new_with_interest(io, Interest::READABLE | Interest::WRITABLE)
@@ -97,6 +100,7 @@ impl<E: Source> PollEvented<E> {
/// a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter)
/// function.
#[track_caller]
#[cfg_attr(feature = "signal", allow(unused))]
pub(crate) fn new_with_interest(io: E, interest: Interest) -> io::Result<Self> {
Self::new_with_interest_and_handle(io, interest, Handle::current())
@@ -134,7 +138,7 @@ impl<E: Source> PollEvented<E> {
}
feature! {
#![any(feature = "net", feature = "process")]
#![any(feature = "net", all(unix, feature = "process"))]
use crate::io::ReadBuf;
use std::task::{Context, Poll};
@@ -151,16 +155,32 @@ feature! {
{
use std::io::Read;
let n = ready!(self.registration.poll_read_io(cx, || {
let b = &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit<u8>] as *mut [u8]);
self.io.as_ref().unwrap().read(b)
}))?;
loop {
let evt = ready!(self.registration.poll_read_ready(cx))?;
// Safety: We trust `TcpStream::read` to have filled up `n` bytes in the
// buffer.
buf.assume_init(n);
buf.advance(n);
Poll::Ready(Ok(()))
let b = &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit<u8>] as *mut [u8]);
let len = b.len();
match self.io.as_ref().unwrap().read(b) {
Ok(n) => {
// if we read a partially full buffer, this is sufficient on unix to show
// that the socket buffer has been drained
if n > 0 && (!cfg!(windows) && n < len) {
self.registration.clear_readiness(evt);
}
// Safety: We trust `TcpStream::read` to have filled up `n` bytes in the
// buffer.
buf.assume_init(n);
buf.advance(n);
return Poll::Ready(Ok(()));
},
Err(e) if e.kind() == io::ErrorKind::WouldBlock => {
self.registration.clear_readiness(evt);
}
Err(e) => return Poll::Ready(Err(e)),
}
}
}
pub(crate) fn poll_write<'a>(&'a self, cx: &mut Context<'_>, buf: &[u8]) -> Poll<io::Result<usize>>
@@ -168,7 +188,26 @@ feature! {
&'a E: io::Write + 'a,
{
use std::io::Write;
self.registration.poll_write_io(cx, || self.io.as_ref().unwrap().write(buf))
loop {
let evt = ready!(self.registration.poll_write_ready(cx))?;
match self.io.as_ref().unwrap().write(buf) {
Ok(n) => {
// if we write only part of our buffer, this is sufficient on unix to show
// that the socket buffer is full
if n > 0 && (!cfg!(windows) && n < buf.len()) {
self.registration.clear_readiness(evt);
}
return Poll::Ready(Ok(n));
},
Err(e) if e.kind() == io::ErrorKind::WouldBlock => {
self.registration.clear_readiness(evt);
}
Err(e) => return Poll::Ready(Err(e)),
}
}
}
#[cfg(feature = "net")]
+1 -1
View File
@@ -69,7 +69,7 @@ cfg_io_util! {
/// Creates a future which will rewind to the beginning of the stream.
///
/// This is convenience method, equivalent to to `self.seek(SeekFrom::Start(0))`.
/// This is convenience method, equivalent to `self.seek(SeekFrom::Start(0))`.
fn rewind(&mut self) -> Seek<'_, Self>
where
Self: Unpin,
+88 -5
View File
@@ -16,6 +16,7 @@
))]
#![cfg_attr(docsrs, feature(doc_cfg))]
#![cfg_attr(docsrs, allow(unused_attributes))]
#![cfg_attr(loom, allow(dead_code, unreachable_pub))]
//! A runtime for writing reliable network applications without compromising speed.
//!
@@ -152,7 +153,7 @@
//! provide the functionality you need.
//!
//! Using the runtime requires the "rt" or "rt-multi-thread" feature flags, to
//! enable the basic [single-threaded scheduler][rt] and the [thread-pool
//! enable the current-thread [single-threaded scheduler][rt] and the [multi-thread
//! scheduler][rt-multi-thread], respectively. See the [`runtime` module
//! documentation][rt-features] for details. In addition, the "macros" feature
//! flag enables the `#[tokio::main]` and `#[tokio::test]` attributes.
@@ -309,7 +310,7 @@
//! need.
//!
//! - `full`: Enables all features listed below except `test-util` and `tracing`.
//! - `rt`: Enables `tokio::spawn`, the basic (current thread) scheduler,
//! - `rt`: Enables `tokio::spawn`, the current-thread scheduler,
//! and non-scheduler utilities.
//! - `rt-multi-thread`: Enables the heavier, multi-threaded, work-stealing scheduler.
//! - `io-util`: Enables the IO based `Ext` traits.
@@ -347,9 +348,12 @@
//!
//! Likewise, some parts of the API are only available with the same flag:
//!
//! - [`task::JoinSet`]
//! - [`task::Builder`]
//!
//! - Some methods on [`task::JoinSet`]
//! - [`runtime::RuntimeMetrics`]
//! - [`runtime::Builder::unhandled_panic`]
//! - [`task::Id`]
//!
//! This flag enables **unstable** features. The public API of these features
//! may break in 1.x releases. To enable these features, the `--cfg
//! tokio_unstable` argument must be passed to `rustc` when compiling. This
@@ -379,6 +383,39 @@
//!
//! [unstable features]: https://internals.rust-lang.org/t/feature-request-unstable-opt-in-non-transitive-crate-features/16193#why-not-a-crate-feature-2
//! [feature flags]: https://doc.rust-lang.org/cargo/reference/manifest.html#the-features-section
//!
//! ## WASM support
//!
//! Tokio has some limited support for the WASM platform. Without the
//! `tokio_unstable` flag, the following features are supported:
//!
//! * `sync`
//! * `macros`
//! * `io-util`
//! * `rt`
//! * `time`
//!
//! Enabling any other feature (including `full`) will cause a compilation
//! failure.
//!
//! The `time` module will only work on WASM platforms that have support for
//! timers (e.g. wasm32-wasi). The timing functions will panic if used on a WASM
//! platform that does not support timers.
//!
//! Note also that if the runtime becomes indefinitely idle, it will panic
//! immediately instead of blocking forever. On platforms that don't support
//! time, this means that the runtime can never be idle in any way.
//!
//! ### Unstable WASM support
//!
//! Tokio also has unstable support for some additional WASM features. This
//! requires the use of the `tokio_unstable` flag.
//!
//! Using this flag enables the use of `tokio::net` on the wasm32-wasi target.
//! However, not all methods are available on the networking types as WASI
//! currently does not support the creation of new sockets from within WASM.
//! Because of this, sockets must currently be created via the `FromRawFd`
//! trait.
// Test that pointer width is compatible. This asserts that e.g. usize is at
// least 32 bits, which a lot of components in Tokio currently assumes.
@@ -393,6 +430,37 @@ compile_error! {
"Tokio requires the platform pointer width to be 32, 64, or 128 bits"
}
// Ensure that our build script has correctly set cfg flags for wasm.
//
// Each condition is written all(a, not(b)). This should be read as
// "if a, then we must also have b".
#[cfg(any(
all(target_arch = "wasm32", not(tokio_wasm)),
all(target_arch = "wasm64", not(tokio_wasm)),
all(target_family = "wasm", not(tokio_wasm)),
all(target_os = "wasi", not(tokio_wasm)),
all(target_os = "wasi", not(tokio_wasi)),
all(target_os = "wasi", tokio_wasm_not_wasi),
all(tokio_wasm, not(any(target_arch = "wasm32", target_arch = "wasm64"))),
all(tokio_wasm_not_wasi, not(tokio_wasm)),
all(tokio_wasi, not(tokio_wasm))
))]
compile_error!("Tokio's build script has incorrectly detected wasm.");
#[cfg(all(
not(tokio_unstable),
tokio_wasm,
any(
feature = "fs",
feature = "io-std",
feature = "net",
feature = "process",
feature = "rt-multi-thread",
feature = "signal"
)
))]
compile_error!("Only features sync,macros,io-util,rt,time are supported on wasm.");
// Includes re-exports used by macros.
//
// This module is not intended to be part of the public API. In general, any
@@ -417,12 +485,27 @@ cfg_process! {
pub mod process;
}
#[cfg(any(feature = "net", feature = "fs", feature = "io-std"))]
#[cfg(any(
feature = "fs",
feature = "io-std",
feature = "net",
all(windows, feature = "process"),
))]
mod blocking;
cfg_rt! {
pub mod runtime;
}
cfg_not_rt! {
// The `runtime` module is used when the IO or time driver is needed.
#[cfg(any(
feature = "net",
feature = "time",
all(unix, feature = "process"),
all(unix, feature = "signal"),
))]
pub(crate) mod runtime;
}
pub(crate) mod coop;
+5
View File
@@ -38,3 +38,8 @@ pub(crate) mod sys {
2
}
}
pub(crate) mod thread {
pub use loom::lazy_static::AccessError;
pub use loom::thread::*;
}
+2 -2
View File
@@ -102,7 +102,7 @@ pub(crate) mod thread {
#[allow(unused_imports)]
pub(crate) use std::thread::{
current, panicking, park, park_timeout, sleep, spawn, Builder, JoinHandle, LocalKey,
Result, Thread, ThreadId,
current, panicking, park, park_timeout, sleep, spawn, AccessError, Builder, JoinHandle,
LocalKey, Result, Thread, ThreadId,
};
}
+53
View File
@@ -0,0 +1,53 @@
//! This module defines a macro that lets you go from a raw pointer to a struct
//! to a raw pointer to a field of the struct.
#[cfg(not(tokio_no_addr_of))]
macro_rules! generate_addr_of_methods {
(
impl<$($gen:ident)*> $struct_name:ty {$(
$(#[$attrs:meta])*
$vis:vis unsafe fn $fn_name:ident(self: NonNull<Self>) -> NonNull<$field_type:ty> {
&self$(.$field_name:tt)+
}
)*}
) => {
impl<$($gen)*> $struct_name {$(
$(#[$attrs])*
$vis unsafe fn $fn_name(me: ::core::ptr::NonNull<Self>) -> ::core::ptr::NonNull<$field_type> {
let me = me.as_ptr();
let field = ::std::ptr::addr_of_mut!((*me) $(.$field_name)+ );
::core::ptr::NonNull::new_unchecked(field)
}
)*}
};
}
// The `addr_of_mut!` macro is only available for MSRV at least 1.51.0. This
// version of the macro uses a workaround for older versions of rustc.
#[cfg(tokio_no_addr_of)]
macro_rules! generate_addr_of_methods {
(
impl<$($gen:ident)*> $struct_name:ty {$(
$(#[$attrs:meta])*
$vis:vis unsafe fn $fn_name:ident(self: NonNull<Self>) -> NonNull<$field_type:ty> {
&self$(.$field_name:tt)+
}
)*}
) => {
impl<$($gen)*> $struct_name {$(
$(#[$attrs])*
$vis unsafe fn $fn_name(me: ::core::ptr::NonNull<Self>) -> ::core::ptr::NonNull<$field_type> {
let me = me.as_ptr();
let me_u8 = me as *mut u8;
let field_offset = {
let me_ref = &*me;
let field_ref_u8 = (&me_ref $(.$field_name)+ ) as *const $field_type as *const u8;
field_ref_u8.offset_from(me_u8)
};
::core::ptr::NonNull::new_unchecked(me_u8.offset(field_offset).cast())
}
)*}
};
}
+35 -8
View File
@@ -61,6 +61,7 @@ macro_rules! cfg_fs {
($($item:item)*) => {
$(
#[cfg(feature = "fs")]
#[cfg(not(tokio_wasi))]
#[cfg_attr(docsrs, doc(cfg(feature = "fs")))]
$item
)*
@@ -69,7 +70,11 @@ macro_rules! cfg_fs {
macro_rules! cfg_io_blocking {
($($item:item)*) => {
$( #[cfg(any(feature = "io-std", feature = "fs"))] $item )*
$( #[cfg(any(
feature = "io-std",
feature = "fs",
all(windows, feature = "process"),
))] $item )*
}
}
@@ -78,12 +83,12 @@ macro_rules! cfg_io_driver {
$(
#[cfg(any(
feature = "net",
feature = "process",
all(unix, feature = "process"),
all(unix, feature = "signal"),
))]
#[cfg_attr(docsrs, doc(cfg(any(
feature = "net",
feature = "process",
all(unix, feature = "process"),
all(unix, feature = "signal"),
))))]
$item
@@ -96,7 +101,7 @@ macro_rules! cfg_io_driver_impl {
$(
#[cfg(any(
feature = "net",
feature = "process",
all(unix, feature = "process"),
all(unix, feature = "signal"),
))]
$item
@@ -109,7 +114,7 @@ macro_rules! cfg_not_io_driver {
$(
#[cfg(not(any(
feature = "net",
feature = "process",
all(unix, feature = "process"),
all(unix, feature = "signal"),
)))]
$item
@@ -247,6 +252,7 @@ macro_rules! cfg_process {
#[cfg(feature = "process")]
#[cfg_attr(docsrs, doc(cfg(feature = "process")))]
#[cfg(not(loom))]
#[cfg(not(tokio_wasi))]
$item
)*
}
@@ -275,6 +281,7 @@ macro_rules! cfg_signal {
#[cfg(feature = "signal")]
#[cfg_attr(docsrs, doc(cfg(feature = "signal")))]
#[cfg(not(loom))]
#[cfg(not(tokio_wasi))]
$item
)*
}
@@ -334,7 +341,7 @@ macro_rules! cfg_not_rt {
macro_rules! cfg_rt_multi_thread {
($($item:item)*) => {
$(
#[cfg(feature = "rt-multi-thread")]
#[cfg(all(feature = "rt-multi-thread", not(tokio_wasi)))]
#[cfg_attr(docsrs, doc(cfg(feature = "rt-multi-thread")))]
$item
)*
@@ -451,7 +458,8 @@ macro_rules! cfg_has_atomic_u64 {
target_arch = "arm",
target_arch = "mips",
target_arch = "powerpc",
target_arch = "riscv32"
target_arch = "riscv32",
tokio_wasm
)))]
$item
)*
@@ -465,9 +473,28 @@ macro_rules! cfg_not_has_atomic_u64 {
target_arch = "arm",
target_arch = "mips",
target_arch = "powerpc",
target_arch = "riscv32"
target_arch = "riscv32",
tokio_wasm
))]
$item
)*
}
}
macro_rules! cfg_not_wasi {
($($item:item)*) => {
$(
#[cfg(not(tokio_wasi))]
$item
)*
}
}
macro_rules! cfg_is_wasm_not_wasi {
($($item:item)*) => {
$(
#[cfg(tokio_wasm_not_wasi)]
$item
)*
}
}
+3
View File
@@ -15,6 +15,9 @@ mod ready;
#[macro_use]
mod thread_local;
#[macro_use]
mod addr_of;
cfg_trace! {
#[macro_use]
mod trace;
+9 -5
View File
@@ -23,8 +23,10 @@
//! [`UnixDatagram`]: UnixDatagram
mod addr;
#[cfg(feature = "net")]
pub(crate) use addr::to_socket_addrs;
cfg_not_wasi! {
#[cfg(feature = "net")]
pub(crate) use addr::to_socket_addrs;
}
pub use addr::ToSocketAddrs;
cfg_net! {
@@ -33,11 +35,13 @@ cfg_net! {
pub mod tcp;
pub use tcp::listener::TcpListener;
pub use tcp::socket::TcpSocket;
pub use tcp::stream::TcpStream;
cfg_not_wasi! {
pub use tcp::socket::TcpSocket;
mod udp;
pub use udp::UdpSocket;
mod udp;
pub use udp::UdpSocket;
}
}
cfg_net_unix! {
+94 -62
View File
@@ -1,6 +1,9 @@
use crate::io::{Interest, PollEvented};
use crate::net::tcp::TcpStream;
use crate::net::{to_socket_addrs, ToSocketAddrs};
cfg_not_wasi! {
use crate::net::{to_socket_addrs, ToSocketAddrs};
}
use std::convert::TryFrom;
use std::fmt;
@@ -55,68 +58,70 @@ cfg_net! {
}
impl TcpListener {
/// Creates a new TcpListener, which will be bound to the specified address.
///
/// The returned listener is ready for accepting connections.
///
/// Binding with a port number of 0 will request that the OS assigns a port
/// to this listener. The port allocated can be queried via the `local_addr`
/// method.
///
/// The address type can be any implementor of the [`ToSocketAddrs`] trait.
/// If `addr` yields multiple addresses, bind will be attempted with each of
/// the addresses until one succeeds and returns the listener. If none of
/// the addresses succeed in creating a listener, the error returned from
/// the last attempt (the last address) is returned.
///
/// This function sets the `SO_REUSEADDR` option on the socket.
///
/// To configure the socket before binding, you can use the [`TcpSocket`]
/// type.
///
/// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs
/// [`TcpSocket`]: struct@crate::net::TcpSocket
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpListener;
///
/// use std::io;
///
/// #[tokio::main]
/// async fn main() -> io::Result<()> {
/// let listener = TcpListener::bind("127.0.0.1:2345").await?;
///
/// // use the listener
///
/// # let _ = listener;
/// Ok(())
/// }
/// ```
pub async fn bind<A: ToSocketAddrs>(addr: A) -> io::Result<TcpListener> {
let addrs = to_socket_addrs(addr).await?;
cfg_not_wasi! {
/// Creates a new TcpListener, which will be bound to the specified address.
///
/// The returned listener is ready for accepting connections.
///
/// Binding with a port number of 0 will request that the OS assigns a port
/// to this listener. The port allocated can be queried via the `local_addr`
/// method.
///
/// The address type can be any implementor of the [`ToSocketAddrs`] trait.
/// If `addr` yields multiple addresses, bind will be attempted with each of
/// the addresses until one succeeds and returns the listener. If none of
/// the addresses succeed in creating a listener, the error returned from
/// the last attempt (the last address) is returned.
///
/// This function sets the `SO_REUSEADDR` option on the socket.
///
/// To configure the socket before binding, you can use the [`TcpSocket`]
/// type.
///
/// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs
/// [`TcpSocket`]: struct@crate::net::TcpSocket
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpListener;
///
/// use std::io;
///
/// #[tokio::main]
/// async fn main() -> io::Result<()> {
/// let listener = TcpListener::bind("127.0.0.1:2345").await?;
///
/// // use the listener
///
/// # let _ = listener;
/// Ok(())
/// }
/// ```
pub async fn bind<A: ToSocketAddrs>(addr: A) -> io::Result<TcpListener> {
let addrs = to_socket_addrs(addr).await?;
let mut last_err = None;
let mut last_err = None;
for addr in addrs {
match TcpListener::bind_addr(addr) {
Ok(listener) => return Ok(listener),
Err(e) => last_err = Some(e),
for addr in addrs {
match TcpListener::bind_addr(addr) {
Ok(listener) => return Ok(listener),
Err(e) => last_err = Some(e),
}
}
Err(last_err.unwrap_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"could not resolve to any address",
)
}))
}
Err(last_err.unwrap_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"could not resolve to any address",
)
}))
}
fn bind_addr(addr: SocketAddr) -> io::Result<TcpListener> {
let listener = mio::net::TcpListener::bind(addr)?;
TcpListener::new(listener)
fn bind_addr(addr: SocketAddr) -> io::Result<TcpListener> {
let listener = mio::net::TcpListener::bind(addr)?;
TcpListener::new(listener)
}
}
/// Accepts a new incoming connection from this listener.
@@ -216,11 +221,13 @@ impl TcpListener {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
pub fn from_std(listener: net::TcpListener) -> io::Result<TcpListener> {
let io = mio::net::TcpListener::from_std(listener);
let io = PollEvented::new(io)?;
@@ -267,11 +274,22 @@ impl TcpListener {
.map(|io| io.into_raw_socket())
.map(|raw_socket| unsafe { std::net::TcpListener::from_raw_socket(raw_socket) })
}
#[cfg(tokio_wasi)]
{
use std::os::wasi::io::{FromRawFd, IntoRawFd};
self.io
.into_inner()
.map(|io| io.into_raw_fd())
.map(|raw_fd| unsafe { std::net::TcpListener::from_raw_fd(raw_fd) })
}
}
pub(crate) fn new(listener: mio::net::TcpListener) -> io::Result<TcpListener> {
let io = PollEvented::new(listener)?;
Ok(TcpListener { io })
cfg_not_wasi! {
pub(crate) fn new(listener: mio::net::TcpListener) -> io::Result<TcpListener> {
let io = PollEvented::new(listener)?;
Ok(TcpListener { io })
}
}
/// Returns the local address that this listener is bound to.
@@ -384,6 +402,20 @@ mod sys {
}
}
cfg_unstable! {
#[cfg(tokio_wasi)]
mod sys {
use super::TcpListener;
use std::os::wasi::prelude::*;
impl AsRawFd for TcpListener {
fn as_raw_fd(&self) -> RawFd {
self.io.as_raw_fd()
}
}
}
}
#[cfg(windows)]
mod sys {
use super::TcpListener;
+3 -1
View File
@@ -2,7 +2,9 @@
pub(crate) mod listener;
pub(crate) mod socket;
cfg_not_wasi! {
pub(crate) mod socket;
}
mod split;
pub use split::{ReadHalf, WriteHalf};
+83
View File
@@ -398,6 +398,89 @@ impl TcpSocket {
self.inner.linger()
}
/// Gets the value of the `IP_TOS` option for this socket.
///
/// For more information about this option, see [`set_tos`].
///
/// **NOTE:** On Windows, `IP_TOS` is only supported on [Windows 8+ or
/// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)
///
/// [`set_tos`]: Self::set_tos
// https://docs.rs/socket2/0.4.2/src/socket2/socket.rs.html#1178
#[cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
)))]
#[cfg_attr(
docsrs,
doc(cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
))))
)]
pub fn tos(&self) -> io::Result<u32> {
self.inner.tos()
}
/// Sets the value for the `IP_TOS` option on this socket.
///
/// This value sets the time-to-live field that is used in every packet sent
/// from this socket.
///
/// **NOTE:** On Windows, `IP_TOS` is only supported on [Windows 8+ or
/// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)
// https://docs.rs/socket2/0.4.2/src/socket2/socket.rs.html#1178
#[cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
)))]
#[cfg_attr(
docsrs,
doc(cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
))))
)]
pub fn set_tos(&self, tos: u32) -> io::Result<()> {
self.inner.set_tos(tos)
}
/// Gets the value for the `SO_BINDTODEVICE` option on this socket
///
/// This value gets the socket binded device's interface name.
#[cfg(any(target_os = "android", target_os = "fuchsia", target_os = "linux",))]
#[cfg_attr(
docsrs,
doc(cfg(any(target_os = "android", target_os = "fuchsia", target_os = "linux",)))
)]
pub fn device(&self) -> io::Result<Option<Vec<u8>>> {
self.inner.device()
}
/// Sets the value for the `SO_BINDTODEVICE` option on this socket
///
/// If a socket is bound to an interface, only packets received from that
/// particular interface are processed by the socket. Note that this only
/// works for some socket types, particularly `AF_INET` sockets.
///
/// If `interface` is `None` or an empty string it removes the binding.
#[cfg(all(any(target_os = "android", target_os = "fuchsia", target_os = "linux")))]
#[cfg_attr(
docsrs,
doc(cfg(all(any(target_os = "android", target_os = "fuchsia", target_os = "linux"))))
)]
pub fn bind_device(&self, interface: Option<&[u8]>) -> io::Result<()> {
self.inner.bind_device(interface)
}
/// Gets the local address of this socket.
///
/// Will fail on windows if called before `bind`.
+6 -2
View File
@@ -190,8 +190,12 @@ impl ReadHalf<'_> {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
pub fn try_read(&self, buf: &mut [u8]) -> io::Result<usize> {
self.0.try_read(buf)
+6 -2
View File
@@ -245,8 +245,12 @@ impl OwnedReadHalf {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
pub fn try_read(&self, buf: &mut [u8]) -> io::Result<usize> {
self.inner.try_read(buf)
+158 -124
View File
@@ -1,8 +1,12 @@
use crate::future::poll_fn;
cfg_not_wasi! {
use crate::future::poll_fn;
use crate::net::{to_socket_addrs, ToSocketAddrs};
use std::time::Duration;
}
use crate::io::{AsyncRead, AsyncWrite, Interest, PollEvented, ReadBuf, Ready};
use crate::net::tcp::split::{split, ReadHalf, WriteHalf};
use crate::net::tcp::split_owned::{split_owned, OwnedReadHalf, OwnedWriteHalf};
use crate::net::{to_socket_addrs, ToSocketAddrs};
use std::convert::TryFrom;
use std::fmt;
@@ -10,7 +14,6 @@ use std::io;
use std::net::{Shutdown, SocketAddr};
use std::pin::Pin;
use std::task::{Context, Poll};
use std::time::Duration;
cfg_io_util! {
use bytes::BufMut;
@@ -70,86 +73,88 @@ cfg_net! {
}
impl TcpStream {
/// Opens a TCP connection to a remote host.
///
/// `addr` is an address of the remote host. Anything which implements the
/// [`ToSocketAddrs`] trait can be supplied as the address. If `addr`
/// yields multiple addresses, connect will be attempted with each of the
/// addresses until a connection is successful. If none of the addresses
/// result in a successful connection, the error returned from the last
/// connection attempt (the last address) is returned.
///
/// To configure the socket before connecting, you can use the [`TcpSocket`]
/// type.
///
/// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs
/// [`TcpSocket`]: struct@crate::net::TcpSocket
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
/// use tokio::io::AsyncWriteExt;
/// use std::error::Error;
///
/// #[tokio::main]
/// async fn main() -> Result<(), Box<dyn Error>> {
/// // Connect to a peer
/// let mut stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// // Write some data.
/// stream.write_all(b"hello world!").await?;
///
/// Ok(())
/// }
/// ```
///
/// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.
///
/// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all
/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt
pub async fn connect<A: ToSocketAddrs>(addr: A) -> io::Result<TcpStream> {
let addrs = to_socket_addrs(addr).await?;
cfg_not_wasi! {
/// Opens a TCP connection to a remote host.
///
/// `addr` is an address of the remote host. Anything which implements the
/// [`ToSocketAddrs`] trait can be supplied as the address. If `addr`
/// yields multiple addresses, connect will be attempted with each of the
/// addresses until a connection is successful. If none of the addresses
/// result in a successful connection, the error returned from the last
/// connection attempt (the last address) is returned.
///
/// To configure the socket before connecting, you can use the [`TcpSocket`]
/// type.
///
/// [`ToSocketAddrs`]: trait@crate::net::ToSocketAddrs
/// [`TcpSocket`]: struct@crate::net::TcpSocket
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
/// use tokio::io::AsyncWriteExt;
/// use std::error::Error;
///
/// #[tokio::main]
/// async fn main() -> Result<(), Box<dyn Error>> {
/// // Connect to a peer
/// let mut stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// // Write some data.
/// stream.write_all(b"hello world!").await?;
///
/// Ok(())
/// }
/// ```
///
/// The [`write_all`] method is defined on the [`AsyncWriteExt`] trait.
///
/// [`write_all`]: fn@crate::io::AsyncWriteExt::write_all
/// [`AsyncWriteExt`]: trait@crate::io::AsyncWriteExt
pub async fn connect<A: ToSocketAddrs>(addr: A) -> io::Result<TcpStream> {
let addrs = to_socket_addrs(addr).await?;
let mut last_err = None;
let mut last_err = None;
for addr in addrs {
match TcpStream::connect_addr(addr).await {
Ok(stream) => return Ok(stream),
Err(e) => last_err = Some(e),
for addr in addrs {
match TcpStream::connect_addr(addr).await {
Ok(stream) => return Ok(stream),
Err(e) => last_err = Some(e),
}
}
Err(last_err.unwrap_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"could not resolve to any address",
)
}))
}
Err(last_err.unwrap_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"could not resolve to any address",
)
}))
}
/// Establishes a connection to the specified `addr`.
async fn connect_addr(addr: SocketAddr) -> io::Result<TcpStream> {
let sys = mio::net::TcpStream::connect(addr)?;
TcpStream::connect_mio(sys).await
}
pub(crate) async fn connect_mio(sys: mio::net::TcpStream) -> io::Result<TcpStream> {
let stream = TcpStream::new(sys)?;
// Once we've connected, wait for the stream to be writable as
// that's when the actual connection has been initiated. Once we're
// writable we check for `take_socket_error` to see if the connect
// actually hit an error or not.
//
// If all that succeeded then we ship everything on up.
poll_fn(|cx| stream.io.registration().poll_write_ready(cx)).await?;
if let Some(e) = stream.io.take_error()? {
return Err(e);
/// Establishes a connection to the specified `addr`.
async fn connect_addr(addr: SocketAddr) -> io::Result<TcpStream> {
let sys = mio::net::TcpStream::connect(addr)?;
TcpStream::connect_mio(sys).await
}
Ok(stream)
pub(crate) async fn connect_mio(sys: mio::net::TcpStream) -> io::Result<TcpStream> {
let stream = TcpStream::new(sys)?;
// Once we've connected, wait for the stream to be writable as
// that's when the actual connection has been initiated. Once we're
// writable we check for `take_socket_error` to see if the connect
// actually hit an error or not.
//
// If all that succeeded then we ship everything on up.
poll_fn(|cx| stream.io.registration().poll_write_ready(cx)).await?;
if let Some(e) = stream.io.take_error()? {
return Err(e);
}
Ok(stream)
}
}
pub(crate) fn new(connected: mio::net::TcpStream) -> io::Result<TcpStream> {
@@ -181,11 +186,13 @@ impl TcpStream {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
pub fn from_std(stream: std::net::TcpStream) -> io::Result<TcpStream> {
let io = mio::net::TcpStream::from_std(stream);
let io = PollEvented::new(io)?;
@@ -244,6 +251,15 @@ impl TcpStream {
.map(|io| io.into_raw_socket())
.map(|raw_socket| unsafe { std::net::TcpStream::from_raw_socket(raw_socket) })
}
#[cfg(tokio_wasi)]
{
use std::os::wasi::io::{FromRawFd, IntoRawFd};
self.io
.into_inner()
.map(|io| io.into_raw_fd())
.map(|raw_fd| unsafe { std::net::TcpStream::from_raw_fd(raw_fd) })
}
}
/// Returns the local address that this stream is bound to.
@@ -531,8 +547,12 @@ impl TcpStream {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
///
/// # Examples
@@ -1077,52 +1097,54 @@ impl TcpStream {
self.io.set_nodelay(nodelay)
}
/// Reads the linger duration for this socket by getting the `SO_LINGER`
/// option.
///
/// For more information about this option, see [`set_linger`].
///
/// [`set_linger`]: TcpStream::set_linger
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
///
/// # async fn dox() -> Result<(), Box<dyn std::error::Error>> {
/// let stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// println!("{:?}", stream.linger()?);
/// # Ok(())
/// # }
/// ```
pub fn linger(&self) -> io::Result<Option<Duration>> {
socket2::SockRef::from(self).linger()
}
cfg_not_wasi! {
/// Reads the linger duration for this socket by getting the `SO_LINGER`
/// option.
///
/// For more information about this option, see [`set_linger`].
///
/// [`set_linger`]: TcpStream::set_linger
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
///
/// # async fn dox() -> Result<(), Box<dyn std::error::Error>> {
/// let stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// println!("{:?}", stream.linger()?);
/// # Ok(())
/// # }
/// ```
pub fn linger(&self) -> io::Result<Option<Duration>> {
socket2::SockRef::from(self).linger()
}
/// Sets the linger duration of this socket by setting the SO_LINGER option.
///
/// This option controls the action taken when a stream has unsent messages and the stream is
/// closed. If SO_LINGER is set, the system shall block the process until it can transmit the
/// data or until the time expires.
///
/// If SO_LINGER is not specified, and the stream is closed, the system handles the call in a
/// way that allows the process to continue as quickly as possible.
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
///
/// # async fn dox() -> Result<(), Box<dyn std::error::Error>> {
/// let stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// stream.set_linger(None)?;
/// # Ok(())
/// # }
/// ```
pub fn set_linger(&self, dur: Option<Duration>) -> io::Result<()> {
socket2::SockRef::from(self).set_linger(dur)
/// Sets the linger duration of this socket by setting the SO_LINGER option.
///
/// This option controls the action taken when a stream has unsent messages and the stream is
/// closed. If SO_LINGER is set, the system shall block the process until it can transmit the
/// data or until the time expires.
///
/// If SO_LINGER is not specified, and the stream is closed, the system handles the call in a
/// way that allows the process to continue as quickly as possible.
///
/// # Examples
///
/// ```no_run
/// use tokio::net::TcpStream;
///
/// # async fn dox() -> Result<(), Box<dyn std::error::Error>> {
/// let stream = TcpStream::connect("127.0.0.1:8080").await?;
///
/// stream.set_linger(None)?;
/// # Ok(())
/// # }
/// ```
pub fn set_linger(&self, dur: Option<Duration>) -> io::Result<()> {
socket2::SockRef::from(self).set_linger(dur)
}
}
/// Gets the value of the `IP_TTL` option for this socket.
@@ -1315,3 +1337,15 @@ mod sys {
}
}
}
#[cfg(all(tokio_unstable, tokio_wasi))]
mod sys {
use super::TcpStream;
use std::os::wasi::prelude::*;
impl AsRawFd for TcpStream {
fn as_raw_fd(&self) -> RawFd {
self.io.as_raw_fd()
}
}
}
+89
View File
@@ -170,6 +170,7 @@ impl UdpSocket {
UdpSocket::new(sys)
}
#[track_caller]
fn new(socket: mio::net::UdpSocket) -> io::Result<UdpSocket> {
let io = PollEvented::new(socket)?;
Ok(UdpSocket { io })
@@ -210,6 +211,7 @@ impl UdpSocket {
/// # Ok(())
/// # }
/// ```
#[track_caller]
pub fn from_std(socket: net::UdpSocket) -> io::Result<UdpSocket> {
let io = mio::net::UdpSocket::from_std(socket);
UdpSocket::new(io)
@@ -257,6 +259,10 @@ impl UdpSocket {
}
}
fn as_socket(&self) -> socket2::SockRef<'_> {
socket2::SockRef::from(self)
}
/// Returns the local address that this socket is bound to.
///
/// # Example
@@ -1508,6 +1514,89 @@ impl UdpSocket {
self.io.set_ttl(ttl)
}
/// Gets the value of the `IP_TOS` option for this socket.
///
/// For more information about this option, see [`set_tos`].
///
/// **NOTE:** On Windows, `IP_TOS` is only supported on [Windows 8+ or
/// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)
///
/// [`set_tos`]: Self::set_tos
// https://docs.rs/socket2/0.4.2/src/socket2/socket.rs.html#1178
#[cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
)))]
#[cfg_attr(
docsrs,
doc(cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
))))
)]
pub fn tos(&self) -> io::Result<u32> {
self.as_socket().tos()
}
/// Sets the value for the `IP_TOS` option on this socket.
///
/// This value sets the time-to-live field that is used in every packet sent
/// from this socket.
///
/// **NOTE:** On Windows, `IP_TOS` is only supported on [Windows 8+ or
/// Windows Server 2012+.](https://docs.microsoft.com/en-us/windows/win32/winsock/ipproto-ip-socket-options)
// https://docs.rs/socket2/0.4.2/src/socket2/socket.rs.html#1178
#[cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
)))]
#[cfg_attr(
docsrs,
doc(cfg(not(any(
target_os = "fuchsia",
target_os = "redox",
target_os = "solaris",
target_os = "illumos",
))))
)]
pub fn set_tos(&self, tos: u32) -> io::Result<()> {
self.as_socket().set_tos(tos)
}
/// Gets the value for the `SO_BINDTODEVICE` option on this socket
///
/// This value gets the socket-bound device's interface name.
#[cfg(any(target_os = "android", target_os = "fuchsia", target_os = "linux",))]
#[cfg_attr(
docsrs,
doc(cfg(any(target_os = "android", target_os = "fuchsia", target_os = "linux",)))
)]
pub fn device(&self) -> io::Result<Option<Vec<u8>>> {
self.as_socket().device()
}
/// Sets the value for the `SO_BINDTODEVICE` option on this socket
///
/// If a socket is bound to an interface, only packets received from that
/// particular interface are processed by the socket. Note that this only
/// works for some socket types, particularly `AF_INET` sockets.
///
/// If `interface` is `None` or an empty string it removes the binding.
#[cfg(all(any(target_os = "android", target_os = "fuchsia", target_os = "linux")))]
#[cfg_attr(
docsrs,
doc(cfg(all(any(target_os = "android", target_os = "fuchsia", target_os = "linux"))))
)]
pub fn bind_device(&self, interface: Option<&[u8]>) -> io::Result<()> {
self.as_socket().bind_device(interface)
}
/// Executes an operation of the `IP_ADD_MEMBERSHIP` type.
///
/// This function specifies a new multicast group for this socket to join.
+3 -1
View File
@@ -430,7 +430,8 @@ impl UnixDatagram {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a Tokio runtime, otherwise runtime can be set
@@ -457,6 +458,7 @@ impl UnixDatagram {
/// # Ok(())
/// # }
/// ```
#[track_caller]
pub fn from_std(datagram: net::UnixDatagram) -> io::Result<UnixDatagram> {
let socket = mio::net::UnixDatagram::from_std(datagram);
let io = PollEvented::new(socket)?;
+6 -2
View File
@@ -54,11 +54,13 @@ impl UnixListener {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
pub fn bind<P>(path: P) -> io::Result<UnixListener>
where
P: AsRef<Path>,
@@ -77,11 +79,13 @@ impl UnixListener {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
pub fn from_std(listener: net::UnixListener) -> io::Result<UnixListener> {
let listener = mio::net::UnixListener::from_std(listener);
let io = PollEvented::new(listener)?;
+6 -2
View File
@@ -100,8 +100,12 @@ impl ReadHalf<'_> {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
pub fn try_read(&self, buf: &mut [u8]) -> io::Result<usize> {
self.0.try_read(buf)
+6 -2
View File
@@ -155,8 +155,12 @@ impl OwnedReadHalf {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
pub fn try_read(&self, buf: &mut [u8]) -> io::Result<usize> {
self.inner.try_read(buf)
+12 -5
View File
@@ -22,8 +22,8 @@ cfg_io_util! {
cfg_net_unix! {
/// A structure representing a connected Unix socket.
///
/// This socket can be connected directly with `UnixStream::connect` or accepted
/// from a listener with `UnixListener::incoming`. Additionally, a pair of
/// This socket can be connected directly with [`UnixStream::connect`] or accepted
/// from a listener with [`UnixListener::accept`]. Additionally, a pair of
/// anonymous Unix sockets can be created with `UnixStream::pair`.
///
/// To shut down the stream in the write direction, you can call the
@@ -32,6 +32,7 @@ cfg_net_unix! {
/// the stream in one direction.
///
/// [`shutdown()`]: fn@crate::io::AsyncWriteExt::shutdown
/// [`UnixListener::accept`]: crate::net::UnixListener::accept
pub struct UnixStream {
io: PollEvented<mio::net::UnixStream>,
}
@@ -239,8 +240,12 @@ impl UnixStream {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the stream's read half is closed
/// and will no longer yield data. If the stream is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The stream's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the stream is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
///
/// # Examples
@@ -699,11 +704,13 @@ impl UnixStream {
///
/// # Panics
///
/// This function panics if thread-local runtime is not set.
/// This function panics if it is not called from within a runtime with
/// IO enabled.
///
/// The runtime is usually set implicitly when this function is called
/// from a future driven by a tokio runtime, otherwise runtime can be set
/// explicitly with [`Runtime::enter`](crate::runtime::Runtime::enter) function.
#[track_caller]
pub fn from_std(stream: net::UnixStream) -> io::Result<UnixStream> {
let stream = mio::net::UnixStream::from_std(stream);
let io = PollEvented::new(stream)?;
+113 -4
View File
@@ -403,8 +403,12 @@ impl NamedPipeServer {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the pipe's read half is closed
/// and will no longer yield data. If the pipe is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The pipe's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the pipe is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
///
/// # Examples
@@ -1143,8 +1147,12 @@ impl NamedPipeClient {
/// # Return
///
/// If data is successfully read, `Ok(n)` is returned, where `n` is the
/// number of bytes read. `Ok(0)` indicates the pipe's read half is closed
/// and will no longer yield data. If the pipe is not ready to read data
/// number of bytes read. If `n` is `0`, then it can indicate one of two scenarios:
///
/// 1. The pipe's read half is closed and will no longer yield data.
/// 2. The specified buffer was 0 bytes in length.
///
/// If the pipe is not ready to read data,
/// `Err(io::ErrorKind::WouldBlock)` is returned.
///
/// # Examples
@@ -1955,6 +1963,106 @@ impl ServerOptions {
self
}
/// Requests permission to modify the pipe's discretionary access control list.
///
/// This corresponds to setting [`WRITE_DAC`] in dwOpenMode.
///
/// # Examples
///
/// ```
/// use std::{io, os::windows::prelude::AsRawHandle, ptr};
//
/// use tokio::net::windows::named_pipe::ServerOptions;
/// use winapi::{
/// shared::winerror::ERROR_SUCCESS,
/// um::{accctrl::SE_KERNEL_OBJECT, aclapi::SetSecurityInfo, winnt::DACL_SECURITY_INFORMATION},
/// };
///
/// const PIPE_NAME: &str = r"\\.\pipe\write_dac_pipe";
///
/// # #[tokio::main] async fn main() -> io::Result<()> {
/// let mut pipe_template = ServerOptions::new();
/// pipe_template.write_dac(true);
/// let pipe = pipe_template.create(PIPE_NAME)?;
///
/// unsafe {
/// assert_eq!(
/// ERROR_SUCCESS,
/// SetSecurityInfo(
/// pipe.as_raw_handle(),
/// SE_KERNEL_OBJECT,
/// DACL_SECURITY_INFORMATION,
/// ptr::null_mut(),
/// ptr::null_mut(),
/// ptr::null_mut(),
/// ptr::null_mut(),
/// )
/// );
/// }
///
/// # Ok(()) }
/// ```
///
/// ```
/// use std::{io, os::windows::prelude::AsRawHandle, ptr};
//
/// use tokio::net::windows::named_pipe::ServerOptions;
/// use winapi::{
/// shared::winerror::ERROR_ACCESS_DENIED,
/// um::{accctrl::SE_KERNEL_OBJECT, aclapi::SetSecurityInfo, winnt::DACL_SECURITY_INFORMATION},
/// };
///
/// const PIPE_NAME: &str = r"\\.\pipe\write_dac_pipe_fail";
///
/// # #[tokio::main] async fn main() -> io::Result<()> {
/// let mut pipe_template = ServerOptions::new();
/// pipe_template.write_dac(false);
/// let pipe = pipe_template.create(PIPE_NAME)?;
///
/// unsafe {
/// assert_eq!(
/// ERROR_ACCESS_DENIED,
/// SetSecurityInfo(
/// pipe.as_raw_handle(),
/// SE_KERNEL_OBJECT,
/// DACL_SECURITY_INFORMATION,
/// ptr::null_mut(),
/// ptr::null_mut(),
/// ptr::null_mut(),
/// ptr::null_mut(),
/// )
/// );
/// }
///
/// # Ok(()) }
/// ```
///
/// [`WRITE_DAC`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea
pub fn write_dac(&mut self, requested: bool) -> &mut Self {
bool_flag!(self.open_mode, requested, winnt::WRITE_DAC);
self
}
/// Requests permission to modify the pipe's owner.
///
/// This corresponds to setting [`WRITE_OWNER`] in dwOpenMode.
///
/// [`WRITE_OWNER`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea
pub fn write_owner(&mut self, requested: bool) -> &mut Self {
bool_flag!(self.open_mode, requested, winnt::WRITE_OWNER);
self
}
/// Requests permission to modify the pipe's system access control list.
///
/// This corresponds to setting [`ACCESS_SYSTEM_SECURITY`] in dwOpenMode.
///
/// [`ACCESS_SYSTEM_SECURITY`]: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createnamedpipea
pub fn access_system_security(&mut self, requested: bool) -> &mut Self {
bool_flag!(self.open_mode, requested, winnt::ACCESS_SYSTEM_SECURITY);
self
}
/// Indicates whether this server can accept remote clients or not. Remote
/// clients are disabled by default.
///
@@ -2020,6 +2128,7 @@ impl ServerOptions {
/// let builder = ServerOptions::new().max_instances(255);
/// # Ok(()) }
/// ```
#[track_caller]
pub fn max_instances(&mut self, instances: usize) -> &mut Self {
assert!(instances < 255, "cannot specify more than 254 instances");
self.max_instances = instances as DWORD;
-74
View File
@@ -1,74 +0,0 @@
#![cfg_attr(not(feature = "full"), allow(dead_code))]
use crate::park::{Park, Unpark};
use std::fmt;
use std::time::Duration;
pub(crate) enum Either<A, B> {
A(A),
B(B),
}
impl<A, B> Park for Either<A, B>
where
A: Park,
B: Park,
{
type Unpark = Either<A::Unpark, B::Unpark>;
type Error = Either<A::Error, B::Error>;
fn unpark(&self) -> Self::Unpark {
match self {
Either::A(a) => Either::A(a.unpark()),
Either::B(b) => Either::B(b.unpark()),
}
}
fn park(&mut self) -> Result<(), Self::Error> {
match self {
Either::A(a) => a.park().map_err(Either::A),
Either::B(b) => b.park().map_err(Either::B),
}
}
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error> {
match self {
Either::A(a) => a.park_timeout(duration).map_err(Either::A),
Either::B(b) => b.park_timeout(duration).map_err(Either::B),
}
}
fn shutdown(&mut self) {
match self {
Either::A(a) => a.shutdown(),
Either::B(b) => b.shutdown(),
}
}
}
impl<A, B> Unpark for Either<A, B>
where
A: Unpark,
B: Unpark,
{
fn unpark(&self) {
match self {
Either::A(a) => a.unpark(),
Either::B(b) => b.unpark(),
}
}
}
impl<A, B> fmt::Debug for Either<A, B>
where
A: fmt::Debug,
B: fmt::Debug,
{
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Either::A(a) => a.fmt(fmt),
Either::B(b) => b.fmt(fmt),
}
}
}
-80
View File
@@ -34,84 +34,4 @@
//! * `park_timeout` does the same as `park` but allows specifying a maximum
//! time to block the thread for.
cfg_rt! {
pub(crate) mod either;
}
#[cfg(any(feature = "rt", feature = "sync"))]
pub(crate) mod thread;
use std::fmt::Debug;
use std::sync::Arc;
use std::time::Duration;
/// Blocks the current thread.
pub(crate) trait Park {
/// Unpark handle type for the `Park` implementation.
type Unpark: Unpark;
/// Error returned by `park`.
type Error: Debug;
/// Gets a new `Unpark` handle associated with this `Park` instance.
fn unpark(&self) -> Self::Unpark;
/// Blocks the current thread unless or until the token is available.
///
/// A call to `park` does not guarantee that the thread will remain blocked
/// forever, and callers should be prepared for this possibility. This
/// function may wakeup spuriously for any reason.
///
/// # Panics
///
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Park` implementation.
fn park(&mut self) -> Result<(), Self::Error>;
/// Parks the current thread for at most `duration`.
///
/// This function is the same as `park` but allows specifying a maximum time
/// to block the thread for.
///
/// Same as `park`, there is no guarantee that the thread will remain
/// blocked for any amount of time. Spurious wakeups are permitted for any
/// reason.
///
/// # Panics
///
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Park` implementation.
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error>;
/// Releases all resources held by the parker for proper leak-free shutdown.
fn shutdown(&mut self);
}
/// Unblock a thread blocked by the associated `Park` instance.
pub(crate) trait Unpark: Sync + Send + 'static {
/// Unblocks a thread that is blocked by the associated `Park` handle.
///
/// Calling `unpark` atomically makes available the unpark token, if it is
/// not already available.
///
/// # Panics
///
/// This function **should** not panic, but ultimately, panics are left as
/// an implementation detail. Refer to the documentation for the specific
/// `Unpark` implementation.
fn unpark(&self);
}
impl Unpark for Box<dyn Unpark> {
fn unpark(&self) {
(**self).unpark()
}
}
impl Unpark for Arc<dyn Unpark> {
fn unpark(&self) {
(**self).unpark()
}
}
+31 -45
View File
@@ -2,7 +2,6 @@
use crate::loom::sync::atomic::AtomicUsize;
use crate::loom::sync::{Arc, Condvar, Mutex};
use crate::park::{Park, Unpark};
use std::sync::atomic::Ordering::SeqCst;
use std::time::Duration;
@@ -12,8 +11,6 @@ pub(crate) struct ParkThread {
inner: Arc<Inner>,
}
pub(crate) type ParkError = ();
/// Unblocks a thread that was blocked by `ParkThread`.
#[derive(Clone, Debug)]
pub(crate) struct UnparkThread {
@@ -47,28 +44,25 @@ impl ParkThread {
}),
}
}
}
impl Park for ParkThread {
type Unpark = UnparkThread;
type Error = ParkError;
fn unpark(&self) -> Self::Unpark {
pub(crate) fn unpark(&self) -> UnparkThread {
let inner = self.inner.clone();
UnparkThread { inner }
}
fn park(&mut self) -> Result<(), Self::Error> {
pub(crate) fn park(&mut self) {
self.inner.park();
Ok(())
}
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error> {
pub(crate) fn park_timeout(&mut self, duration: Duration) {
// Wasm doesn't have threads, so just sleep.
#[cfg(not(tokio_wasm))]
self.inner.park_timeout(duration);
Ok(())
#[cfg(tokio_wasm)]
std::thread::sleep(duration);
}
fn shutdown(&mut self) {
pub(crate) fn shutdown(&mut self) {
self.inner.shutdown();
}
}
@@ -208,12 +202,13 @@ impl Default for ParkThread {
// ===== impl UnparkThread =====
impl Unpark for UnparkThread {
fn unpark(&self) {
impl UnparkThread {
pub(crate) fn unpark(&self) {
self.inner.unpark();
}
}
use crate::loom::thread::AccessError;
use std::future::Future;
use std::marker::PhantomData;
use std::mem;
@@ -237,24 +232,38 @@ impl CachedParkThread {
}
}
pub(crate) fn get_unpark(&self) -> Result<UnparkThread, ParkError> {
pub(crate) fn waker(&self) -> Result<Waker, AccessError> {
self.unpark().map(|unpark| unpark.into_waker())
}
fn unpark(&self) -> Result<UnparkThread, AccessError> {
self.with_current(|park_thread| park_thread.unpark())
}
pub(crate) fn park(&mut self) {
self.with_current(|park_thread| park_thread.inner.park())
.unwrap();
}
pub(crate) fn park_timeout(&mut self, duration: Duration) {
self.with_current(|park_thread| park_thread.inner.park_timeout(duration))
.unwrap();
}
/// Gets a reference to the `ParkThread` handle for this thread.
fn with_current<F, R>(&self, f: F) -> Result<R, ParkError>
fn with_current<F, R>(&self, f: F) -> Result<R, AccessError>
where
F: FnOnce(&ParkThread) -> R,
{
CURRENT_PARKER.try_with(|inner| f(inner)).map_err(|_| ())
CURRENT_PARKER.try_with(|inner| f(inner))
}
pub(crate) fn block_on<F: Future>(&mut self, f: F) -> Result<F::Output, ParkError> {
pub(crate) fn block_on<F: Future>(&mut self, f: F) -> Result<F::Output, AccessError> {
use std::task::Context;
use std::task::Poll::Ready;
// `get_unpark()` should not return a Result
let waker = self.get_unpark()?.into_waker();
let waker = self.waker()?;
let mut cx = Context::from_waker(&waker);
pin!(f);
@@ -264,34 +273,11 @@ impl CachedParkThread {
return Ok(v);
}
self.park()?;
self.park();
}
}
}
impl Park for CachedParkThread {
type Unpark = UnparkThread;
type Error = ParkError;
fn unpark(&self) -> Self::Unpark {
self.get_unpark().unwrap()
}
fn park(&mut self) -> Result<(), Self::Error> {
self.with_current(|park_thread| park_thread.inner.park())?;
Ok(())
}
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error> {
self.with_current(|park_thread| park_thread.inner.park_timeout(duration))?;
Ok(())
}
fn shutdown(&mut self) {
let _ = self.with_current(|park_thread| park_thread.inner.shutdown());
}
}
impl UnparkThread {
pub(crate) fn into_waker(self) -> Waker {
unsafe {
+10 -12
View File
@@ -1285,41 +1285,39 @@ impl ChildStderr {
impl AsyncWrite for ChildStdin {
fn poll_write(
self: Pin<&mut Self>,
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
self.inner.poll_write(cx, buf)
Pin::new(&mut self.inner).poll_write(cx, buf)
}
fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.inner).poll_flush(cx)
}
fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.inner).poll_shutdown(cx)
}
}
impl AsyncRead for ChildStdout {
fn poll_read(
self: Pin<&mut Self>,
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
// Safety: pipes support reading into uninitialized memory
unsafe { self.inner.poll_read(cx, buf) }
Pin::new(&mut self.inner).poll_read(cx, buf)
}
}
impl AsyncRead for ChildStderr {
fn poll_read(
self: Pin<&mut Self>,
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
// Safety: pipes support reading into uninitialized memory
unsafe { self.inner.poll_read(cx, buf) }
Pin::new(&mut self.inner).poll_read(cx, buf)
}
}
+5 -20
View File
@@ -2,11 +2,9 @@
//! Process driver.
use crate::park::Park;
use crate::process::unix::GlobalOrphanQueue;
use crate::signal::unix::driver::{Driver as SignalDriver, Handle as SignalHandle};
use std::io;
use std::time::Duration;
/// Responsible for cleaning up orphaned child processes on Unix platforms.
@@ -28,31 +26,18 @@ impl Driver {
signal_handle,
}
}
}
// ===== impl Park for Driver =====
impl Park for Driver {
type Unpark = <SignalDriver as Park>::Unpark;
type Error = io::Error;
fn unpark(&self) -> Self::Unpark {
self.park.unpark()
}
fn park(&mut self) -> Result<(), Self::Error> {
self.park.park()?;
pub(crate) fn park(&mut self) {
self.park.park();
GlobalOrphanQueue::reap_orphans(&self.signal_handle);
Ok(())
}
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error> {
self.park.park_timeout(duration)?;
pub(crate) fn park_timeout(&mut self, duration: Duration) {
self.park.park_timeout(duration);
GlobalOrphanQueue::reap_orphans(&self.signal_handle);
Ok(())
}
fn shutdown(&mut self) {
pub(crate) fn shutdown(&mut self) {
self.park.shutdown()
}
}
+49 -6
View File
@@ -29,7 +29,7 @@ use orphan::{OrphanQueue, OrphanQueueImpl, Wait};
mod reap;
use reap::Reaper;
use crate::io::PollEvented;
use crate::io::{AsyncRead, AsyncWrite, PollEvented, ReadBuf};
use crate::process::kill::Kill;
use crate::process::SpawnedChild;
use crate::signal::unix::driver::Handle as SignalHandle;
@@ -177,8 +177,8 @@ impl AsRawFd for Pipe {
}
}
pub(crate) fn convert_to_stdio(io: PollEvented<Pipe>) -> io::Result<Stdio> {
let mut fd = io.into_inner()?.fd;
pub(crate) fn convert_to_stdio(io: ChildStdio) -> io::Result<Stdio> {
let mut fd = io.inner.into_inner()?.fd;
// Ensure that the fd to be inherited is set to *blocking* mode, as this
// is the default that virtually all programs expect to have. Those
@@ -213,7 +213,50 @@ impl Source for Pipe {
}
}
pub(crate) type ChildStdio = PollEvented<Pipe>;
pub(crate) struct ChildStdio {
inner: PollEvented<Pipe>,
}
impl fmt::Debug for ChildStdio {
fn fmt(&self, fmt: &mut fmt::Formatter<'_>) -> fmt::Result {
self.inner.fmt(fmt)
}
}
impl AsRawFd for ChildStdio {
fn as_raw_fd(&self) -> RawFd {
self.inner.as_raw_fd()
}
}
impl AsyncWrite for ChildStdio {
fn poll_write(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
self.inner.poll_write(cx, buf)
}
fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Poll::Ready(Ok(()))
}
}
impl AsyncRead for ChildStdio {
fn poll_read(
self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
// Safety: pipes support reading into uninitialized memory
unsafe { self.inner.poll_read(cx, buf) }
}
}
fn set_nonblocking<T: AsRawFd>(fd: &mut T, nonblocking: bool) -> io::Result<()> {
unsafe {
@@ -238,7 +281,7 @@ fn set_nonblocking<T: AsRawFd>(fd: &mut T, nonblocking: bool) -> io::Result<()>
Ok(())
}
pub(super) fn stdio<T>(io: T) -> io::Result<PollEvented<Pipe>>
pub(super) fn stdio<T>(io: T) -> io::Result<ChildStdio>
where
T: IntoRawFd,
{
@@ -246,5 +289,5 @@ where
let mut pipe = Pipe::from(io);
set_nonblocking(&mut pipe, true)?;
PollEvented::new(pipe)
PollEvented::new(pipe).map(|inner| ChildStdio { inner })
}
+1 -1
View File
@@ -120,7 +120,7 @@ where
#[cfg(all(test, not(loom)))]
pub(crate) mod test {
use super::*;
use crate::io::driver::Driver as IoDriver;
use crate::runtime::io::Driver as IoDriver;
use crate::signal::unix::driver::{Driver as SignalDriver, Handle as SignalHandle};
use crate::sync::watch;
use std::cell::{Cell, RefCell};
+84 -15
View File
@@ -15,22 +15,22 @@
//! `RegisterWaitForSingleObject` and then wait on the other end of the oneshot
//! from then on out.
use crate::io::PollEvented;
use crate::io::{blocking::Blocking, AsyncRead, AsyncWrite, ReadBuf};
use crate::process::kill::Kill;
use crate::process::SpawnedChild;
use crate::sync::oneshot;
use mio::windows::NamedPipe;
use std::fmt;
use std::fs::File as StdFile;
use std::future::Future;
use std::io;
use std::os::windows::prelude::{AsRawHandle, FromRawHandle, IntoRawHandle, RawHandle};
use std::os::windows::prelude::{AsRawHandle, IntoRawHandle, RawHandle};
use std::pin::Pin;
use std::process::Stdio;
use std::process::{Child as StdChild, Command as StdCommand, ExitStatus};
use std::ptr;
use std::task::Context;
use std::task::Poll;
use std::sync::Arc;
use std::task::{Context, Poll};
use winapi::shared::minwindef::{DWORD, FALSE};
use winapi::um::handleapi::{DuplicateHandle, INVALID_HANDLE_VALUE};
use winapi::um::processthreadsapi::GetCurrentProcess;
@@ -167,28 +167,97 @@ unsafe extern "system" fn callback(ptr: PVOID, _timer_fired: BOOLEAN) {
let _ = complete.take().unwrap().send(());
}
pub(crate) type ChildStdio = PollEvented<NamedPipe>;
#[derive(Debug)]
struct ArcFile(Arc<StdFile>);
pub(super) fn stdio<T>(io: T) -> io::Result<PollEvented<NamedPipe>>
impl io::Read for ArcFile {
fn read(&mut self, bytes: &mut [u8]) -> io::Result<usize> {
(&*self.0).read(bytes)
}
}
impl io::Write for ArcFile {
fn write(&mut self, bytes: &[u8]) -> io::Result<usize> {
(&*self.0).write(bytes)
}
fn flush(&mut self) -> io::Result<()> {
(&*self.0).flush()
}
}
#[derive(Debug)]
pub(crate) struct ChildStdio {
// Used for accessing the raw handle, even if the io version is busy
raw: Arc<StdFile>,
// For doing I/O operations asynchronously
io: Blocking<ArcFile>,
}
impl AsRawHandle for ChildStdio {
fn as_raw_handle(&self) -> RawHandle {
self.raw.as_raw_handle()
}
}
impl AsyncRead for ChildStdio {
fn poll_read(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &mut ReadBuf<'_>,
) -> Poll<io::Result<()>> {
Pin::new(&mut self.io).poll_read(cx, buf)
}
}
impl AsyncWrite for ChildStdio {
fn poll_write(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
buf: &[u8],
) -> Poll<io::Result<usize>> {
Pin::new(&mut self.io).poll_write(cx, buf)
}
fn poll_flush(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.io).poll_flush(cx)
}
fn poll_shutdown(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Pin::new(&mut self.io).poll_shutdown(cx)
}
}
pub(super) fn stdio<T>(io: T) -> io::Result<ChildStdio>
where
T: IntoRawHandle,
{
let pipe = unsafe { NamedPipe::from_raw_handle(io.into_raw_handle()) };
PollEvented::new(pipe)
use std::os::windows::prelude::FromRawHandle;
let raw = Arc::new(unsafe { StdFile::from_raw_handle(io.into_raw_handle()) });
let io = Blocking::new(ArcFile(raw.clone()));
Ok(ChildStdio { raw, io })
}
pub(crate) fn convert_to_stdio(io: PollEvented<NamedPipe>) -> io::Result<Stdio> {
let named_pipe = io.into_inner()?;
pub(crate) fn convert_to_stdio(child_stdio: ChildStdio) -> io::Result<Stdio> {
let ChildStdio { raw, io } = child_stdio;
drop(io); // Try to drop the Arc count here
Arc::try_unwrap(raw)
.or_else(|raw| duplicate_handle(&*raw))
.map(Stdio::from)
}
fn duplicate_handle<T: AsRawHandle>(io: &T) -> io::Result<StdFile> {
use std::os::windows::prelude::FromRawHandle;
// Mio does not implement `IntoRawHandle` for `NamedPipe`, so we'll manually
// duplicate the handle here...
unsafe {
let mut dup_handle = INVALID_HANDLE_VALUE;
let cur_proc = GetCurrentProcess();
let status = DuplicateHandle(
cur_proc,
named_pipe.as_raw_handle(),
io.as_raw_handle(),
cur_proc,
&mut dup_handle,
0 as DWORD,
@@ -200,6 +269,6 @@ pub(crate) fn convert_to_stdio(io: PollEvented<NamedPipe>) -> io::Result<Stdio>
return Err(io::Error::last_os_error());
}
Ok(Stdio::from_raw_handle(dup_handle))
Ok(StdFile::from_raw_handle(dup_handle))
}
}
+6 -1
View File
@@ -4,15 +4,20 @@
//! compilation.
mod pool;
pub(crate) use pool::{spawn_blocking, BlockingPool, Mandatory, Spawner, Task};
pub(crate) use pool::{spawn_blocking, BlockingPool, Spawner};
cfg_fs! {
pub(crate) use pool::spawn_mandatory_blocking;
}
cfg_trace! {
pub(crate) use pool::Mandatory;
}
mod schedule;
mod shutdown;
mod task;
#[cfg(all(test, not(tokio_wasm)))]
pub(crate) use schedule::NoopSchedule;
pub(crate) use task::BlockingTask;
+133 -9
View File
@@ -3,14 +3,16 @@
use crate::loom::sync::{Arc, Condvar, Mutex};
use crate::loom::thread;
use crate::runtime::blocking::schedule::NoopSchedule;
use crate::runtime::blocking::shutdown;
use crate::runtime::blocking::{shutdown, BlockingTask};
use crate::runtime::builder::ThreadNameFn;
use crate::runtime::context;
use crate::runtime::task::{self, JoinHandle};
use crate::runtime::{Builder, Callback, ToHandle};
use crate::runtime::{Builder, Callback, Handle};
use crate::util::{replace_thread_rng, RngSeedGenerator};
use std::collections::{HashMap, VecDeque};
use std::fmt;
use std::io;
use std::time::Duration;
pub(crate) struct BlockingPool {
@@ -47,6 +49,9 @@ struct Inner {
// Customizable wait timeout.
keep_alive: Duration,
// Random number seed
seed_generator: RngSeedGenerator,
}
struct Shared {
@@ -82,6 +87,25 @@ pub(crate) enum Mandatory {
NonMandatory,
}
pub(crate) enum SpawnError {
/// Pool is shutting down and the task was not scheduled
ShuttingDown,
/// There are no worker threads available to take the task
/// and the OS failed to spawn a new one
NoThreads(io::Error),
}
impl From<SpawnError> for io::Error {
fn from(e: SpawnError) -> Self {
match e {
SpawnError::ShuttingDown => {
io::Error::new(io::ErrorKind::Other, "blocking pool shutting down")
}
SpawnError::NoThreads(e) => e,
}
}
}
impl Task {
pub(crate) fn new(task: task::UnownedTask<NoopSchedule>, mandatory: Mandatory) -> Task {
Task { task, mandatory }
@@ -105,6 +129,7 @@ const KEEP_ALIVE: Duration = Duration::from_secs(10);
/// Tasks will be scheduled as non-mandatory, meaning they may not get executed
/// in case of runtime shutdown.
#[track_caller]
#[cfg_attr(tokio_wasi, allow(dead_code))]
pub(crate) fn spawn_blocking<F, R>(func: F) -> JoinHandle<R>
where
F: FnOnce() -> R + Send + 'static,
@@ -129,7 +154,7 @@ cfg_fs! {
R: Send + 'static,
{
let rt = context::current();
rt.as_inner().spawn_mandatory_blocking(&rt, func)
rt.inner.blocking_spawner().spawn_mandatory_blocking(&rt, func)
}
}
@@ -161,6 +186,7 @@ impl BlockingPool {
before_stop: builder.before_stop.clone(),
thread_cap,
keep_alive,
seed_generator: builder.seed_generator.next_generator(),
}),
},
shutdown_rx,
@@ -220,7 +246,103 @@ impl fmt::Debug for BlockingPool {
// ===== impl Spawner =====
impl Spawner {
pub(crate) fn spawn(&self, task: Task, rt: &dyn ToHandle) -> Result<(), ()> {
#[track_caller]
pub(crate) fn spawn_blocking<F, R>(&self, rt: &Handle, func: F) -> JoinHandle<R>
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let (join_handle, spawn_result) =
if cfg!(debug_assertions) && std::mem::size_of::<F>() > 2048 {
self.spawn_blocking_inner(Box::new(func), Mandatory::NonMandatory, None, rt)
} else {
self.spawn_blocking_inner(func, Mandatory::NonMandatory, None, rt)
};
match spawn_result {
Ok(()) => join_handle,
// Compat: do not panic here, return the join_handle even though it will never resolve
Err(SpawnError::ShuttingDown) => join_handle,
Err(SpawnError::NoThreads(e)) => {
panic!("OS can't spawn worker thread: {}", e)
}
}
}
cfg_fs! {
#[track_caller]
#[cfg_attr(any(
all(loom, not(test)), // the function is covered by loom tests
test
), allow(dead_code))]
pub(crate) fn spawn_mandatory_blocking<F, R>(&self, rt: &Handle, func: F) -> Option<JoinHandle<R>>
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let (join_handle, spawn_result) = if cfg!(debug_assertions) && std::mem::size_of::<F>() > 2048 {
self.spawn_blocking_inner(
Box::new(func),
Mandatory::Mandatory,
None,
rt,
)
} else {
self.spawn_blocking_inner(
func,
Mandatory::Mandatory,
None,
rt,
)
};
if spawn_result.is_ok() {
Some(join_handle)
} else {
None
}
}
}
#[track_caller]
pub(crate) fn spawn_blocking_inner<F, R>(
&self,
func: F,
is_mandatory: Mandatory,
name: Option<&str>,
rt: &Handle,
) -> (JoinHandle<R>, Result<(), SpawnError>)
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let fut = BlockingTask::new(func);
let id = task::Id::next();
#[cfg(all(tokio_unstable, feature = "tracing"))]
let fut = {
use tracing::Instrument;
let location = std::panic::Location::caller();
let span = tracing::trace_span!(
target: "tokio::task::blocking",
"runtime.spawn",
kind = %"blocking",
task.name = %name.unwrap_or_default(),
task.id = id.as_u64(),
"fn" = %std::any::type_name::<F>(),
spawn.location = %format_args!("{}:{}:{}", location.file(), location.line(), location.column()),
);
fut.instrument(span)
};
#[cfg(not(all(tokio_unstable, feature = "tracing")))]
let _ = name;
let (task, handle) = task::unowned(fut, NoopSchedule, id);
let spawned = self.spawn_task(Task::new(task, is_mandatory), rt);
(handle, spawned)
}
fn spawn_task(&self, task: Task, rt: &Handle) -> Result<(), SpawnError> {
let mut shared = self.inner.shared.lock();
if shared.shutdown {
@@ -230,7 +352,7 @@ impl Spawner {
task.task.shutdown();
// no need to even push this task; it would never get picked up
return Err(());
return Err(SpawnError::ShuttingDown);
}
shared.queue.push_back(task);
@@ -261,7 +383,7 @@ impl Spawner {
Err(e) => {
// The OS refused to spawn the thread and there is no thread
// to pick up the task that has just been pushed to the queue.
panic!("OS can't spawn worker thread: {}", e)
return Err(SpawnError::NoThreads(e));
}
}
}
@@ -283,7 +405,7 @@ impl Spawner {
fn spawn_thread(
&self,
shutdown_tx: shutdown::Sender,
rt: &dyn ToHandle,
rt: &Handle,
id: usize,
) -> std::io::Result<thread::JoinHandle<()>> {
let mut builder = thread::Builder::new().name((self.inner.thread_name)());
@@ -292,12 +414,12 @@ impl Spawner {
builder = builder.stack_size(stack_size);
}
let rt = rt.to_handle();
let rt = rt.clone();
builder.spawn(move || {
// Only the reference should be moved into the closure
let _enter = crate::runtime::context::enter(rt.clone());
rt.as_inner().blocking_spawner.inner.run(id);
rt.inner.blocking_spawner().inner.run(id);
drop(shutdown_tx);
})
}
@@ -314,6 +436,8 @@ impl Inner {
if let Some(f) = &self.after_start {
f()
}
// We own this thread so there is no need to replace the RngSeed once we're done.
let _ = replace_thread_rng(self.seed_generator.next_seed());
let mut shared = self.shared.lock();
let mut join_on_thread = None;
+152 -55
View File
@@ -1,5 +1,6 @@
use crate::runtime::handle::Handle;
use crate::runtime::{blocking, driver, Callback, Runtime, Spawner};
use crate::runtime::{blocking, driver, Callback, Runtime};
use crate::util::{RngSeed, RngSeedGenerator};
use std::fmt;
use std::io;
@@ -85,6 +86,14 @@ pub struct Builder {
/// How many ticks before yielding to the driver for timer and I/O events?
pub(super) event_interval: u32,
/// When true, the multi-threade scheduler LIFO slot should not be used.
///
/// This option should only be exposed as unstable.
pub(super) disable_lifo_slot: bool,
/// Specify a random number generator seed to provide deterministic results
pub(super) seed_generator: RngSeedGenerator,
#[cfg(tokio_unstable)]
pub(super) unhandled_panic: UnhandledPanic,
}
@@ -174,7 +183,7 @@ pub(crate) type ThreadNameFn = std::sync::Arc<dyn Fn() -> String + Send + Sync +
pub(crate) enum Kind {
CurrentThread,
#[cfg(feature = "rt-multi-thread")]
#[cfg(all(feature = "rt-multi-thread", not(tokio_wasi)))]
MultiThread,
}
@@ -197,14 +206,16 @@ impl Builder {
Builder::new(Kind::CurrentThread, 31, EVENT_INTERVAL)
}
/// Returns a new builder with the multi thread scheduler selected.
///
/// Configuration methods can be chained on the return value.
#[cfg(feature = "rt-multi-thread")]
#[cfg_attr(docsrs, doc(cfg(feature = "rt-multi-thread")))]
pub fn new_multi_thread() -> Builder {
// The number `61` is fairly arbitrary. I believe this value was copied from golang.
Builder::new(Kind::MultiThread, 61, 61)
cfg_not_wasi! {
/// Returns a new builder with the multi thread scheduler selected.
///
/// Configuration methods can be chained on the return value.
#[cfg(feature = "rt-multi-thread")]
#[cfg_attr(docsrs, doc(cfg(feature = "rt-multi-thread")))]
pub fn new_multi_thread() -> Builder {
// The number `61` is fairly arbitrary. I believe this value was copied from golang.
Builder::new(Kind::MultiThread, 61, 61)
}
}
/// Returns a new runtime builder initialized with default configuration
@@ -248,8 +259,12 @@ impl Builder {
global_queue_interval,
event_interval,
seed_generator: RngSeedGenerator::new(RngSeed::new()),
#[cfg(tokio_unstable)]
unhandled_panic: UnhandledPanic::Ignore,
disable_lifo_slot: false,
}
}
@@ -270,7 +285,11 @@ impl Builder {
/// .unwrap();
/// ```
pub fn enable_all(&mut self) -> &mut Self {
#[cfg(any(feature = "net", feature = "process", all(unix, feature = "signal")))]
#[cfg(any(
feature = "net",
all(unix, feature = "process"),
all(unix, feature = "signal")
))]
self.enable_io();
#[cfg(feature = "time")]
self.enable_time();
@@ -287,11 +306,7 @@ impl Builder {
///
/// The default value is the number of cores available to the system.
///
/// # Panics
///
/// When using the `current_thread` runtime this method will panic, since
/// those variants do not allow setting worker thread counts.
///
/// When using the `current_thread` runtime this method has no effect.
///
/// # Examples
///
@@ -616,8 +631,8 @@ impl Builder {
/// ```
pub fn build(&mut self) -> io::Result<Runtime> {
match &self.kind {
Kind::CurrentThread => self.build_basic_runtime(),
#[cfg(feature = "rt-multi-thread")]
Kind::CurrentThread => self.build_current_thread_runtime(),
#[cfg(all(feature = "rt-multi-thread", not(tokio_wasi)))]
Kind::MultiThread => self.build_threaded_runtime(),
}
}
@@ -626,7 +641,7 @@ impl Builder {
driver::Cfg {
enable_pause_time: match self.kind {
Kind::CurrentThread => true,
#[cfg(feature = "rt-multi-thread")]
#[cfg(all(feature = "rt-multi-thread", not(tokio_wasi)))]
Kind::MultiThread => false,
},
enable_io: self.enable_io,
@@ -736,7 +751,7 @@ impl Builder {
/// * `UnhandledPanic::ShutdownRuntime` will force the runtime to
/// shutdown immediately when a spawned task panics even if that
/// task's `JoinHandle` has not been dropped. All other spawned tasks
/// will immediatetly terminate and further calls to
/// will immediately terminate and further calls to
/// [`Runtime::block_on`] will panic.
///
/// # Unstable
@@ -779,33 +794,108 @@ impl Builder {
self.unhandled_panic = behavior;
self
}
/// Disables the LIFO task scheduler heuristic.
///
/// The multi-threaded scheduler includes a heuristic for optimizing
/// message-passing patterns. This heuristic results in the **last**
/// scheduled task being polled first.
///
/// To implement this heuristic, each worker thread has a slot which
/// holds the task that should be polled next. However, this slot cannot
/// be stolen by other worker threads, which can result in lower total
/// throughput when tasks tend to have longer poll times.
///
/// This configuration option will disable this heuristic resulting in
/// all scheduled tasks being pushed into the worker-local queue, which
/// is stealable.
///
/// Consider trying this option when the task "scheduled" time is high
/// but the runtime is underutilized. Use tokio-rs/tokio-metrics to
/// collect this data.
///
/// # Unstable
///
/// This configuration option is considered a workaround for the LIFO
/// slot not being stealable. When the slot becomes stealable, we will
/// revisit whther or not this option is necessary. See
/// tokio-rs/tokio#4941.
///
/// # Examples
///
/// ```
/// use tokio::runtime;
///
/// let rt = runtime::Builder::new_multi_thread()
/// .disable_lifo_slot()
/// .build()
/// .unwrap();
/// ```
pub fn disable_lifo_slot(&mut self) -> &mut Self {
self.disable_lifo_slot = true;
self
}
/// Specifies the random number generation seed to use within all threads associated
/// with the runtime being built.
///
/// This option is intended to make certain parts of the runtime deterministic.
/// Specifically, it affects the [`tokio::select!`] macro and the work stealing
/// algorithm. In the case of [`tokio::select!`] it will ensure that the order that
/// branches are polled is deterministic.
///
/// In the case of work stealing, it's a little more complicated. Each worker will
/// be given a deterministic seed so that the starting peer for each work stealing
/// search will be deterministic.
///
/// In addition to the code specifying `rng_seed` and interacting with the runtime,
/// the internals of Tokio and the Rust compiler may affect the sequences of random
/// numbers. In order to ensure repeatable results, the version of Tokio, the versions
/// of all other dependencies that interact with Tokio, and the Rust compiler version
/// should also all remain constant.
///
/// # Examples
///
/// ```
/// # use tokio::runtime::{self, RngSeed};
/// # pub fn main() {
/// let seed = RngSeed::from_bytes(b"place your seed here");
/// let rt = runtime::Builder::new_current_thread()
/// .rng_seed(seed)
/// .build();
/// # }
/// ```
///
/// [`tokio::select!`]: crate::select
pub fn rng_seed(&mut self, seed: RngSeed) -> &mut Self {
self.seed_generator = RngSeedGenerator::new(seed);
self
}
}
fn build_basic_runtime(&mut self) -> io::Result<Runtime> {
use crate::runtime::basic_scheduler::Config;
use crate::runtime::{BasicScheduler, HandleInner, Kind};
fn build_current_thread_runtime(&mut self) -> io::Result<Runtime> {
use crate::runtime::scheduler::{self, CurrentThread};
use crate::runtime::{Config, Scheduler};
let (driver, resources) = driver::Driver::new(self.get_cfg())?;
let (driver, driver_handle) = driver::Driver::new(self.get_cfg())?;
// Blocking pool
let blocking_pool = blocking::create_blocking_pool(self, self.max_blocking_threads);
let blocking_spawner = blocking_pool.spawner().clone();
let handle_inner = HandleInner {
io_handle: resources.io_handle,
time_handle: resources.time_handle,
signal_handle: resources.signal_handle,
clock: resources.clock,
blocking_spawner,
};
// Generate a rng seed for this runtime.
let seed_generator_1 = self.seed_generator.next_generator();
let seed_generator_2 = self.seed_generator.next_generator();
// And now put a single-threaded scheduler on top of the timer. When
// there are no futures ready to do something, it'll let the timer or
// the reactor to generate some new stimuli for the futures to continue
// in their life.
let scheduler = BasicScheduler::new(
let scheduler = CurrentThread::new(
driver,
handle_inner,
driver_handle,
blocking_spawner,
seed_generator_2,
Config {
before_park: self.before_park.clone(),
after_unpark: self.after_unpark.clone(),
@@ -813,13 +903,16 @@ impl Builder {
event_interval: self.event_interval,
#[cfg(tokio_unstable)]
unhandled_panic: self.unhandled_panic.clone(),
disable_lifo_slot: self.disable_lifo_slot,
seed_generator: seed_generator_1,
},
);
let spawner = Spawner::Basic(scheduler.spawner().clone());
let handle = scheduler::Handle::CurrentThread(scheduler.handle().clone());
Ok(Runtime {
kind: Kind::CurrentThread(scheduler),
handle: Handle { spawner },
scheduler: Scheduler::CurrentThread(scheduler),
handle: Handle { inner: handle },
blocking_pool,
})
}
@@ -901,45 +994,49 @@ cfg_rt_multi_thread! {
impl Builder {
fn build_threaded_runtime(&mut self) -> io::Result<Runtime> {
use crate::loom::sys::num_cpus;
use crate::runtime::{HandleInner, Kind, ThreadPool};
use crate::runtime::{Config, Scheduler};
use crate::runtime::scheduler::{self, MultiThread};
let core_threads = self.worker_threads.unwrap_or_else(num_cpus);
let (driver, resources) = driver::Driver::new(self.get_cfg())?;
let (driver, driver_handle) = driver::Driver::new(self.get_cfg())?;
// Create the blocking pool
let blocking_pool =
blocking::create_blocking_pool(self, self.max_blocking_threads + core_threads);
let blocking_spawner = blocking_pool.spawner().clone();
let handle_inner = HandleInner {
io_handle: resources.io_handle,
time_handle: resources.time_handle,
signal_handle: resources.signal_handle,
clock: resources.clock,
blocking_spawner,
};
// Generate a rng seed for this runtime.
let seed_generator_1 = self.seed_generator.next_generator();
let seed_generator_2 = self.seed_generator.next_generator();
let (scheduler, launch) = ThreadPool::new(
let (scheduler, launch) = MultiThread::new(
core_threads,
driver,
handle_inner,
self.before_park.clone(),
self.after_unpark.clone(),
self.global_queue_interval,
self.event_interval,
driver_handle,
blocking_spawner,
seed_generator_2,
Config {
before_park: self.before_park.clone(),
after_unpark: self.after_unpark.clone(),
global_queue_interval: self.global_queue_interval,
event_interval: self.event_interval,
#[cfg(tokio_unstable)]
unhandled_panic: self.unhandled_panic.clone(),
disable_lifo_slot: self.disable_lifo_slot,
seed_generator: seed_generator_1,
},
);
let spawner = Spawner::ThreadPool(scheduler.spawner().clone());
// Create the runtime handle
let handle = Handle { spawner };
let handle = scheduler::Handle::MultiThread(scheduler.handle().clone());
let handle = Handle { inner: handle };
// Spawn the thread pool workers
let _enter = crate::runtime::context::enter(handle.clone());
launch.launch();
Ok(Runtime {
kind: Kind::ThreadPool(scheduler),
scheduler: Scheduler::MultiThread(scheduler),
handle,
blocking_pool,
})
+34
View File
@@ -0,0 +1,34 @@
#![cfg_attr(any(not(feature = "full"), tokio_wasm), allow(dead_code))]
use crate::runtime::Callback;
use crate::util::RngSeedGenerator;
pub(crate) struct Config {
/// How many ticks before pulling a task from the global/remote queue?
pub(crate) global_queue_interval: u32,
/// How many ticks before yielding to the driver for timer and I/O events?
pub(crate) event_interval: u32,
/// Callback for a worker parking itself
pub(crate) before_park: Option<Callback>,
/// Callback for a worker unparking itself
pub(crate) after_unpark: Option<Callback>,
/// The multi-threaded scheduler includes a per-worker LIFO slot used to
/// store the last scheduled task. This can improve certain usage patterns,
/// especially message passing between tasks. However, this LIFO slot is not
/// currently stealable.
///
/// Eventually, the LIFO slot **will** become stealable, however as a
/// stop-gap, this unstable option lets users disable the LIFO task.
pub(crate) disable_lifo_slot: bool,
/// Random number generator seed to configure runtimes to act in a
/// deterministic way.
pub(crate) seed_generator: RngSeedGenerator,
#[cfg(tokio_unstable)]
/// How to respond to unhandled task panics.
pub(crate) unhandled_panic: crate::runtime::UnhandledPanic,
}
+35 -30
View File
@@ -1,5 +1,6 @@
//! Thread local runtime context
use crate::runtime::{Handle, TryCurrentError};
use crate::util::{replace_thread_rng, RngSeed};
use std::cell::RefCell;
@@ -24,10 +25,16 @@ pub(crate) fn current() -> Handle {
}
cfg_io_driver! {
#[track_caller]
pub(crate) fn io_handle() -> crate::runtime::driver::IoHandle {
match CONTEXT.try_with(|ctx| {
let ctx = ctx.borrow();
ctx.as_ref().expect(crate::util::error::CONTEXT_MISSING_ERROR).as_inner().io_handle.clone()
ctx.as_ref()
.expect(crate::util::error::CONTEXT_MISSING_ERROR)
.inner
.driver()
.io
.clone()
}) {
Ok(io_handle) => io_handle,
Err(_) => panic!("{}", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),
@@ -40,7 +47,11 @@ cfg_signal_internal! {
pub(crate) fn signal_handle() -> crate::runtime::driver::SignalHandle {
match CONTEXT.try_with(|ctx| {
let ctx = ctx.borrow();
ctx.as_ref().expect(crate::util::error::CONTEXT_MISSING_ERROR).as_inner().signal_handle.clone()
ctx.as_ref()
.expect(crate::util::error::CONTEXT_MISSING_ERROR)
.inner
.signal()
.clone()
}) {
Ok(signal_handle) => signal_handle,
Err(_) => panic!("{}", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),
@@ -49,19 +60,14 @@ cfg_signal_internal! {
}
cfg_time! {
pub(crate) fn time_handle() -> crate::runtime::driver::TimeHandle {
match CONTEXT.try_with(|ctx| {
let ctx = ctx.borrow();
ctx.as_ref().expect(crate::util::error::CONTEXT_MISSING_ERROR).as_inner().time_handle.clone()
}) {
Ok(time_handle) => time_handle,
Err(_) => panic!("{}", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),
}
}
cfg_test_util! {
pub(crate) fn clock() -> Option<crate::runtime::driver::Clock> {
match CONTEXT.try_with(|ctx| (*ctx.borrow()).as_ref().map(|ctx| ctx.as_inner().clock.clone())) {
match CONTEXT.try_with(|ctx| {
let ctx = ctx.borrow();
ctx
.as_ref()
.map(|ctx| ctx.inner.clock().clone())
}) {
Ok(clock) => clock,
Err(_) => panic!("{}", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),
}
@@ -69,15 +75,6 @@ cfg_time! {
}
}
cfg_rt! {
pub(crate) fn spawn_handle() -> Option<crate::runtime::Spawner> {
match CONTEXT.try_with(|ctx| (*ctx.borrow()).as_ref().map(|ctx| ctx.spawner.clone())) {
Ok(spawner) => spawner,
Err(_) => panic!("{}", crate::util::error::THREAD_LOCAL_DESTROYED_ERROR),
}
}
}
/// Sets this [`Handle`] as the current active [`Handle`].
///
/// [`Handle`]: Handle
@@ -92,21 +89,29 @@ pub(crate) fn enter(new: Handle) -> EnterGuard {
///
/// [`Handle`]: Handle
pub(crate) fn try_enter(new: Handle) -> Option<EnterGuard> {
CONTEXT
.try_with(|ctx| {
let old = ctx.borrow_mut().replace(new);
EnterGuard(old)
})
.ok()
let rng_seed = new.inner.seed_generator().next_seed();
let old_handle = CONTEXT.try_with(|ctx| ctx.borrow_mut().replace(new)).ok()?;
let old_seed = replace_thread_rng(rng_seed);
Some(EnterGuard {
old_handle,
old_seed,
})
}
#[derive(Debug)]
pub(crate) struct EnterGuard(Option<Handle>);
pub(crate) struct EnterGuard {
old_handle: Option<Handle>,
old_seed: RngSeed,
}
impl Drop for EnterGuard {
fn drop(&mut self) {
CONTEXT.with(|ctx| {
*ctx.borrow_mut() = self.0.take();
*ctx.borrow_mut() = self.old_handle.take();
});
// We discard the RngSeed associated with this guard
let _ = replace_thread_rng(self.old_seed.clone());
}
}
+195 -84
View File
@@ -1,45 +1,192 @@
//! Abstracts out the entire chain of runtime sub-drivers into common types.
use crate::park::thread::ParkThread;
use crate::park::Park;
// Eventually, this file will see significant refactoring / cleanup. For now, we
// don't need to worry much about dead code with certain feature permutations.
#![cfg_attr(not(feature = "full"), allow(dead_code))]
use crate::park::thread::{ParkThread, UnparkThread};
use std::io;
use std::time::Duration;
#[derive(Debug)]
pub(crate) struct Driver {
inner: TimeDriver,
}
#[derive(Debug)]
pub(crate) struct Handle {
/// IO driver handle
pub(crate) io: IoHandle,
/// Signal driver handle
#[cfg_attr(any(not(unix), loom), allow(dead_code))]
pub(crate) signal: SignalHandle,
/// Time driver handle
pub(crate) time: TimeHandle,
/// Source of `Instant::now()`
#[cfg_attr(not(all(feature = "time", feature = "test-util")), allow(dead_code))]
pub(crate) clock: Clock,
}
pub(crate) struct Cfg {
pub(crate) enable_io: bool,
pub(crate) enable_time: bool,
pub(crate) enable_pause_time: bool,
pub(crate) start_paused: bool,
}
impl Driver {
pub(crate) fn new(cfg: Cfg) -> io::Result<(Self, Handle)> {
let (io_stack, io_handle, signal_handle) = create_io_stack(cfg.enable_io)?;
let clock = create_clock(cfg.enable_pause_time, cfg.start_paused);
let (time_driver, time_handle) =
create_time_driver(cfg.enable_time, io_stack, clock.clone());
Ok((
Self { inner: time_driver },
Handle {
io: io_handle,
signal: signal_handle,
time: time_handle,
clock,
},
))
}
pub(crate) fn park(&mut self) {
self.inner.park()
}
pub(crate) fn park_timeout(&mut self, duration: Duration) {
self.inner.park_timeout(duration)
}
pub(crate) fn shutdown(&mut self) {
self.inner.shutdown()
}
}
impl Handle {
pub(crate) fn unpark(&self) {
#[cfg(feature = "time")]
if let Some(handle) = &self.time {
handle.unpark();
}
self.io.unpark();
}
}
// ===== io driver =====
cfg_io_driver! {
type IoDriver = crate::io::driver::Driver;
type IoStack = crate::park::either::Either<ProcessDriver, ParkThread>;
pub(crate) type IoHandle = Option<crate::io::driver::Handle>;
pub(crate) type IoDriver = crate::runtime::io::Driver;
#[derive(Debug)]
pub(crate) enum IoStack {
Enabled(ProcessDriver),
Disabled(ParkThread),
}
#[derive(Debug, Clone)]
pub(crate) enum IoHandle {
Enabled(crate::runtime::io::Handle),
Disabled(UnparkThread),
}
fn create_io_stack(enabled: bool) -> io::Result<(IoStack, IoHandle, SignalHandle)> {
use crate::park::either::Either;
#[cfg(loom)]
assert!(!enabled);
let ret = if enabled {
let io_driver = crate::io::driver::Driver::new()?;
let io_driver = crate::runtime::io::Driver::new()?;
let io_handle = io_driver.handle();
let (signal_driver, signal_handle) = create_signal_driver(io_driver)?;
let process_driver = create_process_driver(signal_driver);
(Either::A(process_driver), Some(io_handle), signal_handle)
(IoStack::Enabled(process_driver), IoHandle::Enabled(io_handle), signal_handle)
} else {
(Either::B(ParkThread::new()), Default::default(), Default::default())
let park_thread = ParkThread::new();
let unpark_thread = park_thread.unpark();
(IoStack::Disabled(park_thread), IoHandle::Disabled(unpark_thread), Default::default())
};
Ok(ret)
}
impl IoStack {
/*
pub(crate) fn handle(&self) -> IoHandle {
match self {
IoStack::Enabled(v) => IoHandle::Enabled(v.handle()),
IoStack::Disabled(v) => IoHandle::Disabled(v.unpark()),
}
}]
*/
pub(crate) fn park(&mut self) {
match self {
IoStack::Enabled(v) => v.park(),
IoStack::Disabled(v) => v.park(),
}
}
pub(crate) fn park_timeout(&mut self, duration: Duration) {
match self {
IoStack::Enabled(v) => v.park_timeout(duration),
IoStack::Disabled(v) => v.park_timeout(duration),
}
}
pub(crate) fn shutdown(&mut self) {
match self {
IoStack::Enabled(v) => v.shutdown(),
IoStack::Disabled(v) => v.shutdown(),
}
}
}
impl IoHandle {
pub(crate) fn unpark(&self) {
match self {
IoHandle::Enabled(handle) => handle.unpark(),
IoHandle::Disabled(handle) => handle.unpark(),
}
}
#[track_caller]
pub(crate) fn expect(self, msg: &'static str) -> crate::runtime::io::Handle {
match self {
IoHandle::Enabled(v) => v,
IoHandle::Disabled(..) => panic!("{}", msg),
}
}
cfg_unstable! {
pub(crate) fn as_ref(&self) -> Option<&crate::runtime::io::Handle> {
match self {
IoHandle::Enabled(v) => Some(v),
IoHandle::Disabled(..) => None,
}
}
}
}
}
cfg_not_io_driver! {
pub(crate) type IoHandle = ();
type IoStack = ParkThread;
pub(crate) type IoHandle = UnparkThread;
pub(crate) type IoStack = ParkThread;
fn create_io_stack(_enabled: bool) -> io::Result<(IoStack, IoHandle, SignalHandle)> {
Ok((ParkThread::new(), Default::default(), Default::default()))
let park_thread = ParkThread::new();
let unpark_thread = park_thread.unpark();
Ok((park_thread, unpark_thread, Default::default()))
}
}
@@ -98,10 +245,17 @@ cfg_not_process_driver! {
// ===== time driver =====
cfg_time! {
type TimeDriver = crate::park::either::Either<crate::time::driver::Driver<IoStack>, IoStack>;
#[derive(Debug)]
pub(crate) enum TimeDriver {
Enabled {
driver: crate::runtime::time::Driver,
handle: crate::runtime::time::Handle,
},
Disabled(IoStack),
}
pub(crate) type Clock = crate::time::Clock;
pub(crate) type TimeHandle = Option<crate::time::driver::Handle>;
pub(crate) type TimeHandle = Option<crate::runtime::time::Handle>;
fn create_clock(enable_pausing: bool, start_paused: bool) -> Clock {
crate::time::Clock::new(enable_pausing, start_paused)
@@ -112,15 +266,35 @@ cfg_time! {
io_stack: IoStack,
clock: Clock,
) -> (TimeDriver, TimeHandle) {
use crate::park::either::Either;
if enable {
let driver = crate::time::driver::Driver::new(io_stack, clock);
let handle = driver.handle();
let (driver, handle) = crate::runtime::time::Driver::new(io_stack, clock);
(Either::A(driver), Some(handle))
(TimeDriver::Enabled { driver, handle: handle.clone() }, Some(handle))
} else {
(Either::B(io_stack), None)
(TimeDriver::Disabled(io_stack), None)
}
}
impl TimeDriver {
pub(crate) fn park(&mut self) {
match self {
TimeDriver::Enabled { driver, handle } => driver.park(handle),
TimeDriver::Disabled(v) => v.park(),
}
}
pub(crate) fn park_timeout(&mut self, duration: Duration) {
match self {
TimeDriver::Enabled { driver, handle } => driver.park_timeout(handle, duration),
TimeDriver::Disabled(v) => v.park_timeout(duration),
}
}
pub(crate) fn shutdown(&mut self) {
match self {
TimeDriver::Enabled { driver, handle } => driver.shutdown(handle),
TimeDriver::Disabled(v) => v.shutdown(),
}
}
}
}
@@ -143,66 +317,3 @@ cfg_not_time! {
(io_stack, ())
}
}
// ===== runtime driver =====
#[derive(Debug)]
pub(crate) struct Driver {
inner: TimeDriver,
}
pub(crate) struct Resources {
pub(crate) io_handle: IoHandle,
pub(crate) signal_handle: SignalHandle,
pub(crate) time_handle: TimeHandle,
pub(crate) clock: Clock,
}
pub(crate) struct Cfg {
pub(crate) enable_io: bool,
pub(crate) enable_time: bool,
pub(crate) enable_pause_time: bool,
pub(crate) start_paused: bool,
}
impl Driver {
pub(crate) fn new(cfg: Cfg) -> io::Result<(Self, Resources)> {
let (io_stack, io_handle, signal_handle) = create_io_stack(cfg.enable_io)?;
let clock = create_clock(cfg.enable_pause_time, cfg.start_paused);
let (time_driver, time_handle) =
create_time_driver(cfg.enable_time, io_stack, clock.clone());
Ok((
Self { inner: time_driver },
Resources {
io_handle,
signal_handle,
time_handle,
clock,
},
))
}
}
impl Park for Driver {
type Unpark = <TimeDriver as Park>::Unpark;
type Error = <TimeDriver as Park>::Error;
fn unpark(&self) -> Self::Unpark {
self.inner.unpark()
}
fn park(&mut self) -> Result<(), Self::Error> {
self.inner.park()
}
fn park_timeout(&mut self, duration: Duration) -> Result<(), Self::Error> {
self.inner.park_timeout(duration)
}
fn shutdown(&mut self) {
self.inner.shutdown()
}
}
+6 -7
View File
@@ -25,8 +25,6 @@ pub(crate) struct Enter {
}
cfg_rt! {
use crate::park::thread::ParkError;
use std::time::Duration;
/// Marks the current thread as being within the dynamic extent of an
@@ -139,10 +137,12 @@ cfg_rt_multi_thread! {
}
cfg_rt! {
use crate::loom::thread::AccessError;
impl Enter {
/// Blocks the thread on the specified future, returning the value with
/// which that future completes.
pub(crate) fn block_on<F>(&mut self, f: F) -> Result<F::Output, ParkError>
pub(crate) fn block_on<F>(&mut self, f: F) -> Result<F::Output, AccessError>
where
F: std::future::Future,
{
@@ -156,18 +156,17 @@ cfg_rt! {
///
/// If the future completes before `timeout`, the result is returned. If
/// `timeout` elapses, then `Err` is returned.
pub(crate) fn block_on_timeout<F>(&mut self, f: F, timeout: Duration) -> Result<F::Output, ParkError>
pub(crate) fn block_on_timeout<F>(&mut self, f: F, timeout: Duration) -> Result<F::Output, ()>
where
F: std::future::Future,
{
use crate::park::Park;
use crate::park::thread::CachedParkThread;
use std::task::Context;
use std::task::Poll::Ready;
use std::time::Instant;
let mut park = CachedParkThread::new();
let waker = park.get_unpark()?.into_waker();
let waker = park.waker().map_err(|_| ())?;
let mut cx = Context::from_waker(&waker);
pin!(f);
@@ -184,7 +183,7 @@ cfg_rt! {
return Err(());
}
park.park_timeout(when - now)?;
park.park_timeout(when - now);
}
}
}
+12 -155
View File
@@ -1,11 +1,4 @@
use crate::runtime::blocking::{BlockingTask, NoopSchedule};
use crate::runtime::task::{self, JoinHandle};
use crate::runtime::{blocking, context, driver, Spawner};
use crate::util::error::{CONTEXT_MISSING_ERROR, THREAD_LOCAL_DESTROYED_ERROR};
use std::future::Future;
use std::marker::PhantomData;
use std::{error, fmt};
use crate::runtime::scheduler;
/// Handle to the runtime.
///
@@ -14,47 +7,19 @@ use std::{error, fmt};
///
/// [`Runtime::handle`]: crate::runtime::Runtime::handle()
#[derive(Debug, Clone)]
// When the `rt` feature is *not* enabled, this type is still defined, but not
// included in the public API.
pub struct Handle {
pub(super) spawner: Spawner,
pub(crate) inner: scheduler::Handle,
}
/// All internal handles that are *not* the scheduler's spawner.
#[derive(Debug)]
pub(crate) struct HandleInner {
/// Handles to the I/O drivers
#[cfg_attr(
not(any(feature = "net", feature = "process", all(unix, feature = "signal"))),
allow(dead_code)
)]
pub(super) io_handle: driver::IoHandle,
use crate::runtime::context;
use crate::runtime::task::JoinHandle;
use crate::util::error::{CONTEXT_MISSING_ERROR, THREAD_LOCAL_DESTROYED_ERROR};
/// Handles to the signal drivers
#[cfg_attr(
any(
loom,
not(all(unix, feature = "signal")),
not(all(unix, feature = "process")),
),
allow(dead_code)
)]
pub(super) signal_handle: driver::SignalHandle,
/// Handles to the time drivers
#[cfg_attr(not(feature = "time"), allow(dead_code))]
pub(super) time_handle: driver::TimeHandle,
/// Source of `Instant::now()`
#[cfg_attr(not(all(feature = "time", feature = "test-util")), allow(dead_code))]
pub(super) clock: driver::Clock,
/// Blocking pool spawner
pub(super) blocking_spawner: blocking::Spawner,
}
/// Create a new runtime handle.
pub(crate) trait ToHandle {
fn to_handle(&self) -> Handle;
}
use std::future::Future;
use std::marker::PhantomData;
use std::{error, fmt};
/// Runtime context guard.
///
@@ -204,11 +169,7 @@ impl Handle {
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
self.as_inner().spawn_blocking(self, func)
}
pub(crate) fn as_inner(&self) -> &HandleInner {
self.spawner.as_handle_inner()
self.inner.blocking_spawner().spawn_blocking(self, func)
}
/// Runs a future to completion on this `Handle`'s associated `Runtime`.
@@ -308,17 +269,7 @@ impl Handle {
let id = crate::runtime::task::Id::next();
#[cfg(all(tokio_unstable, feature = "tracing"))]
let future = crate::util::trace::task(future, "task", _name, id.as_u64());
self.spawner.spawn(future, id)
}
pub(crate) fn shutdown(mut self) {
self.spawner.shutdown();
}
}
impl ToHandle for Handle {
fn to_handle(&self) -> Handle {
self.clone()
self.inner.spawn(future, id)
}
}
@@ -334,100 +285,6 @@ cfg_metrics! {
}
}
impl HandleInner {
#[track_caller]
pub(crate) fn spawn_blocking<F, R>(&self, rt: &dyn ToHandle, func: F) -> JoinHandle<R>
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let (join_handle, _was_spawned) = if cfg!(debug_assertions)
&& std::mem::size_of::<F>() > 2048
{
self.spawn_blocking_inner(Box::new(func), blocking::Mandatory::NonMandatory, None, rt)
} else {
self.spawn_blocking_inner(func, blocking::Mandatory::NonMandatory, None, rt)
};
join_handle
}
cfg_fs! {
#[track_caller]
#[cfg_attr(any(
all(loom, not(test)), // the function is covered by loom tests
test
), allow(dead_code))]
pub(crate) fn spawn_mandatory_blocking<F, R>(&self, rt: &dyn ToHandle, func: F) -> Option<JoinHandle<R>>
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let (join_handle, was_spawned) = if cfg!(debug_assertions) && std::mem::size_of::<F>() > 2048 {
self.spawn_blocking_inner(
Box::new(func),
blocking::Mandatory::Mandatory,
None,
rt,
)
} else {
self.spawn_blocking_inner(
func,
blocking::Mandatory::Mandatory,
None,
rt,
)
};
if was_spawned {
Some(join_handle)
} else {
None
}
}
}
#[track_caller]
pub(crate) fn spawn_blocking_inner<F, R>(
&self,
func: F,
is_mandatory: blocking::Mandatory,
name: Option<&str>,
rt: &dyn ToHandle,
) -> (JoinHandle<R>, bool)
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
let fut = BlockingTask::new(func);
let id = super::task::Id::next();
#[cfg(all(tokio_unstable, feature = "tracing"))]
let fut = {
use tracing::Instrument;
let location = std::panic::Location::caller();
let span = tracing::trace_span!(
target: "tokio::task::blocking",
"runtime.spawn",
kind = %"blocking",
task.name = %name.unwrap_or_default(),
task.id = id.as_u64(),
"fn" = %std::any::type_name::<F>(),
spawn.location = %format_args!("{}:{}:{}", location.file(), location.line(), location.column()),
);
fut.instrument(span)
};
#[cfg(not(all(tokio_unstable, feature = "tracing")))]
let _ = name;
let (task, handle) = task::unowned(fut, NoopSchedule, id);
let spawned = self
.blocking_spawner
.spawn(blocking::Task::new(task, is_mandatory), rt);
(handle, spawned.is_ok())
}
}
/// Error returned by `try_current` when no Runtime has been started
#[derive(Debug)]
pub struct TryCurrentError {
@@ -1,13 +1,5 @@
#![cfg_attr(not(feature = "rt"), allow(dead_code))]
mod interest;
#[allow(unreachable_pub)]
pub use interest::Interest;
mod ready;
#[allow(unreachable_pub)]
pub use ready::Ready;
mod registration;
pub(crate) use registration::Registration;
@@ -16,7 +8,8 @@ use scheduled_io::ScheduledIo;
mod metrics;
use crate::park::{Park, Unpark};
use crate::io::interest::Interest;
use crate::io::ready::Ready;
use crate::util::slab::{self, Slab};
use crate::{loom::sync::RwLock, util::bit};
@@ -72,6 +65,8 @@ pub(super) struct Inner {
io_dispatch: RwLock<IoDispatcher>,
/// Used to wake up the reactor from a call to `turn`.
/// Not supported on Wasi due to lack of threading support.
#[cfg(not(tokio_wasi))]
waker: mio::Waker,
metrics: IoDriverMetrics,
@@ -115,6 +110,7 @@ impl Driver {
/// creation.
pub(crate) fn new() -> io::Result<Driver> {
let poll = mio::Poll::new()?;
#[cfg(not(tokio_wasi))]
let waker = mio::Waker::new(poll.registry(), TOKEN_WAKEUP)?;
let registry = poll.registry().try_clone()?;
@@ -129,6 +125,7 @@ impl Driver {
inner: Arc::new(Inner {
registry,
io_dispatch: RwLock::new(IoDispatcher::new(allocator)),
#[cfg(not(tokio_wasi))]
waker,
metrics: IoDriverMetrics::default(),
}),
@@ -147,7 +144,26 @@ impl Driver {
}
}
fn turn(&mut self, max_wait: Option<Duration>) -> io::Result<()> {
pub(crate) fn park(&mut self) {
self.turn(None);
}
pub(crate) fn park_timeout(&mut self, duration: Duration) {
self.turn(Some(duration));
}
pub(crate) fn shutdown(&mut self) {
if self.inner.shutdown() {
self.resources.for_each(|io| {
// If a task is waiting on the I/O resource, notify it. The task
// will then attempt to use the I/O resource and fail due to the
// driver being shutdown. And shutdown will clear all wakers.
io.shutdown();
});
}
}
fn turn(&mut self, max_wait: Option<Duration>) {
// How often to call `compact()` on the resource slab
const COMPACT_INTERVAL: u8 = 255;
@@ -164,7 +180,12 @@ impl Driver {
match self.poll.poll(&mut events, max_wait) {
Ok(_) => {}
Err(ref e) if e.kind() == io::ErrorKind::Interrupted => {}
Err(e) => return Err(e),
#[cfg(tokio_wasi)]
Err(e) if e.kind() == io::ErrorKind::InvalidInput => {
// In case of wasm32_wasi this error happens, when trying to poll without subscriptions
// just return from the park, as there would be nothing, which wakes us up.
}
Err(e) => panic!("unexpected error when polling the I/O driver: {:?}", e),
}
// Process all the events that came in, dispatching appropriately
@@ -181,8 +202,6 @@ impl Driver {
self.inner.metrics.incr_ready_count_by(ready_count);
self.events = Some(events);
Ok(())
}
fn dispatch(&mut self, token: mio::Token, ready: Ready) {
@@ -206,42 +225,6 @@ impl Driver {
}
}
impl Drop for Driver {
fn drop(&mut self) {
self.shutdown();
}
}
impl Park for Driver {
type Unpark = Handle;
type Error = io::Error;
fn unpark(&self) -> Self::Unpark {
self.handle()
}
fn park(&mut self) -> io::Result<()> {
self.turn(None)?;
Ok(())
}
fn park_timeout(&mut self, duration: Duration) -> io::Result<()> {
self.turn(Some(duration))?;
Ok(())
}
fn shutdown(&mut self) {
if self.inner.shutdown() {
self.resources.for_each(|io| {
// If a task is waiting on the I/O resource, notify it. The task
// will then attempt to use the I/O resource and fail due to the
// driver being shutdown. And shutdown will clear all wakers.
io.shutdown();
});
}
}
}
impl fmt::Debug for Driver {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Driver")
@@ -259,7 +242,7 @@ cfg_rt! {
/// This function panics if there is no current reactor set and `rt` feature
/// flag is not enabled.
#[track_caller]
pub(super) fn current() -> Self {
pub(crate) fn current() -> Self {
crate::runtime::context::io_handle().expect("A Tokio 1.x context was found, but IO is disabled. Call `enable_io` on the runtime builder to enable IO.")
}
}
@@ -273,7 +256,8 @@ cfg_not_rt! {
///
/// This function panics if there is no current reactor set, or if the `rt`
/// feature flag is not enabled.
pub(super) fn current() -> Self {
#[track_caller]
pub(crate) fn current() -> Self {
panic!("{}", crate::util::error::CONTEXT_MISSING_ERROR)
}
}
@@ -299,17 +283,12 @@ impl Handle {
/// after this method has been called. If the reactor is not currently
/// blocked in `turn`, then the next call to `turn` will not block and
/// return immediately.
fn wakeup(&self) {
pub(crate) fn unpark(&self) {
#[cfg(not(tokio_wasi))]
self.inner.waker.wake().expect("failed to wake I/O driver");
}
}
impl Unpark for Handle {
fn unpark(&self) {
self.wakeup();
}
}
impl fmt::Debug for Handle {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Handle")
@@ -1,6 +1,7 @@
#![cfg_attr(not(feature = "net"), allow(dead_code))]
use crate::io::driver::{Direction, Handle, Interest, ReadyEvent, ScheduledIo};
use crate::io::interest::Interest;
use crate::runtime::io::{Direction, Handle, ReadyEvent, ScheduledIo};
use crate::util::slab;
use mio::event::Source;
@@ -56,10 +57,8 @@ unsafe impl Sync for Registration {}
// ===== impl Registration =====
impl Registration {
/// Registers the I/O resource with the default reactor, for a specific
/// `Interest`. `new_with_interest` should be used over `new` when you need
/// control over the readiness state, such as when a file descriptor only
/// allows reads. This does not add `hup` or `error` so if you are
/// Registers the I/O resource with the reactor for the provided handle, for
/// a specific `Interest`. This does not add `hup` or `error` so if you are
/// interested in those states, you will need to add them to the readiness
/// state passed to this function.
///
@@ -115,6 +114,7 @@ impl Registration {
// Uses the poll path, requiring the caller to ensure mutual exclusion for
// correctness. Only the last task to call this function is notified.
#[cfg(not(tokio_wasi))]
pub(crate) fn poll_read_io<R>(
&self,
cx: &mut Context<'_>,

Some files were not shown because too many files have changed in this diff Show More