Compare commits

..
215 Commits
Author SHA1 Message Date
Minh Vu d6bf379eba sync: avoid atomic reader count in broadcast slots (#8357) 2026-09-08 13:11:14 +02:00
Tim Vilgot Mikael Fredenberg 6bea73e4c1 time: clean up Instant overflow prevention (#8128) 2026-09-08 12:31:49 +02:00
Yizhou Feng 1309451987 runtime: fix idle bookkeeping when a task dump wakes parked workers (#8372)
Requesting a task dump calls `notify_all`, which unparks every worker
thread directly rather than going through `Idle`. A worker that was
parked at that point is woken to be traced while the scheduler still
counts it as a sleeper.

`transition_from_parked` did not account for that. With no tasks
queued, the worker saw itself as still parked in `Idle` and returned
without performing the logical unpark. Once tracing finished and the
worker parked again, `transition_worker_to_parked` pushed it onto the
sleeper list a second time and decremented `num_unparked` a second
time, underflowing it. From then on `notify_should_wakeup` was always
false, so no worker was ever notified of newly spawned work and the
runtime silently stopped running tasks submitted from outside of it.

Treat a worker woken for tracing like a worker woken with tasks queued
and perform the logical unpark. This mirrors `transition_to_parked`,
which already refuses to park a worker that is about to be traced.
2026-09-08 09:27:02 +00:00
Breeze 13d4800fb7 time: avoid interval deadline overflow (#8385) 2026-09-07 18:19:37 +02:00
Minh Vu 483e4b9ee7 stream: handle overflowing timer durations (#8354) 2026-09-07 18:17:16 +02:00
Alex Gaynor 6b3c90cc58 rt: move the multi-thread inject queue to its own mutex (#8382)
This migrates the inject queue to its own lock, instead of sharing the
scheduler's `synced` mutex with the idle worker state. They originally
shared a lock as part of #5747 and #5754, but the unified critical
sections that was intended to enable were removed with the alternative
multi-threaded scheduler in #7275.

This is a first step towards #7973.
2026-09-07 18:00:23 +02:00
Revantark bbb5076068 runtime: fix spawn_blocking hang when only scheduler workers exist (#8408) 2026-09-07 12:24:52 +00:00
Alice Ryhl 855e5b8830 tracing: serialize tracing_task tests (#8428) 2026-09-07 14:06:54 +02:00
Alice Ryhl 0d38ab5480 sync: make tracing_sync tests synchronous (#8427) 2026-09-07 12:39:32 +02:00
Alice Ryhl bd8f8f2028 net: remove leftover debug loop counter in try_read_buf (#8426) 2026-09-07 11:51:07 +02:00
Alice Ryhl b9c075388e Merge 'tokio-1.51.x' into 'master' (#8425) 2026-09-07 07:29:36 +00:00
Alice Ryhl e4be0cfe24 Revert "ci: pin cargo-fuzz to 0.13.1 for check-fuzzing (#8403)" (#8425)
This reverts commit 16a6a239b03a74340d859ebc5740449492163b2f.
2026-09-07 07:28:30 +00:00
Alice Ryhl e456a12f89 Revert "tests: mark failing taskdump tests as #[ignore] (#8403)" (#8425)
This reverts commit 66f836e61a.
2026-09-07 07:28:24 +00:00
Alice Ryhl b62fa166aa Revert "ci: use Rust 1.98 for wasm32-wasip2 (#8412)" (#8425)
This reverts commit 3e6eb7d33a.
2026-09-07 07:28:19 +00:00
Alice Ryhl 3e6eb7d33a ci: use Rust 1.98 for wasm32-wasip2 (#8412) 2026-09-07 06:25:02 +00:00
Kiryl Mialeshka dde3f86922 time: handle wrapped top-level timer-wheel slots (#8334) 2026-09-07 09:43:41 +08:00
Phil Phaulerandphilphauler a316820fa4 tests: handle EPERM in io_uring_supported helper (#8416)
Co-authored-by: philphauler <[email protected]>
2026-09-07 00:05:49 +08:00
Phil Phaulerandphilphauler 7d0d729d8f sync: clarify panic behavior of broadcast::channel() (#8420)
Co-authored-by: philphauler <[email protected]>
2026-09-06 20:20:39 +08:00
Phil Phaulerandphilphauler 0069aef281 io: retry ErrorKind::Interrupted in read_exact (#8417)
Co-authored-by: philphauler <[email protected]>
2026-09-06 20:17:28 +08:00
Phil Phaulerandphilphauler 787697f76f runtime: fix misleading signal driver panic message (#8419)
Co-authored-by: philphauler <[email protected]>
2026-09-06 20:02:36 +08:00
Alice Ryhl 26e6ee89c0 Merge 'tokio-1.47.x' into 'tokio-1.51.x' (#8412) 2026-09-05 09:16:06 +00:00
Guy Bedford 060cc4e46a wasm: support the wasm32-unknown-emscripten target (#8281) 2026-09-05 10:35:22 +02:00
rifuki 4917528a97 rt: make enable_io available whenever rt is enabled (#8387) 2026-09-05 10:03:46 +02:00
Paco Cartones dc9f6ede19 runtime: re-enable ignored io shutdown tests in rt_handle_block_on (#8404)
These tests were added in #3569 (2021) but immediately marked
`#[ignore]` because of a then-known bug where shutting down the io
driver while concurrently registering new resources was unsound.

That bug class was resolved by the io driver rewrite in #5833, which
replaced the slab-based registration with a `RegistrationSet` guarded by
a mutex-protected `Synced`. Allocating a new registration now checks
`is_shutdown` under the lock and returns a proper error instead of
racing, so binding a resource after (or concurrently with) runtime
shutdown fails deterministically.

Re-enable the 7 ignored tests (across the 3 scheduler configurations).
Five of them pass unchanged. The two `unix_listener_shutdown_after_*`
tests asserted the old `"reactor gone"` message; #5833 unified that path
onto the single `RUNTIME_SHUTTING_DOWN_ERROR` string, so update those two
assertions to the current message.
2026-09-04 15:59:55 +00:00
Dylan Pulver bb5a0fce23 time: return the earliest key from DelayQueue::peek (#8402) 2026-09-04 14:29:10 +02:00
Dahale Aditya Dnyaneshwar 103d29f808 sync: use usize for permit count in SemaphorePermit (#8405) 2026-09-04 14:05:03 +02:00
rifuki 89f4d133ba io: read lines as bytes so a partial line survives an I/O error (#8400)
`Lines` accumulated into a `String`, so a partial line interrupted by an
I/O error could not be kept when it ended mid multi-byte character. The
next poll then tripped `debug_assert!(output.is_empty())`, or
underflowed `vector.len() - num_bytes_read` in `put_back_original_data`,
which panics in release builds too on the `.expect` below it.

`Lines` now holds a single `buf: Vec<u8>` and calls
`read_until_internal` directly the way `Split` does, converting to
`String` only once a whole line is available.

An `InvalidData` error now carries the utf-8 error itself rather than a
fixed string. `Lines` owns the line, so it hands over the whole
`FromUtf8Error` and the caller can still recover the bytes; `read_line`
and `read_to_string` have to put those bytes back into the caller's
`String`, so they carry `Utf8Error` instead.

`read_line_internal` has no callers outside `read_line.rs` and is now
private.
2026-09-04 11:33:54 +00:00
Alice Ryhl 66f836e61a tests: mark failing taskdump tests as #[ignore] (#8403) 2026-09-03 14:31:54 +00:00
Alice Ryhl 16a6a2390c ci: pin cargo-fuzz to 0.13.1 for check-fuzzing (#8403) 2026-09-03 13:58:02 +00:00
Joel Dice 705989c98b ci: pin Wasmtime version(s) in CI (#8314)
Per https://github.com/bytecodealliance/wasmtime/pull/13558, Wasmtime v46.0.1
was the last release to support `wasm32-wasip1-threads`, so we use that for the
WASIp1 testing.

For WASIp2, we should be able to use any recent version of Wasmtime, but we pin
to a specific version anyway to avoid surprises.

(cherry picked from commit 5760ccdc37)
2026-09-03 13:58:02 +00:00
Carl Lerche 231a8faf20 tokio: cargo check -p --all-features without tokio_unstable (#8401) 2026-09-03 13:25:46 +00:00
cui fliter 192ef9a1c1 io: preserve the full ScheduledIo tick in readiness events (#8331) 2026-09-03 13:10:43 +00:00
Rachit2323 6858348ad2 io: panic when in-memory pipe is created with zero capacity (#8397) 2026-09-03 14:53:56 +02:00
fly1d 9c880b082b ci: skip workspace semver on release pushes (#8390)
`github.event.pull_request.base.ref` is empty for push events, so the workspace semver check runs on `tokio-1.*.x` branches. Also check `github.ref_name` to preserve the intended release-branch exclusion while leaving pull request behavior unchanged.

Fixes: #8389
2026-08-27 11:14:39 +02:00
K-tecchan ea91b33ca5 rt: rename shared to sharded in internal ShardedList (#8364) 2026-08-20 08:57:24 +02:00
Rachit2323 625954f365 sync: add blocking_acquire methods to Semaphore (#8269) 2026-08-11 09:51:50 +02:00
Minh Vu af93763009 io: handle empty vectored writes in simplex (#8353) 2026-08-10 12:19:30 +02:00
Dhruv Vaishnav 43e6ef50e4 codec: support byte slices in LengthDelimitedCodec (#8355) 2026-08-10 12:18:22 +02:00
Alex Gaynor 8b13642a1f runtime: add an opt-in sharded spawn_blocking queue (#8337)
Re-lands the sharded queue from #7757 (reverted due to #8056), disabled by
default. Opt in via the unstable `Builder::enable_sharded_blocking_queue` or
the `TOKIO_UNSTABLE_SHARDED_BLOCKING_QUEUE` environment variable.
2026-08-09 16:35:12 +00:00
Rachit2323 d4569bb550 stream: add inner stream accessors to adaptors (#8272) 2026-08-09 18:11:51 +02:00
soreavis d4fb4bb9e6 sync: use acquire/release orderings in Notify (#8325)
Every atomic operation on `Notify::state` used `SeqCst`. Nothing needs
the global total order: every lock-avoidance decision is made by an RMW,
which always reads the latest value in that atomic's modification order,
and every stale load is re-validated either by a following RMW or by a
re-load under the `waiters` mutex.

What the `state` orderings do have to carry is the happens-before for
data published before `notify_one`, consumed through the permit
compare-exchange, and for data published before `notify_waiters`,
consumed through the counter check. Acquire/release on `state` provides
both. The waiter list is ordered by the mutex, and `AtomicNotification`
by its own release/acquire pair, so neither depends on these orderings.

Loads become `Acquire`, stores `Release`, compare-exchange
`(AcqRel, Acquire)`, and the `notify_waiters` counter increment
`AcqRel`. All nineteen sites are converted, so no `SeqCst` is left
alongside weaker orderings.

Fixes: #6266
2026-08-09 18:06:29 +02:00
Tim Vilgot Mikael Fredenberg 011f7f4b47 time: simplify wheel constants (#8335) 2026-08-09 17:42:42 +02:00
GuTS805 b6ed00435d codec: fix broken length_delimited builder doc examples (#8350)
The Builder::new, new_codec, and new_read doc examples combined
length_adjustment(0) with num_skip(0). This leaves the length
header bytes in the buffer without accounting for them, so decoding
returns a frame that includes the raw header at the front and is
short at the back, corrupting the following frame.

Drop num_skip(0) so the default (skip the header) behavior applies,
and turn all three examples into executable doctests that perform a
real encode/decode round-trip and assert on the payload, so this
class of bug is caught automatically going forward.

Fixes: #8348
2026-08-09 17:31:23 +02:00
Fodesu e10d614cb9 task: use LocalRuntime in LocalPoolHandle (#7852) 2026-08-09 15:24:13 +00:00
Nikolas Kilian 83e9c57cee fs: restore File internal state when op fails (#8291) 2026-08-09 23:06:54 +08:00
Rachit2323 ddc60948ab stream: add peek_mut, poll_peek to Peekable (#8262) 2026-08-09 16:54:18 +02:00
MildlyMeticulous d87d860cc8 codec: optimize buffer reserve for LengthDelimitedCodec::encode (#8333)
The reservation used `n`, the length after `length_adjustment` has been
applied, but the encoder goes on to write `length_field_len` bytes of
header plus `data.len()` bytes of payload. A positive `length_adjustment`
therefore under-reserved by exactly the adjustment on every frame, and a
negative one over-reserved, so the reservation did not match the write in
either of the adjusted configurations shown in the module docs.
2026-08-09 16:47:16 +02:00
Minh Vu ed25636141 time: wake DelayQueue when cleared (#8320) 2026-08-09 16:45:35 +02:00
Minh Vu 7f86b2ace5 codec: clamp runtime frame length to field width (#8275) 2026-08-09 16:39:18 +02:00
Russell Cohen 6b62ac48ed runtime: expose schedule latency in task hooks (#8282)
Add an explicit tracking knob and expose the sampled task schedule
latency through TaskMeta. Reuse the histogram poll timestamp where
possible and preserve grouped histogram accounting for LIFO polls.

Document activation and interval semantics and cover current-thread,
multi-thread, LIFO, disabled, and non-poll callback behavior.
2026-08-09 14:38:39 +00:00
Thomas Zander ecd621dd2c net: support ucred.rs on Fuchsia (#8257)
In order to compile successfully with the net feature on Fuchsia, a cfg
attribute for Fuchsia in `impl_noproc` is required.
2026-08-07 17:42:19 +02:00
kai-xlr 4f7b8d6239 fs: document cancellation behavior of tokio::fs (#8265) 2026-08-07 22:14:05 +08:00
MAAZIZ Adel Ayoub a5b2d1ff69 tokio-util: stop polling StreamReader after EOF (#8332) 2026-08-07 21:59:51 +08:00
Tim Vilgot Mikael Fredenberg cc8c053421 tokio: simplify Option handling with idiomatic combinators (#8336) 2026-08-07 21:18:39 +08:00
Minh Vu dd344a550c io: complete zero-length memory stream operations (#8323) 2026-08-06 12:01:12 +02:00
Alex Gaynor 108d6d3dc0 runtime: refactor the spawn blocking queue to make adding a new sharded implementation easy (#8135) 2026-07-31 22:27:54 +02:00
Motoyuki Kimura adc2ae7af2 rt: add error documentation for runtime::Builder (#8297) 2026-07-30 12:29:09 +02:00
Geoffry Song dc48a05132 runtime: move block_in_place setup into a non-generic function (#8315) 2026-07-30 11:05:04 +02:00
Dirkjan Ochtman 4308c74a65 chore: prepare tokio-macros v2.7.2 (#8330) 2026-07-29 15:14:27 +02:00
Joel Dice 1a2dbbaa21 net: enable various tests for WASI (#8313)
These tests were temporarily disabled until required `wasi-libc` fixes made
their way into a Rust release.  Now that that has happened, we can enable them.

Note that `send_to_recv_closed_returns_err` remains disabled for a bit longer.
The applicable `wasi-libc` bug was masking a separate bug in Wasmtime, fixed
[here](https://github.com/bytecodealliance/wasmtime/pull/13933).  Once that fix
makes its way into a release (presumably v48.0.0), we'll finally be able enable
that test, and that should be the last of the
temporarily-disabled-on-WASI-due-to-bugs tests.
2026-07-28 11:38:34 +02:00
K-tecchan df28ffe61d rt: correct tick to tick_op in set_readiness internal docs (#8322) 2026-07-27 08:11:09 +02:00
Paolo Barbolini 6a058770e9 macros: upgrade syn to v3 (#8304) 2026-07-26 11:26:19 +00:00
Alice Ryhl 818e2dd866 time: adjust tests::time_rt::tickspace (#8318) 2026-07-25 22:09:09 +02:00
Tim Vilgot Mikael Fredenberg 460dc16d19 tokio-test: fix semicolon_in_expressions_from_macros lint (#8317) 2026-07-25 19:39:39 +02:00
dependabot[bot]andMattia Pitossi 2120bee47f ci: bump actions/labeler from 6 to 7 (#8316)
Bumps [actions/labeler](https://github.com/actions/labeler) from 6 to 7.
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](https://github.com/actions/labeler/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mattia Pitossi <[email protected]>
2026-07-23 06:31:42 +00:00
kai-xlr d1d639fbe8 docs: add step-by-step workflow in contributing guidelines (#3865) (#8306) 2026-07-23 08:00:51 +02:00
Joel Dice 5760ccdc37 ci: pin Wasmtime version(s) in CI (#8314)
Per https://github.com/bytecodealliance/wasmtime/pull/13558, Wasmtime v46.0.1
was the last release to support `wasm32-wasip1-threads`, so we use that for the
WASIp1 testing.

For WASIp2, we should be able to use any recent version of Wasmtime, but we pin
to a specific version anyway to avoid surprises.
2026-07-23 07:57:22 +02:00
Alice Ryhl bc0933ccff chore: prepare tokio-stream v0.1.19 (#8310) 2026-07-22 09:49:29 +02:00
Alex Touchet e3786d0090 readme: remove obsolete TokioConf notices (#8311) 2026-07-22 07:36:16 +00:00
Alice Ryhl f2189d3bd6 chore: prepare tokio-util v0.7.19 (#8309) 2026-07-21 14:09:36 +02:00
Joel Dice 52f2745c18 net: re-enable tcp_stream::try_read_buf test for WASI (#8305) 2026-07-21 10:12:54 +02:00
Mattia Pitossi ac6869a431 rt: remove unstable cfgs leftovers after local runtime stabilization (#8298) 2026-07-20 20:12:30 +02:00
ADD-SP 75fef53d0a chore: prepare Tokio v1.53.1 (#8303) 2026-07-20 19:05:08 +02:00
Jens Holdgaard Pedersen ae9d011213 signal: restore MSRV by removing OnceLock::wait from the Windows handler (#8300)
OnceLock::wait was stabilized in Rust 1.86, so its use in the Windows
console ctrl handler broke tokio's declared rust-version of 1.71 on
windows targets in 1.53.0 (any cargo check with a 1.71..1.86 toolchain
fails with E0599).

The wait existed only because SetConsoleCtrlHandler was called inside
REGISTRY's get_or_init closure, i.e. before the OnceLock was actually
initialized, leaving a window where an invoked handler could observe an
uninitialized REGISTRY. Initialize the registry first and register the
OS handler afterwards (exactly once, through a second OnceLock that
also caches a registration failure so every subsequent call reports the
same error, matching the previous behavior). The handler can then rely
on plain get(): registration happens-after initialization, so an
invoked handler always finds the registry.

Verified with cargo +1.71 check -p tokio --features full
--target x86_64-pc-windows-msvc (fails with the reported E0599 before
this change, clean after) and --all-targets on stable for the same
target.

Fixes #8299
2026-07-20 19:04:08 +02:00
ADD-SP eb4988dc2e time: fix the loom test of the race between cancellation/insertion (#8302) 2026-07-20 19:10:31 +08:00
Alexander KireyevandADD-SP 91d3b4c0bc time: fix alt timer cancellation and insertion race (#8252)
Co-authored-by: ADD-SP <[email protected]>
2026-07-20 18:32:46 +08:00
K-tecchan a46338401b runtime: remove dead link definition in Runtime::block_on (#8301) 2026-07-20 13:47:27 +08:00
Alice Ryhl be689a35f5 chore: prepare Tokio v1.53.0 (#8294) 2026-07-17 10:12:50 +02:00
Alice Ryhl 50f76c71ec chore: prepare tokio-macros v2.7.1 (#8295) 2026-07-17 10:12:38 +02:00
Alice Ryhl f61fccad3c Merge 'tokio-1.52.4' into 'master' (#8290) 2026-07-16 13:27:30 +00:00
Alice Ryhl efdba5fcf0 chore: prepare Tokio v1.52.4 (#8289) 2026-07-16 15:23:48 +02:00
Alice Ryhl b0ba02e755 Merge 'tokio-1.51.4' into 'tokio-1.52.x' (#8288) 2026-07-16 12:18:21 +00:00
Alice Ryhl 7bcd2d343d taskdump: remove crate disambiguators from output (#8288) 2026-07-16 12:17:13 +00:00
Alice Ryhl f84b209126 chore: prepare Tokio v1.51.4 (#8286) 2026-07-16 13:59:24 +02:00
Amey Pawar eacb98e189 runtime: don't skip the driver when before_park schedules work (#8222) 2026-07-16 13:16:47 +02:00
Minh Vu 5e16ee00fa task: avoid replacing the JoinQueue waker in try_join_next (#8279)
Do not poll pending join handles with a noop waker, since that can replace the waker registered by poll_join_next and leave its caller asleep.
2026-07-16 13:15:11 +02:00
cong-or 88212ab64a sync: document memory ordering guarantees for Semaphore (#8119) 2026-07-16 09:43:52 +02:00
kai-xlr 9cae638de6 examples: add UDP binding to all interfaces docs (#8283)
Closes #6737
2026-07-15 17:05:27 +00:00
Tobias Bucher 315a320e96 io: fix typo in SimplexStream docs (#8284) 2026-07-15 16:51:52 +00:00
MAAZIZ Adel Ayoub dac81bf8c8 sync: wake mpsc receiver when a queued reserve[_many] returns permits (#8260) 2026-07-14 09:56:18 +02:00
Minh Vu 145f124d98 ci: preserve QEMU exit status in uring kernel tests (#8271) 2026-07-14 09:52:32 +02:00
Minh Vu 6e8475dd60 time: wake DelayQueue after resetting to expired (#8274)
Resetting an item into the expired stack bypassed the existing Sleep reset
path, leaving a pending consumer asleep until an unrelated deadline.
2026-07-14 09:33:59 +02:00
Mattia Pitossi b6f30cae9a rt: flush CQE in case of CQE overflow (#8277) 2026-07-13 19:33:23 +08:00
Rachit2323 4638b65f9c stream: implement Peekable::size_hint (#8109) 2026-07-13 13:00:59 +02:00
kai-xlr fe258f5e6d net: use getpeereid for QNX peer credentials (#8270)
QNX (nto) does not provide LOCAL_PEEREID, which is used by the
impl_netbsd module. This causes a compilation failure on QNX.

Use getpeereid() instead, following the same pattern as impl_dragonfly
and impl_aix. This provides uid/gid but not pid, so UCred::pid()
returns None on QNX.
2026-07-13 12:36:10 +02:00
kai-xlr 33f46a5395 io: add warning about stdout reordering with multiple handles (#8276) 2026-07-12 21:12:48 +08:00
linkmauve 9c465e2f42 tokio-stream: Simplify TakeWhile with Option::filter() (#8268)
Option::filter() exists since Rust 1.27.0, and is much more readable
than the open coded variant that was there before, using
Option::and_then().

This has been found by clippy.
2026-07-10 21:33:24 +02:00
linkmauve d5950a8890 refactor: fix variable name typos (#8267) 2026-07-10 20:53:52 +02:00
linkmauve c793a631f7 fs: add safe impl From<OwnedFd> for File (#8266)
This impl was missing to be able to create a tokio::fs::File directly
from an OwnedFd, which can be done in a safe way.  Going through RawFd
required unsafe for the same operation.

And same for Windows using a OwnedHandle instead.
2026-07-10 20:02:38 +02:00
bfd4ddf597 docs: correct spelling typos in comments and doc strings (#8263)
Fix several spelling errors found in comments and documentation:
- mulithreading -> multithreading (tcp_shutdown.rs)
- succeded -> succeeded (signal/windows/sys.rs)
- implementor/implementors -> implementer/implementers (multiple files)

Co-authored-by: maxtaran2010 <[email protected]>
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
Co-authored-by: Alice Ryhl <[email protected]>
2026-07-10 11:03:18 +00:00
Alice Ryhl cd3bcca32b taskdump: remove crate disambiguators from output (#8264) 2026-07-10 12:09:45 +02:00
wanglei01 c4c6265a07 net: support Nuttx target os (#8259)
Signed-off-by: wanglei <[email protected]>
2026-07-07 16:01:24 +02:00
Dongpo Liu bb2815ae23 task: explain why yield_now defers its waker (#8254)
Since #5223, `yield_now` does not wake the task immediately. Instead,
the waker is handed to the scheduler via `context::defer`, which wakes
it only after running out of ready tasks and polling the IO/timer
driver. Add a comment explaining this, as the reasoning is not obvious
from the bare `context::defer` call.
2026-07-07 15:40:18 +02:00
Leo Blöcher e06f16259d sync: reset Chan::rx_waker in chan::Rx's Drop impl (#8095)
I recently fixed a memory leak in an application where tokio's RawTask
storage was being kept alive by a leaked Waker. The task itself was
polling an `mpsc::Receiver` before then being aborted. During cleanup,
all references to the RawTask were dropped, except for the one stored in
`mpsc::chan::Chan::rx_waker`. While the `Receiver` was dropped as part
of the task's future, one of the channel's `Sender`s was leaked outside
the task. This meant the `Chan` was never dropped and its `rx_waker`
contained the leaked Waker.

I fixed the leak by properly cleaning up the `Sender`, but I also think
keeping `rx_waker` around in this case is unnecessary. Once `chan::Rx`
is dropped, it can't be polled anymore, so waking up the registered task
will always be spurious.

The commit includes a regression test to illustrate the problem that is
fixed by removing the waker explicitly.
2026-07-06 14:15:51 +02:00
Minh Vu 962420a4f3 codec: document UdpFramed decoder errors (#8248) 2026-07-06 13:44:17 +02:00
c637f6e73d fs: implement rename using io-uring (#7800)
---------

Co-authored-by: Mattia Pitossi <[email protected]>
Co-authored-by: vrtgs <[email protected]>
Co-authored-by: Daksh <[email protected]>
2026-07-03 11:18:20 +02:00
MAAZIZ Adel Ayoub dd683aba3f io: do not treat zero-length reads as EOF in Chain (#8251) 2026-07-03 08:31:33 +02:00
yoda77777 9fe3c5619d sync: clarify broadcast lagging semantics and pin them with tests (#8239)
Expand module and API docs around RecvError::Lagged / TryRecvError::Lagged
so capacity rounding, miss counts, and post-lag resume behavior are explicit.
Add integration tests covering slow receivers within capacity, overflow,
per-receiver lag, async recv, and cursor advancement after Lagged.
2026-07-01 16:34:17 +00:00
ebubekir karaca 448d1227a1 io: add #[inline] to IO trait impls for in-memory types (#8242)
Add #[inline] hints to poll_read, poll_write, start_seek, poll_complete, poll_fill_buf, and consume implementations for in-memory types (&[u8], Vec<u8>, Cursor<T>).

These are small, leaf implementations that benefit from cross-crate inlining, enabling LLVM to optimize call sites in downstream crates (including bounds-check elision and dead-code path elimination).

Benchmarks show ~16% improvement for slice reads and ~20% for cursor writes.
2026-07-01 18:28:37 +02:00
Minh Vu 98104c36f1 stream: honor StreamMap::next_many limit (#8215) 2026-07-01 17:55:20 +02:00
WhySoBad 61aeb33f51 net: re-enable miri for tests with readable readiness for a socket after a short read (#8238) 2026-07-01 17:52:48 +02:00
MAAZIZ Adel Ayoub 7b354d22a9 stream: stop polling the underlying stream once map_while yields None (#8233) 2026-06-30 11:18:31 +08:00
Shuang Li 4bbd2f4d7c docs: fix missing brace in reactor-refactor.md (#8236) 2026-06-28 23:42:35 +03:00
Tim Vilgot Mikael Fredenberg 930ca7436b signal: merge windows statics (#8231) 2026-06-28 10:24:29 -07:00
46c830117b fs(tests): do not leak FDs with shutdown background (#8184)
---------

Co-authored-by: Martin Grigorov <[email protected]>
Co-authored-by: Mattia Pitossi <[email protected]>
2026-06-25 17:20:53 +02:00
dependabot[bot] 8f37021c16 ci: bump actions/checkout from 6 to 7 (#8229)
Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v6...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-25 09:25:48 +03:00
dependabot[bot] cdaa78fc37 ci: bump actions/cache from 5 to 6 (#8230)
Bumps [actions/cache](https://github.com/actions/cache) from 5 to 6.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-25 08:51:00 +03:00
Matteo Monti dba1f20585 sync: implement cancellation token getter for drop guard (#8226) 2026-06-24 15:16:23 +02:00
Nanasi 95b8895da8 stream: update coop handling for empty and once (#8227) 2026-06-24 12:38:36 +00:00
Giorgio Maria Federico Birnthaler 73e8937cc5 util: fix typo in blocking_check.rs comment (#8223)
Replace 'i dont' with 'i don't' in the cfg(not(unix)) stub of
'check_socket_for_blocking'. The line is a casual internal comment
about why WASI is not yet supported there; it just needs the missing
apostrophe.
2026-06-23 20:03:19 +00:00
Ebrahim Eldesoky 630ec12a4f stream: use cooperative budgeting in tokio_stream::iter (#8218) 2026-06-23 11:42:03 +00:00
GraymanandTim Vilgot Mikael Fredenberg 060f66c665 docs: clarify cancel safety wording (#8181)
Co-authored-by: Tim Vilgot Mikael Fredenberg <[email protected]>
2026-06-23 09:13:01 +00:00
Prashant Singh Chouhan a59f9a0a94 test: add Spawn::poll_until_idle (#8213) 2026-06-23 10:25:14 +02:00
Robert Holt aee321206b metrics: add task schedule latency metric (#7986) 2026-06-23 10:23:34 +02:00
Tim Vilgot Mikael Fredenberg 5f52f113d4 tracing: remove unnecessary span clone (#8126) 2026-06-23 07:51:55 +00:00
Minh Vu f59aae423e stream: fix overflow in StreamMap::size_hint (#8216) 2026-06-23 09:49:54 +02:00
Minh Vu dc3a883b99 examples: connect proxy upstream per client (#8217) 2026-06-23 09:49:33 +02:00
Patrick WehbeandPatrick Wehbe 66e29121b3 time: fix reversed poll order in timeout doc (#8214)
The timeout doc said the timeout is checked before polling the future,
but the impl polls the future first and only then checks the delay (see
the Future impl for Timeout, which calls me.value.poll before
poll_delay). Reverse the clause to match the implementation. This is
also what makes the rest of the sentence correct: the future can
complete and exceed the timeout without an error precisely because it is
polled before the timeout is checked.

Co-authored-by: Patrick Wehbe <[email protected]>
2026-06-19 22:32:30 +02:00
Tim Vilgot Mikael Fredenberg daa653d94f tokio: remove needless generic type on LinkedList (#8188) 2026-06-19 23:28:28 +08:00
Mahdi Ali-RaihanandVrtgs 7892f6020d Implemented io-uring Op<Statx> and applied to read_uring and fs::try_exists (#8080)
* implement Op<Statx> and use it to implement fs::try_exists

* fix statx on unavailable platforms

* complete using only io-uring operations for read_uring

* Implemented io-uring::Op<Statx> and apply it accordingly to read_uring, try_exists

* Added test for io uring statx operations. Checks for cancellations, shutdown, stating multiple files, ELOOP, ENAMETOOLONG, EACCES

* Removed musl as supported platform for io_uring statx operations, as statx is supported on 1.25+ musl, and MSRV that uses 1.25 on all *-linux-musl platforms is 1.93

* Removed pending checks for cancel_op_future since io_uring not available on Linux <5.1, removed stat permission denied test case since it doesn't work on Linux 4.19

* Removed redundant cfg attributes on functions, added STATX_BTIME flag in statx operation, use assert pending in cancel ops

* Uncommented stat_permission_denied test and make sure it doesn't run on platforms that don't support io_uring, removed unnecessary comments, and added TODO on symlink_metadata for when Metadata::from_statx is stabilized

* Statx fd leak drop test added and added STATX_BTIME to file_metadata

* Remove unnecessary utils function, refactored code in statx and statx fd leak test, and removed cfg_io_uring gates (localized the feature gate to the pertinent part of the code that uses it in read_uring)

---------

Co-authored-by: Vrtgs <[email protected]>
2026-06-13 13:30:47 -04:00
ADD-SPandAlice Ryhl da044f27d7 net: disable some Miri tests for TCP socket (#8205)
Co-authored-by: Alice Ryhl <[email protected]>
2026-06-12 10:12:08 +08:00
Mukhammedali Berektassuly ecb5125a67 runtime: document interaction with fork() (#8202) 2026-06-10 12:05:27 +00:00
Jean SIMARD bde8967853 task: add Stream wrapper for JoinSet (#8189) 2026-06-10 09:04:27 +02:00
Satyam Gupta 21e4a2a282 taskdump: support taskdumps on s390x (#8192)
Signed-off-by: satyamg1620 <[email protected]>
2026-06-10 08:55:09 +02:00
Amey Pawar 2e7930fe58 net: accept ConnectionReset in shutdown_after_tcp_reset test (#8196)
The test asserts shutdown() returns Ok(()) after the peer resets the
connection (linger = 0). This holds on Linux and macOS, but on FreeBSD the
kernel can finish processing the RST before shutdown() runs, so it returns
ConnectionReset and the test fails intermittently -- the oneshot only
synchronizes the application-level drop, not the kernel's RST processing.

Accept Ok(()) or ConnectionReset for the post-reset shutdown, since both
are valid for a connection the peer has already reset.
2026-06-09 15:00:24 +00:00
Sabnock 67637b348a macros: clarify tokio::main expansion (#8193) 2026-06-09 14:22:46 +02:00
Nicolò Paternoster 778e9d97d9 tests: fix typo in io_uring support functions docs (#8186) 2026-06-03 17:37:57 +02:00
elomatreb 71362aa609 net: add SocketAddr methods to Unix sockets (#8144) 2026-06-03 09:23:30 +02:00
WhySoBad 2de86e557c net: enable more Miri tests for TCP socket (#8180) 2026-06-02 23:50:54 +08:00
RyanStewart 326bd2cc44 codec: use libc::memchr for LinesCodec delimiter scan (#8141) 2026-06-02 17:23:18 +02:00
WhySoBad 32312ae0d6 net: enable Miri tests for TCP socket (#8156) 2026-05-26 10:40:09 +08:00
Qiqi Zhang 37ced33efd time: ensure timers stay in the same runtime after .reset() (#8169) 2026-05-26 10:29:50 +08:00
Tim Vilgot Mikael Fredenberg 923e72345c time: move lazy-registration state into Sleep (#8132) 2026-05-25 08:11:09 +08:00
Mattia Pitossi f619fc0587 docs: improve contributing guidelines (#8166) 2026-05-24 09:10:16 +02:00
吴杨帆 82fe082ef1 fs: clarify create_dir_all succeeds if path exists (#8149) 2026-05-21 15:43:17 +02:00
Joe Grund 1fe1b0e727 task: add JoinMap::try_join_next (#8099) 2026-05-21 15:41:55 +02:00
Minh Vu c6af672353 io: advance partially written buffers correctly in write_all_vectored (#8159) 2026-05-21 01:21:35 -07:00
Abhinav 2a05f364b7 ci: set copyback to false for FreeBSD jobs (#8155) 2026-05-20 14:32:23 +02:00
Mattia Pitossi c6d58ce7e7 ci: fix macOS runners (#8145) 2026-05-15 08:24:07 +02:00
vip892766gma de360cfc7e tokio-stream: fix duplicated word in changelog (#8143) 2026-05-14 20:33:43 -07:00
RyanStewart 7d3b0ad192 sync: remove useless conversion in oneshot Receiver::poll (#8142) 2026-05-13 09:05:35 +02:00
Russell Cohen 0121120b6d taskdump: skip double wake on Trace::capture/Trace::trace_with (#8043) 2026-05-12 19:56:50 +02:00
Mattia Pitossi bdcea6b2cd fs: skip some io_uring tests on Kernels that don't support it (#8134) 2026-05-11 13:58:50 +02:00
Tim Vilgot Mikael Fredenberg ee0dc90926 time: consolidate mutex locks on spurious poll (#8124) 2026-05-08 11:02:54 -07:00
Alice Ryhl 78594a7497 Merge 'tokio-1.52.3' into 'master' (#8131) 2026-05-08 13:00:22 +00:00
Alice Ryhl d87569164f chore: prepare Tokio v1.52.3 (#8130) 2026-05-08 14:52:32 +02:00
Alice Ryhl e1aebb031c Merge 'tokio-1.51.3' into 'tokio-1.52.x' (#8129) 2026-05-08 09:30:59 +00:00
Alice Ryhl fd63094ee0 chore: prepare Tokio v1.51.3 (#8127) 2026-05-08 10:45:32 +02:00
xtqqczze 067f229371 io: replace Vec method truncate(0) with clear (#8125)
https://rust-lang.github.io/rust-clippy/master/index.html#manual_clear
2026-05-08 08:38:49 +02:00
Alice Ryhl 8c600d0fd2 Merge 'tokio-1.47.5' into 'tokio-1.51.x' (#8123) 2026-05-07 13:59:06 +02:00
Alice Ryhl 11bfc1345b chore: prepare Tokio v1.47.5 (#8122) 2026-05-07 13:55:35 +02:00
Alice Ryhl f085b6211b sync: notify receivers in mpsc OwnedPermit::release() method (#8075) 2026-05-07 09:32:14 +02:00
Alice Ryhl 30d25ccb8b sync: require that an RwLock has max_readers != 0 (#8076) 2026-05-07 09:31:12 +02:00
Alice Ryhl 9fccf5339d sync: return Empty from try_recv() when mpsc is closed with outstanding permits (#8074) 2026-05-07 09:30:50 +02:00
Alice Ryhl ebf61b45b5 sync: fix underflow in mpsc channel len() (#8062) 2026-05-07 09:29:33 +02:00
TimoandMartin Tzvetanov Grigorov 02ff0833e0 sync: implement PartialEq and Eq for CancellationToken (#8110)
Co-authored-by: Martin Tzvetanov Grigorov <[email protected]>
2026-05-05 13:46:35 +02:00
Alice Ryhl e56ff72fe7 Merge 'tokio-1.52.2' into 'master' (#8117) 2026-05-04 19:53:32 +02:00
Alice Ryhl 4abe9d732e chore: prepare Tokio v1.52.2 (#8115) 2026-05-04 14:39:25 +02:00
Alice Ryhl f82bcf3f45 Merge 'tokio-1.51.2' into 'tokio-1.52.x' (#8114) 2026-05-04 12:16:52 +02:00
Alice Ryhl 7db9bc41f1 test: revert "remove churn() task from lifo_stealable" (#8114)
This reverts commit 6c03e03898.
2026-05-04 12:15:37 +02:00
Alice Ryhl 64834ec701 chore: prepare Tokio v1.51.2 (#8113) 2026-05-04 12:11:07 +02:00
Alice Ryhl 967f5715a7 runtime: revert "steal tasks from the LIFO slot" (#8100)
This reverts commit eeb55c733b.
2026-05-04 10:08:30 +02:00
HueCodes e77885a494 net: implement UCred::pid on FreeBSD (#8086) 2026-05-03 10:28:31 +02:00
Tim Vilgot Mikael Fredenberg 0926ab195a time: defer waker clone on spurious poll (#8107) 2026-05-02 20:39:08 -07:00
Alice RyhlandMartin Tzvetanov Grigorov 50c23d300a runtime: avoid illegal state in FastRand (#8078)
Co-authored-by: Martin Tzvetanov Grigorov <[email protected]>
2026-05-02 19:01:15 +02:00
Mattia Pitossi 1cd6e65d76 ci: small improvements for freeBSD jobs (#8102) 2026-05-02 17:44:40 +02:00
Alice Ryhl 5ee26af3db Merge tokio-1.52.x (for #8101) into master (#8106) 2026-05-02 15:13:54 +02:00
Alice Ryhl 9271e3ed05 Merge tokio-1.51.x (for #8101) into tokio-1.52.x (#8106) 2026-05-02 15:13:06 +02:00
Alice Ryhl cd1823f43e Revert "Pin stable to 1.94 for tokio-1.51.x" (#8106)
As we are merging tokio-1.51.x into tokio-1.52.x and then master, we
should not include this change.

This reverts commit bde3f20b0f.
2026-05-02 15:12:29 +02:00
Alice Ryhl a97cf12ed9 Merge tokio-1.47.x (commit 670a907c55) into tokio-1.51.x (#8105) 2026-05-02 13:52:07 +02:00
Alice Ryhl bde3f20b0f Pin stable to 1.94 for tokio-1.51.x (#8105) 2026-05-02 13:51:31 +02:00
Alice RyhlandMattia Pitossi 670a907c55 ci: fix CI on tokio-1.47.x (#8101)
Co-authored-by: Mattia Pitossi <[email protected]>
2026-05-02 13:33:40 +02:00
Rachit2323 5030b3005e tokio-stream: implement FusedStream for various stream adaptors (#8096)
`MapWhile` is intentionally excluded because its current implementation
does not track when the closure returns `None` early, making a correct
`is_terminated()` impossible without a separate semantic change.
2026-05-01 14:09:28 +02:00
Tim Vilgot Mikael Fredenberg 26dee92b53 chore: use functional slice building (#8097)
Noticed some redundant procedural code.
2026-04-30 11:36:55 +02:00
Mawer d3565a2923 fs: rename Windows symlink dir test (#8098) 2026-04-30 10:47:45 +02:00
Alice Ryhl 8f81e0814b time: avoid stack overflow in runtime constructor (#8093) 2026-04-29 05:26:57 +00:00
stormshield-fabs 47cc31590f tokio-stream: implement FusedStream for Fuse (#8090) 2026-04-28 17:33:54 +02:00
Alex Gaynor dc0f728162 blocking: introduce a regression test for #8056 (#8068) 2026-04-28 12:47:49 +02:00
Ralf Jung 89713ad1ec miri tests: fstat is supported now (#8088) 2026-04-27 15:23:40 +03:00
Eliza Weisman 6c03e03898 test: remove churn() task from lifo_stealable (#8070)
Currently, the `rt_threaded::lifo_stealable` test I added in #7431
spawns an additional task which sleeps on a 4ms timer in a loop. This
ensures that no worker remains permanently parked. This was added
because it was necessary to stop the LIFO slot deadlock from occurring
prior to changes in the logic for determining whether to notify another
worker, which is what @Darksonn  was referring to in [this comment][1].
Removing the `churn()` test makes the test actually validate that
another worker is notified to steal the LIFO task, and that the changes
from #7431 will *always* prevent a LIFO slot deadlock, regardless of the
behavior of other tasks on the runtime. See also [this comment][2] for
further discussion.

[1]: https://github.com/tokio-rs/tokio/pull/7431#discussion_r2184724657
[2]: https://github.com/tokio-rs/tokio/pull/8069#issuecomment-4274244723
2026-04-20 21:03:42 +02:00
Alice Ryhl 16bc4e2e28 time: add #[track_caller] and panic docs to timeout_at() (#8077) 2026-04-20 20:22:36 +02:00
BarryandMattia Pitossi 1afb391350 net: document pipe try_read*/try_write* readiness behavior (#8032)
Add a Notes section to all five try_* methods on pipe::Sender and
pipe::Receiver explaining that the runtime's I/O driver only delivers
readiness events after control is yielded back to it, so calling
try_read/try_write before any .await returns WouldBlock even when the
operation could otherwise succeed.

This is the same readiness model used by every other Tokio I/O type;
the pipe docs simply did not previously call it out. Refs #7625.

---------

Co-authored-by: Mattia Pitossi <[email protected]>
2026-04-19 19:52:41 +02:00
Mattia Pitossi b010b5ddaf test taskdump docs (#8064) 2026-04-17 23:32:22 +03:00
Eliza Weisman 905c146aed chore: prepare to release v1.52.1 (#8059)
# 1.52.1 (April 16th, 2026)

## Fixed

- runtime: revert [#7757] to fix [a regression][#8056] that causes
  `spawn_blocking` to hang ([#8057])

[#7757]: https://github.com/tokio-rs/tokio/pull/7757
[#8056]: https://github.com/tokio-rs/tokio/pull/8056
[#8057]: https://github.com/tokio-rs/tokio/pull/8057
2026-04-16 21:28:05 +00:00
Eliza Weisman 56aaa43e91 rt: revert #7757 to fix regression in spawn_blocking (#8057)
This reverts commit 1604bc3351.

Unfortunately, this commit introduced a regression that causes programs
using `spawn_blocking` to hang (see #8056). To fix the regression, we
need to undo this change and publish a v1.52.1 release as soon as
possible.

In the future, we may wish to bring back a sharded queue for
`spawn_blocking` tasks, either based on the implementation added in
#7757 or a new one. However, since this is a substantial change to the
runtime internals, I think such a change should probably be done as an
unstable, opt-in `tokio::runtime::Builder` setting initially, so that we
don't regress existing users. I had hoped we could do this now, but
unfortunately, the sharded queue implementation from #7757 is kind of
tightly coupled with the rest of the `spawn_blocking` machinery and
cannot be easily swapped out --- and the hang still occurs with
`NUM_SHARDS` set to 1, so there isn't an easy way to turn it on and off.
Therefore, in the interest of getting a fix out ASAP, this is just a
simple revert.

Fixes #8056
2026-04-16 20:57:22 +00:00
Eliza Weisman 57ff47ab58 ci: update trybuild to expect output from rustc 1.95.0 (#8058) 2026-04-16 20:32:53 +00:00
dependabot[bot] 812de3e134 ci: bump taiki-e/cache-cargo-install-action from 1 to 3 (#8053)
Bumps [taiki-e/cache-cargo-install-action](https://github.com/taiki-e/cache-cargo-install-action) from 1 to 3.
- [Release notes](https://github.com/taiki-e/cache-cargo-install-action/releases)
- [Changelog](https://github.com/taiki-e/cache-cargo-install-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/cache-cargo-install-action/compare/v1...v3)

---
updated-dependencies:
- dependency-name: taiki-e/cache-cargo-install-action
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-16 09:30:55 +02:00
Martin Grigorov ba82e73c7b ci: use Dependabot to keep github actions up to date (#8052) 2026-04-16 08:51:22 +02:00
Mattia Pitossi 2e85f9ddf8 ci: replace cirrus-ci with freebsd-vm (#8041) 2026-04-16 08:40:04 +02:00
Alice Ryhl a7e1cd8ff8 ci: update GitHub Actions workflows to use latest tool versions (#8047) 2026-04-15 16:57:43 +00:00
Eliza Weisman 5f7be0ac42 chore: perpare 1.52.0 (#8045)
# 1.52.0 (April 14th, 2026)

## Added

- io: `AioSource::register_borrowed` for I/O safety support ([#7992])
- net: add `try_io` function to `unix::pipe` sender and receiver types
  ([#8030])

## Added (unstable)

- runtime: `Builder::enable_eager_driver_handoff` setting enable eager
  hand off of the I/O and time drivers before polling tasks ([#8010])
- taskdump: add `trace_with()` for customized task dumps ([#8025])
- taskdump: allow `impl FnMut()` in `trace_with` instead of just `fn()`
  ([#8040])
- fs: support `io_uring` in `AsyncRead` for `File` ([#7907])

## Changed

- runtime: improve `spawn_blocking` scalability with sharded queue
  ([#7757])
- runtime: use `compare_exchange_weak()` in worker queue ([#8028])

## Fixed

- runtime: overflow second half of tasks when local queue is filled
  instead of first half ([#8029])

## Documented

- docs: fix typo in `oneshot::Sender::send` docs ([#8026])
- docs: hide #[tokio::main] attribute in the docs of `sync::watch`
  ([#8035])
- net: add docs on `ConnectionRefused` errors with UDP sockets ([#7870])

[#7757]: https://github.com/tokio-rs/tokio/pull/7757
[#7870]: https://github.com/tokio-rs/tokio/pull/7870
[#7907]: https://github.com/tokio-rs/tokio/pull/7907
[#7992]: https://github.com/tokio-rs/tokio/pull/7992
[#8010]: https://github.com/tokio-rs/tokio/pull/8010
[#8025]: https://github.com/tokio-rs/tokio/pull/8025
[#8026]: https://github.com/tokio-rs/tokio/pull/8026
[#8028]: https://github.com/tokio-rs/tokio/pull/8028
[#8029]: https://github.com/tokio-rs/tokio/pull/8029
[#8030]: https://github.com/tokio-rs/tokio/pull/8030
[#8035]: https://github.com/tokio-rs/tokio/pull/8035
[#8040]: https://github.com/tokio-rs/tokio/pull/8040
2026-04-14 11:56:01 -07:00
Alice Ryhl 36d12d2686 taskdump: allow impl FnMut() in taskdumps instead of just fn() (#8040) 2026-04-14 19:18:56 +02:00
Carter Green f943312865 fs: support io-uring in AsyncRead for File (#7907) 2026-04-13 20:36:23 +02:00
Timo 5db10f538b net: add 'try_io' function to 'unix::pipe' sender and receiver types (#8030) 2026-04-12 15:06:06 +02:00
Russell Cohen bbdba7101d taskdump: add trace_with for customized task dumps (#8025)
provided trace function.
2026-04-12 11:13:24 +02:00
xtqqczze 7cfce54386 ci: update FreeBSD image to 14.4 (#8038) 2026-04-11 22:20:55 +02:00
Alice Ryhl 81370e6202 net: add docs on ConnectionRefused errors with udp sockets (#7870) 2026-04-11 20:06:08 +00:00
Alice Ryhl 203af02126 runtime: overflow second half of tasks when local queue is filled instead of first half (#8029) 2026-04-11 12:14:26 +02:00
Joel Dice de230926dc net: temporarily disable tcp_stream try_read_buf test on WASI (#8036)
This test is flaky on WASI until
https://github.com/bytecodealliance/wasmtime/issues/13040 has been addressed.

See https://github.com/tokio-rs/tokio/issues/8034 for additional details.
2026-04-11 12:06:46 +02:00
Abhinav 432ec3f2b2 sync: hide #[tokio::main] attribute in the docs of sync::watch (#8035) 2026-04-10 13:15:37 -07:00
Alex Gaynor 1604bc3351 rt: improve spawn_blocking scalability with sharded queue (#7757) 2026-04-10 16:06:19 +02:00
Alice Ryhl 4c7d8b2a14 runtime: use compare_exchange_weak() in worker queue (#8028) 2026-04-10 09:05:18 +02:00
Alan Somers d7db722bb4 io: AioSource now employs IO Safety (#7992) 2026-04-08 22:59:32 +02:00
Eliza Weisman fccc28f1d0 runtime: optional eager I/O driver/timer handoff when polling tasks (#8010) 2026-04-08 22:46:27 +02:00
winningMove 927df0e9d9 sync: fix typo in oneshot send doc (#8026) 2026-04-08 19:31:22 +02:00
404 changed files with 13845 additions and 3325 deletions
-71
View File
@@ -1,71 +0,0 @@
only_if: $CIRRUS_TAG == '' && ($CIRRUS_PR != '' || $CIRRUS_BRANCH == 'master' || $CIRRUS_BRANCH =~ 'tokio-.*')
auto_cancellation: $CIRRUS_BRANCH != 'master' && $CIRRUS_BRANCH !=~ 'tokio-.*'
freebsd_instance:
image_family: freebsd-14-3
env:
RUST_STABLE: stable
RUST_NIGHTLY: nightly-2025-10-12
RUSTFLAGS: -D warnings
# This excludes unstable features like io_uring, which require '--cfg tokio_unstable'.
TOKIO_STABLE_FEATURES: full,test-util
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 64-bit
setup_script:
- pkg install -y bash
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain $RUST_STABLE
- . $HOME/.cargo/env
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
test_script:
- . $HOME/.cargo/env
- cargo test --workspace --features $TOKIO_STABLE_FEATURES
# Free the disk space before the next build,
# otherwise cirrus-ci complains about "No space left on device".
- cargo clean
# Enable all unstable features except `taskdump`, which is Linux-only.
- |
RUSTFLAGS="$RUSTFLAGS --cfg tokio_unstable" \
RUSTDOCFLAGS="$RUSTDOCFLAGS --cfg tokio_unstable" \
cargo test \
--features $TOKIO_STABLE_FEATURES,io-uring,tracing
task:
name: FreeBSD docs
env:
RUSTFLAGS: --cfg docsrs --cfg tokio_unstable
RUSTDOCFLAGS: --cfg docsrs --cfg tokio_unstable -Dwarnings
setup_script:
- pkg install -y bash
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain $RUST_NIGHTLY
- . $HOME/.cargo/env
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
test_script:
- . $HOME/.cargo/env
# We use `--features $TOKIO_STABLE_FEATURES,io-uring,tracing` instead of
# `--all-features` to exclude `taskdump`, which is Linux-only.
- cargo doc --lib --no-deps --features $TOKIO_STABLE_FEATURES,io-uring,tracing --document-private-items
task:
name: FreeBSD 32-bit
setup_script:
- pkg install -y bash
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y --profile minimal --default-toolchain $RUST_STABLE
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
echo "~~~~ rustc --version ~~~~"
rustc --version
test_script:
- . $HOME/.cargo/env
- cargo test --workspace --features $TOKIO_STABLE_FEATURES --target i686-unknown-freebsd
+7
View File
@@ -0,0 +1,7 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
open-pull-requests-limit: 10
+31 -17
View File
@@ -1,28 +1,42 @@
R-loom-blocking:
- tokio/src/runtime/blocking/*
- tokio/src/runtime/blocking/**/*
- changed-files:
- any-glob-to-any-file:
- tokio/src/runtime/blocking/*
- tokio/src/runtime/blocking/**/*
R-loom-sync:
- tokio/src/sync/*
- tokio/src/sync/**/*
- changed-files:
- any-glob-to-any-file:
- tokio/src/sync/*
- tokio/src/sync/**/*
R-loom-time-driver:
- tokio/src/runtime/time/*
- tokio/src/runtime/time/**/*
- changed-files:
- any-glob-to-any-file:
- tokio/src/runtime/time/*
- tokio/src/runtime/time/**/*
- tokio/src/runtime/time_alt/*
- tokio/src/runtime/time_alt/**/*
R-loom-current-thread:
- tokio/src/runtime/scheduler/*
- tokio/src/runtime/scheduler/current_thread/*
- tokio/src/runtime/task/*
- tokio/src/runtime/task/**
- changed-files:
- any-glob-to-any-file:
- tokio/src/runtime/scheduler/*
- tokio/src/runtime/scheduler/current_thread/*
- tokio/src/runtime/task/*
- tokio/src/runtime/task/**
R-loom-multi-thread:
- tokio/src/runtime/scheduler/*
- tokio/src/runtime/scheduler/multi_thread/*
- tokio/src/runtime/scheduler/multi_thread/**
- tokio/src/runtime/task/*
- tokio/src/runtime/task/**
- changed-files:
- any-glob-to-any-file:
- tokio/src/runtime/scheduler/*
- tokio/src/runtime/scheduler/multi_thread/*
- tokio/src/runtime/scheduler/multi_thread/**
- tokio/src/runtime/task/*
- tokio/src/runtime/task/**
R-loom-util:
- tokio-util/src/*
- tokio-util/src/**/*
- changed-files:
- any-glob-to-any-file:
- tokio-util/src/*
- tokio-util/src/**/*
+1 -1
View File
@@ -20,5 +20,5 @@ jobs:
issues: write
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- uses: EmbarkStudios/cargo-deny-action@v2
+234 -52
View File
@@ -18,7 +18,9 @@ env:
rust_stable: stable
rust_nightly: nightly-2025-10-12
# Pin a specific miri version
rust_miri_nightly: nightly-2025-11-13
rust_miri_nightly: nightly-2026-06-29
rust_emscripten_nightly: nightly-2026-08-17
emsdk_version: '6.0.8'
rust_clippy: '1.88'
# When updating this, also update:
# - README.md
@@ -64,7 +66,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -75,6 +77,9 @@ jobs:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
with:
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
# Run `tokio` with stable features. This excludes testing utilities which
# can alter the runtime behavior of Tokio.
@@ -96,7 +101,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -107,6 +112,9 @@ jobs:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
with:
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
- name: test --features ${{ env.TOKIO_STABLE_FEATURES }}
run: |
@@ -135,7 +143,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -146,6 +154,9 @@ jobs:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
with:
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
- name: test --features ${{ env.TOKIO_STABLE_FEATURES }} panic=abort
run: |
@@ -168,7 +179,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -179,6 +190,9 @@ jobs:
tool: cargo-hack
- uses: Swatinem/rust-cache@v2
with:
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
# Run integration tests for each feature
- name: test tests-integration --each-feature
@@ -208,7 +222,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -234,7 +248,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -275,7 +289,7 @@ jobs:
- { os: ubuntu-latest, extra_features: io-uring }
- { os: macos-latest, extra_features: "" }
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -287,6 +301,9 @@ jobs:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
with:
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
# Run `tokio` with "unstable" cfg flag.
- name: test tokio full --cfg unstable
run: |
@@ -300,6 +317,30 @@ jobs:
# the unstable cfg to RustDoc
RUSTDOCFLAGS: --cfg tokio_unstable
# Run the test suite with the sharded `spawn_blocking` queue.
test-sharded-blocking-queue:
name: test tokio full with sharded blocking queue
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
toolchain: ${{ env.rust_stable }}
- name: Install cargo-nextest
uses: taiki-e/install-action@v2
with:
tool: cargo-nextest
- uses: Swatinem/rust-cache@v2
- name: test tokio full with sharded blocking queue
run: cargo nextest run --features full
working-directory: tokio
env:
TOKIO_UNSTABLE_SHARDED_BLOCKING_QUEUE: "1"
test-unstable-taskdump:
name: test tokio full --unstable --taskdump
needs: basics
@@ -309,7 +350,7 @@ jobs:
include:
- os: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -343,7 +384,7 @@ jobs:
include:
- os: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -372,7 +413,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_miri_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -395,7 +436,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_miri_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -418,7 +459,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_miri_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -437,7 +478,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install llvm
# Required to resolve symbols in sanitizer output
run: sudo apt-get install -y llvm
@@ -459,7 +500,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Check `tokio` semver
uses: obi1kenobi/cargo-semver-checks-action@v2
with:
@@ -470,7 +511,7 @@ jobs:
# We don't care about the semver of unstable tokio features.
features: ${{ env.TOKIO_STABLE_FEATURES }}
- name: Check semver for rest of the workspace
if: ${{ !startsWith(github.event.pull_request.base.ref, 'tokio-1.') }}
if: ${{ !startsWith(github.event.pull_request.base.ref, 'tokio-1.') && !startsWith(github.ref_name, 'tokio-1.') }}
uses: obi1kenobi/cargo-semver-checks-action@v2
with:
rust-toolchain: ${{ env.rust_stable }}
@@ -488,7 +529,7 @@ jobs:
- powerpc64-unknown-linux-gnu
- arm-linux-androideabi
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -513,7 +554,7 @@ jobs:
# - name: armv7-sony-vita-newlibeabihf
# exclude_features: "process,signal,rt-process-signal,full,taskdump"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@nightly
with:
@@ -547,7 +588,7 @@ jobs:
- target: aarch64-pc-windows-msvc
os: windows-11-arm
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
with:
@@ -597,7 +638,7 @@ jobs:
- target: aarch64-pc-windows-msvc
os: windows-11-arm
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Rust stable
uses: dtolnay/rust-toolchain@stable
with:
@@ -638,7 +679,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -670,7 +711,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -686,8 +727,9 @@ jobs:
# https://github.com/tokio-rs/tokio/pull/5356
# https://github.com/tokio-rs/tokio/issues/5373
- name: Check
# We use `--skip io-uring` since io-uring crate doesn't provide a binding for the i686 target.
run: cargo hack check -Zbuild-std --target target-specs/i686-unknown-linux-gnu.json -p tokio --feature-powerset --skip io-uring --depth 2 --keep-going
# We use `--skip io-uring,schedule-latency` since io-uring crate doesn't provide a binding for the i686 target
# and schedule latency tracking is only supported on 64-bit targets.
run: cargo hack check -Zbuild-std --target target-specs/i686-unknown-linux-gnu.json -p tokio --feature-powerset --skip io-uring,schedule-latency --depth 2 --keep-going
env:
RUSTFLAGS: --cfg tokio_unstable -Dwarnings
@@ -700,15 +742,15 @@ jobs:
include:
- name: ""
rustflags: ""
exclude_features: "io-uring,taskdump"
exclude_features: "io-uring,taskdump,schedule-latency"
- name: "--unstable"
rustflags: "--cfg tokio_unstable -Dwarnings"
exclude_features: "io-uring,taskdump"
- name: "--unstable io-uring,taskdump"
exclude_features: "io-uring,taskdump,schedule-latency"
- name: "--unstable io-uring,taskdump,schedule-latency"
rustflags: "--cfg tokio_unstable -Dwarnings"
exclude_features: ""
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -727,7 +769,7 @@ jobs:
name: minrust
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_min }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -739,15 +781,16 @@ jobs:
- uses: Swatinem/rust-cache@v2
- name: "cargo check"
run: |
# `cargo check -p tokio --all-features` must pass without `tokio_unstable`;
# unstable Cargo features must be inert unless `tokio_unstable` is also enabled.
if [[ "${{ github.event.pull_request.base.ref }}" =~ ^tokio-1\..* ]]; then
# Only check `tokio` crate as the PR is backporting to an earlier tokio release.
cargo check -p tokio --features $TOKIO_STABLE_FEATURES
cargo check -p tokio --all-features
else
# Check all crates in the workspace
cargo check -p tokio --features $TOKIO_STABLE_FEATURES
# Other crates doesn't have unstable features, so we can use --all-features.
cargo check -p tokio --all-features
cargo hack check -p tokio-macros -p tokio-stream -p tokio-util -p tokio-test --all-features
fi
@@ -756,7 +799,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -789,7 +832,7 @@ jobs:
name: fmt
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -809,7 +852,7 @@ jobs:
name: clippy
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_clippy }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -849,7 +892,7 @@ jobs:
extra_features: "tracing,io-uring,taskdump"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -866,7 +909,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -882,7 +925,7 @@ jobs:
name: Check README
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify that both READMEs are identical
run: diff README.md tokio/README.md
@@ -901,7 +944,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -937,6 +980,8 @@ jobs:
# relative to the `$workspace` and defaults to "target" if not explicitly given.
# default: ". -> target"
workspaces: "./hyper"
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
- name: Test hyper
run: cargo test --features full
@@ -953,7 +998,7 @@ jobs:
- ubuntu-latest
- macos-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -985,6 +1030,8 @@ jobs:
# relative to the `$workspace` and defaults to "target" if not explicitly given.
# default: ". -> target"
workspaces: "./quinn"
# FIXME: temporary workaround, see: https://github.com/Swatinem/rust-cache/issues/341
cache-bin: ${{ matrix.os != 'macos-latest' }}
- name: Test Quinn
working-directory: quinn
@@ -997,7 +1044,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1014,7 +1061,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@master
with:
@@ -1038,7 +1085,7 @@ jobs:
- name: macros sync time rt
features: "macros sync time rt"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1061,7 +1108,7 @@ jobs:
- wasm32-wasip1
- wasm32-wasip1-threads
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1072,7 +1119,10 @@ jobs:
- name: Install cargo-hack, wasmtime
uses: taiki-e/install-action@v2
with:
tool: cargo-hack,wasmtime
# Wasmtime v46.0.1 is the last version to support
# `wasm32-wasip1-threads` (which was an experiment that was never
# standardized):
tool: cargo-hack,[email protected]
- uses: Swatinem/rust-cache@v2
- name: WASI test tokio full
@@ -1122,7 +1172,7 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1132,7 +1182,9 @@ jobs:
- name: Install cargo-nextest, wasmtime
uses: taiki-e/install-action@v2
with:
tool: cargo-nextest,wasmtime-cli
# Wasmtime v47.0.0 or later should work, but we stick with a known,
# specific version to avoid surprises:
tool: cargo-nextest,[email protected]
- uses: Swatinem/rust-cache@v2
@@ -1143,6 +1195,59 @@ jobs:
RUSTFLAGS: --cfg tokio_unstable
CARGO_TARGET_WASM32_WASIP2_RUNNER: wasmtime run -Sinherit-network
wasm32-unknown-emscripten:
name: test tokio for wasm32-unknown-emscripten
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
toolchain: ${{ env.rust_stable }}
targets: wasm32-unknown-emscripten
- name: Install Emscripten
uses: mymindstorm/setup-emsdk@v14
with:
version: ${{ env.emsdk_version }}
- uses: actions/setup-node@v4
with:
node-version: 26
- uses: Swatinem/rust-cache@v2
- name: Install cargo-hack
uses: taiki-e/install-action@v2
with:
tool: cargo-hack
- name: Check tokio feature matrix for emscripten
run: cargo hack check -p tokio --each-feature --exclude-features full,net,process,signal,rt-multi-thread,io-uring,taskdump,schedule-latency --target wasm32-unknown-emscripten
working-directory: tokio
- name: Test tokio for emscripten
run: cargo test -p tokio --target wasm32-unknown-emscripten --features "rt,time,sync,macros,fs,io-util,io-std,test-util" --tests
working-directory: tokio
env:
CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUNNER: node
RUSTFLAGS: "-Dwarnings -Clink-args=-sALLOW_MEMORY_GROWTH=1 -Clink-args=-sEXIT_RUNTIME=1 -Clink-args=-sSTACK_SIZE=1048576"
- name: Install Rust ${{ env.rust_emscripten_nightly }}
uses: dtolnay/rust-toolchain@nightly
with:
toolchain: ${{ env.rust_emscripten_nightly }}
targets: wasm32-unknown-emscripten
components: rust-src
- name: Test tokio multi-thread runtime for emscripten (pthreads)
run: cargo +${{ env.rust_emscripten_nightly }} test -Zbuild-std=std,panic_abort -p tokio --target wasm32-unknown-emscripten --features "rt,rt-multi-thread,time,sync,macros" --test rt_multi_thread_emscripten
working-directory: tokio
env:
CARGO_TARGET_WASM32_UNKNOWN_EMSCRIPTEN_RUNNER: node
RUSTFLAGS: "-Dwarnings -Ctarget-feature=+atomics,+bulk-memory,+mutable-globals -Clink-args=-pthread -Clink-args=-sPTHREAD_POOL_SIZE=8 -Clink-args=-sINITIAL_MEMORY=134217728 -Clink-args=-sEXIT_RUNTIME=1 -Clink-args=-sSTACK_SIZE=1048576"
check-external-types:
name: check-external-types (${{ matrix.os }})
needs: basics
@@ -1157,7 +1262,7 @@ jobs:
# includes all unstable features.
extra_features: "tracing,io-uring,taskdump"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ matrix.rust }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1166,7 +1271,7 @@ jobs:
toolchain: nightly-2025-08-06
- uses: Swatinem/rust-cache@v2
- name: Install cargo-check-external-types
uses: taiki-e/cache-cargo-install-action@v1
uses: taiki-e/cache-cargo-install-action@v3
with:
tool: [email protected]
- name: check-external-types
@@ -1181,7 +1286,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_nightly }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1201,7 +1306,7 @@ jobs:
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@stable
with:
@@ -1210,7 +1315,7 @@ jobs:
uses: taiki-e/install-action@v2
with:
tool: cargo-spellcheck
- uses: actions/checkout@v4
- uses: actions/checkout@v7
- name: Make sure dictionary words are sorted and unique
run: |
FILE="spellcheck.dic"
@@ -1292,3 +1397,80 @@ jobs:
uses: ./.github/workflows/uring-kernel-version-test.yml
with:
kernel_version: ${{ matrix.kernel_version }}
freebsd-x86_64:
name: FreeBSD x86_64
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Test in FreeBSD
uses: vmactions/freebsd-vm@v1
with:
release: '14.4'
envs: "TOKIO_STABLE_FEATURES RUSTFLAGS"
sync: rsync
copyback: false
prepare: |
pkg install -y curl
curl https://sh.rustup.rs -sSf --output rustup.sh
sh rustup.sh -y --profile minimal --default-toolchain ${{ env.rust_stable }}
run: |
. $HOME/.cargo/env
cargo test --workspace --features $TOKIO_STABLE_FEATURES
# Enable all unstable features except `io_uring` and `taskdump`,
# which are Linux-only features.
RUSTFLAGS="$RUSTFLAGS --cfg tokio_unstable" \
cargo test \
--features $TOKIO_STABLE_FEATURES,tracing
freebsd-docs:
name: FreeBSD docs
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Build docs in FreeBSD
uses: vmactions/freebsd-vm@v1
env:
RUSTFLAGS: --cfg docsrs --cfg tokio_unstable
RUSTDOCFLAGS: --cfg docsrs --cfg tokio_unstable -Dwarnings
with:
release: '14.4'
envs: "TOKIO_STABLE_FEATURES RUSTDOCFLAGS RUSTFLAGS"
sync: rsync
copyback: false
prepare: |
pkg install -y curl
curl https://sh.rustup.rs -sSf --output rustup.sh
sh rustup.sh -y --profile minimal --default-toolchain ${{ env.rust_nightly }}
run: |
. $HOME/.cargo/env
# We use `--features $TOKIO_STABLE_FEATURES,tracing` instead of
# `--all-features` to exclude `taskdump` and `io_uring`, which are Linux-only
# features.
cargo doc --lib --no-deps --features $TOKIO_STABLE_FEATURES,tracing \
--document-private-items
freebsd-i686:
name: FreeBSD i686
needs: basics
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Test in FreeBSD
uses: vmactions/freebsd-vm@v1
with:
release: '14.4'
envs: "TOKIO_STABLE_FEATURES RUSTFLAGS"
sync: rsync
copyback: false
prepare: |
pkg install -y curl
curl https://sh.rustup.rs -sSf --output rustup.sh
sh rustup.sh -y --profile minimal --default-toolchain ${{ env.rust_stable }}
run: |
. $HOME/.cargo/env
rustup target add i686-unknown-freebsd
cargo test --workspace --features $TOKIO_STABLE_FEATURES \
--target i686-unknown-freebsd
+1 -1
View File
@@ -19,7 +19,7 @@ jobs:
runs-on: ubuntu-latest
if: github.repository_owner == 'tokio-rs'
steps:
- uses: actions/labeler@v3
- uses: actions/labeler@v7
with:
repo-token: "${{ secrets.GITHUB_TOKEN }}"
sync-labels: true
+15 -6
View File
@@ -28,8 +28,15 @@ jobs:
# base_ref is null when it's not a pull request
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-blocking') || (github.base_ref == null))
runs-on: ubuntu-latest
strategy:
matrix:
# Run the blocking pool loom tests against both `spawn_blocking`
# queue implementations.
include:
- sharded_blocking_queue: "0"
- sharded_blocking_queue: "1"
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -38,6 +45,8 @@ jobs:
- name: run tests
run: cargo test --lib --release --features full -- --nocapture loom_blocking
working-directory: tokio
env:
TOKIO_UNSTABLE_SHARDED_BLOCKING_QUEUE: ${{ matrix.sharded_blocking_queue }}
loom-sync:
name: loom tokio::sync
@@ -45,7 +54,7 @@ jobs:
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-sync') || (github.base_ref == null))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -61,7 +70,7 @@ jobs:
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-time-driver') || (github.base_ref == null))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -77,7 +86,7 @@ jobs:
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-current-thread') || (github.base_ref == null))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -100,7 +109,7 @@ jobs:
- scope: loom_multi_thread::group_c
- scope: loom_multi_thread::group_d
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -118,7 +127,7 @@ jobs:
if: github.repository_owner == 'tokio-rs' && (contains(github.event.pull_request.labels.*.name, 'R-loom-util') || (github.base_ref == null))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
+1 -1
View File
@@ -19,5 +19,5 @@ jobs:
cargo-deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- uses: EmbarkStudios/cargo-deny-action@v2
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
stress-test:
- simple_echo_tcp
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install Rust ${{ env.rust_stable }}
uses: dtolnay/rust-toolchain@master
with:
@@ -14,7 +14,7 @@ jobs:
env:
KERNEL_VERSION: ${{ inputs.kernel_version }}
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Install system dependencies
run: |
@@ -25,7 +25,7 @@ jobs:
- name: Cache Linux source
id: cache-kernel
uses: actions/cache@v4
uses: actions/cache@v6
with:
path: linux-${{ env.KERNEL_VERSION }}
key: kernel-${{ env.KERNEL_VERSION }}
@@ -40,6 +40,8 @@ jobs:
make defconfig
make -j$(nproc)
# We are running tests also on a Kernel version that does not support io_uring
# to check if the fallback mechanism works
- name: Generate test binaries with io_uring enabled
run: |
# Build both integration (tokio/tests/) and unit (e.g., tokio/src/fs/file/tests.rs) tests with io_uring enabled
@@ -87,16 +89,31 @@ jobs:
| cpio --null -ov --format=newc | gzip -9 > ../initramfs.cpio.gz)
- name: Run tests in QEMU
shell: bash
run: |
set -euo pipefail
set +e
qemu-system-x86_64 \
-kernel linux-${{ env.KERNEL_VERSION }}/arch/x86/boot/bzImage \
-initrd initramfs.cpio.gz \
-append "console=ttyS0 rootfstype=ramfs panic=1" \
-nographic -no-reboot -m 1024 -action panic=exit-failure 2>&1 | tee qemu-output.log
qemu_status=${PIPESTATUS[0]} tee_status=${PIPESTATUS[1]}
set -e
if [ "$qemu_status" -ne 0 ]; then
echo "QEMU exited with status $qemu_status"
exit "$qemu_status"
fi
if [ "$tee_status" -ne 0 ]; then
echo "tee exited with status $tee_status"
exit "$tee_status"
fi
# qemu always exits with 0, so we check if the tests passed by using grep.
if grep -q "test result: FAILED" qemu-output.log; then
echo "tests failed (QEMU exited abnormally)"
echo "tests reported failures"
exit 1
else
echo "all tests passed"
+1 -5
View File
@@ -1,7 +1,3 @@
*[TokioConf 2026 program and tickets are now available!](https://tokioconf.com)*
---
# Tokio
A runtime for writing reliable, asynchronous, and slim applications with
@@ -60,7 +56,7 @@ Make sure you enable the full features of the tokio crate on Cargo.toml:
```toml
[dependencies]
tokio = { version = "1.51.1", features = ["full"] }
tokio = { version = "1.53.1", features = ["full"] }
```
Then, on your main.rs:
+4 -4
View File
@@ -73,7 +73,7 @@ impl SlowHddWriter {
fn write_bytes(
mut self: std::pin::Pin<&mut Self>,
cx: &mut std::task::Context<'_>,
writeable: usize,
writable: usize,
) -> std::task::Poll<Result<usize, std::io::Error>> {
let service_res = self.as_mut().service_write(cx);
@@ -86,7 +86,7 @@ impl SlowHddWriter {
assert!(service_res.is_pending());
Poll::Pending
} else {
let written = available.min(writeable);
let written = available.min(writable);
self.buffer_used += written;
Poll::Ready(Ok(written))
}
@@ -123,8 +123,8 @@ impl AsyncWrite for SlowHddWriter {
cx: &mut std::task::Context<'_>,
bufs: &[std::io::IoSlice<'_>],
) -> std::task::Poll<Result<usize, std::io::Error>> {
let writeable = bufs.iter().fold(0, |acc, buf| acc + buf.len());
self.write_bytes(cx, writeable)
let writable = bufs.iter().fold(0, |acc, buf| acc + buf.len());
self.write_bytes(cx, writable)
}
fn is_write_vectored(&self) -> bool {
+1 -3
View File
@@ -84,9 +84,7 @@ fn remote_spawn_contention(c: &mut Criterion) {
}
fn parallelism_levels() -> Vec<usize> {
let max_parallelism = std::thread::available_parallelism()
.map(|p| p.get())
.unwrap_or(1);
let max_parallelism = std::thread::available_parallelism().map_or(1, |p| p.get());
[1, 2, 4, 8, 16, 32, 64]
.into_iter()
+1 -3
View File
@@ -15,9 +15,7 @@ const NUM_BATCHES: usize = 100;
const BATCH_SIZE: usize = 16;
fn spawn_blocking_concurrency(c: &mut Criterion) {
let max_parallelism = std::thread::available_parallelism()
.map(|p| p.get())
.unwrap_or(1);
let max_parallelism = std::thread::available_parallelism().map_or(1, |p| p.get());
let parallelism_levels: Vec<usize> = [1, 2, 4, 8, 16, 32, 64]
.into_iter()
+36 -2
View File
@@ -4,7 +4,9 @@ use std::sync::Arc;
use tokio::sync::{broadcast, Notify};
use criterion::measurement::WallTime;
use criterion::{black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion};
use criterion::{
black_box, criterion_group, criterion_main, BenchmarkGroup, Criterion, Throughput,
};
fn rt() -> tokio::runtime::Runtime {
tokio::runtime::Builder::new_multi_thread()
@@ -77,6 +79,38 @@ fn bench_contention(c: &mut Criterion) {
group.finish();
}
criterion_group!(contention, bench_contention);
fn bench_try_recv(c: &mut Criterion) {
let mut group = c.benchmark_group("try_recv");
const MESSAGES: usize = 256;
for receiver_count in [1usize, 4, 16] {
group.throughput(Throughput::Elements((MESSAGES * receiver_count) as u64));
group.bench_function(receiver_count.to_string(), |b| {
let (tx, first_rx) = broadcast::channel::<usize>(MESSAGES);
let mut receivers = Vec::with_capacity(receiver_count);
receivers.push(first_rx);
for _ in 1..receiver_count {
receivers.push(tx.subscribe());
}
b.iter(|| {
for message in 0..MESSAGES {
tx.send(black_box(message)).unwrap();
}
for rx in &mut receivers {
for _ in 0..MESSAGES {
black_box(rx.try_recv().unwrap());
}
}
});
});
}
group.finish();
}
criterion_group!(contention, bench_contention, bench_try_recv);
criterion_main!(contention);
+4 -2
View File
@@ -8,7 +8,8 @@ It should be considered a map to help you navigate the process.
If you are unsure where to begin, use the following guides:
- Want to report or triage a bug? Start with [Contributing in Issues](contributing-in-issues.md).
- Looking for something to work on? Filter issues by [`E-help-wanted`](https://github.com/tokio-rs/tokio/labels/E-help-wanted).
- Looking for something to work on? Check the open [`issues`](https://github.com/tokio-rs/tokio/issues)
(some might be in progress or still under discussion, so leave a comment before starting the work).
- Planning to submit a PR? Read [Pull Requests](pull-requests.md) for the full workflow and required checks.
- Want to understand what the labels on issues mean? See [Keeping track of issues and PRs](keeping-track-of-issues-and-prs.md).
- Interested in code review? See [Reviewing Pull Requests](reviewing-pull-requests.md).
@@ -21,8 +22,9 @@ If you are unsure where to begin, use the following guides:
- [Triaging a Bug Report](contributing-in-issues.md#triaging-a-bug-report)
- [Resolving a Bug Report](contributing-in-issues.md#resolving-a-bug-report)
- [Pull Requests](pull-requests.md)
- [Step-by-Step Contribution Workflow](pull-requests.md#step-by-step-contribution-workflow)
- [Cargo Commands](pull-requests.md#cargo-commands)
- [Performing spellcheck on tokio codebase](pull-requests.md#performing-spellcheck-on-tokio-codebase)
- [Performing spellcheck on Tokio codebase](pull-requests.md#performing-spellcheck-on-tokio-codebase)
- [Tests](pull-requests.md#tests)
- [Integration tests](pull-requests.md#integration-tests)
- [Fuzz tests](pull-requests.md#fuzz-tests)
@@ -81,8 +81,10 @@ The module label provides a more fine grained categorization than **Area**.
Some extra information.
- **T-docs** This is about documentation.
- **T-io-uring** This is about io-uring (Linux).
- **T-performance** This is about performance.
- **T-v0.1.x** This is about old Tokio.
- **T-wasm** This is about Web Assembly.
Any label not listed here is not in active use.
+66 -8
View File
@@ -4,10 +4,71 @@ Pull Requests are the way concrete changes are made to the code, documentation,
and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one-character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increase the likelihood of the PR getting
merged.
documentation) are greatly appreciated.
> [!NOTE]
> Before making a large change, it is usually a good idea to first open an
> issue describing the change to solicit feedback and guidance.
> This will increase the likelihood of the PR getting merged.
### Step-by-Step Contribution Workflow
Once you've found an issue you'd like to work on, follow these steps before opening a Pull Request. This workflow provides a chronological overview of the contribution process and points to the relevant documentation where additional detail is available.
#### 1. Fork and Clone the Repository
Fork the repository to your GitHub account and clone your fork locally.
If you've previously cloned the repository, ensure your local copy is up to date before creating a new branch.
```bash
git checkout master
git pull upstream master
```
---
#### 2. Create a Feature Branch
Never commit directly to your local `master` branch. Instead, create a descriptive feature branch for each change you work on.
```bash
git checkout -b my-feature
```
Using a dedicated branch keeps your default branch clean and makes it easier to update your Pull Request during code review.
---
#### 3. Implement Your Changes
Make the code or documentation changes needed. Check your work with `git diff` to confirm the changes look correct before moving on.
If your changes introduce new functionality or modify existing behavior, consider whether additional tests or documentation should also be added.
---
#### 4. Verify Your Changes
Before opening a Pull Request, run the project's verification steps. Refer to the sections below for details on when each command should be used.
---
#### 5. Commit and Push Your Changes
Commit your changes following the [project's commit message guidelines](#commit-message-guidelines).
```bash
git add .
git commit -m "module: describe your change"
git push origin my-feature
```
---
#### 6. Open a Pull Request
Open a Pull Request from your feature branch to the main repository. See [Opening the Pull Request](#opening-the-pull-request) for what to include and how the review process works.
### Cargo Commands
@@ -23,9 +84,6 @@ cargo check --all-features
cargo test --all-features
```
**NOTE**: there are some features that are not supported in every system, so you might
need to specify which features you want to pass to cargo (e.g., `cargo check --features=full,io-uring`)
Ideally, you should use the same version of clippy as the one used in CI
(defined by `env.rust_clippy` in [ci.yml][ci.yml]), because newer versions
might have new lints:
@@ -99,7 +157,7 @@ MIRIFLAGS="-Zmiri-disable-isolation -Zmiri-strict-provenance" \
cargo +nightly miri test --features full --lib --tests
```
### Performing spellcheck on tokio codebase
### Performing spellcheck on Tokio codebase
You can perform a spell-check on the Tokio codebase. For details of how to use the spellcheck tool, feel free to visit
https://github.com/drahnr/cargo-spellcheck
+12 -2
View File
@@ -5,7 +5,12 @@
#[cfg(all(
tokio_unstable,
target_os = "linux",
any(target_arch = "aarch64", target_arch = "x86", target_arch = "x86_64")
any(
target_arch = "aarch64",
target_arch = "x86",
target_arch = "x86_64",
target_arch = "s390x"
)
))]
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
@@ -82,7 +87,12 @@ async fn main() -> Result<(), Box<dyn std::error::Error>> {
#[cfg(not(all(
tokio_unstable,
target_os = "linux",
any(target_arch = "aarch64", target_arch = "x86", target_arch = "x86_64")
any(
target_arch = "aarch64",
target_arch = "x86",
target_arch = "x86_64",
target_arch = "s390x"
)
)))]
fn main() {
println!("task dumps are not available")
+22
View File
@@ -9,6 +9,28 @@
//! cargo run --example connect-udp 127.0.0.1:8080
//!
//! Each line you type in to the `connect-udp` terminal should be echo'd back to you!
//!
//! # Binding to all interfaces
//!
//! By default this example binds to `127.0.0.1` so it is only reachable
//! from the local machine.
//!
//! To listen on all interfaces instead:
//!
//! ```sh
//! cargo run --example echo-udp -- 0.0.0.0:8080
//! ```
//!
//! Binding to `0.0.0.0` exposes the server on all network interfaces.
//! Only do this in trusted network environments.
//!
//! On multi-homed systems, a UDP socket bound to a wildcard address
//! (`0.0.0.0` or `::`) cannot always send replies from the same local IP
//! that received the packet. Replies may therefore originate from a
//! different address than the client targeted. See [this Cloudflare blog
//! post][udp-blog] for more details.
//!
//! [udp-blog]: https://blog.cloudflare.com/everything-you-ever-wanted-to-know-about-udp-sockets-but-were-afraid-to-ask-part-1
#![warn(rust_2018_idioms)]
+12 -9
View File
@@ -25,7 +25,6 @@
use tokio::io::copy_bidirectional;
use tokio::net::{TcpListener, TcpStream};
use futures::FutureExt;
use std::env;
use std::error::Error;
@@ -44,16 +43,20 @@ async fn main() -> Result<(), Box<dyn Error>> {
let listener = TcpListener::bind(listen_addr).await?;
while let Ok((mut inbound, _)) = listener.accept().await {
let mut outbound = TcpStream::connect(server_addr.clone()).await?;
let server_addr = server_addr.clone();
tokio::spawn(async move {
copy_bidirectional(&mut inbound, &mut outbound)
.map(|r| {
if let Err(e) = r {
println!("Failed to transfer; error={e}");
}
})
.await
let mut outbound = match TcpStream::connect(server_addr).await {
Ok(outbound) => outbound,
Err(e) => {
println!("Failed to connect; error={e}");
return;
}
};
if let Err(e) = copy_bidirectional(&mut inbound, &mut outbound).await {
println!("Failed to transfer; error={e}");
}
});
}
+1 -2
View File
@@ -264,8 +264,7 @@ mod date {
let now = SystemTime::now();
let now_unix = now
.duration_since(SystemTime::UNIX_EPOCH)
.map(|since_epoch| since_epoch.as_secs())
.unwrap_or(0);
.map_or(0, |since_epoch| since_epoch.as_secs());
if cache.unix_date != now_unix {
cache.update(now, now_unix);
}
+18 -1
View File
@@ -1,4 +1,4 @@
314
331
&
+
<
@@ -89,6 +89,7 @@ decrementing
demangled
dequeued
dereferenced
derefs
deregister
deregistered
deregistering
@@ -105,6 +106,8 @@ dns
DNS
DoS
dwOpenMode
Emscripten
Emscripten's
endian
enqueue
enqueued
@@ -138,7 +141,9 @@ hashmaps
HashMaps
hashsets
HdrHistogram
ICMP
ie
iff
Illumos
impl
implementers
@@ -188,14 +193,17 @@ mutex
Mutex
Nagle
namespace
NetBSD
nonblocking
nondecreasing
noop
ntasks
NTO
NUMA
ok
oneshot
opcode
OpenBSD
ORed
os
parker
@@ -207,11 +215,13 @@ plaintext
poller
POSIX
proxied
pthreads
qos
RAII
RCU
reallocations
recv's
Redox
refactors
refcount
refcounting
@@ -223,6 +233,7 @@ reregistering
resize
resized
RMW
RNG
runtime
runtime's
runtimes
@@ -238,9 +249,11 @@ signalling
SmallCrush
Solaris
spawner
spawners
Splitter
spmc
spsc
SQE
src
stabilised
startup
@@ -277,6 +290,7 @@ tokio's
Tokio's
tuple
Tuple
tvOS
tx
udp
UDP
@@ -304,6 +318,7 @@ vec
versa
versioned
versioning
visionOS
vtable
waker
wakers
@@ -311,5 +326,7 @@ Wakers
wakeup
wakeups
WASI
Wasm
watchOS
workstealing
ZST
+1 -1
View File
@@ -37,7 +37,7 @@ help: consider importing this trait
1 + use std::future::Future;
|
error[E0433]: failed to resolve: use of undeclared type `Pin`
error[E0433]: cannot find type `Pin` in this scope
--> tests/fail/macros_join.rs:35:17
|
35 | let _ = Pin::new(&mut x);
@@ -37,7 +37,7 @@ help: consider importing this trait
1 + use std::future::Future;
|
error[E0433]: failed to resolve: use of undeclared type `Pin`
error[E0433]: cannot find type `Pin` in this scope
--> tests/fail/macros_try_join.rs:35:17
|
35 | let _ = Pin::new(&mut x);
-2
View File
@@ -16,7 +16,6 @@ async fn spawning() -> usize {
join.await.unwrap()
}
#[cfg(tokio_unstable)]
#[tokio::main(flavor = "local")]
async fn local_main() -> usize {
let join = tokio::task::spawn_local(async { 1 });
@@ -33,6 +32,5 @@ fn shell() {
assert_eq!(1, basic_main());
assert_eq!(bool::default(), generic_fun::<bool>());
#[cfg(tokio_unstable)]
assert_eq!(1, local_main());
}
+12
View File
@@ -1,3 +1,15 @@
# 2.7.2 (July 29th, 2026)
- macros: upgrade syn to v3 ([#8304])
[#8304]: https://github.com/tokio-rs/tokio/pull/8304
# 2.7.1 (July 17th, 2026)
- macros: clarify `tokio::main` expansion ([#8193])
[#8193]: https://github.com/tokio-rs/tokio/pull/8193
# 2.7.0 (April 3rd, 2026)
- macros: stabilize `LocalRuntime` ([#7557])
+2 -2
View File
@@ -4,7 +4,7 @@ name = "tokio-macros"
# - Remove path dependencies (if any)
# - Update CHANGELOG.md.
# - Create "tokio-macros-x.y.z" git tag.
version = "2.7.0"
version = "2.7.2"
edition = "2021"
rust-version = "1.71"
authors = ["Tokio Contributors <[email protected]>"]
@@ -24,7 +24,7 @@ proc-macro = true
[dependencies]
proc-macro2 = "1.0.60"
quote = "1"
syn = { version = "2.0", features = ["full"] }
syn = { version = "3.0", features = ["full"] }
[dev-dependencies]
tokio = { version = "1.0.0", features = ["full", "test-util"] }
+6 -6
View File
@@ -326,7 +326,7 @@ fn contains_impl_trait(ty: &syn::Type) -> bool {
_ => false,
}),
syn::PathArguments::Parenthesized(args) => {
args.inputs.iter().any(contains_impl_trait)
args.inputs.iter().any(|arg| contains_impl_trait(&arg.ty))
|| matches!(&args.output, syn::ReturnType::Type(_, t) if contains_impl_trait(t))
}
syn::PathArguments::None => false,
@@ -453,13 +453,13 @@ fn parse_knobs(mut input: ItemFn, is_test: bool, config: FinalConfig) -> TokenSt
(start, end)
};
let crate_path = config
.crate_name
.map(ToTokens::into_token_stream)
.unwrap_or_else(|| {
let crate_path = config.crate_name.map_or_else(
|| {
Ident::new("tokio", Span::call_site().located_at(last_stmt_start_span))
.into_token_stream()
});
},
ToTokens::into_token_stream,
);
let use_builder = quote_spanned! {Span::call_site().located_at(last_stmt_start_span)=>
use #crate_path::runtime::Builder;
+6 -4
View File
@@ -29,10 +29,11 @@ use proc_macro::TokenStream;
/// powerful interface.
///
/// Note: This macro can be used on any function and not just the `main`
/// function. Using it on a non-main function makes the function behave as if it
/// was synchronous by starting a new runtime each time it is called. If the
/// function is called often, it is preferable to create the runtime using the
/// runtime builder so the runtime can be reused across calls.
/// function. Although the function is written with `async fn`, this macro
/// expands it to a synchronous function that starts a runtime each time it is
/// called. If the function is called often, it is preferable to create the
/// runtime using the runtime builder so the runtime can be reused across calls.
/// For details on the expansion, see [Bridging with sync code][bridging].
///
/// # Non-worker async function
///
@@ -308,6 +309,7 @@ use proc_macro::TokenStream;
/// [`Builder::unhandled_panic`]: ../tokio/runtime/struct.Builder.html#method.unhandled_panic
/// [unstable]: ../tokio/index.html#unstable-features
/// [local runtime]: ../tokio/runtime/struct.LocalRuntime.html
/// [bridging]: https://tokio.rs/tokio/topics/bridging#what-tokiomain-expands-to
#[proc_macro_attribute]
pub fn main(args: TokenStream, item: TokenStream) -> TokenStream {
entry::main(args.into(), item.into(), true).into()
+36 -1
View File
@@ -1,3 +1,38 @@
# 0.1.19 (July 22nd, 2026)
### Added
- stream: implement `FromStream` for standard collections ([#7954], [#7966])
- stream: implement `FusedStream` for various stream adaptors ([#7854], [#8090], [#8096])
- stream: implement `Peekable::size_hint` ([#8109])
- task: add `JoinSetStream` wrapper for `JoinSet` ([#8189])
### Changed
- stream: bump minimum required `tokio` version to `1.38.0` ([#7887])
- stream: use cooperative budgeting in `tokio_stream::iter` ([#8218])
- stream: update coop handling for `empty` and `once` ([#8227])
### Fixed
- stream: fix overflow in `StreamMap::size_hint` ([#8216])
- stream: honor `StreamMap::next_many` limit ([#8215])
- stream: stop polling underlying stream once `map_while` yields `None` ([#8233])
[#7854]: https://github.com/tokio-rs/tokio/pull/7854
[#7887]: https://github.com/tokio-rs/tokio/pull/7887
[#7954]: https://github.com/tokio-rs/tokio/pull/7954
[#7966]: https://github.com/tokio-rs/tokio/pull/7966
[#8090]: https://github.com/tokio-rs/tokio/pull/8090
[#8096]: https://github.com/tokio-rs/tokio/pull/8096
[#8109]: https://github.com/tokio-rs/tokio/pull/8109
[#8189]: https://github.com/tokio-rs/tokio/pull/8189
[#8215]: https://github.com/tokio-rs/tokio/pull/8215
[#8216]: https://github.com/tokio-rs/tokio/pull/8216
[#8218]: https://github.com/tokio-rs/tokio/pull/8218
[#8227]: https://github.com/tokio-rs/tokio/pull/8227
[#8233]: https://github.com/tokio-rs/tokio/pull/8233
# 0.1.18 (January 4th, 2026)
### Added
@@ -12,7 +47,7 @@
### Documented
- stream: improve the the docs of `TcpListenerStream` ([#7578])
- stream: improve the docs of `TcpListenerStream` ([#7578])
[#7024]: https://github.com/tokio-rs/tokio/pull/7024
[#7492]: https://github.com/tokio-rs/tokio/pull/7492
+3 -1
View File
@@ -4,7 +4,7 @@ name = "tokio-stream"
# - Remove path dependencies (if any)
# - Update CHANGELOG.md.
# - Create "tokio-stream-0.1.x" git tag.
version = "0.1.18"
version = "0.1.19"
edition = "2021"
rust-version = "1.71"
authors = ["Tokio Contributors <[email protected]>"]
@@ -24,6 +24,7 @@ full = [
"net",
"io-util",
"fs",
"rt",
"sync",
"signal"
]
@@ -32,6 +33,7 @@ time = ["tokio/time"]
net = ["tokio/net"]
io-util = ["tokio/io-util"]
fs = ["tokio/fs"]
rt = ["tokio/rt"]
sync = ["tokio/sync", "tokio-util"]
signal = ["tokio/signal"]
+9 -1
View File
@@ -40,7 +40,15 @@ pub const fn empty<T>() -> Empty<T> {
impl<T> Stream for Empty<T> {
type Item = T;
fn poll_next(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Option<T>> {
fn poll_next(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Option<T>> {
#[cfg(feature = "rt")]
{
use tokio::task::coop;
let coop = std::task::ready!(coop::poll_proceed(_cx));
coop.made_progress();
}
Poll::Ready(None)
}
+26 -8
View File
@@ -8,6 +8,7 @@ use core::task::{Context, Poll};
#[must_use = "streams do nothing unless polled"]
pub struct Iter<I> {
iter: I,
#[cfg(not(feature = "rt"))]
yield_amt: usize,
}
@@ -36,6 +37,7 @@ where
{
Iter {
iter: i.into_iter(),
#[cfg(not(feature = "rt"))]
yield_amt: 0,
}
}
@@ -47,17 +49,33 @@ where
type Item = I::Item;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<I::Item>> {
// TODO: add coop back
if self.yield_amt >= 32 {
self.yield_amt = 0;
#[cfg(feature = "rt")]
{
use tokio::task::coop;
cx.waker().wake_by_ref();
let coop = std::task::ready!(coop::poll_proceed(cx));
let item = self.iter.next();
Poll::Pending
} else {
self.yield_amt += 1;
coop.made_progress();
Poll::Ready(self.iter.next())
Poll::Ready(item)
}
#[cfg(not(feature = "rt"))]
{
if self.yield_amt >= 32 {
self.yield_amt = 0;
cx.waker().wake_by_ref();
Poll::Pending
} else {
let item = self.iter.next();
self.yield_amt += 1;
Poll::Ready(item)
}
}
}
+10
View File
@@ -57,3 +57,13 @@ macro_rules! cfg_signal {
)*
}
}
macro_rules! cfg_rt {
($($item:item)*) => {
$(
#[cfg(feature = "rt")]
#[cfg_attr(docsrs, doc(cfg(feature = "rt")))]
$item
)*
}
}
+19 -9
View File
@@ -1,6 +1,5 @@
use crate::{Iter, Stream};
use crate::Stream;
use core::option;
use core::pin::Pin;
use core::task::{Context, Poll};
@@ -8,7 +7,7 @@ use core::task::{Context, Poll};
#[derive(Debug)]
#[must_use = "streams do nothing unless polled"]
pub struct Once<T> {
iter: Iter<option::IntoIter<T>>,
value: Option<T>,
}
impl<I> Unpin for Once<I> {}
@@ -34,19 +33,30 @@ impl<I> Unpin for Once<I> {}
/// # }
/// ```
pub fn once<T>(value: T) -> Once<T> {
Once {
iter: crate::iter(Some(value)),
}
Once { value: Some(value) }
}
impl<T> Stream for Once<T> {
type Item = T;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<T>> {
Pin::new(&mut self.iter).poll_next(cx)
fn poll_next(mut self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<Option<T>> {
#[cfg(feature = "rt")]
{
use tokio::task::coop;
let coop = std::task::ready!(coop::poll_proceed(_cx));
coop.made_progress();
}
Poll::Ready(self.value.take())
}
fn size_hint(&self) -> (usize, Option<usize>) {
self.iter.size_hint()
if self.value.is_some() {
(1, Some(1))
} else {
(0, Some(0))
}
}
}
+11
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::pin::Pin;
use core::task::{ready, Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -48,3 +49,13 @@ where
super::merge_size_hints(self.a.size_hint(), self.b.size_hint())
}
}
impl<T, U> FusedStream for Chain<T, U>
where
T: Stream,
U: FusedStream<Item = T::Item>,
{
fn is_terminated(&self) -> bool {
self.a.is_terminated() && self.b.is_terminated()
}
}
+37
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{ready, Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -30,6 +31,32 @@ impl<St, F> Filter<St, F> {
pub(super) fn new(stream: St, f: F) -> Self {
Self { stream, f }
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F> Stream for Filter<St, F>
@@ -56,3 +83,13 @@ where
(0, self.stream.size_hint().1) // can't know a lower bound, due to the predicate
}
}
impl<St, F> FusedStream for Filter<St, F>
where
St: FusedStream,
F: FnMut(&St::Item) -> bool,
{
fn is_terminated(&self) -> bool {
self.stream.is_terminated()
}
}
+37
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{ready, Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -30,6 +31,32 @@ impl<St, F> FilterMap<St, F> {
pub(super) fn new(stream: St, f: F) -> Self {
Self { stream, f }
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F, T> Stream for FilterMap<St, F>
@@ -56,3 +83,13 @@ where
(0, self.stream.size_hint().1) // can't know a lower bound, due to the predicate
}
}
impl<St, F, T> FusedStream for FilterMap<St, F>
where
St: FusedStream,
F: FnMut(St::Item) -> Option<T>,
{
fn is_terminated(&self) -> bool {
self.stream.is_terminated()
}
}
+10
View File
@@ -1,5 +1,6 @@
use crate::Stream;
use futures_core::FusedStream;
use pin_project_lite::pin_project;
use std::pin::Pin;
use std::task::{ready, Context, Poll};
@@ -51,3 +52,12 @@ where
}
}
}
impl<T> FusedStream for Fuse<T>
where
T: Stream,
{
fn is_terminated(&self) -> bool {
self.stream.is_none()
}
}
+37
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -28,6 +29,32 @@ impl<St, F> Map<St, F> {
pub(super) fn new(stream: St, f: F) -> Self {
Map { stream, f }
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F, T> Stream for Map<St, F>
@@ -49,3 +76,13 @@ where
self.stream.size_hint()
}
}
impl<St, F, T> FusedStream for Map<St, F>
where
St: FusedStream,
F: FnMut(St::Item) -> T,
{
fn is_terminated(&self) -> bool {
self.stream.is_terminated()
}
}
+59 -2
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -12,6 +13,7 @@ pin_project! {
#[pin]
stream: St,
f: F,
done: bool,
}
}
@@ -22,13 +24,44 @@ where
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("MapWhile")
.field("stream", &self.stream)
.field("done", &self.done)
.finish()
}
}
impl<St, F> MapWhile<St, F> {
pub(super) fn new(stream: St, f: F) -> Self {
MapWhile { stream, f }
MapWhile {
stream,
f,
done: false,
}
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
@@ -41,12 +74,36 @@ where
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<T>> {
let me = self.project();
if *me.done {
return Poll::Ready(None);
}
let f = me.f;
me.stream.poll_next(cx).map(|opt| opt.and_then(f))
let done = me.done;
me.stream.poll_next(cx).map(|opt| {
let mapped = opt.and_then(f);
if mapped.is_none() {
*done = true;
}
mapped
})
}
fn size_hint(&self) -> (usize, Option<usize>) {
if self.done {
return (0, Some(0));
}
let (_, upper) = self.stream.size_hint();
(0, upper)
}
}
impl<St, F> FusedStream for MapWhile<St, F>
where
Self: Stream,
{
fn is_terminated(&self) -> bool {
self.done
}
}
+11
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -57,6 +58,16 @@ where
}
}
impl<T, U> FusedStream for Merge<T, U>
where
T: Stream,
U: Stream<Item = T::Item>,
{
fn is_terminated(&self) -> bool {
self.a.is_terminated() && self.b.is_terminated()
}
}
fn poll_next<T, U>(
first: Pin<&mut T>,
second: Pin<&mut U>,
+35
View File
@@ -34,6 +34,33 @@ impl<T: Stream> Peekable<T> {
self.peek.as_ref()
}
}
/// Peek at the next item in the stream as a mutable reference.
pub async fn peek_mut(&mut self) -> Option<&mut T::Item>
where
T: Unpin,
{
if let Some(ref mut it) = self.peek {
Some(it)
} else {
self.peek = self.next().await;
self.peek.as_mut()
}
}
/// Poll to peek at the next item in the stream as a mutable reference.
pub fn poll_peek(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<&mut T::Item>> {
let mut this = self.project();
if this.peek.is_none() {
match this.stream.as_mut().poll_next(cx) {
Poll::Ready(item) => *this.peek = item,
Poll::Pending => return Poll::Pending,
}
}
Poll::Ready(this.peek.as_mut())
}
}
impl<T: Stream> Stream for Peekable<T> {
@@ -47,4 +74,12 @@ impl<T: Stream> Stream for Peekable<T> {
this.stream.poll_next(cx)
}
}
fn size_hint(&self) -> (usize, Option<usize>) {
let peek_len = if self.peek.is_some() { 1 } else { 0 };
let (lo, hi) = self.stream.size_hint();
let lo = lo.saturating_add(peek_len);
let hi = hi.and_then(|x| x.checked_add(peek_len));
(lo, hi)
}
}
+36
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{ready, Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -30,6 +31,32 @@ impl<St> Skip<St> {
pub(super) fn new(stream: St, remaining: usize) -> Self {
Self { stream, remaining }
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St> Stream for Skip<St>
@@ -61,3 +88,12 @@ where
(lower, upper)
}
}
impl<St> FusedStream for Skip<St>
where
St: FusedStream,
{
fn is_terminated(&self) -> bool {
self.stream.is_terminated()
}
}
+37
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{ready, Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -33,6 +34,32 @@ impl<St, F> SkipWhile<St, F> {
predicate: Some(predicate),
}
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F> Stream for SkipWhile<St, F>
@@ -71,3 +98,13 @@ where
(lower, upper)
}
}
impl<St, F> FusedStream for SkipWhile<St, F>
where
St: FusedStream,
F: FnMut(&St::Item) -> bool,
{
fn is_terminated(&self) -> bool {
self.stream.is_terminated()
}
}
+36
View File
@@ -4,6 +4,7 @@ use core::cmp;
use core::fmt;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -31,6 +32,32 @@ impl<St> Take<St> {
pub(super) fn new(stream: St, remaining: usize) -> Self {
Self { stream, remaining }
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St> Stream for Take<St>
@@ -74,3 +101,12 @@ where
(lower, upper)
}
}
impl<St> FusedStream for Take<St>
where
St: Stream,
{
fn is_terminated(&self) -> bool {
self.remaining == 0
}
}
+38 -7
View File
@@ -3,6 +3,7 @@ use crate::Stream;
use core::fmt;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -36,6 +37,32 @@ impl<St, F> TakeWhile<St, F> {
done: false,
}
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F> Stream for TakeWhile<St, F>
@@ -48,13 +75,7 @@ where
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
if !*self.as_mut().project().done {
self.as_mut().project().stream.poll_next(cx).map(|ready| {
let ready = ready.and_then(|item| {
if !(self.as_mut().project().predicate)(&item) {
None
} else {
Some(item)
}
});
let ready = ready.filter(self.as_mut().project().predicate);
if ready.is_none() {
*self.as_mut().project().done = true;
@@ -77,3 +98,13 @@ where
(0, upper)
}
}
impl<St, F> FusedStream for TakeWhile<St, F>
where
St: Stream,
F: FnMut(&St::Item) -> bool,
{
fn is_terminated(&self) -> bool {
self.done
}
}
+38
View File
@@ -4,6 +4,7 @@ use core::fmt;
use core::future::Future;
use core::pin::Pin;
use core::task::{Context, Poll};
use futures_core::FusedStream;
use pin_project_lite::pin_project;
pin_project! {
@@ -37,6 +38,32 @@ impl<St, Fut, F> Then<St, Fut, F> {
f,
}
}
/// Returns a reference to the inner stream.
pub fn get_ref(&self) -> &St {
&self.stream
}
/// Returns a mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_mut(&mut self) -> &mut St {
&mut self.stream
}
/// Returns a pinned mutable reference to the inner stream.
///
/// Mutating the inner stream may confuse this combinator.
pub fn get_pin_mut(self: Pin<&mut Self>) -> Pin<&mut St> {
self.project().stream
}
/// Consumes this combinator and returns the inner stream.
///
/// This may discard intermediate combinator state.
pub fn into_inner(self) -> St {
self.stream
}
}
impl<St, F, Fut> Stream for Then<St, Fut, F>
@@ -81,3 +108,14 @@ where
(lower, upper)
}
}
impl<St, F, Fut> FusedStream for Then<St, Fut, F>
where
St: FusedStream,
Fut: Future,
F: FnMut(St::Item) -> Fut,
{
fn is_terminated(&self) -> bool {
self.future.is_none() && self.stream.is_terminated()
}
}
+3 -3
View File
@@ -1,7 +1,7 @@
//! Slow down a stream by enforcing a delay between items.
use crate::Stream;
use tokio::time::{Duration, Instant, Sleep};
use tokio::time::{sleep, Duration, Sleep};
use std::future::Future;
use std::pin::Pin;
@@ -14,7 +14,7 @@ where
T: Stream,
{
Throttle {
delay: tokio::time::sleep_until(Instant::now() + duration),
delay: sleep(duration),
duration,
has_delayed: true,
stream,
@@ -81,7 +81,7 @@ impl<T: Stream> Stream for Throttle<T> {
if value.is_some() {
if !is_zero(dur) {
me.delay.reset(Instant::now() + dur);
me.delay.set(sleep(dur));
}
*me.has_delayed = false;
+4 -6
View File
@@ -1,6 +1,6 @@
use crate::stream_ext::Fuse;
use crate::Stream;
use tokio::time::{Instant, Sleep};
use tokio::time::{sleep, Sleep};
use core::future::Future;
use core::pin::Pin;
@@ -29,8 +29,7 @@ pub struct Elapsed(());
impl<S: Stream> Timeout<S> {
pub(super) fn new(stream: S, duration: Duration) -> Self {
let next = Instant::now() + duration;
let deadline = tokio::time::sleep_until(next);
let deadline = sleep(duration);
Timeout {
stream: Fuse::new(stream),
@@ -45,13 +44,12 @@ impl<S: Stream> Stream for Timeout<S> {
type Item = Result<S::Item, Elapsed>;
fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
let me = self.project();
let mut me = self.project();
match me.stream.poll_next(cx) {
Poll::Ready(v) => {
if v.is_some() {
let next = Instant::now() + *me.duration;
me.deadline.reset(next);
me.deadline.set(sleep(*me.duration));
*me.poll_deadline = true;
}
return Poll::Ready(v.map(Ok));
+7 -3
View File
@@ -632,6 +632,10 @@ where
should_loop = true;
idx = idx.wrapping_add(1) % self.entries.len();
if added == limit {
break;
}
}
Poll::Ready(None) => {
// Remove the entry
@@ -685,15 +689,15 @@ where
}
fn size_hint(&self) -> (usize, Option<usize>) {
let mut ret = (0, Some(0));
let mut ret: (usize, Option<usize>) = (0, Some(0));
for (_, stream) in &self.entries {
let hint = stream.size_hint();
ret.0 += hint.0;
ret.0 = ret.0.saturating_add(hint.0);
match (ret.1, hint.1) {
(Some(a), Some(b)) => ret.1 = Some(a + b),
(Some(a), Some(b)) => ret.1 = a.checked_add(b),
(Some(_), None) => ret.1 = None,
_ => {}
}
+5
View File
@@ -13,6 +13,11 @@ pub use mpsc_bounded::ReceiverStream;
mod mpsc_unbounded;
pub use mpsc_unbounded::UnboundedReceiverStream;
cfg_rt! {
mod task;
pub use task::JoinSetStream;
}
cfg_sync! {
mod broadcast;
pub use broadcast::BroadcastStream;
+78
View File
@@ -0,0 +1,78 @@
use crate::Stream;
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio::task::{JoinError, JoinSet};
/// A wrapper around [`tokio::task::JoinSet`] that implements [`Stream`].
///
/// # Example
///
/// ```
/// use tokio::task::JoinSet;
/// use tokio_stream::wrappers::JoinSetStream;
/// use tokio_stream::StreamExt;
///
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() -> Result<(), tokio::task::JoinError> {
/// let set: JoinSet<_> = (0..2).map(|i| async move { i }).collect();
///
/// let mut stream = JoinSetStream::new(set);
/// assert_eq!(stream.next().await.transpose()?, Some(0));
/// assert_eq!(stream.next().await.transpose()?, Some(1));
/// assert_eq!(stream.next().await.transpose()?, None);
/// # Ok(())
/// # }
/// ```
///
/// [`tokio::task::JoinSet`]: struct@tokio::task::JoinSet
/// [`Stream`]: trait@crate::Stream
#[derive(Debug)]
pub struct JoinSetStream<T> {
inner: JoinSet<T>,
}
impl<T> JoinSetStream<T> {
/// Create a new `JoinSetStream`.
pub fn new(join_set: JoinSet<T>) -> Self {
Self { inner: join_set }
}
/// Get back the inner `JoinSet`.
pub fn into_inner(self) -> JoinSet<T> {
self.inner
}
}
impl<T: 'static> Stream for JoinSetStream<T> {
type Item = Result<T, JoinError>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.inner.poll_join_next(cx)
}
/// Returns the bounds of the stream based on the underlying `JoinSet`.
///
/// It returns `(set.len(), Some(set.len()))`.
fn size_hint(&self) -> (usize, Option<usize>) {
let size = self.inner.len();
(size, Some(size))
}
}
impl<T> AsRef<JoinSet<T>> for JoinSetStream<T> {
fn as_ref(&self) -> &JoinSet<T> {
&self.inner
}
}
impl<T> AsMut<JoinSet<T>> for JoinSetStream<T> {
fn as_mut(&mut self) -> &mut JoinSet<T> {
&mut self.inner
}
}
impl<T> From<JoinSet<T>> for JoinSetStream<T> {
fn from(join_set: JoinSet<T>) -> Self {
Self::new(join_set)
}
}
+39
View File
@@ -0,0 +1,39 @@
#![cfg(feature = "rt")]
use futures::{Stream, StreamExt};
use std::collections::HashSet;
use tokio::task::JoinSet;
use tokio_stream::wrappers::JoinSetStream;
#[tokio::test]
async fn size_hint_stream() {
let set: JoinSet<_> = (0..2).map(|i| async move { i }).collect();
let mut stream = JoinSetStream::new(set);
assert_eq!(stream.size_hint(), (2, Some(2)));
stream.next().await;
assert_eq!(stream.size_hint(), (1, Some(1)));
stream.next().await;
assert_eq!(stream.size_hint(), (0, Some(0)));
}
#[tokio::test]
async fn join_set_as_stream() {
let set: JoinSet<_> = (0..2).map(|i| async move { i }).collect();
let stream = JoinSetStream::new(set);
let values: HashSet<_> = stream.map(|result| result.unwrap()).collect().await;
assert_eq!(values, HashSet::from([0, 1]));
}
// Cannot run this test when “unwind” is disabled
// since `JoinSet` use it to catch futures that panics.
#[cfg(panic = "unwind")]
#[tokio::test]
async fn join_set_as_stream_panics_with_error() {
let set: JoinSet<_> = std::iter::once(async move { panic!("boom!") }).collect();
let mut stream = JoinSetStream::new(set);
let result = stream.next().await.transpose();
assert!(matches!(result, Err(e) if e.is_panic()));
}
+152
View File
@@ -0,0 +1,152 @@
use std::pin::Pin;
use tokio_stream::{self as stream, Iter, StreamExt};
fn base() -> Iter<std::vec::IntoIter<i32>> {
stream::iter(vec![1, 2, 3])
}
#[tokio::test]
async fn map_accessors() {
let mut map = base().map(|x| x * 2);
let _: &Iter<_> = map.get_ref();
let _: &mut Iter<_> = map.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut map).get_pin_mut();
assert_eq!(map.next().await, Some(2));
// The recovered inner stream continues from where the combinator left it.
let mut inner = map.into_inner();
assert_eq!(inner.next().await, Some(2));
assert_eq!(inner.next().await, Some(3));
assert_eq!(inner.next().await, None);
}
#[tokio::test]
async fn take_accessors() {
let mut take = base().take(2);
let _: &Iter<_> = take.get_ref();
let _: &mut Iter<_> = take.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut take).get_pin_mut();
assert_eq!(take.next().await, Some(1));
// `take(2)` would stop after item 2, but the inner stream still has
// everything that was not yet pulled.
let mut inner = take.into_inner();
assert_eq!(inner.next().await, Some(2));
assert_eq!(inner.next().await, Some(3));
}
#[tokio::test]
async fn skip_accessors() {
let mut skip = base().skip(1);
let _: &Iter<_> = skip.get_ref();
let _: &mut Iter<_> = skip.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut skip).get_pin_mut();
assert_eq!(skip.next().await, Some(2));
let mut inner = skip.into_inner();
assert_eq!(inner.next().await, Some(3));
}
#[tokio::test]
async fn filter_accessors() {
let mut filter = base().filter(|&x| x % 2 == 1);
let _: &Iter<_> = filter.get_ref();
let _: &mut Iter<_> = filter.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut filter).get_pin_mut();
assert_eq!(filter.next().await, Some(1));
let mut inner = filter.into_inner();
assert_eq!(inner.next().await, Some(2));
}
#[tokio::test]
async fn filter_map_accessors() {
let mut filter_map = base().filter_map(|x| (x % 2 == 1).then_some(x * 10));
let _: &Iter<_> = filter_map.get_ref();
let _: &mut Iter<_> = filter_map.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut filter_map).get_pin_mut();
assert_eq!(filter_map.next().await, Some(10));
let mut inner = filter_map.into_inner();
assert_eq!(inner.next().await, Some(2));
}
#[tokio::test]
async fn map_while_accessors() {
let mut map_while = base().map_while(|x| (x < 3).then_some(x + 100));
let _: &Iter<_> = map_while.get_ref();
let _: &mut Iter<_> = map_while.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut map_while).get_pin_mut();
assert_eq!(map_while.next().await, Some(101));
let mut inner = map_while.into_inner();
assert_eq!(inner.next().await, Some(2));
}
#[tokio::test]
async fn take_while_accessors() {
let mut take_while = base().take_while(|&x| x < 3);
let _: &Iter<_> = take_while.get_ref();
let _: &mut Iter<_> = take_while.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut take_while).get_pin_mut();
assert_eq!(take_while.next().await, Some(1));
let mut inner = take_while.into_inner();
assert_eq!(inner.next().await, Some(2));
}
#[tokio::test]
async fn skip_while_accessors() {
let mut skip_while = base().skip_while(|&x| x < 2);
let _: &Iter<_> = skip_while.get_ref();
let _: &mut Iter<_> = skip_while.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut skip_while).get_pin_mut();
assert_eq!(skip_while.next().await, Some(2));
let mut inner = skip_while.into_inner();
assert_eq!(inner.next().await, Some(3));
}
#[tokio::test]
async fn then_accessors() {
let mut then = base().then(|x| std::future::ready(x + 1));
let _: &Iter<_> = then.get_ref();
let _: &mut Iter<_> = then.get_mut();
let _: Pin<&mut Iter<_>> = Pin::new(&mut then).get_pin_mut();
assert_eq!(then.next().await, Some(2));
let mut inner = then.into_inner();
assert_eq!(inner.next().await, Some(2));
}
#[tokio::test]
async fn get_mut_can_mutate_inner() {
// Mutating through `get_mut` is observed by the combinator.
let mut map = base().map(|x| x * 2);
// Skip one item directly on the inner stream.
let inner = map.get_mut();
assert_eq!(inner.next().await, Some(1));
// The combinator now sees the stream from item 2 onwards.
assert_eq!(map.next().await, Some(4));
}
+7
View File
@@ -1,3 +1,4 @@
use futures_core::FusedStream;
use tokio_stream::{Stream, StreamExt};
use std::pin::Pin;
@@ -37,16 +38,22 @@ async fn basic_usage() {
// however, once it is fused
let mut stream = stream.fuse();
assert!(!stream.is_terminated());
assert_eq!(stream.size_hint(), (0, None));
assert_eq!(stream.next().await, Some(4));
assert!(!stream.is_terminated());
assert_eq!(stream.size_hint(), (0, None));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
// it will always return `None` after the first time.
assert_eq!(stream.size_hint(), (0, Some(0)));
assert_eq!(stream.next().await, None);
assert_eq!(stream.size_hint(), (0, Some(0)));
assert!(stream.is_terminated());
}
#[tokio::test]
+208
View File
@@ -0,0 +1,208 @@
use futures_core::FusedStream;
use tokio_stream::StreamExt;
// Helper: a fused base stream built from a vec
fn fused_iter<T>(items: Vec<T>) -> impl FusedStream<Item = T> {
tokio_stream::iter(items).fuse()
}
// ── map ──────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn map_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2]).map(|x| x * 2);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn map_terminated_after_inner_done() {
let mut stream = fused_iter(vec![1]).map(|x| x * 2);
assert_eq!(stream.next().await, Some(2));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── filter ───────────────────────────────────────────────────────────────────
#[tokio::test]
async fn filter_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2]).filter(|x| *x > 0);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn filter_terminated_after_inner_done() {
let mut stream = fused_iter(vec![1]).filter(|x| *x > 0);
assert_eq!(stream.next().await, Some(1));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── filter_map ───────────────────────────────────────────────────────────────
#[tokio::test]
async fn filter_map_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2]).filter_map(Some);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn filter_map_terminated_after_inner_done() {
let mut stream = fused_iter(vec![1]).filter_map(|x| Some(x * 10));
assert_eq!(stream.next().await, Some(10));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── skip ─────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn skip_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2, 3]).skip(1);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn skip_terminated_after_inner_done() {
let mut stream = fused_iter(vec![1, 2]).skip(1);
assert_eq!(stream.next().await, Some(2));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── skip_while ───────────────────────────────────────────────────────────────
#[tokio::test]
async fn skip_while_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2, 3]).skip_while(|x| *x < 2);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn skip_while_terminated_after_inner_done() {
let mut stream = fused_iter(vec![1, 2]).skip_while(|x| *x < 2);
assert_eq!(stream.next().await, Some(2));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── take ─────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn take_not_terminated_before_limit() {
let stream = tokio_stream::iter(vec![1, 2, 3]).take(2);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn take_terminated_when_remaining_zero() {
let mut stream = tokio_stream::iter(vec![1, 2]).take(2);
assert_eq!(stream.next().await, Some(1));
assert!(!stream.is_terminated());
assert_eq!(stream.next().await, Some(2));
// remaining hits 0 after getting the second item
assert!(stream.is_terminated());
assert_eq!(stream.next().await, None);
}
#[tokio::test]
async fn take_zero_is_immediately_terminated() {
let stream = tokio_stream::iter(vec![1, 2]).take(0);
assert!(stream.is_terminated());
}
// ── take_while ───────────────────────────────────────────────────────────────
#[tokio::test]
async fn take_while_not_terminated_before_predicate_fails() {
let stream = tokio_stream::iter(vec![1, 2, 3]).take_while(|x| *x < 10);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn take_while_terminated_after_predicate_fails() {
let mut stream = tokio_stream::iter(vec![1, 5, 2]).take_while(|x| *x < 3);
assert_eq!(stream.next().await, Some(1));
assert!(!stream.is_terminated());
// predicate fails on 5 → done flag set
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── map_while ─────────────────────────────────────────────────────────────────
#[tokio::test]
async fn map_while_not_terminated_before_closure_returns_none() {
let stream =
tokio_stream::iter(vec![1, 2, 3]).map_while(|x| if x < 10 { Some(x) } else { None });
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn map_while_terminated_after_closure_returns_none() {
let mut stream =
tokio_stream::iter(vec![1, 5, 2]).map_while(|x| if x < 3 { Some(x) } else { None });
assert_eq!(stream.next().await, Some(1));
assert!(!stream.is_terminated());
// closure returns `None` on 5 → done flag set
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
}
// ── then ─────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn then_not_terminated_before_done() {
let stream = fused_iter(vec![1, 2]).then(|x| async move { x * 2 });
tokio::pin!(stream);
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn then_terminated_after_inner_done_and_no_pending_future() {
let stream = fused_iter(vec![1]).then(|x| async move { x * 2 });
tokio::pin!(stream);
assert_eq!(stream.next().await, Some(2));
assert_eq!(stream.next().await, None);
// inner stream done AND no in-flight future
assert!(stream.is_terminated());
}
// ── chain ────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn chain_not_terminated_while_either_has_items() {
let stream = fused_iter(vec![1]).chain(fused_iter(vec![2]));
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn chain_terminated_only_after_both_done() {
let mut stream = fused_iter(vec![1]).chain(fused_iter(vec![2]));
assert_eq!(stream.next().await, Some(1));
assert!(!stream.is_terminated()); // b still has items
assert_eq!(stream.next().await, Some(2));
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated()); // both done now
}
// ── merge ────────────────────────────────────────────────────────────────────
#[tokio::test]
async fn merge_not_terminated_while_either_has_items() {
let stream = fused_iter(vec![1]).merge(fused_iter(vec![2]));
assert!(!stream.is_terminated());
}
#[tokio::test]
async fn merge_terminated_only_after_both_done() {
let mut stream = fused_iter(vec![1]).merge(fused_iter(vec![2]));
// drain both
let mut collected = vec![];
while let Some(x) = stream.next().await {
collected.push(x);
}
assert_eq!(stream.next().await, None);
assert!(stream.is_terminated());
assert_eq!(collected.len(), 2);
}
+37 -3
View File
@@ -1,7 +1,6 @@
use tokio_stream as stream;
use tokio_test::task;
use std::iter;
use tokio_stream::{self as stream, Stream};
use tokio_test::{assert_pending, assert_ready, task};
#[tokio::test]
async fn coop() {
@@ -9,6 +8,7 @@ async fn coop() {
for _ in 0..10_000 {
if stream.poll_next().is_pending() {
tokio::task::yield_now().await;
assert!(stream.is_woken());
return;
}
@@ -16,3 +16,37 @@ async fn coop() {
panic!("did not yield");
}
#[tokio::test]
async fn test_iter_coop_budget() {
let mut stream = task::spawn(stream::iter(iter::repeat(1)));
// Tokio's default budget is 128.
// Fallback yield_amt is 32.
let limit = if cfg!(feature = "rt") { 128 } else { 32 };
for i in 0..limit {
let res = stream.poll_next();
assert!(res.is_ready(), "Should be ready at index {i}");
}
// Next poll should be pending
assert_pending!(stream.poll_next());
tokio::task::yield_now().await;
assert!(stream.is_woken());
}
#[tokio::test]
async fn test_iter_size_hint() {
let stream = stream::iter(vec![1, 2, 3]);
assert_eq!(stream.size_hint(), (3, Some(3)));
}
#[tokio::test]
async fn test_iter_eof_behavior() {
let mut stream = task::spawn(stream::iter(vec![1]));
assert_ready!(stream.poll_next());
assert_ready!(stream.poll_next()); // EOF should be ready None
}
+22
View File
@@ -0,0 +1,22 @@
use tokio_stream::StreamExt;
#[tokio::test]
async fn map_while_yields_until_closure_returns_none() {
let mut stream =
tokio_stream::iter(1..=10).map_while(|x| if x < 4 { Some(x + 3) } else { None });
assert_eq!(stream.next().await, Some(4));
assert_eq!(stream.next().await, Some(5));
assert_eq!(stream.next().await, Some(6));
assert_eq!(stream.next().await, None);
}
#[tokio::test]
async fn map_while_does_not_poll_after_closure_returns_none() {
// Once the closure returns `None`, the underlying stream must not be polled
// again, so the trailing `2` is never yielded.
let mut stream =
tokio_stream::iter(vec![1, 5, 2]).map_while(|x| if x < 3 { Some(x) } else { None });
assert_eq!(stream.next().await, Some(1));
assert_eq!(stream.next().await, None);
assert_eq!(stream.next().await, None);
}
+217
View File
@@ -0,0 +1,217 @@
use tokio_stream::{self as stream, Stream, StreamExt};
use tokio_test::{assert_pending, assert_ready, task};
use std::pin::Pin;
use std::task::{Context, Poll};
#[tokio::test]
async fn size_hint_without_peek() {
let mut s = stream::iter(vec![1, 2, 3]).peekable();
assert_eq!(s.size_hint(), (3, Some(3)));
s.next().await;
assert_eq!(s.size_hint(), (2, Some(2)));
s.next().await;
assert_eq!(s.size_hint(), (1, Some(1)));
s.next().await;
assert_eq!(s.size_hint(), (0, Some(0)));
}
#[tokio::test]
async fn size_hint_with_peek() {
let mut s = stream::iter(vec![1, 2, 3]).peekable();
// before peek: all items are in inner stream
assert_eq!(s.size_hint(), (3, Some(3)));
// after peek: one item moves into self.peek buffer — total must still be 3
let _ = s.peek().await;
assert_eq!(s.size_hint(), (3, Some(3)));
// consume the peeked item via next()
assert_eq!(s.next().await, Some(1));
assert_eq!(s.size_hint(), (2, Some(2)));
// peek again
let _ = s.peek().await;
assert_eq!(s.size_hint(), (2, Some(2)));
s.next().await;
assert_eq!(s.size_hint(), (1, Some(1)));
// peek the last item
let _ = s.peek().await;
assert_eq!(s.size_hint(), (1, Some(1)));
s.next().await;
assert_eq!(s.size_hint(), (0, Some(0)));
}
#[tokio::test]
async fn size_hint_empty_stream() {
let mut s = stream::iter(Vec::<i32>::new()).peekable();
assert_eq!(s.size_hint(), (0, Some(0)));
assert_eq!(s.peek().await, None);
assert_eq!(s.size_hint(), (0, Some(0)));
}
#[tokio::test]
async fn peek_returns_correct_item() {
let mut s = stream::iter(vec![10, 20, 30]).peekable();
assert_eq!(s.peek().await, Some(&10));
assert_eq!(s.peek().await, Some(&10)); // second peek returns same item
assert_eq!(s.next().await, Some(10)); // next() gives same item
assert_eq!(s.next().await, Some(20));
assert_eq!(s.next().await, Some(30));
assert_eq!(s.next().await, None);
}
#[tokio::test]
async fn size_hint_overflow() {
// When inner stream reports usize::MAX upper bound and an item is peeked,
// checked_add must return None rather than wrapping.
struct MaxHint(bool); // bool = whether to return one item
impl Stream for MaxHint {
type Item = ();
fn poll_next(
mut self: std::pin::Pin<&mut Self>,
_cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<()>> {
if self.0 {
self.0 = false;
std::task::Poll::Ready(Some(()))
} else {
std::task::Poll::Ready(None)
}
}
fn size_hint(&self) -> (usize, Option<usize>) {
(usize::MAX, Some(usize::MAX))
}
}
let mut s = MaxHint(true).peekable();
// before peek: delegates directly to inner
assert_eq!(s.size_hint(), (usize::MAX, Some(usize::MAX)));
// after peek: peek_len=1, checked_add(1) on usize::MAX must give None not panic
let _ = s.peek().await;
assert_eq!(s.size_hint(), (usize::MAX, None));
}
#[tokio::test]
async fn size_hint_unbounded_upper() {
// A stream that reports unknown upper bound
struct Unbounded;
impl Stream for Unbounded {
type Item = u32;
fn poll_next(
self: std::pin::Pin<&mut Self>,
_cx: &mut std::task::Context<'_>,
) -> std::task::Poll<Option<u32>> {
std::task::Poll::Ready(Some(42))
}
fn size_hint(&self) -> (usize, Option<usize>) {
(1, None)
}
}
let mut s = Unbounded.peekable();
assert_eq!(s.size_hint(), (1, None));
let _ = s.peek().await;
assert_eq!(s.size_hint(), (2, None)); // still unbounded after peek
}
#[tokio::test]
async fn peek_does_not_consume() {
let mut stream = stream::iter(vec![1, 2, 3]).peekable();
assert_eq!(stream.peek().await, Some(&1));
assert_eq!(stream.peek().await, Some(&1));
assert_eq!(stream.next().await, Some(1));
assert_eq!(stream.next().await, Some(2));
}
#[tokio::test]
async fn peek_mut_mutates_yielded_item() {
let mut stream = stream::iter(vec![1, 2, 3]).peekable();
if let Some(item) = stream.peek_mut().await {
*item += 10;
}
assert_eq!(stream.next().await, Some(11));
assert_eq!(stream.next().await, Some(2));
}
#[tokio::test]
async fn peek_on_empty_stream() {
let mut stream = stream::iter(Vec::<i32>::new()).peekable();
assert_eq!(stream.peek().await, None);
assert_eq!(stream.peek_mut().await, None);
assert_eq!(stream.next().await, None);
}
#[test]
fn poll_peek_does_not_advance_stream() {
let mut stream = task::spawn(stream::iter(vec![1, 2, 3]).peekable());
let first = stream.enter(|cx, s| s.poll_peek(cx).map(|opt| opt.copied()));
assert_eq!(assert_ready!(first), Some(1));
let second = stream.enter(|cx, s| s.poll_peek(cx).map(|opt| opt.copied()));
assert_eq!(assert_ready!(second), Some(1));
let next = stream.enter(|cx, s| s.poll_next(cx));
assert_eq!(assert_ready!(next), Some(1));
let next = stream.enter(|cx, s| s.poll_next(cx));
assert_eq!(assert_ready!(next), Some(2));
}
#[test]
fn poll_peek_mutates_buffered_item() {
let mut stream = task::spawn(stream::iter(vec![1, 2, 3]).peekable());
stream.enter(|cx, s| {
if let Poll::Ready(Some(item)) = s.poll_peek(cx) {
*item += 100;
}
});
let next = stream.enter(|cx, s| s.poll_next(cx));
assert_eq!(assert_ready!(next), Some(101));
}
struct PendingOnce {
polled: bool,
}
impl Stream for PendingOnce {
type Item = i32;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<i32>> {
if self.polled {
Poll::Ready(Some(7))
} else {
self.polled = true;
cx.waker().wake_by_ref();
Poll::Pending
}
}
}
#[test]
fn poll_peek_propagates_pending() {
let mut stream = task::spawn(PendingOnce { polled: false }.peekable());
assert_pending!(stream.enter(|cx, s| s.poll_peek(cx).map(|opt| opt.copied())));
let ready = stream.enter(|cx, s| s.poll_peek(cx).map(|opt| opt.copied()));
assert_eq!(assert_ready!(ready), Some(7));
}
+66
View File
@@ -225,6 +225,30 @@ fn size_hint_without_upper() {
assert_eq!(size_hint, (3, None));
}
#[test]
fn size_hint_overflow() {
struct Monster;
impl Stream for Monster {
type Item = ();
fn poll_next(self: Pin<&mut Self>, _cx: &mut std::task::Context<'_>) -> Poll<Option<()>> {
panic!()
}
fn size_hint(&self) -> (usize, Option<usize>) {
(usize::MAX, Some(usize::MAX))
}
}
let mut map = StreamMap::new();
map.insert("a", Monster);
map.insert("b", Monster);
assert_eq!(map.size_hint(), (usize::MAX, None));
}
#[test]
fn new_capacity_zero() {
let map = StreamMap::<&str, stream::Pending<()>>::new();
@@ -405,6 +429,27 @@ async fn poll_next_many_enough() {
assert!(buffer.contains(&(1, 1)));
}
#[tokio::test]
async fn poll_next_many_does_not_exceed_limit() {
let mut stream_map: StreamMap<usize, UsizeStream> = StreamMap::new();
stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);
stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);
let mut buffer = vec![];
let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 1)).await;
assert_eq!(n, 1);
assert_eq!(buffer.len(), 1);
let n = poll_fn(|cx| stream_map.poll_next_many(cx, &mut buffer, 1)).await;
assert_eq!(n, 1);
assert_eq!(buffer.len(), 2);
assert!(buffer.contains(&(0, 0)));
assert!(buffer.contains(&(1, 1)));
}
#[tokio::test]
async fn poll_next_many_correctly_loops_around() {
for _ in 0..10 {
@@ -519,6 +564,27 @@ async fn next_many_enough() {
assert!(buffer.contains(&(1, 1)));
}
#[tokio::test]
async fn next_many_does_not_exceed_limit() {
let mut stream_map: StreamMap<usize, UsizeStream> = StreamMap::new();
stream_map.insert(0, Box::pin(iter([0usize].into_iter())) as UsizeStream);
stream_map.insert(1, Box::pin(iter([1usize].into_iter())) as UsizeStream);
let mut buffer = vec![];
let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 1)).poll(cx)).await;
assert_eq!(n, 1);
assert_eq!(buffer.len(), 1);
let n = poll_fn(|cx| pin!(stream_map.next_many(&mut buffer, 1)).poll(cx)).await;
assert_eq!(n, 1);
assert_eq!(buffer.len(), 2);
assert!(buffer.contains(&(0, 0)));
assert!(buffer.contains(&(1, 1)));
}
#[tokio::test]
async fn next_many_correctly_loops_around() {
for _ in 0..10 {
+8
View File
@@ -107,3 +107,11 @@ async fn no_timeouts() {
assert_ready_eq!(stream.poll_next(), Some(Ok(5)));
assert_ready_eq!(stream.poll_next(), None);
}
#[tokio::test]
async fn duration_max_does_not_overflow() {
let stream = stream::iter([1]).timeout(Duration::MAX);
let mut stream = task::spawn(stream);
assert_ready_eq!(stream.poll_next(), Some(Ok(1)));
}
+7
View File
@@ -26,3 +26,10 @@ async fn usage() {
assert_ready!(stream.poll_next());
}
#[tokio::test]
async fn duration_max_does_not_overflow() {
let mut stream = task::spawn(futures::stream::iter([1]).throttle(Duration::MAX));
assert_ready_eq!(stream.poll_next(), Some(1));
}
+2 -8
View File
@@ -198,10 +198,7 @@ macro_rules! assert_ready_eq {
/// ```
#[macro_export]
macro_rules! assert_ok {
($e:expr) => {
assert_ok!($e,)
};
($e:expr,) => {{
($e:expr $(,)?) => {{
use std::result::Result::*;
match $e {
Ok(v) => v,
@@ -241,10 +238,7 @@ macro_rules! assert_ok {
/// ```
#[macro_export]
macro_rules! assert_err {
($e:expr) => {
assert_err!($e,);
};
($e:expr,) => {{
($e:expr $(,)?) => {{
use std::result::Result::*;
match $e {
Ok(v) => panic!("assertion failed: Ok({:?})", v),
+40
View File
@@ -70,6 +70,9 @@ const IDLE: usize = 0;
const WAKE: usize = 1;
const SLEEP: usize = 2;
/// Default maximum number of poll iterations in [`Spawn::poll_until_idle`].
const POLL_UNTIL_IDLE_MAX_ITERATIONS: usize = 150;
impl<T> Spawn<T> {
/// Consumes `self` returning the inner value
pub fn into_inner(self) -> T
@@ -123,6 +126,43 @@ impl<T: Future> Spawn<T> {
let fut = self.future.as_mut();
self.task.enter(|cx| fut.poll(cx))
}
/// Polls the future until it is idle.
///
/// A future is considered idle when it either completes, or returns
/// [`Poll::Pending`] without a pending wake notification.
///
/// Unlike [`poll`](Self::poll), this method keeps polling while the future
/// returns [`Poll::Pending`] but has received a wake notification, advancing
/// the future as far as possible without waiting for external events.
///
/// Polling is bounded to avoid infinite loops when a future wakes without
/// making progress.
///
/// # Panics
///
/// Panics if the iteration limit is exceeded.
///
/// # Example
///
/// ```
/// use tokio_test::task;
///
/// let mut task = task::spawn(async { 42 });
///
/// assert!(task.poll_until_idle().is_ready());
/// ```
pub fn poll_until_idle(&mut self) -> Poll<T::Output> {
for _ in 0..POLL_UNTIL_IDLE_MAX_ITERATIONS {
let result = self.poll();
if result.is_ready() || !self.is_woken() {
return result;
}
}
panic!(
"poll_until_idle exceeded {POLL_UNTIL_IDLE_MAX_ITERATIONS} iterations; future may be waking without making progress"
);
}
}
impl<T: Stream> Spawn<T> {
+79
View File
@@ -1,3 +1,4 @@
use std::future::{pending, Future};
use std::pin::Pin;
use std::task::{Context, Poll};
use tokio_stream::Stream;
@@ -23,3 +24,81 @@ fn test_spawn_stream_size_hint() {
let spawn = task::spawn(SizedStream);
assert_eq!(spawn.size_hint(), (100, Some(200)));
}
#[test]
fn poll_until_idle_ready() {
let mut task = task::spawn(async { 42 });
assert_eq!(task.poll_until_idle(), Poll::Ready(42));
}
#[test]
fn poll_until_idle_pending_not_woken() {
let mut task = task::spawn(pending::<()>());
assert!(task.poll_until_idle().is_pending());
}
struct WakeThenReady {
step: u8,
}
impl Future for WakeThenReady {
type Output = ();
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
match self.step {
0 => {
self.step = 1;
cx.waker().wake_by_ref();
Poll::Pending
}
_ => Poll::Ready(()),
}
}
}
#[test]
fn poll_until_idle_advances_on_wake() {
let mut task = task::spawn(WakeThenReady { step: 0 });
assert!(task.poll_until_idle().is_ready());
}
struct WakeNTimes {
remaining: u8,
}
impl Future for WakeNTimes {
type Output = u8;
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<u8> {
if self.remaining == 0 {
return Poll::Ready(0);
}
self.remaining -= 1;
cx.waker().wake_by_ref();
Poll::Pending
}
}
#[test]
fn poll_until_idle_multiple_wakes() {
let mut task = task::spawn(WakeNTimes { remaining: 3 });
assert_eq!(task.poll_until_idle(), Poll::Ready(0));
}
struct WakeForever;
impl Future for WakeForever {
type Output = ();
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<()> {
cx.waker().wake_by_ref();
Poll::Pending
}
}
#[test]
#[should_panic(expected = "poll_until_idle exceeded 150 iterations")]
fn poll_until_idle_panics_on_infinite_wake() {
let mut task = task::spawn(WakeForever);
let _ = task.poll_until_idle();
}
+36
View File
@@ -1,3 +1,39 @@
# 0.7.19 (July 21st, 2026)
### Added
- io: add `write_all_vectored` ([#7768], [#8159])
- sync: add `DropGuard::token` for cancellation tokens ([#8226])
- sync: implement `PartialEq` and `Eq` for `CancellationToken` ([#8110])
- task: add `AbortOnDrop` ([#7855])
- task: add `JoinMap::try_join_next` ([#8099])
### Changed
- codec: use `libc::memchr` for `LinesCodec` delimiter scan ([#8141])
### Fixed
- codec: fix `is_readable` when buffer is not empty ([#7912])
- task: avoid replacing the `JoinQueue` waker in `try_join_next` ([#8279])
- time: wake `DelayQueue` after resetting to expired ([#8274])
### Documented
- codec: document `UdpFramed` decoder errors ([#8248])
[#7768]: https://github.com/tokio-rs/tokio/pull/7768
[#7855]: https://github.com/tokio-rs/tokio/pull/7855
[#7912]: https://github.com/tokio-rs/tokio/pull/7912
[#8099]: https://github.com/tokio-rs/tokio/pull/8099
[#8110]: https://github.com/tokio-rs/tokio/pull/8110
[#8141]: https://github.com/tokio-rs/tokio/pull/8141
[#8159]: https://github.com/tokio-rs/tokio/pull/8159
[#8226]: https://github.com/tokio-rs/tokio/pull/8226
[#8248]: https://github.com/tokio-rs/tokio/pull/8248
[#8274]: https://github.com/tokio-rs/tokio/pull/8274
[#8279]: https://github.com/tokio-rs/tokio/pull/8279
# 0.7.18 (January 4th, 2026)
### Added
+5 -2
View File
@@ -4,7 +4,7 @@ name = "tokio-util"
# - Remove path dependencies (if any)
# - Update CHANGELOG.md.
# - Create "tokio-util-0.7.x" git tag.
version = "0.7.18"
version = "0.7.19"
edition = "2021"
rust-version = "1.71"
authors = ["Tokio Contributors <[email protected]>"]
@@ -25,7 +25,7 @@ full = ["codec", "compat", "io-util", "time", "net", "rt", "join-map"]
net = ["tokio/net"]
compat = ["futures-io"]
codec = []
codec = ["libc"]
time = ["tokio/time", "slab"]
io = []
io-util = ["io", "tokio/rt", "tokio/io-util"]
@@ -46,6 +46,9 @@ slab = { version = "0.4.4", optional = true } # Backs `DelayQueue`
tracing = { version = "0.1.29", default-features = false, features = ["std"], optional = true }
hashbrown = { version = "0.15.0", default-features = false, optional = true }
[target.'cfg(unix)'.dependencies]
libc = { version = "0.2.168", optional = true } # Backs the LinesCodec delimiter scan via libc::memchr
[dev-dependencies]
tokio = { version = "1.0.0", features = ["full"] }
tokio-test = "0.4.0"
+1 -1
View File
@@ -47,7 +47,7 @@ pub trait Decoder {
/// implementation of `decode_eof` to yield an `io::Error` when the decoder
/// fails to consume all available data.
///
/// Note that implementors of this trait can simply indicate `type Error =
/// Note that implementers of this trait can simply indicate `type Error =
/// io::Error` to use I/O errors as this type.
///
/// [`FramedRead`]: crate::codec::FramedRead
+60 -27
View File
@@ -493,8 +493,12 @@ impl LengthDelimitedCodec {
/// words, if a frame is currently in process of being decoded with a frame
/// size greater than `val` but less than the max frame length in effect
/// before calling this function, then the frame will be allowed.
///
/// If `val` is larger than what the length field can represent, it is
/// clipped to the maximum representable value.
pub fn set_max_frame_length(&mut self, val: usize) {
self.builder.max_frame_length(val);
self.builder.adjust_max_frame_len();
}
fn decode_head(&mut self, src: &mut BytesMut) -> io::Result<Option<usize>> {
@@ -599,10 +603,10 @@ impl Decoder for LengthDelimitedCodec {
}
}
impl Encoder<Bytes> for LengthDelimitedCodec {
impl Encoder<&[u8]> for LengthDelimitedCodec {
type Error = io::Error;
fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), io::Error> {
fn encode(&mut self, data: &[u8], dst: &mut BytesMut) -> Result<(), io::Error> {
let n = data.len();
if n > self.builder.max_frame_len {
@@ -627,8 +631,8 @@ impl Encoder<Bytes> for LengthDelimitedCodec {
})?;
// Reserve capacity in the destination buffer to fit the frame and
// length field (plus adjustment).
dst.reserve(self.builder.length_field_len + n);
// length field.
dst.reserve(self.builder.length_field_len + data.len());
if self.builder.length_field_is_big_endian {
dst.put_uint(n as u64, self.builder.length_field_len);
@@ -637,12 +641,20 @@ impl Encoder<Bytes> for LengthDelimitedCodec {
}
// Write the frame to the buffer
dst.extend_from_slice(&data[..]);
dst.extend_from_slice(data);
Ok(())
}
}
impl Encoder<Bytes> for LengthDelimitedCodec {
type Error = io::Error;
fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), io::Error> {
Encoder::<&[u8]>::encode(self, data.as_ref(), dst)
}
}
impl Default for LengthDelimitedCodec {
fn default() -> Self {
Self::new()
@@ -675,18 +687,21 @@ impl Builder {
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
/// use tokio_stream::StreamExt;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() {
/// let io: &[u8] = b"\x00\x0bhello world";
/// let mut reader = LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_type::<u16>()
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
///
/// let frame = reader.next().await.unwrap().unwrap();
/// assert_eq!(&frame[..], b"hello world");
/// # }
/// # pub fn main() {}
/// ```
pub fn new() -> Builder {
Builder {
@@ -949,14 +964,20 @@ impl Builder {
/// # Examples
///
/// ```
/// use tokio_util::codec::LengthDelimitedCodec;
/// use bytes::{Bytes, BytesMut};
/// use tokio_util::codec::{Decoder, Encoder, LengthDelimitedCodec};
///
/// # pub fn main() {
/// LengthDelimitedCodec::builder()
/// let mut codec = LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_type::<u16>()
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_codec();
///
/// let mut buf = BytesMut::new();
/// codec.encode(Bytes::from_static(b"hello world"), &mut buf).unwrap();
/// let frame = codec.decode(&mut buf).unwrap().unwrap();
/// assert_eq!(&frame[..], b"hello world");
/// # }
/// ```
pub fn new_codec(&self) -> LengthDelimitedCodec {
@@ -975,18 +996,21 @@ impl Builder {
/// # Examples
///
/// ```
/// # use tokio::io::AsyncRead;
/// use tokio_util::codec::LengthDelimitedCodec;
/// use tokio_stream::StreamExt;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// LengthDelimitedCodec::builder()
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() {
/// let io: &[u8] = b"\x00\x0bhello world";
/// let mut reader = LengthDelimitedCodec::builder()
/// .length_field_offset(0)
/// .length_field_type::<u16>()
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
///
/// let frame = reader.next().await.unwrap().unwrap();
/// assert_eq!(&frame[..], b"hello world");
/// # }
/// # pub fn main() {}
/// ```
pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
where
@@ -1040,7 +1064,7 @@ impl Builder {
fn num_head_bytes(&self) -> usize {
let num = self.length_field_offset + self.length_field_len;
cmp::max(num, self.num_skip.unwrap_or(0))
cmp::max(num, self.num_skip.unwrap_or_default())
}
fn get_num_skip(&self) -> usize {
@@ -1049,16 +1073,25 @@ impl Builder {
}
fn adjust_max_frame_len(&mut self) {
// Calculate the maximum number that can be represented using `length_field_len` bytes.
let max_number = match 1u64.checked_shl((8 * self.length_field_len) as u32) {
let max_allowed_len = self.max_allowed_frame_len();
if self.max_frame_len > max_allowed_len {
self.max_frame_len = max_allowed_len;
}
}
fn max_allowed_frame_len(&self) -> usize {
let max_allowed_len = self
.max_length_field_value()
.saturating_add_signed(self.length_adjustment as i64);
usize::try_from(max_allowed_len).unwrap_or(usize::MAX)
}
fn max_length_field_value(&self) -> u64 {
match 1u64.checked_shl((8 * self.length_field_len) as u32) {
Some(shl) => shl - 1,
None => u64::MAX,
};
let max_allowed_len = max_number.saturating_add_signed(self.length_adjustment as i64);
if self.max_frame_len as u64 > max_allowed_len {
self.max_frame_len = usize::try_from(max_allowed_len).unwrap_or(usize::MAX);
}
}
}
+1 -3
View File
@@ -115,9 +115,7 @@ impl Decoder for LinesCodec {
// there's no max_length set, we'll read to the end of the buffer.
let read_to = cmp::min(self.max_length.saturating_add(1), buf.len());
let newline_offset = buf[self.next_index..read_to]
.iter()
.position(|b| *b == b'\n');
let newline_offset = crate::util::memchr::memchr(b'\n', &buf[self.next_index..read_to]);
match (self.is_discarding, newline_offset) {
(true, Some(offset)) => {
+2 -2
View File
@@ -12,7 +12,7 @@
//! The following example demonstrates how to use a codec such as [`LinesCodec`] to
//! write framed data. [`FramedWrite`] can be used to achieve this. Data sent to
//! [`FramedWrite`] are first framed according to a specific codec, and then sent to
//! an implementor of [`AsyncWrite`].
//! an implementer of [`AsyncWrite`].
//!
//! ```
//! use futures::sink::SinkExt;
@@ -43,7 +43,7 @@
//! # Example decoding using `LinesCodec`
//! The following example demonstrates how to use a codec such as [`LinesCodec`] to
//! read a stream of framed data. [`FramedRead`] can be used to achieve this. [`FramedRead`]
//! will keep reading from an [`AsyncRead`] implementor until a whole frame, according to a codec,
//! will keep reading from an [`AsyncRead`] implementer until a whole frame, according to a codec,
//! can be parsed.
//!
//!```
+4
View File
@@ -291,6 +291,10 @@ impl AsyncWrite for Sender {
return Poll::Ready(Err(IoError::new(IoErrorKind::BrokenPipe, CLOSED_ERROR_MSG)));
}
if bufs.iter().all(|buf| buf.is_empty()) {
return Poll::Ready(Ok(0));
}
let free = inner
.backpressure_boundary
.checked_sub(inner.buf.len())
+10 -1
View File
@@ -158,6 +158,7 @@ pub struct StreamReader<S, B> {
inner: S,
// This field is not pinned.
chunk: Option<B>,
eof: bool,
}
impl<S, B, E> StreamReader<S, B>
@@ -179,6 +180,7 @@ where
Self {
inner: stream,
chunk: None,
eof: false,
}
}
@@ -277,6 +279,8 @@ where
// This unwrap is very sad, but it can't be avoided.
let buf = self.project().chunk.as_ref().unwrap().chunk();
return Poll::Ready(Ok(buf));
} else if *self.as_mut().project().eof {
return Poll::Ready(Ok(&[]));
} else {
match self.as_mut().project().inner.poll_next(cx) {
Poll::Ready(Some(Ok(chunk))) => {
@@ -284,7 +288,10 @@ where
*self.as_mut().project().chunk = Some(chunk);
}
Poll::Ready(Some(Err(err))) => return Poll::Ready(Err(err.into())),
Poll::Ready(None) => return Poll::Ready(Ok(&[])),
Poll::Ready(None) => {
*self.as_mut().project().eof = true;
return Poll::Ready(Ok(&[]));
}
Poll::Pending => return Poll::Pending,
}
}
@@ -311,6 +318,7 @@ impl<S: Unpin, B> Unpin for StreamReader<S, B> {}
struct StreamReaderProject<'a, S, B> {
inner: Pin<&'a mut S>,
chunk: &'a mut Option<B>,
eof: &'a mut bool,
}
impl<S, B> StreamReader<S, B> {
@@ -322,6 +330,7 @@ impl<S, B> StreamReader<S, B> {
StreamReaderProject {
inner: unsafe { Pin::new_unchecked(&mut me.inner) },
chunk: &mut me.chunk,
eof: &mut me.eof,
}
}
}
+2 -2
View File
@@ -147,8 +147,8 @@ fn advance_slices<'a>(bufs: &mut &mut [IoSlice<'a>], n: usize) {
*bufs = &mut std::mem::take(bufs)[remove..];
if let Some(first) = bufs.first_mut() {
let buf = &first[..left];
// necessary due to limitating in the borrow checker,
let buf = &first[left..];
// Necessary due to a limitation in the borrow checker,
// when tokio MSRV reaches 1.81.0 this entire function
// can be replaced with `IoSlice::advance_slices`
//
+20
View File
@@ -122,6 +122,26 @@ impl Clone for CancellationToken {
}
}
impl PartialEq for CancellationToken {
/// Checks if two tokens are equal in terms of their cancellation operation.
///
/// Two tokens are considered equal if cancelling one will always also cancel the other and vice
/// versa. This is only true for cloned tokens and not for tokens in a parent-child
/// relationship.
fn eq(&self, other: &CancellationToken) -> bool {
Arc::ptr_eq(&self.inner, &other.inner)
}
}
impl Eq for CancellationToken {}
impl core::hash::Hash for CancellationToken {
#[inline]
fn hash<H: core::hash::Hasher>(&self, state: &mut H) {
Arc::as_ptr(&self.inner).hash(state);
}
}
impl Drop for CancellationToken {
fn drop(&mut self) {
tree_node::decrease_handle_refcount(&self.inner);
@@ -10,6 +10,13 @@ pub struct DropGuard {
}
impl DropGuard {
/// Returns a reference to the cancellation token wrapped by this guard.
pub fn token(&self) -> &CancellationToken {
self.inner
.as_ref()
.expect("`inner` can only be None in a destructor")
}
/// Returns stored cancellation token and removes this drop guard instance
/// (i.e. it will no longer cancel token). Other guards for this token
/// are not affected.
@@ -13,6 +13,13 @@ pub struct DropGuardRef<'a> {
}
impl<'a> DropGuardRef<'a> {
/// Returns a reference to the cancellation token wrapped by this guard.
pub fn token(&self) -> &CancellationToken {
self.inner
.as_ref()
.expect("`inner` can only be None in a destructor")
}
/// Returns stored cancellation token and removes this drop guard instance
/// (i.e. it will no longer cancel token). Other guards for this token
/// are not affected.
+55 -1
View File
@@ -446,7 +446,7 @@ where
/// * `Some((key, Ok(value)))` if one of the tasks in this `JoinMap` has
/// completed. The `value` is the return value of that ask, and `key` is
/// the key associated with the task.
/// * `Some((key, Err(err))` if one of the tasks in this `JoinMap` has
/// * `Some((key, Err(err)))` if one of the tasks in this `JoinMap` has
/// panicked or been aborted. `key` is the key associated with the task
/// that panicked or was aborted.
/// * `None` if the `JoinMap` is empty.
@@ -468,6 +468,60 @@ where
}
}
/// Tries to join one of the tasks in the map that has completed and
/// returns its output, along with the key corresponding to that task.
///
/// Returns `None` if there are no completed tasks, or if the map is empty.
///
/// # Returns
///
/// This function returns:
///
/// * `Some((key, Ok(value)))` if one of the tasks in this `JoinMap` has
/// completed. The `value` is the return value of that task, and `key`
/// is the key associated with the task.
/// * `Some((key, Err(err)))` if one of the tasks in this `JoinMap` has
/// panicked or been aborted. `key` is the key associated with the task
/// that panicked or was aborted.
/// * `None` if there are no completed tasks ready to be joined, or the
/// `JoinMap` is empty.
///
/// # Examples
///
/// ```
/// use tokio_util::task::JoinMap;
///
/// # #[tokio::main(flavor = "current_thread")]
/// # async fn main() {
/// let mut map = JoinMap::new();
/// map.spawn("answer", async { 42 });
///
/// let (key, res) = loop {
/// if let Some(joined) = map.try_join_next() {
/// break joined;
/// }
/// tokio::task::yield_now().await;
/// };
///
/// assert_eq!(key, "answer");
/// assert_eq!(res.unwrap(), 42);
/// # }
/// ```
pub fn try_join_next(&mut self) -> Option<(K, Result<V, JoinError>)> {
loop {
let (res, id) = match self.tasks.try_join_next_with_id()? {
Ok((id, output)) => (Ok(output), id),
Err(e) => {
let id = e.id();
(Err(e), id)
}
};
if let Some(key) = self.remove_by_id(id) {
break Some((key, res));
}
}
}
/// Aborts all tasks and waits for them to finish shutting down.
///
/// Calling this method is equivalent to calling [`abort_all`] and then calling [`join_next`] in
+4
View File
@@ -188,6 +188,10 @@ impl<T> JoinQueue<T> {
/// Note that on success the handle will panic on subsequent polls
/// since it becomes consumed.
fn try_poll_handle(jh: &mut AbortOnDropHandle<T>) -> Option<Result<T, JoinError>> {
if !jh.is_finished() {
return None;
}
let waker = futures_util::task::noop_waker();
let mut cx = Context::from_waker(&waker);
+38 -32
View File
@@ -4,19 +4,18 @@ use std::fmt::{Debug, Formatter};
use std::future::Future;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use tokio::runtime::Builder;
use tokio::sync::mpsc::{unbounded_channel, UnboundedReceiver, UnboundedSender};
use tokio::sync::oneshot;
use tokio::task::{spawn_local, JoinHandle, LocalSet};
use tokio::task::{spawn_local, JoinHandle};
/// A cloneable handle to a local pool, used for spawning `!Send` tasks.
///
/// Internally the local pool uses a [`tokio::task::LocalSet`] for each worker thread
/// Internally the local pool uses a [`tokio::runtime::LocalRuntime`] for each worker thread
/// in the pool. Consequently you can also use [`tokio::task::spawn_local`] (which will
/// execute on the same thread) inside the Future you supply to the various spawn methods
/// of `LocalPoolHandle`.
///
/// [`tokio::task::LocalSet`]: tokio::task::LocalSet
/// [`tokio::runtime::LocalRuntime`]: tokio::runtime::LocalRuntime
/// [`tokio::task::spawn_local`]: tokio::task::spawn_local
///
/// # Examples
@@ -238,10 +237,10 @@ impl LocalPool {
let _abort_guard = AbortGuard(abort_handle);
// Inside the future we can't run spawn_local yet because we're not
// in the context of a LocalSet. We need to send create_task to the
// LocalSet task for spawning.
// in the context of a LocalRuntime. We need to send create_task to the
// LocalRuntime task for spawning.
let spawn_task = Box::new(move || {
// Once we're in the LocalSet context we can call spawn_local
// Once we're in the LocalRuntime context we can call spawn_local
let join_handle =
spawn_local(
async move { Abortable::new(create_task(), abort_registration).await },
@@ -255,7 +254,7 @@ impl LocalPool {
}
});
// Send the callback to the LocalSet task
// Send the callback to the LocalRuntime task
if let Err(e) = worker_spawner.send(spawn_task) {
// Propagate the error as a panic in the join handle.
panic!("Failed to send job to worker: {e}");
@@ -379,15 +378,17 @@ impl LocalWorkerHandle {
/// Create a new worker for executing pinned tasks
fn new_worker() -> LocalWorkerHandle {
let (sender, receiver) = unbounded_channel();
let runtime = Builder::new_current_thread()
.enable_all()
.build()
.expect("Failed to start a pinned worker thread runtime");
let runtime_handle = runtime.handle().clone();
let (handle_sender, handle_receiver) = std::sync::mpsc::channel();
let task_count = Arc::new(AtomicUsize::new(0));
let task_count_clone = Arc::clone(&task_count);
std::thread::spawn(|| Self::run(runtime, receiver, task_count_clone));
std::thread::spawn(|| Self::run(handle_sender, receiver, task_count_clone));
let runtime_handle = handle_receiver
.recv()
.expect("Failed to recv local runtime init result")
.expect("Failed to start local runtime");
LocalWorkerHandle {
runtime_handle,
@@ -397,28 +398,38 @@ impl LocalWorkerHandle {
}
fn run(
runtime: tokio::runtime::Runtime,
handle_sender: std::sync::mpsc::Sender<std::io::Result<tokio::runtime::Handle>>,
mut task_receiver: UnboundedReceiver<PinnedFutureSpawner>,
task_count: Arc<AtomicUsize>,
) {
let local_set = LocalSet::new();
local_set.block_on(&runtime, async {
let runtime = match tokio::runtime::LocalRuntime::new() {
Ok(runtime) => runtime,
Err(err) => {
let _ = handle_sender.send(Err(err));
return;
}
};
let runtime_handle = runtime.handle().clone();
handle_sender
.send(Ok(runtime_handle))
.expect("Failed to send local runtime handle");
drop(handle_sender);
runtime.block_on(async {
while let Some(spawn_task) = task_receiver.recv().await {
// Calls spawn_local(future)
(spawn_task)();
}
});
// If there are any tasks on the runtime associated with a LocalSet task
// that has already completed, but whose output has not yet been
// reported, let that task complete.
// If there are any tasks on the runtime that has already completed,
// but whose output has not yet been reported, let that task complete.
//
// Since the task_count is decremented when the runtime task exits,
// reading that counter lets us know if any such tasks completed during
// the call to `block_on`.
//
// Tasks on the LocalSet can't complete during this loop since they're
// stored on the LocalSet and we aren't accessing it.
let mut previous_task_count = task_count.load(Ordering::SeqCst);
loop {
// This call will also run tasks spawned on the runtime.
@@ -431,15 +442,10 @@ impl LocalWorkerHandle {
}
}
// It's now no longer possible for a task on the runtime to be
// associated with a LocalSet task that has completed. Drop both the
// LocalSet and runtime to let tasks on the runtime be cancelled if and
// only if they are still on the LocalSet.
//
// Drop the LocalSet task first so that anyone awaiting the runtime
// JoinHandle will see the cancelled error after the LocalSet task
// destructor has completed.
drop(local_set);
// It's now no longer possible for a task on the local runtime
// associated with task that has completed. Drop both
// local runtime to let tasks on the runtime be cancelled if and
// only if they are still on the runtime.
drop(runtime);
}
}
+42 -10
View File
@@ -580,12 +580,7 @@ impl<T> DelayQueue<T> {
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the queue is exhausted.
pub fn poll_expired(&mut self, cx: &mut task::Context<'_>) -> Poll<Option<Expired<T>>> {
if !self
.waker
.as_ref()
.map(|w| w.will_wake(cx.waker()))
.unwrap_or(false)
{
if !self.waker.as_ref().is_some_and(|w| w.will_wake(cx.waker())) {
self.waker = Some(cx.waker().clone());
}
@@ -864,11 +859,19 @@ impl<T> DelayQueue<T> {
self.slab[*key].expired = false;
self.insert_idx(when, *key);
let inserted_expired = self.slab[*key].expired;
let next_deadline = self.next_deadline();
if let (Some(ref mut delay), Some(deadline)) = (&mut self.delay, next_deadline) {
// This should awaken us if necessary (ie, if already expired)
delay.as_mut().reset(deadline);
match (next_deadline, &mut self.delay) {
(None, _) => self.delay = None,
(Some(deadline), Some(delay)) => delay.as_mut().reset(deadline),
(Some(deadline), None) => self.delay = Some(Box::pin(sleep_until(deadline))),
}
if inserted_expired {
if let Some(waker) = self.waker.take() {
waker.wake();
}
}
}
@@ -948,7 +951,7 @@ impl<T> DelayQueue<T> {
pub fn peek(&self) -> Option<Key> {
use self::wheel::Stack;
self.expired.peek().or_else(|| self.wheel.peek())
self.expired.peek().or_else(|| self.wheel.peek(&self.slab))
}
/// Returns the next time to poll as determined by the wheel.
@@ -1028,10 +1031,18 @@ impl<T> DelayQueue<T> {
/// # }
/// ```
pub fn clear(&mut self) {
let had_entries = !self.slab.is_empty();
self.slab.clear();
self.expired = Stack::default();
self.wheel = Wheel::new();
self.delay = None;
if had_entries {
if let Some(waker) = self.waker.take() {
waker.wake();
}
}
}
/// Returns the number of elements the queue can hold without reallocating.
@@ -1249,6 +1260,27 @@ impl<T> wheel::Stack for Stack<T> {
self.head
}
fn peek_earliest(&self, store: &Self::Store) -> Option<Self::Owned> {
let head = self.head?;
let mut earliest = (head, store[head].when);
let mut curr = store[head].next;
while let Some(key) = curr {
let data = &store[key];
// The comparison is strict so that the first entry seen wins a tie,
// which agrees with `pop` when every entry in the slot shares a
// deadline.
if data.when < earliest.1 {
earliest = (key, data.when);
}
curr = data.next;
}
Some(earliest.0)
}
#[track_caller]
fn remove(&mut self, item: &Self::Borrowed, store: &mut Self::Store) {
let key = *item;
+2 -2
View File
@@ -148,8 +148,8 @@ impl<T: Stack> Level<T> {
ret
}
pub(crate) fn peek_entry_slot(&self, slot: usize) -> Option<T::Owned> {
self.slot[slot].peek()
pub(crate) fn peek_entry_slot(&self, slot: usize, store: &T::Store) -> Option<T::Owned> {
self.slot[slot].peek_earliest(store)
}
}
+5 -6
View File
@@ -111,8 +111,7 @@ where
debug_assert!({
self.levels[level]
.next_expiration(self.elapsed)
.map(|e| e.deadline >= self.elapsed)
.unwrap_or(true)
.map_or(true, |e| e.deadline >= self.elapsed)
});
Ok(())
@@ -141,9 +140,9 @@ where
}
/// Next key that will expire
pub(crate) fn peek(&self) -> Option<T::Owned> {
pub(crate) fn peek(&self, store: &T::Store) -> Option<T::Owned> {
self.next_expiration()
.and_then(|expiration| self.peek_entry(&expiration))
.and_then(|expiration| self.peek_entry(&expiration, store))
}
/// Advances the timer up to the instant represented by `now`.
@@ -251,8 +250,8 @@ where
self.levels[expiration.level].pop_entry_slot(expiration.slot, store)
}
fn peek_entry(&self, expiration: &Expiration) -> Option<T::Owned> {
self.levels[expiration.level].peek_entry_slot(expiration.slot)
fn peek_entry(&self, expiration: &Expiration, store: &T::Store) -> Option<T::Owned> {
self.levels[expiration.level].peek_entry_slot(expiration.slot, store)
}
fn level_for(&self, when: u64) -> usize {
+7
View File
@@ -25,6 +25,13 @@ pub(crate) trait Stack: Default {
/// Peek into the stack.
fn peek(&self) -> Option<Self::Owned>;
/// Peek at the item in the stack with the earliest deadline.
///
/// Unlike `peek`, this does not have to agree with `pop`: a slot in a level
/// above zero spans a range of deadlines, so its entries are only ordered
/// once they cascade down.
fn peek_earliest(&self, store: &Self::Store) -> Option<Self::Owned>;
fn remove(&mut self, item: &Self::Borrowed, store: &mut Self::Store);
fn when(item: &Self::Borrowed, store: &Self::Store) -> u64;
+6
View File
@@ -22,6 +22,12 @@ use std::{
/// handle encoding and decoding of messages frames. Note that the incoming and
/// outgoing frame types may be distinct.
///
/// A single datagram may decode into multiple frames. `UdpFramed` will keep
/// calling [`Decoder::decode_eof`] with the current datagram until it returns
/// `Ok(None)`. If a decoder wants to discard a malformed datagram and continue
/// receiving later datagrams, it should consume or clear the remaining bytes
/// from the buffer before returning `Err`.
///
/// This function returns a *single* object that is both [`Stream`] and [`Sink`];
/// grouping this into a single object is often useful for layering things which
/// require both read and write access to the underlying object.
+102
View File
@@ -0,0 +1,102 @@
//! Search for a byte in a byte array using libc.
//!
//! When nothing pulls in libc, then just use a trivial implementation. Note
//! that we only depend on libc on unix.
#[cfg(not(all(unix, feature = "libc")))]
fn memchr_inner(needle: u8, haystack: &[u8]) -> Option<usize> {
haystack.iter().position(|val| needle == *val)
}
#[cfg(all(unix, feature = "libc"))]
fn memchr_inner(needle: u8, haystack: &[u8]) -> Option<usize> {
let start = haystack.as_ptr();
// SAFETY: `start` is valid for `haystack.len()` bytes.
let ptr = (unsafe { libc::memchr(start.cast(), needle as _, haystack.len()) })
.cast::<u8>()
.cast_const();
if ptr.is_null() {
None
} else {
// SAFETY: `ptr` will always be in bounds, since libc guarantees that the ptr will either
// be to an element inside the array or the ptr will be null
// since the ptr is in bounds the offset must also always be non null
// and there can't be more than isize::MAX elements inside an array
// as rust guarantees that the maximum number of bytes a allocation
// may occupy is isize::MAX
unsafe {
// TODO(MSRV 1.87): When bumping MSRV, switch to `ptr.byte_offset_from_unsigned(start)`.
Some(usize::try_from(ptr.offset_from(start)).unwrap_unchecked())
}
}
}
pub(crate) fn memchr(needle: u8, haystack: &[u8]) -> Option<usize> {
let index = memchr_inner(needle, haystack)?;
// SAFETY: `memchr_inner` returns Some(index) and in that case index must point to an element in haystack
// or `memchr_inner` None which is guarded by the `?` operator above
// therefore the index must **always** point to an element in the array
// and so this indexing operation is safe
// TODO(MSRV 1.81): When bumping MSRV, switch to `std::hint::assert_unchecked(haystack.get(..=index).is_some());`
unsafe {
if haystack.get(..=index).is_none() {
std::hint::unreachable_unchecked()
}
}
Some(index)
}
#[cfg(test)]
mod tests {
use super::memchr;
#[test]
fn memchr_test() {
let haystack = b"123abc456\0\xffabc\n";
assert_eq!(memchr(b'1', haystack), Some(0));
assert_eq!(memchr(b'2', haystack), Some(1));
assert_eq!(memchr(b'3', haystack), Some(2));
assert_eq!(memchr(b'4', haystack), Some(6));
assert_eq!(memchr(b'5', haystack), Some(7));
assert_eq!(memchr(b'6', haystack), Some(8));
assert_eq!(memchr(b'7', haystack), None);
assert_eq!(memchr(b'a', haystack), Some(3));
assert_eq!(memchr(b'b', haystack), Some(4));
assert_eq!(memchr(b'c', haystack), Some(5));
assert_eq!(memchr(b'd', haystack), None);
assert_eq!(memchr(b'A', haystack), None);
assert_eq!(memchr(0, haystack), Some(9));
assert_eq!(memchr(0xff, haystack), Some(10));
assert_eq!(memchr(0xfe, haystack), None);
assert_eq!(memchr(1, haystack), None);
assert_eq!(memchr(b'\n', haystack), Some(14));
assert_eq!(memchr(b'\r', haystack), None);
}
#[test]
fn memchr_all() {
let mut arr = Vec::new();
for b in 0..=255 {
arr.push(b);
}
for b in 0..=255 {
assert_eq!(memchr(b, &arr), Some(b as usize));
}
arr.reverse();
for b in 0..=255 {
assert_eq!(memchr(b, &arr), Some(255 - b as usize));
}
}
#[test]
fn memchr_empty() {
for b in 0..=255 {
assert_eq!(memchr(b, b""), None);
}
}
}
+2
View File
@@ -1,4 +1,6 @@
mod maybe_dangling;
#[cfg(feature = "codec")]
pub(crate) mod memchr;
#[cfg(any(feature = "io", feature = "codec"))]
mod poll_buf;
+13
View File
@@ -354,3 +354,16 @@ async fn poll_write_vectored_3() {
let n = assert_ready!(tx.poll_write_vectored(&mut noop_context(), io_slices)).unwrap();
assert_eq!(n, 0);
}
/// The `Sender::poll_write_vectored` should return `Poll::Ready(Ok(0))`
/// if all the input buffers have zero length, even when the channel is full.
#[tokio::test]
async fn poll_write_vectored_4() {
let (mut tx, _rx) = simplex::new(1);
tx.write_all(&[1]).await.unwrap();
let io_slices = &[IoSlice::new(&[]), IoSlice::new(&[])];
tokio::pin!(tx);
let n = assert_ready!(tx.poll_write_vectored(&mut noop_context(), io_slices)).unwrap();
assert_eq!(n, 0);
}
+19
View File
@@ -33,3 +33,22 @@ async fn test_stream_reader() -> std::io::Result<()> {
Ok(())
}
#[tokio::test]
async fn test_stream_reader_does_not_poll_after_eof() -> std::io::Result<()> {
// the first poll of this stream will return `Poll::Ready(None)`,
// and the second poll will panic
let stream = futures::stream::unfold((), |_| async { None::<(std::io::Result<Bytes>, ())> });
let read = StreamReader::new(stream);
tokio::pin!(read);
let mut buf = [0; 1];
// the first poll hits the inner stream,
// and the inner stream returns `Poll::Ready(None)`.
assert_eq!(read.read(&mut buf).await?, 0);
// the second poll doesn't hit the inner stream,
// so this `.read()` doesn't panic.
assert_eq!(read.read(&mut buf).await?, 0);
Ok(())
}
+75
View File
@@ -10,6 +10,55 @@ use std::io::IoSlice;
use std::pin::Pin;
use std::task::{Context, Poll};
struct PartialVectoredWriter {
buf: BytesMut,
max_write: usize,
}
impl AsyncWrite for PartialVectoredWriter {
fn poll_write(
self: Pin<&mut Self>,
_cx: &mut Context<'_>,
_buf: &[u8],
) -> Poll<io::Result<usize>> {
panic!("shouldn't be called")
}
fn poll_flush(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Ok(()).into()
}
fn poll_shutdown(self: Pin<&mut Self>, _cx: &mut Context<'_>) -> Poll<io::Result<()>> {
Ok(()).into()
}
fn poll_write_vectored(
mut self: Pin<&mut Self>,
_cx: &mut Context<'_>,
bufs: &[io::IoSlice<'_>],
) -> Poll<Result<usize, io::Error>> {
let mut remaining = self.max_write;
let mut written = 0;
for buf in bufs {
if remaining == 0 {
break;
}
let n = remaining.min(buf.len());
self.buf.extend_from_slice(&buf[..n]);
remaining -= n;
written += n;
}
Ok(written).into()
}
fn is_write_vectored(&self) -> bool {
true
}
}
#[tokio::test]
async fn test_write_all_vectored() {
struct Wr {
@@ -140,3 +189,29 @@ async fn write_all_vectored_with_empty_slice() {
write_all_vectored(&mut wr, buf).await.unwrap();
assert_eq!(&wr.buf[..], b"hello");
}
#[tokio::test]
async fn write_all_vectored_should_continue_with_unwritten_suffix_if_write_stops_inside_buffer() {
let mut wr = PartialVectoredWriter {
buf: BytesMut::with_capacity(64),
max_write: 3,
};
let buf = &mut [IoSlice::new(b"hello"), IoSlice::new(b"world")];
write_all_vectored(&mut wr, buf).await.unwrap();
assert_eq!(&wr.buf[..], b"helloworld");
}
#[tokio::test]
async fn write_all_vectored_should_continue_with_next_buffer_if_write_ends_on_boundary() {
let mut wr = PartialVectoredWriter {
buf: BytesMut::with_capacity(64),
max_write: 2,
};
let buf = &mut [IoSlice::new(b"ab"), IoSlice::new(b"cd")];
write_all_vectored(&mut wr, buf).await.unwrap();
assert_eq!(&wr.buf[..], b"abcd");
}
+100 -31
View File
@@ -70,6 +70,26 @@ macro_rules! assert_done {
}};
}
fn poll_ready_bytes<T>(
sink: Pin<&mut T>,
cx: &mut Context<'_>,
) -> Poll<Result<(), <T as Sink<Bytes>>::Error>>
where
T: Sink<Bytes>,
{
Sink::<Bytes>::poll_ready(sink, cx)
}
fn poll_flush_bytes<T>(
sink: Pin<&mut T>,
cx: &mut Context<'_>,
) -> Poll<Result<(), <T as Sink<Bytes>>::Error>>
where
T: Sink<Bytes>,
{
Sink::<Bytes>::poll_flush(sink, cx)
}
#[test]
fn read_empty_io_yields_nothing() {
let io = Box::pin(FramedRead::new(mock!(), LengthDelimitedCodec::new()));
@@ -423,9 +443,9 @@ fn write_single_frame_length_adjusted() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
@@ -436,7 +456,7 @@ fn write_nothing_yields_nothing() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io, cx));
});
}
@@ -453,9 +473,31 @@ fn write_single_frame_one_packet() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
#[test]
fn write_single_frame_from_slice() {
let io = FramedWrite::new(
mock! {
data(b"\x00\x00\x00\x09"),
data(b"abcdefghi"),
flush(),
},
LengthDelimitedCodec::new(),
);
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
let data: &[u8] = b"abcdefghi";
assert_ready_ok!(Sink::<&[u8]>::poll_ready(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(data));
assert_ready_ok!(Sink::<&[u8]>::poll_flush(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
@@ -477,16 +519,16 @@ fn write_single_multi_frame_one_packet() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("123")));
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("hello world")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
@@ -510,20 +552,20 @@ fn write_single_multi_frame_multi_packet() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("123")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("hello world")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
@@ -544,12 +586,12 @@ fn write_single_frame_would_block() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_pending!(io.as_mut().poll_flush(cx));
assert_pending!(io.as_mut().poll_flush(cx));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_pending!(poll_flush_bytes(io.as_mut(), cx));
assert_pending!(poll_flush_bytes(io.as_mut(), cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
@@ -567,10 +609,10 @@ fn write_single_frame_little_endian() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
}
@@ -587,10 +629,10 @@ fn write_single_frame_with_short_length_field() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdefghi")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
@@ -604,7 +646,7 @@ fn write_max_frame_len() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_err!(io.as_mut().start_send(Bytes::from("abcdef")));
assert!(io.get_ref().calls.is_empty());
@@ -621,10 +663,10 @@ fn write_update_max_frame_len_at_rest() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdef")));
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
io.encoder_mut().set_max_frame_length(5);
@@ -646,14 +688,14 @@ fn write_update_max_frame_len_in_flight() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdef")));
assert_pending!(io.as_mut().poll_flush(cx));
assert_pending!(poll_flush_bytes(io.as_mut(), cx));
io.encoder_mut().set_max_frame_length(5);
assert_ready_ok!(io.as_mut().poll_flush(cx));
assert_ready_ok!(poll_flush_bytes(io.as_mut(), cx));
assert_err!(io.as_mut().start_send(Bytes::from("abcdef")));
assert!(io.get_ref().calls.is_empty());
@@ -666,10 +708,10 @@ fn write_zero() {
pin_mut!(io);
task::spawn(()).enter(|cx, _| {
assert_ready_ok!(io.as_mut().poll_ready(cx));
assert_ready_ok!(poll_ready_bytes(io.as_mut(), cx));
assert_ok!(io.as_mut().start_send(Bytes::from("abcdef")));
assert_ready_err!(io.as_mut().poll_flush(cx));
assert_ready_err!(poll_flush_bytes(io.as_mut(), cx));
assert!(io.get_ref().calls.is_empty());
});
@@ -699,6 +741,33 @@ fn frame_does_not_fit() {
assert_eq!(codec.max_frame_length(), 255);
}
#[test]
fn runtime_max_frame_len_respects_length_field() {
for (adjustment, max_frame_len) in [(-1, 254), (0, 255), (1, 256)] {
let mut codec = LengthDelimitedCodec::builder()
.length_field_length(1)
.length_adjustment(adjustment)
.new_codec();
codec.set_max_frame_length(1_000);
assert_eq!(codec.max_frame_length(), max_frame_len);
let mut dst = BytesMut::new();
codec
.encode(Bytes::from(vec![0; max_frame_len]), &mut dst)
.unwrap();
assert_eq!(dst[0], u8::MAX);
let mut dst = BytesMut::from(&b"prefix"[..]);
let original = dst.clone();
let result = codec.encode(Bytes::from(vec![0; max_frame_len + 1]), &mut dst);
assert!(result.is_err());
assert_eq!(result.unwrap_err().kind(), io::ErrorKind::InvalidInput);
assert_eq!(dst, original);
}
}
#[test]
fn neg_adjusted_frame_does_not_fit() {
let codec = LengthDelimitedCodec::builder()
@@ -5,8 +5,10 @@ use tokio::sync::oneshot;
use tokio_util::sync::{CancellationToken, WaitForCancellationFuture};
use core::future::Future;
use core::hash::Hash;
use core::task::{Context, Poll};
use futures_test::task::new_count_waker;
use std::hash::{DefaultHasher, Hasher};
#[test]
fn cancel_token() {
@@ -563,3 +565,51 @@ fn run_until_cancelled_owned_test() {
);
}
}
#[test]
fn cloned_cancellation_tokens_are_considered_equal() {
let token = CancellationToken::new();
let token_clone = token.clone();
assert_eq!(token, token_clone);
}
#[test]
fn child_cancellation_tokens_are_not_considered_equal() {
let token = CancellationToken::new();
let token_clone = token.child_token();
assert_ne!(token, token_clone);
}
#[test]
fn independent_cancellation_tokens_are_not_considered_equal() {
let token1 = CancellationToken::new();
let token2 = CancellationToken::new();
assert_ne!(token1, token2);
}
#[test]
fn cloned_cancellation_tokens_have_same_hash() {
let token1 = CancellationToken::new();
let token2 = token1.clone();
let mut state1 = DefaultHasher::default();
token1.hash(&mut state1);
let mut state2 = DefaultHasher::default();
token2.hash(&mut state2);
assert_eq!(state1.finish(), state2.finish());
}
#[test]
fn different_cancellation_tokens_have_different_hash() {
let token1 = CancellationToken::new();
let token2 = CancellationToken::new();
let mut state1 = DefaultHasher::default();
token1.hash(&mut state1);
let mut state2 = DefaultHasher::default();
token2.hash(&mut state2);
assert_ne!(state1.finish(), state2.finish());
}
+125 -2
View File
@@ -358,6 +358,131 @@ async fn abort_all() {
}
}
#[tokio::test]
async fn try_join_next_empty() {
let mut map: JoinMap<usize, ()> = JoinMap::new();
assert!(map.try_join_next().is_none());
}
#[tokio::test]
async fn try_join_next_no_ready_task() {
let mut map = JoinMap::new();
let (_tx, rx) = oneshot::channel::<()>();
map.spawn("pending", async move {
let _ = rx.await;
});
// Task is not yet ready.
assert!(map.try_join_next().is_none());
assert_eq!(map.len(), 1);
}
#[tokio::test]
async fn try_join_next_completed_task() {
let mut map = JoinMap::new();
map.spawn("hello", async { 42 });
let mut got = None;
while got.is_none() {
got = map.try_join_next();
if got.is_none() {
tokio::task::yield_now().await;
}
}
let (key, res) = got.unwrap();
assert_eq!(key, "hello");
assert_eq!(res.unwrap(), 42);
assert!(map.is_empty());
}
#[tokio::test]
async fn try_join_next_aborted_task() {
let mut map = JoinMap::new();
map.spawn("forever", async {
futures::future::pending::<()>().await;
});
assert!(map.abort("forever"));
let mut got = None;
while got.is_none() {
got = map.try_join_next();
if got.is_none() {
tokio::task::yield_now().await;
}
}
let (key, res) = got.unwrap();
assert_eq!(key, "forever");
assert!(res.unwrap_err().is_cancelled());
assert!(map.is_empty());
}
#[tokio::test(flavor = "current_thread")]
async fn try_join_next_advances_through_multiple() {
const N: u32 = 8;
static SEM: tokio::sync::Semaphore = tokio::sync::Semaphore::const_new(0);
let mut map = JoinMap::new();
for i in 0..N {
map.spawn(i, async move {
SEM.add_permits(1);
i
});
}
// Wait until all tasks have signalled completion. On the current_thread
// runtime this means they have actually finished.
let _ = SEM.acquire_many(N).await.unwrap();
let mut seen = vec![false; N as usize];
let mut count = 0;
loop {
match map.try_join_next() {
Some((key, res)) => {
let v = res.expect("task should have completed successfully");
assert_eq!(key, v);
seen[v as usize] = true;
count += 1;
}
None if map.is_empty() => break,
None => tokio::task::yield_now().await,
}
}
assert_eq!(count, N);
assert!(seen.into_iter().all(|b| b));
assert!(map.try_join_next().is_none());
}
#[tokio::test]
async fn try_join_next_skips_replaced_task() {
let mut map = JoinMap::new();
let (tx1, rx1) = oneshot::channel::<()>();
map.spawn(1, async {
let _ = rx1.await;
11
});
tx1.send(()).unwrap();
tokio::task::yield_now().await;
let (tx2, rx2) = oneshot::channel::<()>();
map.spawn(1, async {
let _ = rx2.await;
22
});
tx2.send(()).unwrap();
tokio::task::yield_now().await;
let (key, res) = map.try_join_next().unwrap();
assert_eq!(key, 1);
assert_eq!(res.unwrap(), 22);
assert!(map.try_join_next().is_none());
assert!(map.is_empty());
}
#[tokio::test]
async fn duplicate_keys() {
let mut map = JoinMap::new();
@@ -456,7 +581,6 @@ mod spawn_local {
map.spawn_local((), async {});
}
#[cfg(tokio_unstable)]
mod local_runtime {
use super::*;
@@ -568,7 +692,6 @@ mod spawn_local {
mod spawn_local_on {
use super::*;
#[cfg(tokio_unstable)]
mod local_runtime {
use super::*;
+57
View File
@@ -1,5 +1,8 @@
#![warn(rust_2018_idioms)]
use std::task::Context;
use futures_test::task::new_count_waker;
use tokio::sync::oneshot;
use tokio::task::yield_now;
use tokio::time::Duration;
@@ -276,6 +279,60 @@ async fn test_join_queue_try_join_next() {
check_try_join_next_is_noop(&mut queue);
}
#[tokio::test]
async fn test_join_queue_try_join_next_does_not_replace_waker() {
let (send, recv) = oneshot::channel();
let mut queue = JoinQueue::new();
queue.spawn(async move {
recv.await.unwrap();
42
});
let (waker, wake_count) = new_count_waker();
let mut cx = Context::from_waker(&waker);
assert_pending!(queue.poll_join_next(&mut cx));
assert_eq!(wake_count, 0);
assert!(queue.try_join_next().is_none());
send.send(()).unwrap();
yield_now().await;
assert_eq!(wake_count, 1);
assert_eq!(
assert_ready!(queue.poll_join_next(&mut cx))
.unwrap()
.unwrap(),
42
);
}
#[tokio::test]
async fn test_join_queue_try_join_next_with_id_does_not_replace_waker() {
let (send, recv) = oneshot::channel();
let mut queue = JoinQueue::new();
queue.spawn(async move {
recv.await.unwrap();
42
});
let (waker, wake_count) = new_count_waker();
let mut cx = Context::from_waker(&waker);
assert_pending!(queue.poll_join_next_with_id(&mut cx));
assert_eq!(wake_count, 0);
assert!(queue.try_join_next_with_id().is_none());
send.send(()).unwrap();
yield_now().await;
assert_eq!(wake_count, 1);
let (_, output) = assert_ready!(queue.poll_join_next_with_id(&mut cx))
.unwrap()
.unwrap();
assert_eq!(output, 42);
}
#[tokio::test]
async fn test_join_queue_try_join_next_disabled_coop() {
// This number is large enough to trigger coop. Without using `tokio::task::coop::unconstrained`
-4
View File
@@ -1,7 +1,6 @@
#![warn(rust_2018_idioms)]
use futures::future::pending;
#[cfg(tokio_unstable)]
use std::rc::Rc;
use tokio::sync::mpsc;
use tokio::task::LocalSet;
@@ -182,7 +181,6 @@ fn notify_many() {
}
}
#[cfg(tokio_unstable)]
mod spawn {
use super::*;
@@ -209,7 +207,6 @@ mod spawn {
}
}
#[cfg(tokio_unstable)]
mod spawn_local {
use super::*;
@@ -278,7 +275,6 @@ mod spawn_local {
mod spawn_local_on {
use super::*;
#[cfg(tokio_unstable)]
mod local_runtime {
use super::*;
+70
View File
@@ -202,6 +202,27 @@ async fn reset_entry() {
assert!(entry.is_none())
}
#[tokio::test]
async fn reset_to_past_wakes_pending_queue() {
time::pause();
let mut queue = task::spawn(DelayQueue::new());
let key = queue.insert("foo", ms(10_000));
assert_pending!(poll!(queue));
assert!(!queue.is_woken());
queue.reset_at(&key, Instant::now() - ms(100));
assert!(queue.is_woken());
let entry = assert_ready_some!(poll!(queue));
assert_eq!(*entry.get_ref(), "foo");
let entry = assert_ready!(poll!(queue));
assert!(entry.is_none());
}
// Reproduces tokio-rs/tokio#849.
#[tokio::test]
async fn reset_much_later() {
@@ -881,6 +902,41 @@ async fn peek() {
assert!(queue.peek().is_none());
}
#[tokio::test(start_paused = true)]
async fn peek_entries_sharing_a_wheel_slot() {
// Only the lowest level of the timer wheel has one deadline per slot. A
// level-one slot spans 64ms, so entries with different deadlines share it
// and are only ordered once the slot cascades down.
let mut queue = task::spawn(DelayQueue::new());
let now = Instant::now();
// 64ms..=127ms all fall in the same level-one slot. Entries are pushed onto
// the front of a slot, so inserting the later deadline second puts it at the
// head.
let early = queue.insert_at("early", now + ms(100));
let late = queue.insert_at("late", now + ms(120));
assert_eq!(queue.peek(), Some(early));
sleep(ms(105)).await;
assert_eq!(queue.peek(), Some(early));
let entry = assert_ready_some!(poll!(queue));
assert_eq!(entry.key(), early);
assert_eq!(entry.get_ref(), &"early");
assert_eq!(queue.peek(), Some(late));
sleep(ms(20)).await;
let entry = assert_ready_some!(poll!(queue));
assert_eq!(entry.key(), late);
assert!(queue.peek().is_none());
}
#[tokio::test(start_paused = true)]
async fn wake_after_remove_last() {
let mut queue = task::spawn(DelayQueue::new());
@@ -894,6 +950,20 @@ async fn wake_after_remove_last() {
assert!(assert_ready!(poll!(queue)).is_none());
}
#[tokio::test(start_paused = true)]
async fn wake_after_clear() {
let mut queue = task::spawn(DelayQueue::new());
queue.insert("foo", ms(1000));
assert_pending!(poll!(queue));
assert!(!queue.is_woken());
queue.clear();
assert!(queue.is_woken());
assert!(assert_ready!(poll!(queue)).is_none());
}
fn ms(n: u64) -> Duration {
Duration::from_millis(n)
}
+1 -1
View File
@@ -1,6 +1,6 @@
#![warn(rust_2018_idioms)]
#![cfg(not(target_os = "wasi"))] // Wasi doesn't support UDP
#![cfg(not(miri))] // No `socket` in Miri.
#![cfg(not(miri))] // No UDP sockets in Miri.
#![cfg(not(loom))] // No udp / UdpFramed in loom
use tokio::net::UdpSocket;
+221
View File
@@ -1,3 +1,210 @@
# 1.53.1 (July 20th, 2026)
### Fixed
- signal: restore MSRV by removing `OnceLock::wait` from the Windows handler ([#8300])
### Fixed (unstable)
- time: fix alt timer cancellation and insertion race ([#8252])
### Documented
- runtime: remove dead link definition in Runtime::block_on ([#8301])
[#8252]: https://github.com/tokio-rs/tokio/pull/8252
[#8300]: https://github.com/tokio-rs/tokio/pull/8300
[#8301]: https://github.com/tokio-rs/tokio/pull/8301
# 1.53.0 (July 17th, 2026)
### Added
- fs: implement `From<OwnedFd>` and `From<OwnedHandle>` for `File` ([#8266])
- metrics: add task schedule latency metric ([#7986])
- net: add `SocketAddr` methods to Unix sockets ([#8144])
### Changed
- io: add `#[inline]` to IO trait impls for in-memory types ([#8242])
- net: implement UCred::pid on FreeBSD ([#8086])
- net: support Nuttx target os ([#8259])
- signal: refactor global variables on Windows ([#8231])
- sync: `mpsc::{Receiver,UnboundedReceiver}` now drops waker on drop, even if there are still senders ([#8095])
- taskdump: support taskdumps on s390x ([#8192])
- time: add `#[track_caller]` to `timeout_at()` ([#8077])
- time: consolidate mutex locks on spurious poll ([#8124])
- time: defer waker clone on spurious poll ([#8107])
- time: move lazy-registration state into `Sleep` ([#8132])
- tracing: remove unnecessary span clone ([#8126])
### Fixed
- io: do not treat zero-length reads as EOF in `Chain` ([#8251])
- net: use getpeereid for QNX peer credentials ([#8270])
- runtime: avoid illegal state in `FastRand` ([#8078])
- sync: wake mpsc receiver when a queued `reserve[_many]` returns permits ([#8260])
- taskdump: skip double wake on `Trace::capture`/`Trace::trace_with` ([#8043])
- time: avoid stack overflow in runtime constructor ([#8093])
- time (alt timer): ensure timers stay in the same runtime after `.reset()` ([#8169])
### IO uring (unstable)
- fs: use io-uring for `fs::try_exists` ([#8080])
- fs: use io-uring for renaming files ([#7800])
- rt: flush io-uring CQE in case of CQE overflow ([#8277])
### Documented
- docs: clarify cancel safety wording ([#8181])
- fs: clarify `create_dir_all` succeeds if path exists ([#8149])
- io: add warning about stdout reordering with multiple handles ([#8276])
- net: document pipe `try_read*`/`try_write*` readiness behavior ([#8032])
- runtime: document interaction with fork() ([#8202])
- sync: clarify broadcast lagging semantics ([#8239])
- sync: document memory ordering guarantees for Semaphore ([#8119])
- task: explain why `yield_now` defers its waker ([#8254])
- time: add panic docs to `timeout_at()` ([#8077])
- time: fix reversed poll order in timeout doc ([#8214])
[#7800]: https://github.com/tokio-rs/tokio/pull/7800
[#7986]: https://github.com/tokio-rs/tokio/pull/7986
[#8032]: https://github.com/tokio-rs/tokio/pull/8032
[#8043]: https://github.com/tokio-rs/tokio/pull/8043
[#8077]: https://github.com/tokio-rs/tokio/pull/8077
[#8078]: https://github.com/tokio-rs/tokio/pull/8078
[#8080]: https://github.com/tokio-rs/tokio/pull/8080
[#8086]: https://github.com/tokio-rs/tokio/pull/8086
[#8093]: https://github.com/tokio-rs/tokio/pull/8093
[#8095]: https://github.com/tokio-rs/tokio/pull/8095
[#8107]: https://github.com/tokio-rs/tokio/pull/8107
[#8119]: https://github.com/tokio-rs/tokio/pull/8119
[#8124]: https://github.com/tokio-rs/tokio/pull/8124
[#8126]: https://github.com/tokio-rs/tokio/pull/8126
[#8132]: https://github.com/tokio-rs/tokio/pull/8132
[#8144]: https://github.com/tokio-rs/tokio/pull/8144
[#8149]: https://github.com/tokio-rs/tokio/pull/8149
[#8169]: https://github.com/tokio-rs/tokio/pull/8169
[#8181]: https://github.com/tokio-rs/tokio/pull/8181
[#8192]: https://github.com/tokio-rs/tokio/pull/8192
[#8193]: https://github.com/tokio-rs/tokio/pull/8193
[#8202]: https://github.com/tokio-rs/tokio/pull/8202
[#8214]: https://github.com/tokio-rs/tokio/pull/8214
[#8231]: https://github.com/tokio-rs/tokio/pull/8231
[#8239]: https://github.com/tokio-rs/tokio/pull/8239
[#8242]: https://github.com/tokio-rs/tokio/pull/8242
[#8251]: https://github.com/tokio-rs/tokio/pull/8251
[#8254]: https://github.com/tokio-rs/tokio/pull/8254
[#8259]: https://github.com/tokio-rs/tokio/pull/8259
[#8260]: https://github.com/tokio-rs/tokio/pull/8260
[#8266]: https://github.com/tokio-rs/tokio/pull/8266
[#8270]: https://github.com/tokio-rs/tokio/pull/8270
[#8276]: https://github.com/tokio-rs/tokio/pull/8276
[#8277]: https://github.com/tokio-rs/tokio/pull/8277
# 1.52.4 (July 16th, 2026)
### Fixed
- runtime: don't skip the driver when `before_park` schedules work ([#8222])
### Fixed (unstable)
- taskdump: remove crate disambiguators from output ([#8264])
[#8264]: https://github.com/tokio-rs/tokio/pull/8264
# 1.52.3 (May 8th, 2026)
### Fixed
* sync: fix underflow in mpsc channel `len()` ([#8062])
* sync: notify receivers in mpsc `OwnedPermit::release()` method ([#8075])
* sync: require that an `RwLock` has `max_readers != 0` ([#8076])
* sync: return `Empty` from `try_recv()` when mpsc is closed with outstanding permits ([#8074])
# 1.52.2 (May 4th, 2026)
This release reverts the LIFO slot stealing change introduced in 1.51.0
([#7431]), due to [its performance impact][#8065]. ([#8100])
# 1.52.1 (April 16th, 2026)
## Fixed
- runtime: revert [#7757] to fix [a regression][#8056] that causes `spawn_blocking` to hang ([#8057])
[#7757]: https://github.com/tokio-rs/tokio/pull/7757
[#8056]: https://github.com/tokio-rs/tokio/pull/8056
[#8057]: https://github.com/tokio-rs/tokio/pull/8057
# 1.52.0 (April 14th, 2026)
## Added
- io: `AioSource::register_borrowed` for I/O safety support ([#7992])
- net: add `try_io` function to `unix::pipe` sender and receiver types ([#8030])
## Added (unstable)
- runtime: `Builder::enable_eager_driver_handoff` setting enable eager hand off of the I/O and time drivers before polling tasks ([#8010])
- taskdump: add `trace_with()` for customized task dumps ([#8025])
- taskdump: allow `impl FnMut()` in `trace_with` instead of just `fn()` ([#8040])
- fs: support `io_uring` in `AsyncRead` for `File` ([#7907])
## Changed
- runtime: improve `spawn_blocking` scalability with sharded queue ([#7757])
- runtime: use `compare_exchange_weak()` in worker queue ([#8028])
## Fixed
- runtime: overflow second half of tasks when local queue is filled instead of first half ([#8029])
## Documented
- docs: fix typo in `oneshot::Sender::send` docs ([#8026])
- docs: hide #[tokio::main] attribute in the docs of `sync::watch` ([#8035])
- net: add docs on `ConnectionRefused` errors with UDP sockets ([#7870])
[#7757]: https://github.com/tokio-rs/tokio/pull/7757
[#7870]: https://github.com/tokio-rs/tokio/pull/7870
[#7907]: https://github.com/tokio-rs/tokio/pull/7907
[#7992]: https://github.com/tokio-rs/tokio/pull/7992
[#8010]: https://github.com/tokio-rs/tokio/pull/8010
[#8025]: https://github.com/tokio-rs/tokio/pull/8025
[#8026]: https://github.com/tokio-rs/tokio/pull/8026
[#8028]: https://github.com/tokio-rs/tokio/pull/8028
[#8029]: https://github.com/tokio-rs/tokio/pull/8029
[#8030]: https://github.com/tokio-rs/tokio/pull/8030
[#8035]: https://github.com/tokio-rs/tokio/pull/8035
[#8040]: https://github.com/tokio-rs/tokio/pull/8040
# 1.51.4 (July 16th, 2026)
### Fixed
- runtime: don't skip the driver when `before_park` schedules work ([#8222])
[#8222]: https://github.com/tokio-rs/tokio/pull/8222
# 1.51.3 (May 8th, 2026)
### Fixed
* sync: fix underflow in mpsc channel `len()` ([#8062])
* sync: notify receivers in mpsc `OwnedPermit::release()` method ([#8075])
* sync: require that an `RwLock` has `max_readers != 0` ([#8076])
* sync: return `Empty` from `try_recv()` when mpsc is closed with outstanding permits ([#8074])
# 1.51.2 (May 4th, 2026)
This release reverts the LIFO slot stealing change introduced in 1.51.0
([#7431]), due to [its performance impact][#8065]. ([#8100])
[#8065]: https://github.com/tokio-rs/tokio/pull/8065
[#8100]: https://github.com/tokio-rs/tokio/pull/8100
# 1.51.1 (April 8th, 2026)
### Fixed
@@ -322,6 +529,20 @@ The MSRV is increased to 1.71.
[#7672]: https://github.com/tokio-rs/tokio/pull/7672
[#7675]: https://github.com/tokio-rs/tokio/pull/7675
# 1.47.5 (May 7th, 2026)
### Fixed
* sync: fix underflow in mpsc channel `len()` ([#8062])
* sync: notify receivers in mpsc `OwnedPermit::release()` method ([#8075])
* sync: require that an `RwLock` has `max_readers != 0` ([#8076])
* sync: return `Empty` from `try_recv()` when mpsc is closed with outstanding permits ([#8074])
[#8062]: https://github.com/tokio-rs/tokio/pull/8062
[#8074]: https://github.com/tokio-rs/tokio/pull/8074
[#8075]: https://github.com/tokio-rs/tokio/pull/8075
[#8076]: https://github.com/tokio-rs/tokio/pull/8076
# 1.47.4 (April 2nd, 2026)
### Fixed

Some files were not shown because too many files have changed in this diff Show More