Removes the `stream` feature flag from `Cargo.toml` and removes the
`futures-core` dependency. Once `Stream` lands in `std`, a feature flag
is most likely not needed.
This change removes all references to `Stream` from
within the `tokio` crate and moves them into a new
`tokio-stream` crate. Most types have had their
`impl Stream` removed as well in-favor of their
inherent methods.
Closes#2870
## Motivation
In Tokio 0.2, `AsyncRead` and `AsyncWrite` had `poll_write_buf` and
`poll_read_buf` methods for reading and writing to implementers of
`bytes` `Buf` and `BufMut` traits. In 0.3, these were removed, but
`poll_read_buf` was added as a free function in `tokio-util`. However,
there is currently no `poll_write_buf`.
Now that `AsyncWrite` has regained support for vectored writes in #3149,
there's a lot of potential benefit in having a `poll_write_buf` that
uses vectored writes when supported and non-vectored writes when not
supported, so that users don't have to reimplement this.
## Solution
This PR adds a `poll_write_buf` function to `tokio_util::io`, analogous
to the existing `poll_read_buf` function.
This function writes from a `Buf` to an `AsyncWrite`, advancing the
`Buf`'s internal cursor. In addition, when the `AsyncWrite` supports
vectored writes (i.e. its `is_write_vectored` method returns `true`),
it will use vectored IO.
I copied the documentation for this functions from the docs from Tokio
0.2's `AsyncWrite::poll_write_buf` , with some minor modifications as
appropriate.
Finally, I fixed a minor issue in the existing docs for `poll_read_buf`
and `read_buf`, and updated `tokio_util::codec` to use `poll_write_buf`.
Signed-off-by: Eliza Weisman <[email protected]>
This reverts commit fe2b997.
We are avoiding adding poll_read_buf to tokio itself for now. The patch is
reverted now in order to not block the v0.3.2 release (#3059).
This change is intended to do the minimum to unblock 0.3; as such, for now, we
duplicate the internal `time::wheel` structures in tokio-util, rather than trying
to refactor things at this stage.
Co-authored-by: Bryan Donlan <[email protected]>
This refactors I/O registration in a few ways:
- Cleans up the cached readiness in `PollEvented`. This cache used to
be helpful when readiness was a linked list of `*mut Node`s in
`Registration`. Previous refactors have turned `Registration` into just
an `AtomicUsize` holding the current readiness, so the cache is just
extra work and complexity. Gone.
- Polling the `Registration` for readiness now gives a `ReadyEvent`,
which includes the driver tick. This event must be passed back into
`clear_readiness`, so that the readiness is only cleared from `Registration`
if the tick hasn't changed. Previously, it was possible to clear the
readiness even though another thread had *just* polled the driver and
found the socket ready again.
- Registration now also contains an `async fn readiness`, which stores
wakers in an instrusive linked list. This allows an unbounded number
of tasks to register for readiness (previously, only 1 per direction (read
and write)). By using the intrusive linked list, there is no concern of
leaking the storage of the wakers, since they are stored inside the `async fn`
and released when the future is dropped.
- Registration retains a `poll_readiness(Direction)` method, to support
`AsyncRead` and `AsyncWrite`. They aren't able to use `async fn`s, and
so there are 2 reserved slots for those methods.
- IO types where it makes sense to have multiple tasks waiting on them
now take advantage of this new `async fn readiness`, such as `UdpSocket`
and `UnixDatagram`.
Additionally, this makes the `io-driver` "feature" internal-only (no longer
documented, not part of public API), and adds a second internal-only
feature, `io-readiness`, to group together linked list part of registration
that is only used by some of the IO types.
After a bit of discussion, changing stream-based transports (like
`TcpStream`) to have `async fn read(&self)` is punted, since that
is likely too easy of a footgun to activate.
Refs: #2779, #2728
* sync: move CancellationToken to tokio-util
The `CancellationToken` utility is only available with the
`tokio_unstable` flag. This was done as the API is not final, but it
adds friction for users.
This patch moves `CancellationToken` to tokio-util where it is generally
available. The tokio-util crate does not have any constraints on
breaking change releases.
* fix clippy
* clippy again
tokio-util uses tokio::stream::StreamExt, which was not introduced until
tokio v0.2.5. The current dependency specification is incorrect, and
breaks with cargo update -Z minimal-versions.
* util: add futures-io/tokio::io compatibility layer
This PR adds a compatibility layer with conversions between the
`tokio::io` and `futures-io` versions of the `AsyncRead` and
`AsyncWrite` traits.
I initially opened this PR against `tokio-compat`, but we decided that
a compatibility layer for current versions of the `tokio` and
`futures-io` crates (rather than for compatibility with legacy code)
ought to go in `tokio-util` instead. See:
https://github.com/tokio-rs/tokio-compat/pull/2#issuecomment-551310953
This is based on code originally written by @Nemo157 as part of the
`futures-tokio-compat` crate, and is contributed on behalf of the
original author:
https://github.com/Nemo157/futures-tokio-compat/issues/2#issuecomment-544118866Closestokio-rs/tokio-compat#2
Co-authored-by: Wim Looman <[email protected]>
Signed-off-by: Eliza Weisman <[email protected]>
Introduces `StreamExt` trait. This trait will be used to add utility functions
to make working with streams easier. This patch includes two functions:
* `next`: a future returning the item in the stream.
* `map`: transform each item in the stream.
Taiki Endo