Compare commits

...
Author SHA1 Message Date
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
Carl Lerche 11e2af66a8 Bump Tokio to v0.1.15. (#869)
Also bumps:

- tokio-sync (0.1.0)
- tokio-threadpool (0.1.11)
- tokio-timer (0.2.9)
2019-01-25 10:20:09 -08:00
Carl Lerche a4aae1459c chore: move enumerate test to correct location (#867) 2019-01-24 20:47:45 -08:00
Zahari Dichev 12546d1d9c tokio-timer: fix DelayQueue bug when inserting shorter delay (#863)
Reset the delay of the queue in case an item that expires sooner than the last inserted is put
into the queue.
2019-01-24 14:36:41 -08:00
Zahari Dichev fbad6297c5 Add enumerate combinator to Stream (#832) 2019-01-24 11:50:34 -08:00
Jon Gjengset 0ec8986b0b Make reason for try_send errors clearer (#864) 2019-01-23 15:06:24 -08:00
Jon Gjengset c6f8bdb249 Remove T: Debug bound on mpsc Debug impls (#866)
Following from https://github.com/tokio-rs/tokio/pull/865, this PR
removes `#[derive(Debug)]` on `mpsc` sender and receiver types in favor
of explicit `impl fmt::Debug` blocks that don't have a `T: fmt::Debug`
bound.
2019-01-23 18:04:00 -05:00
Jon Gjengset c6f9a069a5 Explicit impl Clone for tx to avoid T: Clone (#865)
`#[derive(Clone)]` on a type `struct Foo<T>` adds an impl that requires that
`T: Clone`:

```rust
impl<T: Clone> Clone for Foo<T>
```

which is unfortunate in the case of senders, because we don't want to require
that the items being sent are `Clone` for the channel sender to be `Clone`.
This PR adds an explicit `impl Clone` for the bounded and unbounded sender
types which does not have the `T: Clone` bound.

Note that this is _also_ an issue with `#[derive(Debug)]`, but that one is
harder to work around as `chan::Tx` _also_ has `#[derive(Debug)]`, as does
`chan::Chan`, so we'd have to add explicit impls for all of them to make
progress.
2019-01-23 15:51:44 -05:00
Sean McArthur 9f356d6244 tokio-sync: add into_inner for TrySendErrors (#862) 2019-01-22 14:48:22 -08:00
Carl Lerche 13083153aa Introduce tokio-sync crate containing synchronization primitives. (#839)
Introduce a tokio-sync crate containing useful synchronization primitives for programs
written using Tokio.

The initial release contains:

* An mpsc channel
* A oneshot channel
* A semaphore implementation
* An `AtomicTask` primitive.

The `oneshot` and `mpsc` channels are new implementations providing improved
performance characteristics. In some benchmarks, the new mpsc channel shows
up to 7x improvement over the version provided by the `futures` crate. Unfortunately,
the `oneshot` implementation only provides a slight performance improvement as it
is mostly limited by the `futures` 0.1 task system. Once updated to the `std` version
of `Future` (currently nightly only), much greater performance improvements should
be achievable by `oneshot`.

Additionally, he implementations provided here are checked using
[Loom](http://github.com/carllerche/loom/), which provides greater confidence of
correctness.
2019-01-22 11:37:26 -08:00
rmcteggart-r7 91f20e33a4 docs: deal with Result instead of using unwrap (#860) 2019-01-20 14:21:17 -05:00
Eliza Weisman 983e9d1b67 timer: Fix DelayQueue delay reset logic (#851) 2019-01-20 08:38:39 -05:00
Stjepan Glavina 4c8f274db9 threadpool: drop incomplete tasks on shutdown (#722)
## Motivation

When the thread pool shuts down, futures that have been polled at least once but not completed yet are simply leaked. We should drop them instead.

## Solution

Multiple changes are introduced:

* Tasks are assigned a home worker the first time they are polled.

* Each worker contains a set of tasks (`Arc<Task>`) it is home to. When a task is assigned a home worker, it is registered in that worker's set of tasks. When the task is completed, it is unregistered from the set.

* When the thread pool shuts down and after all worker threads stop, the remaining tasks in workers' sets are aborted, i.e. they are switched to the `Aborted` state and their `Future`s are dropped.

* The thread pool shutdown process is refactored to make it more robust. We don't  track the number of active threads manually anymore. Instead, there's  `Arc<ShutdownTrigger>` that aborts remaining tasks and completes the `Shutdown` future once it gets destroyed (when all `Worker`s and `ThreadPool` get dropped because they're the only ones to contain strong references to the `ShutdownTrigger`).

Closes #424 
Closes #428
2019-01-17 22:12:25 +01:00
Marek Kotewicz c980837581 docs: missing links in tokio-timer::delay_queue (#845) 2019-01-13 21:20:08 +01:00
Marek Kotewicz eec370cae8 docs: fixed links in tokio-timer (#844)
* docs: fixed links in tokio-timer/src/timer/mod.rs

* docs: fixed links in tokio-timer::clock
2019-01-12 10:06:55 -08:00
Marek Kotewicz 733d432b80 docs: fixed links to tokio_timer::clock::Now (#842)
* docs: fixed links to tokio_timer::clock::Now in tokio-timer/src/timer/mod.rs

* docs: fixed links to std::time::Instant in tokio-timer/src/timer/mod.rs
2019-01-10 23:49:13 +01:00
Carl Lerche 74c473d68f travis: allow nightly Rust CI to fail (#843) 2019-01-10 11:27:58 -08:00
Sean McArthur d95c697781 tokio: update tokio-threadpool minimum version (#838) 2019-01-07 16:49:08 -08:00
Carl Lerche 25e835c5b7 tcp: specify version for tokio dev dependency
This is required for publishing to crates.io
2019-01-06 23:31:24 -08:00
Carl Lerche 961aae41c4 Bump version to 0.1.14. (#836)
Also bumps:

* tokio-async-await (0.1.5)
* tokio-executor (0.1.6)
* tokio-fs (0.1.5)
* tokio-io (0.1.11)
* tokio-reactor (0.1.8)
* tokio-tcp (0.1.3)
* tokio-threadpool (0.1.10)
* tokio-tls (0.2.1)
* tokio-uds (0.2.5)

...and updates LICENSE files to 2019.
2019-01-06 23:25:55 -08:00
Carl Lerche 74c73b218e Revert "util: implement stream debounce combinator (#747)" (#834)
This reverts commit 7a49ebb65e.

The commit conflicted with another change that was merged, causing CI to fail. The public API
also requires a bit more refinement (#833) and Tokio crates need to be released.
2019-01-06 16:56:49 -08:00
Moritz Gunz 7a49ebb65e util: implement stream debounce combinator (#747) 2019-01-05 11:08:12 -05:00
Stjepan Glavina a687922746 tcp: deprecate TcpStream::try_clone() (#824) 2019-01-05 10:55:58 -05:00
Stjepan Glavina df299ced45 threadpool: panic if a worker thread cannot be spawned (#826) 2019-01-05 10:53:38 -05:00
Carl Lerche 78d1fe0eb0 ci: limit min rust version to cargo check (#829) 2019-01-05 10:52:26 -05:00
Ryan Huang fc8cde383a docs: fix link to ThreadPool (#830) 2019-01-05 10:51:17 -05:00
Sean McArthur 76198f63d7 Provide optional features on tokio crate (#808)
Disabling all features means the only dependency is `futures`.

Relevant pieces of the API can then be enabled with the following features:

- `codec`
- `fs`
- `io`
- `reactor`
- `tcp`
- `timer`
- `udp`
- `uds`

This also introduces the beginnings of enabling only certain pieces of the `Runtime`. As a start, the entire default runtime API is enabled via the `rt-full` feature.
2019-01-04 11:42:33 -08:00
Carl Lerche 39dc5706b7 travis: remove commented out code. (#828)
The commented out lines are no longer relevant and will not be brought
back.
2019-01-03 22:04:09 -08:00
Carl Lerche cbecb87797 executor: fix build (#825)
Two unrelated PRs to the same file resulted in a broken build. This
patch fixes the build by including `Arc`.
2019-01-03 11:26:58 -08:00
Carl Lerche f0bdf1980c threadpool: remove unused fn (#822)
The unused lint on nightly has discovered a new unused fn.
2019-01-03 09:34:37 -08:00
Stjepan Glavina 5e2d93f060 Use Crossbeam's Parker/Unparker (#528) 2019-01-02 21:51:22 -08:00
Taiki Endo 9a8d087c69 Allow deprecated Error::cause (#818)
Error::cause is deprecated in Rust 1.33, but this allows Error::cause
until the minimum supported version of tokio is Rust 1.30.

When the minimum support version of tokio reaches Rust 1.30,
replace Error::cause with Error::source.

Fixes: #817
2019-01-02 14:12:11 -08:00
gralpli 30f59670c8 Clarify what NoopWaker does (#819) 2019-01-02 12:29:30 -08:00
jq-rs 9e4ddaeaf3 examples: single-threaded chat combinator example (#794) 2018-12-29 10:16:30 -05:00
Balthild Ires 03e2e864f3 Stablize pin feature (#814)
Box::pinned has been renamed to Box::pin. Meanwhile, the pin feature
no longer requires an attribute to enable.

Fixes: #813
2018-12-28 12:12:31 -08:00
Sean McArthur c8a990eda4 tokio-reactor: deprecates Handle::current() (#805)
The side effects of calling `Handle::current()` from outside of a
runtime could be very surprising, since it would start up a background
reactor.
2018-12-28 12:09:35 -08:00
Pavel Strakhov 1a5026324f executor: impl Unpark for Arc<Unpark> (#802) 2018-12-28 14:40:04 -05:00
Stjepan Glavina fdf4aba621 threadpool: introduce a global task queue (#798) 2018-12-28 14:34:54 -05:00
Roman 201b6ce53a ci: remove ALLOW_FAILURES=false in travis for nightly cargo doc (#816) 2018-12-28 10:06:00 -05:00
Roman db69275202 docs: fix warnings for nightly docs (#792) 2018-12-17 15:20:46 -05:00
Roman af85cb3430 ci: improve travis run times (#793) 2018-12-17 15:18:27 -05:00
Christian Bourjau 36f1a19ac8 Minor change in documentation of Decoder::decode (#797)
`None` -> `Ok(None)`
2018-12-13 11:14:38 -08:00
Stjepan Glavina 6aa990ea75 threadpool: fix semaphore deadlock (#795) 2018-12-12 16:42:18 -05:00
Simon Farnsworth 760a7667d6 threadpool: improve the documentation of blocking (#789) 2018-12-05 15:20:19 -05:00
Matt Gathu 2283b63e9e fs: added usage examples/doctests to File (#786) 2018-12-01 21:07:48 -05:00
Felix Obenhuber 8263e5f18d uds: fix WouldBlock case in UnixDatagram send methods (#782) 2018-11-30 19:40:03 -05:00
Carl Lerche b3e57b60d0 examples: remove reference to tokio-core (#780) 2018-11-28 14:55:31 -05:00
Matt Gathu 1cd0ebfc5e tcp: add usage examples to TcpListener and TcpStream (#775)
Refs: https://github.com/rust-lang-nursery/wg-net/issues/54
2018-11-28 13:05:35 -05:00
David Kellum 4797d79950 reactor: update to parking_lot 0.7 (#778) 2018-11-28 09:29:31 -08:00
Steven Fackler e7d9ba7e51 tls: make TlsConnector and TlsAcceptor derive Clone (#777) 2018-11-27 07:52:17 -05:00
luben karavelov 527dc0a66f net: export UnixDatagram and UnixDatagramFramed (#772) 2018-11-23 08:41:32 -05:00
Carl Lerche b117fc1d65 Bump version to v0.1.13 (#771)
This also bumps the following sub crate versions:

* tokio-current-thread (0.1.4)
* tokio-reactor (0.1.7)
* tokio-signal (0.2.7)
* tokio-threadpool (0.1.9)
* tokio-timer (0.2.8)
* tokio-udp (0.1.3)
* tokio-uds (0.2.4)
2018-11-21 17:11:31 -08:00
Felix Obenhuber 272e09d349 threadpool: remove smoke example (#764) (#770) 2018-11-21 14:23:36 -08:00
Stjepan Glavina 3235749006 threadpool: refactor pool shutdown (#769) 2018-11-20 21:43:23 +01:00
Stjepan Glavina 9c037044c4 threadpool: rename inner to something more descriptive (#768)
`inner` is a fitting name for variables of type named `Inner`, but in other cases I find them confusing - sometimes `inner` refers to a `Pool`, sometimes to a `Sender`. I renamed a bunch of variables named `inner` to be more descriptive.

This PR is the first step in an effort of splitting https://github.com/tokio-rs/tokio/pull/722#issuecomment-439552671 into multiple PRs.
2018-11-20 20:05:14 +01:00
Patrick Barrett 3658e10045 uds: implement UnixDatagramFramed (#453)
Implement `Stream + Sink` layer on top of unix domain sockets
using codecs.
2018-11-20 09:19:34 -08:00
Carl Lerche ed3ece266b current-thread: fix shutdown on idle (#763)
When spawning using `Handle` while on the executor, tasks were being
double counted. This prevented the number of active tasks to reach zero,
thus preventing the executor from shutting down.

This changes `spawn` to check if being called from the executor
**before** incrementing the number of active tasks.

Fixes #760
2018-11-20 09:17:07 -08:00
Liran Ringel 9b1a45cc6a tests: handle errors properly in examples (#748) 2018-11-20 11:10:36 -05:00
Carl Lerche 477fa5580a ci: Don't deploy docs if $TARGET is set (#762) 2018-11-19 21:22:28 -08:00
Toby Lawrence bb6cca8ff0 tests: switch to Windows Server 2016 for AppVeyor builds. (#761)
Should hopefully fix the underlying bug that was causing tokio-tls tests to occasionally fail on Windows.

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 20:18:37 -05:00
Moritz Gunz e166c4d912 Implement throttle combinator (#736)
Throttle down a stream by enforcing a fixed delay between items.
2018-11-19 15:04:55 -08:00
Toby Lawrence b7506cf663 Allow nightly builds to fail. (#743)
* tests: allow nightly builds to fail

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 17:13:56 -05:00
Carl Lerche dc4a29359f io: allow deprecated code in length_delimited test (#759)
This file is testing deprecated code, so it should be permitted to
access deprecated code.
2018-11-19 14:11:46 -08:00
Bastian Köcher d3dca4552b Expose after_start and before_stop in runtime::Builder (#756)
Closes #705
2018-11-19 09:04:58 -08:00
andoks 42a0df1ea4 Fix async await README example (#758)
* async-await: fix README example dependencies

As per commit "async-await: track nightly changes (#661)" ( commit
2f690d30bc)

> The `tokio-async-await` crate is no longer a facade. Instead, the
> `tokio` crate provides a feature flag to enable async/await support.

Ensure the example in the async-await README file also works by
correctly declaring this updated dependency

* async-await: remove unnecessary 'edition' declaration from README

As the "edition" feature was stabilized in rust v1.30 and async-await
specifies that the nightly toolchain must be used, remove the use of the
"edition" feature gate since it is enabled by default.
2018-11-17 20:58:19 -08:00
Brian Myers a98eab6eff rt: fix Builder docs to no longer use deprecated methods (#749) 2018-11-16 14:58:08 -08:00
Ivan Petkov 5a5dde70b3 signal: miscellaneous tweaks and improvements (#751)
* Minimize allocation needed for channels

* Use a newtype for signal ids

* We can just cast the raw pointer to a `usize` and still perform a
simple identity check, without incurring any implications of storing a
raw pointer (e.g. previously Signal was !Sync and had an unsafe impl of
Send, and now it is naturally Sync+Send)

* Broadcast with `try_send` instead of `start_send`

The `Stream::start_send` method uses backpressure and schedules the
current task to be notified whenever the channel has additional room,
which means we'll generate a lot of unnecessary wakeups whenever a
channel gets full

By changing to `try_send` and handling any errors, we ensure the
Driver's task won't get woken up when a Signal finally consumes its
notification, since we're coalescing things anyway
2018-11-16 14:56:43 -08:00
Alex Gaynor d0963774a3 chore: bump rand dependency to 0.6 (#753) 2018-11-16 14:54:14 -08:00
Felix Obenhuber c83355235c uds: minor doc fix in UnixStream and UnixDatagram (#754) 2018-11-16 14:53:19 -08:00
Kazuyoshi Kato 33a216e4c1 fs: add more tests (#755)
Fixes #704.
2018-11-16 14:50:06 -08:00
Toralf Wittner 09f2ac85bf udp: add into_parts to RecvDgram (#710)
* udp: add `into_parts` to `RecvDgram`

If `RecvDgram` can not be driven to completion it may become necessary to get back the `UdpSocket` it contains which is currently not possible.

This adds`into_parts` to get the socket as well as the buffer back. Both methods consume `RecvDgram`.

Note that after the future has completed, `into_parts` must not be used, or else a panic will happen.
2018-11-15 10:55:34 -05:00
Ohad Ravid 32a152630f uds: added solaris support in the ucred module (#733) 2018-11-15 10:30:37 -05:00
Kazuyoshi Kato 9153067d66 fs: add tests for directory-related functions (#704) (#724)
This change adds a few tests around directory-related functions.
2018-11-13 18:28:52 -05:00
Kazuyoshi Kato d246964bdf fs: gen_ascii_chars has been deprecated (#735)
Use sample_iter() instead.
2018-11-10 21:35:38 -05:00
Alex Gaynor e700607554 Bumped crossbeam-utils version (#746)
## Motivation

tokio depends on an out of date version of crossbeam-utils, which results in multiple versions of that package being linked in binaries which use other popular libraries.

## Solution

Bump the version; there's no API changes and tests still pass.
2018-11-10 10:39:09 +01:00
Benjamin Saunders 5321550534 Derive Clone for delay_queue::Key (#730)
Improves API ergonomics with minimal forwards-compatibility hazard.
2018-11-09 15:11:17 -08:00
Stjepan Glavina 32e1cafb57 fix tsan errors (#745) 2018-11-09 15:06:46 -08:00
Josh Leverette 49bc4025dd reactor: reduce log level of loop process (#734) 2018-11-07 16:53:53 -05:00
Carl Lerche 51e36e41bc Add tokio-buf and a BufStream trait (#611)
The `BufStream` trait provides an improved API for working with
asynchronous streams of bytes compared to `Stream<Item = [u8]>`
2018-10-29 13:43:48 -07:00
Carl Lerche d011b92b9a rt: fix Runtime::reactor() as used by tokio-core (#721)
* rt: fix `Runtime::reactor()` as used by tokio-core

Up until Tokio v0.1.11, the handle returned by `Runtime::reactor()`
pointed to a reactor instance running in a background thread. The thread
was eagerly spawned.

As of v0.1.12, a reactor instance is created per runtime worker thread.
`Runtime::reactor()` was deprecated and updated to point to the reactor
for one of the worker threads.

A problem occurs when attempting to use the reactor before spawning a
task. Worker threads are spawned lazily, which means that the reactor
referenced by `Runtime::reactor()` is not yet running.

This patch changes `Runtime::reactor` back to a dedicated reactor
running on a background thread. However, the background thread is now
spawned lazily when the deprecated function is first called.

Fixes #720

* Fix comment

Co-Authored-By: carllerche <[email protected]>
2018-10-25 11:23:54 +02:00
Carl Lerche f929576f0e Bump version to 0.1.12 (#718)
Also bumps the following sub-crates:

* tokio-fs (0.1.4)
* tokio-io (0.1.10)
* tokio-signal (0.2.6)
* tokio-threadpool (0.1.8)
* tokio-uds (0.2.3)
2018-10-23 22:00:49 -07:00
Ivan Petkov b0f001a05a signal: Bump version to 0.2.6 (#714)
* Also Update the CHANGELOG to match the rest of the project
2018-10-23 20:39:19 -07:00
Iku Iwasa 2291ba9d0d uds: add NetBSD support (#715) 2018-10-23 20:15:42 -04:00
Name 7f84f6b4ca contributing: fix an invalid link (#716)
Just move a dot to the right place.
2018-10-21 16:56:01 +00:00
Andrew Audibert 5f61bd5252 fix a typo in the contributing guide (#711) 2018-10-19 09:25:11 -07:00
Ryan Dahl bffa3ed558 fs: expose fs::File::from_std() (#696) 2018-10-17 19:51:46 -04:00
Sean McArthur 7b5ef61aeb runtime: check Enter in more places when blocking (#708)
- `tokio::run` checks Enter before creating a new threadpool and
  spawning the main future.
- `Runtime::block_on` now checks Enter
- `Runtime::block_on_all` now checks Enter
2018-10-17 15:25:40 -07:00
Stjepan Glavina 753336de8e threadpool: Arc instead of Inner in Notifier (#702) 2018-10-15 13:24:00 -07:00
Ryan Levick 65aea16ad1 tokio: change hello world to new, simpler example (#690) 2018-10-12 12:42:19 -04:00
nickelc 796fee6364 fs: fix minor documentation error for MetadataFuture (#698) 2018-10-12 12:41:19 -04:00
Stjepan Glavina adb0ba71d4 threadpool: worker threads shouldn't respect keep_alive (#692)
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.

Bug fixes and new features should include tests.

Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->

## Motivation

Now that each worker thread drives its own reactor, reactors have to be driven until the threadpool shuts down. We mustn't use the `keep_alive` setting to shut down a worker thread if it doesn't receive an event from the reactor for a certain duration of time.

<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->

## Solution

Just ignore the `keep_alive` setting when parking in `Worker::sleep`.

<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
2018-10-10 09:05:36 +02:00
David Ross bfa6766f3c re-export tokio_io::read in tokio::io (#689)
Fixes: #688
2018-10-09 19:50:03 -07:00
Nikolay Kim a2f457fa48 io: expose underlying codec (#686) 2018-10-06 19:23:05 -04:00
Eliza Weisman 1879bc49ce codec: Fix panic in LengthDelimitedCodec::encode (#682)
Fixes: #681 

## Motivation

Currently, a potential panic exists in `LengthDelimitedCodec::encode`.
Writing the length field to the `dst` buffer can exceed the buffer
capacity, as `BufMut::put_uint_{le,be}` doesn't reserve more capacity. 

## Solution

This branch adds a call to `dst.reserve` to ensure that there's 
sufficient remaining buffer capacity to hold the length field and
the frame, prior to writing the length field. Previously, capacity
was only reserved later in the function, when writing the frame
to the buffer, and we never reserved capacity for the length field.

I've also added a test that reproduces the issue. The test panics on
master, but passes after making this change.

Signed-off-by: Eliza Weisman <[email protected]>
2018-10-04 12:46:57 -07:00
Sven Marnach 678f6382b8 io: implement prepare_uninitialized_buffer for Take and Chain (#678) 2018-10-04 11:03:43 -07:00
Stjepan Glavina e27b0a46ba threadpool: spawn new tasks onto a random worker (#683)
* threadpool: submit new tasks to a random worker

* Revert unnecessary version bumps
2018-10-03 23:09:20 +02:00
Stjepan Glavina d35d0518f5 runtime: create reactor per worker (#660) 2018-10-02 18:19:27 -07:00
Sven Marnach 886511c0a6 io: fix minor documentation errors for Async{Read,Write} (#677) 2018-10-01 19:34:24 -04:00
Steven Fackler d06bd6b216 Expose keep_alive on the Runtime builder (#676)
This was overlooked when delegating the rest of the threadpool builder
methods from Runtime's builder.
2018-09-28 21:00:50 -07:00
Carl Lerche 2c85cd0991 Bump version to v0.1.11 (#675)
This fixes the dependency on `tokio-async-await` to not be scoped to
unix platforms.

Fixes #673
2018-09-28 11:32:52 -07:00
Carl Lerche 1e45237a28 Bump tokio-uds to v0.2.2 2018-09-27 20:05:23 -07:00
Sean McArthur 3a88d85538 ads: fix UdsStream::read_buf to clear read (not write) readiness (#672) 2018-09-27 20:02:29 -07:00
Carl Lerche b47ad24268 Bump version to v0.1.10, fixing minimal versions (#671)
Some minimal versions were not correctly updated.

Also updates:

* tokio-current-thread (v0.1.3).
2018-09-27 13:00:53 -07:00
429 changed files with 23954 additions and 7523 deletions
-20
View File
@@ -1,20 +0,0 @@
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+41
View File
@@ -0,0 +1,41 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_DURATION: 10
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
cargo_cache:
folder: $HOME/.cargo/registry
test_script:
- . $HOME/.cargo/env
- cargo test --all --no-fail-fast
- cargo doc --all
i686_test_script:
- . $HOME/.cargo/env
- |
cargo test --all --exclude tokio-tls --no-fail-fast --target i686-unknown-freebsd
before_cache_script:
- rm -rf $HOME/.cargo/registry/index
-108
View File
@@ -1,108 +0,0 @@
---
language: rust
sudo: false
cache:
- apt
- cargo
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
- rust: 1.26.0
- rust: stable
- rust: beta
- rust: nightly
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script:
- |
set -e
if [[ "$TRAVIS_RUST_VERSION" == nightly ]]
then
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
fi
- |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
cargo test --all --no-fail-fast
# Disable these tests for now as they are buggy
#
# cargo test --features unstable-futures
# cargo test --manifest-path tokio-threadpool/Cargo.toml --features unstable-futures
# cargo test --manifest-path tokio-reactor/Cargo.toml --features unstable-futures
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = linux
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+3 -3
View File
@@ -109,7 +109,7 @@ and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increasethe likelihood of the PR getting
feedback and guidance. This will increase the likelihood of the PR getting
merged.
### Tests
@@ -380,8 +380,8 @@ left). When doing so, it is courteous to give the original contributor credit
for the work they started (either by preserving their name and email address in
the commit log, or by using an `Author: ` meta-data tag in the commit.
_Adapted from the [Node.js contributing guide][node]_
_Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
+5 -72
View File
@@ -1,31 +1,9 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Update doc URL.
# - Create "v0.1.x" git tag.
version = "0.1.9"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.9/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio",
"tokio-async-await",
"tokio-channel",
"tokio-buf",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
@@ -33,58 +11,13 @@ members = [
"tokio-io",
"tokio-reactor",
"tokio-signal",
"tokio-sync",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-trace",
"tokio-trace/tokio-trace-core",
"tokio-udp",
"tokio-uds",
]
[features]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
bytes = "0.4"
tokio-codec = { version = "0.1.0", path = "tokio-codec" }
tokio-current-thread = { version = "0.1.1", path = "tokio-current-thread" }
tokio-io = { version = "0.1.6", path = "tokio-io" }
tokio-executor = { version = "0.1.2", path = "tokio-executor" }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.4", path = "tokio-threadpool" }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp" }
tokio-udp = { version = "0.1.0", path = "tokio-udp" }
tokio-timer = { version = "0.2.6", path = "tokio-timer" }
tokio-fs = { version = "0.1.3", path = "tokio-fs" }
futures = "0.1.20"
# Needed until `reactor` is removed from `tokio`.
mio = "0.6.14"
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds" }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+8 -11
View File
@@ -14,29 +14,26 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio) |
[API Docs](https://docs.rs/tokio/0.1.18/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[master-dox]: https://tokio-rs.github.io/tokio/doc/tokio/
## Overview
@@ -52,9 +49,9 @@ level, it provides a few major components:
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/0.1.18/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.18/tokio/reactor/index.html
[scheduler]: https://docs.rs/tokio/0.1.18/tokio/runtime/index.html
## Example
+115
View File
@@ -0,0 +1,115 @@
trigger: ["master"]
pr: ["master"]
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
name: rustfmt
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
displayName: Test tokio
cross: true
crates:
- tokio
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates -
cross: true
crates:
- tokio-fs
- tokio-reactor
- tokio-signal
- tokio-tcp
- tokio-tls
- tokio-udp
- tokio-uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
crates:
- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
- tokio-io
- tokio-sync
- tokio-threadpool
- tokio-timer
- tokio-trace
- tokio-trace/tokio-trace-core
- template: ci/azure-cargo-check.yml
parameters:
name: features
displayName: Check feature permtuations
rust: stable
crates:
tokio:
- codec
- fs
- io
- reactor
- rt-full
- tcp
- timer
- udp
- uds
tokio-buf:
- util
# Check async / await
- template: ci/azure-cargo-check.yml
parameters:
name: async_await
displayName: Async / Await
rust: nightly-2019-02-28
noDefaultFeatures: ''
benches: true
crates:
tokio:
- async-await-preview
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross_32bit_linux
target: i686-unknown-linux-gnu
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust_version: 1.26.0
- template: ci/azure-tsan.yml
parameters:
name: tsan
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
- rustfmt
- test_tokio
- test_sub_cross
- test_linux
- features
- async_await
- cross_32bit_linux
- minrust
- tsan
+1 -3
View File
@@ -10,8 +10,8 @@ use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
@@ -57,7 +57,6 @@ fn udp_echo_latency(b: &mut Bencher) {
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
@@ -67,7 +66,6 @@ fn udp_echo_latency(b: &mut Bencher) {
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
+8 -9
View File
@@ -3,14 +3,13 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
extern crate test;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
use mio::{PollOpt, Ready, Token};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
@@ -22,8 +21,8 @@ fn mio_register_deregister(b: &mut Bencher) {
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
poll.deregister(&sock).unwrap();
});
}
@@ -36,12 +35,12 @@ fn mio_reregister(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.reregister(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
});
poll.deregister(&sock).unwrap();
}
+62 -49
View File
@@ -11,18 +11,18 @@ pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::reactor::Reactor;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::io::{self, Read, Write};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
pub use test::{self, Bencher};
}
mod connect_churn {
use ::prelude::*;
use prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
@@ -36,25 +36,29 @@ mod connect_churn {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
Ok(TcpStream::connect(&addr).and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
let connects_concurrent = connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(connects_concurrent)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
}
@@ -65,8 +69,7 @@ mod connect_churn {
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
@@ -75,47 +78,56 @@ mod connect_churn {
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(shutdown_rx)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
use std::sync::{Arc, Barrier};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
let threads: Vec<_> = (0..n)
.map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()))
.wait()
.unwrap();
})
})
}).collect();
.collect();
barrier.wait();
@@ -141,7 +153,7 @@ mod connect_churn {
}
mod transfer {
use ::prelude::*;
use prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
@@ -200,7 +212,8 @@ mod transfer {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
let server = listener
.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
@@ -210,17 +223,17 @@ mod transfer {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
drain
.map(|_| ())
.map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
.and_then(move |sock| Transfer {
sock: sock,
rem: MB,
chunk: write_size,
})
.map_err(|e| panic!("client err: {:?}", e));
@@ -229,7 +242,7 @@ mod transfer {
}
mod small_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
@@ -238,7 +251,7 @@ mod transfer {
}
mod big_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
+33
View File
@@ -0,0 +1,33 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- ${{ if parameters.benches }}:
- script: cargo check --benches --all
displayName: Check benchmarks
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+24
View File
@@ -0,0 +1,24 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
# Always patch
- template: azure-patch-crates.yml
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+38
View File
@@ -0,0 +1,38 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --all --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+27
View File
@@ -0,0 +1,27 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain %RUSTUP_TOOLCHAIN%
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%s' | grep -q '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+16
View File
@@ -0,0 +1,16 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
rustup component add rustfmt
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+41
View File
@@ -0,0 +1,41 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }} (PATCHED)
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-threadpool --tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: nightly-2018-11-18
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+22
View File
@@ -0,0 +1,22 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-async-await = { path = "tokio-async-await" }
tokio-buf = { path = "tokio-buf" }
tokio-codec = { path = "tokio-codec" }
tokio-current-thread = { path = "tokio-current-thread" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-io = { path = "tokio-io" }
tokio-reactor = { path = "tokio-reactor" }
tokio-signal = { path = "tokio-signal" }
tokio-sync = { path = "tokio-sync" }
tokio-threadpool = { path = "tokio-threadpool" }
tokio-timer = { path = "tokio-timer" }
tokio-tcp = { path = "tokio-tcp" }
tokio-tls = { path = "tokio-tls" }
tokio-trace = { path = "tokio-trace" }
tokio-trace-core = { path = "tokio-trace/tokio-trace-core" }
tokio-udp = { path = "tokio-udp" }
tokio-uds = { path = "tokio-uds" }
+4
View File
@@ -31,3 +31,7 @@ race:crossbeam_deque*steal
race:Backup::next_sleeper
race:Backup::set_next_sleeper
race:WorkerEntry::set_next_sleeper
# This ignores a false positive caused by `thread::park()`/`thread::unpark()`.
# See: https://github.com/rust-lang/rust/pull/54806#issuecomment-436193353
race:pthread_cond_destroy
+172
View File
@@ -0,0 +1,172 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! Because we are here running the reactor/executor on the same thread instead
//! of a threadpool, we can avoid full synchronization with Arc + Mutex and use
//! Rc + RefCell instead. The max performance is however limited to a CPU HW
//! thread.
//!
//! You can test this out by running:
//!
//! cargo run --example chat-combinator-current-thread
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::runtime::current_thread::{Runtime, TaskExecutor};
use std::cell::RefCell;
use std::collections::HashMap;
use std::env;
use std::io::BufReader;
use std::iter;
use std::rc::Rc;
fn main() -> Result<(), Box<std::error::Error>> {
let mut runtime = Runtime::new().unwrap();
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio current_thread runtime, so it will be single-
// threaded. The `Rc<RefCell<...>>` allows state to be shared across the tasks.
let connections = Rc::new(RefCell::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
let mut conns = connections.borrow_mut();
conns.insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.borrow_mut();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn locally a task to process the connection
TaskExecutor::current()
.spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
})))
.unwrap();
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// Spawn srv itself
runtime.spawn(srv);
// Execute server
runtime.run().unwrap();
Ok(())
}
+23 -17
View File
@@ -21,25 +21,25 @@
#![deny(warnings)]
extern crate tokio;
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::io::BufReader;
use std::iter;
use std::sync::{Arc, Mutex};
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse().unwrap();
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr).unwrap();
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
@@ -48,11 +48,15 @@ fn main() {
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr().unwrap();
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
@@ -91,9 +95,7 @@ fn main() {
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
@@ -105,15 +107,17 @@ fn main() {
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
@@ -143,8 +147,10 @@ fn main() {
}));
Ok(())
});
})
.map_err(|err| println!("error occurred: {:?}", err));
// execute server
tokio::run(srv);
Ok(())
}
+32 -33
View File
@@ -31,12 +31,12 @@ extern crate tokio;
extern crate futures;
extern crate bytes;
use bytes::{BufMut, Bytes, BytesMut};
use futures::future::{self, Either};
use futures::sync::mpsc;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use std::collections::HashMap;
use std::net::SocketAddr;
@@ -131,10 +131,7 @@ impl Shared {
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
fn new(name: BytesMut, state: Arc<Mutex<Shared>>, lines: Lines) -> Peer {
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
@@ -142,8 +139,7 @@ impl Peer {
let (tx, rx) = mpsc::unbounded();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
state.lock().unwrap().peers.insert(addr, tx);
Peer {
name,
@@ -198,7 +194,7 @@ impl Future for Peer {
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
if i + 1 == LINES_PER_TICK {
task::current().notify();
}
}
@@ -256,8 +252,7 @@ impl Future for Peer {
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
self.state.lock().unwrap().peers.remove(&self.addr);
}
}
@@ -333,7 +328,10 @@ impl Stream for Lines {
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
let pos = self
.rd
.windows(2)
.enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
@@ -373,7 +371,8 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
let connection = lines
.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
@@ -408,10 +407,7 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
let peer = Peer::new(name, state, lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
@@ -426,7 +422,7 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
tokio::spawn(connection);
}
pub fn main() {
pub fn main() -> Result<(), Box<std::error::Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
@@ -434,27 +430,29 @@ pub fn main() {
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
let addr = "127.0.0.1:6142".parse()?;
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
let listener = TcpListener::bind(&addr)?;
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
let server = listener
.incoming()
.for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
@@ -471,4 +469,5 @@ pub fn main() {
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
Ok(())
}
+68 -56
View File
@@ -16,20 +16,20 @@
#![deny(warnings)]
extern crate bytes;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use std::env;
use std::io::{self, Read, Write};
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
use tokio::prelude::*;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
@@ -41,10 +41,11 @@ fn main() {
};
// Parse what address we're going to connect to
let addr = args.first().unwrap_or_else(|| {
panic!("this program requires at least one argument")
});
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = match args.first() {
Some(addr) => addr,
None => Err("this program requires at least one argument")?,
};
let addr = addr.parse::<SocketAddr>()?;
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
@@ -52,15 +53,15 @@ fn main() {
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!()); // errors not possible on rx
let stdin_rx = stdin_rx.map_err(|_| panic!("errors not possible on rx"));
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))
tcp::connect(&addr, Box::new(stdin_rx))?
} else {
udp::connect(&addr, Box::new(stdin_rx))
udp::connect(&addr, Box::new(stdin_rx))?
};
// And now with our stream of bytes to write to stdout, we execute that in
@@ -72,17 +73,16 @@ fn main() {
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.for_each(move |chunk| out.write_all(&chunk))
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
Ok(())
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio::codec::{Encoder, Decoder};
use std::io;
use tokio::codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -120,20 +120,21 @@ mod codec {
mod tcp {
use tokio;
use tokio::codec::Decoder;
use tokio::net::TcpStream;
use tokio::prelude::*;
use tokio::codec::Decoder;
use bytes::BytesMut;
use codec::Bytes;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
pub fn connect(
addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
@@ -151,45 +152,52 @@ mod tcp {
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
Box::new(tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
let stream = Box::new(
tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream())
stream
})
.flatten_stream(),
);
Ok(stream)
}
}
mod udp {
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use tokio;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
pub fn connect(
&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse().unwrap()
"0.0.0.0:0".parse()?
} else {
"[::]:0".parse().unwrap()
"[::]:0".parse()?
};
let udp = match UdpSocket::bind(&addr_to_bind) {
Ok(udp) => udp,
Err(_) => Err("failed to bind socket")?,
};
let udp = UdpSocket::bind(&addr_to_bind)
.expect("failed to bind socket");
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
@@ -199,14 +207,15 @@ mod udp {
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
});
let forward_stdin = stdin
.map(move |chunk| (chunk, addr))
.forward(sink)
.then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
@@ -218,10 +227,14 @@ mod udp {
}
});
Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream())
let stream = Box::new(
future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
})
.flatten_stream(),
);
Ok(stream)
}
}
@@ -232,8 +245,7 @@ fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Err(_) | Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
+7 -6
View File
@@ -16,11 +16,11 @@
extern crate futures;
extern crate tokio;
use std::{env, io};
use std::net::SocketAddr;
use std::{env, io};
use tokio::prelude::*;
use tokio::net::UdpSocket;
use tokio::prelude::*;
struct Server {
socket: UdpSocket,
@@ -50,12 +50,12 @@ impl Future for Server {
}
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
let socket = UdpSocket::bind(&addr).unwrap();
println!("Listening on: {}", socket.local_addr().unwrap());
let socket = UdpSocket::bind(&addr)?;
println!("Listening on: {}", socket.local_addr()?);
let server = Server {
socket: socket,
@@ -70,4 +70,5 @@ fn main() {
//
// `tokio::run` spawns the task on the Tokio runtime and starts running.
tokio::run(server.map_err(|e| println!("server error = {:?}", e)));
Ok(())
}
+6 -5
View File
@@ -30,19 +30,19 @@ use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
@@ -54,7 +54,8 @@ fn main() {
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
@@ -89,7 +90,6 @@ fn main() {
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
@@ -111,4 +111,5 @@ fn main() {
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
+27 -39
View File
@@ -1,58 +1,45 @@
//! Hello world server.
//!
//! A simple server that accepts connections, writes "hello world\n", and closes
//! A simple client that opens a TCP stream, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! cargo run --example hello_world
//! ncat -l 6142
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! cargo run --example hello_world
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::net::TcpStream;
use tokio::prelude::*;
pub fn main() {
let addr = "127.0.0.1:6142".parse().unwrap();
pub fn main() -> Result<(), Box<std::error::Error>> {
let addr = "127.0.0.1:6142".parse()?;
// Bind a TCP listener to the socket address.
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(|socket| {
println!("accepted socket; addr={:?}", socket.peer_addr().unwrap());
let connection = io::write_all(socket, "hello world\n")
.then(|res| {
println!("wrote message; success={:?}", res.is_ok());
// Note that this is the Tokio TcpStream, which is fully async.
let client = TcpStream::connect(&addr)
.and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
});
// Spawn a new task that processes the socket:
tokio::spawn(connection);
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
})
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
// Start the Tokio runtime.
//
@@ -63,8 +50,9 @@ pub fn main() {
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
println!("About to create the stream and write to it...");
tokio::run(client);
println!("Stream has been created and written to.");
Ok(())
}
+3 -2
View File
@@ -60,7 +60,7 @@ fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
Ok(())
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by tokio_current_thread::spawn().
// It also can use the default reactor and create timeouts.
@@ -82,5 +82,6 @@ fn main() {
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
})).unwrap();
}))?;
Ok(())
}
+6 -5
View File
@@ -57,27 +57,27 @@
extern crate tokio;
extern crate tokio_codec;
use tokio_codec::BytesCodec;
use tokio::codec::Decoder;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::codec::Decoder;
use tokio_codec::BytesCodec;
use std::env;
use std::net::SocketAddr;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
@@ -146,4 +146,5 @@ fn main() {
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
+22 -20
View File
@@ -24,28 +24,29 @@
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use std::net::{Shutdown, SocketAddr};
use std::sync::{Arc, Mutex};
use tokio::io::{copy, shutdown};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>().unwrap();
let listen_addr = listen_addr.parse::<SocketAddr>()?;
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>().unwrap();
let server_addr = server_addr.parse::<SocketAddr>()?;
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr).unwrap();
let socket = TcpListener::bind(&listen_addr)?;
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
@@ -68,25 +69,25 @@ fn main() {
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
.and_then(|(n, _, server_writer)| shutdown(server_writer).map(move |_| n));
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
.and_then(|(n, _, client_writer)| shutdown(client_writer).map(move |_| n));
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
let msg = amounts
.map(move |(from_client, from_server)| {
println!(
"client wrote {} bytes and received {} bytes",
from_client, from_server
);
})
.map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
@@ -94,6 +95,7 @@ fn main() {
});
tokio::run(done);
Ok(())
}
// This is a custom type used to have a custom implementation of the
+48 -27
View File
@@ -44,8 +44,8 @@
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::io::BufReader;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
@@ -69,17 +69,26 @@ enum Request {
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let listener = TcpListener::bind(&addr).expect("failed to bind");
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).map_err(|_| "failed to bind")?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
@@ -93,7 +102,8 @@ fn main() {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
let done = listener
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
@@ -124,15 +134,22 @@ fn main() {
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
}
}
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
Response::Set {
key,
value,
previous,
}
}
}
});
@@ -156,6 +173,7 @@ fn main() {
});
tokio::run(done);
Ok(())
}
impl Request {
@@ -168,9 +186,11 @@ impl Request {
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
return Err(format!("GET's key must not be followed by anything"));
}
Ok(Request::Get { key: key.to_string() })
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
@@ -181,7 +201,10 @@ impl Request {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
@@ -192,15 +215,13 @@ impl Request {
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
+74 -57
View File
@@ -1,9 +1,9 @@
//! A "tiny" example of HTTP request/response handling using just tokio-core
//! A "tiny" example of HTTP request/response handling using transports.
//!
//! This example is intended for *learning purposes* to see how various pieces
//! hook up together and how HTTP can get up and running. Note that this example
//! is written with the restriction that it *can't* use any "big" library other
//! than tokio-core, if you'd like a "real world" HTTP library you likely want a
//! than Tokio, if you'd like a "real world" HTTP library you likely want a
//! crate like Hyper.
//!
//! Code here is based on the `echo-threads` example and implements two paths,
@@ -23,34 +23,36 @@ extern crate time;
extern crate tokio;
extern crate tokio_io;
use std::{env, fmt, io};
use std::net::SocketAddr;
use std::{env, fmt, io};
use tokio::net::{TcpStream, TcpListener};
use tokio::codec::{Decoder, Encoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use tokio::codec::{Encoder, Decoder};
use bytes::BytesMut;
use http::header::HeaderValue;
use http::{Request, Response, StatusCode};
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Parse the arguments, bind the TCP socket we'll be listening to, spin up
// our worker threads, and start shipping sockets to those worker threads.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).expect("failed to bind");
let listener = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
tokio::run({
listener.incoming()
listener
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(|socket| {
process(socket);
Ok(())
})
});
Ok(())
}
fn process(socket: TcpStream) {
@@ -63,14 +65,13 @@ fn process(socket: TcpStream) {
.split();
// Map all requests into responses and send them back to the client.
let task = tx.send_all(rx.and_then(respond))
.then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
let task = tx.send_all(rx.and_then(respond)).then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
Ok(())
});
Ok(())
});
// Spawn the task that handles the connection.
tokio::spawn(task);
@@ -81,31 +82,37 @@ fn process(socket: TcpStream) {
/// This function is a map from and HTTP request to a future of a response and
/// represents the various handling a server might do. Currently the contents
/// here are pretty uninteresting.
fn respond(req: Request<()>)
-> Box<Future<Item = Response<String>, Error = io::Error> + Send>
{
let mut ret = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
ret.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
ret.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
fn respond(req: Request<()>) -> Box<Future<Item = Response<String>, Error = io::Error> + Send> {
let f = future::lazy(move || {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
serde_json::to_string(&Message { message: "Hello, World!" })
.unwrap()
}
_ => {
ret.status(StatusCode::NOT_FOUND);
String::new()
}
};
Box::new(future::ok(ret.body(body).unwrap()))
"/json" => {
response.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
}
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
});
Box::new(f)
}
struct Http;
@@ -119,12 +126,19 @@ impl Encoder for Http {
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(BytesWrite(dst), "\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
", item.status(), item.body().len(), date::now()).unwrap();
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
@@ -193,13 +207,18 @@ impl Decoder for Http {
headers[i] = Some((k, v));
}
(toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt)
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(io::ErrorKind::Other, "only HTTP/1.1 accepted"))
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
@@ -211,15 +230,13 @@ impl Decoder for Http {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = unsafe {
HeaderValue::from_shared_unchecked(data.slice(v.0, v.1))
};
let value = unsafe { HeaderValue::from_shared_unchecked(data.slice(v.0, v.1)) };
ret.header(&data[k.0..k.1], value);
}
let req = ret.body(()).map_err(|e| {
io::Error::new(io::ErrorKind::Other, e)
})?;
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
+12 -16
View File
@@ -35,29 +35,28 @@ use std::net::SocketAddr;
use tokio::net::UdpSocket;
use tokio::prelude::*;
fn get_stdin_data() -> Vec<u8> {
fn get_stdin_data() -> Result<Vec<u8>, Box<std::error::Error>> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf).unwrap();
buf
stdin().read_to_end(&mut buf)?;
Ok(buf)
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or("127.0.0.1:8080".into())
.parse()
.unwrap();
.parse()?;
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}.parse()
.unwrap();
let socket = UdpSocket::bind(&local_addr).unwrap();
}
.parse()?;
let socket = UdpSocket::bind(&local_addr)?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
let processing = socket
.send_dgram(get_stdin_data(), &remote_addr)
socket
.send_dgram(get_stdin_data()?, &remote_addr)
.and_then(|(socket, _)| socket.recv_dgram(vec![0u8; MAX_DATAGRAM_SIZE]))
.map(|(_, data, len, _)| {
println!(
@@ -66,9 +65,6 @@ fn main() {
String::from_utf8_lossy(&data[..len])
)
})
.wait();
match processing {
Ok(_) => {}
Err(e) => eprintln!("Encountered an error: {}", e),
}
.wait()?;
Ok(())
}
+8 -7
View File
@@ -8,26 +8,26 @@
#![deny(warnings)]
extern crate env_logger;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
use std::net::SocketAddr;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_codec::BytesCodec;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
let addr: SocketAddr = "127.0.0.1:0".parse()?;
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr).unwrap();
let b = UdpSocket::bind(&addr).unwrap();
let b_addr = b.local_addr().unwrap();
let a = UdpSocket::bind(&addr)?;
let b = UdpSocket::bind(&addr)?;
let b_addr = b.local_addr()?;
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
@@ -61,4 +61,5 @@ fn main() {
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
Ok(())
}
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
+4 -5
View File
@@ -3,12 +3,12 @@ name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.4"
version = "0.1.6"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.3"
documentation = "https://docs.rs/tokio-async-await/0.1.6"
description = """
Experimental async/await support for Tokio
"""
@@ -21,10 +21,9 @@ async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
# tokio-codec = { version = "0.1.0", path = "../tokio-codec" }
tokio = "0.1.8"
hyper = "0.12.8"
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+3 -5
View File
@@ -8,19 +8,17 @@ guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate.
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.34.0-nightly or later.
Add this to your `Cargo.toml`:
```toml
# At the very top of the file
cargo-features = ["edition"]
# In the `[packages]` section
edition = "2018"
# In the `[dependencies]` section
tokio-async-await = "0.1.0"
tokio = {version = "0.1.15", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
+1 -1
View File
@@ -61,7 +61,7 @@ async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()>
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
await!(lines_tx.send_async(line)).unwrap();
}
});
@@ -1,4 +1,4 @@
#![feature(await_macro, async_await)]
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
+4 -4
View File
@@ -2,15 +2,15 @@
#[macro_export]
macro_rules! await {
($e:expr) => {{
use $crate::std_await;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
#[allow(unused_imports)]
use $crate::compat::backward::IntoAwaitable as IntoAwaitableBackward;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
use $crate::std_await;
#[allow(unused_mut)]
let mut e = $e;
let e = e.into_awaitable();
std_await!(e)
}}
}};
}
+27 -33
View File
@@ -1,16 +1,9 @@
use futures::{Future, Poll};
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::future::{
Future as StdFuture,
};
use std::ptr::NonNull;
use std::task::{
LocalWaker,
Poll as StdPoll,
UnsafeWake,
Waker,
};
use std::ptr;
use std::task::{Poll as StdPoll, RawWaker, RawWakerVTable, Waker};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
@@ -19,7 +12,7 @@ pub struct Compat<T>(Pin<Box<T>>);
impl<T> Compat<T> {
/// Create a new `Compat` backed by `future`.
pub fn new(future: T) -> Compat<T> {
Compat(Box::pinned(future))
Compat(Box::pin(future))
}
}
@@ -31,7 +24,8 @@ pub trait IntoAwaitable {
}
impl<T> IntoAwaitable for T
where T: StdFuture,
where
T: StdFuture,
{
type Awaitable = Self;
@@ -41,7 +35,8 @@ where T: StdFuture,
}
impl<T, Item, Error> Future for Compat<T>
where T: StdFuture<Output = Result<Item, Error>>,
where
T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
@@ -49,9 +44,9 @@ where T: StdFuture<Output = Result<Item, Error>>,
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let local_waker = noop_local_waker();
let waker = noop_waker();
let res = self.0.as_mut().poll(&local_waker);
let res = self.0.as_mut().poll(&waker);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
@@ -63,27 +58,26 @@ where T: StdFuture<Output = Result<Item, Error>>,
// ===== NoopWaker =====
struct NoopWaker;
fn noop_local_waker() -> LocalWaker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { LocalWaker::new(w) }
fn noop_raw_waker() -> RawWaker {
RawWaker::new(ptr::null(), &NOOP_WAKER_VTABLE)
}
fn noop_waker() -> Waker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { Waker::new(w) }
unsafe { Waker::new_unchecked(noop_raw_waker()) }
}
unsafe impl UnsafeWake for NoopWaker {
unsafe fn clone_raw(&self) -> Waker {
noop_waker()
}
unsafe fn drop_raw(&self) {
}
unsafe fn wake(&self) {
panic!("NoopWake cannot wake");
}
unsafe fn clone_raw(_data: *const ()) -> RawWaker {
noop_raw_waker()
}
unsafe fn drop_raw(_data: *const ()) {}
unsafe fn wake(_data: *const ()) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
const NOOP_WAKER_VTABLE: RawWakerVTable = RawWakerVTable {
clone: clone_raw,
drop: drop_raw,
wake,
};
+10 -11
View File
@@ -1,17 +1,15 @@
use futures::{Async, Future};
use futures::{Future, Async};
use std::marker::Unpin;
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{LocalWaker, Poll as StdPoll};
use std::task::{Poll as StdPoll, Waker};
/// Converts an 0.1 `Future` into an 0.3 `Future`.
#[derive(Debug)]
pub struct Compat<T>(T);
pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T, E>> {
use futures::Async::{Ready, NotReady};
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
@@ -21,9 +19,9 @@ pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T,
}
pub(crate) fn convert_poll_stream<T, E>(
poll: Result<Async<Option<T>>, E>) -> StdPoll<Option<Result<T, E>>>
{
use futures::Async::{Ready, NotReady};
poll: Result<Async<Option<T>>, E>,
) -> StdPoll<Option<Result<T, E>>> {
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(Some(val))) => StdPoll::Ready(Some(Ok(val))),
@@ -50,12 +48,13 @@ impl<T: Future + Unpin> IntoAwaitable for T {
}
impl<T> StdFuture for Compat<T>
where T: Future + Unpin
where
T: Future + Unpin,
{
type Output = Result<T::Item, T::Error>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> StdPoll<Self::Output> {
use futures::Async::{Ready, NotReady};
fn poll(mut self: Pin<&mut Self>, _waker: &Waker) -> StdPoll<Self::Output> {
use futures::Async::{NotReady, Ready};
// TODO: wire in cx
+1 -1
View File
@@ -1,4 +1,4 @@
#![doc(hidden)]
pub mod forward;
pub mod backward;
pub mod forward;
+3 -5
View File
@@ -1,11 +1,9 @@
use tokio_io::AsyncWrite;
use std::io;
use std::future::Future;
use std::marker::Unpin;
use std::io;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Poll, Waker};
/// A future used to fully flush an I/O object.
#[derive(Debug)]
@@ -25,7 +23,7 @@ impl<'a, T: AsyncWrite + ?Sized> Flush<'a, T> {
impl<'a, T: AsyncWrite + ?Sized> Future for Flush<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _wx: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _wx: &Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
convert_poll(self.writer.poll_flush())
}
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future which can be used to read bytes.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Read<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> Read<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> Read<'a, T> {
Read {
reader,
buf,
}
Read { reader, buf }
}
}
impl<'a, T: AsyncRead + ?Sized> Future for Read<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for ReadExact<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> ReadExact<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> ReadExact<'a, T> {
ReadExact {
reader,
buf,
}
ReadExact { reader, buf }
}
}
@@ -34,7 +30,7 @@ fn eof() -> io::Error {
impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -47,7 +43,7 @@ impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
this.buf = rest;
}
if n == 0 {
return Poll::Ready(Err(eof()))
return Poll::Ready(Err(eof()));
}
}
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future used to write data.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Write<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Write<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> Write<'a, T> {
Write {
writer,
buf,
}
Write { writer, buf }
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Write<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<usize>> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<io::Result<usize>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for WriteAll<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> WriteAll<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> WriteAll<'a, T> {
WriteAll {
writer,
buf,
}
WriteAll { writer, buf }
}
}
@@ -34,7 +30,7 @@ fn zero_write() -> io::Error {
impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<()>> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<io::Result<()>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -48,7 +44,7 @@ impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
}
if n == 0 {
return Poll::Ready(Err(zero_write()))
return Poll::Ready(Err(zero_write()));
}
}
+5 -85
View File
@@ -1,14 +1,6 @@
#![cfg(feature = "async-await-preview")]
#![feature(
rust_2018_preview,
arbitrary_self_types,
async_await,
await_macro,
futures_api,
pin,
)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.4")]
#![feature(rust_2018_preview, async_await, await_macro, futures_api)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.6")]
#![deny(missing_docs, missing_debug_implementations)]
#![cfg_attr(test, deny(warnings))]
@@ -24,12 +16,10 @@ macro_rules! try_ready {
($x:expr) => {
match $x {
std::task::Poll::Ready(Ok(x)) => x,
std::task::Poll::Ready(Err(e)) =>
return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending =>
return std::task::Poll::Pending,
std::task::Poll::Ready(Err(e)) => return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending => return std::task::Poll::Pending,
}
}
};
}
#[macro_use]
@@ -39,77 +29,7 @@ pub mod io;
pub mod sink;
pub mod stream;
/*
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_main::prelude::*;
#[doc(inline)]
pub use crate::async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt,
},
};
}
*/
// Rename the `await` macro in `std`. This is used by the redefined
// `await` macro in this crate.
#[doc(hidden)]
pub use std::await as std_await;
/*
use std::future::{Future as StdFuture};
fn run<T: futures::Future<Item = (), Error = ()>>(t: T) {
drop(t);
}
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
run(future);
unimplemented!();
}
*/
/*
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use crate::async_await::compat::backward;
spawn(backward::Compat::new(async || {
let _ = await!(future);
Ok(())
}));
}
*/
-2
View File
@@ -6,8 +6,6 @@ pub use self::send::Send;
use futures::Sink;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Sink` types.
pub trait SinkExt: Sink {
/// Send an item into the sink.
+2 -3
View File
@@ -3,7 +3,6 @@ use futures::Sink;
use std::future::Future;
use std::task::{self, Poll};
use std::marker::Unpin;
use std::pin::Pin;
/// Future for the `SinkExt::send_async` combinator, which sends a value to a
@@ -28,9 +27,9 @@ impl<'a, T: Sink + Unpin + ?Sized> Send<'a, T> {
impl<T: Sink + Unpin + ?Sized> Future for Send<'_, T> {
type Output = Result<(), T::SinkError>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
use futures::AsyncSink::{Ready, NotReady};
use futures::AsyncSink::{NotReady, Ready};
if let Some(item) = self.item.take() {
match self.sink.start_send(item) {
-2
View File
@@ -6,8 +6,6 @@ pub use self::next::Next;
use futures::Stream;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Stream` types.
pub trait StreamExt: Stream {
/// Creates a future that resolves to the next item in the stream.
+2 -3
View File
@@ -1,9 +1,8 @@
use futures::Stream;
use std::future::Future;
use std::marker::Unpin;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Poll, Waker};
/// A future of the next element of a stream.
#[derive(Debug)]
@@ -22,7 +21,7 @@ impl<'a, T: Stream + Unpin> Next<'a, T> {
impl<'a, T: Stream + Unpin> Future for Next<'a, T> {
type Output = Option<Result<T::Item, T::Error>>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll_stream;
convert_poll_stream(self.stream.poll())
+3
View File
@@ -0,0 +1,3 @@
# 0.1.0 (February 23, 2019)
* Initial release
@@ -1,8 +1,11 @@
[package]
name = "tokio-channel"
name = "tokio-buf"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
@@ -10,11 +13,17 @@ authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-channel/0.1.0"
documentation = "https://docs.rs/tokio-buf/0.1.0/tokio_buf"
description = """
Channels for asynchronous communication using Tokio.
Asynchronous stream of byte buffers
"""
categories = ["asynchronous"]
[dependencies]
bytes = "0.4.10"
either = { version = "1.5", optional = true}
futures = "0.1.23"
[features]
default = ["util"]
util = ["bytes/either", "either"]
+25
View File
@@ -0,0 +1,25 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+35
View File
@@ -0,0 +1,35 @@
# tokio-buf
Asynchronous stream of byte buffers
[Documenation](https://docs.rs/tokio-buf)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-buf = "0.1.0"
```
Next, add this to your crate:
```rust
extern crate tokio_buf;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-buf) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+99
View File
@@ -0,0 +1,99 @@
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.0")]
#![deny(missing_docs, missing_debug_implementations, unreachable_pub)]
#![cfg_attr(test, deny(warnings))]
//! Asynchronous stream of bytes.
//!
//! This crate contains the `BufStream` trait and a number of combinators for
//! this trait. The trait is similar to `Stream` in the `futures` library, but
//! instead of yielding arbitrary values, it only yields types that implement
//! `Buf` (i.e, byte collections).
extern crate bytes;
#[cfg(feature = "util")]
extern crate either;
#[allow(unused)]
#[macro_use]
extern crate futures;
mod never;
mod size_hint;
mod str;
mod u8;
#[cfg(feature = "util")]
pub mod util;
pub use self::size_hint::SizeHint;
#[doc(inline)]
#[cfg(feature = "util")]
pub use util::BufStreamExt;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
}
+22
View File
@@ -0,0 +1,22 @@
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
+56
View File
@@ -0,0 +1,56 @@
use std::u64;
/// A `BufStream` size hint
///
/// The default implementation returns:
///
/// * 0 for `available`
/// * 0 for `lower`
/// * `None` for `upper`.
#[derive(Debug, Default, Clone)]
pub struct SizeHint {
lower: u64,
upper: Option<u64>,
}
impl SizeHint {
/// Returns a new `SizeHint` with default values
pub fn new() -> SizeHint {
SizeHint::default()
}
/// Returns the lower bound of data that the `BufStream` will yield before
/// completing.
pub fn lower(&self) -> u64 {
self.lower
}
/// Set the value of the `lower` hint.
///
/// # Panics
///
/// The function panics if `value` is less than `upper`.
pub fn set_lower(&mut self, value: u64) {
assert!(value <= self.upper.unwrap_or(u64::MAX));
self.lower = value;
}
/// Returns the upper bound of data the `BufStream` will yield before
/// completing, or `None` if the value is unknown.
pub fn upper(&self) -> Option<u64> {
self.upper
}
/// Set the value of the `upper` hint value.
///
/// # Panics
///
/// This function panics if `value` is less than `lower`.
pub fn set_upper(&mut self, value: u64) {
// There is no need to check `available` as that is guaranteed to be
// less than or equal to `lower`.
assert!(value >= self.lower, "`value` is less than than `lower`");
self.upper = Some(value);
}
}
+39
View File
@@ -0,0 +1,39 @@
use never::Never;
use BufStream;
use futures::Poll;
use std::io;
use std::mem;
impl BufStream for String {
type Item = io::Cursor<Vec<u8>>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
let bytes = mem::replace(self, Default::default()).into_bytes();
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
}
impl BufStream for &'static str {
type Item = io::Cursor<&'static [u8]>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
let bytes = mem::replace(self, Default::default()).as_bytes();
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
}
+66
View File
@@ -0,0 +1,66 @@
use bytes::{Bytes, BytesMut};
use futures::Poll;
use never::Never;
use std::io;
use BufStream;
impl BufStream for Vec<u8> {
type Item = io::Cursor<Vec<u8>>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for &'static [u8] {
type Item = io::Cursor<&'static [u8]>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for Bytes {
type Item = io::Cursor<Bytes>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
impl BufStream for BytesMut {
type Item = io::Cursor<BytesMut>;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.is_empty() {
return Ok(None.into());
}
poll_bytes(self)
}
}
fn poll_bytes<T: Default>(buf: &mut T) -> Poll<Option<io::Cursor<T>>, Never> {
use std::mem;
let bytes = mem::replace(buf, Default::default());
let buf = io::Cursor::new(bytes);
Ok(Some(buf).into())
}
+46
View File
@@ -0,0 +1,46 @@
use BufStream;
use either::Either;
use futures::Poll;
/// A buf stream that sequences two buf streams together.
///
/// `Chain` values are produced by the `chain` function on `BufStream`.
#[derive(Debug)]
pub struct Chain<T, U> {
left: Option<T>,
right: U,
}
impl<T, U> Chain<T, U> {
pub(crate) fn new(left: T, right: U) -> Chain<T, U> {
Chain {
left: Some(left),
right,
}
}
}
impl<T, U> BufStream for Chain<T, U>
where
T: BufStream,
U: BufStream<Error = T::Error>,
{
type Item = Either<T::Item, U::Item>;
type Error = T::Error;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if let Some(ref mut stream) = self.left {
let res = try_ready!(stream.poll_buf());
if res.is_some() {
return Ok(res.map(Either::Left).into());
}
}
self.left = None;
let res = try_ready!(self.right.poll_buf());
Ok(res.map(Either::Right).into())
}
}
+101
View File
@@ -0,0 +1,101 @@
use super::FromBufStream;
use BufStream;
use futures::{Future, Poll};
/// Consumes a buf stream, collecting the data into a single byte container.
///
/// `Collect` values are produced by `BufStream::collect`.
#[derive(Debug)]
pub struct Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
stream: T,
builder: Option<U::Builder>,
}
/// Errors returned from `Collect` future.
#[derive(Debug)]
pub struct CollectError<T, U> {
inner: Error<T, U>,
}
#[derive(Debug)]
enum Error<T, U> {
Stream(T),
Collect(U),
}
impl<T, U> Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
pub(crate) fn new(stream: T) -> Collect<T, U> {
let builder = U::builder(&stream.size_hint());
Collect {
stream,
builder: Some(builder),
}
}
}
impl<T, U> Future for Collect<T, U>
where
T: BufStream,
U: FromBufStream<T::Item>,
{
type Item = U;
type Error = CollectError<T::Error, U::Error>;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
loop {
let res = self.stream.poll_buf().map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
match try_ready!(res) {
Some(mut buf) => {
let builder = self.builder.as_mut().expect("cannot poll after done");
U::extend(builder, &mut buf, &self.stream.size_hint()).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
}
None => {
let builder = self.builder.take().expect("cannot poll after done");
let value = U::build(builder).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
return Ok(value.into());
}
}
}
}
}
// ===== impl CollectError =====
impl<T, U> CollectError<T, U> {
/// Returns `true` if the error was caused by polling the stream.
pub fn is_stream_err(&self) -> bool {
match self.inner {
Error::Stream(_) => true,
_ => false,
}
}
/// Returns `true` if the error happened while collecting the data.
pub fn is_collect_err(&self) -> bool {
match self.inner {
Error::Collect(_) => true,
_ => false,
}
}
}
+112
View File
@@ -0,0 +1,112 @@
use SizeHint;
use bytes::{Buf, BufMut};
use std::usize;
/// Conversion from a `BufStream`.
///
/// By implementing `FromBufStream` for a type, you define how it will be
/// created from a buf stream. This is common for types which describe byte
/// storage of some kind.
///
/// `FromBufStream` is rarely called explicitly, and it is instead used through
/// `BufStream`'s `collect` method.
pub trait FromBufStream<T: Buf>: Sized {
/// Type that is used to build `Self` while the `BufStream` is being
/// consumed.
type Builder;
/// Error that might happen on conversion.
type Error;
/// Create a new, empty, builder. The provided `hint` can be used to inform
/// reserving capacity.
fn builder(hint: &SizeHint) -> Self::Builder;
/// Extend the builder with the `Buf`.
///
/// This method is called whenever a new `Buf` value is obtained from the
/// buf stream.
///
/// The provided size hint represents the state of the stream **after**
/// `buf` has been yielded. The lower bound represents the minimum amount of
/// data that will be provided after this call to `extend` returns.
fn extend(builder: &mut Self::Builder, buf: &mut T, hint: &SizeHint)
-> Result<(), Self::Error>;
/// Finalize the building of `Self`.
///
/// Called once the buf stream is fully consumed.
fn build(builder: Self::Builder) -> Result<Self, Self::Error>;
}
/// Error returned from collecting into a `Vec<u8>`
#[derive(Debug)]
pub struct CollectVecError {
_p: (),
}
impl<T: Buf> FromBufStream<T> for Vec<u8> {
type Builder = Vec<u8>;
type Error = CollectVecError;
fn builder(_hint: &SizeHint) -> Vec<u8> {
Vec::new()
}
fn extend(builder: &mut Self, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
let lower = hint.lower();
// If the lower bound is greater than `usize::MAX` then we have a
// problem
if lower > usize::MAX as u64 {
return Err(CollectVecError { _p: () });
}
let mut reserve = lower as usize;
// If `upper` is set, use this value if it is less than or equal to 64.
// This only really impacts the first iteration.
match hint.upper() {
Some(upper) if upper <= 64 => {
reserve = upper as usize;
}
_ => {}
}
// hint.lower() represents the minimum amount of data that will be
// received *after* this function call. We reserve this amount on top of
// the amount of data in `buf`.
reserve = match reserve.checked_add(buf.remaining()) {
Some(n) => n,
None => return Err(CollectVecError { _p: () }),
};
// Always reserve 64 bytes the first time, unless `upper` is set and is
// less than 64.
if builder.is_empty() {
reserve = reserve.max(match hint.upper() {
Some(upper) if upper < 64 => upper as usize,
_ => 64,
});
}
// Make sure overflow won't happen when reserving
if reserve.checked_add(builder.len()).is_none() {
return Err(CollectVecError { _p: () });
}
// Reserve space
builder.reserve(reserve);
// Copy the data
builder.put(buf);
Ok(())
}
fn build(builder: Self) -> Result<Self, Self::Error> {
Ok(builder)
}
}
+76
View File
@@ -0,0 +1,76 @@
use BufStream;
use bytes::Buf;
use futures::Poll;
/// Limits the stream to a maximum amount of data.
#[derive(Debug)]
pub struct Limit<T> {
stream: T,
remaining: u64,
}
/// Errors returned from `Limit`.
#[derive(Debug)]
pub struct LimitError<T> {
/// When `None`, limit was reached
inner: Option<T>,
}
impl<T> Limit<T> {
pub(crate) fn new(stream: T, amount: u64) -> Limit<T> {
Limit {
stream,
remaining: amount,
}
}
}
impl<T> BufStream for Limit<T>
where
T: BufStream,
{
type Item = T::Item;
type Error = LimitError<T::Error>;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
use futures::Async::Ready;
if self.stream.size_hint().lower() > self.remaining {
return Err(LimitError { inner: None });
}
let res = self
.stream
.poll_buf()
.map_err(|err| LimitError { inner: Some(err) });
match res {
Ok(Ready(Some(ref buf))) => {
if buf.remaining() as u64 > self.remaining {
self.remaining = 0;
return Err(LimitError { inner: None });
}
self.remaining -= buf.remaining() as u64;
}
_ => {}
}
res
}
}
// ===== impl LimitError =====
impl<T> LimitError<T> {
/// Returns `true` if the error was caused by polling the stream.
pub fn is_stream_err(&self) -> bool {
self.inner.is_some()
}
/// Returns `true` if the stream reached its limit.
pub fn is_limit_err(&self) -> bool {
self.inner.is_none()
}
}
+73
View File
@@ -0,0 +1,73 @@
//! Types and utilities for working with `BufStream`.
mod chain;
mod collect;
mod from;
mod limit;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::limit::Limit;
pub mod error {
//! Error types
pub use super::collect::CollectError;
pub use super::from::CollectVecError;
pub use super::limit::LimitError;
}
use BufStream;
impl<T> BufStreamExt for T where T: BufStream {}
/// An extension trait for `BufStream`'s that provides a variety of convenient
/// adapters.
pub trait BufStreamExt: BufStream {
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
}
+66
View File
@@ -0,0 +1,66 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use tokio_buf::{BufStream, SizeHint};
#[macro_use]
mod support;
// ===== test `SizeHint` =====
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
// ===== BufStream impelmentations for misc types =====
#[test]
fn str_buf_stream() {
let mut bs = "hello world".to_string();
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
let mut bs = "hello world";
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
}
+145
View File
@@ -0,0 +1,145 @@
#![cfg(feature = "ext")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Future;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
// ===== test `chain()` =====
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"]).chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[Ok(Ready("foo")), Ok(NotReady), Ok(Ready("bar"))]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
// ===== Test `collect()` =====
#[test]
fn collect_vec() {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world, one two three");
}
// ===== Test limit() =====
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world").limit(5).collect::<Vec<_>>().wait();
assert!(res.is_err());
let res = one("hello world").limit(10).collect::<Vec<_>>().wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"]).limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
+132
View File
@@ -0,0 +1,132 @@
#![allow(unused)]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Poll;
use tokio_buf::{BufStream, SizeHint};
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
};
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
};
}
// ===== Test utils =====
pub fn one(buf: &'static str) -> Mock {
list(&[buf])
}
pub fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
pub fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
pub struct Mock {
pub polls: VecDeque<Poll<&'static [u8], ()>>,
pub size_hint: SizeHint,
}
#[derive(Debug)]
pub struct MockBuf {
pub data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
View File
-51
View File
@@ -1,51 +0,0 @@
Copyright (c) 2018 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
Copyright (c) 2016 futures-rs Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
View File
-14
View File
@@ -1,14 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio-channel/0.1.0")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
//! Asynchronous channels.
//!
//! This crate provides channels that can be used to communicate between
//! asynchronous tasks.
extern crate futures;
pub mod mpsc;
pub mod oneshot;
mod lock;
-105
View File
@@ -1,105 +0,0 @@
//! A "mutex" which only supports `try_lock`
//!
//! As a futures library the eventual call to an event loop should be the only
//! thing that ever blocks, so this is assisted with a fast user-space
//! implementation of a lock that can only have a `try_lock` operation.
use std::cell::UnsafeCell;
use std::ops::{Deref, DerefMut};
use std::sync::atomic::Ordering::SeqCst;
use std::sync::atomic::AtomicBool;
/// A "mutex" around a value, similar to `std::sync::Mutex<T>`.
///
/// This lock only supports the `try_lock` operation, however, and does not
/// implement poisoning.
#[derive(Debug)]
pub struct Lock<T> {
locked: AtomicBool,
data: UnsafeCell<T>,
}
/// Sentinel representing an acquired lock through which the data can be
/// accessed.
pub struct TryLock<'a, T: 'a> {
__ptr: &'a Lock<T>,
}
// The `Lock` structure is basically just a `Mutex<T>`, and these two impls are
// intended to mirror the standard library's corresponding impls for `Mutex<T>`.
//
// If a `T` is sendable across threads, so is the lock, and `T` must be sendable
// across threads to be `Sync` because it allows mutable access from multiple
// threads.
unsafe impl<T: Send> Send for Lock<T> {}
unsafe impl<T: Send> Sync for Lock<T> {}
impl<T> Lock<T> {
/// Creates a new lock around the given value.
pub fn new(t: T) -> Lock<T> {
Lock {
locked: AtomicBool::new(false),
data: UnsafeCell::new(t),
}
}
/// Attempts to acquire this lock, returning whether the lock was acquired or
/// not.
///
/// If `Some` is returned then the data this lock protects can be accessed
/// through the sentinel. This sentinel allows both mutable and immutable
/// access.
///
/// If `None` is returned then the lock is already locked, either elsewhere
/// on this thread or on another thread.
pub fn try_lock(&self) -> Option<TryLock<T>> {
if !self.locked.swap(true, SeqCst) {
Some(TryLock { __ptr: self })
} else {
None
}
}
}
impl<'a, T> Deref for TryLock<'a, T> {
type Target = T;
fn deref(&self) -> &T {
// The existence of `TryLock` represents that we own the lock, so we
// can safely access the data here.
unsafe { &*self.__ptr.data.get() }
}
}
impl<'a, T> DerefMut for TryLock<'a, T> {
fn deref_mut(&mut self) -> &mut T {
// The existence of `TryLock` represents that we own the lock, so we
// can safely access the data here.
//
// Additionally, we're the *only* `TryLock` in existence so mutable
// access should be ok.
unsafe { &mut *self.__ptr.data.get() }
}
}
impl<'a, T> Drop for TryLock<'a, T> {
fn drop(&mut self) {
self.__ptr.locked.store(false, SeqCst);
}
}
#[cfg(test)]
mod tests {
use super::Lock;
#[test]
fn smoke() {
let a = Lock::new(1);
let mut a1 = a.try_lock().unwrap();
assert!(a.try_lock().is_none());
assert_eq!(*a1, 1);
*a1 = 2;
drop(a1);
assert_eq!(*a.try_lock().unwrap(), 2);
assert_eq!(*a.try_lock().unwrap(), 2);
}
}
-989
View File
@@ -1,989 +0,0 @@
//! A multi-producer, single-consumer, futures-aware, FIFO queue with back pressure.
//!
//! A channel can be used as a communication primitive between tasks running on
//! `futures-rs` executors. Channel creation provides `Receiver` and `Sender`
//! handles. `Receiver` implements `Stream` and allows a task to read values
//! out of the channel. If there is no message to read from the channel, the
//! current task will be notified when a new value is sent. `Sender` implements
//! the `Sink` trait and allows a task to send messages into the channel. If
//! the channel is at capacity, then send will be rejected and the task will be
//! notified when additional capacity is available.
//!
//! # Disconnection
//!
//! When all `Sender` handles have been dropped, it is no longer possible to
//! send values into the channel. This is considered the termination event of
//! the stream. As such, `Sender::poll` will return `Ok(Ready(None))`.
//!
//! If the receiver handle is dropped, then messages can no longer be read out
//! of the channel. In this case, a `send` will result in an error.
//!
//! # Clean Shutdown
//!
//! If the `Receiver` is simply dropped, then it is possible for there to be
//! messages still in the channel that will not be processed. As such, it is
//! usually desirable to perform a "clean" shutdown. To do this, the receiver
//! will first call `close`, which will prevent any further messages to be sent
//! into the channel. Then, the receiver consumes the channel to completion, at
//! which point the receiver can be dropped.
// At the core, the channel uses an atomic FIFO queue for message passing. This
// queue is used as the primary coordination primitive. In order to enforce
// capacity limits and handle back pressure, a secondary FIFO queue is used to
// send parked task handles.
//
// The general idea is that the channel is created with a `buffer` size of `n`.
// The channel capacity is `n + num-senders`. Each sender gets one "guaranteed"
// slot to hold a message. This allows `Sender` to know for a fact that a send
// will succeed *before* starting to do the actual work of sending the value.
// Since most of this work is lock-free, once the work starts, it is impossible
// to safely revert.
//
// If the sender is unable to process a send operation, then the current
// task is parked and the handle is sent on the parked task queue.
//
// Note that the implementation guarantees that the channel capacity will never
// exceed the configured limit, however there is no *strict* guarantee that the
// receiver will wake up a parked task *immediately* when a slot becomes
// available. However, it will almost always unpark a task when a slot becomes
// available and it is *guaranteed* that a sender will be unparked when the
// message that caused the sender to become parked is read out of the channel.
//
// The steps for sending a message are roughly:
//
// 1) Increment the channel message count
// 2) If the channel is at capacity, push the task handle onto the wait queue
// 3) Push the message onto the message queue.
//
// The steps for receiving a message are roughly:
//
// 1) Pop a message from the message queue
// 2) Pop a task handle from the wait queue
// 3) Decrement the channel message count.
//
// It's important for the order of operations on lock-free structures to happen
// in reverse order between the sender and receiver. This makes the message
// queue the primary coordination structure and establishes the necessary
// happens-before semantics required for the acquire / release semantics used
// by the queue structure.
use mpsc::queue::{Queue, PopResult};
use futures::task::{self, Task};
use futures::{Async, AsyncSink, Poll, StartSend, Sink, Stream};
use std::fmt;
use std::error::Error;
use std::any::Any;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::SeqCst;
use std::sync::{Arc, Mutex};
use std::thread;
use std::usize;
mod queue;
/// The transmission end of a channel which is used to send values.
///
/// This is created by the `channel` method.
#[derive(Debug)]
pub struct Sender<T> {
// Channel state shared between the sender and receiver.
inner: Arc<Inner<T>>,
// Handle to the task that is blocked on this sender. This handle is sent
// to the receiver half in order to be notified when the sender becomes
// unblocked.
sender_task: Arc<Mutex<SenderTask>>,
// True if the sender might be blocked. This is an optimization to avoid
// having to lock the mutex most of the time.
maybe_parked: bool,
}
/// The transmission end of a channel which is used to send values.
///
/// This is created by the `unbounded` method.
#[derive(Debug)]
pub struct UnboundedSender<T>(Sender<T>);
trait AssertKinds: Send + Sync + Clone {}
impl AssertKinds for UnboundedSender<u32> {}
/// The receiving end of a channel which implements the `Stream` trait.
///
/// This is a concrete implementation of a stream which can be used to represent
/// a stream of values being computed elsewhere. This is created by the
/// `channel` method.
#[derive(Debug)]
pub struct Receiver<T> {
inner: Arc<Inner<T>>,
}
/// Error type for sending, used when the receiving end of a channel is
/// dropped
#[derive(Clone, PartialEq, Eq)]
pub struct SendError<T>(T);
/// Error type returned from `try_send`
#[derive(Clone, PartialEq, Eq)]
pub struct TrySendError<T> {
kind: TrySendErrorKind<T>,
}
#[derive(Clone, PartialEq, Eq)]
enum TrySendErrorKind<T> {
Full(T),
Disconnected(T),
}
impl<T> fmt::Debug for SendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_tuple("SendError")
.field(&"...")
.finish()
}
}
impl<T> fmt::Display for SendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "send failed because receiver is gone")
}
}
impl<T: Any> Error for SendError<T>
{
fn description(&self) -> &str {
"send failed because receiver is gone"
}
}
impl<T> SendError<T> {
/// Returns the message that was attempted to be sent but failed.
pub fn into_inner(self) -> T {
self.0
}
}
impl<T> fmt::Debug for TrySendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_tuple("TrySendError")
.field(&"...")
.finish()
}
}
impl<T> fmt::Display for TrySendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
if self.is_full() {
write!(fmt, "send failed because channel is full")
} else {
write!(fmt, "send failed because receiver is gone")
}
}
}
impl<T: Any> Error for TrySendError<T> {
fn description(&self) -> &str {
if self.is_full() {
"send failed because channel is full"
} else {
"send failed because receiver is gone"
}
}
}
impl<T> TrySendError<T> {
/// Returns true if this error is a result of the channel being full
pub fn is_full(&self) -> bool {
use self::TrySendErrorKind::*;
match self.kind {
Full(_) => true,
_ => false,
}
}
/// Returns true if this error is a result of the receiver being dropped
pub fn is_disconnected(&self) -> bool {
use self::TrySendErrorKind::*;
match self.kind {
Disconnected(_) => true,
_ => false,
}
}
/// Returns the message that was attempted to be sent but failed.
pub fn into_inner(self) -> T {
use self::TrySendErrorKind::*;
match self.kind {
Full(v) | Disconnected(v) => v,
}
}
}
#[derive(Debug)]
struct Inner<T> {
// Max buffer size of the channel. If `None` then the channel is unbounded.
buffer: Option<usize>,
// Internal channel state. Consists of the number of messages stored in the
// channel as well as a flag signalling that the channel is closed.
state: AtomicUsize,
// Atomic, FIFO queue used to send messages to the receiver
message_queue: Queue<Option<T>>,
// Atomic, FIFO queue used to send parked task handles to the receiver.
parked_queue: Queue<Arc<Mutex<SenderTask>>>,
// Number of senders in existence
num_senders: AtomicUsize,
// Handle to the receiver's task.
recv_task: Mutex<ReceiverTask>,
}
// Struct representation of `Inner::state`.
#[derive(Debug, Clone, Copy)]
struct State {
// `true` when the channel is open
is_open: bool,
// Number of messages in the channel
num_messages: usize,
}
#[derive(Debug)]
struct ReceiverTask {
unparked: bool,
task: Option<Task>,
}
// Returned from Receiver::try_park()
enum TryPark {
Parked,
Closed,
NotEmpty,
}
// The `is_open` flag is stored in the left-most bit of `Inner::state`
const OPEN_MASK: usize = usize::MAX - (usize::MAX >> 1);
// When a new channel is created, it is created in the open state with no
// pending messages.
const INIT_STATE: usize = OPEN_MASK;
// The maximum number of messages that a channel can track is `usize::MAX >> 1`
const MAX_CAPACITY: usize = !(OPEN_MASK);
// The maximum requested buffer size must be less than the maximum capacity of
// a channel. This is because each sender gets a guaranteed slot.
const MAX_BUFFER: usize = MAX_CAPACITY >> 1;
// Sent to the consumer to wake up blocked producers
#[derive(Debug)]
struct SenderTask {
task: Option<Task>,
is_parked: bool,
}
impl SenderTask {
fn new() -> Self {
SenderTask {
task: None,
is_parked: false,
}
}
fn notify(&mut self) {
self.is_parked = false;
if let Some(task) = self.task.take() {
task.notify();
}
}
}
/// Creates an in-memory channel implementation of the `Stream` trait with
/// bounded capacity.
///
/// This method creates a concrete implementation of the `Stream` trait which
/// can be used to send values across threads in a streaming fashion. This
/// channel is unique in that it implements back pressure to ensure that the
/// sender never outpaces the receiver. The channel capacity is equal to
/// `buffer + num-senders`. In other words, each sender gets a guaranteed slot
/// in the channel capacity, and on top of that there are `buffer` "first come,
/// first serve" slots available to all senders.
///
/// The `Receiver` returned implements the `Stream` trait and has access to any
/// number of the associated combinators for transforming the result.
pub fn channel<T>(buffer: usize) -> (Sender<T>, Receiver<T>) {
// Check that the requested buffer size does not exceed the maximum buffer
// size permitted by the system.
assert!(buffer < MAX_BUFFER, "requested buffer size too large");
channel2(Some(buffer))
}
/// Creates an in-memory channel implementation of the `Stream` trait with
/// unbounded capacity.
///
/// This method creates a concrete implementation of the `Stream` trait which
/// can be used to send values across threads in a streaming fashion. A `send`
/// on this channel will always succeed as long as the receive half has not
/// been closed. If the receiver falls behind, messages will be buffered
/// internally.
///
/// **Note** that the amount of available system memory is an implicit bound to
/// the channel. Using an `unbounded` channel has the ability of causing the
/// process to run out of memory. In this case, the process will be aborted.
pub fn unbounded<T>() -> (UnboundedSender<T>, Receiver<T>) {
let (tx, rx) = channel2(None);
(UnboundedSender(tx), rx)
}
fn channel2<T>(buffer: Option<usize>) -> (Sender<T>, Receiver<T>) {
let inner = Arc::new(Inner {
buffer: buffer,
state: AtomicUsize::new(INIT_STATE),
message_queue: Queue::new(),
parked_queue: Queue::new(),
num_senders: AtomicUsize::new(1),
recv_task: Mutex::new(ReceiverTask {
unparked: false,
task: None,
}),
});
let tx = Sender {
inner: inner.clone(),
sender_task: Arc::new(Mutex::new(SenderTask::new())),
maybe_parked: false,
};
let rx = Receiver {
inner: inner,
};
(tx, rx)
}
/*
*
* ===== impl Sender =====
*
*/
impl<T> Sender<T> {
/// Attempts to send a message on this `Sender<T>` without blocking.
///
/// This function, unlike `start_send`, is safe to call whether it's being
/// called on a task or not. Note that this function, however, will *not*
/// attempt to block the current task if the message cannot be sent.
///
/// It is not recommended to call this function from inside of a future,
/// only from an external thread where you've otherwise arranged to be
/// notified when the channel is no longer full.
pub fn try_send(&mut self, msg: T) -> Result<(), TrySendError<T>> {
// If the sender is currently blocked, reject the message
if !self.poll_unparked(false).is_ready() {
return Err(TrySendError {
kind: TrySendErrorKind::Full(msg),
});
}
// The channel has capacity to accept the message, so send it
self.do_send(Some(msg), false)
.map_err(|SendError(v)| {
TrySendError {
kind: TrySendErrorKind::Disconnected(v),
}
})
}
// Do the send without failing
// None means close
fn do_send(&mut self, msg: Option<T>, do_park: bool) -> Result<(), SendError<T>> {
// First, increment the number of messages contained by the channel.
// This operation will also atomically determine if the sender task
// should be parked.
//
// None is returned in the case that the channel has been closed by the
// receiver. This happens when `Receiver::close` is called or the
// receiver is dropped.
let park_self = match self.inc_num_messages(msg.is_none()) {
Some(park_self) => park_self,
None => {
// The receiver has closed the channel. Only abort if actually
// sending a message. It is important that the stream
// termination (None) is always sent. This technically means
// that it is possible for the queue to contain the following
// number of messages:
//
// num-senders + buffer + 1
//
if let Some(msg) = msg {
return Err(SendError(msg));
} else {
return Ok(());
}
}
};
// If the channel has reached capacity, then the sender task needs to
// be parked. This will send the task handle on the parked task queue.
//
// However, when `do_send` is called while dropping the `Sender`,
// `task::current()` can't be called safely. In this case, in order to
// maintain internal consistency, a blank message is pushed onto the
// parked task queue.
if park_self {
self.park(do_park);
}
self.queue_push_and_signal(msg);
Ok(())
}
// Do the send without parking current task.
//
// To be called from unbounded sender.
fn do_send_nb(&self, msg: T) -> Result<(), SendError<T>> {
match self.inc_num_messages(false) {
Some(park_self) => assert!(!park_self),
None => return Err(SendError(msg)),
};
self.queue_push_and_signal(Some(msg));
Ok(())
}
// Push message to the queue and signal to the receiver
fn queue_push_and_signal(&self, msg: Option<T>) {
// Push the message onto the message queue
self.inner.message_queue.push(msg);
// Signal to the receiver that a message has been enqueued. If the
// receiver is parked, this will unpark the task.
self.signal();
}
// Increment the number of queued messages. Returns if the sender should
// block.
fn inc_num_messages(&self, close: bool) -> Option<bool> {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
// The receiver end closed the channel.
if !state.is_open {
return None;
}
// This probably is never hit? Odds are the process will run out of
// memory first. It may be worth to return something else in this
// case?
assert!(state.num_messages < MAX_CAPACITY, "buffer space exhausted; \
sending this messages would overflow the state");
state.num_messages += 1;
// The channel is closed by all sender handles being dropped.
if close {
state.is_open = false;
}
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => {
// Block if the current number of pending messages has exceeded
// the configured buffer size
let park_self = match self.inner.buffer {
Some(buffer) => state.num_messages > buffer,
None => false,
};
return Some(park_self)
}
Err(actual) => curr = actual,
}
}
}
// Signal to the receiver task that a message has been enqueued
fn signal(&self) {
// TODO
// This logic can probably be improved by guarding the lock with an
// atomic.
//
// Do this step first so that the lock is dropped when
// `unpark` is called
let task = {
let mut recv_task = self.inner.recv_task.lock().unwrap();
// If the receiver has already been unparked, then there is nothing
// more to do
if recv_task.unparked {
return;
}
// Setting this flag enables the receiving end to detect that
// an unpark event happened in order to avoid unnecessarily
// parking.
recv_task.unparked = true;
recv_task.task.take()
};
if let Some(task) = task {
task.notify();
}
}
fn park(&mut self, can_park: bool) {
// TODO: clean up internal state if the task::current will fail
let task = if can_park {
Some(task::current())
} else {
None
};
{
let mut sender = self.sender_task.lock().unwrap();
sender.task = task;
sender.is_parked = true;
}
// Send handle over queue
let t = self.sender_task.clone();
self.inner.parked_queue.push(t);
// Check to make sure we weren't closed after we sent our task on the
// queue
let state = decode_state(self.inner.state.load(SeqCst));
self.maybe_parked = state.is_open;
}
/// Polls the channel to determine if there is guaranteed to be capacity to send at least one
/// item without waiting.
///
/// Returns `Ok(Async::Ready(_))` if there is sufficient capacity, or returns
/// `Ok(Async::NotReady)` if the channel is not guaranteed to have capacity. Returns
/// `Err(SendError(_))` if the receiver has been dropped.
///
/// # Panics
///
/// This method will panic if called from outside the context of a task or future.
pub fn poll_ready(&mut self) -> Poll<(), SendError<()>> {
let state = decode_state(self.inner.state.load(SeqCst));
if !state.is_open {
return Err(SendError(()));
}
Ok(self.poll_unparked(true))
}
fn poll_unparked(&mut self, do_park: bool) -> Async<()> {
// First check the `maybe_parked` variable. This avoids acquiring the
// lock in most cases
if self.maybe_parked {
// Get a lock on the task handle
let mut task = self.sender_task.lock().unwrap();
if !task.is_parked {
self.maybe_parked = false;
return Async::Ready(())
}
// At this point, an unpark request is pending, so there will be an
// unpark sometime in the future. We just need to make sure that
// the correct task will be notified.
//
// Update the task in case the `Sender` has been moved to another
// task
task.task = if do_park {
Some(task::current())
} else {
None
};
Async::NotReady
} else {
Async::Ready(())
}
}
}
impl<T> Sink for Sender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
// If the sender is currently blocked, reject the message before doing
// any work.
if !self.poll_unparked(true).is_ready() {
return Ok(AsyncSink::NotReady(msg));
}
// The channel has capacity to accept the message, so send it.
self.do_send(Some(msg), true)?;
Ok(AsyncSink::Ready)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<T> UnboundedSender<T> {
/// Sends the provided message along this channel.
///
/// This is an unbounded sender, so this function differs from `Sink::send`
/// by ensuring the return type reflects that the channel is always ready to
/// receive messages.
#[deprecated(note = "renamed to `unbounded_send`")]
#[doc(hidden)]
pub fn send(&self, msg: T) -> Result<(), SendError<T>> {
self.unbounded_send(msg)
}
/// Sends the provided message along this channel.
///
/// This is an unbounded sender, so this function differs from `Sink::send`
/// by ensuring the return type reflects that the channel is always ready to
/// receive messages.
pub fn unbounded_send(&self, msg: T) -> Result<(), SendError<T>> {
self.0.do_send_nb(msg)
}
}
impl<T> Sink for UnboundedSender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
self.0.start_send(msg)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
self.0.poll_complete()
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<'a, T> Sink for &'a UnboundedSender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
self.0.do_send_nb(msg)?;
Ok(AsyncSink::Ready)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<T> Clone for UnboundedSender<T> {
fn clone(&self) -> UnboundedSender<T> {
UnboundedSender(self.0.clone())
}
}
impl<T> Clone for Sender<T> {
fn clone(&self) -> Sender<T> {
// Since this atomic op isn't actually guarding any memory and we don't
// care about any orderings besides the ordering on the single atomic
// variable, a relaxed ordering is acceptable.
let mut curr = self.inner.num_senders.load(SeqCst);
loop {
// If the maximum number of senders has been reached, then fail
if curr == self.inner.max_senders() {
panic!("cannot clone `Sender` -- too many outstanding senders");
}
debug_assert!(curr < self.inner.max_senders());
let next = curr + 1;
let actual = self.inner.num_senders.compare_and_swap(curr, next, SeqCst);
// The ABA problem doesn't matter here. We only care that the
// number of senders never exceeds the maximum.
if actual == curr {
return Sender {
inner: self.inner.clone(),
sender_task: Arc::new(Mutex::new(SenderTask::new())),
maybe_parked: false,
};
}
curr = actual;
}
}
}
impl<T> Drop for Sender<T> {
fn drop(&mut self) {
// Ordering between variables don't matter here
let prev = self.inner.num_senders.fetch_sub(1, SeqCst);
if prev == 1 {
let _ = self.do_send(None, false);
}
}
}
/*
*
* ===== impl Receiver =====
*
*/
impl<T> Receiver<T> {
/// Closes the receiving half
///
/// This prevents any further messages from being sent on the channel while
/// still enabling the receiver to drain messages that are buffered.
pub fn close(&mut self) {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
if !state.is_open {
break
}
state.is_open = false;
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => break,
Err(actual) => curr = actual,
}
}
// Wake up any threads waiting as they'll see that we've closed the
// channel and will continue on their merry way.
loop {
match unsafe { self.inner.parked_queue.pop() } {
PopResult::Data(task) => {
task.lock().unwrap().notify();
}
PopResult::Empty => break,
PopResult::Inconsistent => thread::yield_now(),
}
}
}
fn next_message(&mut self) -> Async<Option<T>> {
// Pop off a message
loop {
match unsafe { self.inner.message_queue.pop() } {
PopResult::Data(msg) => {
return Async::Ready(msg);
}
PopResult::Empty => {
// The queue is empty, return NotReady
return Async::NotReady;
}
PopResult::Inconsistent => {
// Inconsistent means that there will be a message to pop
// in a short time. This branch can only be reached if
// values are being produced from another thread, so there
// are a few ways that we can deal with this:
//
// 1) Spin
// 2) thread::yield_now()
// 3) task::current().unwrap() & return NotReady
//
// For now, thread::yield_now() is used, but it would
// probably be better to spin a few times then yield.
thread::yield_now();
}
}
}
}
// Unpark a single task handle if there is one pending in the parked queue
fn unpark_one(&mut self) {
loop {
match unsafe { self.inner.parked_queue.pop() } {
PopResult::Data(task) => {
task.lock().unwrap().notify();
return;
}
PopResult::Empty => {
// Queue empty, no task to wake up.
return;
}
PopResult::Inconsistent => {
// Same as above
thread::yield_now();
}
}
}
}
// Try to park the receiver task
fn try_park(&self) -> TryPark {
let curr = self.inner.state.load(SeqCst);
let state = decode_state(curr);
// If the channel is closed, then there is no need to park.
if !state.is_open && state.num_messages == 0 {
return TryPark::Closed;
}
// First, track the task in the `recv_task` slot
let mut recv_task = self.inner.recv_task.lock().unwrap();
if recv_task.unparked {
// Consume the `unpark` signal without actually parking
recv_task.unparked = false;
return TryPark::NotEmpty;
}
recv_task.task = Some(task::current());
TryPark::Parked
}
fn dec_num_messages(&self) {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
state.num_messages -= 1;
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => break,
Err(actual) => curr = actual,
}
}
}
}
impl<T> Stream for Receiver<T> {
type Item = T;
type Error = ();
fn poll(&mut self) -> Poll<Option<T>, ()> {
loop {
// Try to read a message off of the message queue.
let msg = match self.next_message() {
Async::Ready(msg) => msg,
Async::NotReady => {
// There are no messages to read, in this case, attempt to
// park. The act of parking will verify that the channel is
// still empty after the park operation has completed.
match self.try_park() {
TryPark::Parked => {
// The task was parked, and the channel is still
// empty, return NotReady.
return Ok(Async::NotReady);
}
TryPark::Closed => {
// The channel is closed, there will be no further
// messages.
return Ok(Async::Ready(None));
}
TryPark::NotEmpty => {
// A message has been sent while attempting to
// park. Loop again, the next iteration is
// guaranteed to get the message.
continue;
}
}
}
};
// If there are any parked task handles in the parked queue, pop
// one and unpark it.
self.unpark_one();
// Decrement number of messages
self.dec_num_messages();
// Return the message
return Ok(Async::Ready(msg));
}
}
}
impl<T> Drop for Receiver<T> {
fn drop(&mut self) {
// Drain the channel of all pending messages
self.close();
while self.next_message().is_ready() {
// ...
}
}
}
/*
*
* ===== impl Inner =====
*
*/
impl<T> Inner<T> {
// The return value is such that the total number of messages that can be
// enqueued into the channel will never exceed MAX_CAPACITY
fn max_senders(&self) -> usize {
match self.buffer {
Some(buffer) => MAX_CAPACITY - buffer,
None => MAX_BUFFER,
}
}
}
unsafe impl<T: Send> Send for Inner<T> {}
unsafe impl<T: Send> Sync for Inner<T> {}
/*
*
* ===== Helpers =====
*
*/
fn decode_state(num: usize) -> State {
State {
is_open: num & OPEN_MASK == OPEN_MASK,
num_messages: num & MAX_CAPACITY,
}
}
fn encode_state(state: &State) -> usize {
let mut num = state.num_messages;
if state.is_open {
num |= OPEN_MASK;
}
num
}
-151
View File
@@ -1,151 +0,0 @@
/* Copyright (c) 2010-2011 Dmitry Vyukov. All rights reserved.
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY DMITRY VYUKOV "AS IS" AND ANY EXPRESS OR IMPLIED
* WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
* SHALL DMITRY VYUKOV OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
* OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are
* those of the authors and should not be interpreted as representing official
* policies, either expressed or implied, of Dmitry Vyukov.
*/
//! A mostly lock-free multi-producer, single consumer queue.
//!
//! This module contains an implementation of a concurrent MPSC queue. This
//! queue can be used to share data between threads, and is also used as the
//! building block of channels in rust.
//!
//! Note that the current implementation of this queue has a caveat of the `pop`
//! method, and see the method for more information about it. Due to this
//! caveat, this queue may not be appropriate for all use-cases.
// http://www.1024cores.net/home/lock-free-algorithms
// /queues/non-intrusive-mpsc-node-based-queue
// NOTE: this implementation is lifted from the standard library and only
// slightly modified
pub use self::PopResult::*;
use std::prelude::v1::*;
use std::cell::UnsafeCell;
use std::ptr;
use std::sync::atomic::{AtomicPtr, Ordering};
/// A result of the `pop` function.
pub enum PopResult<T> {
/// Some data has been popped
Data(T),
/// The queue is empty
Empty,
/// The queue is in an inconsistent state. Popping data should succeed, but
/// some pushers have yet to make enough progress in order allow a pop to
/// succeed. It is recommended that a pop() occur "in the near future" in
/// order to see if the sender has made progress or not
Inconsistent,
}
#[derive(Debug)]
struct Node<T> {
next: AtomicPtr<Node<T>>,
value: Option<T>,
}
/// The multi-producer single-consumer structure. This is not cloneable, but it
/// may be safely shared so long as it is guaranteed that there is only one
/// popper at a time (many pushers are allowed).
#[derive(Debug)]
pub struct Queue<T> {
head: AtomicPtr<Node<T>>,
tail: UnsafeCell<*mut Node<T>>,
}
unsafe impl<T: Send> Send for Queue<T> { }
unsafe impl<T: Send> Sync for Queue<T> { }
impl<T> Node<T> {
unsafe fn new(v: Option<T>) -> *mut Node<T> {
Box::into_raw(Box::new(Node {
next: AtomicPtr::new(ptr::null_mut()),
value: v,
}))
}
}
impl<T> Queue<T> {
/// Creates a new queue that is safe to share among multiple producers and
/// one consumer.
pub fn new() -> Queue<T> {
let stub = unsafe { Node::new(None) };
Queue {
head: AtomicPtr::new(stub),
tail: UnsafeCell::new(stub),
}
}
/// Pushes a new value onto this queue.
pub fn push(&self, t: T) {
unsafe {
let n = Node::new(Some(t));
let prev = self.head.swap(n, Ordering::AcqRel);
(*prev).next.store(n, Ordering::Release);
}
}
/// Pops some data from this queue.
///
/// Note that the current implementation means that this function cannot
/// return `Option<T>`. It is possible for this queue to be in an
/// inconsistent state where many pushes have succeeded and completely
/// finished, but pops cannot return `Some(t)`. This inconsistent state
/// happens when a pusher is preempted at an inopportune moment.
///
/// This inconsistent state means that this queue does indeed have data, but
/// it does not currently have access to it at this time.
///
/// This function is unsafe because only one thread can call it at a time.
pub unsafe fn pop(&self) -> PopResult<T> {
let tail = *self.tail.get();
let next = (*tail).next.load(Ordering::Acquire);
if !next.is_null() {
*self.tail.get() = next;
assert!((*tail).value.is_none());
assert!((*next).value.is_some());
let ret = (*next).value.take().unwrap();
drop(Box::from_raw(tail));
return Data(ret);
}
if self.head.load(Ordering::Acquire) == tail {Empty} else {Inconsistent}
}
}
impl<T> Drop for Queue<T> {
fn drop(&mut self) {
unsafe {
let mut cur = *self.tail.get();
while !cur.is_null() {
let next = (*cur).next.load(Ordering::Relaxed);
drop(Box::from_raw(cur));
cur = next;
}
}
}
}
-426
View File
@@ -1,426 +0,0 @@
//! A one-shot, futures-aware channel
use lock::Lock;
use futures::{Future, Poll, Async};
use futures::task::{self, Task};
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::sync::atomic::Ordering::SeqCst;
use std::error::Error;
use std::fmt;
/// A future representing the completion of a computation happening elsewhere in
/// memory.
///
/// This is created by the `oneshot::channel` function.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct Receiver<T> {
inner: Arc<Inner<T>>,
}
/// Represents the completion half of a oneshot through which the result of a
/// computation is signaled.
///
/// This is created by the `oneshot::channel` function.
#[derive(Debug)]
pub struct Sender<T> {
inner: Arc<Inner<T>>,
}
/// Internal state of the `Receiver`/`Sender` pair above. This is all used as
/// the internal synchronization between the two for send/recv operations.
#[derive(Debug)]
struct Inner<T> {
/// Indicates whether this oneshot is complete yet. This is filled in both
/// by `Sender::drop` and by `Receiver::drop`, and both sides interpret it
/// appropriately.
///
/// For `Receiver`, if this is `true`, then it's guaranteed that `data` is
/// unlocked and ready to be inspected.
///
/// For `Sender` if this is `true` then the oneshot has gone away and it
/// can return ready from `poll_cancel`.
complete: AtomicBool,
/// The actual data being transferred as part of this `Receiver`. This is
/// filled in by `Sender::complete` and read by `Receiver::poll`.
///
/// Note that this is protected by `Lock`, but it is in theory safe to
/// replace with an `UnsafeCell` as it's actually protected by `complete`
/// above. I wouldn't recommend doing this, however, unless someone is
/// supremely confident in the various atomic orderings here and there.
data: Lock<Option<T>>,
/// Field to store the task which is blocked in `Receiver::poll`.
///
/// This is filled in when a oneshot is polled but not ready yet. Note that
/// the `Lock` here, unlike in `data` above, is important to resolve races.
/// Both the `Receiver` and the `Sender` halves understand that if they
/// can't acquire the lock then some important interference is happening.
rx_task: Lock<Option<Task>>,
/// Like `rx_task` above, except for the task blocked in
/// `Sender::poll_cancel`. Additionally, `Lock` cannot be `UnsafeCell`.
tx_task: Lock<Option<Task>>,
}
/// Creates a new futures-aware, one-shot channel.
///
/// This function is similar to Rust's channels found in the standard library.
/// Two halves are returned, the first of which is a `Sender` handle, used to
/// signal the end of a computation and provide its value. The second half is a
/// `Receiver` which implements the `Future` trait, resolving to the value that
/// was given to the `Sender` handle.
///
/// Each half can be separately owned and sent across threads/tasks.
///
/// # Examples
///
/// ```
/// extern crate tokio_channel;
/// extern crate futures;
///
/// use tokio_channel::oneshot;
/// use futures::*;
/// use std::thread;
///
/// # fn main() {
/// let (p, c) = oneshot::channel::<i32>();
///
/// thread::spawn(|| {
/// c.map(|i| {
/// println!("got: {}", i);
/// }).wait();
/// });
///
/// p.send(3).unwrap();
/// # }
/// ```
pub fn channel<T>() -> (Sender<T>, Receiver<T>) {
let inner = Arc::new(Inner::new());
let receiver = Receiver {
inner: inner.clone(),
};
let sender = Sender {
inner: inner,
};
(sender, receiver)
}
impl<T> Inner<T> {
fn new() -> Inner<T> {
Inner {
complete: AtomicBool::new(false),
data: Lock::new(None),
rx_task: Lock::new(None),
tx_task: Lock::new(None),
}
}
fn send(&self, t: T) -> Result<(), T> {
if self.complete.load(SeqCst) {
return Err(t)
}
// Note that this lock acquisition may fail if the receiver
// is closed and sets the `complete` flag to true, whereupon
// the receiver may call `poll()`.
if let Some(mut slot) = self.data.try_lock() {
assert!(slot.is_none());
*slot = Some(t);
drop(slot);
// If the receiver called `close()` between the check at the
// start of the function, and the lock being released, then
// the receiver may not be around to receive it, so try to
// pull it back out.
if self.complete.load(SeqCst) {
// If lock acquisition fails, then receiver is actually
// receiving it, so we're good.
if let Some(mut slot) = self.data.try_lock() {
if let Some(t) = slot.take() {
return Err(t);
}
}
}
Ok(())
} else {
// Must have been closed
Err(t)
}
}
fn poll_cancel(&self) -> Poll<(), ()> {
// Fast path up first, just read the flag and see if our other half is
// gone. This flag is set both in our destructor and the oneshot
// destructor, but our destructor hasn't run yet so if it's set then the
// oneshot is gone.
if self.complete.load(SeqCst) {
return Ok(Async::Ready(()))
}
// If our other half is not gone then we need to park our current task
// and move it into the `notify_cancel` slot to get notified when it's
// actually gone.
//
// If `try_lock` fails, then the `Receiver` is in the process of using
// it, so we can deduce that it's now in the process of going away and
// hence we're canceled. If it succeeds then we just store our handle.
//
// Crucially we then check `oneshot_gone` *again* before we return.
// While we were storing our handle inside `notify_cancel` the `Receiver`
// may have been dropped. The first thing it does is set the flag, and
// if it fails to acquire the lock it assumes that we'll see the flag
// later on. So... we then try to see the flag later on!
let handle = task::current();
match self.tx_task.try_lock() {
Some(mut p) => *p = Some(handle),
None => return Ok(Async::Ready(())),
}
if self.complete.load(SeqCst) {
Ok(Async::Ready(()))
} else {
Ok(Async::NotReady)
}
}
fn is_canceled(&self) -> bool {
self.complete.load(SeqCst)
}
fn drop_tx(&self) {
// Flag that we're a completed `Sender` and try to wake up a receiver.
// Whether or not we actually stored any data will get picked up and
// translated to either an item or cancellation.
//
// Note that if we fail to acquire the `rx_task` lock then that means
// we're in one of two situations:
//
// 1. The receiver is trying to block in `poll`
// 2. The receiver is being dropped
//
// In the first case it'll check the `complete` flag after it's done
// blocking to see if it succeeded. In the latter case we don't need to
// wake up anyone anyway. So in both cases it's ok to ignore the `None`
// case of `try_lock` and bail out.
//
// The first case crucially depends on `Lock` using `SeqCst` ordering
// under the hood. If it instead used `Release` / `Acquire` ordering,
// then it would not necessarily synchronize with `inner.complete`
// and deadlock might be possible, as was observed in
// https://github.com/rust-lang-nursery/futures-rs/pull/219.
self.complete.store(true, SeqCst);
if let Some(mut slot) = self.rx_task.try_lock() {
if let Some(task) = slot.take() {
drop(slot);
task.notify();
}
}
}
fn close_rx(&self) {
// Flag our completion and then attempt to wake up the sender if it's
// blocked. See comments in `drop` below for more info
self.complete.store(true, SeqCst);
if let Some(mut handle) = self.tx_task.try_lock() {
if let Some(task) = handle.take() {
drop(handle);
task.notify()
}
}
}
fn recv(&self) -> Poll<T, Canceled> {
let mut done = false;
// Check to see if some data has arrived. If it hasn't then we need to
// block our task.
//
// Note that the acquisition of the `rx_task` lock might fail below, but
// the only situation where this can happen is during `Sender::drop`
// when we are indeed completed already. If that's happening then we
// know we're completed so keep going.
if self.complete.load(SeqCst) {
done = true;
} else {
let task = task::current();
match self.rx_task.try_lock() {
Some(mut slot) => *slot = Some(task),
None => done = true,
}
}
// If we're `done` via one of the paths above, then look at the data and
// figure out what the answer is. If, however, we stored `rx_task`
// successfully above we need to check again if we're completed in case
// a message was sent while `rx_task` was locked and couldn't notify us
// otherwise.
//
// If we're not done, and we're not complete, though, then we've
// successfully blocked our task and we return `NotReady`.
if done || self.complete.load(SeqCst) {
// If taking the lock fails, the sender will realise that the we're
// `done` when it checks the `complete` flag on the way out, and will
// treat the send as a failure.
if let Some(mut slot) = self.data.try_lock() {
if let Some(data) = slot.take() {
return Ok(data.into());
}
}
Err(Canceled)
} else {
Ok(Async::NotReady)
}
}
fn drop_rx(&self) {
// Indicate to the `Sender` that we're done, so any future calls to
// `poll_cancel` are weeded out.
self.complete.store(true, SeqCst);
// If we've blocked a task then there's no need for it to stick around,
// so we need to drop it. If this lock acquisition fails, though, then
// it's just because our `Sender` is trying to take the task, so we
// let them take care of that.
if let Some(mut slot) = self.rx_task.try_lock() {
let task = slot.take();
drop(slot);
drop(task);
}
// Finally, if our `Sender` wants to get notified of us going away, it
// would have stored something in `tx_task`. Here we try to peel that
// out and unpark it.
//
// Note that the `try_lock` here may fail, but only if the `Sender` is
// in the process of filling in the task. If that happens then we
// already flagged `complete` and they'll pick that up above.
if let Some(mut handle) = self.tx_task.try_lock() {
if let Some(task) = handle.take() {
drop(handle);
task.notify()
}
}
}
}
impl<T> Sender<T> {
#[deprecated(note = "renamed to `send`", since = "0.1.11")]
#[doc(hidden)]
#[cfg(feature = "with-deprecated")]
pub fn complete(self, t: T) {
drop(self.send(t));
}
/// Completes this oneshot with a successful result.
///
/// This function will consume `self` and indicate to the other end, the
/// `Receiver`, that the value provided is the result of the computation this
/// represents.
///
/// If the value is successfully enqueued for the remote end to receive,
/// then `Ok(())` is returned. If the receiving end was deallocated before
/// this function was called, however, then `Err` is returned with the value
/// provided.
pub fn send(self, t: T) -> Result<(), T> {
self.inner.send(t)
}
/// Polls this `Sender` half to detect whether the `Receiver` this has
/// paired with has gone away.
///
/// This function can be used to learn about when the `Receiver` (consumer)
/// half has gone away and nothing will be able to receive a message sent
/// from `send`.
///
/// If `Ready` is returned then it means that the `Receiver` has disappeared
/// and the result this `Sender` would otherwise produce should no longer
/// be produced.
///
/// If `NotReady` is returned then the `Receiver` is still alive and may be
/// able to receive a message if sent. The current task, however, is
/// scheduled to receive a notification if the corresponding `Receiver` goes
/// away.
///
/// # Panics
///
/// Like `Future::poll`, this function will panic if it's not called from
/// within the context of a task. In other words, this should only ever be
/// called from inside another future.
///
/// If you're calling this function from a context that does not have a
/// task, then you can use the `is_canceled` API instead.
pub fn poll_cancel(&mut self) -> Poll<(), ()> {
self.inner.poll_cancel()
}
/// Tests to see whether this `Sender`'s corresponding `Receiver`
/// has gone away.
///
/// This function can be used to learn about when the `Receiver` (consumer)
/// half has gone away and nothing will be able to receive a message sent
/// from `send`.
///
/// Note that this function is intended to *not* be used in the context of a
/// future. If you're implementing a future you probably want to call the
/// `poll_cancel` function which will block the current task if the
/// cancellation hasn't happened yet. This can be useful when working on a
/// non-futures related thread, though, which would otherwise panic if
/// `poll_cancel` were called.
pub fn is_canceled(&self) -> bool {
self.inner.is_canceled()
}
}
impl<T> Drop for Sender<T> {
fn drop(&mut self) {
self.inner.drop_tx()
}
}
/// Error returned from a `Receiver<T>` whenever the corresponding `Sender<T>`
/// is dropped.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct Canceled;
impl fmt::Display for Canceled {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "oneshot canceled")
}
}
impl Error for Canceled {
fn description(&self) -> &str {
"oneshot canceled"
}
}
impl<T> Receiver<T> {
/// Gracefully close this receiver, preventing sending any future messages.
///
/// Any `send` operation which happens after this method returns is
/// guaranteed to fail. Once this method is called the normal `poll` method
/// can be used to determine whether a message was actually sent or not. If
/// `Canceled` is returned from `poll` then no message was sent.
pub fn close(&mut self) {
self.inner.close_rx()
}
}
impl<T> Future for Receiver<T> {
type Item = T;
type Error = Canceled;
fn poll(&mut self) -> Poll<T, Canceled> {
self.inner.recv()
}
}
impl<T> Drop for Receiver<T> {
fn drop(&mut self) {
self.inner.drop_rx()
}
}
-22
View File
@@ -1,22 +0,0 @@
extern crate tokio_channel;
extern crate futures;
use tokio_channel::mpsc::*;
use futures::prelude::*;
use std::thread;
#[test]
fn smoke() {
let (mut sender, receiver) = channel(1);
let t = thread::spawn(move ||{
while let Ok(s) = sender.send(42).wait() {
sender = s;
}
});
receiver.take(3).for_each(|_| Ok(())).wait().unwrap();
t.join().unwrap()
}
-481
View File
@@ -1,481 +0,0 @@
extern crate tokio_channel;
#[macro_use]
extern crate futures;
mod support;
use support::*;
use tokio_channel::mpsc;
use tokio_channel::oneshot;
use futures::prelude::*;
use futures::future::lazy;
use std::thread;
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicUsize, Ordering};
trait AssertSend: Send {}
impl AssertSend for mpsc::Sender<i32> {}
impl AssertSend for mpsc::Receiver<i32> {}
#[test]
fn send_recv() {
let (tx, rx) = mpsc::channel::<i32>(16);
let mut rx = rx.wait();
tx.send(1).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(1));
}
#[test]
fn send_recv_no_buffer() {
let (mut tx, mut rx) = mpsc::channel::<i32>(0);
// Run on a task context
lazy(move || {
assert!(tx.poll_complete().unwrap().is_ready());
assert!(tx.poll_ready().unwrap().is_ready());
// Send first message
let res = tx.start_send(1).unwrap();
assert!(is_ready(&res));
assert!(tx.poll_ready().unwrap().is_not_ready());
// Send second message
let res = tx.start_send(2).unwrap();
assert!(!is_ready(&res));
// Take the value
assert_eq!(rx.poll().unwrap(), Async::Ready(Some(1)));
assert!(tx.poll_ready().unwrap().is_ready());
let res = tx.start_send(2).unwrap();
assert!(is_ready(&res));
assert!(tx.poll_ready().unwrap().is_not_ready());
// Take the value
assert_eq!(rx.poll().unwrap(), Async::Ready(Some(2)));
assert!(tx.poll_ready().unwrap().is_ready());
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn send_shared_recv() {
let (tx1, rx) = mpsc::channel::<i32>(16);
let tx2 = tx1.clone();
let mut rx = rx.wait();
tx1.send(1).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(1));
tx2.send(2).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(2));
}
#[test]
fn send_recv_threads() {
let (tx, rx) = mpsc::channel::<i32>(16);
let mut rx = rx.wait();
thread::spawn(move|| {
tx.send(1).wait().unwrap();
});
assert_eq!(rx.next().unwrap(), Ok(1));
}
#[test]
fn send_recv_threads_no_capacity() {
let (tx, rx) = mpsc::channel::<i32>(0);
let mut rx = rx.wait();
let (readytx, readyrx) = mpsc::channel::<()>(2);
let mut readyrx = readyrx.wait();
let t = thread::spawn(move|| {
let readytx = readytx.sink_map_err(|_| panic!());
let (a, b) = tx.send(1).join(readytx.send(())).wait().unwrap();
a.send(2).join(b.send(())).wait().unwrap();
});
drop(readyrx.next().unwrap());
assert_eq!(rx.next().unwrap(), Ok(1));
drop(readyrx.next().unwrap());
assert_eq!(rx.next().unwrap(), Ok(2));
t.join().unwrap();
}
#[test]
fn recv_close_gets_none() {
let (mut tx, mut rx) = mpsc::channel::<i32>(10);
// Run on a task context
lazy(move || {
rx.close();
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
assert!(tx.poll_ready().is_err());
drop(tx);
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn tx_close_gets_none() {
let (_, mut rx) = mpsc::channel::<i32>(10);
// Run on a task context
lazy(move || {
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn stress_shared_unbounded() {
const AMT: u32 = 10000;
const NTHREADS: u32 = 8;
let (tx, rx) = mpsc::unbounded::<i32>();
let mut rx = rx.wait();
let t = thread::spawn(move|| {
for _ in 0..AMT * NTHREADS {
assert_eq!(rx.next().unwrap(), Ok(1));
}
if rx.next().is_some() {
panic!();
}
});
for _ in 0..NTHREADS {
let tx = tx.clone();
thread::spawn(move|| {
for _ in 0..AMT {
tx.unbounded_send(1).unwrap();
}
});
}
drop(tx);
t.join().ok().unwrap();
}
#[test]
fn stress_shared_bounded_hard() {
const AMT: u32 = 10000;
const NTHREADS: u32 = 8;
let (tx, rx) = mpsc::channel::<i32>(0);
let mut rx = rx.wait();
let t = thread::spawn(move|| {
for _ in 0..AMT * NTHREADS {
assert_eq!(rx.next().unwrap(), Ok(1));
}
if rx.next().is_some() {
panic!();
}
});
for _ in 0..NTHREADS {
let mut tx = tx.clone();
thread::spawn(move|| {
for _ in 0..AMT {
tx = tx.send(1).wait().unwrap();
}
});
}
drop(tx);
t.join().ok().unwrap();
}
#[test]
fn stress_receiver_multi_task_bounded_hard() {
const AMT: usize = 10_000;
const NTHREADS: u32 = 2;
let (mut tx, rx) = mpsc::channel::<usize>(0);
let rx = Arc::new(Mutex::new(Some(rx)));
let n = Arc::new(AtomicUsize::new(0));
let mut th = vec![];
for _ in 0..NTHREADS {
let rx = rx.clone();
let n = n.clone();
let t = thread::spawn(move || {
let mut i = 0;
loop {
i += 1;
let mut lock = rx.lock().ok().unwrap();
match lock.take() {
Some(mut rx) => {
if i % 5 == 0 {
let (item, rest) = rx.into_future().wait().ok().unwrap();
if item.is_none() {
break;
}
n.fetch_add(1, Ordering::Relaxed);
*lock = Some(rest);
} else {
// Just poll
let n = n.clone();
let r = lazy(move || {
let r = match rx.poll().unwrap() {
Async::Ready(Some(_)) => {
n.fetch_add(1, Ordering::Relaxed);
*lock = Some(rx);
false
}
Async::Ready(None) => {
true
}
Async::NotReady => {
*lock = Some(rx);
false
}
};
Ok::<bool, ()>(r)
}).wait().unwrap();
if r {
break;
}
}
}
None => break,
}
}
});
th.push(t);
}
for i in 0..AMT {
tx = tx.send(i).wait().unwrap();
}
drop(tx);
for t in th {
t.join().unwrap();
}
assert_eq!(AMT, n.load(Ordering::Relaxed));
}
/// Stress test that receiver properly receives all the messages
/// after sender dropped.
#[test]
fn stress_drop_sender() {
fn list() -> Box<Stream<Item=i32, Error=u32>> {
let (tx, rx) = mpsc::channel(1);
tx.send(Ok(1))
.and_then(|tx| tx.send(Ok(2)))
.and_then(|tx| tx.send(Ok(3)))
.forget();
Box::new(rx.then(|r| r.unwrap()))
}
for _ in 0..10000 {
assert_eq!(list().wait().collect::<Result<Vec<_>, _>>(),
Ok(vec![1, 2, 3]));
}
}
/// Stress test that after receiver dropped,
/// no messages are lost.
fn stress_close_receiver_iter() {
let (tx, rx) = mpsc::unbounded();
let (unwritten_tx, unwritten_rx) = std::sync::mpsc::channel();
let th = thread::spawn(move || {
for i in 1.. {
if let Err(_) = tx.unbounded_send(i) {
unwritten_tx.send(i).expect("unwritten_tx");
return;
}
}
});
let mut rx = rx.wait();
// Read one message to make sure thread effectively started
assert_eq!(Some(Ok(1)), rx.next());
rx.get_mut().close();
for i in 2.. {
match rx.next() {
Some(Ok(r)) => assert!(i == r),
Some(Err(_)) => unreachable!(),
None => {
let unwritten = unwritten_rx.recv().expect("unwritten_rx");
assert_eq!(unwritten, i);
th.join().unwrap();
return;
}
}
}
}
#[test]
fn stress_close_receiver() {
for _ in 0..10000 {
stress_close_receiver_iter();
}
}
/// Tests that after `poll_ready` indicates capacity a channel can always send without waiting.
#[test]
fn stress_poll_ready() {
// A task which checks channel capacity using poll_ready, and pushes items onto the channel when
// ready.
struct SenderTask {
sender: mpsc::Sender<u32>,
count: u32,
}
impl Future for SenderTask {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
// In a loop, check if the channel is ready. If so, push an item onto the channel
// (asserting that it doesn't attempt to block).
while self.count > 0 {
try_ready!(self.sender.poll_ready().map_err(|_| ()));
assert!(self.sender.start_send(self.count).unwrap().is_ready());
self.count -= 1;
}
Ok(Async::Ready(()))
}
}
const AMT: u32 = 1000;
const NTHREADS: u32 = 8;
/// Run a stress test using the specified channel capacity.
fn stress(capacity: usize) {
let (tx, rx) = mpsc::channel(capacity);
let mut threads = Vec::new();
for _ in 0..NTHREADS {
let sender = tx.clone();
threads.push(thread::spawn(move || {
SenderTask {
sender: sender,
count: AMT,
}.wait()
}));
}
drop(tx);
let mut rx = rx.wait();
for _ in 0..AMT * NTHREADS {
assert!(rx.next().is_some());
}
assert!(rx.next().is_none());
for thread in threads {
thread.join().unwrap().unwrap();
}
}
stress(0);
stress(1);
stress(8);
stress(16);
}
fn is_ready<T>(res: &AsyncSink<T>) -> bool {
match *res {
AsyncSink::Ready => true,
_ => false,
}
}
#[test]
fn try_send_1() {
const N: usize = 3000;
let (mut tx, rx) = mpsc::channel(0);
let t = thread::spawn(move || {
for i in 0..N {
loop {
if tx.try_send(i).is_ok() {
break
}
}
}
});
for (i, j) in rx.wait().enumerate() {
assert_eq!(i, j.unwrap());
}
t.join().unwrap();
}
#[test]
fn try_send_2() {
let (mut tx, rx) = mpsc::channel(0);
tx.try_send("hello").unwrap();
let (readytx, readyrx) = oneshot::channel::<()>();
let th = thread::spawn(|| {
lazy(|| {
assert!(tx.start_send("fail").unwrap().is_not_ready());
Ok::<_, ()>(())
}).wait().unwrap();
drop(readytx);
tx.send("goodbye").wait().unwrap();
});
let mut rx = rx.wait();
drop(readyrx.wait());
assert_eq!(rx.next(), Some(Ok("hello")));
assert_eq!(rx.next(), Some(Ok("goodbye")));
assert!(rx.next().is_none());
th.join().unwrap();
}
#[test]
fn try_send_fail() {
let (mut tx, rx) = mpsc::channel(0);
let mut rx = rx.wait();
tx.try_send("hello").unwrap();
// This should fail
assert!(tx.try_send("fail").is_err());
assert_eq!(rx.next(), Some(Ok("hello")));
tx.try_send("goodbye").unwrap();
drop(tx);
assert_eq!(rx.next(), Some(Ok("goodbye")));
assert!(rx.next().is_none());
}
-124
View File
@@ -1,124 +0,0 @@
extern crate tokio_channel;
extern crate futures;
mod support;
use support::*;
use tokio_channel::oneshot::*;
use futures::prelude::*;
use futures::future::{lazy, ok};
use std::sync::mpsc;
use std::thread;
#[test]
fn smoke_poll() {
let (mut tx, rx) = channel::<u32>();
lazy(|| {
assert!(tx.poll_cancel().unwrap().is_not_ready());
assert!(tx.poll_cancel().unwrap().is_not_ready());
drop(rx);
assert!(tx.poll_cancel().unwrap().is_ready());
assert!(tx.poll_cancel().unwrap().is_ready());
ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn cancel_notifies() {
let (tx, rx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
WaitForCancel { tx: tx }.then(move |v| tx2.send(v)).forget();
drop(rx);
rx2.recv().unwrap().unwrap();
}
struct WaitForCancel {
tx: Sender<u32>,
}
impl Future for WaitForCancel {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.tx.poll_cancel()
}
}
#[test]
fn cancel_lots() {
let (tx, rx) = mpsc::channel::<(Sender<_>, mpsc::Sender<_>)>();
let t = thread::spawn(move || {
for (tx, tx2) in rx {
WaitForCancel { tx: tx }.then(move |v| tx2.send(v)).forget();
}
});
for _ in 0..20000 {
let (otx, orx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
tx.send((otx, tx2)).unwrap();
drop(orx);
rx2.recv().unwrap().unwrap();
}
drop(tx);
t.join().unwrap();
}
#[test]
fn close() {
let (mut tx, mut rx) = channel::<u32>();
rx.close();
assert!(rx.poll().is_err());
assert!(tx.poll_cancel().unwrap().is_ready());
}
#[test]
fn close_wakes() {
let (tx, mut rx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
let t = thread::spawn(move || {
rx.close();
rx2.recv().unwrap();
});
WaitForCancel { tx: tx }.wait().unwrap();
tx2.send(()).unwrap();
t.join().unwrap();
}
#[test]
fn is_canceled() {
let (tx, rx) = channel::<u32>();
assert!(!tx.is_canceled());
drop(rx);
assert!(tx.is_canceled());
}
#[test]
fn cancel_sends() {
let (tx, rx) = mpsc::channel::<Sender<_>>();
let t = thread::spawn(move || {
for otx in rx {
let _ = otx.send(42);
}
});
for _ in 0..20000 {
let (otx, mut orx) = channel::<u32>();
tx.send(otx).unwrap();
orx.close();
// Not necessary to wrap in a task because the implementation of oneshot
// never calls `task::current()` if the channel has been closed already.
let _ = orx.poll();
}
drop(tx);
t.join().unwrap();
}
-16
View File
@@ -1,16 +0,0 @@
use futures::Future;
pub trait ForgetExt {
fn forget(self);
}
impl<F> ForgetExt for F
where F: Future + Sized + Send + 'static,
F::Item: Send,
F::Error: Send
{
fn forget(self) {
use std::thread;
thread::spawn(|| self.wait());
}
}
+1 -1
View File
@@ -18,6 +18,6 @@ Utilities for encoding and decoding frames.
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
bytes = "0.4.7"
futures = "0.1.18"
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+5 -3
View File
@@ -1,6 +1,6 @@
use bytes::{Bytes, BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use bytes::{BufMut, Bytes, BytesMut};
use std::io;
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -8,7 +8,9 @@ pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec { BytesCodec(()) }
pub fn new() -> BytesCodec {
BytesCodec(())
}
}
impl Decoder for BytesCodec {
+1 -8
View File
@@ -19,14 +19,7 @@ extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use tokio_io::_tokio_codec::{Decoder, Encoder, Framed, FramedParts, FramedRead, FramedWrite};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
+4 -6
View File
@@ -1,6 +1,6 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{cmp, io, str, usize};
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -103,10 +103,8 @@ impl LinesCodec {
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
str::from_utf8(buf)
.map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
@@ -153,7 +151,7 @@ impl Decoder for LinesCodec {
self.is_discarding = true;
Err(io::Error::new(
io::ErrorKind::Other,
"line length limit exceeded"
"line length limit exceeded",
))
} else {
// We didn't find a line or reach the length limit, so the next
+33 -9
View File
@@ -1,8 +1,8 @@
extern crate tokio_codec;
extern crate bytes;
extern crate tokio_codec;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
use bytes::{BufMut, Bytes, BytesMut};
use tokio_codec::{BytesCodec, Decoder, Encoder, LinesCodec};
#[test]
fn bytes_decoder() {
@@ -27,13 +27,17 @@ fn bytes_encoder() {
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf)
.unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf)
.unwrap();
}
#[test]
@@ -68,17 +72,32 @@ fn lines_decoder_max_length() {
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 2", line);
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 4", line);
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("", line);
assert_eq!(None, codec.decode(buf).unwrap());
@@ -87,7 +106,12 @@ fn lines_decoder_max_length() {
assert_eq!(None, codec.decode(buf).unwrap());
let line = codec.decode_eof(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("\rk", line);
assert_eq!(None, codec.decode(buf).unwrap());
+10 -10
View File
@@ -1,13 +1,13 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use futures::{Stream, Future};
use bytes::{Buf, BufMut, BytesMut, IntoBuf};
use futures::{Future, Stream};
use std::io::{self, Read};
use tokio_codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_codec::{Decoder, Encoder, Framed, FramedParts};
use tokio_io::AsyncRead;
use bytes::{BytesMut, Buf, BufMut, IntoBuf};
const INITIAL_CAPACITY: usize = 8 * 1024;
@@ -45,8 +45,10 @@ struct DontReadIntoThis;
impl Read for DontReadIntoThis {
fn read(&mut self, _: &mut [u8]) -> io::Result<usize> {
Err(io::Error::new(io::ErrorKind::Other,
"Read into something you weren't supposed to."))
Err(io::Error::new(
io::ErrorKind::Other,
"Read into something you weren't supposed to.",
))
}
}
@@ -61,9 +63,7 @@ fn can_read_from_existing_buf() {
let num = framed
.into_future()
.map(|(first_num, _)| {
first_num.unwrap()
})
.map(|(first_num, _)| first_num.unwrap())
.wait()
.map_err(|e| e.0)
.unwrap();
+7 -8
View File
@@ -1,17 +1,17 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, FramedRead};
use tokio_io::AsyncRead;
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf};
use bytes::{Buf, BytesMut, IntoBuf};
use futures::Async::{NotReady, Ready};
use futures::Stream;
use futures::Async::{Ready, NotReady};
use std::io::{self, Read};
use std::collections::VecDeque;
use std::io::{self, Read};
macro_rules! mock {
($($x:expr,)*) => {{
@@ -212,5 +212,4 @@ impl Read for Mock {
}
}
impl AsyncRead for Mock {
}
impl AsyncRead for Mock {}
+6 -6
View File
@@ -1,16 +1,16 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_io::AsyncWrite;
use tokio_codec::{Encoder, FramedWrite};
use tokio_io::AsyncWrite;
use futures::{Sink, Poll};
use bytes::{BytesMut, BufMut};
use bytes::{BufMut, BytesMut};
use futures::{Poll, Sink};
use std::io::{self, Write};
use std::collections::VecDeque;
use std::io::{self, Write};
macro_rules! mock {
($($x:expr,)*) => {{
+18
View File
@@ -1,3 +1,21 @@
# 0.1.6 (March 22, 2019)
### Added
- implement `TypedExecutor` (#993).
# 0.1.5 (March 1, 2019)
### Fixed
- Documentation typos (#882).
# 0.1.4 (November 21, 2018)
* Fix shutdown on idle (#763).
# 0.1.3 (September 27, 2018)
* Fix minimal versions
# 0.1.2 (September 26, 2018)
* Implement `futures::Executor` for executor types (#563)
+6 -4
View File
@@ -3,11 +3,13 @@ name = "tokio-current-thread"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc URL
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.2"
documentation = "https://docs.rs/tokio-current-thread/0.1.2/tokio_current_thread"
version = "0.1.6"
documentation = "https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -19,5 +21,5 @@ keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
tokio-executor = { version = "0.1.2", path = "../tokio-executor" }
tokio-executor = "0.1.7"
futures = "0.1.19"
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
+1 -1
View File
@@ -2,7 +2,7 @@
Single threaded executor for Tokio.
[Documentation](https://tokio-rs.github.io/tokio/tokio_current_thread/)
[Documentation](https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread/)
## Overview
+109 -100
View File
@@ -1,4 +1,4 @@
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.2")]
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.6")]
#![deny(warnings, missing_docs, missing_debug_implementations)]
//! A single-threaded executor which executes tasks on the same thread from which
@@ -32,19 +32,19 @@ mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::park::{Park, ParkThread, Unpark};
use tokio_executor::{Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::future::{ExecuteError, ExecuteErrorKind, Executor};
use futures::{executor, Async, Future};
use futures::future::{Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::rc::Rc;
use std::sync::{atomic, mpsc, Arc};
use std::time::{Duration, Instant};
use std::thread;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
@@ -86,7 +86,7 @@ pub struct TaskExecutor {
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
polled: bool,
}
impl Turn {
@@ -194,17 +194,21 @@ struct CurrentRunner {
id: Cell<Option<u64>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
});
thread_local! {
/// Current thread's task runner. This is set in `TaskRunner::with`
static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
}
}
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one referred
/// to by a `Handle` so that direct task dispatch can be used.
thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
thread_local! {
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one
/// referred to by a `Handle` so that direct task dispatch can be used.
static EXECUTOR_ID: Cell<u64> = Cell::new(0)
}
/// Run the executor bootstrapping the execution with the provided future.
///
@@ -222,7 +226,8 @@ thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
where
F: Future,
{
let mut current_thread = CurrentThread::new();
@@ -246,7 +251,8 @@ where F: Future,
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
TaskExecutor::current()
.spawn_local(Box::new(future))
@@ -312,7 +318,8 @@ impl<P: Park> CurrentThread<P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future), false);
self
@@ -331,41 +338,33 @@ impl<P: Park> CurrentThread<P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).turn(duration)
}
@@ -432,11 +431,24 @@ impl tokio_executor::Executor for CurrentThread {
}
}
impl<T> tokio_executor::TypedExecutor<T> for CurrentThread
where
T: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
self.borrow().spawn_local(Box::new(future), false);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures.load(atomic::Ordering::SeqCst))
.field(
"num_futures",
&self.num_futures.load(atomic::Ordering::SeqCst),
)
.finish()
}
}
@@ -448,7 +460,8 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future), false);
self
@@ -467,17 +480,18 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
let res = self
.executor
.borrow()
.enter(self.enter, || future.poll_future_notify(&notify, 0));
match res {
Ok(Async::Ready(e)) => return Ok(e),
@@ -496,24 +510,19 @@ impl<'a, P: Park> Entered<'a, P> {
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
self.run_timeout2(None).map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
@@ -542,9 +551,7 @@ impl<'a, P: Park> Entered<'a, P> {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
fn run_timeout2(&mut self, dur: Option<Duration>) -> Result<(), RunTimeoutError> {
if self.executor.is_idle() {
// Nothing to do
return Ok(());
@@ -602,10 +609,9 @@ impl<'a, P: Park> Entered<'a, P> {
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
borrow.id,
&mut *self.enter,
borrow.num_futures)
borrow
.scheduler
.tick(borrow.id, &mut *self.enter, borrow.num_futures)
}
}
@@ -653,6 +659,13 @@ impl Handle {
where
F: Future<Item = (), Error = ()> + Send + 'static,
{
if thread::current().id() == self.thread {
let mut e = TaskExecutor::current();
if e.id() == Some(self.id) {
return e.spawn_local(Box::new(future));
}
}
if self.shut_down.get() {
return Err(SpawnError::shutdown());
}
@@ -669,14 +682,8 @@ impl Handle {
return Err(SpawnError::shutdown());
}
if thread::current().id() == self.thread {
let mut e = TaskExecutor::current();
if e.id() == Some(self.id) {
return e.spawn_local(Box::new(future));
}
}
self.sender.send(Box::new(future))
self.sender
.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
@@ -718,51 +725,53 @@ impl TaskExecutor {
/// Get the current executor's thread-local ID.
fn id(&self) -> Option<u64> {
CURRENT.with(|current| {
current.id.get()
})
CURRENT.with(|current| current.id.get())
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
pub fn spawn_local(
&mut self,
future: Box<Future<Item = (), Error = ()>>,
) -> Result<(), SpawnError> {
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => Err(SpawnError::shutdown()),
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
self.spawn_local(future)
}
}
impl<F> tokio_executor::TypedExecutor<F> for TaskExecutor
where
F: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: F) -> Result<(), SpawnError> {
self.spawn_local(Box::new(future))
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future)),
})
}
}
@@ -771,13 +780,12 @@ where F: Future<Item = (), Error = ()> + 'static
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
where
F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.id.set(Some(self.id));
current.set_spawn(self, || {
f()
})
current.set_spawn(self, || f())
})
}
}
@@ -797,7 +805,8 @@ impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
where
F: FnOnce() -> R,
{
struct Reset<'a>(&'a CurrentRunner);

Some files were not shown because too many files have changed in this diff Show More