Compare commits

...
Author SHA1 Message Date
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
385 changed files with 14847 additions and 5076 deletions
-22
View File
@@ -1,22 +0,0 @@
image: Visual Studio 2017
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- set LOOM_MAX_DURATION=10
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+41
View File
@@ -0,0 +1,41 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_DURATION: 10
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
cargo_cache:
folder: $HOME/.cargo/registry
test_script:
- . $HOME/.cargo/env
- cargo test --all --no-fail-fast
- cargo doc --all
i686_test_script:
- . $HOME/.cargo/env
- |
cargo test --all --exclude tokio-tls --no-fail-fast --target i686-unknown-freebsd
before_cache_script:
- rm -rf $HOME/.cargo/registry/index
-131
View File
@@ -1,131 +0,0 @@
---
language: rust
sudo: false
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
- rust: stable
- rust: beta
- rust: nightly
env: ALLOW_FAILURES=true
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
#
# Tests are not run as tests may require newer versions of rust.
- rust: 1.26.0
script: |
cargo check --all
# Test combinations of enabled features.
- rust: stable
script: |
shopt -s expand_aliases
alias check="cargo check --no-default-features"
check
check --features codec
check --features fs
check --features io
check --features reactor
check --features rt-full
check --features tcp
check --features timer
check --features udp
check --features uds
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
# This runs TSAN against nightly and allows failures to propagate up.
- rust: nightly-2018-11-18
env: TSAN=yes
script: |
set -e
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# This runs cargo +nightly doc
- name: nightly_docs
rust: nightly
env: ALLOW_FAILURES=true
script: cargo doc
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script: |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
# Limit the execution time of loom tests.
export LOOM_MAX_DURATION=10
cargo test --all --no-fail-fast
cargo doc --all
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = "linux" && $TARGET = ""
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+3 -124
View File
@@ -1,30 +1,7 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.15"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.15/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio",
"tokio-async-await",
"tokio-buf",
"tokio-codec",
@@ -39,106 +16,8 @@ members = [
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-trace",
"tokio-trace/tokio-trace-core",
"tokio-udp",
"tokio-uds",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"sync",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
]
sync = ["tokio-sync"]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.3", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-sync = { version = "0.1.0", path = "tokio-sync", optional = true }
tokio-threadpool = { version = "0.1.8", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
#tokio = { path = "." }
#tokio-async-await = { path = "./tokio-async-await" }
#tokio-codec = { path = "./tokio-codec" }
#tokio-current-thread = { path = "./tokio-current-thread" }
#tokio-executor = { path = "./tokio-executor" }
#tokio-fs = { path = "./tokio-fs" }
#tokio-io = { path = "./tokio-io" }
#tokio-reactor = { path = "./tokio-reactor" }
#tokio-signal = { path = "./tokio-signal" }
#tokio-tcp = { path = "./tokio-tcp" }
#tokio-threadpool = { path = "./tokio-threadpool" }
#tokio-timer = { path = "./tokio-timer" }
#tokio-tls = { path = "./tokio-tls" }
#tokio-udp = { path = "./tokio-udp" }
#tokio-uds = { path = "./tokio-uds" }
+8 -11
View File
@@ -14,29 +14,26 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio/0.1.15/tokio) |
[API Docs](https://docs.rs/tokio/0.1.18/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[master-dox]: https://tokio-rs.github.io/tokio/doc/tokio/
## Overview
@@ -52,9 +49,9 @@ level, it provides a few major components:
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/0.1.18/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.18/tokio/reactor/index.html
[scheduler]: https://docs.rs/tokio/0.1.18/tokio/runtime/index.html
## Example
+115
View File
@@ -0,0 +1,115 @@
trigger: ["master"]
pr: ["master"]
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
name: rustfmt
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
displayName: Test tokio
cross: true
crates:
- tokio
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates -
cross: true
crates:
- tokio-fs
- tokio-reactor
- tokio-signal
- tokio-tcp
- tokio-tls
- tokio-udp
- tokio-uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
crates:
- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
- tokio-io
- tokio-sync
- tokio-threadpool
- tokio-timer
- tokio-trace
- tokio-trace/tokio-trace-core
- template: ci/azure-cargo-check.yml
parameters:
name: features
displayName: Check feature permtuations
rust: stable
crates:
tokio:
- codec
- fs
- io
- reactor
- rt-full
- tcp
- timer
- udp
- uds
tokio-buf:
- util
# Check async / await
- template: ci/azure-cargo-check.yml
parameters:
name: async_await
displayName: Async / Await
rust: nightly-2019-02-28
noDefaultFeatures: ''
benches: true
crates:
tokio:
- async-await-preview
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross_32bit_linux
target: i686-unknown-linux-gnu
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust_version: 1.26.0
- template: ci/azure-tsan.yml
parameters:
name: tsan
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
- rustfmt
- test_tokio
- test_sub_cross
- test_linux
- features
- async_await
- cross_32bit_linux
- minrust
- tsan
+1 -3
View File
@@ -10,8 +10,8 @@ use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
@@ -57,7 +57,6 @@ fn udp_echo_latency(b: &mut Bencher) {
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
@@ -67,7 +66,6 @@ fn udp_echo_latency(b: &mut Bencher) {
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
+8 -9
View File
@@ -3,14 +3,13 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
extern crate test;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
use mio::{PollOpt, Ready, Token};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
@@ -22,8 +21,8 @@ fn mio_register_deregister(b: &mut Bencher) {
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
poll.deregister(&sock).unwrap();
});
}
@@ -36,12 +35,12 @@ fn mio_reregister(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.reregister(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
});
poll.deregister(&sock).unwrap();
}
+62 -49
View File
@@ -11,18 +11,18 @@ pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::reactor::Reactor;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::io::{self, Read, Write};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
pub use test::{self, Bencher};
}
mod connect_churn {
use ::prelude::*;
use prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
@@ -36,25 +36,29 @@ mod connect_churn {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
Ok(TcpStream::connect(&addr).and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
let connects_concurrent = connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(connects_concurrent)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
}
@@ -65,8 +69,7 @@ mod connect_churn {
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
@@ -75,47 +78,56 @@ mod connect_churn {
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(shutdown_rx)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
use std::sync::{Arc, Barrier};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
let threads: Vec<_> = (0..n)
.map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()))
.wait()
.unwrap();
})
})
}).collect();
.collect();
barrier.wait();
@@ -141,7 +153,7 @@ mod connect_churn {
}
mod transfer {
use ::prelude::*;
use prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
@@ -200,7 +212,8 @@ mod transfer {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
let server = listener
.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
@@ -210,17 +223,17 @@ mod transfer {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
drain
.map(|_| ())
.map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
.and_then(move |sock| Transfer {
sock: sock,
rem: MB,
chunk: write_size,
})
.map_err(|e| panic!("client err: {:?}", e));
@@ -229,7 +242,7 @@ mod transfer {
}
mod small_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
@@ -238,7 +251,7 @@ mod transfer {
}
mod big_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
+33
View File
@@ -0,0 +1,33 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- ${{ if parameters.benches }}:
- script: cargo check --benches --all
displayName: Check benchmarks
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+24
View File
@@ -0,0 +1,24 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
# Always patch
- template: azure-patch-crates.yml
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+38
View File
@@ -0,0 +1,38 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --all --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+27
View File
@@ -0,0 +1,27 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain %RUSTUP_TOOLCHAIN%
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%s' | grep -q '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+16
View File
@@ -0,0 +1,16 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
rustup component add rustfmt
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+41
View File
@@ -0,0 +1,41 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }} (PATCHED)
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-threadpool --tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: nightly-2018-11-18
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+22
View File
@@ -0,0 +1,22 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-async-await = { path = "tokio-async-await" }
tokio-buf = { path = "tokio-buf" }
tokio-codec = { path = "tokio-codec" }
tokio-current-thread = { path = "tokio-current-thread" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-io = { path = "tokio-io" }
tokio-reactor = { path = "tokio-reactor" }
tokio-signal = { path = "tokio-signal" }
tokio-sync = { path = "tokio-sync" }
tokio-threadpool = { path = "tokio-threadpool" }
tokio-timer = { path = "tokio-timer" }
tokio-tcp = { path = "tokio-tcp" }
tokio-tls = { path = "tokio-tls" }
tokio-trace = { path = "tokio-trace" }
tokio-trace-core = { path = "tokio-trace/tokio-trace-core" }
tokio-udp = { path = "tokio-udp" }
tokio-uds = { path = "tokio-uds" }
+27 -22
View File
@@ -26,21 +26,20 @@
#![deny(warnings)]
extern crate tokio;
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::runtime::current_thread::{Runtime, TaskExecutor};
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::rc::Rc;
use std::cell::RefCell;
use std::collections::HashMap;
use std::env;
use std::io::BufReader;
use std::iter;
use std::rc::Rc;
fn main() -> Result<(), Box<std::error::Error>> {
let mut runtime = Runtime::new().unwrap();
@@ -58,8 +57,12 @@ fn main() -> Result<(), Box<std::error::Error>> {
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
@@ -102,9 +105,7 @@ fn main() -> Result<(), Box<std::error::Error>> {
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
@@ -116,15 +117,17 @@ fn main() -> Result<(), Box<std::error::Error>> {
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
@@ -147,12 +150,14 @@ fn main() -> Result<(), Box<std::error::Error>> {
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn locally a task to process the connection
TaskExecutor::current().spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
}))).unwrap();
TaskExecutor::current()
.spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
})))
.unwrap();
Ok(())
})
+16 -12
View File
@@ -21,17 +21,17 @@
#![deny(warnings)]
extern crate tokio;
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::io::BufReader;
use std::iter;
use std::sync::{Arc, Mutex};
fn main() -> Result<(), Box<std::error::Error>> {
@@ -48,8 +48,12 @@ fn main() -> Result<(), Box<std::error::Error>> {
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| {println!("failed to accept socket; error = {:?}", e); e})
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
@@ -91,9 +95,7 @@ fn main() -> Result<(), Box<std::error::Error>> {
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
@@ -105,15 +107,17 @@ fn main() -> Result<(), Box<std::error::Error>> {
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
+28 -30
View File
@@ -31,12 +31,12 @@ extern crate tokio;
extern crate futures;
extern crate bytes;
use bytes::{BufMut, Bytes, BytesMut};
use futures::future::{self, Either};
use futures::sync::mpsc;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use std::collections::HashMap;
use std::net::SocketAddr;
@@ -131,10 +131,7 @@ impl Shared {
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
fn new(name: BytesMut, state: Arc<Mutex<Shared>>, lines: Lines) -> Peer {
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
@@ -142,8 +139,7 @@ impl Peer {
let (tx, rx) = mpsc::unbounded();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
state.lock().unwrap().peers.insert(addr, tx);
Peer {
name,
@@ -198,7 +194,7 @@ impl Future for Peer {
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
if i + 1 == LINES_PER_TICK {
task::current().notify();
}
}
@@ -256,8 +252,7 @@ impl Future for Peer {
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
self.state.lock().unwrap().peers.remove(&self.addr);
}
}
@@ -333,7 +328,10 @@ impl Stream for Lines {
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
let pos = self
.rd
.windows(2)
.enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
@@ -373,7 +371,8 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
let connection = lines
.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
@@ -408,10 +407,7 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
let peer = Peer::new(name, state, lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
@@ -443,18 +439,20 @@ pub fn main() -> Result<(), Box<std::error::Error>> {
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
let server = listener
.incoming()
.for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
+48 -44
View File
@@ -16,18 +16,18 @@
#![deny(warnings)]
extern crate bytes;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use std::env;
use std::io::{self, Read, Write};
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
use tokio::prelude::*;
fn main() -> Result<(), Box<std::error::Error>> {
// Determine if we're going to run in TCP or UDP mode
@@ -73,18 +73,16 @@ fn main() -> Result<(), Box<std::error::Error>> {
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.for_each(move |chunk| out.write_all(&chunk))
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
Ok(())
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio::codec::{Encoder, Decoder};
use std::io;
use tokio::codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -122,9 +120,9 @@ mod codec {
mod tcp {
use tokio;
use tokio::codec::Decoder;
use tokio::net::TcpStream;
use tokio::prelude::*;
use tokio::codec::Decoder;
use bytes::BytesMut;
use codec::Bytes;
@@ -133,10 +131,10 @@ mod tcp {
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
pub fn connect(
addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
@@ -154,18 +152,21 @@ mod tcp {
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
let stream = Box::new(tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
let stream = Box::new(
tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream());
stream
})
.flatten_stream(),
);
Ok(stream)
}
}
@@ -175,17 +176,17 @@ mod udp {
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use tokio;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>>
{
pub fn connect(
&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
@@ -206,14 +207,15 @@ mod udp {
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
let forward_stdin = stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
let forward_stdin = stdin
.map(move |chunk| (chunk, addr))
.forward(sink)
.then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
@@ -225,10 +227,13 @@ mod udp {
}
});
let stream = Box::new(future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
}).flatten_stream());
let stream = Box::new(
future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
})
.flatten_stream(),
);
Ok(stream)
}
}
@@ -240,8 +245,7 @@ fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Err(_) | Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
+2 -2
View File
@@ -16,11 +16,11 @@
extern crate futures;
extern crate tokio;
use std::{env, io};
use std::net::SocketAddr;
use std::{env, io};
use tokio::prelude::*;
use tokio::net::UdpSocket;
use tokio::prelude::*;
struct Server {
socket: UdpSocket,
+2 -2
View File
@@ -54,7 +54,8 @@ fn main() -> Result<(), Box<std::error::Error>> {
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
@@ -89,7 +90,6 @@ fn main() -> Result<(), Box<std::error::Error>> {
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
+14 -13
View File
@@ -25,20 +25,21 @@ pub fn main() -> Result<(), Box<std::error::Error>> {
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpStream, which is fully async.
let client = TcpStream::connect(&addr).and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
let client = TcpStream::connect(&addr)
.and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
})
})
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
// Start the Tokio runtime.
//
+2 -2
View File
@@ -57,10 +57,10 @@
extern crate tokio;
extern crate tokio_codec;
use tokio_codec::BytesCodec;
use tokio::codec::Decoder;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::codec::Decoder;
use tokio_codec::BytesCodec;
use std::env;
use std::net::SocketAddr;
+17 -16
View File
@@ -24,10 +24,10 @@
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use std::net::{Shutdown, SocketAddr};
use std::sync::{Arc, Mutex};
use tokio::io::{copy, shutdown};
use tokio::net::{TcpListener, TcpStream};
@@ -45,7 +45,8 @@ fn main() -> Result<(), Box<std::error::Error>> {
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
@@ -68,25 +69,25 @@ fn main() -> Result<(), Box<std::error::Error>> {
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
.and_then(|(n, _, server_writer)| shutdown(server_writer).map(move |_| n));
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
.and_then(|(n, _, client_writer)| shutdown(client_writer).map(move |_| n));
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
let msg = amounts
.map(move |(from_client, from_server)| {
println!(
"client wrote {} bytes and received {} bytes",
from_client, from_server
);
})
.map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
+44 -24
View File
@@ -44,8 +44,8 @@
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::io::BufReader;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
@@ -69,9 +69,18 @@ enum Request {
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
fn main() -> Result<(), Box<std::error::Error>> {
@@ -93,7 +102,8 @@ fn main() -> Result<(), Box<std::error::Error>> {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
let done = listener
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
@@ -124,15 +134,22 @@ fn main() -> Result<(), Box<std::error::Error>> {
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
}
}
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
Response::Set {
key,
value,
previous,
}
}
}
});
@@ -169,9 +186,11 @@ impl Request {
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
return Err(format!("GET's key must not be followed by anything"));
}
Ok(Request::Get { key: key.to_string() })
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
@@ -182,7 +201,10 @@ impl Request {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
@@ -193,15 +215,13 @@ impl Request {
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
+45 -33
View File
@@ -23,12 +23,12 @@ extern crate time;
extern crate tokio;
extern crate tokio_io;
use std::{env, fmt, io};
use std::net::SocketAddr;
use std::{env, fmt, io};
use tokio::net::{TcpStream, TcpListener};
use tokio::codec::{Decoder, Encoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use tokio::codec::{Encoder, Decoder};
use bytes::BytesMut;
use http::header::HeaderValue;
@@ -44,7 +44,8 @@ fn main() -> Result<(), Box<std::error::Error>> {
println!("Listening on: {}", addr);
tokio::run({
listener.incoming()
listener
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(|socket| {
process(socket);
@@ -64,14 +65,13 @@ fn process(socket: TcpStream) {
.split();
// Map all requests into responses and send them back to the client.
let task = tx.send_all(rx.and_then(respond))
.then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
let task = tx.send_all(rx.and_then(respond)).then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
Ok(())
});
Ok(())
});
// Spawn the task that handles the connection.
tokio::spawn(task);
@@ -82,9 +82,7 @@ fn process(socket: TcpStream) {
/// This function is a map from and HTTP request to a future of a response and
/// represents the various handling a server might do. Currently the contents
/// here are pretty uninteresting.
fn respond(req: Request<()>)
-> Box<Future<Item = Response<String>, Error = io::Error> + Send>
{
fn respond(req: Request<()>) -> Box<Future<Item = Response<String>, Error = io::Error> + Send> {
let f = future::lazy(move || {
let mut response = Response::builder();
let body = match req.uri().path() {
@@ -99,14 +97,18 @@ fn respond(req: Request<()>)
struct Message {
message: &'static str,
}
serde_json::to_string(&Message { message: "Hello, World!" })?
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response.body(body).map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
});
@@ -124,12 +126,19 @@ impl Encoder for Http {
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(BytesWrite(dst), "\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
", item.status(), item.body().len(), date::now()).unwrap();
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
@@ -198,13 +207,18 @@ impl Decoder for Http {
headers[i] = Some((k, v));
}
(toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt)
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(io::ErrorKind::Other, "only HTTP/1.1 accepted"))
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
@@ -216,15 +230,13 @@ impl Decoder for Http {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = unsafe {
HeaderValue::from_shared_unchecked(data.slice(v.0, v.1))
};
let value = unsafe { HeaderValue::from_shared_unchecked(data.slice(v.0, v.1)) };
ret.header(&data[k.0..k.1], value);
}
let req = ret.body(()).map_err(|e| {
io::Error::new(io::ErrorKind::Other, e)
})?;
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
+2 -1
View File
@@ -51,7 +51,8 @@ fn main() -> Result<(), Box<std::error::Error>> {
"0.0.0.0:0"
} else {
"[::]:0"
}.parse()?;
}
.parse()?;
let socket = UdpSocket::bind(&local_addr)?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
socket
+2 -2
View File
@@ -8,15 +8,15 @@
#![deny(warnings)]
extern crate env_logger;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate env_logger;
use std::net::SocketAddr;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_codec::BytesCodec;
fn main() -> Result<(), Box<std::error::Error>> {
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
+4 -4
View File
@@ -3,12 +3,12 @@ name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.5"
version = "0.1.6"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.3"
documentation = "https://docs.rs/tokio-async-await/0.1.6"
description = """
Experimental async/await support for Tokio
"""
@@ -21,9 +21,9 @@ async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
tokio = "0.1.8"
hyper = "0.12.8"
+2 -2
View File
@@ -9,7 +9,7 @@ guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.33.0-nightly or later.
1.34.0-nightly or later.
Add this to your `Cargo.toml`:
@@ -18,7 +18,7 @@ Add this to your `Cargo.toml`:
edition = "2018"
# In the `[dependencies]` section
tokio = {version = "0.1.0", features = ["async-await-preview"]}
tokio = {version = "0.1.15", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
+1 -1
View File
@@ -61,7 +61,7 @@ async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()>
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
await!(lines_tx.send_async(line)).unwrap();
}
});
@@ -1,4 +1,4 @@
#![feature(await_macro, async_await)]
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
+4 -4
View File
@@ -2,15 +2,15 @@
#[macro_export]
macro_rules! await {
($e:expr) => {{
use $crate::std_await;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
#[allow(unused_imports)]
use $crate::compat::backward::IntoAwaitable as IntoAwaitableBackward;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
use $crate::std_await;
#[allow(unused_mut)]
let mut e = $e;
let e = e.into_awaitable();
std_await!(e)
}}
}};
}
+26 -32
View File
@@ -1,16 +1,9 @@
use futures::{Future, Poll};
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::future::{
Future as StdFuture,
};
use std::ptr::NonNull;
use std::task::{
LocalWaker,
Poll as StdPoll,
UnsafeWake,
Waker,
};
use std::ptr;
use std::task::{Poll as StdPoll, RawWaker, RawWakerVTable, Waker};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
@@ -31,7 +24,8 @@ pub trait IntoAwaitable {
}
impl<T> IntoAwaitable for T
where T: StdFuture,
where
T: StdFuture,
{
type Awaitable = Self;
@@ -41,7 +35,8 @@ where T: StdFuture,
}
impl<T, Item, Error> Future for Compat<T>
where T: StdFuture<Output = Result<Item, Error>>,
where
T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
@@ -49,9 +44,9 @@ where T: StdFuture<Output = Result<Item, Error>>,
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let local_waker = noop_local_waker();
let waker = noop_waker();
let res = self.0.as_mut().poll(&local_waker);
let res = self.0.as_mut().poll(&waker);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
@@ -63,27 +58,26 @@ where T: StdFuture<Output = Result<Item, Error>>,
// ===== NoopWaker =====
struct NoopWaker;
fn noop_local_waker() -> LocalWaker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { LocalWaker::new(w) }
fn noop_raw_waker() -> RawWaker {
RawWaker::new(ptr::null(), &NOOP_WAKER_VTABLE)
}
fn noop_waker() -> Waker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { Waker::new(w) }
unsafe { Waker::new_unchecked(noop_raw_waker()) }
}
unsafe impl UnsafeWake for NoopWaker {
unsafe fn clone_raw(&self) -> Waker {
noop_waker()
}
unsafe fn drop_raw(&self) {
}
unsafe fn wake(&self) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
unsafe fn clone_raw(_data: *const ()) -> RawWaker {
noop_raw_waker()
}
unsafe fn drop_raw(_data: *const ()) {}
unsafe fn wake(_data: *const ()) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
const NOOP_WAKER_VTABLE: RawWakerVTable = RawWakerVTable {
clone: clone_raw,
drop: drop_raw,
wake,
};
+10 -11
View File
@@ -1,17 +1,15 @@
use futures::{Async, Future};
use futures::{Future, Async};
use std::marker::Unpin;
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{LocalWaker, Poll as StdPoll};
use std::task::{Poll as StdPoll, Waker};
/// Converts an 0.1 `Future` into an 0.3 `Future`.
#[derive(Debug)]
pub struct Compat<T>(T);
pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T, E>> {
use futures::Async::{Ready, NotReady};
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
@@ -21,9 +19,9 @@ pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T,
}
pub(crate) fn convert_poll_stream<T, E>(
poll: Result<Async<Option<T>>, E>) -> StdPoll<Option<Result<T, E>>>
{
use futures::Async::{Ready, NotReady};
poll: Result<Async<Option<T>>, E>,
) -> StdPoll<Option<Result<T, E>>> {
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(Some(val))) => StdPoll::Ready(Some(Ok(val))),
@@ -50,12 +48,13 @@ impl<T: Future + Unpin> IntoAwaitable for T {
}
impl<T> StdFuture for Compat<T>
where T: Future + Unpin
where
T: Future + Unpin,
{
type Output = Result<T::Item, T::Error>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> StdPoll<Self::Output> {
use futures::Async::{Ready, NotReady};
fn poll(mut self: Pin<&mut Self>, _waker: &Waker) -> StdPoll<Self::Output> {
use futures::Async::{NotReady, Ready};
// TODO: wire in cx
+1 -1
View File
@@ -1,4 +1,4 @@
#![doc(hidden)]
pub mod forward;
pub mod backward;
pub mod forward;
+3 -5
View File
@@ -1,11 +1,9 @@
use tokio_io::AsyncWrite;
use std::io;
use std::future::Future;
use std::marker::Unpin;
use std::io;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Poll, Waker};
/// A future used to fully flush an I/O object.
#[derive(Debug)]
@@ -25,7 +23,7 @@ impl<'a, T: AsyncWrite + ?Sized> Flush<'a, T> {
impl<'a, T: AsyncWrite + ?Sized> Future for Flush<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _wx: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _wx: &Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
convert_poll(self.writer.poll_flush())
}
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future which can be used to read bytes.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Read<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> Read<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> Read<'a, T> {
Read {
reader,
buf,
}
Read { reader, buf }
}
}
impl<'a, T: AsyncRead + ?Sized> Future for Read<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for ReadExact<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> ReadExact<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> ReadExact<'a, T> {
ReadExact {
reader,
buf,
}
ReadExact { reader, buf }
}
}
@@ -34,7 +30,7 @@ fn eof() -> io::Error {
impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -47,7 +43,7 @@ impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
this.buf = rest;
}
if n == 0 {
return Poll::Ready(Err(eof()))
return Poll::Ready(Err(eof()));
}
}
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future used to write data.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Write<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Write<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> Write<'a, T> {
Write {
writer,
buf,
}
Write { writer, buf }
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Write<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<usize>> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<io::Result<usize>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for WriteAll<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> WriteAll<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> WriteAll<'a, T> {
WriteAll {
writer,
buf,
}
WriteAll { writer, buf }
}
}
@@ -34,7 +30,7 @@ fn zero_write() -> io::Error {
impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<()>> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<io::Result<()>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -48,7 +44,7 @@ impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
}
if n == 0 {
return Poll::Ready(Err(zero_write()))
return Poll::Ready(Err(zero_write()));
}
}
+5 -84
View File
@@ -1,13 +1,6 @@
#![cfg(feature = "async-await-preview")]
#![feature(
rust_2018_preview,
arbitrary_self_types,
async_await,
await_macro,
futures_api,
)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.5")]
#![feature(rust_2018_preview, async_await, await_macro, futures_api)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.6")]
#![deny(missing_docs, missing_debug_implementations)]
#![cfg_attr(test, deny(warnings))]
@@ -23,12 +16,10 @@ macro_rules! try_ready {
($x:expr) => {
match $x {
std::task::Poll::Ready(Ok(x)) => x,
std::task::Poll::Ready(Err(e)) =>
return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending =>
return std::task::Poll::Pending,
std::task::Poll::Ready(Err(e)) => return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending => return std::task::Poll::Pending,
}
}
};
}
#[macro_use]
@@ -38,77 +29,7 @@ pub mod io;
pub mod sink;
pub mod stream;
/*
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_main::prelude::*;
#[doc(inline)]
pub use crate::async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt,
},
};
}
*/
// Rename the `await` macro in `std`. This is used by the redefined
// `await` macro in this crate.
#[doc(hidden)]
pub use std::await as std_await;
/*
use std::future::{Future as StdFuture};
fn run<T: futures::Future<Item = (), Error = ()>>(t: T) {
drop(t);
}
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
run(future);
unimplemented!();
}
*/
/*
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use crate::async_await::compat::backward;
spawn(backward::Compat::new(async || {
let _ = await!(future);
Ok(())
}));
}
*/
-2
View File
@@ -6,8 +6,6 @@ pub use self::send::Send;
use futures::Sink;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Sink` types.
pub trait SinkExt: Sink {
/// Send an item into the sink.
+2 -3
View File
@@ -3,7 +3,6 @@ use futures::Sink;
use std::future::Future;
use std::task::{self, Poll};
use std::marker::Unpin;
use std::pin::Pin;
/// Future for the `SinkExt::send_async` combinator, which sends a value to a
@@ -28,9 +27,9 @@ impl<'a, T: Sink + Unpin + ?Sized> Send<'a, T> {
impl<T: Sink + Unpin + ?Sized> Future for Send<'_, T> {
type Output = Result<(), T::SinkError>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
use futures::AsyncSink::{Ready, NotReady};
use futures::AsyncSink::{NotReady, Ready};
if let Some(item) = self.item.take() {
match self.sink.start_send(item) {
-2
View File
@@ -6,8 +6,6 @@ pub use self::next::Next;
use futures::Stream;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Stream` types.
pub trait StreamExt: Stream {
/// Creates a future that resolves to the next item in the stream.
+2 -3
View File
@@ -1,9 +1,8 @@
use futures::Stream;
use std::future::Future;
use std::marker::Unpin;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Poll, Waker};
/// A future of the next element of a stream.
#[derive(Debug)]
@@ -22,7 +21,7 @@ impl<'a, T: Stream + Unpin> Next<'a, T> {
impl<'a, T: Stream + Unpin> Future for Next<'a, T> {
type Output = Option<Result<T::Item, T::Error>>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _waker: &Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll_stream;
convert_poll_stream(self.stream.poll())
+1 -1
View File
@@ -1,3 +1,3 @@
# 0.1.0 (unreleased)
# 0.1.0 (February 23, 2019)
* Initial release
+10 -3
View File
@@ -3,6 +3,9 @@ name = "tokio-buf"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
@@ -10,13 +13,17 @@ authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-buf/0.1.0"
documentation = "https://docs.rs/tokio-buf/0.1.0/tokio_buf"
description = """
Asynchronous stream of byte buffers
"""
categories = ["asynchronous"]
[dependencies]
bytes = { version = "0.4.10", features = [ "either" ] }
either = "1.5"
bytes = "0.4.10"
either = { version = "1.5", optional = true}
futures = "0.1.23"
[features]
default = ["util"]
util = ["bytes/either", "either"]
+35
View File
@@ -0,0 +1,35 @@
# tokio-buf
Asynchronous stream of byte buffers
[Documenation](https://docs.rs/tokio-buf)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-buf = "0.1.0"
```
Next, add this to your crate:
```rust
extern crate tokio_buf;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-buf) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-32
View File
@@ -1,32 +0,0 @@
//! Error types
pub use super::collect::CollectError;
pub use super::from::CollectVecError;
pub use super::limit::LimitError;
// Being crate-private, we should be able to swap the type out in a
// backwards compatible way.
pub(crate) mod internal {
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
}
-163
View File
@@ -1,163 +0,0 @@
//! Types and utilities for working with `BufStream`.
mod bytes;
mod chain;
mod collect;
pub mod errors;
mod from;
mod limit;
mod size_hint;
mod str;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::limit::Limit;
pub use self::size_hint::SizeHint;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
/// Indicates to the `BufStream` how much data the consumer is currently
/// able to process.
///
/// The consume hint allows the stream to perform certain optimizations that
/// are dependent on the consumer's readiness. For example, the consume hint
/// may be used to request a remote peer to start sending up to `amount`
/// data.
///
/// Calling `consume_hint` is not a requirement. If `consume_hint` is never
/// called, the stream should assume a default behavior. When `consume_hint`
/// is called, the stream should make a best effort to honor by the request.
///
/// `amount` represents the number of bytes that the caller would like to
/// receive at the time the function is called. For example, if
/// `consume_hint` is called with 20, the consumer requests 20 bytes. The
/// stream may yield less than that. If the next call to `poll_buf` returns
/// 5 bytes, the consumer still has 15 bytes requested. At this point,
/// invoking `consume_hint` again with 20 resets the amount requested back
/// to 20 bytes.
///
/// Calling `consume_hint` with 0 as the argument informs the stream that
/// the caller does not intend to call `poll_buf`. If `poll_buf` **is**
/// called, the stream may, but is not obligated to, return `NotReady` even
/// if it could produce data at that point. If it chooses to return
/// `NotReady`, when `consume_hint` is called with a non-zero argument, the
/// task must be notified in order to respect the `poll_buf` contract.
fn consume_hint(&mut self, amount: usize) {
// By default, this function does nothing
drop(amount);
}
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
}
+82 -3
View File
@@ -1,5 +1,5 @@
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.0")]
#![deny(missing_docs, missing_debug_implementations)]
#![deny(missing_docs, missing_debug_implementations, unreachable_pub)]
#![cfg_attr(test, deny(warnings))]
//! Asynchronous stream of bytes.
@@ -10,11 +10,90 @@
//! `Buf` (i.e, byte collections).
extern crate bytes;
#[cfg(feature = "util")]
extern crate either;
#[allow(unused)]
#[macro_use]
extern crate futures;
pub mod buf_stream;
mod never;
mod size_hint;
mod str;
mod u8;
#[cfg(feature = "util")]
pub mod util;
pub use self::size_hint::SizeHint;
#[doc(inline)]
pub use buf_stream::BufStream;
#[cfg(feature = "util")]
pub use util::BufStreamExt;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
}
+22
View File
@@ -0,0 +1,22 @@
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
@@ -1,5 +1,5 @@
use never::Never;
use BufStream;
use buf_stream::errors::internal::Never;
use futures::Poll;
@@ -1,10 +1,8 @@
use BufStream;
use buf_stream::errors::internal::Never;
use bytes::{Bytes, BytesMut};
use futures::Poll;
use never::Never;
use std::io;
use BufStream;
impl BufStream for Vec<u8> {
type Item = io::Cursor<Vec<u8>>;
@@ -58,9 +56,7 @@ impl BufStream for BytesMut {
}
}
fn poll_bytes<T: Default>(buf: &mut T)
-> Poll<Option<io::Cursor<T>>, Never>
{
fn poll_bytes<T: Default>(buf: &mut T) -> Poll<Option<io::Cursor<T>>, Never> {
use std::mem;
let bytes = mem::replace(buf, Default::default());
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use either::Either;
use futures::Poll;
@@ -43,9 +43,4 @@ where
let res = try_ready!(self.right.poll_buf());
Ok(res.map(Either::Right).into())
}
fn size_hint(&self) -> SizeHint {
// TODO: Implement
SizeHint::default()
}
}
@@ -1,4 +1,5 @@
use super::{BufStream, FromBufStream};
use super::FromBufStream;
use BufStream;
use futures::{Future, Poll};
@@ -52,29 +53,26 @@ where
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
loop {
let res = self.stream.poll_buf()
.map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
let res = self.stream.poll_buf().map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
match try_ready!(res) {
Some(mut buf) => {
let builder = self.builder.as_mut().expect("cannot poll after done");
U::extend(builder, &mut buf, &self.stream.size_hint())
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
U::extend(builder, &mut buf, &self.stream.size_hint()).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
}
None => {
let builder = self.builder.take().expect("cannot poll after done");
let value = U::build(builder)
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
let value = U::build(builder).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
return Ok(value.into());
}
}
@@ -1,4 +1,4 @@
use super::SizeHint;
use SizeHint;
use bytes::{Buf, BufMut};
@@ -43,7 +43,9 @@ pub trait FromBufStream<T: Buf>: Sized {
/// Error returned from collecting into a `Vec<u8>`
#[derive(Debug)]
pub struct CollectVecError { _p: () }
pub struct CollectVecError {
_p: (),
}
impl<T: Buf> FromBufStream<T> for Vec<u8> {
type Builder = Vec<u8>;
@@ -70,7 +72,7 @@ impl<T: Buf> FromBufStream<T> for Vec<u8> {
Some(upper) if upper <= 64 => {
reserve = upper as usize;
}
_ => {},
_ => {}
}
// hint.lower() represents the minimum amount of data that will be
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use bytes::Buf;
use futures::Poll;
@@ -40,10 +40,10 @@ where
return Err(LimitError { inner: None });
}
let res = self.stream.poll_buf()
.map_err(|err| {
LimitError { inner: Some(err) }
});
let res = self
.stream
.poll_buf()
.map_err(|err| LimitError { inner: Some(err) });
match res {
Ok(Ready(Some(ref buf))) => {
@@ -59,22 +59,6 @@ where
res
}
fn size_hint(&self) -> SizeHint {
let mut hint = self.stream.size_hint();
let upper = hint.upper()
.map(|upper| upper.min(self.remaining))
.unwrap_or(self.remaining);
hint.set_upper(upper);
hint
}
fn consume_hint(&mut self, amount: usize) {
// TODO: Should this be capped by `self.remaining`?
self.stream.consume_hint(amount)
}
}
// ===== impl LimitError =====
+73
View File
@@ -0,0 +1,73 @@
//! Types and utilities for working with `BufStream`.
mod chain;
mod collect;
mod from;
mod limit;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::limit::Limit;
pub mod error {
//! Error types
pub use super::collect::CollectError;
pub use super::from::CollectVecError;
pub use super::limit::LimitError;
}
use BufStream;
impl<T> BufStreamExt for T where T: BufStream {}
/// An extension trait for `BufStream`'s that provides a variety of convenient
/// adapters.
pub trait BufStreamExt: BufStream {
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
}
+4 -267
View File
@@ -1,46 +1,13 @@
extern crate tokio_buf;
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use tokio_buf::buf_stream::{BufStream, SizeHint};
use bytes::Buf;
use futures::{Future, Poll};
use futures::Async::*;
use tokio_buf::{BufStream, SizeHint};
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
}
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
}
}
#[macro_use]
mod support;
// ===== test `SizeHint` =====
@@ -83,149 +50,6 @@ fn size_hint_upper_less_than_lower() {
hint.set_upper(100);
}
// ===== test `chain()` =====
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"])
.chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[
Ok(Ready("foo")),
Ok(NotReady),
Ok(Ready("bar"))
]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
// ===== Test `collect()` =====
#[test]
fn collect_vec() {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world, one two three");
}
// ===== Test limit() =====
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world")
.limit(5)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let res = one("hello world")
.limit(10)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"])
.limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
// ===== BufStream impelmentations for misc types =====
#[test]
@@ -240,90 +64,3 @@ fn str_buf_stream() {
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
}
// ===== Test utils =====
fn one(buf: &'static str) -> Mock {
list(&[buf])
}
fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
struct Mock {
polls: VecDeque<Poll<&'static [u8], ()>>,
size_hint: SizeHint,
}
#[derive(Debug)]
struct MockBuf {
data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
+145
View File
@@ -0,0 +1,145 @@
#![cfg(feature = "ext")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Future;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
// ===== test `chain()` =====
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"]).chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[Ok(Ready("foo")), Ok(NotReady), Ok(Ready("bar"))]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
// ===== Test `collect()` =====
#[test]
fn collect_vec() {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: Vec<u8> = bs.collect().wait().unwrap();
assert_eq!(vec, b"hello world, one two three");
}
// ===== Test limit() =====
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world").limit(5).collect::<Vec<_>>().wait();
assert!(res.is_err());
let res = one("hello world").limit(10).collect::<Vec<_>>().wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"]).limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
+132
View File
@@ -0,0 +1,132 @@
#![allow(unused)]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Poll;
use tokio_buf::{BufStream, SizeHint};
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
};
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
};
}
// ===== Test utils =====
pub fn one(buf: &'static str) -> Mock {
list(&[buf])
}
pub fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
pub fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
pub struct Mock {
pub polls: VecDeque<Poll<&'static [u8], ()>>,
pub size_hint: SizeHint,
}
#[derive(Debug)]
pub struct MockBuf {
pub data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
+1 -1
View File
@@ -18,6 +18,6 @@ Utilities for encoding and decoding frames.
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
bytes = "0.4.7"
futures = "0.1.18"
+5 -3
View File
@@ -1,6 +1,6 @@
use bytes::{Bytes, BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use bytes::{BufMut, Bytes, BytesMut};
use std::io;
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -8,7 +8,9 @@ pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec { BytesCodec(()) }
pub fn new() -> BytesCodec {
BytesCodec(())
}
}
impl Decoder for BytesCodec {
+1 -8
View File
@@ -19,14 +19,7 @@ extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use tokio_io::_tokio_codec::{Decoder, Encoder, Framed, FramedParts, FramedRead, FramedWrite};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
+4 -6
View File
@@ -1,6 +1,6 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{cmp, io, str, usize};
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -103,10 +103,8 @@ impl LinesCodec {
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
str::from_utf8(buf)
.map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
@@ -153,7 +151,7 @@ impl Decoder for LinesCodec {
self.is_discarding = true;
Err(io::Error::new(
io::ErrorKind::Other,
"line length limit exceeded"
"line length limit exceeded",
))
} else {
// We didn't find a line or reach the length limit, so the next
+33 -9
View File
@@ -1,8 +1,8 @@
extern crate tokio_codec;
extern crate bytes;
extern crate tokio_codec;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
use bytes::{BufMut, Bytes, BytesMut};
use tokio_codec::{BytesCodec, Decoder, Encoder, LinesCodec};
#[test]
fn bytes_decoder() {
@@ -27,13 +27,17 @@ fn bytes_encoder() {
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf)
.unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf)
.unwrap();
}
#[test]
@@ -68,17 +72,32 @@ fn lines_decoder_max_length() {
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 2", line);
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 4", line);
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("", line);
assert_eq!(None, codec.decode(buf).unwrap());
@@ -87,7 +106,12 @@ fn lines_decoder_max_length() {
assert_eq!(None, codec.decode(buf).unwrap());
let line = codec.decode_eof(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("\rk", line);
assert_eq!(None, codec.decode(buf).unwrap());
+10 -10
View File
@@ -1,13 +1,13 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use futures::{Stream, Future};
use bytes::{Buf, BufMut, BytesMut, IntoBuf};
use futures::{Future, Stream};
use std::io::{self, Read};
use tokio_codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_codec::{Decoder, Encoder, Framed, FramedParts};
use tokio_io::AsyncRead;
use bytes::{BytesMut, Buf, BufMut, IntoBuf};
const INITIAL_CAPACITY: usize = 8 * 1024;
@@ -45,8 +45,10 @@ struct DontReadIntoThis;
impl Read for DontReadIntoThis {
fn read(&mut self, _: &mut [u8]) -> io::Result<usize> {
Err(io::Error::new(io::ErrorKind::Other,
"Read into something you weren't supposed to."))
Err(io::Error::new(
io::ErrorKind::Other,
"Read into something you weren't supposed to.",
))
}
}
@@ -61,9 +63,7 @@ fn can_read_from_existing_buf() {
let num = framed
.into_future()
.map(|(first_num, _)| {
first_num.unwrap()
})
.map(|(first_num, _)| first_num.unwrap())
.wait()
.map_err(|e| e.0)
.unwrap();
+7 -8
View File
@@ -1,17 +1,17 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, FramedRead};
use tokio_io::AsyncRead;
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf};
use bytes::{Buf, BytesMut, IntoBuf};
use futures::Async::{NotReady, Ready};
use futures::Stream;
use futures::Async::{Ready, NotReady};
use std::io::{self, Read};
use std::collections::VecDeque;
use std::io::{self, Read};
macro_rules! mock {
($($x:expr,)*) => {{
@@ -212,5 +212,4 @@ impl Read for Mock {
}
}
impl AsyncRead for Mock {
}
impl AsyncRead for Mock {}
+6 -6
View File
@@ -1,16 +1,16 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_io::AsyncWrite;
use tokio_codec::{Encoder, FramedWrite};
use tokio_io::AsyncWrite;
use futures::{Sink, Poll};
use bytes::{BytesMut, BufMut};
use bytes::{BufMut, BytesMut};
use futures::{Poll, Sink};
use std::io::{self, Write};
use std::collections::VecDeque;
use std::io::{self, Write};
macro_rules! mock {
($($x:expr,)*) => {{
+10
View File
@@ -1,3 +1,13 @@
# 0.1.6 (March 22, 2019)
### Added
- implement `TypedExecutor` (#993).
# 0.1.5 (March 1, 2019)
### Fixed
- Documentation typos (#882).
# 0.1.4 (November 21, 2018)
* Fix shutdown on idle (#763).
+6 -4
View File
@@ -3,11 +3,13 @@ name = "tokio-current-thread"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc URL
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.4"
documentation = "https://docs.rs/tokio-current-thread/0.1.4/tokio_current_thread"
version = "0.1.6"
documentation = "https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -19,5 +21,5 @@ keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
tokio-executor = { version = "0.1.5", path = "../tokio-executor" }
tokio-executor = "0.1.7"
futures = "0.1.19"
+1 -1
View File
@@ -2,7 +2,7 @@
Single threaded executor for Tokio.
[Documentation](https://tokio-rs.github.io/tokio/tokio_current_thread/)
[Documentation](https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread/)
## Overview
+102 -93
View File
@@ -1,4 +1,4 @@
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.4")]
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.6")]
#![deny(warnings, missing_docs, missing_debug_implementations)]
//! A single-threaded executor which executes tasks on the same thread from which
@@ -32,19 +32,19 @@ mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::park::{Park, ParkThread, Unpark};
use tokio_executor::{Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::future::{ExecuteError, ExecuteErrorKind, Executor};
use futures::{executor, Async, Future};
use futures::future::{Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::rc::Rc;
use std::sync::{atomic, mpsc, Arc};
use std::time::{Duration, Instant};
use std::thread;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
@@ -86,7 +86,7 @@ pub struct TaskExecutor {
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
polled: bool,
}
impl Turn {
@@ -194,17 +194,21 @@ struct CurrentRunner {
id: Cell<Option<u64>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
});
thread_local! {
/// Current thread's task runner. This is set in `TaskRunner::with`
static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
}
}
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one referred
/// to by a `Handle` so that direct task dispatch can be used.
thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
thread_local! {
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one
/// referred to by a `Handle` so that direct task dispatch can be used.
static EXECUTOR_ID: Cell<u64> = Cell::new(0)
}
/// Run the executor bootstrapping the execution with the provided future.
///
@@ -222,7 +226,8 @@ thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
where
F: Future,
{
let mut current_thread = CurrentThread::new();
@@ -246,7 +251,8 @@ where F: Future,
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
TaskExecutor::current()
.spawn_local(Box::new(future))
@@ -312,7 +318,8 @@ impl<P: Park> CurrentThread<P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future), false);
self
@@ -331,41 +338,33 @@ impl<P: Park> CurrentThread<P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).turn(duration)
}
@@ -432,11 +431,24 @@ impl tokio_executor::Executor for CurrentThread {
}
}
impl<T> tokio_executor::TypedExecutor<T> for CurrentThread
where
T: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
self.borrow().spawn_local(Box::new(future), false);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures.load(atomic::Ordering::SeqCst))
.field(
"num_futures",
&self.num_futures.load(atomic::Ordering::SeqCst),
)
.finish()
}
}
@@ -448,7 +460,8 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future), false);
self
@@ -467,17 +480,18 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
let res = self
.executor
.borrow()
.enter(self.enter, || future.poll_future_notify(&notify, 0));
match res {
Ok(Async::Ready(e)) => return Ok(e),
@@ -496,24 +510,19 @@ impl<'a, P: Park> Entered<'a, P> {
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
self.run_timeout2(None).map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
@@ -542,9 +551,7 @@ impl<'a, P: Park> Entered<'a, P> {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
fn run_timeout2(&mut self, dur: Option<Duration>) -> Result<(), RunTimeoutError> {
if self.executor.is_idle() {
// Nothing to do
return Ok(());
@@ -602,10 +609,9 @@ impl<'a, P: Park> Entered<'a, P> {
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
borrow.id,
&mut *self.enter,
borrow.num_futures)
borrow
.scheduler
.tick(borrow.id, &mut *self.enter, borrow.num_futures)
}
}
@@ -676,7 +682,8 @@ impl Handle {
return Err(SpawnError::shutdown());
}
self.sender.send(Box::new(future))
self.sender
.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
@@ -718,51 +725,53 @@ impl TaskExecutor {
/// Get the current executor's thread-local ID.
fn id(&self) -> Option<u64> {
CURRENT.with(|current| {
current.id.get()
})
CURRENT.with(|current| current.id.get())
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
pub fn spawn_local(
&mut self,
future: Box<Future<Item = (), Error = ()>>,
) -> Result<(), SpawnError> {
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => Err(SpawnError::shutdown()),
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
self.spawn_local(future)
}
}
impl<F> tokio_executor::TypedExecutor<F> for TaskExecutor
where
F: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: F) -> Result<(), SpawnError> {
self.spawn_local(Box::new(future))
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future)),
})
}
}
@@ -771,13 +780,12 @@ where F: Future<Item = (), Error = ()> + 'static
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
where
F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.id.set(Some(self.id));
current.set_spawn(self, || {
f()
})
current.set_spawn(self, || f())
})
}
}
@@ -797,7 +805,8 @@ impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
where
F: FnOnce() -> R,
{
struct Reset<'a>(&'a CurrentRunner);
+17 -21
View File
@@ -1,20 +1,20 @@
use super::Borrow;
use tokio_executor::Enter;
use tokio_executor::park::Unpark;
use tokio_executor::Enter;
use futures::{Future, Async};
use futures::executor::{self, Spawn, UnsafeNotify, NotifyHandle};
use futures::executor::{self, NotifyHandle, Spawn, UnsafeNotify};
use futures::{Async, Future};
use std::cell::UnsafeCell;
use std::fmt::{self, Debug};
use std::marker::PhantomData;
use std::mem;
use std::ptr;
use std::sync::atomic::Ordering::{Relaxed, SeqCst, Acquire, Release, AcqRel};
use std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release, SeqCst};
use std::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize};
use std::sync::{Arc, Weak};
use std::usize;
use std::thread;
use std::marker::PhantomData;
use std::usize;
/// A generic task-aware scheduler.
///
@@ -135,7 +135,8 @@ pub struct Scheduled<'a, U: 'a> {
}
impl<U> Scheduler<U>
where U: Unpark,
where
U: Unpark,
{
/// Constructs a new, empty `Scheduler`
///
@@ -200,9 +201,7 @@ where U: Unpark,
pub fn has_pending_futures(&mut self) -> bool {
// See function definition for why the unsafe is needed and
// correctly used here
unsafe {
self.inner.has_pending_futures()
}
unsafe { self.inner.has_pending_futures() }
}
/// Advance the scheduler state, returning `true` if any futures were
@@ -210,11 +209,9 @@ where U: Unpark,
///
/// This function should be called whenever the caller is notified via a
/// wakeup.
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool
{
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool {
let mut ret = false;
let tick = self.inner.tick_num.fetch_add(1, SeqCst)
.wrapping_add(1);
let tick = self.inner.tick_num.fetch_add(1, SeqCst).wrapping_add(1);
loop {
let node = match unsafe { self.inner.dequeue(Some(tick)) } {
@@ -246,7 +243,7 @@ where U: Unpark,
let node = ptr2arc(node);
assert!((*node.next_all.get()).is_null());
assert!((*node.prev_all.get()).is_null());
continue
continue;
};
// We're going to need to be very careful if the `poll`
@@ -369,8 +366,7 @@ impl Task {
impl fmt::Debug for Task {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Task")
.finish()
fmt.debug_struct("Task").finish()
}
}
@@ -559,7 +555,7 @@ impl<U> List<U> {
}
}
/// Prepends an element to the back of the list
/// Appends an element to the back of the list
fn push_back(&mut self, node: Arc<Node<U>>) -> *const Node<U> {
let ptr = arc2ptr(node);
@@ -580,7 +576,7 @@ impl<U> List<U> {
self.len += 1;
return ptr
return ptr;
}
/// Pop an element from the front of the list
@@ -632,7 +628,7 @@ impl<U> List<U> {
self.len -= 1;
return node
return node;
}
}
@@ -749,7 +745,7 @@ impl<U> Drop for Node<U> {
fn arc2ptr<T>(ptr: Arc<T>) -> *const T {
let addr = &*ptr as *const T;
mem::forget(ptr);
return addr
return addr;
}
unsafe fn ptr2arc<T>(ptr: *const T) -> Arc<T> {
+155 -120
View File
@@ -1,6 +1,6 @@
extern crate futures;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate futures;
use tokio_current_thread::{block_on_all, CurrentThread};
@@ -10,8 +10,8 @@ use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::task;
// This is not actually unused --- we need this trait to be in scope for
// the tests that sue TaskExecutor::current().execute(). The compiler
// doesn't realise that.
@@ -22,7 +22,7 @@ use futures::sync::oneshot;
mod from_block_on_all {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()>>) + 'static>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()>>) + 'static>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -36,7 +36,8 @@ mod from_block_on_all {
})));
Ok::<_, ()>("hello")
})).unwrap();
}))
.unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
@@ -72,7 +73,8 @@ fn block_waits() {
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
}))
.unwrap();
assert_eq!(1, cnt2.get());
}
@@ -100,11 +102,14 @@ fn spawn_many() {
mod does_not_set_global_executor_by_default {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()> + Send>) -> Result<(), E> + 'static, E>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()> + Send>) -> Result<(), E> + 'static, E>(
spawn: F,
) {
block_on_all(lazy(|| {
spawn(Box::new(lazy(|| ok()))).unwrap_err();
ok()
})).unwrap()
}))
.unwrap()
}
#[test]
@@ -123,20 +128,22 @@ mod from_block_on_future {
use super::*;
fn test<F: Fn(Box<Future<Item = (), Error = ()>>)>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let cnt = Rc::new(Cell::new(0));
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
tokio_current_thread
.block_on(lazy(|| {
let cnt = cnt.clone();
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
Ok::<_, ()>(())
})).unwrap();
Ok::<_, ()>(())
}))
.unwrap();
tokio_current_thread.run().unwrap();
@@ -150,7 +157,11 @@ mod from_block_on_future {
#[test]
fn execute() {
test(|f| { tokio_current_thread::TaskExecutor::current().execute(f).unwrap(); });
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
});
}
}
@@ -170,8 +181,8 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut rc = Rc::new(());
@@ -189,10 +200,12 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
})).unwrap();
tokio_current_thread
.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
}))
.unwrap();
drop(tokio_current_thread);
@@ -202,12 +215,15 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -216,7 +232,9 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
@@ -225,12 +243,11 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
mod run_in_future {
@@ -241,13 +258,12 @@ mod run_in_future {
fn spawn() {
block_on_all(lazy(|| {
tokio_current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}));
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -256,18 +272,16 @@ mod run_in_future {
block_on_all(lazy(|| {
tokio_current_thread::TaskExecutor::current()
.execute(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}))
.unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
@@ -288,9 +302,7 @@ fn tick_on_infini_future() {
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.spawn(Infini { num: num.clone() })
.turn(None)
.unwrap();
@@ -347,7 +359,8 @@ mod tasks_are_scheduled_fairly {
});
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -359,8 +372,8 @@ mod tasks_are_scheduled_fairly {
fn execute() {
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
.execute(f)
.unwrap();
})
}
}
@@ -370,8 +383,8 @@ mod and_turn {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -379,24 +392,25 @@ mod and_turn {
let mut tokio_current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
Ok(())
})));
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || Ok(()))));
// Turn once...
tokio_current_thread.turn(None).unwrap();
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
c.set(1 + c.get());
// Spawn!
spawn(Box::new(lazy(move || {
dotspawn(
&mut tokio_current_thread,
Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})));
// Spawn!
spawn(Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})),
);
// This does not run the newly spawned thread
tokio_current_thread.turn(None).unwrap();
@@ -409,12 +423,15 @@ mod and_turn {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -423,11 +440,12 @@ mod and_turn {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
mod in_drop {
@@ -455,23 +473,24 @@ mod in_drop {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut tokio_current_thread = CurrentThread::new();
let mut tokio_current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
dotspawn(&mut tokio_current_thread, Box::new(
MyFuture {
dotspawn(
&mut tokio_current_thread,
Box::new(MyFuture {
_data: Box::new(OnDrop(Some(move || {
spawn(Box::new(lazy(move || {
tx.send(()).unwrap();
Ok(())
})));
}))),
}
));
}),
);
tokio_current_thread.block_on(rx).unwrap();
tokio_current_thread.run().unwrap();
@@ -479,12 +498,15 @@ mod in_drop {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -493,7 +515,9 @@ mod in_drop {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
@@ -562,13 +586,17 @@ fn turn_has_polled() {
tokio_current_thread.spawn(receiver.then(|_| Ok(())));
// Turn once...
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver once, but considered it not ready
assert!(res.has_polled());
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled nothing, the receiver is not ready yet
assert!(!res.has_polled());
@@ -577,14 +605,18 @@ fn turn_has_polled() {
sender.send(()).unwrap();
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver, it's ready now
assert!(res.has_polled());
// Now the executor should be empty
assert!(tokio_current_thread.is_idle());
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// So should've polled nothing
assert!(!res.has_polled());
@@ -646,46 +678,41 @@ fn turn_fair() {
// Once an item is received on the oneshot channel, it will immediately
// immediately make the second oneshot channel ready
tokio_current_thread.spawn(receiver
.map_err(|_| unreachable!())
.and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
tokio_current_thread.spawn(receiver.map_err(|_| unreachable!()).and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
Ok(())
})
);
Ok(())
}));
let receiver_2_done = Rc::new(Cell::new(false));
let receiver_2_done_clone = receiver_2_done.clone();
tokio_current_thread.spawn(receiver_2
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_2.map_err(|_| unreachable!()).and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
}));
// The third receiver is only woken up from our Park implementation, it simulates
// e.g. a socket that first has to be polled to know if it is ready now
let receiver_3_done = Rc::new(Cell::new(false));
let receiver_3_done_clone = receiver_3_done.clone();
tokio_current_thread.spawn(receiver_3
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_3.map_err(|_| unreachable!()).and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
}));
// First turn should've polled both and considered them not ready
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(res.has_polled());
// Next turn should've polled nothing
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(!res.has_polled());
assert!(!receiver_1_done.get());
@@ -736,10 +763,12 @@ fn spawn_from_other_thread() {
let (sender, receiver) = oneshot::channel::<()>();
thread::spawn(move || {
handle.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
}))
.unwrap();
});
let _ = current_thread.block_on(receiver).unwrap();
@@ -758,10 +787,12 @@ fn spawn_from_other_thread_unpark() {
thread::spawn(move || {
let _ = receiver_2.recv().unwrap();
handle.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
}))
.unwrap();
});
// Ensure that unparking the executor works correctly. It will first
@@ -769,13 +800,15 @@ fn spawn_from_other_thread_unpark() {
// lazy future below which will cause the future to be spawned from
// the other thread. Then the executor will park but should be woken
// up because *now* we have a new future to schedule
let _ = current_thread.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1)
).unwrap();
let _ = current_thread
.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1),
)
.unwrap();
}
#[test]
@@ -785,10 +818,12 @@ fn spawn_from_executor_with_handle() {
let (tx, rx) = oneshot::channel();
current_thread.spawn(lazy(move || {
handle.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}))
.unwrap();
Ok::<_, ()>(())
}));
+5
View File
@@ -1,3 +1,8 @@
# 0.1.7 (March 22, 2019)
### Added
- `TypedExecutor` for spawning futures of a specific type (#993).
# 0.1.6 (January 6, 2019)
* Implement `Unpark` for `Arc<Unpark>` (#802).
+5 -2
View File
@@ -8,8 +8,8 @@ name = "tokio-executor"
# - Cargo.toml
# - README.md
# - Create "v0.1.x" git tag.
version = "0.1.6"
documentation = "https://docs.rs/tokio-executor/0.1.6/tokio_executor"
version = "0.1.7"
documentation = "https://docs.rs/tokio-executor/0.1.7/tokio_executor"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -23,3 +23,6 @@ categories = ["concurrency", "asynchronous"]
[dependencies]
crossbeam-utils = "0.6.2"
futures = "0.1.19"
[dev-dependencies]
tokio = "0.1.18"
+5 -5
View File
@@ -2,7 +2,7 @@
Task execution related traits and utilities.
[Documentation](https://docs.rs/tokio-executor/0.1.6/tokio_executor)
[Documentation](https://docs.rs/tokio-executor/0.1.7/tokio_executor)
## Overview
@@ -31,10 +31,10 @@ executor, including:
* [`Park`] abstracts over blocking and unblocking the current thread.
[`Executor`]: https://tokio-rs.github.io/tokio/tokio_executor/trait.Executor.html
[`enter`]: https://tokio-rs.github.io/tokio/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://tokio-rs.github.io/tokio/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://tokio-rs.github.io/tokio/tokio_executor/park/index.html
[`Executor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/trait.Executor.html
[`enter`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/park/trait.Park.html
## License
+6 -4
View File
@@ -1,7 +1,7 @@
use std::prelude::v1::*;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::prelude::v1::*;
use futures::{self, Future};
@@ -70,7 +70,10 @@ pub fn enter() -> Result<Enter, EnterError> {
impl Enter {
/// Register a callback to be invoked if and when the thread
/// ceased to act as an executor.
pub fn on_exit<F>(&mut self, f: F) where F: FnOnce() + 'static {
pub fn on_exit<F>(&mut self, f: F)
where
F: FnOnce() + 'static,
{
self.on_exit.push(Box::new(f));
}
@@ -88,7 +91,6 @@ impl Enter {
pub fn block_on<F: Future>(&mut self, f: F) -> Result<F::Item, F::Error> {
futures::executor::spawn(f).wait_future()
}
}
impl fmt::Debug for Enter {
@@ -103,7 +105,7 @@ impl Drop for Enter {
assert!(c.get());
if self.permanent {
return
return;
}
for callback in self.on_exit.drain(..) {
+50
View File
@@ -0,0 +1,50 @@
use std::error::Error;
use std::fmt;
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
+151
View File
@@ -0,0 +1,151 @@
use futures::Future;
use SpawnError;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// This trait is primarily intended to implemented by executors and used to
/// back `tokio::spawn`. Libraries and applications **may** use this trait to
/// bound generics, but doing so will limit usage to futures that implement
/// `Send`. Instead, libraries and applications are recommended to use
/// [`TypedExecutor`] as a bound.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
/// [`TypedExecutor`]: ../trait.TypedExecutor.html
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
+41 -24
View File
@@ -1,4 +1,4 @@
use super::{Executor, Enter, SpawnError};
use super::{Enter, Executor, SpawnError};
use futures::{future, Future};
@@ -33,24 +33,22 @@ impl DefaultExecutor {
/// `DefaultExecutor::current()` on thread A and then sending the result to
/// thread B will _not_ reference the default executor that was set on thread A.
pub fn current() -> DefaultExecutor {
DefaultExecutor {
_dummy: (),
}
DefaultExecutor { _dummy: () }
}
#[inline]
fn with_current<F: FnOnce(&mut Executor) -> R, R>(f: F) -> Option<R> {
EXECUTOR.with(|current_executor| {
match current_executor.replace(State::Active) {
EXECUTOR.with(
|current_executor| match current_executor.replace(State::Active) {
State::Ready(executor_ptr) => {
let executor = unsafe { &mut *executor_ptr };
let result = f(executor);
current_executor.set(State::Ready(executor_ptr));
Some(result)
},
}
State::Empty | State::Active => None,
}
})
},
)
}
}
@@ -61,18 +59,21 @@ enum State {
// default executor is defined and ready to be used
Ready(*mut Executor),
// default executor is currently active (used to detect recursive calls)
Active
Active,
}
/// Thread-local tracking the current executor
thread_local!(static EXECUTOR: Cell<State> = Cell::new(State::Empty));
thread_local! {
/// Thread-local tracking the current executor
static EXECUTOR: Cell<State> = Cell::new(State::Empty)
}
// ===== impl DefaultExecutor =====
impl super::Executor for DefaultExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
DefaultExecutor::with_current(|executor| executor.spawn(future))
.unwrap_or_else(|| Err(SpawnError::shutdown()))
}
@@ -83,8 +84,22 @@ impl super::Executor for DefaultExecutor {
}
}
impl<T> super::TypedExecutor<T> for DefaultExecutor
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
super::Executor::spawn(self, Box::new(future))
}
fn status(&self) -> Result<(), SpawnError> {
super::Executor::status(self)
}
}
impl<T> future::Executor<T> for DefaultExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
if let Err(e) = super::Executor::status(self) {
@@ -144,10 +159,10 @@ where T: Future<Item = (), Error = ()> + Send + 'static,
/// # pub fn main() {}
/// ```
pub fn spawn<T>(future: T)
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
DefaultExecutor::current().spawn(Box::new(future))
.unwrap()
DefaultExecutor::current().spawn(Box::new(future)).unwrap()
}
/// Set the default executor for the duration of the closure
@@ -156,13 +171,15 @@ pub fn spawn<T>(future: T)
///
/// This function panics if there already is a default executor set.
pub fn with_default<T, F, R>(executor: &mut T, enter: &mut Enter, f: F) -> R
where T: Executor,
F: FnOnce(&mut Enter) -> R
where
T: Executor,
F: FnOnce(&mut Enter) -> R,
{
EXECUTOR.with(|cell| {
match cell.get() {
State::Ready(_) | State::Active =>
panic!("default executor already set for execution context"),
State::Ready(_) | State::Active => {
panic!("default executor already set for execution context")
}
_ => {}
}
@@ -200,7 +217,7 @@ unsafe fn hide_lt<'a>(p: *mut (Executor + 'a)) -> *mut (Executor + 'static) {
#[cfg(test)]
mod tests {
use super::{Executor, DefaultExecutor, with_default};
use super::{with_default, DefaultExecutor, Executor};
#[test]
fn default_executor_is_send_and_sync() {
+25 -195
View File
@@ -1,5 +1,5 @@
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.6")]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.7")]
//! Task execution related traits and utilities.
//!
@@ -17,8 +17,12 @@
//! This crate provides traits and utilities that are necessary for building an
//! executor, including:
//!
//! * The [`Executor`] trait describes the API for spawning a future onto an
//! executor.
//! * The [`Executor`] trait spawns future object onto an executor.
//!
//! * The [`TypedExecutor`] trait spawns futures of a specific type onto an
//! executor. This is used to be generic over executors that spawn futures
//! that are either `Send` or `!Send` or implement executors that apply to
//! specific futures.
//!
//! * [`enter`] marks that the current thread is entering an execution
//! context. This prevents a second executor from accidentally starting from
@@ -29,7 +33,19 @@
//!
//! * [`Park`] abstracts over blocking and unblocking the current thread.
//!
//! # Implementing an executor
//!
//! Executors should always implement `TypedExecutor`. This usually is the bound
//! that applications and libraries will use when generic over an executor. See
//! the [trait documentation][`TypedExecutor`] for more details.
//!
//! If the executor is able to spawn all futures that are `Send`, then the
//! executor should also implement the `Executor` trait. This trait is rarely
//! used directly by applications and libraries. Instead, `tokio::spawn` is
//! configured to dispatch to type that implements `Executor`.
//!
//! [`Executor`]: trait.Executor.html
//! [`TypedExecutor`]: trait.TypedExecutor.html
//! [`enter`]: fn.enter.html
//! [`DefaultExecutor`]: struct.DefaultExecutor.html
//! [`Park`]: park/index.html
@@ -39,200 +55,14 @@ extern crate crossbeam_utils;
extern crate futures;
mod enter;
mod error;
mod executor;
mod global;
pub mod park;
mod typed;
pub use enter::{enter, Enter, EnterError};
pub use error::SpawnError;
pub use executor::Executor;
pub use global::{spawn, with_default, DefaultExecutor};
use futures::Future;
use std::error::Error;
use std::fmt;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementers are encouraged to avoid panics. However, a panic is
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
pub use typed::TypedExecutor;
+2 -1
View File
@@ -190,7 +190,8 @@ impl ParkThread {
/// Get a reference to the `ParkThread` handle for this thread.
fn with_current<F, R>(&self, f: F) -> R
where F: FnOnce(&Parker) -> R,
where
F: FnOnce(&Parker) -> R,
{
CURRENT_PARKER.with(|inner| f(inner))
}
+181
View File
@@ -0,0 +1,181 @@
use SpawnError;
/// A value that spawns futures of a specific type.
///
/// The trait is generic over `T`: the type of future that can be spawened. This
/// is useful for implementing an executor that is only able to spawn a specific
/// type of future.
///
/// The [`spawn`] function is used to submit the future to the executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// This trait is useful as a bound for applications and libraries in order to
/// be generic over futures that are `Send` vs. `!Send`.
///
/// # Examples
///
/// Consider a function that provides an API for draining a `Stream` in the
/// background. To do this, a task must be spawned to perform the draining. As
/// such, the function takes a stream and an executor on which the background
/// task is spawned.
///
/// ```rust
/// #[macro_use]
/// extern crate futures;
/// extern crate tokio;
///
/// use futures::{Future, Stream, Poll};
/// use tokio::executor::TypedExecutor;
/// use tokio::sync::oneshot;
///
/// pub fn drain<T, E>(stream: T, executor: &mut E)
/// -> impl Future<Item = (), Error = ()>
/// where
/// T: Stream,
/// E: TypedExecutor<Drain<T>>
/// {
/// let (tx, rx) = oneshot::channel();
///
/// executor.spawn(Drain {
/// stream,
/// tx: Some(tx),
/// }).unwrap();
///
/// rx.map_err(|_| ())
/// }
///
/// // The background task
/// pub struct Drain<T: Stream> {
/// stream: T,
/// tx: Option<oneshot::Sender<()>>,
/// }
///
/// impl<T: Stream> Future for Drain<T> {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
/// loop {
/// let item = try_ready!(
/// self.stream.poll()
/// .map_err(|_| ())
/// );
///
/// if item.is_none() { break; }
/// }
///
/// self.tx.take().unwrap().send(()).map_err(|_| ());
/// Ok(().into())
/// }
/// }
/// # pub fn main() {}
/// ```
///
/// By doing this, the `drain` fn can accept a stream that is `!Send` as long as
/// the supplied executor is able to spawn `!Send` types.
pub trait TypedExecutor<T> {
/// Spawns a future to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// executor takes ownership of the future and becomes responsible for
/// driving the future to completion.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// my_executor.spawn(MyFuture).unwrap();
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: T) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// if my_executor.status().is_ok() {
/// my_executor.spawn(MyFuture).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E, T> TypedExecutor<T> for Box<E>
where
E: TypedExecutor<T>,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
+5 -4
View File
@@ -1,15 +1,16 @@
extern crate tokio_executor;
extern crate futures;
extern crate tokio_executor;
use tokio_executor::*;
use futures::{Future, future::lazy};
use futures::{future::lazy, Future};
use tokio_executor::DefaultExecutor;
mod out_of_executor_context {
use super::*;
use tokio_executor::Executor;
fn test<F, E>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) -> Result<(), E>,
F: Fn(Box<Future<Item = (), Error = ()> + Send>) -> Result<(), E>,
{
let res = spawn(Box::new(lazy(|| Ok(()))));
assert!(res.is_err());
+6
View File
@@ -1,3 +1,9 @@
# 0.1.6 (March 1, 2019)
### Added
- File::try_clone() (#850).
- Async equivalent of read / write file helpers being added to `std` (#896).
# 0.1.5 (January 6, 2019)
* Add examples to `File` API docs (#786).
+7 -7
View File
@@ -8,13 +8,13 @@ name = "tokio-fs"
# - Cargo.toml
# - README.md
# - Create "v0.1.x" git tag.
version = "0.1.5"
version = "0.1.6"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-fs/0.1.5/tokio_fs"
documentation = "https://docs.rs/tokio-fs/0.1.6/tokio_fs"
description = """
Filesystem API for Tokio.
"""
@@ -23,13 +23,13 @@ categories = ["asynchronous", "network-programming", "filesystem"]
[dependencies]
futures = "0.1.21"
tokio-threadpool = { version = "0.1.3", path = "../tokio-threadpool" }
tokio-io = { version = "0.1.6", path = "../tokio-io" }
tokio-threadpool = "0.1.3"
tokio-io = "0.1.6"
[dev-dependencies]
rand = "0.6"
tempfile = "3"
tempdir = "0.3"
tokio-io = { version = "0.1.6", path = "../tokio-io" }
tokio-codec = { version = "0.1.0", path = "../tokio-codec" }
tokio = { version = "0.1.7", path = ".." }
tokio-io = "0.1.6"
tokio-codec = "0.1.0"
tokio = "0.1.7"
+1 -1
View File
@@ -2,7 +2,7 @@
Asynchronous filesystem manipulation operations (and stdin, stdout, stderr).
[Documentation](https://docs.rs/tokio-fs/0.1.5/tokio_fs)
[Documentation](https://docs.rs/tokio-fs/0.1.6/tokio_fs)
## Overview
+17 -19
View File
@@ -2,39 +2,35 @@
#![deny(deprecated, warnings)]
extern crate futures;
extern crate tokio_fs;
extern crate tokio_codec;
extern crate tokio_fs;
extern crate tokio_threadpool;
use tokio_fs::{stdin, stdout, stderr};
use tokio_codec::{FramedRead, FramedWrite, LinesCodec};
use tokio_fs::{stderr, stdin, stdout};
use tokio_threadpool::Builder;
use futures::{Future, Stream, Sink};
use futures::{Future, Sink, Stream};
use std::io;
pub fn main() -> Result<(), Box<std::error::Error>> {
let pool = Builder::new()
.pool_size(1)
.build();
let pool = Builder::new().pool_size(1).build();
pool.spawn({
let input = FramedRead::new(stdin(), LinesCodec::new());
let output = FramedWrite::new(stdout(), LinesCodec::new())
.with(|line: String| {
let mut out = "OUT: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let output = FramedWrite::new(stdout(), LinesCodec::new()).with(|line: String| {
let mut out = "OUT: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let error = FramedWrite::new(stderr(), LinesCodec::new())
.with(|line: String| {
let mut out = "ERR: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let error = FramedWrite::new(stderr(), LinesCodec::new()).with(|line: String| {
let mut out = "ERR: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let dst = output.fanout(error);
@@ -44,6 +40,8 @@ pub fn main() -> Result<(), Box<std::error::Error>> {
.map_err(|e| panic!("io error = {:?}", e))
});
pool.shutdown_on_idle().wait().map_err(|_| "failed to shutdown the thread pool")?;
pool.shutdown_on_idle()
.wait()
.map_err(|_| "failed to shutdown the thread pool")?;
Ok(())
}
+5 -7
View File
@@ -17,30 +17,28 @@ pub fn create_dir<P: AsRef<Path>>(path: P) -> CreateDirFuture<P> {
#[derive(Debug)]
pub struct CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
path: P,
}
impl<P> CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
fn new(path: P) -> CreateDirFuture<P> {
CreateDirFuture {
path: path,
}
CreateDirFuture { path: path }
}
}
impl<P> Future for CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
::blocking_io(|| fs::create_dir(&self.path) )
::blocking_io(|| fs::create_dir(&self.path))
}
}

Some files were not shown because too many files have changed in this diff Show More