Compare commits

...
Author SHA1 Message Date
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
Carl Lerche 11e2af66a8 Bump Tokio to v0.1.15. (#869)
Also bumps:

- tokio-sync (0.1.0)
- tokio-threadpool (0.1.11)
- tokio-timer (0.2.9)
2019-01-25 10:20:09 -08:00
Carl Lerche a4aae1459c chore: move enumerate test to correct location (#867) 2019-01-24 20:47:45 -08:00
Zahari Dichev 12546d1d9c tokio-timer: fix DelayQueue bug when inserting shorter delay (#863)
Reset the delay of the queue in case an item that expires sooner than the last inserted is put
into the queue.
2019-01-24 14:36:41 -08:00
Zahari Dichev fbad6297c5 Add enumerate combinator to Stream (#832) 2019-01-24 11:50:34 -08:00
Jon Gjengset 0ec8986b0b Make reason for try_send errors clearer (#864) 2019-01-23 15:06:24 -08:00
Jon Gjengset c6f8bdb249 Remove T: Debug bound on mpsc Debug impls (#866)
Following from https://github.com/tokio-rs/tokio/pull/865, this PR
removes `#[derive(Debug)]` on `mpsc` sender and receiver types in favor
of explicit `impl fmt::Debug` blocks that don't have a `T: fmt::Debug`
bound.
2019-01-23 18:04:00 -05:00
Jon Gjengset c6f9a069a5 Explicit impl Clone for tx to avoid T: Clone (#865)
`#[derive(Clone)]` on a type `struct Foo<T>` adds an impl that requires that
`T: Clone`:

```rust
impl<T: Clone> Clone for Foo<T>
```

which is unfortunate in the case of senders, because we don't want to require
that the items being sent are `Clone` for the channel sender to be `Clone`.
This PR adds an explicit `impl Clone` for the bounded and unbounded sender
types which does not have the `T: Clone` bound.

Note that this is _also_ an issue with `#[derive(Debug)]`, but that one is
harder to work around as `chan::Tx` _also_ has `#[derive(Debug)]`, as does
`chan::Chan`, so we'd have to add explicit impls for all of them to make
progress.
2019-01-23 15:51:44 -05:00
Sean McArthur 9f356d6244 tokio-sync: add into_inner for TrySendErrors (#862) 2019-01-22 14:48:22 -08:00
Carl Lerche 13083153aa Introduce tokio-sync crate containing synchronization primitives. (#839)
Introduce a tokio-sync crate containing useful synchronization primitives for programs
written using Tokio.

The initial release contains:

* An mpsc channel
* A oneshot channel
* A semaphore implementation
* An `AtomicTask` primitive.

The `oneshot` and `mpsc` channels are new implementations providing improved
performance characteristics. In some benchmarks, the new mpsc channel shows
up to 7x improvement over the version provided by the `futures` crate. Unfortunately,
the `oneshot` implementation only provides a slight performance improvement as it
is mostly limited by the `futures` 0.1 task system. Once updated to the `std` version
of `Future` (currently nightly only), much greater performance improvements should
be achievable by `oneshot`.

Additionally, he implementations provided here are checked using
[Loom](http://github.com/carllerche/loom/), which provides greater confidence of
correctness.
2019-01-22 11:37:26 -08:00
rmcteggart-r7 91f20e33a4 docs: deal with Result instead of using unwrap (#860) 2019-01-20 14:21:17 -05:00
Eliza Weisman 983e9d1b67 timer: Fix DelayQueue delay reset logic (#851) 2019-01-20 08:38:39 -05:00
Stjepan Glavina 4c8f274db9 threadpool: drop incomplete tasks on shutdown (#722)
## Motivation

When the thread pool shuts down, futures that have been polled at least once but not completed yet are simply leaked. We should drop them instead.

## Solution

Multiple changes are introduced:

* Tasks are assigned a home worker the first time they are polled.

* Each worker contains a set of tasks (`Arc<Task>`) it is home to. When a task is assigned a home worker, it is registered in that worker's set of tasks. When the task is completed, it is unregistered from the set.

* When the thread pool shuts down and after all worker threads stop, the remaining tasks in workers' sets are aborted, i.e. they are switched to the `Aborted` state and their `Future`s are dropped.

* The thread pool shutdown process is refactored to make it more robust. We don't  track the number of active threads manually anymore. Instead, there's  `Arc<ShutdownTrigger>` that aborts remaining tasks and completes the `Shutdown` future once it gets destroyed (when all `Worker`s and `ThreadPool` get dropped because they're the only ones to contain strong references to the `ShutdownTrigger`).

Closes #424 
Closes #428
2019-01-17 22:12:25 +01:00
Marek Kotewicz c980837581 docs: missing links in tokio-timer::delay_queue (#845) 2019-01-13 21:20:08 +01:00
Marek Kotewicz eec370cae8 docs: fixed links in tokio-timer (#844)
* docs: fixed links in tokio-timer/src/timer/mod.rs

* docs: fixed links in tokio-timer::clock
2019-01-12 10:06:55 -08:00
Marek Kotewicz 733d432b80 docs: fixed links to tokio_timer::clock::Now (#842)
* docs: fixed links to tokio_timer::clock::Now in tokio-timer/src/timer/mod.rs

* docs: fixed links to std::time::Instant in tokio-timer/src/timer/mod.rs
2019-01-10 23:49:13 +01:00
Carl Lerche 74c473d68f travis: allow nightly Rust CI to fail (#843) 2019-01-10 11:27:58 -08:00
Sean McArthur d95c697781 tokio: update tokio-threadpool minimum version (#838) 2019-01-07 16:49:08 -08:00
Carl Lerche 25e835c5b7 tcp: specify version for tokio dev dependency
This is required for publishing to crates.io
2019-01-06 23:31:24 -08:00
Carl Lerche 961aae41c4 Bump version to 0.1.14. (#836)
Also bumps:

* tokio-async-await (0.1.5)
* tokio-executor (0.1.6)
* tokio-fs (0.1.5)
* tokio-io (0.1.11)
* tokio-reactor (0.1.8)
* tokio-tcp (0.1.3)
* tokio-threadpool (0.1.10)
* tokio-tls (0.2.1)
* tokio-uds (0.2.5)

...and updates LICENSE files to 2019.
2019-01-06 23:25:55 -08:00
Carl Lerche 74c73b218e Revert "util: implement stream debounce combinator (#747)" (#834)
This reverts commit 7a49ebb65e.

The commit conflicted with another change that was merged, causing CI to fail. The public API
also requires a bit more refinement (#833) and Tokio crates need to be released.
2019-01-06 16:56:49 -08:00
Moritz Gunz 7a49ebb65e util: implement stream debounce combinator (#747) 2019-01-05 11:08:12 -05:00
Stjepan Glavina a687922746 tcp: deprecate TcpStream::try_clone() (#824) 2019-01-05 10:55:58 -05:00
Stjepan Glavina df299ced45 threadpool: panic if a worker thread cannot be spawned (#826) 2019-01-05 10:53:38 -05:00
Carl Lerche 78d1fe0eb0 ci: limit min rust version to cargo check (#829) 2019-01-05 10:52:26 -05:00
Ryan Huang fc8cde383a docs: fix link to ThreadPool (#830) 2019-01-05 10:51:17 -05:00
Sean McArthur 76198f63d7 Provide optional features on tokio crate (#808)
Disabling all features means the only dependency is `futures`.

Relevant pieces of the API can then be enabled with the following features:

- `codec`
- `fs`
- `io`
- `reactor`
- `tcp`
- `timer`
- `udp`
- `uds`

This also introduces the beginnings of enabling only certain pieces of the `Runtime`. As a start, the entire default runtime API is enabled via the `rt-full` feature.
2019-01-04 11:42:33 -08:00
Carl Lerche 39dc5706b7 travis: remove commented out code. (#828)
The commented out lines are no longer relevant and will not be brought
back.
2019-01-03 22:04:09 -08:00
Carl Lerche cbecb87797 executor: fix build (#825)
Two unrelated PRs to the same file resulted in a broken build. This
patch fixes the build by including `Arc`.
2019-01-03 11:26:58 -08:00
Carl Lerche f0bdf1980c threadpool: remove unused fn (#822)
The unused lint on nightly has discovered a new unused fn.
2019-01-03 09:34:37 -08:00
Stjepan Glavina 5e2d93f060 Use Crossbeam's Parker/Unparker (#528) 2019-01-02 21:51:22 -08:00
Taiki Endo 9a8d087c69 Allow deprecated Error::cause (#818)
Error::cause is deprecated in Rust 1.33, but this allows Error::cause
until the minimum supported version of tokio is Rust 1.30.

When the minimum support version of tokio reaches Rust 1.30,
replace Error::cause with Error::source.

Fixes: #817
2019-01-02 14:12:11 -08:00
gralpli 30f59670c8 Clarify what NoopWaker does (#819) 2019-01-02 12:29:30 -08:00
jq-rs 9e4ddaeaf3 examples: single-threaded chat combinator example (#794) 2018-12-29 10:16:30 -05:00
Balthild Ires 03e2e864f3 Stablize pin feature (#814)
Box::pinned has been renamed to Box::pin. Meanwhile, the pin feature
no longer requires an attribute to enable.

Fixes: #813
2018-12-28 12:12:31 -08:00
Sean McArthur c8a990eda4 tokio-reactor: deprecates Handle::current() (#805)
The side effects of calling `Handle::current()` from outside of a
runtime could be very surprising, since it would start up a background
reactor.
2018-12-28 12:09:35 -08:00
Pavel Strakhov 1a5026324f executor: impl Unpark for Arc<Unpark> (#802) 2018-12-28 14:40:04 -05:00
Stjepan Glavina fdf4aba621 threadpool: introduce a global task queue (#798) 2018-12-28 14:34:54 -05:00
Roman 201b6ce53a ci: remove ALLOW_FAILURES=false in travis for nightly cargo doc (#816) 2018-12-28 10:06:00 -05:00
Roman db69275202 docs: fix warnings for nightly docs (#792) 2018-12-17 15:20:46 -05:00
Roman af85cb3430 ci: improve travis run times (#793) 2018-12-17 15:18:27 -05:00
Christian Bourjau 36f1a19ac8 Minor change in documentation of Decoder::decode (#797)
`None` -> `Ok(None)`
2018-12-13 11:14:38 -08:00
Stjepan Glavina 6aa990ea75 threadpool: fix semaphore deadlock (#795) 2018-12-12 16:42:18 -05:00
Simon Farnsworth 760a7667d6 threadpool: improve the documentation of blocking (#789) 2018-12-05 15:20:19 -05:00
Matt Gathu 2283b63e9e fs: added usage examples/doctests to File (#786) 2018-12-01 21:07:48 -05:00
Felix Obenhuber 8263e5f18d uds: fix WouldBlock case in UnixDatagram send methods (#782) 2018-11-30 19:40:03 -05:00
Carl Lerche b3e57b60d0 examples: remove reference to tokio-core (#780) 2018-11-28 14:55:31 -05:00
Matt Gathu 1cd0ebfc5e tcp: add usage examples to TcpListener and TcpStream (#775)
Refs: https://github.com/rust-lang-nursery/wg-net/issues/54
2018-11-28 13:05:35 -05:00
David Kellum 4797d79950 reactor: update to parking_lot 0.7 (#778) 2018-11-28 09:29:31 -08:00
Steven Fackler e7d9ba7e51 tls: make TlsConnector and TlsAcceptor derive Clone (#777) 2018-11-27 07:52:17 -05:00
luben karavelov 527dc0a66f net: export UnixDatagram and UnixDatagramFramed (#772) 2018-11-23 08:41:32 -05:00
Carl Lerche b117fc1d65 Bump version to v0.1.13 (#771)
This also bumps the following sub crate versions:

* tokio-current-thread (0.1.4)
* tokio-reactor (0.1.7)
* tokio-signal (0.2.7)
* tokio-threadpool (0.1.9)
* tokio-timer (0.2.8)
* tokio-udp (0.1.3)
* tokio-uds (0.2.4)
2018-11-21 17:11:31 -08:00
Felix Obenhuber 272e09d349 threadpool: remove smoke example (#764) (#770) 2018-11-21 14:23:36 -08:00
Stjepan Glavina 3235749006 threadpool: refactor pool shutdown (#769) 2018-11-20 21:43:23 +01:00
Stjepan Glavina 9c037044c4 threadpool: rename inner to something more descriptive (#768)
`inner` is a fitting name for variables of type named `Inner`, but in other cases I find them confusing - sometimes `inner` refers to a `Pool`, sometimes to a `Sender`. I renamed a bunch of variables named `inner` to be more descriptive.

This PR is the first step in an effort of splitting https://github.com/tokio-rs/tokio/pull/722#issuecomment-439552671 into multiple PRs.
2018-11-20 20:05:14 +01:00
Patrick Barrett 3658e10045 uds: implement UnixDatagramFramed (#453)
Implement `Stream + Sink` layer on top of unix domain sockets
using codecs.
2018-11-20 09:19:34 -08:00
Carl Lerche ed3ece266b current-thread: fix shutdown on idle (#763)
When spawning using `Handle` while on the executor, tasks were being
double counted. This prevented the number of active tasks to reach zero,
thus preventing the executor from shutting down.

This changes `spawn` to check if being called from the executor
**before** incrementing the number of active tasks.

Fixes #760
2018-11-20 09:17:07 -08:00
Liran Ringel 9b1a45cc6a tests: handle errors properly in examples (#748) 2018-11-20 11:10:36 -05:00
Carl Lerche 477fa5580a ci: Don't deploy docs if $TARGET is set (#762) 2018-11-19 21:22:28 -08:00
Toby Lawrence bb6cca8ff0 tests: switch to Windows Server 2016 for AppVeyor builds. (#761)
Should hopefully fix the underlying bug that was causing tokio-tls tests to occasionally fail on Windows.

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 20:18:37 -05:00
Moritz Gunz e166c4d912 Implement throttle combinator (#736)
Throttle down a stream by enforcing a fixed delay between items.
2018-11-19 15:04:55 -08:00
Toby Lawrence b7506cf663 Allow nightly builds to fail. (#743)
* tests: allow nightly builds to fail

Signed-off-by: Toby Lawrence <[email protected]>
2018-11-19 17:13:56 -05:00
Carl Lerche dc4a29359f io: allow deprecated code in length_delimited test (#759)
This file is testing deprecated code, so it should be permitted to
access deprecated code.
2018-11-19 14:11:46 -08:00
Bastian Köcher d3dca4552b Expose after_start and before_stop in runtime::Builder (#756)
Closes #705
2018-11-19 09:04:58 -08:00
andoks 42a0df1ea4 Fix async await README example (#758)
* async-await: fix README example dependencies

As per commit "async-await: track nightly changes (#661)" ( commit
2f690d30bc)

> The `tokio-async-await` crate is no longer a facade. Instead, the
> `tokio` crate provides a feature flag to enable async/await support.

Ensure the example in the async-await README file also works by
correctly declaring this updated dependency

* async-await: remove unnecessary 'edition' declaration from README

As the "edition" feature was stabilized in rust v1.30 and async-await
specifies that the nightly toolchain must be used, remove the use of the
"edition" feature gate since it is enabled by default.
2018-11-17 20:58:19 -08:00
Brian Myers a98eab6eff rt: fix Builder docs to no longer use deprecated methods (#749) 2018-11-16 14:58:08 -08:00
Ivan Petkov 5a5dde70b3 signal: miscellaneous tweaks and improvements (#751)
* Minimize allocation needed for channels

* Use a newtype for signal ids

* We can just cast the raw pointer to a `usize` and still perform a
simple identity check, without incurring any implications of storing a
raw pointer (e.g. previously Signal was !Sync and had an unsafe impl of
Send, and now it is naturally Sync+Send)

* Broadcast with `try_send` instead of `start_send`

The `Stream::start_send` method uses backpressure and schedules the
current task to be notified whenever the channel has additional room,
which means we'll generate a lot of unnecessary wakeups whenever a
channel gets full

By changing to `try_send` and handling any errors, we ensure the
Driver's task won't get woken up when a Signal finally consumes its
notification, since we're coalescing things anyway
2018-11-16 14:56:43 -08:00
Alex Gaynor d0963774a3 chore: bump rand dependency to 0.6 (#753) 2018-11-16 14:54:14 -08:00
Felix Obenhuber c83355235c uds: minor doc fix in UnixStream and UnixDatagram (#754) 2018-11-16 14:53:19 -08:00
Kazuyoshi Kato 33a216e4c1 fs: add more tests (#755)
Fixes #704.
2018-11-16 14:50:06 -08:00
Toralf Wittner 09f2ac85bf udp: add into_parts to RecvDgram (#710)
* udp: add `into_parts` to `RecvDgram`

If `RecvDgram` can not be driven to completion it may become necessary to get back the `UdpSocket` it contains which is currently not possible.

This adds`into_parts` to get the socket as well as the buffer back. Both methods consume `RecvDgram`.

Note that after the future has completed, `into_parts` must not be used, or else a panic will happen.
2018-11-15 10:55:34 -05:00
Ohad Ravid 32a152630f uds: added solaris support in the ucred module (#733) 2018-11-15 10:30:37 -05:00
Kazuyoshi Kato 9153067d66 fs: add tests for directory-related functions (#704) (#724)
This change adds a few tests around directory-related functions.
2018-11-13 18:28:52 -05:00
Kazuyoshi Kato d246964bdf fs: gen_ascii_chars has been deprecated (#735)
Use sample_iter() instead.
2018-11-10 21:35:38 -05:00
Alex Gaynor e700607554 Bumped crossbeam-utils version (#746)
## Motivation

tokio depends on an out of date version of crossbeam-utils, which results in multiple versions of that package being linked in binaries which use other popular libraries.

## Solution

Bump the version; there's no API changes and tests still pass.
2018-11-10 10:39:09 +01:00
Benjamin Saunders 5321550534 Derive Clone for delay_queue::Key (#730)
Improves API ergonomics with minimal forwards-compatibility hazard.
2018-11-09 15:11:17 -08:00
Stjepan Glavina 32e1cafb57 fix tsan errors (#745) 2018-11-09 15:06:46 -08:00
Josh Leverette 49bc4025dd reactor: reduce log level of loop process (#734) 2018-11-07 16:53:53 -05:00
Carl Lerche 51e36e41bc Add tokio-buf and a BufStream trait (#611)
The `BufStream` trait provides an improved API for working with
asynchronous streams of bytes compared to `Stream<Item = [u8]>`
2018-10-29 13:43:48 -07:00
Carl Lerche d011b92b9a rt: fix Runtime::reactor() as used by tokio-core (#721)
* rt: fix `Runtime::reactor()` as used by tokio-core

Up until Tokio v0.1.11, the handle returned by `Runtime::reactor()`
pointed to a reactor instance running in a background thread. The thread
was eagerly spawned.

As of v0.1.12, a reactor instance is created per runtime worker thread.
`Runtime::reactor()` was deprecated and updated to point to the reactor
for one of the worker threads.

A problem occurs when attempting to use the reactor before spawning a
task. Worker threads are spawned lazily, which means that the reactor
referenced by `Runtime::reactor()` is not yet running.

This patch changes `Runtime::reactor` back to a dedicated reactor
running on a background thread. However, the background thread is now
spawned lazily when the deprecated function is first called.

Fixes #720

* Fix comment

Co-Authored-By: carllerche <[email protected]>
2018-10-25 11:23:54 +02:00
Carl Lerche f929576f0e Bump version to 0.1.12 (#718)
Also bumps the following sub-crates:

* tokio-fs (0.1.4)
* tokio-io (0.1.10)
* tokio-signal (0.2.6)
* tokio-threadpool (0.1.8)
* tokio-uds (0.2.3)
2018-10-23 22:00:49 -07:00
Ivan Petkov b0f001a05a signal: Bump version to 0.2.6 (#714)
* Also Update the CHANGELOG to match the rest of the project
2018-10-23 20:39:19 -07:00
Iku Iwasa 2291ba9d0d uds: add NetBSD support (#715) 2018-10-23 20:15:42 -04:00
Name 7f84f6b4ca contributing: fix an invalid link (#716)
Just move a dot to the right place.
2018-10-21 16:56:01 +00:00
Andrew Audibert 5f61bd5252 fix a typo in the contributing guide (#711) 2018-10-19 09:25:11 -07:00
Ryan Dahl bffa3ed558 fs: expose fs::File::from_std() (#696) 2018-10-17 19:51:46 -04:00
Sean McArthur 7b5ef61aeb runtime: check Enter in more places when blocking (#708)
- `tokio::run` checks Enter before creating a new threadpool and
  spawning the main future.
- `Runtime::block_on` now checks Enter
- `Runtime::block_on_all` now checks Enter
2018-10-17 15:25:40 -07:00
Stjepan Glavina 753336de8e threadpool: Arc instead of Inner in Notifier (#702) 2018-10-15 13:24:00 -07:00
Ryan Levick 65aea16ad1 tokio: change hello world to new, simpler example (#690) 2018-10-12 12:42:19 -04:00
nickelc 796fee6364 fs: fix minor documentation error for MetadataFuture (#698) 2018-10-12 12:41:19 -04:00
Stjepan Glavina adb0ba71d4 threadpool: worker threads shouldn't respect keep_alive (#692)
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.

Bug fixes and new features should include tests.

Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->

## Motivation

Now that each worker thread drives its own reactor, reactors have to be driven until the threadpool shuts down. We mustn't use the `keep_alive` setting to shut down a worker thread if it doesn't receive an event from the reactor for a certain duration of time.

<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->

## Solution

Just ignore the `keep_alive` setting when parking in `Worker::sleep`.

<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
2018-10-10 09:05:36 +02:00
David Ross bfa6766f3c re-export tokio_io::read in tokio::io (#689)
Fixes: #688
2018-10-09 19:50:03 -07:00
Nikolay Kim a2f457fa48 io: expose underlying codec (#686) 2018-10-06 19:23:05 -04:00
Eliza Weisman 1879bc49ce codec: Fix panic in LengthDelimitedCodec::encode (#682)
Fixes: #681 

## Motivation

Currently, a potential panic exists in `LengthDelimitedCodec::encode`.
Writing the length field to the `dst` buffer can exceed the buffer
capacity, as `BufMut::put_uint_{le,be}` doesn't reserve more capacity. 

## Solution

This branch adds a call to `dst.reserve` to ensure that there's 
sufficient remaining buffer capacity to hold the length field and
the frame, prior to writing the length field. Previously, capacity
was only reserved later in the function, when writing the frame
to the buffer, and we never reserved capacity for the length field.

I've also added a test that reproduces the issue. The test panics on
master, but passes after making this change.

Signed-off-by: Eliza Weisman <[email protected]>
2018-10-04 12:46:57 -07:00
Sven Marnach 678f6382b8 io: implement prepare_uninitialized_buffer for Take and Chain (#678) 2018-10-04 11:03:43 -07:00
Stjepan Glavina e27b0a46ba threadpool: spawn new tasks onto a random worker (#683)
* threadpool: submit new tasks to a random worker

* Revert unnecessary version bumps
2018-10-03 23:09:20 +02:00
Stjepan Glavina d35d0518f5 runtime: create reactor per worker (#660) 2018-10-02 18:19:27 -07:00
Sven Marnach 886511c0a6 io: fix minor documentation errors for Async{Read,Write} (#677) 2018-10-01 19:34:24 -04:00
Steven Fackler d06bd6b216 Expose keep_alive on the Runtime builder (#676)
This was overlooked when delegating the rest of the threadpool builder
methods from Runtime's builder.
2018-09-28 21:00:50 -07:00
Carl Lerche 2c85cd0991 Bump version to v0.1.11 (#675)
This fixes the dependency on `tokio-async-await` to not be scoped to
unix platforms.

Fixes #673
2018-09-28 11:32:52 -07:00
Carl Lerche 1e45237a28 Bump tokio-uds to v0.2.2 2018-09-27 20:05:23 -07:00
Sean McArthur 3a88d85538 ads: fix UdsStream::read_buf to clear read (not write) readiness (#672) 2018-09-27 20:02:29 -07:00
Carl Lerche b47ad24268 Bump version to v0.1.10, fixing minimal versions (#671)
Some minimal versions were not correctly updated.

Also updates:

* tokio-current-thread (v0.1.3).
2018-09-27 13:00:53 -07:00
Carl Lerche cab9a44e01 Bump version to v0.1.9 (#666)
This also includes bumps to subcrates.

* tokio-async-await (0.1.4)
* tokio-codec (0.1.1)
* tokio-current-thread (0.1.2)
* tokio-executor (0.1.5)
* tokio-io (0.1.9)
* tokio-reactor (0.1.6)
* tokio-tcp (0.1.2)
* tokio-threadpool (0.1.7)
* tokio-timer (0.2.7)
2018-09-26 22:32:51 -07:00
Carl Lerche 964afb2ce3 split facade modules into separate files (#665) 2018-09-26 15:28:57 -07:00
Carl Lerche 2f690d30bc async-await: track nightly changes (#661)
The `tokio-async-await` crate is no longer a facade. Instead, the `tokio` crate
provides a feature flag to enable async/await support.
2018-09-26 10:10:47 -07:00
Stjepan Glavina 331a88cee6 reactor: turn bench-poll into a proper benchmark (#662) 2018-09-26 08:52:35 -07:00
Andrew Tunnell-Jones 46353737e7 tinydb: Update doc to reflect change from RefCell to Mutex (#663)
Fixes: #658
2018-09-26 11:47:31 +02:00
Carl Lerche ffd73a64e7 async-await: streaming hyper body example (#656)
Add reading the hyper body in the async/await example.
2018-09-21 19:58:46 -07:00
Toby Lawrence 1119d572ee io: ensure ReadHalf/WriteHalf do not return WouldBlock directly (#655)
* io: ensure ReadHalf/WriteHalf do not return WouldBlock directly

These facades were passing back WouldBlock when the internal BiLock
couldn't be acquired, which does not fit the intended behavior.

Signed-off-by: Toby Lawrence <[email protected]>

* io: pull from the local crate, not crates.io
2018-09-21 14:59:07 -04:00
Stjepan Glavina 20ca59114a threadpool: impl Drop for Queue (#649)
We need to drain the queue when dropping, or else those `Arc<Task>`s
will be leaked.

Fixes #542
2018-09-21 10:20:41 -07:00
Eliza Weisman 3dd95a9ff1 Add max line length to LinesCodec (#632)
## Motivation

Currently, there is a potential denial of service vulnerability in the
`lines` codec. Since there is no bound on the buffer that holds data
before it is split into a new line, an attacker could send an unbounded
amount of data without sending a `\n` character. 

## Solution

This branch adds a `new_with_max_length` constructor for `LinesCodec`
that configures a limit on the maximum number of bytes per line. When
the limit is reached, the the overly long line will be discarded (in 
`max_length`-sized increments until a newline character or the end of the
buffer is reached. It was also necessary to add some special-case logic
to avoid creating an empty line when the length limit is reached at the 
character immediately _before_ a `\n` character.

Additionally, this branch adds new tests for this function, including a
test for changing the line limit in-flight.

## Notes

This branch makes the following changes from my original PR with
this change (#590):

- The whole too-long line is discarded at once in the first call to `decode`
  that encounters it.
- Only one error is emitted per too-long line.
- Made all the changes requested by @carllerche in
  https://github.com/tokio-rs/tokio/pull/590#issuecomment-420735023

Fixes: #186 

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-20 17:08:00 -07:00
Sven Marnach be67eda117 fix deprecation warning in test for FutureExt::deadline() (#651)
* silence deprecation warnings for deadline in tests
* add new integration test for timeout
2018-09-20 15:26:56 -07:00
Alexander Polakov e267a1922d Reexport TaskExecutor from tokio_current_thread (#652) 2018-09-19 14:46:13 -07:00
Eliza Weisman 9b456f48d9 Set RUST_BACKTRACE=1 on AppVeyor (#650)
## Motivation

Currently, the `RUST_BACKTRACE` environment variable is set to `1` on
Travis CI builds:
https://github.com/tokio-rs/tokio/blob/0ca973a7ebc5b8a29beac1ccb6c73ef26ddcbf22/.travis.yml#L49
However, it's not set on AppVeyor. This can make debugging
Windows-specific CI failures challenging for developers on other
operating systems.

## Solution

This branch sets `RUST_BACKTRACE=1` on AppVeyor.

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 11:53:11 -07:00
Eliza Weisman 0ca973a7eb tokio: deprecate and replace runtime::threadpool_builder (#645)
* Deprecate and hide runtime::Builder::threadpool_builder

* Add functions to runtime::Builder wrapping threadpool builder functions

Signed-off-by: Eliza Weisman <[email protected]>
2018-09-19 10:37:45 -04:00
RT df6acf0c2a tokio-timer: reset timeout after elapsed in stream (#648) 2018-09-19 09:47:39 -04:00
Eliza Weisman 98d23b8b29 Make tokio::run panic if called from inside tokio::run (#646)
This is implemented by creating an `Enter` instance from within `run`.

This patch also introduces `Enter::block_on`.

Fixes #504
2018-09-18 21:57:21 -07:00
Eliza Weisman 85f8522536 tokio-executor: hide deprecated tokio-threadpool reexports (#644)
Fixes: #643
Signed-off-by: Eliza Weisman <[email protected]>
2018-09-18 23:57:17 -04:00
Liran Ringel d275341fb2 Fix tokio-async-await tests compile errors (#630) 2018-09-18 13:49:08 -07:00
Nick Cameron d735e5d527 async-await: update deps in tokio-async-await (#639) 2018-09-18 10:08:35 -07:00
Carl Lerche 4019198706 Add some missing future::Executor implementations (#563)
This adds an implementation of future::Executor for
`executor::DefaultExecutor` and `runtime::current_thread::Handle`.
2018-09-17 22:23:48 -07:00
Ivan Petkov 24dc85dc5e ci: Run cargo test with the --no-fail-fast flag (#635)
Since the CI runs all tests for all tokio crates, it is possible that a
sporadic failure in one crate can mask failures/successes of other
crates' tests.

Using the `--no-fail-fast` flag instructs cargo to run *all* tests
before failing the build. This will allow checking to see if any
relevant test cases still pass even if an unrelated test has failed.
2018-09-15 00:41:41 +00:00
Ivan Petkov aaa5adb7fd Merge pull request #634 from vorner/import-signal-2
Import the `tokio-signal` source from its original repo

Original repository can be found at https://github.com/alexcrichton/tokio-signal
2018-09-14 22:58:19 +00:00
Michal 'vorner' Vaner 5f68b3aaa1 signal: Remove Apache license
Whole tokio is MIT only, unifying.
2018-09-14 23:28:56 +02:00
Michal 'vorner' Vaner 2f69acbe9f signal: Fix tests after importing & linking
* Don't use tokio-core any more for tests. That one brings tokio from
  crates.io instead of the current workspace and two versions of that
  don't want to cooperate.
* Guard unix-specific examples on windows.
* Leave CI setup to top-level directory.
2018-09-14 23:28:47 +02:00
Michal 'vorner' Vaner 7e12f5c39e signal: Link tokio-signal and tokio crates
References in the Cargo.toml, various links.
2018-09-14 23:28:33 +02:00
Michal 'vorner' Vaner 462882b356 Merge tokio with tokio-signal 2018-09-14 23:27:22 +02:00
Michal 'vorner' Vaner 35687f1d18 signal: Move to tokio-signal subdirectory
As a preparation to merge with tokio.
2018-09-14 23:25:57 +02:00
Michal 'vorner' Vaner e7dc3a1091 signal: Use signal-hook for registration of signals
This saves some code and gets rid of quite some amount of unsafe code.
2018-09-14 23:25:21 +02:00
Ivan Petkov b594e240f9 signal: Bump version to 0.2.5 2018-09-14 23:25:21 +02:00
Ivan Petkov 605708dca6 signal: Fix a possible starvation with concurrent Signal polls
* Originally reported in alexcrichton/tokio-process#42
* The root cause appears to be due to two different PollEvented
instances trying to consume readiness events from the same file
descriptor.
* Previously we would simply swallow any `AlreadyExists` errors when
attempting to register the pipe receiver with the event loop. I'm not
sure if this means the PollEvented wrapper wasn't fully registered to
receive events, or maybe there is a potential race condition with how
PollEvented consumes mio readiness events. Using a fresh/duplicate file
descriptor appears to mitigate the issue, however.
* I was also not able to reproduce the issue as an isolated test case so
there is no regression test available within this crate (but we can add
one in tokio-process)
2018-09-14 23:25:08 +02:00
Carl Lerche cc40a4e7f0 Revert "Add max line length to LinesCodec (#590)"
This reverts commit 4ae6c997ee.
2018-09-12 10:34:37 -07:00
Eliza Weisman 4ae6c997ee Add max line length to LinesCodec (#590)
* codec: add new constructor `with_max_length ` to `LinesCodec`
* codec: add security note to docs

Signed-off-by: Eliza Weisman <[email protected]>

* Fix Rust 1.25 compatibility

* codec: Fix incorrect line lengths in tests (and add assertions)

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix off-by-one error in lines codec

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix call to decode rather than decode_eof in test

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix incorrect LinesCodec::decode_max_line_length

This bug was introduced after the fix for the off-by-one error.
Fortunately, the doctests caught it.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Minor style improvements

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Don't allow LinesCodec length limit to be set after construction

Signed-off-by: Eliza Weisman <[email protected]>

* codec: change LinesCodec to error and discard line when at max length

* codec: Fix build on Rust 1.25

The slice patterns syntax wasn't supported yet in that release.

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for out-of-bounds index when peeking

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Fix out of bounds index

* codec: Fix incomplete comment

Signed-off-by: Eliza Weisman <[email protected]>

* codec: Add test for line decoder buffer underrun
2018-09-12 13:02:57 -04:00
Stjepan Glavina 0f44adf5f6 reactor: use LocalKey::try_with in sharded RW lock (#628)
@jonhoo reported a panic in the call to `LocalKey::with`, which occurs
when the reactor is dropped in the middle of TLS teardown. This PR
changes the call to `LocalKey::try_with` and handles the case when the
thread-local value has already been destroyed.
2018-09-11 16:48:06 -04:00
Flux Xu 19d5565442 Expose thread_pool::SpawnHandle (#604) 2018-09-10 15:51:15 -07:00
Ivan Petkov 98e76d9bc6 signal: Bump version to to 0.2.4 2018-09-10 11:30:08 -07:00
Alan Somers b7f5bc95fe signal: Actually make unix::bsd public 2018-09-10 11:30:07 -07:00
Ivan Petkov 90ea2f6c5b signal: Bump version to 0.2.3 2018-09-10 11:30:07 -07:00
Ivan Petkov 837c3934d5 signal: Also cfg gate the entire unix::bsd module 2018-09-10 11:30:07 -07:00
Alan Somers bcd42d11d9 signal: Move SIGINFO to a BSD-specific submodule 2018-09-10 11:30:07 -07:00
Alan Somers 8ad66d296f signal: Add CHANGELOG entry for SIGINFO. 2018-09-10 11:30:07 -07:00
Alan Somers d9edc26e97 signal: export SIGINFO on supported platforms. 2018-09-10 11:30:07 -07:00
Alan Somers f0ac62151b signal: Update tokio dependency
tokio::runtime::current_thread was added in 0.1.6
2018-09-10 11:30:06 -07:00
Ivan Petkov 214722a296 signal: Update CHANGELOG 2018-09-10 11:30:06 -07:00
Niv Kaminer b8f8145b62 signal: cast SIGINFO to be the same type as sa_flags regardless of platform 2018-09-10 11:30:06 -07:00
Ivan Petkov 32d3e0e1f9 signal: Bump version to 0.2.2 2018-09-10 11:30:06 -07:00
Ivan Petkov 266919add6 signal: Refactor Signal tests
* Added timeouts to all tests that were missing them
 - any issue we have will likely result in deadlocks/starvation so its
   best if all tests quickly timeout rather than require getting killed
   or have the CI timeout itself
* Added a `support` module and put a bunch of helpers there to DRY the
tests
2018-09-10 11:30:06 -07:00
Ivan Petkov c9ffd98b1e signal: Fix a potential Signal starvation based on creation order
* As observed in alexcrichton/tokio-signal#38, Signal instances can starve based on the order
they are created in, and this ordering appears to be platform/OS
specific
* The crux of the issue is that we woud only *attempt* to broadcast any
pending signals if we successfully read out at least one byte from the
global pipe.
* For reasons unclear to me, the affected Signal instance would get
woken up after the signal handler writes to the global pipe, but it
would immediately hit a WouldBlock error and give up, bypassing the
broadcast attempt (even though the pending flag was correctly set).
 - Maybe this has to do with OS specifics with how the bytes are
   delivered (or not), or with some complex interaction with tokio and
   the pipe registration. It seems fishy since strace logs didn't show
   the signal handler pipe write fail either, but I'm all out of ideas
* The fix appears simple: unconditionally attempt to broadcast any
pending signals *any* time a Driver instance is woken up.
* Since we perform an atomic check for each pending signal, we know that
each (coalesced) signal broadcast will happen at most once. If we were
supuriously woken up and no signals were pending, then nothing will be
yielded to any pollers of Signal
* The down side is that since each Signal instance polls a Driver
instance, each poll to Signal will essentially perform N atomic
operations (N = number of signals we support) in an attempt to broadcast
any pending signals.
 - However, we can revisit optimizing this better in the future

Fixes alexcrichton/tokio-signal#38
2018-09-10 11:30:06 -07:00
Ivan Petkov 2d4bfa1485 signal: Fix starvation of signal streams on drop of another instance
* We introduce a new global structure which keeps track of how many
signal streams have been registered with a given event loop (the event
loop is identified by its OS file descriptor)
* We only attempt to deregister our global evented pipe from any event
loop if and only if we are the last signal that was registered with it
2018-09-10 11:30:05 -07:00
Ivan Petkov b6ecfa251c signal: Add (failing) test case which exibits starvation on drop
* Currently, whenever a new signal stream is created we attempt to
register a global pipe with the event loop to drive events.
* We also (correctly) swallow any descriptor-already-registered errors
since the same pipe is always used
* However, we currently *deregister* the same global pipe *any time* a
Signal stream is dropped.
* This means that if 2 or more of Signal instances exist simultaneously
(even if listening for different signals) and one of them is dropped,
the remainder will starve (until any new signal is created again).
2018-09-10 11:30:05 -07:00
Ivan Petkov 3a81d7746a signal: Split up all integration tests to run in their own process
* Cargo runs each integration-style-test in its own process. Since the
tests use global data structures specific to the process, we should run
them in an isolated manner to avoid having cross-test interactions
* Fixes alexcrichton/tokio-signal#39
2018-09-10 11:30:05 -07:00
Daniel Wagner-Hall 6e1a833825 signal: Update mio dependency to 0.6.14
This allows tokio-signal to build with `-Z minimal-versions` - see
https://github.com/rust-lang/cargo/issues/5657#issuecomment-401110172
for more details.

Earlier versions depend on log 0.3.1, which itself depends on libc
0.1, which doesn't build on any post-1.0 version of rust.
2018-09-10 11:30:05 -07:00
Niv Kaminer 3f80953dee signal: account for definition mismatch on aarch64 android 2018-09-10 11:30:05 -07:00
Michael Hadley 4374f5be70 signal: Fix typo in README.md 2018-09-10 11:30:05 -07:00
jjl b23ab94cd5 signal: Change reference from 'tokio-core' to 'tokio' in README 2018-09-10 11:30:04 -07:00
Markus Westerlind 40c77bd17e signal: Version 0.2 2018-09-10 11:30:04 -07:00
Markus Westerlind 31b51004f2 signal: Increase number of signals to 33 for the sake of FreeBSD
Fixes alexcrichton/tokio-signal#21
2018-09-10 11:30:04 -07:00
Markus Westerlind 9a4e4f2308 signal: Don't use the depreceated new method of Registration 2018-09-10 11:30:04 -07:00
Markus Westerlind 484fda7a23 signal: Add an appveyor build file
It is not possible to test much on windows but this will at least verify
that it can be built
2018-09-10 11:30:04 -07:00
Markus Westerlind 45ba6e2652 signal: Remove test that were accidentally included in the ctrl-c example 2018-09-10 11:30:04 -07:00
Markus Westerlind e73b8a0cc9 signal: refactor: Prefer the implicit handle passing used by tokio 2018-09-10 11:30:03 -07:00
Markus Westerlind 209232befd signal: Ensure that the driver dies once the signal does
`tokio::run` expects that all futures finish processing so we can't
leave `Driver` around forever or `tokio::run` would never return.
2018-09-10 11:30:03 -07:00
Markus Westerlind f759e4d70f signal: panic 2018-09-10 11:30:03 -07:00
Markus Westerlind 1fdff707b8 signal: test: Add a test for ctrl_c on unix 2018-09-10 11:30:03 -07:00
Markus Westerlind e97e8cb7fe signal: Update the windows implementation to work with tokio
BREAKING CHANGE

`ctrl_c` now takes a `tokio_reactor::Handle`
2018-09-10 11:30:03 -07:00
Markus Westerlind 2848df9b6c signal: Run rustfmt 0.4.2 2018-09-10 11:30:03 -07:00
Alex Crichton 7a24ed7509 signal: Bump to 0.1.5 2018-09-10 11:30:02 -07:00
Alex Crichton 9c9760cfbb signal: Fix a bug with most recent tokio-core release 2018-09-10 11:30:02 -07:00
Alex Crichton 5ecd929b1a signal: Bump to 0.1.4 2018-09-10 11:30:02 -07:00
Alex Crichton 8ddebf4309 signal: Fix compile on Android
Closes alexcrichton/tokio-signal#19
2018-09-10 11:30:02 -07:00
Alex Crichton 0df1882f21 signal: Bump to 0.1.3 2018-09-10 11:30:02 -07:00
Alex Crichton 4fa1b2b58c signal: Update to winapi 0.3 2018-09-10 11:30:02 -07:00
Alex Crichton a4895fe364 signal: Tweak travis config 2018-09-10 11:30:02 -07:00
Alex Crichton 7ab97f99c8 signal: Clarify wording of license information in README.
This text historically was copied verbatim from rust-lang/rust's own README [1]
with the intention of licensing projects the same as rustc's own license, namely
a dual MIT/Apache-2.0 license. The clause about "various BSD-like licenses"
isn't actually correct for almost all projects other than rust-lang/rust and
the wording around "both" was slightly ambiguous.

This commit updates the wording to match more precisely what's in the
standard library [2], namely clarifying that there aren't any BSD-like licenses
in this repository and that the source is licensable under either license, at
your own discretion.

[1]: https://github.com/rust-lang/rust/tree/f0fe716dbcbf2363ab8f929325d32a17e51039d0#license
[2]: https://github.com/rust-lang/rust/blob/f0fe716dbcbf2363ab8f929325d32a17e51039d0/src/libstd/lib.rs#L5-L9
2018-09-10 11:30:01 -07:00
Alex Crichton 9bf3228f73 signal: Add an example for waiting on two signals
Relies on `Stream::select` to merge streams.

Closes alexcrichton/tokio-signal#16
2018-09-10 11:30:01 -07:00
Raphael Nestler d41c60e21d signal: Fix typo in README 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 934c596133 signal: Ctrl+C example: quit after 10 signals. 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 72e2209bd8 signal: Ctrl+C example: more explanations 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 6a7092b9f7 signal: Ctrl+C example: Defer stream initialisation (and explain how/why) 2018-09-10 11:30:01 -07:00
Jules Kerssemakers 1c893ef6d3 signal: undo nested example cargo project
.. after learning about `cargo run --example`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 20e7598e8d signal: Ctrl+C example: Don't forget proper attribution for original example 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 3db92496f6 signal: Ctrl+C example: highlight power of Stream::for_each() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers da47cfbd58 signal: Ctrl+C example: clarify control flow after receiving Ctrl+C: unreachable!() 2018-09-10 11:30:00 -07:00
Jules Kerssemakers f5eadc74f1 signal: Ctrl+C example: add explanatory comments 2018-09-10 11:30:00 -07:00
Jules Kerssemakers 175f9afea9 signal: ctrl+C example: Prompt user to do something.
So we don't stay at a blank terminal without any feedback after `cargo run`
2018-09-10 11:30:00 -07:00
Jules Kerssemakers 48eda3fe2f signal: Upgrade ctrl+c example into cargo run-able version with proper Cargo.toml 2018-09-10 11:29:59 -07:00
Alex Crichton 010c2223ca signal: Touch up the sighup-example slightly 2018-09-10 11:29:59 -07:00
Jules Kerssemakers 36b58d8fa8 signal: new example: SIGHUP, shows how to receive other signals than ctrl+C 2018-09-10 11:29:59 -07:00
Alex Crichton c601f68c9f signal: Add some examples to crate docs
Closes alexcrichton/tokio-signal#11
2018-09-10 11:29:59 -07:00
Alex Crichton 78ca103f3a signal: Update to tokio-io 2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner 7da00f3832 signal: Use IDs that don't run out
Replace the sequential counting (which might be exhausted) by an address
of an object (in a box, so it doesn't change). This is also a unique, so
it is acceptable ID.
2018-09-10 11:29:59 -07:00
Michal 'vorner' Vaner edba77e8df signal: A test running multiple event loops
Run multiple loops (both in parallel and sequentially) to make sure
broadcasting to multiple of them works and we work even after the
initial loop has gone away.
2018-09-10 11:29:58 -07:00
Michal 'vorner' Vaner cf1afd2d90 signal: Style: Replace tabs with spaces
Mixing tabs and spaces breaks indentation for people (and github) if
they use different tab width.
2018-09-10 11:29:58 -07:00
Alex Crichton 8e58a9d8d4 signal: Add badges/categories 2018-09-10 11:29:58 -07:00
Alex Crichton 4afef9391a signal: Bump to 0.1.2 2018-09-10 11:29:58 -07:00
Alex Crichton b6bacc1ca3 signal: Clarify a comment 2018-09-10 11:29:58 -07:00
Alex Crichton 955cd2836d signal: Clear out old Signal on drop 2018-09-10 11:29:58 -07:00
Alex Crichton 70e4ed67ad signal: Touch up more impls and comments 2018-09-10 11:29:58 -07:00
Alex Crichton 699b9ab89e signal: Handle a few more errors 2018-09-10 11:29:57 -07:00
Alex Crichton ba0921a01d signal: Various cleanups:
* Drop nix/lazy_static
* Use previously registered handlers
* Handle some more errors
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 367cb56e02 signal: The driver task
Add the driver task, connecting the signal handler wakeups to the
wakeups of of the streams.

It is a prototype-quality code, a lot of cleanups and similar is needed.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 04d949c380 signal: Provide the new kind of Signal stream
Which is just a wrapper around the futures::sync::mpsc. The sender is in
a global registry.

The part that connects the wakeups to the senders in the registry
doesn't yet exist.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner ed4359bb26 signal: Implement the wake-up part of the new signal handling
Register the signal handler that wakes up someone through a self-pipe.
That someone doesn't yet exist, though.

Some dependencies (nix, lazy_static) added to speed up the prototyping
process. They are likely to be dropped in some future commits.

Some features (eg. preserving the previous signal handlers) are still
missing.
2018-09-10 11:29:57 -07:00
Michal 'vorner' Vaner 4142dc2fae signal: Use tokio-core from git
Just for now, as we need some yet unreleased features.
2018-09-10 11:29:57 -07:00
Alex Crichton 468b037e4e signal: Update docs urls and such 2018-09-10 11:29:56 -07:00
Alex Crichton b33ae3cdd6 signal: Remove deprecated API usage on Windows 2018-09-10 11:29:56 -07:00
Alex Crichton 4519ac8e17 signal: Remove use of deprecated APIs on Unix 2018-09-10 11:29:56 -07:00
Alex Crichton 92b93ee176 signal: Bump to 0.1.1 2018-09-10 11:29:56 -07:00
Alex Crichton 635149e3ab signal: Ignore errors in signal handler
Closes alexcrichton/tokio-signal#3
2018-09-10 11:29:56 -07:00
Alex Crichton e28c350e31 signal: Update travis token 2018-09-10 11:29:56 -07:00
Chris Emerson 1a122018a2 signal: Trivial typo fix. 2018-09-10 11:29:55 -07:00
Alex Crichton f3f8ee431e signal: Remove SIGKILL reexport 2018-09-10 11:29:55 -07:00
Alex Crichton 61c4047c6a signal: Add symbolic reexports for common signals
Means you don't have to import libc!

Closes alexcrichton/tokio-signal#1
2018-09-10 11:29:55 -07:00
Alex Crichton 3486a61a0f signal: Update deps to point to crates.io 2018-09-10 11:29:55 -07:00
Alex Crichton 8291c3d462 signal: Start adding windows support 2018-09-10 11:29:55 -07:00
Alex Crichton 1b6893b6f6 signal: Track tokio-core master 2018-09-10 11:29:55 -07:00
Alex Crichton eca7f0760f signal: Initial commit 2018-09-10 11:29:50 -07:00
Ben Boeckel 89d969d518 StreamExt: add a trait for additional Stream methods (#573)
Primarily, it offers a `timeout` method for streams.
2018-09-07 15:43:03 -07:00
Carl Lerche 16664189c1 async-await: move examples into dedicated crate (#608)
This works around a bug in the cargo renaming feature as well as allows
the use of `[patch]` in the `Cargo.toml`.
2018-09-06 13:36:59 -04:00
Carl Lerche 6828870608 async-await: bump version to v0.1.2 (#619) 2018-09-04 15:02:15 -07:00
Nimi Wariboko Jr 89d0cda2e2 async-await: use new PinMut/PinBox location (#613)
The types moved in `std`. This patch updates tokio-async-await to
import `PinMut` and `PinBox` from the new location.

Ref: rust-lang/rust#53227
2018-09-04 13:19:10 -07:00
Stjepan Glavina 8052a9b348 guide: fix a few typos (#612) 2018-09-03 10:18:53 -07:00
ksqsf a5ac6c8b72 Fix undesired multi-line error message (#605) 2018-08-31 09:53:41 -07:00
Léo Gaspard 3a59526523 Document that timeout-ed futures will be polled at least once (#603)
tokio-util: document behavior of `StreamExt::timeout` when timeout = 0
2018-08-31 09:26:41 -04:00
Zachary Stewart 322a94f72f Update documentation for AsyncRead and AsyncWrite (#596)
tokio-io: update documentation for AsyncRead and AsyncWrite
2018-08-31 09:00:32 -04:00
Eunchong Yu c03b23355b Fix minimum version to export tokio::codec module (#594)
tokio-async-await: fix minimum version to export tokio::codec module (#594)
2018-08-31 08:02:02 -04:00
Eliza Weisman bc91bc5022 Fix non-terminating loop in tokio_io::length_delimited::FramedWrite (#576)
* tokio-io: fix non-terminating loop in length_delimited::FramedWrite (#497)
2018-08-31 06:31:43 -04:00
Flux Xu a7b053372f Add ThreadPool::spawn_handle (#602)
## Motivation

`tokio_threadpool::ThreadPool::spawn` has no return value.

## Solution

Add `ThreadPool::spawn_handle` which calls
`futures::sync::oneshot::spawn` to return a future represents the return
value.
2018-08-30 16:53:05 -07:00
Eliza Weisman 673fdb5cb3 Refactor codec::length_delimited (#575)
This patch refactors `length_delimited` to be implemented as a `Codec` and
use the default `Framed` wrapper types.

The original implementation did not do this in order to support vectored writes in the
write half. However, this implementation would be more efficient with small frames anyway.

If vectored writes are to be explored in the future, then it should be done holistically.

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-30 14:50:32 -07:00
Carl Lerche 97618746de readme: fix section ordering (#600) 2018-08-30 14:46:40 -07:00
Carl Lerche d8f8b59df9 guide: add a testing section to the contributing guide (#598) 2018-08-30 12:26:24 -07:00
Jon Gjengset 0745a9b88a Use spawn_local to spawn from local Handles (#565)
Previously, every call to `current_thread::Handle::spawn` would go
through a `mpsc` channel. This is unnecessary when the `Handle` is still
on the same thread as the current thread executor. This patch fixes that
by storing the `ThreadId` of the executor when it is created, and then
comparing against that when `Handle::spawn` is called. If the call is
made from the same thread, `spawn_local` is used directly.

Fixes #562.
2018-08-30 11:24:59 -07:00
Eliza Weisman a7f5ba28ba Bump minimum supported version & document support policy (#599)
* Bump minimum supported version & document support policy

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-29 20:35:27 -04:00
Josef Brandl cc3b6af7a3 Fix tokio-uds version (#580) 2018-08-28 15:12:45 -07:00
Martin Chaine 07e30ae923 net: rework tokio_tcp and tokio_udp re-exports (#548)
This patch keeps the primary net types in `tokio::net` and moves
secondary types to a protocol specific submodules.

Primary types are the ones that users are most likely to name (`TcpStream`,
`TcpListener`, `UdpSocket`, ...)

Secondary types are the operation futures.
2018-08-28 14:13:06 -07:00
Jason Davies 69d90ac7ee Fix a few typos in timer docs. (#569) 2018-08-28 11:00:42 -07:00
Carl Lerche d16032cf06 async-await: misc fixes and typos (#585) 2018-08-27 15:16:32 -07:00
Carl Lerche b479ce78d3 add experimental async/await support. (#582)
This patch adds experimental async/await support to Tokio. It does this
by adding feature flags to existing libs only where necessary in order
to add nightly specific code (mostly `Unpin` implementations). It then
provides a new crate: `tokio-async-await` which is a shim layer on top
of `tokio`.

The `tokio-async-await` crate is expected to look exactly like `tokio`
does, but with async / await support. This strategy reduces the amount
of cfg guarding in the main libraries.

This patch also adds `tokio-channel`, which is copied from futures-rs
0.1 and adds the necessary `Unpin` implementations. In general, futures
0.1 is mostly unmaintained, so it will make sense for Tokio to take over
maintainership of key components regardless of async / await support.
2018-08-27 12:24:51 -07:00
Michal 'vorner' Vaner 6e45e0ac61 re-export tokio-current-thread::spawn (#579)
Re-export it inside the tokio::runtime::current_thread, as the original
place (tokio::executor::current_thread) is hidden from documentation and
users need some way to spawn non-Send futures.
2018-08-25 12:51:49 -07:00
Ben Boeckel 82c5baa09b Spelling fixes (#571)
* docs: fix spelling and whitespace errors
2018-08-25 15:26:41 -04:00
Carl Lerche 7dc6404726 draft initial CONTRIBUTING guide (#567)
This guide was adopted from the node.js project.
2018-08-24 13:03:34 -07:00
Eliza Weisman 2e88e29fe9 Move tokio_io::codec::length_delimited module to tokio::codec (#568)
* Deprecate tokio-io::length_delimited
* Move `length_delimited` into `tokio::codec`

Signed-off-by: Eliza Weisman <[email protected]>
2018-08-24 15:54:42 -04:00
Carl Lerche 07203408de Bump version to v0.1.8 (#566)
This also bumps a number of sub crates:

* tokio-executor (0.1.3)
* tokio-io (0.1.8)
* tokio-reactor (0.1.4)
* tokio-threadpool (0.1.6)
* tokio-timer (0.2.6)
* tokio-udp (0.1.2)
2018-08-24 08:58:26 -07:00
Carl Lerche 8bf2e9aeb0 Introduce Timeout and deprecate Deadline. (#558)
This patch introduces `Timeout`. This new type allows setting a timeout
both using a duration and an instant. Given this overlap with
`Deadline`, `Deadline` is deprecated.

In addition to supporting future timeouts, the `Timeout` combinator is
able to provide timeout functionality to streams. It does this by
applying a duration based timeout to each item being yielded.

The main reason for introducing `Timeout` is that a deadline approach
does not work with streams. Since `Timeout` needed to be introduced
anyway, keeping `Deadline` around does not make sense.
2018-08-22 20:39:46 -07:00
Carl Lerche cf184eb326 timer: Reduce size of Delay struct (#554)
* Remove `counted` field on `timer::Entry`.

It turns out that a better indicator of whether or not the number of
active timeouts should be decremented is if the `Entry` has been
associated with a timer. In other words, if `Entry::inner` can be
upgraded, then the count should be decremented on drop.

* timer: Tweak link between `Delay` and the driver

This tweaks the struct layout / details regarding how a `Delay` instance
is linked to a driver (timer instance). Instead of lazily allocating the
`Entry` (node shared between `Delay` and the timer), `Entry` is
allocated immediately when `Delay` is created. This allows using the
entry store data used by `Delay`.

This is in anticipation of further timer improvements that would
otherwise require the size of `Delay` to grow further. Since an
allocation is already made, the idea is to shrink the size of the
`Delay` struct.
2018-08-21 21:48:40 -07:00
Carl Lerche d822b721b4 Add DelayQueue implementation to tokio-timer (#550)
This patch adds a `DelayQueue` to tokio_timer. The `DelayQueue` allows
inserting elements as well as specifying a time at which the element
should be returned to the user. This allows handling more complex
timeout situations.
2018-08-20 21:47:10 -07:00
Carl Lerche c66b56c3fb Implement Default for tokio_timer::Handle (#553)
This patch implements `Default` for `tokio_timer::Handle`. It returns a
`Handle` instance that is not bound to a specific timer. Instead, it
will use the timer for the current execution context. This is the same
strategy used by `tokio_reactor::Handle`.

Fixes #547
2018-08-20 13:01:39 -07:00
Carl Lerche 89639ec48b Bump tokio-uds version to v0.2.1 (#552)
Fixes #551
2018-08-19 21:23:49 -07:00
Martin Chaine 2b1b0ac858 Expose tokio_uds from the root crate (#526) 2018-08-15 21:26:10 -07:00
Carl Lerche 6b84c73f12 tokio::codec docs + additional exports (#546) 2018-08-15 21:25:25 -07:00
Jason Ish 767b370c21 Add tokio-tls echo example. (#541)
Based on the current example on the front page of tokio.rs.
2018-08-15 07:54:24 -04:00
Gary M. Josack 28010b5962 Update lines_encoder test to use LinesCodec (#544)
The `lines_encoder` test is a copy/paste of `bytes_encoder` and not
testing the LinesCodec encoding at all. This updates the test to do
simple validations of the LinesCodec encoding.
2018-08-15 07:51:42 -04:00
Roman 2e343f9e42 Reexport Encoder, Decoder, Framed* from tokio::codec (#499) 2018-08-14 11:18:54 -07:00
Mateusz Mikuła 31f71dedee Routine dependencies update (#533)
* Update dependencies

* Replace deprecated tempdir with tempfile
2018-08-10 12:37:45 -07:00
Stjepan Glavina 989262fe6e Enable sanitizer tests for tokio-threadpool (#537)
Closes #536.
2018-08-10 19:13:51 +02:00
Carl Lerche d91c775f36 Remove dead futures2 code. (#538)
The futures 0.2 crate is not intended for widespread usage. Also, the
futures team is exploring the compat shim route.

If futures 0.3 support is added to Tokio 0.1, then a different
integration route will be explored, making the current code unhelpful.
2018-08-09 21:56:53 -07:00
Stjepan Glavina 96b556fbff Steal multiple tasks from another worker at a time (#534)
* Steal multiple tasks from another worker at a time
* Better spinning and failing pop
* Update crossbeam-deque and simplify spinning
2018-08-09 12:14:13 -07:00
Stjepan Glavina fd36054ae4 Use a scalable RW lock in tokio-reactor (#517) 2018-08-09 11:23:45 -07:00
Carl Lerche 89d6bfc5cb Bump tokio-tls to v0.2.0 (#531)
This prepares the crate for release.
2018-08-08 10:17:28 -07:00
David Kellum decc83e959 Update to crossbeam-utils 0.5.0, fix imports (#519) 2018-08-08 08:57:25 -07:00
Sean McArthur afcfefd7e3 Move tokio-tls into workspace (#529) 2018-08-08 08:36:17 -07:00
Roman c89b0b4c8c Fix num CPUs in threadpool::builder::Builder::new (#530)
Closes #400
2018-08-08 08:33:01 -07:00
Stjepan Glavina 6b1e4ab0a3 Implement Error for a few error types (#511) 2018-08-07 19:45:58 -07:00
David Kellum 4153cc4076 Fix more rustdoc links (#518)
* Fix a cut-paste error with -reactor rustdoc links
* Fix more ::reactor rustdoc broken links
* Minor rustdoc typo
* Consistently reference std::io::{Read, Write} in rustdoc/links
2018-08-07 19:45:39 -07:00
Serho Liu 5304557d1d Fix tokio threadpool readme examples (#521) 2018-08-07 19:44:45 -07:00
Andrew Cann fdb2f61357 Udp socket readiness methods (#522) 2018-08-07 19:41:27 -07:00
Carl Lerche e964c4136c Bump subcrate versions (#524)
* tokio-current-thread 0.1.1
* tokio-executor 0.1.3
* tokio-fs 0.1.3
* tokio-reactor 0.1.3
* tokio-tcp 0.1.1
* tokio-timer 0.2.5
2018-08-06 20:36:50 -07:00
Brian Olsen 0490280d66 tokio-fs: Add async versions of most of std::fs (#494)
* create_dir
* create_dir_all
* hard_link
* read_dir
* read_link
* remove_dir
* remove_file
* rename
* set_permissions that works with path
* symlink_metadata
* symlink on unix
* symlink_dir on windows
* symlink_file on windows
2018-07-31 21:39:27 -07:00
Sam Rijs 0f76470172 detect and handle recursive calls to DefaultExecutor (#473) 2018-07-30 20:59:08 -07:00
Stjepan Glavina 9352249c3e Terminate backup threads when idle (#489) 2018-07-30 20:48:53 -07:00
Stjepan Glavina e5b2681513 Fix a race in thread wakeup (#507) 2018-07-30 20:46:46 -07:00
Stjepan Glavina 629c9f0698 Small fixes (#508)
* Make Shutdown public
* Remove unused import
* Fix documentation mistake
* Fix typo
2018-07-30 20:46:04 -07:00
Alan Somers 5d0d2a2e12 Ignore tokio-uds's test_socket_pair on FreeBSD. (#493)
It requires FreeBSD 12.0 or later.  Also, fix a spelling mistake in a
comment.
2018-07-24 14:00:01 -07:00
kohensu ad4693a18f Fix the doc of read_to_end method (#482) 2018-07-24 13:57:15 -07:00
Laurențiu Nicola c85bde3170 tokio: expose tokio_fs::metadata (#479) 2018-07-24 13:56:42 -07:00
Jon Gjengset 1e90e27720 Count in-transit spawned futures to current thread executor as pending (#478) 2018-07-24 13:49:01 -07:00
Carl Lerche f212a2ab9d Fix Weak tsan whitelist (#505) 2018-07-24 13:37:48 -07:00
Michal 'vorner' Vaner 84db325628 RunError and few more error types implements Error (#501)
This allows them to be used with things like `failure`.
2018-07-24 13:27:57 -07:00
Douman 365efec24a Add Interval::interval shortcut for a better usability (#492) 2018-07-23 23:08:49 -07:00
David Kellum 491f15827b General rustdoc improvements (#450)
* Normalize links to docs.rs/CRATE/M.N/...

docs.rs is smart enough to show docs for the latest M.N.P release when
M.N is used in the link. For example:

  https://docs.rs/mio/0.6/mio/struct.Poll.html

..will show mio 0.6.14 and later docs. While using the `M.N.*`
(ASTERISK) syntax also works, `M.N` is the more common usage, so
standarize a few existing links to that format.

* Fix missing or malformed rustdoc links

* executor lib rustdoc minor format change

* Promote tokio-threadpool crate level comments to rustdoc

* Replace hidden tokio::executor::thread_pool docs with deprecation note

* Fix typo/simplify util module rustdoc

* Reuse some tokio::executor::thread_pool rustdoc for the crate

Relates to #421
2018-07-22 13:35:30 -07:00
Stjepan Glavina c17ecb53e7 Pad fields to cacheline size to avoid false sharing (#475) 2018-07-16 14:22:48 -07:00
Jon Gjengset 6ba8e7621d Add free block_on_all in current thread Runtime (#477) 2018-07-11 15:32:58 -07:00
Laurențiu Nicola 39c95d6206 tokio-fs: Bump version to 0.1.2 (#469)
* Add a couple of missing full stops in the documentation
2018-07-11 15:09:37 -07:00
Sam Rijs 78b6bd4ca5 implement Send and Sync for DefaultExecutor (#472)
Fxes #376
2018-07-11 12:35:32 -07:00
Richard Dodd (dodj) b3ff9e315c Update lib.rs (#471)
Fix build failure on nightly (combination of warning for "cannot be resolved" and lint deny(warnings))
2018-07-11 12:30:51 -07:00
Stjepan Glavina 990186ec9d Optimize spinning in Worker::run (#470) 2018-07-11 12:30:26 -07:00
Stjepan Glavina 19da6ff59a New version of crossbeam-deque (#468) 2018-07-11 12:24:10 -07:00
Roman 36c817f0c3 Update rand dep from 0.4 to 0.5 (#458) 2018-07-11 12:14:40 -07:00
David Kellum 35123f7ae4 Additional details for tokio-fs rustdoc (#454) 2018-07-11 12:13:16 -07:00
João Oliveira 54b7c1b10d tokio-tcp: add tokio::net::TcpStream::try_clone (#448) 2018-07-11 11:54:08 -07:00
Patrick Barrett e6fc3d209d return NotReady when recv_from wouldblock in uds (#452) 2018-07-11 11:37:57 -07:00
Carl Lerche f98b81e527 Bump minimum supported Rust to 1.25. (#465)
Currenty, 1.27 is the latest released Rust version.
2018-07-06 14:14:05 -07:00
Stjepan Glavina dc7202cfa9 Replace XorShiftRng with a custom RNG (#466) 2018-07-06 13:33:52 -07:00
Carl Lerche f1a7caea3f Bump tokio-threadpool to v0.1.5 (#462) 2018-07-05 10:22:03 -07:00
Stjepan Glavina b019532bc2 Implement status() for DefaultExecutor (#463) 2018-07-05 10:19:49 -07:00
Stjepan Glavina 7fb579c667 Fix a race in thread wakeup (#459) 2018-07-03 16:28:50 -07:00
Stjepan Glavina dbefa67058 Make WorkerId public (#460) 2018-07-02 13:34:08 -07:00
Roman 24d99c029e Add a verbose error message for BlockingError (#451)
Add a verbose error message for EnterError while trying to run
tokio_threadpool::blocking on a current_thread::Runtime
2018-06-26 08:37:48 -07:00
Laurențiu Nicola 3fecd0154c Add an explicit wait for the test to finish (#445) 2018-06-22 14:07:02 -07:00
Laurențiu Nicola 0440343a11 tokio-fs: add changelog for 0.1.2 (#444) 2018-06-22 14:06:50 -07:00
Roman 3cf56b7bfa Fix unneeded mut and some deprecated api (#442) 2018-06-21 09:47:21 -07:00
Roman 7153d8d6ce Add a verbose error message for EnterError (#441)
Add a verbose error message for EnterError while trying to run an
executor while another executor is already running.

Fixes: #410
2018-06-21 09:46:45 -07:00
Laurențiu Nicola ecfe2f6a05 tokio-fs: add tokio_fs::File::seek (#434) 2018-06-21 09:43:35 -07:00
Laurențiu Nicola 5753553ba3 Move metadata to a submodule (#439) 2018-06-21 09:41:38 -07:00
Laurențiu Nicola 04a4bfd455 tokio-fs: add tokio_fs::metadata (#433) 2018-06-20 13:12:06 -07:00
Jake Goulding b2f77dcebe Add a dedicated Future for retrieving the metadata of a file (#385) 2018-06-18 16:00:43 -07:00
Carl Lerche 85cf47de86 Enable backtraces in CI & disable TSAN (#436)
This PR enables backtraces when running tests and disables tsan for the thread pool.

The thread sanitizer was generating too many false positives. Once #329 lands, then it can
be re-enabled.
2018-06-18 15:15:45 -07:00
Steven Fackler 45bcea6c4f Reexport tokio_uds::ConnectFuture (#430) 2018-06-18 13:26:06 -07:00
Carl Lerche 3fac7ce68c Add some thread pool docs (#421) 2018-06-15 15:20:25 -07:00
Marc-Antoine Perennou 71c8f561e3 runtime: add block_on_all (#398)
Signed-off-by: Marc-Antoine Perennou <[email protected]>
2018-06-14 22:13:39 -07:00
Sean McArthur 011ebf44eb Implement Executor for Box<E: Executor> (#420) 2018-06-14 16:28:23 -07:00
Carl Lerche c25ea78ec9 Bump version of a number of sub crates (#414)
This includes:

* tokio-codec (0.1.0)
* tokio-current-thread (0.1.0)
* tokio-fs (0.1.1)
* tokio-io (0.1.7)
* tokio-reactor (0.1.2)
* tokio-udp (0.1.1)
2018-06-13 10:24:56 -07:00
Carl Lerche 2e0cd292d2 Fix some broken doc links (#413) 2018-06-13 09:02:46 -07:00
Sylwek 4ebaf18c27 Typo (#415) 2018-06-13 09:02:34 -07:00
Carl Lerche ab07733d66 Deprecate executor re-exports (#412) 2018-06-12 14:41:12 -07:00
Mat Sadler d1f825ca13 Add OpenOptions to tokio-fs (#390)
Add an `OpenOptions` struct to `tokio-fs` that mirrors the one found in
`std`. Also provide a conversion from a `std` instance to a Tokio instance.
2018-06-12 10:47:24 -07:00
Laurențiu Nicola 4cf7d73b22 tokio-fs: add into_std (#403) 2018-06-12 10:40:43 -07:00
jpbriquet 2cd854c2c7 tokio-current-thread crate (#370)
Extract `tokio::executor::current_thread` to a tokio-current-thread
crate. Deprecated fns stay in the old location. The new crate only
contains thee most recent API.
2018-06-12 10:26:03 -07:00
Carl Lerche ba05c39d65 Fix a deadlock that can happen when shutting down (#409)
There is a deadlock that can occur when the concurrent runtime shuts
down. This patch adds a test and fix.

Fixes #401.
2018-06-12 09:41:18 -07:00
Alyssa Ross 64b8884911 Fix typo in comment (#402) 2018-06-11 15:26:55 -07:00
pravic d391e63418 Duplicated word in documentation. (#405) 2018-06-11 15:17:09 -07:00
Carl Lerche 8d8c895a1c Remove tokio-codec dependency from tokio (#397)
This will be added again later once types are re-exported.
2018-06-08 09:56:40 -07:00
Carl Lerche dba5c27296 Bump version to v0.1.7 (#396)
This also bumps the versions of:

* tokio-threadpool
* tokio-timer
2018-06-06 20:14:35 -07:00
Carl Lerche db620b42ec Another attempt at abstracting Instant::now (#381)
Currently, the timer uses a `Now` trait to abstract the source of time.
This allows time to be mocked out. However, the current implementation
has a number of limitations as represented by #288 and #296.

The main issues are that `Now` requires `&mut self` which prevents a
value from being easily used in a concurrent environment. Also, when
wanting to write code that is abstract over the source of time, generics
get out of hand.

This patch provides an alternate solution. A new type, `Clock` is
provided which defaults to `Instant::now` as the source of time, but
allows configuring the actual source using a new iteration of the `Now`
trait. This time, `Now` is `Send + Sync + 'static`. Internally, `Clock`
stores the now value in an `Arc<Now>` value, which introduces dynamism
and allows `Clock` values to be cloned and be `Sync`.

Also, the current clock can be set for the current execution context
using the `with_default` pattern.

Because using the `Instant::now` will be the most common case by far, it
is special cased in order to avoid the need to allocate an `Arc` and use
dynamic dispatch.
2018-06-06 16:04:39 -07:00
David Kellum 9013ed9bd4 Fix description of BlockingError as io::Error (#384) 2018-06-06 14:34:55 -07:00
Carl Lerche 06325fa63b Bump tokio-uds to v0.2.0 (#395) 2018-06-06 14:09:07 -07:00
Sebastian Dröge 0d41ba7a08 Implement a Send Handle for the single-threaded Runtime (#340)
Implement a Send'able Handle for the single-threaded `Runtime` and
`CurrentThread` executor to spawn new tasks from other threads.
2018-06-05 16:56:15 -07:00
Carl Lerche c07a7b26d3 Cleanup FramedParts in new tokio-codec (#394) 2018-06-05 15:31:01 -07:00
Bryan Burgers f723d10087 Create tokio-codec (#360)
Create a new tokio-codec crate with many of the contents of
`tokio_io::codec`.
2018-06-04 20:36:06 -07:00
Jon Gjengset 3d7263d3a0 Implement Runtime::block_on using oneshot (#391) 2018-06-04 20:09:17 -07:00
Carl Lerche 9caec1c15d Remove futures2 crate (#380) 2018-05-29 16:28:00 -07:00
Carl Lerche 703f07ca17 Remove threadpool disclaimer (#378) 2018-05-29 15:59:37 -07:00
Michal 'vorner' Vaner db9371126d Include a manually built runtime example (#306) 2018-05-29 14:44:28 -07:00
Carl Lerche eb1cf8fc9b Unpin Rust nightly version (#379) 2018-05-29 14:36:52 -07:00
Carl Lerche 4af6109398 Fix bug related to spawning optimization (#375)
The thread pool optimizes cases where a task currently running on the
pool spawns a new future. However, the optimization did not factor in
cases where two thread pools interacted.

This patch fixes the optimization and includes a test.

Fixes #342
2018-05-24 22:06:32 -07:00
Roman Zeyde 96f3ec903c Fix a small typo in README.md (#373) 2018-05-23 12:07:46 -07:00
Chris Pick 8c791fd0bf Fix Runtime::new's doc link to tokio::run (#371) 2018-05-22 15:29:15 -07:00
Rijenkii c0747a5fc1 tokio-io: Fix the link to the repository (#372) 2018-05-22 15:28:28 -07:00
Carl Lerche c8e710d39e Import tokio-uds (#365)
This imports tokio-uds from the dedicated repo.
2018-05-14 14:48:32 -07:00
Carl Lerche e281e4f4cb Remove fuchsia references as it is not supported. (#355) 2018-05-14 12:00:19 -07:00
Carl Lerche 6598334021 Add Gitter badge to README (#358) 2018-05-14 12:00:10 -07:00
main() 35f3351c97 Document Handle::default() behavior (#359) 2018-05-14 11:11:28 -07:00
Jason Davies 1f5bb121e2 Fix typo in doc comment. (#361) 2018-05-14 11:10:25 -07:00
sbstp 88801bb613 timer: add sleep free function (#347) 2018-05-11 09:16:08 -07:00
Carl Lerche a850063211 Handle::default() should lazily bind to reactor. (#350)
Currently, not specifying a `Handle` is different than using
`Handle::default()`. This is because `Handle::default()` will
immediately bind to the reactor for the current context vs. not
specifying a `Handle`, which binds to a reactor when it is polled.

This patch changes the `Handle::default()` behavior, bringing it inline
with actual defaults.

`Handle::current()` still immediately binds to the current reactor.

Fixes #307
2018-05-11 08:32:03 -07:00
Marek Kotewicz 14ec268b8a Fixed broken link in tokio-fs documentation (#352) 2018-05-11 08:31:06 -07:00
Thijs Vermeir 363b207f2b Fix typo in documentation (#346) 2018-05-08 11:44:50 -07:00
Julian Tescher 06b2c40222 Fix typos (#348) 2018-05-08 11:44:17 -07:00
Thijs Vermeir 68b82f5721 Fix typo in documentation (#341) 2018-05-04 07:06:47 -07:00
Thijs Vermeir 7cca6499a9 Fix typo in documentation (#338) 2018-05-03 10:28:48 -07:00
Carl Lerche 8235eefbf0 Fix some dependency versions (#337) 2018-05-02 13:12:33 -07:00
Carl Lerche 14b31bdba5 Bump version to v0.1.6 (#336) 2018-05-02 12:14:44 -07:00
Carl Lerche f768163982 Filesystem manipulation APIs. (#323)
This patch adds a new crate: tokio-fs. This crate provides a wrapper
around `std` functionality that can only be performed using blocking
operations. This primarily includes filesystem operations, but it also
includes standard input, output, and error access as these streams
cannot be safely switched to non-blocking mode in a portable way.

These wrappers call the `std` functions from within a `blocking`
annotation which allows the runtime to compensate for the fact that the
thread will potentially remain blocked in a system call.
2018-05-02 11:19:58 -07:00
Carl Lerche 7a2b5db15c Remove futures2 feature from Cargo.toml files (#334)
Currently, the state of the futures2 integration is pretty broken. This
patch removes the feature flag, preventing users from trying to use it.
In the future, it can be brought back when the implementation is fixed.
2018-05-02 10:48:58 -07:00
Roman 2465483845 Current thread runtime (#308)
This patch introduces a version of `Runtime` that runs all components on
the current thread. This allows users to spawn futures that do not implement
`Send`.
2018-05-02 09:40:42 -07:00
Stefan Bühler 6a0ecef81a Timer: always park nested Park (#327)
The nested `Park` might need to do some work, even if the duration is 0
seconds (e.g. a `Reactor`).

Similar to what #313 did for CurrentThread.
2018-05-01 16:33:41 -07:00
Stefan Bühler 6defeeb2ba current_thread: make underlying Park instance accessible 2018-05-01 14:33:38 -07:00
Stefan Bühler b36a73059d tokio-io: require bytes-0.4.7 for Buf::get_uint_be 2018-05-01 14:33:38 -07:00
Roman d1d4fe4d07 Stop using deprecated bytes APIs in tests (#324) (#331) 2018-04-30 10:02:48 -07:00
Carl Lerche 9aaa8f06d1 Stop using deprecated bytes APIs (#324)
This also adds a filter for another treiber stack expected data race. The
race is expected as part of the algorithm.
2018-04-28 12:25:22 -07:00
Sebastian Dröge 6ea00162b9 Make CurrentThread::turn() more fair by always parking with 0 timeout… (#313)
This ensures that all fd-based futures are put into the queue for the
current tick, if the CurrentThread is parking via the Reactor.

Otherwise, if there are queued up futures already, only those would be
polled in the turn. These futures could then notify others/themselves to
have the queue still non-empty on the next turn. Which then potentially
allows the reactor to never be polled, and thus fd-based futures are
never queued up and polled.

Also return in the Turn return value whether any futures were polled at
all, which allows the caller to know if any work was done at all in this
turn and based on that adjust behavior.
2018-04-25 10:37:18 -07:00
Carl Lerche 61d635e8ad Threadpool blocking (#317)
This patch adds a `blocking` to `tokio-threadpool`. This function serves
as a way to annotate sections of code that will perform blocking
operations. This informs the thread pool that an additional thread needs
to be spawned to replace the current thread, which will no longer be
able to process the work queue.
2018-04-15 12:29:22 -07:00
Carl Lerche 372400ed34 Add additional timer::Error docs. (#311)
Closes #302
2018-04-10 14:28:37 -07:00
Roman ba9d849ef0 Fix warning: variable does not need to be mutable (#309) 2018-04-10 13:33:05 -07:00
Roman 5b677934fe Add example that prints each packet from tcp client (#301) 2018-04-10 13:08:55 -07:00
Sam Rijs dbcd8353b0 Update futures2 to use the futures 0.2 release (#304) 2018-04-08 20:23:33 -07:00
Carl Lerche 3be6b69e1b Refactor threadpool task types (#300)
Replaces homegrown Arc with std Arc

Is this safer? Unknown. At least we don't have to maintain an arc
implementation anymore. This will also make it easier to filter out tsan
false positives.

Also split task/mod.rs into multiple files.
2018-04-05 10:57:05 -07:00
Carl Lerche 0bcf9b0ae6 ThreadPool refactoring (#299) 2018-04-04 13:30:54 -07:00
Carl Lerche c715739599 Add arc::Weak to tsan filter. (#298) 2018-04-04 12:54:49 -07:00
David 6aea9c43e8 Update Cargo.toml (#293) 2018-04-04 09:18:56 -07:00
Igor Gnatenko 82f6a52d1a threadpool: bump minimal version of executor (#292) 2018-04-04 09:18:40 -07:00
Leandro Pacheco a6b307cfbe re-export io::{ReadHalf/WriteHalf} timer::Error (#290) 2018-04-04 09:18:12 -07:00
Roman dcb20b289c Build 32/64-bit Linux and FreeBSD on Travis CI (#286) 2018-04-04 08:37:28 -07:00
Carl Lerche 79afc7ee68 Threadpool refactor (#294)
* Switch worker lifecycle to an enum
* Move some files around
* Rename State -> PoolState
2018-04-03 22:35:59 -07:00
Kam Y. Tse 3ba5595233 Fix typo (#275) 2018-04-02 13:11:06 -07:00
Carl Lerche 7232ba6d55 Bump tokio-timer to v0.2.1 (#287) 2018-04-02 11:06:22 -07:00
Roman a14de909eb Build both x86 and x64 on Windows (#282) 2018-04-02 09:37:56 -07:00
laizy d8789cd379 fix panic in chat example (#279) 2018-04-02 09:00:40 -07:00
Roman 8d4be0361e Fix unused variable in tokio-threadpool\tests\threadpool.rs:581:9 (#284) 2018-04-02 09:00:14 -07:00
Daniel Griffen 3f2710397d Fix tokio-timer on 32bit systems (#274) 2018-04-02 08:53:43 -07:00
Roman 8895a7d3ab Fix Appveyor badge on crates.io page (#280) 2018-04-01 16:16:22 -07:00
Carl Lerche 10cb9dd468 Actually bump tokio to v0.1.5 (#273) 2018-03-30 15:37:52 -07:00
Carl Lerche 2ca214bd2c Fix tokio dependency versions (#272) 2018-03-30 15:32:25 -07:00
Carl Lerche 2a01c26d58 Bump version to v0.1.5 (#271)
This also bumps:

* tokio-executor to v0.1.2
* tokio-threadpool to v0.1.2
* tokio-timer to v0.2.0
2018-03-30 15:28:44 -07:00
Carl Lerche ea172537aa Rename Sleep to Delay (#270)
This patch renames `Sleep` from tokio-timer and the tokio facade to
`Delay`. Given that the future does not actually put anything to sleep,
the `Delay` name feels more appropriate.

Fixes #263
2018-03-30 14:21:48 -07:00
Carl Lerche baa2502ec6 Integrate timers with runtime. (#266)
This patch integrate the new timer implementation with the runtime by
initializing a timer per worker thread. This allows minimizing the
amount of synchronization needed for using timers.
2018-03-30 11:50:02 -07:00
Carl Lerche d4d17392fe Remove println from tests (#267) 2018-03-29 20:58:02 -07:00
Carl Lerche 6807363efd Fix permissions (#268) 2018-03-29 20:42:59 -07:00
Carl Lerche 1c5d131245 Allow customizing the threadpool's parker (#264)
* Allow customizing the threadpool's parker

This patch allows the user of threadpool to customize how the worker
threads park themselves. This allows custom parking logic to be
injected. For example, this allows embedding a timer on each worker
thread.

* Call `park` instance every so often.

Since the `park` is now customizable, it might have logic that must be
called every so often. For example, a timer might have timeouts that it
must expire.

Currently, if a worker is very busy, it won't call into the `park`
instance. This patch changes this so that after every 32 task
invocations, `park` is called with a duration of zero.
2018-03-29 13:47:08 -07:00
Carl Lerche 19500f7df8 Provide a timer implementation (#249)
This patch adds a new crate: tokio-timer. This crate provides an
efficient timer implemeentation designed for use in Tokio based
applications.

The timer users a hierarchical hashed timer wheel algorithm with six
levels, each having 64 slots. This allows the timer to have a resolution
of 1ms while maintaining O(1) complexity for insert, removal, and firing
of timeouts.

There already exists a tokio-timer crate. This is a complete rewrite
which solves the outstanding problems with the existing tokio-timer
library.

Closes #146.
2018-03-28 22:26:47 -07:00
Roman ad189826f4 Split tokio-threadpool lib.rs into files (#233)
* Builder -> src/builder.rs
* Callback -> src/callback.rs
* Config -> src/config.rs
* Futures2Wake -> src/futures2_wake.rs
* Inner -> src/inner.rs
* Notifier-> src/notifier.rs
* Sender -> src/sender.rs
* Shutdown -> src/shutdown.rs
* ShutdownTask -> src/shutdown_task.rs
* SleepStack -> src/sleep_stack.rs
* State -> src/state.rs
* ThreadPool -> src/thread_pool.rs
* Worker -> src/worker.rs
* WorkerEntry -> src/worker_entry.rs
* WorkerState -> src/worker_state.rs
2018-03-27 15:56:21 -07:00
Klaus Purer a612736f54 fix(cargo): Bump dependencies so that Tokio compiles with minimal versions (#258) 2018-03-27 15:47:11 -07:00
Carl Lerche bda8dd5113 Update futures2 Cargo.toml (#256) 2018-03-27 15:46:19 -07:00
Sam Rijs 415a786049 Fix unstable-futures feature flag propagation (#261) 2018-03-27 15:46:02 -07:00
Carl Lerche 2edc35a45d Disable future 0.2 tests (#259) 2018-03-24 14:01:41 -07:00
Carl Lerche 9cffda59c9 Bump version to v0.1.4 (#252)
This also bumps:

* tokio-executor: v0.1.1
* tokio-reactor: v0.1.1
* tokio-threadpool: v0.1.1
2018-03-23 10:34:42 -07:00
Carl Lerche 23d95b44e7 Change hammer test consts (#255) 2018-03-23 10:27:58 -07:00
Denis 69e45f4be4 Fix a typo in tokio_reactor::PollEvented (#250)
`read_readiness` -> `write_readiness` in *write* methods
2018-03-22 20:15:31 -07:00
Denis 6bdfa159a7 Fix connect example for UDP (#241)
Close #241
2018-03-22 10:02:01 -07:00
Michal 'vorner' Vaner 5d87a9cee1 Docs: warn about errors from TcpListener::incoming (#247) 2018-03-22 10:01:05 -07:00
Denis 16d3540ce9 Add UDP client example (send/recv_dgram) (#239) 2018-03-22 10:00:27 -07:00
Sean McArthur e5ebd02885 implement poll_vectored_* and initializer method for futures2 (#242) 2018-03-22 09:59:13 -07:00
Carl Lerche 08c21e7bac Fix race condition related bugs (#243)
* Fix races.

This mostly pulls in changes from rust-lang-nursery/futures-rs#881, but
also updates Registration to be a bit more obvious as to what is going
on.

* Reduce spurious wakeups caused by Reactor

This patch adds an ABA guard on token values before registering them
with Mio. This allows catching token reuse and avoid the notification.

This is needed for OS X as the notification is used to determine that a
TCP connect has completed. A spurious notification can potentially cause
write failures.
2018-03-22 09:57:40 -07:00
Aaron Turon 8786741ba9 Update to futures 0.2.0-beta (#246) 2018-03-21 14:21:02 -07:00
Roman 494f0dc176 Runtime builder (#234)
* Split runtime module into files
* Add runtime::Builder to set up thread pool.
2018-03-21 11:23:36 -07:00
Cyril Plisko df9025594c Lapsus clavis (#245) 2018-03-21 11:17:16 -07:00
Aaron Turon 7b1306e6c2 Add top-level tests for futures 0.2 integration (#231) 2018-03-15 16:38:12 -07:00
Sean McArthur b7f4e337be set Runtime thread pool name prefix (#232) 2018-03-15 16:37:32 -07:00
Hiroaki Nakamura d1046db735 Fix condition for updating the current date buffer (#230) 2018-03-15 09:29:55 -07:00
Sam Rijs 923a80e098 Move tokio::net module into tokio tcp/udp crates (#224) 2018-03-14 09:38:59 -07:00
Carl Lerche 64435f5b35 Travis: Move before_deploy out of matrix block (#229) 2018-03-13 15:58:53 -07:00
Aaron Turon d304791c0e Simultaneous futures compat (#172)
This patch adds opt-in support for futures 0.2.
2018-03-13 13:57:35 -07:00
Carl Lerche 5846b3fc2a Reduce AtomicTask spurious notifications (#227) 2018-03-13 13:28:41 -07:00
Carl Lerche 8eb3e58b7d Shutdown the runtime on drop (#214)
Currently, the runtime does not shutdown if the runtime handle is
dropped. This can happen during a panic or when the value is simply
dropped.

This patch forces the runtime to shutdown if it is not explicitly
shutdown.

Fixes #209
2018-03-13 13:14:51 -07:00
Carl Lerche c0a2cc1f9e Add LICENSE file to all sub crates (#226)
* Add LICENSE file to all sub crates
* Update links in README
2018-03-13 13:14:28 -07:00
Carl Lerche 61b2889881 Try compiling Tokio on FreeBSD (#228) 2018-03-13 13:14:12 -07:00
hcpl 5dab821b29 Fix docs markup (#225) 2018-03-13 11:43:05 -07:00
Wesley Moore 2abeff01a5 Fix build on FreeBSD (#218) 2018-03-13 11:24:17 -07:00
Carl Lerche 96a542451d Handle futures that panic on a threadpool (#216)
If a future panics from within the context of a thread pool, the pool
should not be impacted. To do this, polling the future is wrapped with a
catch_unwind. Extra care is taken to ensure that `thread::panicking()`
is set from within the future's drop handle.

Fixes #209
2018-03-13 09:44:14 -07:00
Gray Olson 95899e007d Update comment in udp-codec example (#222) 2018-03-12 10:39:32 -07:00
Jeehoon Kang e6e3c49e0e Remove uses of futures_cpupool (#220) 2018-03-11 11:37:10 -07:00
Carl Lerche 4d514b7eb3 Relicense Tokio exclusively under the MIT license. (#215)
This patch relicenses the Tokio project exclusively under the MIT
license. Before this, the project was dual licensed under MIT and Apache
2. As such, switching to only MIT is permitted.

Fixes #202
2018-03-09 20:07:09 -08:00
Tosil Velkov 2a1585157e Fix wrong file link in examples readme.md (#208) 2018-03-09 12:26:09 -08:00
Igor Gnatenko 189d6baac4 tokio-threadpool: bump rand to 0.4 (#205) 2018-03-09 12:25:38 -08:00
Igor Gnatenko 3ad27e99ec tokio-reactor: bump mio to 0.6.14 (#204)
With 0.6.13 it doesn't compile:
no method named `as_usize` found for type `mio::Ready` in the current scope
2018-03-09 12:25:16 -08:00
Carl Lerche bf1305c421 Bump version to v0.1.3 (#213) 2018-03-09 11:41:31 -08:00
Carl Lerche cf7435ba30 CurrentThread::turn should block on idle. (#212)
This patch fixes a bug where `CurrentThread::turn` is expected to block
even if the executor is idle.

The `turn` API is the low level interface for callers to interact with
the `Sleep` instance used by the `CurrentThread` instance. As such, a
call to `turn` is expected to call `sleep` once if the executor did not
perform any work.
2018-03-09 11:09:10 -08:00
445 changed files with 49102 additions and 6744 deletions
-16
View File
@@ -1,16 +0,0 @@
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host x86_64-pc-windows-msvc
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- if NOT "%TARGET%" == "x86_64-pc-windows-msvc" rustup target add %TARGET%
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --target %TARGET%
+28
View File
@@ -0,0 +1,28 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_DURATION: 10
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
cargo_cache:
folder: $HOME/.cargo/registry
test_script:
- . $HOME/.cargo/env
- cargo test --all --no-fail-fast
- cargo doc --all
i686_test_script:
- . $HOME/.cargo/env
- cargo test --all --exclude tokio-tls --no-fail-fast --target i686-unknown-freebsd
before_cache_script:
- rm -rf $HOME/.cargo/registry/index
+51
View File
@@ -0,0 +1,51 @@
<!--
Thank you for reporting an issue.
Please fill in as much of the template below as you're able.
-->
## Version
<!--
List the versions of all `tokio` crates you are using. The easiest way to get
this information is using `cargo-tree`.
`cargo install cargo-tree`
(see install here: https://github.com/sfackler/cargo-tree)
Then:
`cargo tree | grep tokio`
-->
## Platform
<!---
Output of `uname -a` (UNIX), or version and 32 or 64-bit (Windows)
-->
## Subcrates
<!--
If known, please specify the affected Tokio sub crates. Otherwise, delete this
section.
-->
## Description
<!--
Enter your issue details below this comment.
One way to structure the description:
<short summary of the bug>
I tried this code:
<code sample that causes the bug>
I expected to see this happen: <explanation>
Instead, this happened: <explanation>
-->
+23
View File
@@ -0,0 +1,23 @@
<!--
Thank you for your Pull Request. Please provide a description above and review
the requirements below.
Bug fixes and new features should include tests.
Contributors guide: https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md
-->
## Motivation
<!--
Explain the context and why you're making that change. What is the problem
you're trying to solve? In some cases there is not a problem and this can be
thought of as being the motivation for your change.
-->
## Solution
<!--
Summarize the solution and provide any necessary context needed to understand
the code change.
-->
-40
View File
@@ -1,40 +0,0 @@
---
language: rust
sudo: false
matrix:
include:
- rust: 1.21.0
- rust: stable
before_deploy: cargo doc --all --no-deps
- os: osx
- rust: beta
- rust: nightly
script:
- |
if [[ "$TRAVIS_RUST_VERSION" == nightly ]]
then
cargo build --benches --all
fi
- cargo test --all
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = linux
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+105
View File
@@ -1,3 +1,108 @@
This changelog only applies to the `tokio` crate proper. Each sub crate
maintains its own changelog tracking changes made in each respective sub crate.
# 0.1.17 (March 13, 2019)
### Added
- Propagate trace subscriber in the runtime (#966).
# 0.1.16 (March 1, 2019)
### Fixed
- async-await: track latest nightly changes (#940).
### Added
- `sync::Watch`, a single value broadcast channel (#922).
- Async equivalent of read / write file helpers being added to `std` (#896).
# 0.1.15 (January 24, 2019)
### Added
- Re-export tokio-sync APIs (#839).
- Stream enumerate combinator (#832).
# 0.1.14 (January 6, 2019)
* Use feature flags to break up the crate, allowing users to pick & choose
components (#808).
* Export `UnixDatagram` and `UnixDatagramFramed` (#772).
# 0.1.13 (November 21, 2018)
* Fix `Runtime::reactor()` when no tasks are spawned (#721).
* `runtime::Builder` no longer uses deprecated methods (#749).
* Provide `after_start` and `before_stop` configuration settings for
`Runtime` (#756).
* Implement throttle stream combinator (#736).
# 0.1.12 (October 23, 2018)
* runtime: expose `keep_alive` on runtime builder (#676).
* runtime: create a reactor per worker thread (#660).
* codec: fix panic in `LengthDelimitedCodec` (#682).
* io: re-export `tokio_io::io::read` function (#689).
* runtime: check for executor re-entry in more places (#708).
# 0.1.11 (September 28, 2018)
* Fix `tokio-async-await` dependency (#675).
# 0.1.10 (September 27, 2018)
* Fix minimal versions
# 0.1.9 (September 27, 2018)
* Experimental async/await improvements (#661).
* Re-export `TaskExecutor` from `tokio-current-thread` (#652).
* Improve `Runtime` builder API (#645).
* `tokio::run` panics when called from the context of an executor
(#646).
* Introduce `StreamExt` with a `timeout` helper (#573).
* Move `length_delimited` into `tokio` (#575).
* Re-organize `tokio::net` module (#548).
* Re-export `tokio-current-thread::spawn` in current_thread runtime
(#579).
# 0.1.8 (August 23, 2018)
* Extract tokio::executor::current_thread to a sub crate (#370)
* Add `Runtime::block_on` (#398)
* Add `runtime::current_thread::block_on_all` (#477)
* Misc documentation improvements (#450)
* Implement `std::error::Error` for error types (#501)
# 0.1.7 (June 6, 2018)
* Add `Runtime::block_on` for concurrent runtime (#391).
* Provide handle to `current_thread::Runtime` that allows spawning tasks from
other threads (#340).
* Provide `clock::now()`, a configurable source of time (#381).
# 0.1.6 (May 2, 2018)
* Add asynchronous filesystem APIs (#323).
* Add "current thread" runtime variant (#308).
* `CurrentThread`: Expose inner `Park` instance.
* Improve fairness of `CurrentThread` executor (#313).
# 0.1.5 (March 30, 2018)
* Provide timer API (#266)
# 0.1.4 (March 22, 2018)
* Fix build on FreeBSD (#218)
* Shutdown the Runtime when the handle is dropped (#214)
* Set Runtime thread name prefix for worker threads (#232)
* Add builder for Runtime (#234)
* Extract TCP and UDP types into separate crates (#224)
* Optionally support futures 0.2.
# 0.1.3 (March 09, 2018)
* Fix `CurrentThread::turn` to block on idle (#212).
# 0.1.2 (March 09, 2018)
* Introduce Tokio Runtime (#141)
+387
View File
@@ -0,0 +1,387 @@
# Contributing to Tokio
:balloon: Thanks for your help improving the project! We are so happy to have
you!
There are opportunities to contribute to Tokio at any level. It doesn't matter if
you are just getting started with Rust or are the most weathered expert, we can
use your help.
**No contribution is too small and all contributions are valued.**
This guide will help you get started. **Do not let this guide intimidate you**.
It should be considered a map to help you navigate the process.
You may also get help with contributing in the [dev channel][dev], please join
us!
[dev]: https://gitter.im/tokio-rs/dev
## Conduct
The Tokio project adheres to the [Rust Code of Conduct][coc]. This describes
the _minimum_ behavior expected from all contributors.
[coc]: https://github.com/rust-lang/rust/blob/master/CODE_OF_CONDUCT.md
## Contributing in Issues
For any issue, there are fundamentally three ways an individual can contribute:
1. By opening the issue for discussion: For instance, if you believe that you
have uncovered a bug in Tokio, creating a new issue in the tokio-rs/tokio
issue tracker is the way to report it.
2. By helping to triage the issue: This can be done by providing
supporting details (a test case that demonstrates a bug), providing
suggestions on how to address the issue, or ensuring that the issue is tagged
correctly.
3. By helping to resolve the issue: Typically this is done either in the form of
demonstrating that the issue reported is not a problem after all, or more
often, by opening a Pull Request that changes some bit of something in
Tokio in a concrete and reviewable manner.
**Anybody can participate in any stage of contribution**. We urge you to
participate in the discussion around bugs and participate in reviewing PRs.
### Asking for General Help
If you have reviewed existing documentation and still have questions or are
having problems, you can open an issue asking for help.
In exchange for receiving help, we ask that you contribute back a documentation
PR that helps others avoid the problems that you encountered.
### Submitting a Bug Report
When opening a new issue in the Tokio issue tracker, users will be presented
with a [basic template][template] that should be filled in. If you believe that you have
uncovered a bug, please fill out this form, following the template to the best
of your ability. Do not worry if you cannot answer every detail, just fill in
what you can.
The two most important pieces of information we need in order to properly
evaluate the report is a description of the behavior you are seeing and a simple
test case we can use to recreate the problem on our own. If we cannot recreate
the issue, it becomes impossible for us to fix.
In order to rule out the possibility of bugs introduced by userland code, test
cases should be limited, as much as possible, to using only Tokio APIs.
See [How to create a Minimal, Complete, and Verifiable example][mcve].
[mcve]: https://stackoverflow.com/help/mcve
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Triaging a Bug Report
Once an issue has been opened, it is not uncommon for there to be discussion
around it. Some contributors may have differing opinions about the issue,
including whether the behavior being seen is a bug or a feature. This discussion
is part of the process and should be kept focused, helpful, and professional.
Short, clipped responses—that provide neither additional context nor supporting
detail—are not helpful or professional. To many, such responses are simply
annoying and unfriendly.
Contributors are encouraged to help one another make forward progress as much as
possible, empowering one another to solve issues collaboratively. If you choose
to comment on an issue that you feel either is not a problem that needs to be
fixed, or if you encounter information in an issue that you feel is incorrect,
explain why you feel that way with additional supporting context, and be willing
to be convinced that you may be wrong. By doing so, we can often reach the
correct outcome much faster.
### Resolving a Bug Report
In the majority of cases, issues are resolved by opening a Pull Request. The
process for opening and reviewing a Pull Request is similar to that of opening
and triaging issues, but carries with it a necessary review and approval
workflow that ensures that the proposed changes meet the minimal quality and
functional guidelines of the Tokio project.
## Pull Requests
Pull Requests are the way concrete changes are made to the code, documentation,
and dependencies in the Tokio repository.
Even tiny pull requests (e.g., one character pull request fixing a typo in API
documentation) are greatly appreciated. Before making a large change, it is
usually a good idea to first open an issue describing the change to solicit
feedback and guidance. This will increase the likelihood of the PR getting
merged.
### Tests
If the change being proposed alters code (as opposed to only documentation for
example), it is either adding new functionality to Tokio or it is fixing
existing, broken functionality. In both of these cases, the pull request should
include one or more tests to ensure that Tokio does not regress in the future.
There are two ways to write tests: integration tests and documentation tests
(Tokio avoids unit tests as much as possible).
#### Integration tests
Integration tests go in the same crate as the code they are testing. Each sub
crate should have a `dev-dependency` on `tokio` itself. This makes all Tokio
utilities available to use in tests, no matter the crate being tested.
The best strategy for writing a new integration test is to look at existing
integration tests in the crate and follow the style.
#### Documentation tests
Ideally, every API has at least one [documentation test] that demonstrates how to
use the API. Documentation tests are run with `cargo test --doc`. This ensures
that the example is correct and provides additional test coverage.
The trick to documentation tests is striking a balance between being succinct
for a reader to understand and actually testing the API.
Same as with integration tests, when writing a documentation test, the full
`tokio` crate is available. This is especially useful for getting access to the
runtime to run the example.
The documentation tests will be visible from both the crate specific
documentation **and** the `tokio` facade documentation via the re-export. The
example should be written from the point of view of a user that is using the
`tokio` crate. As such, the example should use the API via the facade and not by
directly referencing the crate.
The type level example for `tokio_timer::Timeout` provides a good example of a
documentation test:
```
/// # extern crate futures;
/// # extern crate tokio;
/// // import the `timeout` function, usually this is done
/// // with `use tokio::prelude::*`
/// use tokio::prelude::FutureExt;
/// use futures::Stream;
/// use futures::sync::mpsc;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = mpsc::unbounded();
/// # tx.unbounded_send(()).unwrap();
/// # drop(tx);
///
/// let process = rx.for_each(|item| {
/// // do something with `item`
/// # drop(item);
/// # Ok(())
/// });
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// process.timeout(Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
Given that this is a *type* level documentation test and the primary way users
of `tokio` will create an instance of `Timeout` is by using
`FutureExt::timeout`, this is how the documentation test is structured.
Lines that start with `/// #` are removed when the documentation is generated.
They are only there to get the test to run. The `block_on_all` function is the
easiest way to execute a future from a test.
If this were a documentation test for the `Timeout::new` function, then the
example would explicitly use `Timeout::new`. For example:
```
/// # extern crate futures;
/// # extern crate tokio;
/// use tokio::timer::Timeout;
/// use futures::Future;
/// use futures::sync::oneshot;
/// use std::time::Duration;
///
/// # fn main() {
/// let (tx, rx) = oneshot::channel();
/// # tx.send(()).unwrap();
///
/// # tokio::runtime::current_thread::block_on_all(
/// // Wrap the future with a `Timeout` set to expire in 10 milliseconds.
/// Timeout::new(rx, Duration::from_millis(10))
/// # ).unwrap();
/// # }
```
### Commits
It is a recommended best practice to keep your changes as logically grouped as
possible within individual commits. There is no limit to the number of commits
any single Pull Request may have, and many contributors find it easier to review
changes that are split across multiple commits.
That said, if you have a number of commits that are "checkpoints" and don't
represent a single logical change, please squash those together.
Note that multiple commits often get squashed when they are landed (see the
notes about [commit squashing]).
#### Commit message guidelines
A good commit message should describe what changed and why.
1. The first line should:
* contain a short description of the change (preferably 50 characters or less,
and no more than 72 characters)
* be entirely in lowercase with the exception of proper nouns, acronyms, and
the words that refer to code, like function/variable names
* be prefixed with the name of the sub crate being changed (without the `tokio-`
prefix) and start with an imperative verb. If modifying `tokio` proper,
omit the crate prefix.
Examples:
* timer: introduce `Timeout` and deprecate `Deadline`
* export `Encoder`, `Decoder`, `Framed*` from tokio_codec
2. Keep the second line blank.
3. Wrap all other lines at 72 columns (except for long URLs).
4. If your patch fixes an open issue, you can add a reference to it at the end
of the log. Use the `Fixes: #` prefix and the issue number. For other
references use `Refs: #`. `Refs` may include multiple issues, separated by a
comma.
Examples:
- `Fixes: #1337`
- `Refs: #1234`
Sample complete commit message:
```txt
subcrate: explain the commit in one line
Body of commit message is a few lines of text, explaining things
in more detail, possibly giving some background about the issue
being fixed, etc.
The body of the commit message can be several paragraphs, and
please do proper word-wrap and keep columns shorter than about
72 characters or so. That way, `git log` will show things
nicely even when it is indented.
Fixes: #1337
Refs: #453, #154
```
### Opening the Pull Request
From within GitHub, opening a new Pull Request will present you with a
[template] that should be filled out. Please try to do your best at filling out
the details, but feel free to skip parts if you're not sure what to put.
[template]: .github/PULL_REQUEST_TEMPLATE.md
### Discuss and update
You will probably get feedback or requests for changes to your Pull Request.
This is a big part of the submission process so don't be discouraged! Some
contributors may sign off on the Pull Request right away, others may have
more detailed comments or feedback. This is a necessary part of the process
in order to evaluate whether the changes are correct and necessary.
**Any community member can review a PR and you might get conflicting feedback**.
Keep an eye out for comments from code owners to provide guidance on conflicting
feedback.
**Once the PR is open, do not rebase the commits**. See [Commit Squashing] for
more details.
### Commit Squashing
In most cases, **do not squash commits that you add to your Pull Request during
the review process**. When the commits in your Pull Request land, they may be
squashed into one commit per logical change. Metadata will be added to the
commit message (including links to the Pull Request, links to relevant issues,
and the names of the reviewers). The commit history of your Pull Request,
however, will stay intact on the Pull Request page.
## Reviewing Pull Requests
**Any Tokio community member is welcome to review any pull request**.
All Tokio contributors who choose to review and provide feedback on Pull
Requests have a responsibility to both the project and the individual making the
contribution. Reviews and feedback must be helpful, insightful, and geared
towards improving the contribution as opposed to simply blocking it. If there
are reasons why you feel the PR should not land, explain what those are. Do not
expect to be able to block a Pull Request from advancing simply because you say
"No" without giving an explanation. Be open to having your mind changed. Be open
to working with the contributor to make the Pull Request better.
Reviews that are dismissive or disrespectful of the contributor or any other
reviewers are strictly counter to the Code of Conduct.
When reviewing a Pull Request, the primary goals are for the codebase to improve
and for the person submitting the request to succeed. **Even if a Pull Request
does not land, the submitters should come away from the experience feeling like
their effort was not wasted or unappreciated**. Every Pull Request from a new
contributor is an opportunity to grow the community.
### Review a bit at a time.
Do not overwhelm new contributors.
It is tempting to micro-optimize and make everything about relative performance,
perfect grammar, or exact style matches. Do not succumb to that temptation.
Focus first on the most significant aspects of the change:
1. Does this change make sense for Tokio?
2. Does this change make Tokio better, even if only incrementally?
3. Are there clear bugs or larger scale issues that need attending to?
4. Is the commit message readable and correct? If it contains a breaking change
is it clear enough?
Note that only **incremental** improvement is needed to land a PR. This means
that the PR does not need to be perfect, only better than the status quo. Follow
up PRs may be opened to continue iterating.
When changes are necessary, *request* them, do not *demand* them, and **do not
assume that the submitter already knows how to add a test or run a benchmark**.
Specific performance optimization techniques, coding styles and conventions
change over time. The first impression you give to a new contributor never does.
Nits (requests for small changes that are not essential) are fine, but try to
avoid stalling the Pull Request. Most nits can typically be fixed by the Tokio
Collaborator landing the Pull Request but they can also be an opportunity for
the contributor to learn a bit more about the project.
It is always good to clearly indicate nits when you comment: e.g.
`Nit: change foo() to bar(). But this is not blocking.`
If your comments were addressed but were not folded automatically after new
commits or if they proved to be mistaken, please, [hide them][hiding-a-comment]
with the appropriate reason to keep the conversation flow concise and relevant.
### Be aware of the person behind the code
Be aware that *how* you communicate requests and reviews in your feedback can
have a significant impact on the success of the Pull Request. Yes, we may land
a particular change that makes Tokio better, but the individual might just not
want to have anything to do with Tokio ever again. The goal is not just having
good code.
### Abandoned or Stalled Pull Requests
If a Pull Request appears to be abandoned or stalled, it is polite to first
check with the contributor to see if they intend to continue the work before
checking if they would mind if you took it over (especially if it just has nits
left). When doing so, it is courteous to give the original contributor credit
for the work they started (either by preserving their name and email address in
the commit log, or by using an `Author: ` meta-data tag in the commit.
_Adapted from the [Node.js contributing guide][node]_.
[node]: https://github.com/nodejs/node/blob/master/CONTRIBUTING.md
[hiding-a-comment]: https://help.github.com/articles/managing-disruptive-comments/#hiding-a-comment
[documentation test]: https://doc.rust-lang.org/rustdoc/documentation-tests.html
+88 -19
View File
@@ -1,17 +1,19 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.2"
version = "0.1.17"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT/Apache-2.0"
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.17/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio/0.1"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
@@ -23,30 +25,100 @@ keywords = ["io", "async", "non-blocking", "futures"]
members = [
"./",
"tokio-async-await",
"tokio-buf",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
"tokio-fs",
"tokio-io",
"tokio-reactor",
"tokio-signal",
"tokio-sync",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-trace",
"tokio-trace/tokio-trace-core",
"tokio-udp",
"tokio-uds",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"sync",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["io", "tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
"tokio-trace-core",
]
sync = ["tokio-sync"]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
tokio-io = { version = "0.1.6", path = "tokio-io" }
tokio-executor = { version = "0.1.0", path = "tokio-executor" }
tokio-reactor = { version = "0.1.0", path = "tokio-reactor" }
tokio-threadpool = { version = "0.1.0", path = "tokio-threadpool" }
bytes = "0.4"
log = "0.4"
mio = "0.6.14"
slab = "0.4"
iovec = "0.1"
futures = "0.1.18"
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.6", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-sync = { version = "0.1.3", path = "tokio-sync", optional = true }
tokio-threadpool = { version = "0.1.8", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
tokio-trace-core = { version = "0.1", path = "tokio-trace/tokio-trace-core", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.4", default-features = false }
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
@@ -57,6 +129,3 @@ serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
tokio-io = { path = "tokio-io" }
+1 -1
View File
@@ -1,4 +1,4 @@
Copyright (c) 2018 Tokio Contributors
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
-201
View File
@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
+65 -19
View File
@@ -14,25 +14,26 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio)
[API Docs](https://docs.rs/tokio/0.1.17/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[master-dox]: https://tokio-rs.github.io/tokio/doc/tokio/
## Overview
@@ -49,7 +50,7 @@ These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.1/tokio/reactor/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
## Example
@@ -99,6 +100,24 @@ fn main() {
More examples can be found [here](examples).
## Getting Help
First, see if the answer to your question can be found in the [Guides] or the
[API documentation]. If the answer is not there, there is an active community in
the [Tokio Gitter channel][chat]. We would be happy to try to answer your
question. Last, if that doesn't work, try opening an [issue] with the question.
[chat]: https://gitter.im/tokio-rs/tokio
[issue]: https://github.com/tokio-rs/tokio/issues/new
## Contributing
:balloon: Thanks for your help improving the project! We are so happy to have
you! We have a [contributing guide][guide] to help you get involved in the Tokio
project.
[guide]: CONTRIBUTING.md
## Project layout
The `tokio` crate, found at the root, is primarily intended for use by
@@ -107,34 +126,61 @@ have greater guarantees of stability.
The crates included as part of Tokio are:
* [`tokio-async-await`]: Experimental `async` / `await` support.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-current-thread`]: Schedule the execution of futures on the current
thread.
* [`tokio-executor`]: Task execution related traits and utilities.
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
* [`tokio-tcp`]: TCP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-threadpool`]: Schedules the execution of futures across a pool of
threads.
* [ `tokio-timer`]: Time related APIs.
* [`tokio-udp`]: UDP bindings for use with `tokio-io` and `tokio-reactor`.
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
[`tokio-async-await`]: tokio-async-await
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-io`]: tokio-io
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
[`tokio-timer`]: tokio-timer
[`tokio-udp`]: tokio-udp
[`tokio-uds`]: tokio-uds
## Supported Rust Versions
Tokio is built against the latest stable, nightly, and beta Rust releases. The
minimum version supported is the stable release from three months before the
current stable release version. For example, if the latest stable Rust is 1.29,
the minimum version supported is 1.26. The current Tokio version is not
guaranteed to build on Rust versions earlier than the minimum supported version.
## License
This project is licensed under either of
* Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
http://www.apache.org/licenses/LICENSE-2.0)
* MIT license ([LICENSE-MIT](LICENSE-MIT) or
http://opensource.org/licenses/MIT)
at your option.
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in tokio by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+115
View File
@@ -0,0 +1,115 @@
trigger: ["master"]
pr: ["master"]
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
name: rustfmt
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
displayName: Test tokio
cross: true
crates:
- tokio
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates -
cross: true
crates:
- tokio-fs
- tokio-reactor
- tokio-signal
- tokio-tcp
- tokio-tls
- tokio-udp
- tokio-uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates - Any
crates:
- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
- tokio-io
- tokio-sync
- tokio-threadpool
- tokio-timer
- tokio-trace
- tokio-trace/tokio-trace-core
- template: ci/azure-cargo-check.yml
parameters:
name: features
displayName: Check feature permtuations
rust: stable
crates:
tokio:
- codec
- fs
- io
- reactor
- rt-full
- tcp
- timer
- udp
- uds
tokio-buf:
- util
# Check async / await
- template: ci/azure-cargo-check.yml
parameters:
name: async_await
displayName: Async / Await
rust: nightly-2019-02-28
noDefaultFeatures: ''
benches: true
crates:
tokio:
- async-await-preview
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross_32bit_linux
target: i686-unknown-linux-gnu
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust_version: 1.26.0
- template: ci/azure-tsan.yml
parameters:
name: tsan
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
- rustfmt
- test_tokio
- test_sub_cross
- test_linux
- features
- async_await
- cross_32bit_linux
- minrust
- tsan
+1 -3
View File
@@ -10,8 +10,8 @@ use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
@@ -57,7 +57,6 @@ fn udp_echo_latency(b: &mut Bencher) {
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
@@ -67,7 +66,6 @@ fn udp_echo_latency(b: &mut Bencher) {
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
+8 -9
View File
@@ -3,14 +3,13 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
extern crate test;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
use mio::{PollOpt, Ready, Token};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
@@ -22,8 +21,8 @@ fn mio_register_deregister(b: &mut Bencher) {
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
poll.deregister(&sock).unwrap();
});
}
@@ -36,12 +35,12 @@ fn mio_reregister(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.reregister(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
});
poll.deregister(&sock).unwrap();
}
+62 -50
View File
@@ -11,19 +11,18 @@ pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::executor::current_thread;
pub use tokio::reactor::Reactor;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::io::{self, Read, Write};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
pub use test::{self, Bencher};
}
mod connect_churn {
use ::prelude::*;
use prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
@@ -37,25 +36,29 @@ mod connect_churn {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
Ok(TcpStream::connect(&addr).and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
let connects_concurrent = connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(connects_concurrent)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
}
@@ -66,8 +69,7 @@ mod connect_churn {
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
@@ -76,47 +78,56 @@ mod connect_churn {
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(shutdown_rx)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
use std::sync::{Arc, Barrier};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
let threads: Vec<_> = (0..n)
.map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()))
.wait()
.unwrap();
})
})
}).collect();
.collect();
barrier.wait();
@@ -142,7 +153,7 @@ mod connect_churn {
}
mod transfer {
use ::prelude::*;
use prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
@@ -201,7 +212,8 @@ mod transfer {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
let server = listener
.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
@@ -211,17 +223,17 @@ mod transfer {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
drain
.map(|_| ())
.map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
.and_then(move |sock| Transfer {
sock: sock,
rem: MB,
chunk: write_size,
})
.map_err(|e| panic!("client err: {:?}", e));
@@ -230,7 +242,7 @@ mod transfer {
}
mod small_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
@@ -239,7 +251,7 @@ mod transfer {
}
mod big_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
+27
View File
@@ -0,0 +1,27 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- ${{ if eq(crate.key, 'tokio') }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check features = ${{ feature }}
- ${{ if not(eq(crate.key, 'tokio')) }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- ${{ if parameters.benches }}:
- script: cargo check --benches --all
displayName: Check benchmarks
+12
View File
@@ -0,0 +1,12 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- script: cargo check --all
displayName: cargo check --all
+21
View File
@@ -0,0 +1,21 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+38
View File
@@ -0,0 +1,38 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --all --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+27
View File
@@ -0,0 +1,27 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain %RUSTUP_TOOLCHAIN%
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+16
View File
@@ -0,0 +1,16 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
rustup component add rustfmt
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- ${{ each crate in parameters.crates }}:
- ${{ if eq(crate, 'tokio') }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test
- ${{ if not(eq(crate, 'tokio')) }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+34
View File
@@ -0,0 +1,34 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-threadpool --tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: nightly-2018-11-18
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+37
View File
@@ -0,0 +1,37 @@
# TSAN suppressions file for Tokio
# TSAN does not understand fences and `Arc::drop` is implemented using a fence.
# This causes many false positives.
race:Arc*drop
race:Weak*drop
# `std` mpsc is not used in any Tokio code base. This race is triggered by some
# rust runtime logic.
race:std*mpsc_queue
# Probably more fences in std.
race:__call_tls_dtors
# The epoch-based GC uses fences.
race:crossbeam_epoch
# Push and steal operations in crossbeam-deque may cause data races, but such
# data races are safe. If a data race happens, the value read by `steal` is
# forgotten and the steal operation is then retried.
race:crossbeam_deque*push
race:crossbeam_deque*steal
# This filters out expected data race in the Treiber stack implementations.
# Treiber stacks are inherently racy. The pop operation will attempt to access
# the "next" pointer on the node it is attempting to pop. However, at this
# point it has not gained ownership of the node and another thread might beat
# it and take ownership of the node first (touching the next pointer). The
# original pop operation will fail due to the ABA guard, but tsan still picks
# up the access on the next pointer.
race:Backup::next_sleeper
race:Backup::set_next_sleeper
race:WorkerEntry::set_next_sleeper
# This ignores a false positive caused by `thread::park()`/`thread::unpark()`.
# See: https://github.com/rust-lang/rust/pull/54806#issuecomment-436193353
race:pthread_cond_destroy
+10 -2
View File
@@ -19,6 +19,10 @@ A high level description of each example is:
connections and then echos back any contents that are read from each connected
client.
* [`print_each_packet`](print_each_packet.rs) - this server will create a TCP
listener, accept connections in a loop, and put down in the stdout everything
that's read off of each TCP connection.
* [`echo-udp`](echo-udp.rs) - again your standard "echo server", except for UDP
instead of TCP. This will echo back any packets received to the original
sender.
@@ -34,7 +38,7 @@ A high level description of each example is:
in multiple terminals and use it to chat between the terminals.
* [`chat-combinator`](chat-combinator.rs) - Similar to `chat`, but this uses a
much more functional programming approch using combinators.
much more functional programming approach using combinators.
* [`proxy`](proxy.rs) - an example proxy server that will forward all connected
TCP clients to the remote address specified when starting the program.
@@ -44,9 +48,13 @@ A high level description of each example is:
spawning tasks, and finally framing a TCP connection to discrete
request/response objects.
* [`tinydb`](tinyhttp.rs) - an in-memory database which shows sharing state
* [`tinydb`](tinydb.rs) - an in-memory database which shows sharing state
between all connected clients, notably the key/value store of this database.
* [`udp-client`](udp-client.rs) - a simple `send_dgram`/`recv_dgram` example.
* [`manual-runtime`](manual-runtime.rs) - manually composing a runtime.
If you've got an example you'd like to see here, please feel free to open an
issue. Otherwise if you've got an example you'd like to add, please feel free
to make a PR!
+172
View File
@@ -0,0 +1,172 @@
//! A chat server that broadcasts a message to all connections.
//!
//! This is a line-based server which accepts connections, reads lines from
//! those connections, and broadcasts the lines to all other connected clients.
//!
//! This example is similar to chat.rs, but uses combinators and a much more
//! functional style.
//!
//! Because we are here running the reactor/executor on the same thread instead
//! of a threadpool, we can avoid full synchronization with Arc + Mutex and use
//! Rc + RefCell instead. The max performance is however limited to a CPU HW
//! thread.
//!
//! You can test this out by running:
//!
//! cargo run --example chat-combinator-current-thread
//!
//! And then in another window run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! You can run the second command in multiple windows and then chat between the
//! two, seeing the messages from the other client as they're received. For all
//! connected clients they'll all join the same room and see everyone else's
//! messages.
#![deny(warnings)]
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio::runtime::current_thread::{Runtime, TaskExecutor};
use std::cell::RefCell;
use std::collections::HashMap;
use std::env;
use std::io::BufReader;
use std::iter;
use std::rc::Rc;
fn main() -> Result<(), Box<std::error::Error>> {
let mut runtime = Runtime::new().unwrap();
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio current_thread runtime, so it will be single-
// threaded. The `Rc<RefCell<...>>` allows state to be shared across the tasks.
let connections = Rc::new(RefCell::new(HashMap::new()));
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
// Split the TcpStream into two separate handles. One handle for reading
// and one handle for writing. This lets us use separate tasks for
// reading and writing.
let (reader, writer) = stream.split();
// Create a channel for our stream, which other sockets will use to
// send us messages. Then register our address with the stream to send
// data to us.
let (tx, rx) = futures::sync::mpsc::unbounded();
let mut conns = connections.borrow_mut();
conns.insert(addr, tx);
// Define here what we do for the actual I/O. That is, read a bunch of
// lines from the socket and dispatch them while we also write any lines
// from other sockets.
let connections_inner = connections.clone();
let reader = BufReader::new(reader);
// Model the read portion of this socket by mapping an infinite
// iterator to each line off the socket. This "loop" is then
// terminated with an error once we hit EOF on the socket.
let iter = stream::iter_ok::<_, io::Error>(iter::repeat(()));
let socket_reader = iter.fold(reader, move |reader, _| {
// Read a line off the socket, failing if we're at EOF
let line = io::read_until(reader, b'\n', Vec::new());
let line = line.and_then(|(reader, vec)| {
if vec.len() == 0 {
Err(io::Error::new(io::ErrorKind::BrokenPipe, "broken pipe"))
} else {
Ok((reader, vec))
}
});
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
line.map(move |(reader, message)| {
println!("{}: {:?}", addr, message);
let mut conns = connections.borrow_mut();
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
})
});
// Whenever we receive a string on the Receiver, we write it to
// `WriteHalf<TcpStream>`.
let socket_writer = rx.fold(writer, |writer, msg| {
let amt = io::write_all(writer, msg.into_bytes());
let amt = amt.map(|(writer, _)| writer);
amt.map_err(|_| ())
});
// Now that we've got futures representing each half of the socket, we
// use the `select` combinator to wait for either half to be done to
// tear down the other. Then we spawn off the result.
let connections = connections.clone();
let socket_reader = socket_reader.map_err(|_| ());
let connection = socket_reader.map(|_| ()).select(socket_writer.map(|_| ()));
// Spawn locally a task to process the connection
TaskExecutor::current()
.spawn_local(Box::new(connection.then(move |_| {
let mut conns = connections.borrow_mut();
conns.remove(&addr);
println!("Connection {} closed.", addr);
Ok(())
})))
.unwrap();
Ok(())
})
.map_err(|err| println!("error occurred: {:?}", err));
// Spawn srv itself
runtime.spawn(srv);
// Execute server
runtime.run().unwrap();
Ok(())
}
+23 -17
View File
@@ -21,25 +21,25 @@
#![deny(warnings)]
extern crate tokio;
extern crate futures;
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::prelude::*;
use std::collections::HashMap;
use std::iter;
use std::env;
use std::io::{BufReader};
use std::io::BufReader;
use std::iter;
use std::sync::{Arc, Mutex};
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Create the TCP listener we'll accept connections on.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse().unwrap();
let addr = addr.parse()?;
let socket = TcpListener::bind(&addr).unwrap();
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// This is running on the Tokio runtime, so it will be multi-threaded. The
@@ -48,11 +48,15 @@ fn main() {
// The server task asynchronously iterates over and processes each incoming
// connection.
let srv = socket.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
let srv = socket
.incoming()
.map_err(|e| {
println!("failed to accept socket; error = {:?}", e);
e
})
.for_each(move |stream| {
// The client's socket address
let addr = stream.peer_addr().unwrap();
let addr = stream.peer_addr()?;
println!("New Connection: {}", addr);
@@ -91,9 +95,7 @@ fn main() {
// Convert the bytes we read into a string, and then send that
// string to all other connected clients.
let line = line.map(|(reader, vec)| {
(reader, String::from_utf8(vec))
});
let line = line.map(|(reader, vec)| (reader, String::from_utf8(vec)));
// Move the connection state into the closure below.
let connections = connections_inner.clone();
@@ -105,15 +107,17 @@ fn main() {
if let Ok(msg) = message {
// For each open connection except the sender, send the
// string via the channel.
let iter = conns.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
let iter = conns
.iter_mut()
.filter(|&(&k, _)| k != addr)
.map(|(_, v)| v);
for tx in iter {
tx.unbounded_send(format!("{}: {}", addr, msg)).unwrap();
}
} else {
let tx = conns.get_mut(&addr).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string()).unwrap();
tx.unbounded_send("You didn't send valid UTF-8.".to_string())
.unwrap();
}
reader
@@ -143,8 +147,10 @@ fn main() {
}));
Ok(())
});
})
.map_err(|err| println!("error occurred: {:?}", err));
// execute server
tokio::run(srv);
Ok(())
}
+38 -39
View File
@@ -4,7 +4,7 @@
//! illustrate more concepts.
//!
//! A chat server for telnet clients. After a telnet client connects, the first
//! line should contain the client's name. After that, all lines send by a
//! line should contain the client's name. After that, all lines sent by a
//! client are broadcasted to all other connected clients.
//!
//! Because the client is telnet, lines are delimited by "\r\n".
@@ -31,12 +31,12 @@ extern crate tokio;
extern crate futures;
extern crate bytes;
use bytes::{BufMut, Bytes, BytesMut};
use futures::future::{self, Either};
use futures::sync::mpsc;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use futures::future::{self, Either};
use bytes::{BytesMut, Bytes, BufMut};
use std::collections::HashMap;
use std::net::SocketAddr;
@@ -131,10 +131,7 @@ impl Shared {
impl Peer {
/// Create a new instance of `Peer`.
fn new(name: BytesMut,
state: Arc<Mutex<Shared>>,
lines: Lines) -> Peer
{
fn new(name: BytesMut, state: Arc<Mutex<Shared>>, lines: Lines) -> Peer {
// Get the client socket address
let addr = lines.socket.peer_addr().unwrap();
@@ -142,8 +139,7 @@ impl Peer {
let (tx, rx) = mpsc::unbounded();
// Add an entry for this `Peer` in the shared state map.
state.lock().unwrap()
.peers.insert(addr, tx);
state.lock().unwrap().peers.insert(addr, tx);
Peer {
name,
@@ -157,7 +153,7 @@ impl Peer {
/// This is where a connected client is managed.
///
/// A `Peer` is also a future representing completly processing the client.
/// A `Peer` is also a future representing completely processing the client.
///
/// When a `Peer` is created, the first line (representing the client's name)
/// has already been read. When the socket closes, the `Peer` future completes.
@@ -198,7 +194,7 @@ impl Future for Peer {
// though there could still be lines to read. Because we did
// not reach `Async::NotReady`, we have to notify ourselves
// in order to tell the executor to schedule the task again.
if i+1 == LINES_PER_TICK {
if i + 1 == LINES_PER_TICK {
task::current().notify();
}
}
@@ -216,9 +212,9 @@ impl Future for Peer {
if let Some(message) = line {
// Append the peer's name to the front of the line:
let mut line = self.name.clone();
line.put(": ");
line.put(&message);
line.put("\r\n");
line.extend_from_slice(b": ");
line.extend_from_slice(&message);
line.extend_from_slice(b"\r\n");
// We're using `Bytes`, which allows zero-copy clones (by
// storing the data in an Arc internally).
@@ -256,8 +252,7 @@ impl Future for Peer {
impl Drop for Peer {
fn drop(&mut self) {
self.state.lock().unwrap().peers
.remove(&self.addr);
self.state.lock().unwrap().peers.remove(&self.addr);
}
}
@@ -290,7 +285,7 @@ impl Lines {
fn poll_flush(&mut self) -> Poll<(), io::Error> {
// As long as there is buffered data to write, try to write it.
while !self.wr.is_empty() {
// Try to read some bytes from the socket
// Try to write some bytes to the socket
let n = try_ready!(self.socket.poll_write(&self.wr));
// As long as the wr is not empty, a successful write should
@@ -333,7 +328,10 @@ impl Stream for Lines {
let sock_closed = self.fill_read_buf()?.is_ready();
// Now, try finding lines
let pos = self.rd.windows(2).enumerate()
let pos = self
.rd
.windows(2)
.enumerate()
.find(|&(_, bytes)| bytes == b"\r\n")
.map(|(i, _)| i);
@@ -373,7 +371,8 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
// We use the `into_future` combinator to extract the first item from the
// lines stream. `into_future` takes a `Stream` and converts it to a future
// of `(first, rest)` where `rest` is the original stream instance.
let connection = lines.into_future()
let connection = lines
.into_future()
// `into_future` doesn't have the right error type, so map the error to
// make it work.
.map_err(|(e, _)| e)
@@ -408,10 +407,7 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
//
// This is also a future that processes the connection, only
// completing when the socket closes.
let peer = Peer::new(
name,
state,
lines);
let peer = Peer::new(name, state, lines);
// Wrap `peer` with `Either::B` to make the return type fit.
Either::B(peer)
@@ -426,7 +422,7 @@ fn process(socket: TcpStream, state: Arc<Mutex<Shared>>) {
tokio::spawn(connection);
}
pub fn main() {
pub fn main() -> Result<(), Box<std::error::Error>> {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
@@ -434,27 +430,29 @@ pub fn main() {
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
let addr = "127.0.0.1:6142".parse()?;
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
let listener = TcpListener::bind(&addr)?;
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
let server = listener
.incoming()
.for_each(move |socket| {
// Spawn a task to process the connection
process(socket, state.clone());
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
@@ -471,4 +469,5 @@ pub fn main() {
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
Ok(())
}
+71 -53
View File
@@ -16,20 +16,20 @@
#![deny(warnings)]
extern crate bytes;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate futures;
extern crate bytes;
use std::env;
use std::io::{self, Read, Write};
use std::net::SocketAddr;
use std::thread;
use tokio::prelude::*;
use futures::sync::mpsc;
use tokio::prelude::*;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Determine if we're going to run in TCP or UDP mode
let mut args = env::args().skip(1).collect::<Vec<_>>();
let tcp = match args.iter().position(|a| a == "--udp") {
@@ -41,10 +41,11 @@ fn main() {
};
// Parse what address we're going to connect to
let addr = args.first().unwrap_or_else(|| {
panic!("this program requires at least one argument")
});
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = match args.first() {
Some(addr) => addr,
None => Err("this program requires at least one argument")?,
};
let addr = addr.parse::<SocketAddr>()?;
// Right now Tokio doesn't support a handle to stdin running on the event
// loop, so we farm out that work to a separate thread. This thread will
@@ -52,15 +53,15 @@ fn main() {
// loop over a standard futures channel.
let (stdin_tx, stdin_rx) = mpsc::channel(0);
thread::spawn(|| read_stdin(stdin_tx));
let stdin_rx = stdin_rx.map_err(|_| panic!()); // errors not possible on rx
let stdin_rx = stdin_rx.map_err(|_| panic!("errors not possible on rx"));
// Now that we've got our stdin read we either set up our TCP connection or
// our UDP connection to get a stream of bytes we're going to emit to
// stdout.
let stdout = if tcp {
tcp::connect(&addr, Box::new(stdin_rx))
tcp::connect(&addr, Box::new(stdin_rx))?
} else {
udp::connect(&addr, Box::new(stdin_rx))
udp::connect(&addr, Box::new(stdin_rx))?
};
// And now with our stream of bytes to write to stdout, we execute that in
@@ -72,17 +73,16 @@ fn main() {
tokio::run({
stdout
.for_each(move |chunk| {
out.write_all(&chunk)
})
.for_each(move |chunk| out.write_all(&chunk))
.map_err(|e| println!("error reading stdout; error = {:?}", e))
});
Ok(())
}
mod codec {
use std::io;
use bytes::{BufMut, BytesMut};
use tokio_io::codec::{Encoder, Decoder};
use std::io;
use tokio::codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
///
@@ -120,19 +120,21 @@ mod codec {
mod tcp {
use tokio;
use tokio::codec::Decoder;
use tokio::net::TcpStream;
use tokio::prelude::*;
use bytes::BytesMut;
use codec::Bytes;
use std::error::Error;
use std::io;
use std::net::SocketAddr;
pub fn connect(addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
pub fn connect(
addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
let tcp = TcpStream::connect(addr);
// After the TCP connection has been established, we set up our client
@@ -150,45 +152,52 @@ mod tcp {
// You'll also note that we *spawn* the work to read stdin and write it
// to the TCP stream. This is done to ensure that happens concurrently
// with us reading data from the stream.
Box::new(tcp.map(move |stream| {
let (sink, stream) = stream.framed(Bytes).split();
let stream = Box::new(
tcp.map(move |stream| {
let (sink, stream) = Bytes.framed(stream).split();
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
tokio::spawn(stdin.forward(sink).then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
}));
stream
}).flatten_stream())
stream
})
.flatten_stream(),
);
Ok(stream)
}
}
mod udp {
use std::error::Error;
use std::io;
use std::net::SocketAddr;
use tokio;
use tokio::net::{UdpSocket, UdpFramed};
use tokio::prelude::*;
use bytes::BytesMut;
use tokio;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use codec::Bytes;
pub fn connect(&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>)
-> Box<Stream<Item = BytesMut, Error = io::Error> + Send>
{
pub fn connect(
&addr: &SocketAddr,
stdin: Box<Stream<Item = Vec<u8>, Error = io::Error> + Send>,
) -> Result<Box<Stream<Item = BytesMut, Error = io::Error> + Send>, Box<Error>> {
// We'll bind our UDP socket to a local IP/port, but for now we
// basically let the OS pick both of those.
let addr_to_bind = if addr.ip().is_ipv4() {
"0.0.0.0:0".parse().unwrap()
"0.0.0.0:0".parse()?
} else {
"[::]:0".parse().unwrap()
"[::]:0".parse()?
};
let udp = match UdpSocket::bind(&addr_to_bind) {
Ok(udp) => udp,
Err(_) => Err("failed to bind socket")?,
};
let udp = UdpSocket::bind(&addr_to_bind)
.expect("failed to bind socket");
// Like above with TCP we use an instance of `Bytes` codec to transform
// this UDP socket into a framed sink/stream which operates over
@@ -198,24 +207,34 @@ mod udp {
// All bytes from `stdin` will go to the `addr` specified in our
// argument list. Like with TCP this is spawned concurrently
tokio::spawn(stdin.map(move |chunk| {
(chunk, addr)
}).forward(sink).then(|result| {
if let Err(e) = result {
panic!("failed to write to socket: {}", e)
}
Ok(())
}));
let forward_stdin = stdin
.map(move |chunk| (chunk, addr))
.forward(sink)
.then(|result| {
if let Err(e) = result {
println!("failed to write to socket: {}", e)
}
Ok(())
});
// With UDP we could receive data from any source, so filter out
// anything coming from a different address
Box::new(stream.filter_map(move |(chunk, src)| {
let receive = stream.filter_map(move |(chunk, src)| {
if src == addr {
Some(chunk.into())
} else {
None
}
}))
});
let stream = Box::new(
future::lazy(|| {
tokio::spawn(forward_stdin);
future::ok(receive)
})
.flatten_stream(),
);
Ok(stream)
}
}
@@ -226,8 +245,7 @@ fn read_stdin(mut tx: mpsc::Sender<Vec<u8>>) {
loop {
let mut buf = vec![0; 1024];
let n = match stdin.read(&mut buf) {
Err(_) |
Ok(0) => break,
Err(_) | Ok(0) => break,
Ok(n) => n,
};
buf.truncate(n);
+9 -8
View File
@@ -1,6 +1,6 @@
//! An UDP echo server that just sends back everything that it receives.
//!
//! If you're on unix you can test this out by in one terminal executing:
//! If you're on Unix you can test this out by in one terminal executing:
//!
//! cargo run --example echo-udp
//!
@@ -16,11 +16,11 @@
extern crate futures;
extern crate tokio;
use std::{env, io};
use std::net::SocketAddr;
use std::{env, io};
use tokio::prelude::*;
use tokio::net::UdpSocket;
use tokio::prelude::*;
struct Server {
socket: UdpSocket,
@@ -50,12 +50,12 @@ impl Future for Server {
}
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
let socket = UdpSocket::bind(&addr).unwrap();
println!("Listening on: {}", socket.local_addr().unwrap());
let socket = UdpSocket::bind(&addr)?;
println!("Listening on: {}", socket.local_addr()?);
let server = Server {
socket: socket,
@@ -68,6 +68,7 @@ fn main() {
// `map_err` handles the error by logging it and maps the future to a type
// that can be spawned.
//
// `tokio::run` spanws the task on the Tokio runtime and starts running.
// `tokio::run` spawns the task on the Tokio runtime and starts running.
tokio::run(server.map_err(|e| println!("server error = {:?}", e)));
Ok(())
}
+7 -6
View File
@@ -3,7 +3,7 @@
//! This server will create a TCP listener, accept connections in a loop, and
//! write back everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
@@ -30,19 +30,19 @@ use tokio::prelude::*;
use std::env;
use std::net::SocketAddr;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr).unwrap();
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
@@ -54,7 +54,8 @@ fn main() {
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
@@ -89,7 +90,6 @@ fn main() {
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
@@ -111,4 +111,5 @@ fn main() {
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
+27 -39
View File
@@ -1,58 +1,45 @@
//! Hello world server.
//!
//! A simple server that accepts connections, writes "hello world\n", and closes
//! A simple client that opens a TCP stream, writes "hello world\n", and closes
//! the connection.
//!
//! You can test this out by running:
//!
//! cargo run --example hello_world
//! ncat -l 6142
//!
//! And then in another terminal run:
//!
//! telnet localhost 6142
//!
//! cargo run --example hello_world
#![deny(warnings)]
extern crate tokio;
use tokio::io;
use tokio::net::TcpListener;
use tokio::net::TcpStream;
use tokio::prelude::*;
pub fn main() {
let addr = "127.0.0.1:6142".parse().unwrap();
pub fn main() -> Result<(), Box<std::error::Error>> {
let addr = "127.0.0.1:6142".parse()?;
// Bind a TCP listener to the socket address.
// Open a TCP stream to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
// The server task asynchronously iterates over and processes each
// incoming connection.
let server = listener.incoming().for_each(|socket| {
println!("accepted socket; addr={:?}", socket.peer_addr().unwrap());
let connection = io::write_all(socket, "hello world\n")
.then(|res| {
println!("wrote message; success={:?}", res.is_ok());
// Note that this is the Tokio TcpStream, which is fully async.
let client = TcpStream::connect(&addr)
.and_then(|stream| {
println!("created stream");
io::write_all(stream, "hello world\n").then(|result| {
println!("wrote to stream; success={:?}", result.is_ok());
Ok(())
});
// Spawn a new task that processes the socket:
tokio::spawn(connection);
Ok(())
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("accept error = {:?}", err);
});
println!("server running on localhost:6142");
})
})
.map_err(|err| {
// All tasks must have an `Error` type of `()`. This forces error
// handling and helps avoid silencing failures.
//
// In our example, we are only going to log the error to STDOUT.
println!("connection error = {:?}", err);
});
// Start the Tokio runtime.
//
@@ -63,8 +50,9 @@ pub fn main() {
// This function blocks until the runtime reaches an idle state. Idle is
// defined as all spawned tasks have completed and all I/O resources (TCP
// sockets in our case) have been dropped.
//
// In our example, we have not defined a shutdown strategy, so this will
// block until `ctrl-c` is pressed at the terminal.
tokio::run(server);
println!("About to create the stream and write to it...");
tokio::run(client);
println!("Stream has been created and written to.");
Ok(())
}
+87
View File
@@ -0,0 +1,87 @@
//! An example how to manually assemble a runtime and run some tasks on it.
//!
//! This is closer to the single-threaded runtime than the default tokio one, as it is simpler to
//! grasp. There are conceptually similar, but the multi-threaded one would be more code. If you
//! just want to *use* a single-threaded runtime, use the one provided by tokio directly
//! (`tokio::runtime::current_thread::Runtime::new()`. This is a demonstration only.
//!
//! Note that the error handling is a bit left out. Also, the `run` could be modified to return the
//! result of the provided future.
extern crate futures;
extern crate tokio;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_timer;
use std::io::Error as IoError;
use std::time::{Duration, Instant};
use futures::{future, Future};
use tokio_current_thread::CurrentThread;
use tokio_reactor::Reactor;
use tokio_timer::timer::{self, Timer};
/// Creates a "runtime".
///
/// This is similar to running `tokio::runtime::current_thread::Runtime::new()`.
fn run<F: Future<Item = (), Error = ()>>(f: F) -> Result<(), IoError> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new(reactor);
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor generate some new stimuli for the
// futures to continue in their life.
let mut executor = CurrentThread::new_with_park(timer);
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the current single-threaded
// executor when used. This is a trick, because we need two mutable references to the
// executor (one to run the provided future, another to install as the default one). We
// use the fake one here as the default one.
let mut default_executor = tokio_current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
// Run the provided future
executor.block_on(f).unwrap();
// Run all the other futures that are still left in the executor
executor.run().unwrap();
});
});
});
Ok(())
}
fn main() -> Result<(), Box<std::error::Error>> {
run(future::lazy(|| {
// Here comes the application logic. It can spawn further tasks by tokio_current_thread::spawn().
// It also can use the default reactor and create timeouts.
// Connect somewhere. And then do nothing with it. Yes, useless.
//
// This will use the default reactor which runs in the current thread.
let connect = tokio::net::TcpStream::connect(&"127.0.0.1:53".parse().unwrap())
.map(|_| println!("Connected"))
.map_err(|e| println!("Failed to connect: {}", e));
// We can spawn it without requiring Send. This would panic if we run it outside of the
// `run` (or outside of anything else)
tokio_current_thread::spawn(connect);
// We can also create timeouts.
let deadline = tokio::timer::Delay::new(Instant::now() + Duration::from_secs(5))
.map(|()| println!("5 seconds are over"))
.map_err(|e| println!("Failed to wait: {}", e));
// We can spawn on the default executor, which is also the local one.
tokio::executor::spawn(deadline);
Ok(())
}))?;
Ok(())
}
+150
View File
@@ -0,0 +1,150 @@
//! A "print-each-packet" server with Tokio
//!
//! This server will create a TCP listener, accept connections in a loop, and
//! put down in the stdout everything that's read off of each TCP connection.
//!
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
//! To see this server in action, you can run this in one terminal:
//!
//! cargo run --example print\_each\_packet
//!
//! and in another terminal you can run:
//!
//! cargo run --example connect 127.0.0.1:8080
//!
//! Each line you type in to the `connect` terminal should be written to terminal!
//!
//! Minimal js example:
//!
//! ```js
//! var net = require("net");
//!
//! var listenPort = 8080;
//!
//! var server = net.createServer(function (socket) {
//! socket.on("data", function (bytes) {
//! console.log("bytes", bytes);
//! });
//!
//! socket.on("end", function() {
//! console.log("Socket received FIN packet and closed connection");
//! });
//! socket.on("error", function (error) {
//! console.log("Socket closed with error", error);
//! });
//!
//! socket.on("close", function (with_error) {
//! if (with_error) {
//! console.log("Socket closed with result: Err(SomeError)");
//! } else {
//! console.log("Socket closed with result: Ok(())");
//! }
//! });
//!
//! });
//!
//! server.listen(listenPort);
//!
//! console.log("Listening on:", listenPort);
//! ```
//!
#![deny(warnings)]
extern crate tokio;
extern crate tokio_codec;
use tokio::codec::Decoder;
use tokio::net::TcpListener;
use tokio::prelude::*;
use tokio_codec::BytesCodec;
use std::env;
use std::net::SocketAddr;
fn main() -> Result<(), Box<std::error::Error>> {
// Allow passing an address to listen on as the first argument of this
// program, but otherwise we'll just set up our TCP listener on
// 127.0.0.1:8080 for connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>()?;
// Next up we create a TCP listener which will listen for incoming
// connections. This TCP listener is bound to the address we determined
// above and must be associated with an event loop, so we pass in a handle
// to our event loop. After the socket's created we inform that we're ready
// to go and start accepting connections.
let socket = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
// Here we convert the `TcpListener` to a stream of incoming connections
// with the `incoming` method. We then define how to process each element in
// the stream with the `for_each` method.
//
// This combinator, defined on the `Stream` trait, will allow us to define a
// computation to happen for all items on the stream (in this case TCP
// connections made to the server). The return value of the `for_each`
// method is itself a future representing processing the entire stream of
// connections, and ends up being our server.
let done = socket
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(move |socket| {
// Once we're inside this closure this represents an accepted client
// from our server. The `socket` is the client connection (similar to
// how the standard library operates).
//
// We're parsing each socket with the `BytesCodec` included in `tokio_io`,
// and then we `split` each codec into the reader/writer halves.
//
// See https://docs.rs/tokio-codec/0.1/src/tokio_codec/bytes_codec.rs.html
let framed = BytesCodec::new().framed(socket);
let (_writer, reader) = framed.split();
let processor = reader
.for_each(|bytes| {
println!("bytes: {:?}", bytes);
Ok(())
})
// After our copy operation is complete we just print out some helpful
// information.
.and_then(|()| {
println!("Socket received FIN packet and closed connection");
Ok(())
})
.or_else(|err| {
println!("Socket closed with error: {:?}", err);
// We have to return the error to catch it in the next ``.then` call
Err(err)
})
.then(|result| {
println!("Socket closed with result: {:?}", result);
Ok(())
});
// And this is where much of the magic of this server happens. We
// crucially want all clients to make progress concurrently, rather than
// blocking one on completion of another. To achieve this we use the
// `tokio::spawn` function to execute the work in the background.
//
// This function will transfer ownership of the future (`msg` in this
// case) to the Tokio runtime thread pool that. The thread pool will
// drive the future to completion.
//
// Essentially here we're executing a new task to run concurrently,
// which will allow all of our clients to be processed concurrently.
tokio::spawn(processor)
});
// And finally now that we've define what our server is, we run it!
//
// This starts the Tokio runtime, spawns the server task, and blocks the
// current thread until all tasks complete execution. Since the `done` task
// never completes (it just keeps accepting sockets), `tokio::run` blocks
// forever (until ctrl-c is pressed).
tokio::run(done);
Ok(())
}
+23 -21
View File
@@ -1,7 +1,7 @@
//! A proxy that forwards data to another server and forwards that server's
//! responses back to clients.
//!
//! Because the Tokio runtime uses a thread poool, each TCP connection is
//! Because the Tokio runtime uses a thread pool, each TCP connection is
//! processed concurrently with all other TCP connections across multiple
//! threads.
//!
@@ -24,28 +24,29 @@
extern crate tokio;
use std::sync::{Arc, Mutex};
use std::env;
use std::net::{Shutdown, SocketAddr};
use std::io::{self, Read, Write};
use std::net::{Shutdown, SocketAddr};
use std::sync::{Arc, Mutex};
use tokio::io::{copy, shutdown};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let listen_addr = env::args().nth(1).unwrap_or("127.0.0.1:8081".to_string());
let listen_addr = listen_addr.parse::<SocketAddr>().unwrap();
let listen_addr = listen_addr.parse::<SocketAddr>()?;
let server_addr = env::args().nth(2).unwrap_or("127.0.0.1:8080".to_string());
let server_addr = server_addr.parse::<SocketAddr>().unwrap();
let server_addr = server_addr.parse::<SocketAddr>()?;
// Create a TCP listener which will listen for incoming connections.
let socket = TcpListener::bind(&listen_addr).unwrap();
let socket = TcpListener::bind(&listen_addr)?;
println!("Listening on: {}", listen_addr);
println!("Proxying to: {}", server_addr);
let done = socket.incoming()
let done = socket
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |client| {
let server = TcpStream::connect(&server_addr);
@@ -68,25 +69,25 @@ fn main() {
// After the copy is done we indicate to the remote side that we've
// finished by shutting down the connection.
let client_to_server = copy(client_reader, server_writer)
.and_then(|(n, _, server_writer)| {
shutdown(server_writer).map(move |_| n)
});
.and_then(|(n, _, server_writer)| shutdown(server_writer).map(move |_| n));
let server_to_client = copy(server_reader, client_writer)
.and_then(|(n, _, client_writer)| {
shutdown(client_writer).map(move |_| n)
});
.and_then(|(n, _, client_writer)| shutdown(client_writer).map(move |_| n));
client_to_server.join(server_to_client)
});
let msg = amounts.map(move |(from_client, from_server)| {
println!("client wrote {} bytes and received {} bytes",
from_client, from_server);
}).map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
let msg = amounts
.map(move |(from_client, from_server)| {
println!(
"client wrote {} bytes and received {} bytes",
from_client, from_server
);
})
.map_err(|e| {
// Don't panic. Maybe the client just disconnected too soon.
println!("error: {}", e);
});
tokio::spawn(msg);
@@ -94,6 +95,7 @@ fn main() {
});
tokio::run(done);
Ok(())
}
// This is a custom type used to have a custom implementation of the
+49 -28
View File
@@ -44,8 +44,8 @@
extern crate tokio;
use std::collections::HashMap;
use std::io::BufReader;
use std::env;
use std::io::BufReader;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
@@ -56,7 +56,7 @@ use tokio::prelude::*;
/// The in-memory database shared amongst all clients.
///
/// This database will be shared via `Arc`, so to mutate the internal map we're
/// also going to use a `RefCell` for interior mutability.
/// going to use a `Mutex` for interior mutability.
struct Database {
map: Mutex<HashMap<String, String>>,
}
@@ -69,17 +69,26 @@ enum Request {
/// Responses to the `Request` commands above
enum Response {
Value { key: String, value: String },
Set { key: String, value: String, previous: Option<String> },
Error { msg: String },
Value {
key: String,
value: String,
},
Set {
key: String,
value: String,
previous: Option<String>,
},
Error {
msg: String,
},
}
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Parse the address we're going to run this server on
// and set up our TCP listener to accept connections.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let listener = TcpListener::bind(&addr).expect("failed to bind");
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).map_err(|_| "failed to bind")?;
println!("Listening on: {}", addr);
// Create the shared state of this server that will be shared amongst all
@@ -93,7 +102,8 @@ fn main() {
map: Mutex::new(initial_db),
});
let done = listener.incoming()
let done = listener
.incoming()
.map_err(|e| println!("error accepting socket; error = {:?}", e))
.for_each(move |socket| {
// As with many other small examples, the first thing we'll do is
@@ -124,15 +134,22 @@ fn main() {
let mut db = db.map.lock().unwrap();
match request {
Request::Get { key } => {
match db.get(&key) {
Some(value) => Response::Value { key, value: value.clone() },
None => Response::Error { msg: format!("no key {}", key) },
}
}
Request::Get { key } => match db.get(&key) {
Some(value) => Response::Value {
key,
value: value.clone(),
},
None => Response::Error {
msg: format!("no key {}", key),
},
},
Request::Set { key, value } => {
let previous = db.insert(key.clone(), value.clone());
Response::Set { key, value, previous }
Response::Set {
key,
value,
previous,
}
}
}
});
@@ -156,6 +173,7 @@ fn main() {
});
tokio::run(done);
Ok(())
}
impl Request {
@@ -168,9 +186,11 @@ impl Request {
None => return Err(format!("GET must be followed by a key")),
};
if parts.next().is_some() {
return Err(format!("GET's key must not be followed by anything"))
return Err(format!("GET's key must not be followed by anything"));
}
Ok(Request::Get { key: key.to_string() })
Ok(Request::Get {
key: key.to_string(),
})
}
Some("SET") => {
let key = match parts.next() {
@@ -181,7 +201,10 @@ impl Request {
Some(value) => value,
None => return Err(format!("SET needs a value")),
};
Ok(Request::Set { key: key.to_string(), value: value.to_string() })
Ok(Request::Set {
key: key.to_string(),
value: value.to_string(),
})
}
Some(cmd) => Err(format!("unknown command: {}", cmd)),
None => Err(format!("empty input")),
@@ -192,15 +215,13 @@ impl Request {
impl Response {
fn serialize(&self) -> String {
match *self {
Response::Value { ref key, ref value } => {
format!("{} = {}", key, value)
}
Response::Set { ref key, ref value, ref previous } => {
format!("set {} = `{}`, previous: {:?}", key, value, previous)
}
Response::Error { ref msg } => {
format!("error: {}", msg)
}
Response::Value { ref key, ref value } => format!("{} = {}", key, value),
Response::Set {
ref key,
ref value,
ref previous,
} => format!("set {} = `{}`, previous: {:?}", key, value, previous),
Response::Error { ref msg } => format!("error: {}", msg),
}
}
}
+77 -61
View File
@@ -1,9 +1,9 @@
//! A "tiny" example of HTTP request/response handling using just tokio-core
//! A "tiny" example of HTTP request/response handling using transports.
//!
//! This example is intended for *learning purposes* to see how various pieces
//! hook up together and how HTTP can get up and running. Note that this example
//! is written with the restriction that it *can't* use any "big" library other
//! than tokio-core, if you'd like a "real world" HTTP library you likely want a
//! than Tokio, if you'd like a "real world" HTTP library you likely want a
//! crate like Hyper.
//!
//! Code here is based on the `echo-threads` example and implements two paths,
@@ -23,55 +23,55 @@ extern crate time;
extern crate tokio;
extern crate tokio_io;
use std::{env, fmt, io};
use std::net::SocketAddr;
use std::{env, fmt, io};
use tokio::net::{TcpStream, TcpListener};
use tokio::codec::{Decoder, Encoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use tokio_io::codec::{Encoder, Decoder};
use bytes::BytesMut;
use http::header::HeaderValue;
use http::{Request, Response, StatusCode};
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
// Parse the arguments, bind the TCP socket we'll be listening to, spin up
// our worker threads, and start shipping sockets to those worker threads.
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
let addr = addr.parse::<SocketAddr>()?;
let listener = TcpListener::bind(&addr).expect("failed to bind");
let listener = TcpListener::bind(&addr)?;
println!("Listening on: {}", addr);
tokio::run({
listener.incoming()
listener
.incoming()
.map_err(|e| println!("failed to accept socket; error = {:?}", e))
.for_each(|socket| {
process(socket);
Ok(())
})
});
Ok(())
}
fn process(socket: TcpStream) {
let (tx, rx) = socket
let (tx, rx) =
// Frame the socket using the `Http` protocol. This maps the TCP socket
// to a Stream + Sink of HTTP frames.
.framed(Http)
Http.framed(socket)
// This splits a single `Stream + Sink` value into two separate handles
// that can be used independently (even on different tasks or threads).
.split();
// Map all requests into responses and send them back to the client.
let task = tx.send_all(rx.and_then(respond))
.then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
let task = tx.send_all(rx.and_then(respond)).then(|res| {
if let Err(e) = res {
println!("failed to process connection; error = {:?}", e);
}
Ok(())
});
Ok(())
});
// Spawn the task that handles the connection.
tokio::spawn(task);
@@ -82,31 +82,37 @@ fn process(socket: TcpStream) {
/// This function is a map from and HTTP request to a future of a response and
/// represents the various handling a server might do. Currently the contents
/// here are pretty uninteresting.
fn respond(req: Request<()>)
-> Box<Future<Item = Response<String>, Error = io::Error> + Send>
{
let mut ret = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
ret.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
"/json" => {
ret.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
fn respond(req: Request<()>) -> Box<Future<Item = Response<String>, Error = io::Error> + Send> {
let f = future::lazy(move || {
let mut response = Response::builder();
let body = match req.uri().path() {
"/plaintext" => {
response.header("Content-Type", "text/plain");
"Hello, World!".to_string()
}
serde_json::to_string(&Message { message: "Hello, World!" })
.unwrap()
}
_ => {
ret.status(StatusCode::NOT_FOUND);
String::new()
}
};
Box::new(future::ok(ret.body(body).unwrap()))
"/json" => {
response.header("Content-Type", "application/json");
#[derive(Serialize)]
struct Message {
message: &'static str,
}
serde_json::to_string(&Message {
message: "Hello, World!",
})?
}
_ => {
response.status(StatusCode::NOT_FOUND);
String::new()
}
};
let response = response
.body(body)
.map_err(|err| io::Error::new(io::ErrorKind::Other, err))?;
Ok(response)
});
Box::new(f)
}
struct Http;
@@ -120,12 +126,19 @@ impl Encoder for Http {
fn encode(&mut self, item: Response<String>, dst: &mut BytesMut) -> io::Result<()> {
use std::fmt::Write;
write!(BytesWrite(dst), "\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
", item.status(), item.body().len(), date::now()).unwrap();
write!(
BytesWrite(dst),
"\
HTTP/1.1 {}\r\n\
Server: Example\r\n\
Content-Length: {}\r\n\
Date: {}\r\n\
",
item.status(),
item.body().len(),
date::now()
)
.unwrap();
for (k, v) in item.headers() {
dst.extend_from_slice(k.as_str().as_bytes());
@@ -194,13 +207,18 @@ impl Decoder for Http {
headers[i] = Some((k, v));
}
(toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt)
(
toslice(r.method.unwrap().as_bytes()),
toslice(r.path.unwrap().as_bytes()),
r.version.unwrap(),
amt,
)
};
if version != 1 {
return Err(io::Error::new(io::ErrorKind::Other, "only HTTP/1.1 accepted"))
return Err(io::Error::new(
io::ErrorKind::Other,
"only HTTP/1.1 accepted",
));
}
let data = src.split_to(amt).freeze();
let mut ret = Request::builder();
@@ -212,15 +230,13 @@ impl Decoder for Http {
Some((ref k, ref v)) => (k, v),
None => break,
};
let value = unsafe {
HeaderValue::from_shared_unchecked(data.slice(v.0, v.1))
};
let value = unsafe { HeaderValue::from_shared_unchecked(data.slice(v.0, v.1)) };
ret.header(&data[k.0..k.1], value);
}
let req = ret.body(()).map_err(|e| {
io::Error::new(io::ErrorKind::Other, e)
})?;
let req = ret
.body(())
.map_err(|e| io::Error::new(io::ErrorKind::Other, e))?;
Ok(Some(req))
}
}
@@ -274,7 +290,7 @@ mod date {
LAST.with(|cache| {
let mut cache = cache.borrow_mut();
let now = time::get_time();
if now > cache.next_update {
if now >= cache.next_update {
cache.update(now);
}
f.write_str(cache.buffer())
+70
View File
@@ -0,0 +1,70 @@
//! A UDP client that just sends everything it gets via `stdio` in a single datagram, and then
//! waits for a reply.
//!
//! For the reasons of simplicity data from `stdio` is read until `EOF` in a blocking manner.
//!
//! You can test this out by running an echo server:
//!
//! ```
//! $ cargo run --example echo-udp -- 127.0.0.1:8080
//! ```
//!
//! and running the client in another terminal:
//!
//! ```
//! $ cargo run --example udp-client
//! ```
//!
//! You can optionally provide any custom endpoint address for the client:
//!
//! ```
//! $ cargo run --example udp-client -- 127.0.0.1:8080
//! ```
//!
//! Don't forget to pass `EOF` to the standard input of the client!
//!
//! Please mind that since the UDP protocol doesn't have any capabilities to detect a broken
//! connection the server needs to be run first, otherwise the client will block forever.
extern crate futures;
extern crate tokio;
use std::env;
use std::io::stdin;
use std::net::SocketAddr;
use tokio::net::UdpSocket;
use tokio::prelude::*;
fn get_stdin_data() -> Result<Vec<u8>, Box<std::error::Error>> {
let mut buf = Vec::new();
stdin().read_to_end(&mut buf)?;
Ok(buf)
}
fn main() -> Result<(), Box<std::error::Error>> {
let remote_addr: SocketAddr = env::args()
.nth(1)
.unwrap_or("127.0.0.1:8080".into())
.parse()?;
// We use port 0 to let the operating system allocate an available port for us.
let local_addr: SocketAddr = if remote_addr.is_ipv4() {
"0.0.0.0:0"
} else {
"[::]:0"
}
.parse()?;
let socket = UdpSocket::bind(&local_addr)?;
const MAX_DATAGRAM_SIZE: usize = 65_507;
socket
.send_dgram(get_stdin_data()?, &remote_addr)
.and_then(|(socket, _)| socket.recv_dgram(vec![0u8; MAX_DATAGRAM_SIZE]))
.map(|(_, data, len, _)| {
println!(
"Received {} bytes:\n{}",
len,
String::from_utf8_lossy(&data[..len])
)
})
.wait()?;
Ok(())
}
+12 -10
View File
@@ -8,27 +8,28 @@
#![deny(warnings)]
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
use std::net::SocketAddr;
use tokio::net::{UdpFramed, UdpSocket};
use tokio::prelude::*;
use tokio::net::{UdpSocket, UdpFramed};
use tokio_io::codec::BytesCodec;
use tokio_codec::BytesCodec;
fn main() {
fn main() -> Result<(), Box<std::error::Error>> {
let _ = env_logger::init();
let addr: SocketAddr = "127.0.0.1:0".parse().unwrap();
let addr: SocketAddr = "127.0.0.1:0".parse()?;
// Bind both our sockets and then figure out what ports we got.
let a = UdpSocket::bind(&addr).unwrap();
let b = UdpSocket::bind(&addr).unwrap();
let b_addr = b.local_addr().unwrap();
let a = UdpSocket::bind(&addr)?;
let b = UdpSocket::bind(&addr)?;
let b_addr = b.local_addr()?;
// We're parsing each socket with the `LineCodec` defined above, and then we
// We're parsing each socket with the `BytesCodec` included in `tokio_io`, and then we
// `split` each codec into the sink/stream halves.
let (a_sink, a_stream) = UdpFramed::new(a, BytesCodec::new()).split();
let (b_sink, b_stream) = UdpFramed::new(b, BytesCodec::new()).split();
@@ -60,4 +61,5 @@ fn main() {
.map(|_| ())
.map_err(|e| println!("error = {:?}", e))
});
Ok(())
}
+48
View File
@@ -0,0 +1,48 @@
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{Poll, Waker};
fn map_ok<T: StdFuture>(future: T) -> impl StdFuture<Output = Result<(), ()>> {
MapOk(future)
}
struct MapOk<T>(T);
impl<T> MapOk<T> {
fn future<'a>(self: Pin<&'a mut Self>) -> Pin<&'a mut T> {
unsafe { Pin::map_unchecked_mut(self, |x| &mut x.0) }
}
}
impl<T: StdFuture> StdFuture for MapOk<T> {
type Output = Result<(), ()>;
fn poll(self: Pin<&mut Self>, waker: &Waker) -> Poll<Self::Output> {
match self.future().poll(waker) {
Poll::Ready(_) => Poll::Ready(Ok(())),
Poll::Pending => Poll::Pending,
}
}
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where
F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where
F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
+15
View File
@@ -0,0 +1,15 @@
//! A configurable source of time.
//!
//! This module provides the [`now`][n] function, which returns an `Instant`
//! representing "now". The source of time used by this function is configurable
//! (via the [`tokio-timer`] crate) and allows mocking out the source of time in
//! tests or performing caching operations to reduce the number of syscalls.
//!
//! Note that, because the source of time is configurable, it is possible to
//! observe non-monotonic behavior when calling [`now`][n] from different
//! executors.
//!
//! [n]: fn.now.html
//! [`tokio-timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/clock/index.html
pub use tokio_timer::clock::now;
+982
View File
@@ -0,0 +1,982 @@
//! Frame a stream of bytes based on a length prefix
//!
//! Many protocols delimit their frames by prefacing frame data with a
//! frame head that specifies the length of the frame. The
//! `length_delimited` module provides utilities for handling the length
//! based framing. This allows the consumer to work with entire frames
//! without having to worry about buffering or other framing logic.
//!
//! # Getting started
//!
//! If implementing a protocol from scratch, using length delimited framing
//! is an easy way to get started. [`LengthDelimitedCodec::new()`] will
//! return a length delimited codec using default configuration values.
//! This can then be used to construct a framer to adapt a full-duplex
//! byte stream into a stream of frames.
//!
//! ```
//! # extern crate tokio;
//! use tokio::io::{AsyncRead, AsyncWrite};
//! use tokio::codec::*;
//!
//! fn bind_transport<T: AsyncRead + AsyncWrite>(io: T)
//! -> Framed<T, LengthDelimitedCodec>
//! {
//! Framed::new(io, LengthDelimitedCodec::new())
//! }
//! # pub fn main() {}
//! ```
//!
//! The returned transport implements `Sink + Stream` for `BytesMut`. It
//! encodes the frame with a big-endian `u32` header denoting the frame
//! payload length:
//!
//! ```text
//! +----------+--------------------------------+
//! | len: u32 | frame payload |
//! +----------+--------------------------------+
//! ```
//!
//! Specifically, given the following:
//!
//! ```
//! # extern crate tokio;
//! # extern crate bytes;
//! # extern crate futures;
//! #
//! use tokio::io::{AsyncRead, AsyncWrite};
//! use tokio::codec::*;
//! use bytes::Bytes;
//! use futures::{Sink, Future};
//!
//! fn write_frame<T: AsyncRead + AsyncWrite>(io: T) -> Result<(), Box<std::error::Error>> {
//! let mut transport = Framed::new(io, LengthDelimitedCodec::new());
//! let frame = Bytes::from("hello world");
//!
//! transport.send(frame).wait()?;
//! Ok(())
//! }
//! #
//! # pub fn main() {}
//! ```
//!
//! The encoded frame will look like this:
//!
//! ```text
//! +---- len: u32 ----+---- data ----+
//! | \x00\x00\x00\x0b | hello world |
//! +------------------+--------------+
//! ```
//!
//! # Decoding
//!
//! [`FramedRead`] adapts an [`AsyncRead`] into a `Stream` of [`BytesMut`],
//! such that each yielded [`BytesMut`] value contains the contents of an
//! entire frame. There are many configuration parameters enabling
//! [`FramedRead`] to handle a wide range of protocols. Here are some
//! examples that will cover the various options at a high level.
//!
//! ## Example 1
//!
//! The following will parse a `u16` length field at offset 0, including the
//! frame head in the yielded `BytesMut`.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(0) // default value
//! .num_skip(0) // Do not strip frame header
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +-- len ---+--- Payload ---+
//! | \x00\x0B | Hello world | --> | \x00\x0B | Hello world |
//! +----------+---------------+ +----------+---------------+
//! ```
//!
//! The value of the length field is 11 (`\x0B`) which represents the length
//! of the payload, `hello world`. By default, [`FramedRead`] assumes that
//! the length field represents the number of bytes that **follows** the
//! length field. Thus, the entire frame has a length of 13: 2 bytes for the
//! frame head + 11 bytes for the payload.
//!
//! ## Example 2
//!
//! The following will parse a `u16` length field at offset 0, omitting the
//! frame head in the yielded `BytesMut`.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(0) // default value
//! // `num_skip` is not needed, the default is to skip
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +--- Payload ---+
//! | \x00\x0B | Hello world | --> | Hello world |
//! +----------+---------------+ +---------------+
//! ```
//!
//! This is similar to the first example, the only difference is that the
//! frame head is **not** included in the yielded `BytesMut` value.
//!
//! ## Example 3
//!
//! The following will parse a `u16` length field at offset 0, including the
//! frame head in the yielded `BytesMut`. In this case, the length field
//! **includes** the frame head length.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(0) // default value
//! .length_field_length(2)
//! .length_adjustment(-2) // size of head
//! .num_skip(0)
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT DECODED
//! +-- len ---+--- Payload ---+ +-- len ---+--- Payload ---+
//! | \x00\x0D | Hello world | --> | \x00\x0D | Hello world |
//! +----------+---------------+ +----------+---------------+
//! ```
//!
//! In most cases, the length field represents the length of the payload
//! only, as shown in the previous examples. However, in some protocols the
//! length field represents the length of the whole frame, including the
//! head. In such cases, we specify a negative `length_adjustment` to adjust
//! the value provided in the frame head to represent the payload length.
//!
//! ## Example 4
//!
//! The following will parse a 3 byte length field at offset 0 in a 5 byte
//! frame head, including the frame head in the yielded `BytesMut`.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(0) // default value
//! .length_field_length(3)
//! .length_adjustment(2) // remaining head
//! .num_skip(0)
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +---- len -----+- head -+--- Payload ---+
//! | \x00\x00\x0B | \xCAFE | Hello world |
//! +--------------+--------+---------------+
//!
//! DECODED
//! +---- len -----+- head -+--- Payload ---+
//! | \x00\x00\x0B | \xCAFE | Hello world |
//! +--------------+--------+---------------+
//! ```
//!
//! A more advanced example that shows a case where there is extra frame
//! head data between the length field and the payload. In such cases, it is
//! usually desirable to include the frame head as part of the yielded
//! `BytesMut`. This lets consumers of the length delimited framer to
//! process the frame head as needed.
//!
//! The positive `length_adjustment` value lets `FramedRead` factor in the
//! additional head into the frame length calculation.
//!
//! ## Example 5
//!
//! The following will parse a `u16` length field at offset 1 of a 4 byte
//! frame head. The first byte and the length field will be omitted from the
//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be
//! included.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(1) // length of hdr1
//! .length_field_length(2)
//! .length_adjustment(1) // length of hdr2
//! .num_skip(3) // length of hdr1 + LEN
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+
//! | \xCA | \x00\x0B | \xFE | Hello world |
//! +--------+----------+--------+---------------+
//!
//! DECODED
//! +- hdr2 -+--- Payload ---+
//! | \xFE | Hello world |
//! +--------+---------------+
//! ```
//!
//! The length field is situated in the middle of the frame head. In this
//! case, the first byte in the frame head could be a version or some other
//! identifier that is not needed for processing. On the other hand, the
//! second half of the head is needed.
//!
//! `length_field_offset` indicates how many bytes to skip before starting
//! to read the length field. `length_adjustment` is the number of bytes to
//! skip starting at the end of the length field. In this case, it is the
//! second half of the head.
//!
//! ## Example 6
//!
//! The following will parse a `u16` length field at offset 1 of a 4 byte
//! frame head. The first byte and the length field will be omitted from the
//! yielded `BytesMut`, but the trailing 2 bytes of the frame head will be
//! included. In this case, the length field **includes** the frame head
//! length.
//!
//! ```
//! # extern crate tokio;
//! # use tokio::io::AsyncRead;
//! # use tokio::codec::length_delimited;
//! # fn bind_read<T: AsyncRead>(io: T) {
//! length_delimited::Builder::new()
//! .length_field_offset(1) // length of hdr1
//! .length_field_length(2)
//! .length_adjustment(-3) // length of hdr1 + LEN, negative
//! .num_skip(3)
//! .new_read(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! The following frame will be decoded as such:
//!
//! ```text
//! INPUT
//! +- hdr1 -+-- len ---+- hdr2 -+--- Payload ---+
//! | \xCA | \x00\x0F | \xFE | Hello world |
//! +--------+----------+--------+---------------+
//!
//! DECODED
//! +- hdr2 -+--- Payload ---+
//! | \xFE | Hello world |
//! +--------+---------------+
//! ```
//!
//! Similar to the example above, the difference is that the length field
//! represents the length of the entire frame instead of just the payload.
//! The length of `hdr1` and `len` must be counted in `length_adjustment`.
//! Note that the length of `hdr2` does **not** need to be explicitly set
//! anywhere because it already is factored into the total frame length that
//! is read from the byte stream.
//!
//! # Encoding
//!
//! [`FramedWrite`] adapts an [`AsyncWrite`] into a `Sink` of [`BytesMut`],
//! such that each submitted [`BytesMut`] is prefaced by a length field.
//! There are fewer configuration options than [`FramedRead`]. Given
//! protocols that have more complex frame heads, an encoder should probably
//! be written by hand using [`Encoder`].
//!
//! Here is a simple example, given a `FramedWrite` with the following
//! configuration:
//!
//! ```
//! # extern crate tokio;
//! # extern crate bytes;
//! # use tokio::io::AsyncWrite;
//! # use tokio::codec::length_delimited;
//! # use bytes::BytesMut;
//! # fn write_frame<T: AsyncWrite>(io: T) {
//! # let _ =
//! length_delimited::Builder::new()
//! .length_field_length(2)
//! .new_write(io);
//! # }
//! # pub fn main() {}
//! ```
//!
//! A payload of `hello world` will be encoded as:
//!
//! ```text
//! +- len: u16 -+---- data ----+
//! | \x00\x0b | hello world |
//! +------------+--------------+
//! ```
//!
//! [`LengthDelimitedCodec::new()`]: struct.LengthDelimitedCodec.html#method.new
//! [`FramedRead`]: struct.FramedRead.html
//! [`FramedWrite`]: struct.FramedWrite.html
//! [`AsyncRead`]: ../../trait.AsyncRead.html
//! [`AsyncWrite`]: ../../trait.AsyncWrite.html
//! [`Encoder`]: ../trait.Encoder.html
//! [`BytesMut`]: https://docs.rs/bytes/0.4/bytes/struct.BytesMut.html
use {
codec::{Decoder, Encoder, Framed, FramedRead, FramedWrite},
io::{AsyncRead, AsyncWrite},
};
use bytes::{Buf, BufMut, Bytes, BytesMut, IntoBuf};
use std::error::Error as StdError;
use std::io::{self, Cursor};
use std::{cmp, fmt};
/// Configure length delimited `LengthDelimitedCodec`s.
///
/// `Builder` enables constructing configured length delimited codecs. Note
/// that not all configuration settings apply to both encoding and decoding. See
/// the documentation for specific methods for more detail.
#[derive(Debug, Clone, Copy)]
pub struct Builder {
// Maximum frame length
max_frame_len: usize,
// Number of bytes representing the field length
length_field_len: usize,
// Number of bytes in the header before the length field
length_field_offset: usize,
// Adjust the length specified in the header field by this amount
length_adjustment: isize,
// Total number of bytes to skip before reading the payload, if not set,
// `length_field_len + length_field_offset`
num_skip: Option<usize>,
// Length field byte order (little or big endian)
length_field_is_big_endian: bool,
}
/// An error when the number of bytes read is more than max frame length.
pub struct FrameTooBig {
_priv: (),
}
/// A codec for frames delimited by a frame head specifying their lengths.
///
/// This allows the consumer to work with entire frames without having to worry
/// about buffering or other framing logic.
///
/// See [module level] documentation for more detail.
///
/// [module level]: index.html
#[derive(Debug)]
pub struct LengthDelimitedCodec {
// Configuration values
builder: Builder,
// Read state
state: DecodeState,
}
#[derive(Debug, Clone, Copy)]
enum DecodeState {
Head,
Data(usize),
}
// ===== impl LengthDelimitedCodec ======
impl LengthDelimitedCodec {
/// Creates a new `LengthDelimitedCodec` with the default configuration values.
pub fn new() -> Self {
Self {
builder: Builder::new(),
state: DecodeState::Head,
}
}
/// Returns the current max frame setting
///
/// This is the largest size this codec will accept from the wire. Larger
/// frames will be rejected.
pub fn max_frame_length(&self) -> usize {
self.builder.max_frame_len
}
/// Updates the max frame setting.
///
/// The change takes effect the next time a frame is decoded. In other
/// words, if a frame is currently in process of being decoded with a frame
/// size greater than `val` but less than the max frame length in effect
/// before calling this function, then the frame will be allowed.
pub fn set_max_frame_length(&mut self, val: usize) {
self.builder.max_frame_length(val);
}
fn decode_head(&mut self, src: &mut BytesMut) -> io::Result<Option<usize>> {
let head_len = self.builder.num_head_bytes();
let field_len = self.builder.length_field_len;
if src.len() < head_len {
// Not enough data
return Ok(None);
}
let n = {
let mut src = Cursor::new(&mut *src);
// Skip the required bytes
src.advance(self.builder.length_field_offset);
// match endianess
let n = if self.builder.length_field_is_big_endian {
src.get_uint_be(field_len)
} else {
src.get_uint_le(field_len)
};
if n > self.builder.max_frame_len as u64 {
return Err(io::Error::new(
io::ErrorKind::InvalidData,
FrameTooBig { _priv: () },
));
}
// The check above ensures there is no overflow
let n = n as usize;
// Adjust `n` with bounds checking
let n = if self.builder.length_adjustment < 0 {
n.checked_sub(-self.builder.length_adjustment as usize)
} else {
n.checked_add(self.builder.length_adjustment as usize)
};
// Error handling
match n {
Some(n) => n,
None => {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
"provided length would overflow after adjustment",
));
}
}
};
let num_skip = self.builder.get_num_skip();
if num_skip > 0 {
let _ = src.split_to(num_skip);
}
// Ensure that the buffer has enough space to read the incoming
// payload
src.reserve(n);
return Ok(Some(n));
}
fn decode_data(&self, n: usize, src: &mut BytesMut) -> io::Result<Option<BytesMut>> {
// At this point, the buffer has already had the required capacity
// reserved. All there is to do is read.
if src.len() < n {
return Ok(None);
}
Ok(Some(src.split_to(n)))
}
}
impl Decoder for LengthDelimitedCodec {
type Item = BytesMut;
type Error = io::Error;
fn decode(&mut self, src: &mut BytesMut) -> io::Result<Option<BytesMut>> {
let n = match self.state {
DecodeState::Head => match try!(self.decode_head(src)) {
Some(n) => {
self.state = DecodeState::Data(n);
n
}
None => return Ok(None),
},
DecodeState::Data(n) => n,
};
match try!(self.decode_data(n, src)) {
Some(data) => {
// Update the decode state
self.state = DecodeState::Head;
// Make sure the buffer has enough space to read the next head
src.reserve(self.builder.num_head_bytes());
Ok(Some(data))
}
None => Ok(None),
}
}
}
impl Encoder for LengthDelimitedCodec {
type Item = Bytes;
type Error = io::Error;
fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), io::Error> {
let n = (&data).into_buf().remaining();
if n > self.builder.max_frame_len {
return Err(io::Error::new(
io::ErrorKind::InvalidInput,
FrameTooBig { _priv: () },
));
}
// Adjust `n` with bounds checking
let n = if self.builder.length_adjustment < 0 {
n.checked_add(-self.builder.length_adjustment as usize)
} else {
n.checked_sub(self.builder.length_adjustment as usize)
};
let n = n.ok_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidInput,
"provided length would overflow after adjustment",
)
})?;
// Reserve capacity in the destination buffer to fit the frame and
// length field (plus adjustment).
dst.reserve(self.builder.length_field_len + n);
if self.builder.length_field_is_big_endian {
dst.put_uint_be(n as u64, self.builder.length_field_len);
} else {
dst.put_uint_le(n as u64, self.builder.length_field_len);
}
// Write the frame to the buffer
dst.extend_from_slice(&data[..]);
Ok(())
}
}
// ===== impl Builder =====
impl Builder {
/// Creates a new length delimited codec builder with default configuration
/// values.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new() -> Builder {
Builder {
// Default max frame length of 8MB
max_frame_len: 8 * 1_024 * 1_024,
// Default byte length of 4
length_field_len: 4,
// Default to the header field being at the start of the header.
length_field_offset: 0,
length_adjustment: 0,
// Total number of bytes to skip before reading the payload, if not set,
// `length_field_len + length_field_offset`
num_skip: None,
// Default to reading the length field in network (big) endian.
length_field_is_big_endian: true,
}
}
/// Read the length field as a big endian integer
///
/// This is the default setting.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .big_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn big_endian(&mut self) -> &mut Self {
self.length_field_is_big_endian = true;
self
}
/// Read the length field as a little endian integer
///
/// The default setting is big endian.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .little_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn little_endian(&mut self) -> &mut Self {
self.length_field_is_big_endian = false;
self
}
/// Read the length field as a native endian integer
///
/// The default setting is big endian.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .native_endian()
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn native_endian(&mut self) -> &mut Self {
if cfg!(target_endian = "big") {
self.big_endian()
} else {
self.little_endian()
}
}
/// Sets the max frame length
///
/// This configuration option applies to both encoding and decoding. The
/// default value is 8MB.
///
/// When decoding, the length field read from the byte stream is checked
/// against this setting **before** any adjustments are applied. When
/// encoding, the length of the submitted payload is checked against this
/// setting.
///
/// When frames exceed the max length, an `io::Error` with the custom value
/// of the `FrameTooBig` type will be returned.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .max_frame_length(8 * 1024)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn max_frame_length(&mut self, val: usize) -> &mut Self {
self.max_frame_len = val;
self
}
/// Sets the number of bytes used to represent the length field
///
/// The default value is `4`. The max value is `8`.
///
/// This configuration option applies to both encoding and decoding.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .length_field_length(4)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_field_length(&mut self, val: usize) -> &mut Self {
assert!(val > 0 && val <= 8, "invalid length field length");
self.length_field_len = val;
self
}
/// Sets the number of bytes in the header before the length field
///
/// This configuration option only applies to decoding.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .length_field_offset(1)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_field_offset(&mut self, val: usize) -> &mut Self {
self.length_field_offset = val;
self
}
/// Delta between the payload length specified in the header and the real
/// payload length
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .length_adjustment(-2)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn length_adjustment(&mut self, val: isize) -> &mut Self {
self.length_adjustment = val;
self
}
/// Sets the number of bytes to skip before reading the payload
///
/// Default value is `length_field_len + length_field_offset`
///
/// This configuration option only applies to decoding
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .num_skip(4)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn num_skip(&mut self, val: usize) -> &mut Self {
self.num_skip = Some(val);
self
}
/// Create a configured length delimited `LengthDelimitedCodec`
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
/// # pub fn main() {
/// Builder::new()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_codec();
/// # }
/// ```
pub fn new_codec(&self) -> LengthDelimitedCodec {
LengthDelimitedCodec {
builder: *self,
state: DecodeState::Head,
}
}
/// Create a configured length delimited `FramedRead`
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::io::AsyncRead;
/// use tokio::codec::length_delimited::Builder;
///
/// # fn bind_read<T: AsyncRead>(io: T) {
/// Builder::new()
/// .length_field_offset(0)
/// .length_field_length(2)
/// .length_adjustment(0)
/// .num_skip(0)
/// .new_read(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_read<T>(&self, upstream: T) -> FramedRead<T, LengthDelimitedCodec>
where
T: AsyncRead,
{
FramedRead::new(upstream, self.new_codec())
}
/// Create a configured length delimited `FramedWrite`
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate bytes;
/// # use tokio::io::AsyncWrite;
/// # use tokio::codec::length_delimited;
/// # use bytes::BytesMut;
/// # fn write_frame<T: AsyncWrite>(io: T) {
/// length_delimited::Builder::new()
/// .length_field_length(2)
/// .new_write(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_write<T>(&self, inner: T) -> FramedWrite<T, LengthDelimitedCodec>
where
T: AsyncWrite,
{
FramedWrite::new(inner, self.new_codec())
}
/// Create a configured length delimited `Framed`
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate bytes;
/// # use tokio::io::{AsyncRead, AsyncWrite};
/// # use tokio::codec::length_delimited;
/// # use bytes::BytesMut;
/// # fn write_frame<T: AsyncRead + AsyncWrite>(io: T) {
/// # let _ =
/// length_delimited::Builder::new()
/// .length_field_length(2)
/// .new_framed(io);
/// # }
/// # pub fn main() {}
/// ```
pub fn new_framed<T>(&self, inner: T) -> Framed<T, LengthDelimitedCodec>
where
T: AsyncRead + AsyncWrite,
{
Framed::new(inner, self.new_codec())
}
fn num_head_bytes(&self) -> usize {
let num = self.length_field_offset + self.length_field_len;
cmp::max(num, self.num_skip.unwrap_or(0))
}
fn get_num_skip(&self) -> usize {
self.num_skip
.unwrap_or(self.length_field_offset + self.length_field_len)
}
}
// ===== impl FrameTooBig =====
impl fmt::Debug for FrameTooBig {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("FrameTooBig").finish()
}
}
impl fmt::Display for FrameTooBig {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.write_str(self.description())
}
}
impl StdError for FrameTooBig {
fn description(&self) -> &str {
"frame size too big"
}
}
+19
View File
@@ -0,0 +1,19 @@
//! Utilities for encoding and decoding frames.
//!
//! Contains adapters to go from streams of bytes, [`AsyncRead`] and
//! [`AsyncWrite`], to framed streams implementing [`Sink`] and [`Stream`].
//! Framed streams are also known as [transports].
//!
//! [`AsyncRead`]: ../io/trait.AsyncRead.html
//! [`AsyncWrite`]: ../io/trait.AsyncWrite.html
//! [`Sink`]: https://docs.rs/futures/0.1/futures/sink/trait.Sink.html
//! [`Stream`]: https://docs.rs/futures/0.1/futures/stream/trait.Stream.html
//! [transports]: https://tokio.rs/docs/going-deeper/frames/
pub use tokio_codec::{
BytesCodec, Decoder, Encoder, Framed, FramedParts, FramedRead, FramedWrite, LinesCodec,
};
pub mod length_delimited;
pub use self::length_delimited::LengthDelimitedCodec;
+23 -567
View File
@@ -1,3 +1,5 @@
#![allow(deprecated)]
//! Execute many tasks concurrently on the current thread.
//!
//! [`CurrentThread`] is an executor that keeps tasks on the same thread that
@@ -102,57 +104,24 @@
//! [`CurrentThread`]: struct.CurrentThread.html
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
#![allow(deprecated)]
pub use tokio_current_thread::{
BlockError,
CurrentThread,
Entered,
Handle,
RunError,
RunTimeoutError,
TaskExecutor,
Turn,
TurnError,
block_on_all,
spawn,
};
mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::{self, Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::{executor, Async, Future};
use futures::future::{self, Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::marker::PhantomData;
use std::rc::Rc;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
/// Execute futures and receive unpark notifications.
scheduler: Scheduler<P::Unpark>,
/// Current number of futures being executed
num_futures: usize,
/// Thread park handle
park: P,
}
/// Executes futures on the current thread.
///
/// All futures executed using this executor will be executed on the current
/// thread. As such, `run` will wait for these futures to complete before
/// returning.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
// Prevent the handle from moving across threads.
_p: ::std::marker::PhantomData<Rc<()>>,
}
/// Returned by the `turn` function
#[derive(Debug)]
pub struct Turn(());
/// A `CurrentThread` instance bound to a supplied execution conext.
pub struct Entered<'a, P: Park + 'a> {
executor: &'a mut CurrentThread<P>,
enter: &'a mut Enter,
}
use futures::future::{self};
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
@@ -162,54 +131,17 @@ pub struct Context<'a> {
_p: PhantomData<&'a ()>,
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
_p: (),
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
/// Error returned by the `run_timeout` function.
#[derive(Debug)]
pub struct RunTimeoutError {
timeout: bool,
}
/// Error returned by the `turn` function.
#[derive(Debug)]
pub struct TurnError {
_p: (),
}
/// Error returned by the `block_on` function.
#[derive(Debug)]
pub struct BlockError<T> {
inner: Option<T>,
}
/// This is mostly split out to make the borrow checker happy.
struct Borrow<'a, U: 'a> {
scheduler: &'a mut Scheduler<U>,
num_futures: &'a mut usize,
}
trait SpawnLocal {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>);
}
struct CurrentRunner {
spawn: Cell<Option<*mut SpawnLocal>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
});
#[deprecated(since = "0.1.2", note = "use block_on_all instead")]
#[doc(hidden)]
#[allow(deprecated)]
pub fn run<F, R>(f: F) -> R
where F: FnOnce(&mut Context) -> R
where F: FnOnce(&mut Context) -> R
{
let mut context = Context {
cancel: Cell::new(false),
@@ -230,485 +162,9 @@ where F: FnOnce(&mut Context) -> R
ret
}
/// Run the executor bootstrapping the execution with the provided future.
///
/// This creates a new [`CurrentThread`] executor, spawns the provided future,
/// and blocks the current thread until the provided future and **all**
/// subsequently spawned futures complete. In other words:
///
/// * If the provided boostrap future does **not** spawn any additional tasks,
/// `block_on_all` returns once `future` completes.
/// * If the provided bootstrap future **does** spawn additional tasks, then
/// `block_on_all` returns once **all** spawned futures complete.
///
/// See [module level][mod] documentation for more details.
///
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
{
let mut current_thread = CurrentThread::new();
let ret = current_thread.block_on(future);
current_thread.run().unwrap();
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
}
/// Executes a future on the current thread.
///
/// The provided future must complete or be canceled before `run` will return.
///
/// Unlike [`tokio::spawn`], this function will always spawn on a
/// `CurrentThread` executor and is able to spawn futures that are not `Send`.
///
/// # Panics
///
/// This function can only be invoked from the context of a `run` call; any
/// other use will result in a panic.
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
{
TaskExecutor::current()
.spawn_local(Box::new(future))
.unwrap();
}
// ===== impl CurrentThread =====
impl CurrentThread<ParkThread> {
/// Create a new instance of `CurrentThread`.
pub fn new() -> Self {
CurrentThread::new_with_park(ParkThread::new())
}
}
impl<P: Park> CurrentThread<P> {
/// Create a new instance of `CurrentThread` backed by the given park
/// handle.
pub fn new_with_park(park: P) -> Self {
let unpark = park.unpark();
CurrentThread {
scheduler: Scheduler::new(unpark),
num_futures: 0,
park,
}
}
/// Returns `true` if the executor is currently idle.
///
/// An idle executor is defined by not currently having any spawned tasks.
pub fn is_idle(&self) -> bool {
self.num_futures == 0
}
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter().unwrap();
self.enter(&mut enter).turn(duration)
}
/// Bind `CurrentThread` instance with an execution context.
pub fn enter<'a>(&'a mut self, enter: &'a mut Enter) -> Entered<'a, P> {
Entered {
executor: self,
enter,
}
}
fn borrow(&mut self) -> Borrow<P::Unpark> {
Borrow {
scheduler: &mut self.scheduler,
num_futures: &mut self.num_futures,
}
}
}
impl tokio_executor::Executor for CurrentThread {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.borrow().spawn_local(future);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures)
.finish()
}
}
// ===== impl Entered =====
impl<'a, P: Park> Entered<'a, P> {
/// Spawn the future on the executor.
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future));
self
}
/// Synchronously waits for the provided `future` to complete.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
match res {
Ok(Async::Ready(e)) => return Ok(e),
Err(e) => return Err(BlockError { inner: Some(e) }),
Ok(Async::NotReady) => {}
}
self.tick();
if let Err(_) = self.executor.park.park() {
return Err(BlockError { inner: None });
}
}
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(Turn(()));
}
if !self.tick() {
let res = match duration {
Some(duration) => self.executor.park.park_timeout(duration),
None => self.executor.park.park(),
};
if res.is_err() {
return Err(TurnError { _p: () });
}
self.tick();
}
Ok(Turn(()))
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
if self.executor.is_idle() {
// Nothing to do
return Ok(());
}
let mut time = dur.map(|dur| (Instant::now() + dur, dur));
loop {
self.tick();
if self.executor.is_idle() {
return Ok(());
}
match time {
Some((until, rem)) => {
if let Err(_) = self.executor.park.park_timeout(rem) {
return Err(RunTimeoutError::new(false));
}
let now = Instant::now();
if now >= until {
return Err(RunTimeoutError::new(true));
}
time = Some((until, until - now));
}
None => {
if let Err(_) = self.executor.park.park() {
return Err(RunTimeoutError::new(false));
}
}
}
}
}
/// Returns `true` if any futures were processed
fn tick(&mut self) -> bool {
self.executor.scheduler.tick(
&mut *self.enter,
&mut self.executor.num_futures)
}
}
impl<'a, P: Park> fmt::Debug for Entered<'a, P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Entered")
.field("executor", &self.executor)
.field("enter", &self.enter)
.finish()
}
}
// ===== impl TaskExecutor =====
#[deprecated(since = "0.1.2", note = "use TaskExecutor::current instead")]
#[doc(hidden)]
pub fn task_executor() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
TaskExecutor::current()
}
impl TaskExecutor {
/// Returns an executor that executes futures on the current thread.
///
/// The user of `TaskExecutor` must ensure that when a future is submitted,
/// that it is done within the context of a call to `run`.
///
/// For more details, see the [module level](index.html) documentation.
pub fn current() -> TaskExecutor {
TaskExecutor {
_p: ::std::marker::PhantomData,
}
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
}
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
self.spawn_local(future)
}
fn status(&self) -> Result<(), SpawnError> {
CURRENT.with(|current| {
if current.spawn.get().is_some() {
Ok(())
} else {
Err(SpawnError::shutdown())
}
})
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future)) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
}
})
}
}
// ===== impl Context =====
impl<'a> Context<'a> {
/// Cancels *all* executing futures.
pub fn cancel_all_spawned(&self) {
self.cancel.set(true);
}
}
// ===== impl Borrow =====
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.set_spawn(self, || {
f()
})
})
}
}
impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>) {
*self.num_futures += 1;
self.scheduler.schedule(future);
}
}
// ===== impl CurrentRunner =====
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
{
struct Reset<'a>(&'a CurrentRunner);
impl<'a> Drop for Reset<'a> {
fn drop(&mut self) {
self.0.spawn.set(None);
}
}
let _reset = Reset(self);
let spawn = unsafe { hide_lt(spawn as *mut SpawnLocal) };
self.spawn.set(Some(spawn));
f()
}
}
unsafe fn hide_lt<'a>(p: *mut (SpawnLocal + 'a)) -> *mut (SpawnLocal + 'static) {
use std::mem;
mem::transmute(p)
}
// ===== impl RunTimeoutError =====
impl RunTimeoutError {
fn new(timeout: bool) -> Self {
RunTimeoutError { timeout }
}
/// Returns `true` if the error was caused by the operation timeing out.
pub fn is_timeout(&self) -> bool {
self.timeout
}
}
impl From<tokio_executor::EnterError> for RunTimeoutError {
fn from(_: tokio_executor::EnterError) -> Self {
RunTimeoutError::new(false)
}
}
// ===== impl BlockError =====
impl<T> BlockError<T> {
/// Returns the error yielded by the future being blocked on
pub fn into_inner(self) -> Option<T> {
self.inner
}
}
impl<T> From<tokio_executor::EnterError> for BlockError<T> {
fn from(_: tokio_executor::EnterError) -> Self {
BlockError { inner: None }
}
}
+21 -93
View File
@@ -5,7 +5,7 @@
//! the future must be submitted to an executor. A future that is submitted to
//! an executor is called a "task".
//!
//! The executor executor is responsible for ensuring that [`Future::poll`] is
//! The executor is responsible for ensuring that [`Future::poll`] is
//! called whenever the task is [notified]. Notification happens when the
//! internal state of a task transitions from "not ready" to ready. For
//! example, a socket might have received data and a call to `read` will now be
@@ -13,16 +13,8 @@
//!
//! The specific strategy used to manage the tasks is left up to the
//! executor. There are two main flavors of executors: single-threaded and
//! multithreaded. This module provides both.
//!
//! * **[`current_thread`]**: A single-threaded executor that support spawning
//! tasks that are not `Send`. It guarantees that tasks will be executed on
//! the same thread from which they are spawned.
//!
//! * **[`thread_pool`]**: A multi-threaded executor that maintains a pool of
//! threads. Tasks are spawned to one of the threads in the pool and executed.
//! The pool employes a [work-stealing] strategy for optimizing how tasks get
//! spread across the available threads.
//! multi-threaded. Tokio provides implementation for both of these in the
//! [`runtime`] module.
//!
//! # `Executor` trait.
//!
@@ -36,94 +28,30 @@
//! executor. This value will often be set to the executor itself, but it is
//! possible that the default executor might be set to a different executor.
//!
//! For example, the [`current_thread`] executor might set the default executor
//! to a thread pool instead of itself, allowing futures to spawn new tasks onto
//! the thread pool when those tasks are `Send`.
//! For example, a single threaded executor might set the default executor to a
//! thread pool instead of itself, allowing futures to spawn new tasks onto the
//! thread pool when those tasks are `Send`.
//!
//! [`Future::poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
//! [notified]: https://docs.rs/futures/0.1/futures/executor/trait.Notify.html#tymethod.notify
//! [`current_thread`]: current_thread/index.html
//! [`thread_pool`]: thread_pool/index.html
//! [work-stealing]: https://en.wikipedia.org/wiki/Work_stealing
//! [`tokio-executor`]: #
//! [`Executor`]: #
//! [`spawn`]: #
//! [`runtime`]: ../runtime/index.html
//! [`tokio-executor`]: https://docs.rs/tokio-executor/0.1
//! [`Executor`]: trait.Executor.html
//! [`spawn`]: fn.spawn.html
#[deprecated(
since = "0.1.8",
note = "use tokio-current-thread crate or functions in tokio::runtime::current_thread instead",
)]
#[doc(hidden)]
pub mod current_thread;
#[deprecated(since = "0.1.8", note = "use tokio-threadpool crate instead")]
#[doc(hidden)]
/// Re-exports of [`tokio-threadpool`], deprecated in favor of the crate.
///
/// [`tokio-threadpool`]: https://docs.rs/tokio-threadpool/0.1
pub mod thread_pool {
//! Maintains a pool of threads across which the set of spawned tasks are
//! executed.
//!
//! [`ThreadPool`] is an executor that uses a thread pool for executing
//! tasks concurrently across multiple cores. It uses a thread pool that is
//! optimized for use cases that involve multiplexing large number of
//! independent tasks that perform short(ish) amounts of computation and are
//! mainly waiting on I/O, i.e. the Tokio use case.
//!
//! Usually, users of [`ThreadPool`] will not create pool instances.
//! Instead, they will create a [`Runtime`] instance, which comes with a
//! pre-configured thread pool.
//!
//! At the core, [`ThreadPool`] uses a work-stealing based scheduling
//! strategy. When spawning a task while *external* to the thread pool
//! (i.e., from a thread that is not part of the thread pool), the task is
//! randomly assigned to a worker thread. When spawning a task while
//! *internal* to the thread pool, the task is assigned to the current
//! worker.
//!
//! Each worker maintains its own queue and first focuses on processing all
//! tasks in its queue. When the worker's queue is empty, the worker will
//! attempt to *steal* tasks from other worker queues. This strategy helps
//! ensure that work is evenly distributed across threads while minimizing
//! synchronization between worker threads.
//!
//! # Usage
//!
//! Thread pool instances are created using [`ThreadPool::new`] or
//! [`Builder::new`]. The first option returns a thread pool with default
//! configuration values. The second option allows configuring the thread
//! pool before instantiating it.
//!
//! Once an instance is obtained, futures may be spawned onto it using the
//! [`spawn`] function.
//!
//! A handle to the thread pool is obtained using [`ThreadPool::sender`].
//! This handle is **only** able to spawn futures onto the thread pool. It
//! is unable to affect the lifecycle of the thread pool in any way. This
//! handle can be passed into functions or stored in structs as a way to
//! grant the capability of spawning futures.
//!
//! # Examples
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use tokio::executor::thread_pool::ThreadPool;
//! use futures::future::{Future, lazy};
//!
//! # pub fn main() {
//! // Create a thread pool with default configuration values
//! let thread_pool = ThreadPool::new();
//!
//! thread_pool.spawn(lazy(|| {
//! println!("called from a worker thread");
//! Ok(())
//! }));
//!
//! // Gracefully shutdown the threadpool
//! thread_pool.shutdown().wait().unwrap();
//! # }
//! ```
//!
//! [`ThreadPool`]: struct.ThreadPool.html
//! [`ThreadPool::new`]: struct.ThreadPool.html#method.new
//! [`ThreadPool::sender`]: struct.ThreadPool.html#method.sender
//! [`spawn`]: struct.ThreadPool.html#method.spawn
//! [`Builder::new`]: struct.Builder.html#method.new
//! [`Runtime`]: ../../runtime/struct.Runtime.html
pub use tokio_threadpool::{
Builder,
Sender,
@@ -198,7 +126,7 @@ pub struct Spawn(());
/// onto the default executor returns an error. To avoid the panic, use
/// [`DefaultExecutor`].
///
/// [`DefaultExecutor`]: #
/// [`DefaultExecutor`]: struct.DefaultExecutor.html
pub fn spawn<F>(f: F) -> Spawn
where F: Future<Item = (), Error = ()> + 'static + Send
{
+15
View File
@@ -0,0 +1,15 @@
//! Asynchronous filesystem manipulation operations.
//!
//! This module contains basic methods and types for manipulating the contents
//! of the local filesystem from within the context of the Tokio runtime.
//!
//! Unlike *most* other Tokio APIs, the filesystem APIs **must** be used from
//! the context of the Tokio runtime as they require Tokio specific features to
//! function.
pub use tokio_fs::OpenOptions;
pub use tokio_fs::{
create_dir, create_dir_all, file, hard_link, metadata, os, read_dir, read_link,
};
pub use tokio_fs::{read, write, ReadFile, WriteFile};
pub use tokio_fs::{remove_dir, remove_file, rename, set_permissions, symlink_metadata, File};
+62
View File
@@ -0,0 +1,62 @@
//! Asynchronous I/O.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! # AsyncRead and AsyncWrite
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! # Standard input and output
//!
//! Tokio provides asynchronous APIs to standard [input], [output], and [error].
//! These APIs are very similar to the ones provided by `std`, but they also
//! implement [`AsyncRead`] and [`AsyncWrite`].
//!
//! Unlike *most* other Tokio APIs, the standard input / output APIs
//! **must** be used from the context of the Tokio runtime as they require
//! Tokio specific features to function.
//!
//! [input]: fn.stdin.html
//! [output]: fn.stdout.html
//! [error]: fn.stderr.html
//!
//! # Utility functions
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! # `std` re-exports
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
//! [`AsyncRead`]: trait.AsyncRead.html
//! [`AsyncWrite`]: trait.AsyncWrite.html
//! [`copy`]: fn.copy.html
//! [`Read`]: trait.Read.html
//! [`Write`]: trait.Write.html
//! [`Error`]: struct.Error.html
//! [`ErrorKind`]: enum.ErrorKind.html
//! [`Result`]: type.Result.html
pub use tokio_io::{AsyncRead, AsyncWrite};
// standard input, output, and error
#[cfg(feature = "fs")]
pub use tokio_fs::{stderr, stdin, stdout, Stderr, Stdin, Stdout};
// Utils
pub use tokio_io::io::{
copy, flush, lines, read, read_exact, read_to_end, read_until, shutdown, write_all, Copy,
Flush, Lines, ReadExact, ReadHalf, ReadToEnd, ReadUntil, Shutdown, WriteAll, WriteHalf,
};
// Re-export io::Error so that users don't have to deal
// with conflicts when `use`ing `futures::io` and `std::io`.
pub use std::io::{Error, ErrorKind, Read, Result, Write};
+77 -111
View File
@@ -1,3 +1,10 @@
#![doc(html_root_url = "https://docs.rs/tokio/0.1.17")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
#![cfg_attr(
feature = "async-await-preview",
feature(async_await, await_macro, futures_api,)
)]
//! A runtime for writing reliable, asynchronous, and slim applications.
//!
//! Tokio is an event-driven, non-blocking I/O platform for writing asynchronous
@@ -5,17 +12,19 @@
//! provides a few major components:
//!
//! * A multi threaded, work-stealing based task [scheduler][runtime].
//! * A [reactor][reactor] backed by the operating system's event queue (epoll, kqueue,
//! * A [reactor] backed by the operating system's event queue (epoll, kqueue,
//! IOCP, etc...).
//! * Asynchronous [TCP and UDP][net] sockets.
//! * Asynchronous [filesystem][fs] operations.
//! * [Timer][timer] API for scheduling work in the future.
//!
//! Tokio is built using futures (provided by the [futures] crate) as the
//! abstraction for managing the complexity of asynchronous programming.
//! Tokio is built using [futures] as the abstraction for managing the
//! complexity of asynchronous programming.
//!
//! Guide level documentation is found on the [website].
//!
//! [website]: https://tokio.rs/docs/getting-started/hello-world/
//! [futures]: http://docs.rs/futures
//! [futures]: http://docs.rs/futures/0.1
//!
//! # Examples
//!
@@ -62,128 +71,85 @@
//! }
//! ```
#![doc(html_root_url = "https://docs.rs/tokio/0.1.2")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
macro_rules! if_runtime {
($($i:item)*) => ($(
#[cfg(any(feature = "rt-full"))]
$i
)*)
}
extern crate bytes;
#[macro_use]
extern crate futures;
extern crate iovec;
#[cfg(feature = "io")]
extern crate bytes;
#[cfg(feature = "reactor")]
extern crate mio;
extern crate slab;
#[cfg(feature = "rt-full")]
extern crate num_cpus;
#[cfg(feature = "codec")]
extern crate tokio_codec;
#[cfg(feature = "rt-full")]
extern crate tokio_current_thread;
#[cfg(feature = "fs")]
extern crate tokio_fs;
#[cfg(feature = "io")]
extern crate tokio_io;
extern crate tokio_executor;
#[cfg(feature = "reactor")]
extern crate tokio_reactor;
#[cfg(feature = "sync")]
extern crate tokio_sync;
#[cfg(feature = "tcp")]
extern crate tokio_tcp;
#[cfg(feature = "rt-full")]
extern crate tokio_threadpool;
#[cfg(feature = "timer")]
extern crate tokio_timer;
#[cfg(feature = "udp")]
extern crate tokio_udp;
#[macro_use]
extern crate log;
#[cfg(feature = "async-await-preview")]
extern crate tokio_async_await;
pub mod executor;
#[cfg(all(unix, feature = "uds"))]
extern crate tokio_uds;
#[cfg(feature = "timer")]
pub mod clock;
#[cfg(feature = "codec")]
pub mod codec;
#[cfg(feature = "fs")]
pub mod fs;
#[cfg(feature = "io")]
pub mod io;
#[cfg(any(feature = "tcp", feature = "udp", feature = "uds"))]
pub mod net;
pub mod prelude;
#[cfg(feature = "reactor")]
pub mod reactor;
pub mod runtime;
#[cfg(feature = "sync")]
pub mod sync;
#[cfg(feature = "timer")]
pub mod timer;
pub mod util;
pub use executor::spawn;
pub use runtime::run;
if_runtime! {
extern crate tokio_executor;
extern crate tokio_trace_core;
pub mod executor;
pub mod runtime;
pub mod io {
//! Asynchronous I/O.
//!
//! This module is the asynchronous version of `std::io`. Primarily, it
//! defines two traits, [`AsyncRead`] and [`AsyncWrite`], which extend the
//! `Read` and `Write` traits of the standard library.
//!
//! [`AsyncRead`] and [`AsyncWrite`] must only be implemented for
//! non-blocking I/O types that integrate with the futures type system. In
//! other words, these types must never block the thread, and instead the
//! current task is notified when the I/O resource is ready.
//!
//! Utilities functions are provided for working with [`AsyncRead`] /
//! [`AsyncWrite`] types. For example, [`copy`] asynchronously copies all
//! data from a source to a destination.
//!
//! Additionally, [`Read`], [`Write`], [`Error`], [`ErrorKind`], and
//! [`Result`] are re-exported from `std::io` for ease of use.
//!
//! [`AsyncRead`]: trait.AsyncRead.html
//! [`AsyncWrite`]: trait.AsyncWrite.html
//! [`copy`]: fn.copy.html
//! [`Read`]: trait.Read.html
//! [`Write`]: trait.Write.html
//! [`Error`]: struct.Error.html
//! [`ErrorKind`]: enum.ErrorKind.html
//! [`Result`]: type.Result.html
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
// Utils
pub use tokio_io::io::{
copy,
Copy,
flush,
Flush,
lines,
Lines,
read_exact,
ReadExact,
read_to_end,
ReadToEnd,
read_until,
ReadUntil,
shutdown,
Shutdown,
write_all,
WriteAll,
};
// Re-export io::Error so that users don't have to deal
// with conflicts when `use`ing `futures::io` and `std::io`.
pub use ::std::io::{
Error,
ErrorKind,
Result,
Read,
Write,
};
pub use executor::spawn;
pub use runtime::run;
}
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
// ===== Experimental async/await support =====
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
#[cfg(feature = "async-await-preview")]
mod async_await;
pub use ::std::io::{
Read,
Write,
};
#[cfg(feature = "async-await-preview")]
pub use async_await::{run_async, spawn_async};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
}
#[cfg(feature = "async-await-preview")]
pub use tokio_async_await::await;
+98
View File
@@ -0,0 +1,98 @@
//! TCP/UDP/Unix bindings for `tokio`.
//!
//! This module contains the TCP/UDP/Unix networking types, similar to the standard
//! library, which can be used to implement networking protocols.
//!
//! # Organization
//!
//! * [`TcpListener`] and [`TcpStream`] provide functionality for communication over TCP
//! * [`UdpSocket`] and [`UdpFramed`] provide functionality for communication over UDP
//! * [`UnixListener`] and [`UnixStream`] provide functionality for communication over a
//! Unix Domain Stream Socket **(available on Unix only)**
//! * [`UnixDatagram`] and [`UnixDatagramFramed`] provide functionality for communication
//! over Unix Domain Datagram Socket **(available on Unix only)**
//!
//! [`TcpListener`]: struct.TcpListener.html
//! [`TcpStream`]: struct.TcpStream.html
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`UnixListener`]: struct.UnixListener.html
//! [`UnixStream`]: struct.UnixStream.html
//! [`UnixDatagram`]: struct.UnixDatagram.html
//! [`UnixDatagramFramed`]: struct.UnixDatagramFramed.html
#[cfg(feature = "tcp")]
pub mod tcp {
//! TCP bindings for `tokio`.
//!
//! Connecting to an address, via TCP, can be done using [`TcpStream`]'s
//! [`connect`] method, which returns [`ConnectFuture`]. `ConnectFuture`
//! implements a future which returns a `TcpStream`.
//!
//! To listen on an address [`TcpListener`] can be used. `TcpListener`'s
//! [`incoming`][incoming_method] method can be used to accept new connections.
//! It return the [`Incoming`] struct, which implements a stream which returns
//! `TcpStream`s.
//!
//! [`TcpStream`]: struct.TcpStream.html
//! [`connect`]: struct.TcpStream.html#method.connect
//! [`ConnectFuture`]: struct.ConnectFuture.html
//! [`TcpListener`]: struct.TcpListener.html
//! [incoming_method]: struct.TcpListener.html#method.incoming
//! [`Incoming`]: struct.Incoming.html
pub use tokio_tcp::{ConnectFuture, Incoming, TcpListener, TcpStream};
}
#[cfg(feature = "tcp")]
pub use self::tcp::{TcpListener, TcpStream};
#[cfg(feature = "tcp")]
#[deprecated(note = "use `tokio::net::tcp::ConnectFuture` instead")]
#[doc(hidden)]
pub type ConnectFuture = self::tcp::ConnectFuture;
#[cfg(feature = "tcp")]
#[deprecated(note = "use `tokio::net::tcp::Incoming` instead")]
#[doc(hidden)]
pub type Incoming = self::tcp::Incoming;
#[cfg(feature = "udp")]
pub mod udp {
//! UDP bindings for `tokio`.
//!
//! The main struct for UDP is the [`UdpSocket`], which represents a UDP socket.
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convenience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`RecvDgram`]: struct.RecvDgram.html
//! [`SendDgram`]: struct.SendDgram.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
pub use tokio_udp::{RecvDgram, SendDgram, UdpFramed, UdpSocket};
}
#[cfg(feature = "udp")]
pub use self::udp::{UdpFramed, UdpSocket};
#[cfg(feature = "udp")]
#[deprecated(note = "use `tokio::net::udp::RecvDgram` instead")]
#[doc(hidden)]
pub type RecvDgram<T> = self::udp::RecvDgram<T>;
#[cfg(feature = "udp")]
#[deprecated(note = "use `tokio::net::udp::SendDgram` instead")]
#[doc(hidden)]
pub type SendDgram<T> = self::udp::SendDgram<T>;
#[cfg(all(unix, feature = "uds"))]
pub mod unix {
//! Unix domain socket bindings for `tokio` (only available on unix systems).
pub use tokio_uds::{
ConnectFuture, Incoming, RecvDgram, SendDgram, UCred, UnixDatagram, UnixDatagramFramed,
UnixListener, UnixStream,
};
}
#[cfg(all(unix, feature = "uds"))]
pub use self::unix::{UnixDatagram, UnixDatagramFramed, UnixListener, UnixStream};
-44
View File
@@ -1,44 +0,0 @@
//! TCP/UDP bindings for `tokio`.
//!
//! This module contains the TCP/UDP networking types, similar to the standard
//! library, which can be used to implement networking protocols.
//!
//! # TCP
//!
//! Connecting to an address, via TCP, can be done using [`TcpStream`]'s
//! [`connect`] method, which returns [`ConnectFuture`]. `ConnectFuture`
//! implements a future which returns a `TcpStream`.
//!
//! To listen on an address [`TcpListener`] can be used. `TcpListener`'s
//! [`incoming`][incoming_method] method can be used to accept new connections.
//! It return the [`Incoming`] struct, which implements a stream which returns
//! `TcpStream`s.
//!
//! [`TcpStream`]: struct.TcpStream.html
//! [`connect`]: struct.TcpStream.html#method.connect
//! [`ConnectFuture`]: struct.ConnectFuture.html
//! [`TcpListener`]: struct.TcpListener.html
//! [incoming_method]: struct.TcpListener.html#method.incoming
//! [`Incoming`]: struct.Incoming.html
//!
//! # UDP
//!
//! The main struct for UDP is the [`UdpSocket`], which represents a UDP socket.
//! Reading and writing to it can be done using futures, which return the
//! [`RecvDgram`] and [`SendDgram`] structs respectively.
//!
//! For convience it's also possible to convert raw datagrams into higher-level
//! frames.
//!
//! [`UdpSocket`]: struct.UdpSocket.html
//! [`RecvDgram`]: struct.RecvDgram.html
//! [`SendDgram`]: struct.SendDgram.html
//! [`UdpFramed`]: struct.UdpFramed.html
//! [`framed`]: struct.UdpSocket.html#method.framed
mod tcp;
mod udp;
pub use self::tcp::{TcpStream, ConnectFuture};
pub use self::tcp::{TcpListener, Incoming};
pub use self::udp::{UdpSocket, UdpFramed, SendDgram, RecvDgram};
-8
View File
@@ -1,8 +0,0 @@
mod incoming;
mod listener;
mod stream;
pub use self::incoming::Incoming;
pub use self::listener::TcpListener;
pub use self::stream::TcpStream;
pub use self::stream::ConnectFuture;
-9
View File
@@ -1,9 +0,0 @@
mod frame;
mod socket;
mod send_dgram;
mod recv_dgram;
pub use self::frame::UdpFramed;
pub use self::socket::UdpSocket;
pub use self::send_dgram::SendDgram;
pub use self::recv_dgram::RecvDgram;
-52
View File
@@ -1,52 +0,0 @@
use net::udp::socket::UdpSocket;
use std::io;
use std::net::SocketAddr;
use futures::{Async, Future, Poll};
/// A future used to receive a datagram from a UDP socket.
///
/// This is created by the `UdpSocket::recv_dgram` method.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct RecvDgram<T> {
/// None means future was completed
state: Option<RecvDgramInner<T>>
}
/// A struct is used to represent the full info of RecvDgram.
#[derive(Debug)]
struct RecvDgramInner<T> {
/// Rx socket
socket: UdpSocket,
/// The received data will be put in the buffer
buffer: T
}
impl<T> RecvDgram<T> {
/// Create a new future to receive UDP Datagram
pub(crate) fn new(socket: UdpSocket, buffer: T) -> RecvDgram<T> {
let inner = RecvDgramInner { socket: socket, buffer: buffer };
RecvDgram { state: Some(inner) }
}
}
impl<T> Future for RecvDgram<T>
where T: AsMut<[u8]>,
{
type Item = (UdpSocket, T, usize, SocketAddr);
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, io::Error> {
let (n, addr) = {
let ref mut inner =
self.state.as_mut().expect("RecvDgram polled after completion");
try_ready!(inner.socket.poll_recv_from(inner.buffer.as_mut()))
};
let inner = self.state.take().unwrap();
Ok(Async::Ready((inner.socket, inner.buffer, n, addr)))
}
}
+28
View File
@@ -0,0 +1,28 @@
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
#[cfg(feature = "io")]
pub use tokio_io::{AsyncRead, AsyncWrite};
pub use util::{FutureExt, StreamExt};
pub use std::io::{Read, Write};
pub use futures::{future, stream, task, Async, AsyncSink, Future, IntoFuture, Poll, Sink, Stream};
#[cfg(feature = "async-await-preview")]
#[doc(inline)]
pub use tokio_async_await::{
io::{AsyncReadExt, AsyncWriteExt},
sink::SinkExt,
stream::StreamExt as StreamAsyncExt,
};
+6 -11
View File
@@ -81,7 +81,7 @@
//! ## Implementation
//!
//! The reactor implementation uses [`mio`] to interface with the operating
//! system's event queue. A call to [`Reactor::poll`] results in in a single
//! system's event queue. A call to [`Reactor::poll`] results in a single
//! call to [`Poll::poll`] which in turn results in a single call to the
//! operating system's selector.
//!
@@ -107,8 +107,8 @@
//! There are a couple of ways to do this.
//!
//! If the custom I/O resource implements [`mio::Evented`] and implements
//! [`std::Read`] and / or [`std::Write`], then [`PollEvented`] is the most
//! suited.
//! [`std::io::Read`] and / or [`std::io::Write`], then [`PollEvented`] is the
//! most suited.
//!
//! Otherwise, [`Registration`] can be used directly. This provides the lowest
//! level primitive needed for integrating with the reactor: a stream of
@@ -132,16 +132,11 @@
//! [`Poll::poll`]: https://docs.rs/mio/0.6/mio/struct.Poll.html#method.poll
//! [`mio::Evented`]: https://docs.rs/mio/0.6/mio/trait.Evented.html
//! [`PollEvented`]: struct.PollEvented.html
//! [`std::Read`]: https://doc.rust-lang.org/std/io/trait.Read.html
//! [`std::Write`]: https://doc.rust-lang.org/std/io/trait.Write.html
//! [`std::io::Read`]: https://doc.rust-lang.org/std/io/trait.Read.html
//! [`std::io::Write`]: https://doc.rust-lang.org/std/io/trait.Write.html
pub use tokio_reactor::{
Reactor,
Handle,
Background,
Turn,
Registration,
PollEvented as PollEvented2,
Background, Handle, PollEvented as PollEvented2, Reactor, Registration, Turn,
};
mod poll_evented;
+32 -34
View File
@@ -10,9 +10,9 @@
use std::fmt;
use std::io::{self, Read, Write};
use std::sync::Mutex;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use std::sync::Mutex;
use futures::{task, Async, Poll};
use mio::event::Evented;
@@ -41,9 +41,7 @@ struct Inner {
impl<E: fmt::Debug> fmt::Debug for PollEvented<E> {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
f.debug_struct("PollEvented")
.field("io", &self.io)
.finish()
f.debug_struct("PollEvented").field("io", &self.io).finish()
}
}
@@ -51,7 +49,8 @@ impl<E> PollEvented<E> {
/// Creates a new readiness stream associated with the provided
/// `loop_handle` and for the given `source`.
pub fn new(io: E, handle: &Handle) -> io::Result<PollEvented<E>>
where E: Evented,
where
E: Evented,
{
let registration = Registration::new();
registration.register(&io)?;
@@ -153,7 +152,9 @@ impl<E> PollEvented<E> {
};
// Cache the value
self.inner.write_readiness.store(ready2usize(ready), Relaxed);
self.inner
.write_readiness
.store(ready2usize(ready), Relaxed);
().into()
}
@@ -334,17 +335,17 @@ impl<E> PollEvented<E> {
/// method is called, and will likely return an error if this `PollEvented`
/// was created on a separate event loop from the `handle` specified.
pub fn deregister(&self) -> io::Result<()>
where E: Evented,
where
E: Evented,
{
self.inner.registration.lock().unwrap()
.deregister(&self.io)
self.inner.registration.lock().unwrap().deregister(&self.io)
}
}
impl<E: Read> Read for PollEvented<E> {
fn read(&mut self, buf: &mut [u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_read() {
return Err(io::ErrorKind::WouldBlock.into())
return Err(io::ErrorKind::WouldBlock.into());
}
let r = self.get_mut().read(buf);
@@ -353,14 +354,14 @@ impl<E: Read> Read for PollEvented<E> {
self.need_read()?;
}
return r
return r;
}
}
impl<E: Write> Write for PollEvented<E> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
return Err(io::ErrorKind::WouldBlock.into());
}
let r = self.get_mut().write(buf);
@@ -369,12 +370,12 @@ impl<E: Write> Write for PollEvented<E> {
self.need_write()?;
}
return r
return r;
}
fn flush(&mut self) -> io::Result<()> {
if let Async::NotReady = self.poll_write() {
return Err(io::ErrorKind::WouldBlock.into())
return Err(io::ErrorKind::WouldBlock.into());
}
let r = self.get_mut().flush();
@@ -383,12 +384,11 @@ impl<E: Write> Write for PollEvented<E> {
self.need_write()?;
}
return r
return r;
}
}
impl<E: Read> AsyncRead for PollEvented<E> {
}
impl<E: Read> AsyncRead for PollEvented<E> {}
impl<E: Write> AsyncWrite for PollEvented<E> {
fn shutdown(&mut self) -> Poll<(), io::Error> {
@@ -428,20 +428,10 @@ fn usize2ready(bits: usize) -> Ready {
ready | platform::usize2ready(bits)
}
#[cfg(all(unix, not(target_os = "fuchsia")))]
#[cfg(unix)]
mod platform {
use mio::Ready;
use mio::unix::UnixReady;
#[cfg(target_os = "dragonfly")]
pub fn all() -> Ready {
hup() | UnixReady::aio()
}
#[cfg(target_os = "freebsd")]
pub fn all() -> Ready {
hup() | UnixReady::aio() | UnixReady::lio()
}
use mio::Ready;
const HUP: usize = 1 << 2;
const ERROR: usize = 1 << 3;
@@ -486,14 +476,22 @@ mod platform {
bits
}
#[cfg(any(target_os = "dragonfly", target_os = "freebsd", target_os = "ios",
target_os = "macos"))]
#[cfg(any(
target_os = "dragonfly",
target_os = "freebsd",
target_os = "ios",
target_os = "macos"
))]
fn usize2ready_aio(ready: &mut UnixReady) {
ready.insert(UnixReady::aio());
}
#[cfg(not(any(target_os = "dragonfly",
target_os = "freebsd", target_os = "ios", target_os = "macos")))]
#[cfg(not(any(
target_os = "dragonfly",
target_os = "freebsd",
target_os = "ios",
target_os = "macos"
)))]
fn usize2ready_aio(_ready: &mut UnixReady) {
// aio not available here → empty
}
@@ -526,7 +524,7 @@ mod platform {
}
}
#[cfg(any(windows, target_os = "fuchsia"))]
#[cfg(windows)]
mod platform {
use mio::Ready;
-444
View File
@@ -1,444 +0,0 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [`Runtime`]: struct.Runtime.html
//! [`ThreadPool`]: ../executor/thread_pool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
use reactor::{Reactor, Handle, Background};
use tokio_threadpool::{self as threadpool, ThreadPool, Sender};
use futures::Poll;
use futures::future::{self, Future};
use std::{fmt, io};
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
inner: Sender,
}
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
inner: Box<Future<Item = (), Error = ()> + Send>,
}
#[derive(Debug)]
struct Inner {
/// Reactor running on a background thread.
reactor: Background,
/// Task execution pool.
pool: ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
let mut runtime = Runtime::new().unwrap();
runtime.spawn(future);
runtime.shutdown_on_idle().wait().unwrap();
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
// Spawn a reactor on a background thread.
let reactor = Reactor::new()?.background()?;
// Get a handle to the reactor.
let handle = reactor.handle().clone();
let pool = threadpool::Builder::new()
.around_worker(move |w, enter| {
::tokio_reactor::with_default(&handle, enter, |_| {
w.run();
});
})
.build();
Ok(Runtime {
inner: Some(Inner {
reactor,
pool,
}),
})
}
/// Return a reference to the reactor handle for this runtime instance.
pub fn handle(&self) -> &Handle {
self.inner().reactor.handle()
}
/// Return a handle to the runtime's executor.
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_on_idle().and_then(|_| {
reactor.shutdown_on_idle()
})
});
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = Box::new({
let pool = inner.pool;
let reactor = inner.reactor;
pool.shutdown_now().and_then(|_| {
reactor.shutdown_now()
})
});
Shutdown { inner }
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
// ===== impl TaskExecutor =====
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
}
// ===== impl Shutdown =====
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
+88
View File
@@ -0,0 +1,88 @@
use executor::current_thread::CurrentThread;
use runtime::current_thread::Runtime;
use tokio_reactor::Reactor;
use tokio_timer::clock::Clock;
use tokio_timer::timer::Timer;
use std::io;
/// Builds a Single-threaded runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use tokio::runtime::current_thread::Builder;
/// use tokio_timer::clock::Clock;
///
/// # pub fn main() {
/// // build Runtime
/// let runtime = Builder::new()
/// .clock(Clock::new())
/// .build();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
#[derive(Debug)]
pub struct Builder {
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
Builder {
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Create the configured `Runtime`.
pub fn build(&mut self) -> io::Result<Runtime> {
// We need a reactor to receive events about IO objects from kernel
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
// Place a timer wheel on top of the reactor. If there are no timeouts to fire, it'll let the
// reactor pick up some new external events.
let timer = Timer::new_with_now(reactor, self.clock.clone());
let timer_handle = timer.handle();
// And now put a single-threaded executor on top of the timer. When there are no futures ready
// to do something, it'll let the timer or the reactor to generate some new stimuli for the
// futures to continue in their life.
let executor = CurrentThread::new_with_park(timer);
let runtime = Runtime::new2(
reactor_handle,
timer_handle,
self.clock.clone(),
executor);
Ok(runtime)
}
}
+107
View File
@@ -0,0 +1,107 @@
//! A runtime implementation that runs everything on the current thread.
//!
//! [`current_thread::Runtime`][rt] is similar to the primary
//! [`Runtime`][concurrent-rt] except that it runs all components on the current
//! thread instead of using a thread pool. This means that it is able to spawn
//! futures that do not implement `Send`.
//!
//! Same as the default [`Runtime`][concurrent-rt], the
//! [`current_thread::Runtime`][rt] includes:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! Note that [`current_thread::Runtime`][rt] does not implement `Send` itself
//! and cannot be safely moved to other threads.
//!
//! # Spawning from other threads
//!
//! While [`current_thread::Runtime`][rt] does not implement `Send` and cannot
//! safely be moved to other threads, it provides a `Handle` that can be sent
//! to other threads and allows to spawn new tasks from there.
//!
//! For example:
//!
//! ```
//! # extern crate tokio;
//! # extern crate futures;
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//! use std::thread;
//!
//! # fn main() {
//! let mut runtime = Runtime::new().unwrap();
//! let handle = runtime.handle();
//!
//! thread::spawn(move || {
//! handle.spawn(future::ok(()));
//! }).join().unwrap();
//!
//! # /*
//! runtime.run().unwrap();
//! # */
//! # }
//! ```
//!
//! # Examples
//!
//! Creating a new `Runtime` and running a future `f` until its completion and
//! returning its result.
//!
//! ```
//! use tokio::runtime::current_thread::Runtime;
//! use tokio::prelude::*;
//!
//! let mut runtime = Runtime::new().unwrap();
//!
//! // Use the runtime...
//! // runtime.block_on(f); // where f is a future
//! ```
//!
//! [rt]: struct.Runtime.html
//! [concurrent-rt]: ../struct.Runtime.html
//! [chan]: https://docs.rs/futures/0.1/futures/sync/mpsc/fn.channel.html
//! [reactor]: ../../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../../timer/index.html
mod builder;
mod runtime;
pub use self::builder::Builder;
pub use self::runtime::{Runtime, Handle};
pub use tokio_current_thread::spawn;
pub use tokio_current_thread::TaskExecutor;
use futures::Future;
/// Run the provided future to completion using a runtime running on the current thread.
///
/// This first creates a new [`Runtime`], and calls [`Runtime::block_on`] with the provided future,
/// which blocks the current thread until the provided future completes. It then calls
/// [`Runtime::run`] to wait for any other spawned futures to resolve.
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where
F: Future,
{
let mut r = Runtime::new().expect("failed to start runtime on current thread");
let v = r.block_on(future)?;
r.run().expect("failed to resolve remaining futures");
Ok(v)
}
/// Start a current-thread runtime using the supplied future to bootstrap execution.
///
/// # Panics
///
/// This function panics if called from the context of an executor.
pub fn run<F>(future: F)
where
F: Future<Item = (), Error = ()> + 'static,
{
let mut r = Runtime::new().expect("failed to start runtime on current thread");
r.spawn(future);
r.run().expect("failed to resolve remaining futures");
}
+238
View File
@@ -0,0 +1,238 @@
use tokio_current_thread::{self as current_thread, CurrentThread};
use tokio_current_thread::Handle as ExecutorHandle;
use runtime::current_thread::Builder;
use tokio_reactor::{self, Reactor};
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
use tokio_executor;
use futures::{future, Future};
use std::fmt;
use std::error::Error;
use std::io;
/// Single-threaded runtime provides a way to start reactor
/// and executor on the current thread.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
#[derive(Debug)]
pub struct Runtime {
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>,
}
/// Handle to spawn a future on the corresponding `CurrentThread` runtime instance
#[derive(Debug, Clone)]
pub struct Handle(ExecutorHandle);
impl Handle {
/// Spawn a future onto the `CurrentThread` runtime instance corresponding to this handle
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the `CurrentThread`
/// instance of the `Handle` does not exist anymore.
pub fn spawn<F>(&self, future: F) -> Result<(), tokio_executor::SpawnError>
where F: Future<Item = (), Error = ()> + Send + 'static {
self.0.spawn(future)
}
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
pub fn status(&self) -> Result<(), tokio_executor::SpawnError> {
self.0.status()
}
}
impl<T> future::Executor<T> for Handle
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
if let Err(e) = self.status() {
let kind = if e.is_at_capacity() {
future::ExecuteErrorKind::NoCapacity
} else {
future::ExecuteErrorKind::Shutdown
};
return Err(future::ExecuteError::new(kind, future));
}
let _ = self.spawn(future);
Ok(())
}
}
/// Error returned by the `run` function.
#[derive(Debug)]
pub struct RunError {
inner: current_thread::RunError,
}
impl fmt::Display for RunError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.inner)
}
}
impl Error for RunError {
fn description(&self) -> &str {
self.inner.description()
}
// FIXME(taiki-e): When the minimum support version of tokio reaches Rust 1.30,
// replace this with Error::source.
#[allow(deprecated)]
fn cause(&self) -> Option<&Error> {
self.inner.cause()
}
}
impl Runtime {
/// Returns a new runtime initialized with default configuration values.
pub fn new() -> io::Result<Runtime> {
Builder::new().build()
}
pub(super) fn new2(
reactor_handle: tokio_reactor::Handle,
timer_handle: timer::Handle,
clock: Clock,
executor: CurrentThread<Timer<Reactor>>) -> Runtime
{
Runtime {
reactor_handle,
timer_handle,
clock,
executor,
}
}
/// Get a new handle to spawn futures on the single-threaded Tokio runtime
///
/// Different to the runtime itself, the handle can be sent to different
/// threads.
pub fn handle(&self) -> Handle {
Handle(self.executor.handle().clone())
}
/// Spawn a future onto the single-threaded Tokio runtime.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::current_thread::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("running on the runtime");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
{
self.executor.spawn(future);
self
}
/// Runs the provided future, blocking the current thread until the future
/// completes.
///
/// This function can be used to synchronously block the current thread
/// until the provided `future` has resolved either successfully or with an
/// error. The result of the future is then returned from this function
/// call.
///
/// Note that this function will **also** execute any spawned futures on the
/// current thread, but will **not** block until these other spawned futures
/// have completed. Once the function returns, any uncompleted futures
/// remain pending in the `Runtime` instance. These futures will not run
/// until `block_on` or `run` is called again.
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution by calling `block_on` or `run`.
pub fn block_on<F>(&mut self, f: F) -> Result<F::Item, F::Error>
where F: Future
{
self.enter(|executor| {
// Run the provided future
let ret = executor.block_on(f);
ret.map_err(|e| e.into_inner().expect("unexpected execution error"))
})
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.enter(|executor| executor.run())
.map_err(|e| RunError {
inner: e,
})
}
fn enter<F, R>(&mut self, f: F) -> R
where F: FnOnce(&mut current_thread::Entered<Timer<Reactor>>) -> R
{
let Runtime {
ref reactor_handle,
ref timer_handle,
ref clock,
ref mut executor,
..
} = *self;
// Binds an executor to this thread
let mut enter = tokio_executor::enter().expect("Multiple executors at once");
// This will set the default handle and timer to use inside the closure
// and run the future.
tokio_reactor::with_default(&reactor_handle, &mut enter, |enter| {
clock::with_default(clock, enter, |enter| {
timer::with_default(&timer_handle, enter, |enter| {
// The TaskExecutor is a fake executor that looks into the
// current single-threaded executor when used. This is a trick,
// because we need two mutable references to the executor (one
// to run the provided future, another to install as the default
// one). We use the fake one here as the default one.
let mut default_executor = current_thread::TaskExecutor::current();
tokio_executor::with_default(&mut default_executor, enter, |enter| {
let mut executor = executor.enter(enter);
f(&mut executor)
})
})
})
})
}
}
+125
View File
@@ -0,0 +1,125 @@
//! A batteries included runtime for applications using Tokio.
//!
//! Applications using Tokio require some runtime support in order to work:
//!
//! * A [reactor] to drive I/O resources.
//! * An [executor] to execute tasks that use these I/O resources.
//! * A [timer] for scheduling work to run after a set period of time.
//!
//! While it is possible to setup each component manually, this involves a bunch
//! of boilerplate.
//!
//! [`Runtime`] bundles all of these various runtime components into a single
//! handle that can be started and shutdown together, eliminating the necessary
//! boilerplate to run a Tokio application.
//!
//! Most applications wont need to use [`Runtime`] directly. Instead, they will
//! use the [`run`] function, which uses [`Runtime`] under the hood.
//!
//! Creating a [`Runtime`] does the following:
//!
//! * Spawn a background thread running a [`Reactor`] instance.
//! * Start a [`ThreadPool`] for executing futures.
//! * Run an instance of [`Timer`] **per** thread pool worker thread.
//!
//! The thread pool uses a work-stealing strategy and is configured to start a
//! worker thread for each CPU core available on the system. This tends to be
//! the ideal setup for Tokio applications.
//!
//! A timer per thread pool worker thread is used to minimize the amount of
//! synchronization that is required for working with the timer.
//!
//! # Usage
//!
//! Most applications will use the [`run`] function. This takes a future to
//! "seed" the application, blocking the thread until the runtime becomes
//! [idle].
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! tokio::run(server);
//! # }
//! # pub fn main() {}
//! ```
//!
//! In this function, the `run` function blocks until the runtime becomes idle.
//! See [`shutdown_on_idle`][idle] for more shutdown details.
//!
//! From within the context of the runtime, additional tasks are spawned using
//! the [`tokio::spawn`] function. Futures spawned using this function will be
//! executed on the same thread pool used by the [`Runtime`].
//!
//! A [`Runtime`] instance can also be used directly.
//!
//! ```rust
//! # extern crate tokio;
//! # extern crate futures;
//! # use futures::{Future, Stream};
//! use tokio::runtime::Runtime;
//! use tokio::net::TcpListener;
//!
//! # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
//! # unimplemented!();
//! # }
//! # fn dox() {
//! # let addr = "127.0.0.1:8080".parse().unwrap();
//! let listener = TcpListener::bind(&addr).unwrap();
//!
//! let server = listener.incoming()
//! .map_err(|e| println!("error = {:?}", e))
//! .for_each(|socket| {
//! tokio::spawn(process(socket))
//! });
//!
//! // Create the runtime
//! let mut rt = Runtime::new().unwrap();
//!
//! // Spawn the server task
//! rt.spawn(server);
//!
//! // Wait until the runtime becomes idle and shut it down.
//! rt.shutdown_on_idle()
//! .wait().unwrap();
//! # }
//! # pub fn main() {}
//! ```
//!
//! [reactor]: ../reactor/struct.Reactor.html
//! [executor]: https://tokio.rs/docs/getting-started/runtime-model/#executors
//! [timer]: ../timer/index.html
//! [`Runtime`]: struct.Runtime.html
//! [`Reactor`]: ../reactor/struct.Reactor.html
//! [`ThreadPool`]: https://docs.rs/tokio-threadpool/0.1/tokio_threadpool/struct.ThreadPool.html
//! [`run`]: fn.run.html
//! [idle]: struct.Runtime.html#method.shutdown_on_idle
//! [`tokio::spawn`]: ../executor/fn.spawn.html
//! [`Timer`]: https://docs.rs/tokio-timer/0.2/tokio_timer/timer/struct.Timer.html
pub mod current_thread;
mod threadpool;
pub use self::threadpool::{
Builder,
Runtime,
Shutdown,
TaskExecutor,
run,
};
+378
View File
@@ -0,0 +1,378 @@
use super::{Inner, Runtime};
use reactor::Reactor;
use std::io;
use std::sync::Mutex;
use std::time::Duration;
use num_cpus;
use tokio_reactor;
use tokio_threadpool::Builder as ThreadPoolBuilder;
use tokio_timer::clock::{self, Clock};
use tokio_timer::timer::{self, Timer};
use tokio_trace_core as trace;
/// Builds Tokio Runtime with custom configuration values.
///
/// Methods can be chained in order to set the configuration values. The
/// Runtime is constructed by calling [`build`].
///
/// New instances of `Builder` are obtained via [`Builder::new`].
///
/// See function level documentation for details on the various configuration
/// settings.
///
/// [`build`]: #method.build
/// [`Builder::new`]: #method.new
///
/// # Examples
///
/// ```
/// extern crate tokio;
/// extern crate tokio_timer;
///
/// use std::time::Duration;
///
/// use tokio::runtime::Builder;
/// use tokio_timer::clock::Clock;
///
/// fn main() {
/// // build Runtime
/// let mut runtime = Builder::new()
/// .blocking_threads(4)
/// .clock(Clock::system())
/// .core_threads(4)
/// .keep_alive(Some(Duration::from_secs(60)))
/// .name_prefix("my-custom-name-")
/// .stack_size(3 * 1024 * 1024)
/// .build()
/// .unwrap();
///
/// // use runtime ...
/// }
/// ```
#[derive(Debug)]
pub struct Builder {
/// Thread pool specific builder
threadpool_builder: ThreadPoolBuilder,
/// The number of worker threads
core_threads: usize,
/// The clock to use
clock: Clock,
}
impl Builder {
/// Returns a new runtime builder initialized with default configuration
/// values.
///
/// Configuration methods can be chained on the return value.
pub fn new() -> Builder {
let core_threads = num_cpus::get().max(1);
let mut threadpool_builder = ThreadPoolBuilder::new();
threadpool_builder.name_prefix("tokio-runtime-worker-");
threadpool_builder.pool_size(core_threads);
Builder {
threadpool_builder,
core_threads,
clock: Clock::new(),
}
}
/// Set the `Clock` instance that will be used by the runtime.
pub fn clock(&mut self, clock: Clock) -> &mut Self {
self.clock = clock;
self
}
/// Set builder to set up the thread pool instance.
#[deprecated(
since = "0.1.9",
note = "use the `core_threads`, `blocking_threads`, `name_prefix`, \
`keep_alive`, and `stack_size` functions on `runtime::Builder`, \
instead")]
#[doc(hidden)]
pub fn threadpool_builder(&mut self, val: ThreadPoolBuilder) -> &mut Self {
self.threadpool_builder = val;
self
}
/// Set the maximum number of worker threads for the `Runtime`'s thread pool.
///
/// This must be a number between 1 and 32,768 though it is advised to keep
/// this value on the smaller side.
///
/// The default value is the number of cores available to the system.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .core_threads(4)
/// .build()
/// .unwrap();
/// # }
/// ```
pub fn core_threads(&mut self, val: usize) -> &mut Self {
self.core_threads = val;
self.threadpool_builder.pool_size(val);
self
}
/// Set the maximum number of concurrent blocking sections in the `Runtime`'s
/// thread pool.
///
/// When the maximum concurrent `blocking` calls is reached, any further
/// calls to `blocking` will return `NotReady` and the task is notified once
/// previously in-flight calls to `blocking` return.
///
/// This must be a number between 1 and 32,768 though it is advised to keep
/// this value on the smaller side.
///
/// The default value is 100.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .blocking_threads(200)
/// .build();
/// # }
/// ```
pub fn blocking_threads(&mut self, val: usize) -> &mut Self {
self.threadpool_builder.max_blocking(val);
self
}
/// Set the worker thread keep alive duration for threads in the `Runtime`'s
/// thread pool.
///
/// If set, a worker thread will wait for up to the specified duration for
/// work, at which point the thread will shutdown. When work becomes
/// available, a new thread will eventually be spawned to replace the one
/// that shut down.
///
/// When the value is `None`, the thread will wait for work forever.
///
/// The default value is `None`.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
/// use std::time::Duration;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .keep_alive(Some(Duration::from_secs(30)))
/// .build();
/// # }
/// ```
pub fn keep_alive(&mut self, val: Option<Duration>) -> &mut Self {
self.threadpool_builder.keep_alive(val);
self
}
/// Set name prefix of threads spawned by the `Runtime`'s thread pool.
///
/// Thread name prefix is used for generating thread names. For example, if
/// prefix is `my-pool-`, then threads in the pool will get names like
/// `my-pool-1` etc.
///
/// The default prefix is "tokio-runtime-worker-".
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .name_prefix("my-pool-")
/// .build();
/// # }
/// ```
pub fn name_prefix<S: Into<String>>(&mut self, val: S) -> &mut Self {
self.threadpool_builder.name_prefix(val);
self
}
/// Set the stack size (in bytes) for worker threads.
///
/// The actual stack size may be greater than this value if the platform
/// specifies minimal stack size.
///
/// The default stack size for spawned threads is 2 MiB, though this
/// particular stack size is subject to change in the future.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let mut rt = runtime::Builder::new()
/// .stack_size(32 * 1024)
/// .build();
/// # }
/// ```
pub fn stack_size(&mut self, val: usize) -> &mut Self {
self.threadpool_builder.stack_size(val);
self
}
/// Execute function `f` after each thread is started but before it starts
/// doing work.
///
/// This is intended for bookkeeping and monitoring use cases.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let thread_pool = runtime::Builder::new()
/// .after_start(|| {
/// println!("thread started");
/// })
/// .build();
/// # }
/// ```
pub fn after_start<F>(&mut self, f: F) -> &mut Self
where F: Fn() + Send + Sync + 'static
{
self.threadpool_builder.after_start(f);
self
}
/// Execute function `f` before each thread stops.
///
/// This is intended for bookkeeping and monitoring use cases.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// # use tokio::runtime;
///
/// # pub fn main() {
/// let thread_pool = runtime::Builder::new()
/// .before_stop(|| {
/// println!("thread stopping");
/// })
/// .build();
/// # }
/// ```
pub fn before_stop<F>(&mut self, f: F) -> &mut Self
where F: Fn() + Send + Sync + 'static
{
self.threadpool_builder.before_stop(f);
self
}
/// Create the configured `Runtime`.
///
/// The returned `ThreadPool` instance is ready to spawn tasks.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # use tokio::runtime::Builder;
/// # pub fn main() {
/// let runtime = Builder::new().build().unwrap();
/// // ... call runtime.run(...)
/// # let _ = runtime;
/// # }
/// ```
pub fn build(&mut self) -> io::Result<Runtime> {
// TODO(stjepang): Once we remove the `threadpool_builder` method, remove this line too.
self.threadpool_builder.pool_size(self.core_threads);
let mut reactor_handles = Vec::new();
let mut timer_handles = Vec::new();
let mut timers = Vec::new();
for _ in 0..self.core_threads {
// Create a new reactor.
let reactor = Reactor::new()?;
reactor_handles.push(reactor.handle());
// Create a new timer.
let timer = Timer::new_with_now(reactor, self.clock.clone());
timer_handles.push(timer.handle());
timers.push(Mutex::new(Some(timer)));
}
// Get a handle to the clock for the runtime.
let clock = self.clock.clone();
// Get the current trace dispatcher.
// TODO(eliza): when `tokio-trace-core` is stable enough to take a
// public API dependency, we should allow users to set a custom
// subscriber for the runtime.
let dispatch = trace::dispatcher::get_default(trace::Dispatch::clone);
let pool = self
.threadpool_builder
.around_worker(move |w, enter| {
let index = w.id().to_usize();
tokio_reactor::with_default(&reactor_handles[index], enter, |enter| {
clock::with_default(&clock, enter, |enter| {
timer::with_default(&timer_handles[index], enter, |_| {
trace::dispatcher::with_default(&dispatch, || {
w.run();
})
});
})
});
})
.custom_park(move |worker_id| {
let index = worker_id.to_usize();
timers[index]
.lock()
.unwrap()
.take()
.unwrap()
})
.build();
// To support deprecated `reactor()` function
let reactor = Reactor::new()?;
let reactor_handle = reactor.handle();
Ok(Runtime {
inner: Some(Inner {
reactor_handle,
reactor: Mutex::new(Some(reactor)),
pool,
}),
})
}
}
+395
View File
@@ -0,0 +1,395 @@
mod builder;
mod shutdown;
mod task_executor;
pub use self::builder::Builder;
pub use self::shutdown::Shutdown;
pub use self::task_executor::TaskExecutor;
use reactor::{Handle, Reactor};
use std::io;
use std::sync::Mutex;
use tokio_executor::enter;
use tokio_threadpool as threadpool;
use futures;
use futures::future::Future;
/// Handle to the Tokio runtime.
///
/// The Tokio runtime includes a reactor as well as an executor for running
/// tasks.
///
/// Instances of `Runtime` can be created using [`new`] or [`Builder`]. However,
/// most users will use [`tokio::run`], which uses a `Runtime` internally.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
/// [`new`]: #method.new
/// [`Builder`]: struct.Builder.html
/// [`tokio::run`]: fn.run.html
#[derive(Debug)]
pub struct Runtime {
inner: Option<Inner>,
}
#[derive(Debug)]
struct Inner {
/// A handle to the reactor in the background thread.
reactor_handle: Handle,
// TODO: This should go away in 0.2
reactor: Mutex<Option<Reactor>>,
/// Task execution pool.
pool: threadpool::ThreadPool,
}
// ===== impl Runtime =====
/// Start the Tokio runtime using the supplied future to bootstrap execution.
///
/// This function is used to bootstrap the execution of a Tokio application. It
/// does the following:
///
/// * Start the Tokio runtime using a default configuration.
/// * Spawn the given future onto the thread pool.
/// * Block the current thread until the runtime shuts down.
///
/// Note that the function will not return immediately once `future` has
/// completed. Instead it waits for the entire runtime to become idle.
///
/// See the [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{Future, Stream};
/// use tokio::net::TcpListener;
///
/// # fn process<T>(_: T) -> Box<Future<Item = (), Error = ()> + Send> {
/// # unimplemented!();
/// # }
/// # fn dox() {
/// # let addr = "127.0.0.1:8080".parse().unwrap();
/// let listener = TcpListener::bind(&addr).unwrap();
///
/// let server = listener.incoming()
/// .map_err(|e| println!("error = {:?}", e))
/// .for_each(|socket| {
/// tokio::spawn(process(socket))
/// });
///
/// tokio::run(server);
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if called from the context of an executor.
///
/// [mod]: ../index.html
pub fn run<F>(future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
// Check enter before creating a new Runtime...
let mut entered = enter().expect("nested tokio::run");
let mut runtime = Runtime::new().expect("failed to start new Runtime");
runtime.spawn(future);
entered
.block_on(runtime.shutdown_on_idle())
.expect("shutdown cannot error")
}
impl Runtime {
/// Create a new runtime instance with default configuration values.
///
/// This results in a reactor, thread pool, and timer being initialized. The
/// thread pool will not spawn any worker threads until it needs to, i.e.
/// tasks are scheduled to run.
///
/// Most users will not need to call this function directly, instead they
/// will use [`tokio::run`](fn.run.html).
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// Creating a new `Runtime` with default configuration values.
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn new() -> io::Result<Self> {
Builder::new().build()
}
#[deprecated(since = "0.1.5", note = "use `reactor` instead")]
#[doc(hidden)]
pub fn handle(&self) -> &Handle {
#[allow(deprecated)]
self.reactor()
}
/// Return a reference to the reactor handle for this runtime instance.
///
/// The returned handle reference can be cloned in order to get an owned
/// value of the handle. This handle can be used to initialize I/O resources
/// (like TCP or UDP sockets) that will not be used on the runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let reactor_handle = rt.reactor().clone();
///
/// // use `reactor_handle`
/// ```
#[deprecated(since = "0.1.11", note = "there is now a reactor per worker thread")]
pub fn reactor(&self) -> &Handle {
let mut reactor = self.inner().reactor.lock().unwrap();
if let Some(reactor) = reactor.take() {
if let Ok(background) = reactor.background() {
background.forget();
}
}
&self.inner().reactor_handle
}
/// Return a handle to the runtime's executor.
///
/// The returned handle can be used to spawn tasks that run on this runtime.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// let executor_handle = rt.executor();
///
/// // use `executor_handle`
/// ```
pub fn executor(&self) -> TaskExecutor {
let inner = self.inner().pool.sender().clone();
TaskExecutor { inner }
}
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
///
/// // Spawn a future onto the runtime
/// rt.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner_mut().pool.sender().spawn(future).unwrap();
self
}
/// Run a future to completion on the Tokio runtime.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, and yielding its resolved result. Any tasks or timers which
/// the future spawns internally will be executed on the runtime.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on<F, R, E>(&mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
entered.block_on(rx).unwrap()
}
/// Run a future to completion on the Tokio runtime, then wait for all
/// background futures to complete too.
///
/// This runs the given future on the runtime, blocking until it is
/// complete, waiting for background futures to complete, and yielding
/// its resolved result. Any tasks or timers which the future spawns
/// internally will be executed on the runtime and waited for completion.
///
/// This method should not be called from an asynchronous context.
///
/// # Panics
///
/// This function panics if the executor is at capacity, if the provided
/// future panics, or if called within an asynchronous execution context.
pub fn block_on_all<F, R, E>(mut self, future: F) -> Result<R, E>
where
F: Send + 'static + Future<Item = R, Error = E>,
R: Send + 'static,
E: Send + 'static,
{
let mut entered = enter().expect("nested block_on_all");
let (tx, rx) = futures::sync::oneshot::channel();
self.spawn(future.then(move |r| tx.send(r).map_err(|_| unreachable!())));
let block = rx
.map_err(|_| unreachable!())
.and_then(move |r| {
self.shutdown_on_idle()
.map(move |()| r)
});
entered.block_on(block).unwrap()
}
/// Signals the runtime to shutdown once it becomes idle.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function can be used to perform a graceful shutdown of the runtime.
///
/// The runtime enters an idle state once **all** of the following occur.
///
/// * The thread pool has no tasks to execute, i.e., all tasks that were
/// spawned have completed.
/// * The reactor is not managing any I/O resources.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_on_idle()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_on_idle(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
let inner = inner.pool.shutdown_on_idle();
Shutdown { inner }
}
/// Signals the runtime to shutdown immediately.
///
/// Returns a future that completes once the shutdown operation has
/// completed.
///
/// This function will forcibly shutdown the runtime, causing any
/// in-progress work to become canceled. The shutdown steps are:
///
/// * Drain any scheduled work queues.
/// * Drop any futures that have not yet completed.
/// * Drop the reactor.
///
/// Once the reactor has dropped, any outstanding I/O resources bound to
/// that reactor will no longer function. Calling any method on them will
/// result in an error.
///
/// See [module level][mod] documentation for more details.
///
/// # Examples
///
/// ```
/// use tokio::runtime::Runtime;
/// use tokio::prelude::*;
///
/// let rt = Runtime::new()
/// .unwrap();
///
/// // Use the runtime...
///
/// // Shutdown the runtime
/// rt.shutdown_now()
/// .wait().unwrap();
/// ```
///
/// [mod]: index.html
pub fn shutdown_now(mut self) -> Shutdown {
let inner = self.inner.take().unwrap();
Shutdown::shutdown_now(inner)
}
fn inner(&self) -> &Inner {
self.inner.as_ref().unwrap()
}
fn inner_mut(&mut self) -> &mut Inner {
self.inner.as_mut().unwrap()
}
}
impl Drop for Runtime {
fn drop(&mut self) {
if let Some(inner) = self.inner.take() {
let shutdown = Shutdown::shutdown_now(inner);
let _ = shutdown.wait();
}
}
}
+36
View File
@@ -0,0 +1,36 @@
use super::Inner;
use tokio_threadpool as threadpool;
use std::fmt;
use futures::{Future, Poll};
/// A future that resolves when the Tokio `Runtime` is shut down.
pub struct Shutdown {
pub(super) inner: threadpool::Shutdown,
}
impl Shutdown {
pub(super) fn shutdown_now(inner: Inner) -> Self {
let inner = inner.pool.shutdown_now();
Shutdown { inner }
}
}
impl Future for Shutdown {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
try_ready!(self.inner.poll());
Ok(().into())
}
}
impl fmt::Debug for Shutdown {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Shutdown")
.field("inner", &"Box<Future<Item = (), Error = ()>>")
.finish()
}
}
+75
View File
@@ -0,0 +1,75 @@
use tokio_threadpool::Sender;
use futures::future::{self, Future};
/// Executes futures on the runtime
///
/// All futures spawned using this executor will be submitted to the associated
/// Runtime's executor. This executor is usually a thread pool.
///
/// For more details, see the [module level](index.html) documentation.
#[derive(Debug, Clone)]
pub struct TaskExecutor {
pub(super) inner: Sender,
}
impl TaskExecutor {
/// Spawn a future onto the Tokio runtime.
///
/// This spawns the given future onto the runtime's executor, usually a
/// thread pool. The thread pool is then responsible for polling the future
/// until it completes.
///
/// See [module level][mod] documentation for more details.
///
/// [mod]: index.html
///
/// # Examples
///
/// ```rust
/// # extern crate tokio;
/// # extern crate futures;
/// # use futures::{future, Future, Stream};
/// use tokio::runtime::Runtime;
///
/// # fn dox() {
/// // Create the runtime
/// let mut rt = Runtime::new().unwrap();
/// let executor = rt.executor();
///
/// // Spawn a future onto the runtime
/// executor.spawn(future::lazy(|| {
/// println!("now running on a worker thread");
/// Ok(())
/// }));
/// # }
/// # pub fn main() {}
/// ```
///
/// # Panics
///
/// This function panics if the spawn fails. Failure occurs if the executor
/// is currently at capacity and is unable to spawn a new future.
pub fn spawn<F>(&self, future: F)
where F: Future<Item = (), Error = ()> + Send + 'static,
{
self.inner.spawn(future).unwrap();
}
}
impl<T> future::Executor<T> for TaskExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
self.inner.execute(future)
}
}
impl ::executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), ::executor::SpawnError>
{
self.inner.spawn(future)
}
}
+15
View File
@@ -0,0 +1,15 @@
//! Future-aware synchronization
//!
//! This module is enabled with the **`sync`** feature flag.
//!
//! Tasks sometimes need to communicate with each other. This module contains
//! two basic abstractions for doing so:
//!
//! - [oneshot](oneshot/index.html), a way of sending a single value
//! from one task to another.
//! - [mpsc](mpsc/index.html), a multi-producer, single-consumer channel for
//! sending values between tasks.
//! - [watch](watch/index.html), a single-producer, multi-consumer channel that
//! only stores the **most recently** sent value.
pub use tokio_sync::{mpsc, oneshot, watch};
+94
View File
@@ -0,0 +1,94 @@
//! Utilities for tracking time.
//!
//! This module provides a number of types for executing code after a set period
//! of time.
//!
//! * [`Delay`][Delay] is a future that does no work and completes at a specific `Instant`
//! in time.
//!
//! * [`Interval`][Interval] is a stream yielding a value at a fixed period. It
//! is initialized with a `Duration` and repeatedly yields each time the
//! duration elapses.
//!
//! * [`Timeout`][Timeout]: Wraps a future or stream, setting an upper bound to the
//! amount of time it is allowed to execute. If the future or stream does not
//! complete in time, then it is canceled and an error is returned.
//!
//! * [`DelayQueue`]: A queue where items are returned once the requested delay
//! has expired.
//!
//! These types are sufficient for handling a large number of scenarios
//! involving time.
//!
//! These types must be used from within the context of the
//! [`Runtime`][runtime] or a timer context must be setup explicitly. See the
//! [`tokio-timer`][tokio-timer] crate for more details on how to setup a timer
//! context.
//!
//! # Examples
//!
//! Wait 100ms and print "Hello World!"
//!
//! ```
//! use tokio::prelude::*;
//! use tokio::timer::Delay;
//!
//! use std::time::{Duration, Instant};
//!
//! let when = Instant::now() + Duration::from_millis(100);
//!
//! tokio::run({
//! Delay::new(when)
//! .map_err(|e| panic!("timer failed; err={:?}", e))
//! .and_then(|_| {
//! println!("Hello world!");
//! Ok(())
//! })
//! })
//! ```
//!
//! Require that an operation takes no more than 300ms. Note that this uses the
//! [`timeout`][ext] function on the [`FutureExt`][ext] trait. This trait is
//! included in the prelude.
//!
//! ```
//! # extern crate futures;
//! # extern crate tokio;
//! use tokio::prelude::*;
//!
//! use std::time::{Duration, Instant};
//!
//! fn long_op() -> Box<Future<Item = (), Error = ()> + Send> {
//! // ...
//! # Box::new(futures::future::ok(()))
//! }
//!
//! # fn main() {
//! tokio::run({
//! long_op()
//! .timeout(Duration::from_millis(300))
//! .map_err(|e| {
//! println!("operation timed out");
//! })
//! })
//! # }
//! ```
//!
//! [runtime]: ../runtime/struct.Runtime.html
//! [tokio-timer]: https://docs.rs/tokio-timer
//! [ext]: ../util/trait.FutureExt.html#method.timeout
//! [Timeout]: struct.Timeout.html
//! [Delay]: struct.Delay.html
//! [Interval]: struct.Interval.html
//! [`DelayQueue`]: struct.DelayQueue.html
pub use tokio_timer::{delay_queue, timeout, Delay, DelayQueue, Error, Interval, Timeout};
#[deprecated(since = "0.1.8", note = "use Timeout instead")]
#[allow(deprecated)]
#[doc(hidden)]
pub type Deadline<T> = ::tokio_timer::Deadline<T>;
#[deprecated(since = "0.1.8", note = "use Timeout instead")]
#[allow(deprecated)]
#[doc(hidden)]
pub type DeadlineError<T> = ::tokio_timer::DeadlineError<T>;
+84
View File
@@ -0,0 +1,84 @@
use futures::{Async, Poll, Sink, StartSend, Stream};
/// A stream combinator which combines the yields the current item
/// plus its count starting from 0.
///
/// This structure is produced by the `Stream::enumerate` method.
#[derive(Debug)]
#[must_use = "Does nothing unless polled"]
pub struct Enumerate<T> {
inner: T,
count: usize,
}
impl<T> Enumerate<T> {
pub(crate) fn new(stream: T) -> Self {
Self {
inner: stream,
count: 0,
}
}
/// Acquires a reference to the underlying stream that this combinator is
/// pulling from.
pub fn get_ref(&self) -> &T {
&self.inner
}
/// Acquires a mutable reference to the underlying stream that this
/// combinator is pulling from.
///
/// Note that care must be taken to avoid tampering with the state of the
/// stream which may otherwise confuse this combinator.
pub fn get_mut(&mut self) -> &mut T {
&mut self.inner
}
/// Consumes this combinator, returning the underlying stream.
///
/// Note that this may discard intermediate state of this combinator, so
/// care should be taken to avoid losing resources when this is called.
pub fn into_inner(self) -> T {
self.inner
}
}
impl<T> Stream for Enumerate<T>
where
T: Stream,
{
type Item = (usize, T::Item);
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, T::Error> {
match try_ready!(self.inner.poll()) {
Some(item) => {
let ret = Some((self.count, item));
self.count += 1;
Ok(Async::Ready(ret))
}
None => return Ok(Async::Ready(None)),
}
}
}
// Forwarding impl of Sink from the underlying stream
impl<T> Sink for Enumerate<T>
where
T: Sink,
{
type SinkItem = T::SinkItem;
type SinkError = T::SinkError;
fn start_send(&mut self, item: T::SinkItem) -> StartSend<T::SinkItem, T::SinkError> {
self.inner.start_send(item)
}
fn poll_complete(&mut self) -> Poll<(), T::SinkError> {
self.inner.poll_complete()
}
fn close(&mut self) -> Poll<(), T::SinkError> {
self.inner.close()
}
}
+93
View File
@@ -0,0 +1,93 @@
#[cfg(feature = "timer")]
#[allow(deprecated)]
use tokio_timer::Deadline;
#[cfg(feature = "timer")]
use tokio_timer::Timeout;
use futures::Future;
#[cfg(feature = "timer")]
use std::time::{Duration, Instant};
/// An extension trait for `Future` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there only is a [`timeout`] function, but this will increase
/// over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Future` already implement `FutureExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`timeout`]: #method.timeout
pub trait FutureExt: Future {
/// Creates a new future which allows `self` until `timeout`.
///
/// This combinator creates a new future which wraps the receiving future
/// with a timeout. The returned future is allowed to execute until it
/// completes or `timeout` has elapsed, whichever happens first.
///
/// If the future completes before `timeout` then the future will resolve
/// with that item. Otherwise the future will resolve to an error.
///
/// The future is guaranteed to be polled at least once, even if `timeout`
/// is set to zero.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::Duration;
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let future = long_future()
/// .timeout(Duration::from_secs(1))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(future);
/// # }
/// ```
#[cfg(feature = "timer")]
fn timeout(self, timeout: Duration) -> Timeout<Self>
where
Self: Sized,
{
Timeout::new(self, timeout)
}
#[cfg(feature = "timer")]
#[deprecated(since = "0.1.8", note = "use `timeout` instead")]
#[allow(deprecated)]
#[doc(hidden)]
fn deadline(self, deadline: Instant) -> Deadline<Self>
where
Self: Sized,
{
Deadline::new(self, deadline)
}
}
impl<T: ?Sized> FutureExt for T where T: Future {}
#[cfg(test)]
mod test {
use super::*;
use prelude::future;
#[cfg(feature = "timer")]
#[test]
fn timeout_polls_at_least_once() {
let base_future = future::result::<(), ()>(Ok(()));
let timeouted_future = base_future.timeout(Duration::new(0, 0));
assert!(timeouted_future.wait().is_ok());
}
}
+15
View File
@@ -0,0 +1,15 @@
//! Utilities for working with Tokio.
//!
//! This module contains utilities that are useful for working with Tokio.
//! Currently, this only includes [`FutureExt`] and [`StreamExt`], but this
//! may grow over time.
//!
//! [`FutureExt`]: trait.FutureExt.html
//! [`StreamExt`]: trait.StreamExt.html
mod enumerate;
mod future;
mod stream;
pub use self::future::FutureExt;
pub use self::stream::StreamExt;
+95
View File
@@ -0,0 +1,95 @@
#[cfg(feature = "timer")]
use tokio_timer::{throttle::Throttle, Timeout};
use futures::Stream;
#[cfg(feature = "timer")]
use std::time::Duration;
pub use util::enumerate::Enumerate;
/// An extension trait for `Stream` that provides a variety of convenient
/// combinator functions.
///
/// Currently, there are only [`timeout`] and [`throttle`] functions, but
/// this will increase over time.
///
/// Users are not expected to implement this trait. All types that implement
/// `Stream` already implement `StreamExt`.
///
/// This trait can be imported directly or via the Tokio prelude: `use
/// tokio::prelude::*`.
///
/// [`timeout`]: #method.timeout
pub trait StreamExt: Stream {
/// Throttle down the stream by enforcing a fixed delay between items.
///
/// Errors are also delayed.
#[cfg(feature = "timer")]
fn throttle(self, duration: Duration) -> Throttle<Self>
where
Self: Sized,
{
Throttle::new(self, duration)
}
/// Creates a new stream which gives the current iteration count as well
/// as the next value.
///
/// The stream returned yields pairs `(i, val)`, where `i` is the
/// current index of iteration and `val` is the value returned by the
/// iterator.
///
/// # Overflow Behavior
///
/// The method does no guarding against overflows, so counting elements of
/// an iterator with more than [`std::usize::MAX`] elements either produces the
/// wrong result or panics.
fn enumerate(self) -> Enumerate<Self>
where
Self: Sized,
{
Enumerate::new(self)
}
/// Creates a new stream which allows `self` until `timeout`.
///
/// This combinator creates a new stream which wraps the receiving stream
/// with a timeout. For each item, the returned stream is allowed to execute
/// until it completes or `timeout` has elapsed, whichever happens first.
///
/// If an item completes before `timeout` then the stream will yield
/// with that item. Otherwise the stream will yield to an error.
///
/// # Examples
///
/// ```
/// # extern crate tokio;
/// # extern crate futures;
/// use tokio::prelude::*;
/// use std::time::Duration;
/// # use futures::future::{self, FutureResult};
///
/// # fn long_future() -> FutureResult<(), ()> {
/// # future::ok(())
/// # }
/// #
/// # fn main() {
/// let stream = long_future()
/// .into_stream()
/// .timeout(Duration::from_secs(1))
/// .for_each(|i| future::ok(println!("item = {:?}", i)))
/// .map_err(|e| println!("error = {:?}", e));
///
/// tokio::run(stream);
/// # }
/// ```
#[cfg(feature = "timer")]
fn timeout(self, timeout: Duration) -> Timeout<Self>
where
Self: Sized,
{
Timeout::new(self, timeout)
}
}
impl<T: ?Sized> StreamExt for T where T: Stream {}
+10 -8
View File
@@ -3,26 +3,28 @@ extern crate futures;
extern crate tokio;
extern crate tokio_io;
use std::io::{BufReader, BufWriter, Read, Write};
use std::net::TcpStream;
use std::thread;
use std::io::{Read, Write, BufReader, BufWriter};
use futures::Future;
use futures::stream::Stream;
use tokio_io::io::copy;
use futures::Future;
use tokio::net::TcpListener;
use tokio_io::io::copy;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
($e:expr) => {
match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
}
};
}
#[test]
fn echo_server() {
const N: usize = 1024;
drop(env_logger::init());
drop(env_logger::try_init());
let srv = t!(TcpListener::bind(&t!("127.0.0.1:0".parse())));
let addr = t!(srv.local_addr());
+64
View File
@@ -0,0 +1,64 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_timer;
use tokio::prelude::*;
use tokio::runtime::{self, current_thread};
use tokio::timer::*;
use tokio_timer::clock::Clock;
use std::sync::mpsc;
use std::time::{Duration, Instant};
struct MockNow(Instant);
impl tokio_timer::clock::Now for MockNow {
fn now(&self) -> Instant {
self.0
}
}
#[test]
fn clock_and_timer_concurrent() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = runtime::Builder::new().clock(clock).build().unwrap();
let (tx, rx) = mpsc::channel();
rt.spawn({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn clock_and_timer_single_threaded() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(5_000);
let clock = Clock::new_with_now(MockNow(when));
let mut rt = current_thread::Builder::new().clock(clock).build().unwrap();
rt.block_on({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() < when);
Ok(())
})
})
.unwrap();
}
-395
View File
@@ -1,395 +0,0 @@
extern crate tokio;
extern crate tokio_executor;
extern crate futures;
use tokio::executor::current_thread::{self, block_on_all, CurrentThread};
use std::any::Any;
use std::cell::{Cell, RefCell};
use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::prelude::*;
use futures::sync::oneshot;
#[test]
fn spawn_from_block_on_all() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let msg = current_thread::block_on_all(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok::<_, ()>("hello")
})).unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Rc::new(Cell::new(0));
let cnt2 = cnt.clone();
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
assert_eq!(1, cnt2.get());
}
#[test]
fn spawn_many() {
const ITER: usize = 200;
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
for _ in 0..ITER {
let cnt = cnt.clone();
current_thread.spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok::<(), ()>(())
}));
}
current_thread.run().unwrap();
assert_eq!(cnt.get(), ITER);
}
#[test]
fn does_not_set_global_executor_by_default() {
use tokio_executor::Executor;
block_on_all(lazy(|| {
tokio_executor::DefaultExecutor::current()
.spawn(Box::new(lazy(|| ok())))
.unwrap_err();
ok()
})).unwrap();
}
#[test]
fn spawn_from_block_on_future() {
let cnt = Rc::new(Cell::new(0));
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
current_thread::spawn(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
}));
Ok::<_, ()>(())
})).unwrap();
current_thread.run().unwrap();
assert_eq!(1, cnt.get());
}
struct Never(Rc<()>);
impl Future for Never {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(Async::NotReady)
}
}
#[test]
fn outstanding_tasks_are_dropped_when_executor_is_dropped() {
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.spawn(Never(rc.clone()));
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
// Using the global spawn fn
let mut rc = Rc::new(());
let mut current_thread = CurrentThread::new();
current_thread.block_on(lazy(|| {
current_thread::spawn(Never(rc.clone()));
Ok::<_, ()>(())
})).unwrap();
drop(current_thread);
// Ensure the daemon is dropped
assert!(Rc::get_mut(&mut rc).is_some());
}
#[test]
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
})).unwrap();
}
#[test]
#[should_panic]
fn run_in_future() {
block_on_all(lazy(|| {
current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
ok()
}));
ok()
})).unwrap();
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
struct Infini {
num: Rc<Cell<usize>>,
}
impl Future for Infini {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.num.set(1 + self.num.get());
task::current().notify();
Ok(Async::NotReady)
}
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.turn(None)
.unwrap();
assert_eq!(1, num.get());
}
#[test]
fn tasks_are_scheduled_fairly() {
let state = Rc::new(RefCell::new([0, 0]));
struct Spin {
state: Rc<RefCell<[i32; 2]>>,
idx: usize,
}
impl Future for Spin {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
let mut state = self.state.borrow_mut();
if self.idx == 0 {
let diff = state[0] - state[1];
assert!(diff.abs() <= 1);
if state[0] >= 50 {
return Ok(().into());
}
}
state[self.idx] += 1;
if state[self.idx] >= 100 {
return Ok(().into());
}
task::current().notify();
Ok(Async::NotReady)
}
}
block_on_all(lazy(|| {
current_thread::spawn(Spin {
state: state.clone(),
idx: 0,
});
current_thread::spawn(Spin {
state: state,
idx: 1,
});
ok()
})).unwrap();
}
#[test]
fn spawn_and_turn() {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
let mut current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
current_thread.spawn(lazy(move || {
Ok(())
}));
// Turn once...
current_thread.turn(None).unwrap();
current_thread.spawn(lazy(move || {
c.set(1 + c.get());
// Spawn!
current_thread::spawn(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
}));
Ok(())
}));
// This does not run the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(1, cnt.get());
// This runs the newly spawned thread
current_thread.turn(None).unwrap();
assert_eq!(2, cnt.get());
}
#[test]
fn spawn_in_drop() {
let mut current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
current_thread.spawn({
struct OnDrop<F: FnOnce()>(Option<F>);
impl<F: FnOnce()> Drop for OnDrop<F> {
fn drop(&mut self) {
(self.0.take().unwrap())();
}
}
struct MyFuture {
_data: Box<Any>,
}
impl Future for MyFuture {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
Ok(().into())
}
}
MyFuture {
_data: Box::new(OnDrop(Some(move || {
current_thread::spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}));
}))),
}
});
current_thread.block_on(rx).unwrap();
current_thread.run().unwrap();
}
#[test]
fn hammer_turn() {
use futures::sync::mpsc;
const ITER: usize = 100;
const N: usize = 100;
const THREADS: usize = 4;
for _ in 0..ITER {
let mut ths = vec![];
// Add some jitter
for _ in 0..THREADS {
let th = thread::spawn(|| {
let mut current_thread = CurrentThread::new();
let (tx, rx) = mpsc::unbounded();
current_thread.spawn({
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
rx.for_each(move |_| {
c.set(1 + c.get());
Ok(())
})
.map_err(|e| panic!("err={:?}", e))
.map(move |v| {
assert_eq!(N, cnt.get());
v
})
});
thread::spawn(move || {
for _ in 0..N {
tx.unbounded_send(()).unwrap();
thread::yield_now();
}
});
while !current_thread.is_idle() {
current_thread.turn(None).unwrap();
}
});
ths.push(th);
}
for th in ths {
th.join().unwrap();
}
}
}
fn ok() -> future::FutureResult<(), ()> {
future::ok(())
}
+2 -2
View File
@@ -1,8 +1,8 @@
extern crate tokio;
extern crate futures;
extern crate tokio;
use std::thread;
use std::net;
use std::thread;
use futures::future;
use futures::prelude::*;
+26
View File
@@ -0,0 +1,26 @@
extern crate futures;
extern crate tokio;
extern crate tokio_executor;
extern crate tokio_timer;
use futures::sync::mpsc;
use tokio::util::StreamExt;
#[test]
fn enumerate() {
use futures::*;
let (mut tx, rx) = mpsc::channel(1);
std::thread::spawn(|| {
for i in 0..5 {
tx = tx.send(i * 2).wait().unwrap();
}
});
let result = rx.enumerate().collect();
assert_eq!(
result.wait(),
Ok(vec![(0, 0), (1, 2), (2, 4), (3, 6), (4, 8)])
);
}
+76 -55
View File
@@ -1,40 +1,48 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use std::{io, thread};
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::Relaxed;
use std::sync::Arc;
use std::{io, thread};
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio::net::{TcpListener, TcpStream};
use tokio::runtime::Runtime;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
($e:expr) => {
match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
}
};
}
#[test]
fn hammer() {
let _ = env_logger::init();
fn hammer_old() {
let _ = env_logger::try_init();
let threads = (0..10).map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv.incoming().into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
let threads = (0..10)
.map(|_| {
thread::spawn(|| {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mine = TcpStream::connect(&addr);
let theirs = srv
.incoming()
.into_future()
.map(|(s, _)| s.unwrap())
.map_err(|(s, _)| s);
let (mine, theirs) = t!(mine.join(theirs).wait());
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
assert_eq!(t!(mine.local_addr()), t!(theirs.peer_addr()));
assert_eq!(t!(theirs.local_addr()), t!(mine.peer_addr()));
})
})
}).collect::<Vec<_>>();
.collect::<Vec<_>>();
for thread in threads {
thread.join().unwrap();
}
@@ -49,8 +57,7 @@ impl io::Read for Rd {
}
}
impl tokio_io::AsyncRead for Rd {
}
impl tokio_io::AsyncRead for Rd {}
impl io::Write for Wr {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
@@ -73,48 +80,62 @@ fn hammer_split() {
use tokio_io::io;
const N: usize = 100;
const ITER: usize = 10;
let _ = env_logger::init();
let _ = env_logger::try_init();
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
for _ in 0..ITER {
let srv = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(srv.local_addr());
let mut rt = Runtime::new().unwrap();
let cnt = Arc::new(AtomicUsize::new(0));
fn split(socket: TcpStream) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let mut rt = Runtime::new().unwrap();
let rd = io::read(rd, vec![0; 1])
.map(|_| ())
.map_err(|e| panic!("read error = {:?}", e));
fn split(socket: TcpStream, cnt: Arc<AtomicUsize>) {
let socket = Arc::new(socket);
let rd = Rd(socket.clone());
let wr = Wr(socket);
let wr = io::write_all(wr, b"1")
.map(|_| ())
.map_err(|e| panic!("write error = {:?}", e));
let cnt2 = cnt.clone();
tokio::spawn(rd);
tokio::spawn(wr);
}
let rd = io::read(rd, vec![0; 1])
.map(move |_| {
cnt2.fetch_add(1, Relaxed);
})
.map_err(|e| panic!("read error = {:?}", e));
rt.spawn({
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(|socket| {
split(socket);
Ok(())
})
});
let wr = io::write_all(wr, b"1")
.map(move |_| {
cnt.fetch_add(1, Relaxed);
})
.map_err(move |e| panic!("write error = {:?}", e));
tokio::spawn(rd);
tokio::spawn(wr);
}
for _ in 0..N {
rt.spawn({
TcpStream::connect(&addr)
.map_err(|e| panic!("connect error = {:?}", e))
.map(|socket| split(socket))
let cnt = cnt.clone();
srv.incoming()
.map_err(|e| panic!("accept error = {:?}", e))
.take(N as u64)
.for_each(move |socket| {
split(socket, cnt.clone());
Ok(())
})
});
}
rt.shutdown_on_idle().wait().unwrap();
for _ in 0..N {
rt.spawn({
let cnt = cnt.clone();
TcpStream::connect(&addr)
.map_err(move |e| panic!("connect error = {:?}", e))
.map(move |socket| split(socket, cnt))
});
}
rt.shutdown_on_idle().wait().unwrap();
assert_eq!(N * 4, cnt.load(Relaxed));
}
}
+627
View File
@@ -0,0 +1,627 @@
extern crate bytes;
extern crate futures;
extern crate tokio;
use tokio::codec::*;
use tokio::io::{AsyncRead, AsyncWrite};
use bytes::{BufMut, Bytes, BytesMut};
use futures::Async::*;
use futures::{Poll, Sink, Stream};
use std::collections::VecDeque;
use std::io;
macro_rules! mock {
($($x:expr,)*) => {{
let mut v = VecDeque::new();
v.extend(vec![$($x),*]);
Mock { calls: v }
}};
}
#[test]
fn read_empty_io_yields_nothing() {
let mut io = FramedRead::new(mock!(), LengthDelimitedCodec::new());
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_read(mock! {
Ok(b"\x09\x00\x00\x00abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_one_packet_native_endian() {
let data = if cfg!(target_endian = "big") {
b"\x00\x00\x00\x09abcdefghi"
} else {
b"\x09\x00\x00\x00abcdefghi"
};
let mut io = length_delimited::Builder::new()
.native_endian()
.new_read(mock! {
Ok(data[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x00\x00\x09abcdefghi");
data.extend_from_slice(b"\x00\x00\x00\x03123");
data.extend_from_slice(b"\x00\x00\x00\x0bhello world");
let mut io = FramedRead::new(
mock! {
Ok(data.into()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00"[..].into()),
Ok(b"\x00\x09abc"[..].into()),
Ok(b"defghi"[..].into()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_frame_multi_packet_wait() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_multi_frame_multi_packet_wait() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09abc"[..].into()),
Err(would_block()),
Ok(b"defghi"[..].into()),
Err(would_block()),
Ok(b"\x00\x00\x00\x0312"[..].into()),
Err(would_block()),
Ok(b"3\x00\x00\x00\x0bhello world"[..].into()),
Err(would_block()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_incomplete_head() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00"[..].into()),
},
LengthDelimitedCodec::new(),
);
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_head_multi() {
let mut io = FramedRead::new(
mock! {
Err(would_block()),
Ok(b"\x00"[..].into()),
Err(would_block()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_incomplete_payload() {
let mut io = FramedRead::new(
mock! {
Ok(b"\x00\x00\x00\x09ab"[..].into()),
Err(would_block()),
Ok(b"cd"[..].into()),
Err(would_block()),
},
LengthDelimitedCodec::new(),
);
assert_eq!(io.poll().unwrap(), NotReady);
assert_eq!(io.poll().unwrap(), NotReady);
assert!(io.poll().is_err());
}
#[test]
fn read_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new().new_read(mock! {
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new().new_read(mock! {
Ok(b"\x00\x00\x00\x09abcd"[..].into()),
Err(would_block()),
Ok(b"efghi"[..].into()),
Ok(b"\x00\x00\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), NotReady);
io.decoder_mut().set_max_frame_length(5);
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap_err().kind(), io::ErrorKind::InvalidData);
}
#[test]
fn read_one_byte_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_read(mock! {
Ok(b"\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_header_offset() {
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(4)
.new_read(mock! {
Ok(b"zzzz\x00\x09abcdefghi"[..].into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_skip_none_adjusted() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"xx\x00\x09abcdefghi");
data.extend_from_slice(b"yy\x00\x03123");
data.extend_from_slice(b"zz\x00\x0bhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_field_offset(2)
.num_skip(0)
.length_adjustment(4)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(
io.poll().unwrap(),
Ready(Some(b"xx\x00\x09abcdefghi"[..].into()))
);
assert_eq!(io.poll().unwrap(), Ready(Some(b"yy\x00\x03123"[..].into())));
assert_eq!(
io.poll().unwrap(),
Ready(Some(b"zz\x00\x0bhello world"[..].into()))
);
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn read_single_multi_frame_one_packet_length_includes_head() {
let mut data: Vec<u8> = vec![];
data.extend_from_slice(b"\x00\x0babcdefghi");
data.extend_from_slice(b"\x00\x05123");
data.extend_from_slice(b"\x00\x0dhello world");
let mut io = length_delimited::Builder::new()
.length_field_length(2)
.length_adjustment(-2)
.new_read(mock! {
Ok(data.into()),
});
assert_eq!(io.poll().unwrap(), Ready(Some(b"abcdefghi"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"123"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(Some(b"hello world"[..].into())));
assert_eq!(io.poll().unwrap(), Ready(None));
}
#[test]
fn write_single_frame_length_adjusted() {
let mut io = length_delimited::Builder::new()
.length_adjustment(-2)
.new_write(mock! {
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_nothing_yields_nothing() {
let mut io = FramedWrite::new(mock!(), LengthDelimitedCodec::new());
assert!(io.poll_complete().unwrap().is_ready());
}
#[test]
fn write_single_frame_one_packet() {
let mut io = FramedWrite::new(
mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
},
LengthDelimitedCodec::new(),
);
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_one_packet() {
let mut io = FramedWrite::new(
mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
},
LengthDelimitedCodec::new(),
);
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io
.start_send(Bytes::from("hello world"))
.unwrap()
.is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_multi_frame_multi_packet() {
let mut io = FramedWrite::new(
mock! {
Ok(b"\x00\x00\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x03"[..].into()),
Ok(b"123"[..].into()),
Ok(Flush),
Ok(b"\x00\x00\x00\x0b"[..].into()),
Ok(b"hello world"[..].into()),
Ok(Flush),
},
LengthDelimitedCodec::new(),
);
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.start_send(Bytes::from("123")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io
.start_send(Bytes::from("hello world"))
.unwrap()
.is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_would_block() {
let mut io = FramedWrite::new(
mock! {
Err(would_block()),
Ok(b"\x00\x00"[..].into()),
Err(would_block()),
Ok(b"\x00\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
},
LengthDelimitedCodec::new(),
);
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_little_endian() {
let mut io = length_delimited::Builder::new()
.little_endian()
.new_write(mock! {
Ok(b"\x09\x00\x00\x00"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_single_frame_with_short_length_field() {
let mut io = length_delimited::Builder::new()
.length_field_length(1)
.new_write(mock! {
Ok(b"\x09"[..].into()),
Ok(b"abcdefghi"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdefghi")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_max_frame_len() {
let mut io = length_delimited::Builder::new()
.max_frame_length(5)
.new_write(mock! {});
assert_eq!(
io.start_send(Bytes::from("abcdef")).unwrap_err().kind(),
io::ErrorKind::InvalidInput
);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_at_rest() {
let mut io = length_delimited::Builder::new().new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"abcdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert_eq!(
io.start_send(Bytes::from("abcdef")).unwrap_err().kind(),
io::ErrorKind::InvalidInput
);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_update_max_frame_len_in_flight() {
let mut io = length_delimited::Builder::new().new_write(mock! {
Ok(b"\x00\x00\x00\x06"[..].into()),
Ok(b"ab"[..].into()),
Err(would_block()),
Ok(b"cdef"[..].into()),
Ok(Flush),
});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert!(!io.poll_complete().unwrap().is_ready());
io.encoder_mut().set_max_frame_length(5);
assert!(io.poll_complete().unwrap().is_ready());
assert_eq!(
io.start_send(Bytes::from("abcdef")).unwrap_err().kind(),
io::ErrorKind::InvalidInput
);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn write_zero() {
let mut io = length_delimited::Builder::new().new_write(mock! {});
assert!(io.start_send(Bytes::from("abcdef")).unwrap().is_ready());
assert_eq!(
io.poll_complete().unwrap_err().kind(),
io::ErrorKind::WriteZero
);
assert!(io.get_ref().calls.is_empty());
}
#[test]
fn encode_overflow() {
// Test reproducing tokio-rs/tokio#681.
let mut codec = length_delimited::Builder::new().new_codec();
let mut buf = BytesMut::with_capacity(1024);
// Put some data into the buffer without resizing it to hold more.
let some_as = std::iter::repeat(b'a').take(1024).collect::<Vec<_>>();
buf.put_slice(&some_as[..]);
// Trying to encode the length header should resize the buffer if it won't fit.
codec.encode(Bytes::from("hello"), &mut buf).unwrap();
}
// ===== Test utils =====
fn would_block() -> io::Error {
io::Error::new(io::ErrorKind::WouldBlock, "would block")
}
struct Mock {
calls: VecDeque<io::Result<Op>>,
}
enum Op {
Data(Vec<u8>),
Flush,
}
use self::Op::*;
impl io::Read for Mock {
fn read(&mut self, dst: &mut [u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
debug_assert!(dst.len() >= data.len());
dst[..data.len()].copy_from_slice(&data[..]);
Ok(data.len())
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
}
impl AsyncRead for Mock {}
impl io::Write for Mock {
fn write(&mut self, src: &[u8]) -> io::Result<usize> {
match self.calls.pop_front() {
Some(Ok(Op::Data(data))) => {
let len = data.len();
assert!(src.len() >= len, "expect={:?}; actual={:?}", data, src);
assert_eq!(&data[..], &src[..len]);
Ok(len)
}
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(0),
}
}
fn flush(&mut self) -> io::Result<()> {
match self.calls.pop_front() {
Some(Ok(Op::Flush)) => Ok(()),
Some(Ok(_)) => panic!(),
Some(Err(e)) => Err(e),
None => Ok(()),
}
}
}
impl AsyncWrite for Mock {
fn shutdown(&mut self) -> Poll<(), io::Error> {
Ok(Ready(()))
}
}
impl<'a> From<&'a [u8]> for Op {
fn from(src: &'a [u8]) -> Op {
Op::Data(src.into())
}
}
impl From<Vec<u8>> for Op {
fn from(src: Vec<u8>) -> Op {
Op::Data(src)
}
}
+18 -15
View File
@@ -1,21 +1,20 @@
extern crate bytes;
extern crate env_logger;
extern crate futures;
extern crate futures_cpupool;
extern crate tokio;
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate tokio_threadpool;
use std::io;
use std::net::Shutdown;
use bytes::{BytesMut, BufMut};
use futures::{Future, Stream, Sink};
use futures::future::Executor;
use futures_cpupool::CpuPool;
use bytes::{BufMut, BytesMut};
use futures::{Future, Sink, Stream};
use tokio::net::{TcpListener, TcpStream};
use tokio_io::codec::{Encoder, Decoder};
use tokio_io::io::{write_all, read};
use tokio_io::AsyncRead;
use tokio_codec::{Decoder, Encoder};
use tokio_io::io::{read, write_all};
use tokio_threadpool::Builder;
pub struct LineCodec;
@@ -52,20 +51,24 @@ impl Encoder for LineCodec {
#[test]
fn echo() {
drop(env_logger::init());
drop(env_logger::try_init());
let pool = CpuPool::new(1);
let pool = Builder::new().pool_size(1).build();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
let addr = listener.local_addr().unwrap();
let pool_inner = pool.clone();
let sender = pool.sender().clone();
let srv = listener.incoming().for_each(move |socket| {
let (sink, stream) = socket.framed(LineCodec).split();
pool_inner.execute(sink.send_all(stream).map(|_| ()).map_err(|_| ())).unwrap();
let (sink, stream) = LineCodec.framed(socket).split();
sender
.spawn(sink.send_all(stream).map(|_| ()).map_err(|_| ()))
.unwrap();
Ok(())
});
pool.execute(srv.map_err(|e| panic!("srv error: {}", e))).unwrap();
pool.sender()
.spawn(srv.map_err(|e| panic!("srv error: {}", e)))
.unwrap();
let client = TcpStream::connect(&addr);
let client = client.wait().unwrap();
+28 -13
View File
@@ -13,18 +13,20 @@ use std::os::unix::io::{AsRawFd, FromRawFd};
use std::thread;
use std::time::Duration;
use futures::Future;
use mio::event::Evented;
use mio::unix::{UnixReady, EventedFd};
use mio::unix::{EventedFd, UnixReady};
use mio::{PollOpt, Ready, Token};
use tokio::reactor::{Handle, PollEvented2};
use tokio_io::io::read_to_end;
use futures::Future;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
($e:expr) => {
match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
}
};
}
struct MyFile(File);
@@ -46,13 +48,23 @@ impl io::Read for MyFile {
}
impl Evented for MyFile {
fn register(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
fn register(
&self,
poll: &mio::Poll,
token: Token,
interest: Ready,
opts: PollOpt,
) -> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).register(poll, token, interest | hup, opts)
}
fn reregister(&self, poll: &mio::Poll, token: Token, interest: Ready, opts: PollOpt)
-> io::Result<()> {
fn reregister(
&self,
poll: &mio::Poll,
token: Token,
interest: Ready,
opts: PollOpt,
) -> io::Result<()> {
let hup: Ready = UnixReady::hup().into();
EventedFd(&self.0.as_raw_fd()).reregister(poll, token, interest | hup, opts)
}
@@ -63,13 +75,16 @@ impl Evented for MyFile {
#[test]
fn hup() {
drop(env_logger::init());
drop(env_logger::try_init());
let handle = Handle::default();
unsafe {
let mut pipes = [0; 2];
assert!(libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}", io::Error::last_os_error());
assert!(
libc::pipe(pipes.as_mut_ptr()) != -1,
"pipe error: {}",
io::Error::last_os_error()
);
let read = File::from_raw_fd(pipes[0]);
let mut write = File::from_raw_fd(pipes[1]);
let t = thread::spawn(move || {
+91
View File
@@ -0,0 +1,91 @@
extern crate futures;
extern crate tokio_executor;
extern crate tokio_reactor;
extern crate tokio_tcp;
use tokio_reactor::Reactor;
use tokio_tcp::TcpListener;
use futures::executor::{spawn, Notify, Spawn};
use futures::{Future, Stream};
use std::mem;
use std::net::TcpStream;
use std::sync::{Arc, Mutex};
#[test]
fn test_drop_on_notify() {
// When the reactor receives a kernel notification, it notifies the
// task that holds the associated socket. If this notification results in
// the task being dropped, the socket will also be dropped.
//
// Previously, there was a deadlock scenario where the reactor, while
// notifying, held a lock and the task being dropped attempted to acquire
// that same lock in order to clean up state.
//
// To simulate this case, we create a fake executor that does nothing when
// the task is notified. This simulates an executor in the process of
// shutting down. Then, when the task handle is dropped, the task itself is
// dropped.
struct MyNotify;
type Task = Mutex<Spawn<Box<Future<Item = (), Error = ()>>>>;
impl Notify for MyNotify {
fn notify(&self, _: usize) {
// Do nothing
}
fn clone_id(&self, id: usize) -> usize {
let ptr = id as *const Task;
let task = unsafe { Arc::from_raw(ptr) };
mem::forget(task.clone());
mem::forget(task);
id
}
fn drop_id(&self, id: usize) {
let ptr = id as *const Task;
let _ = unsafe { Arc::from_raw(ptr) };
}
}
let addr = "127.0.0.1:0".parse().unwrap();
let mut reactor = Reactor::new().unwrap();
// Create a listener
let listener = TcpListener::bind(&addr).unwrap();
let addr = listener.local_addr().unwrap();
// Define a task that just drains the listener
let task = Box::new({
listener
.incoming()
.for_each(|_| Ok(()))
.map_err(|_| panic!())
}) as Box<Future<Item = (), Error = ()>>;
let task = Arc::new(Mutex::new(spawn(task)));
let notify = Arc::new(MyNotify);
let mut enter = tokio_executor::enter().unwrap();
tokio_reactor::with_default(&reactor.handle(), &mut enter, |_| {
let id = &*task as *const Task as usize;
task.lock()
.unwrap()
.poll_future_notify(&notify, id)
.unwrap();
});
drop(task);
// Establish a connection to the acceptor
let _s = TcpStream::connect(&addr).unwrap();
reactor.turn(None).unwrap();
}
+522 -42
View File
@@ -1,52 +1,532 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
extern crate env_logger;
use futures::prelude::*;
use tokio::net::{TcpStream, TcpListener};
use tokio_io::io;
use futures::sync::oneshot;
use std::sync::{atomic, Arc, Mutex};
use std::thread;
use tokio::io;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::future::lazy;
use tokio::prelude::*;
use tokio::runtime::Runtime;
// this import is used in all child modules that have it in scope
// from importing super::*, but the compiler doesn't realise that
// and warns about it.
pub use futures::future::Executor;
macro_rules! t {
($e:expr) => (match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
})
($e:expr) => {
match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {:?}", stringify!($e), e),
}
};
}
fn create_client_server_future() -> Box<Future<Item = (), Error = ()> + Send> {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server
.incoming()
.take(1)
.map_err(|e| panic!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| ())
.map_err(|e| panic!("write err = {:?}", e))
})
})
.map(|_| ());
let client = client
.map_err(|e| panic!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| ())
.map_err(|e| panic!("read err = {:?}", e))
});
let future = server.join(client).map(|_| ());
Box::new(future)
}
#[test]
fn basic_runtime_usage() {
let _ = env_logger::init();
fn runtime_tokio_run() {
let _ = env_logger::try_init();
// TODO: Don't require the lazy wrapper
tokio::run(::futures::future::lazy(|| {
let server = t!(TcpListener::bind(&"127.0.0.1:0".parse().unwrap()));
let addr = t!(server.local_addr());
let client = TcpStream::connect(&addr);
let server = server.incoming().take(1)
.map_err(|e| println!("accept err = {:?}", e))
.for_each(|socket| {
tokio::spawn({
io::write_all(socket, b"hello")
.map(|_| println!("write done"))
.map_err(|e| println!("write err = {:?}", e))
})
})
.map(|_| println!("accept done"));
let client = client
.map_err(|e| println!("connect err = {:?}", e))
.and_then(|client| {
// Read all
io::read_to_end(client, vec![])
.map(|_| println!("read done"))
.map_err(|e| println!("read err = {:?}", e))
});
tokio::spawn({
server.join(client)
.map(|_| println!("done"))
})
}));
tokio::run(create_client_server_future());
}
#[test]
fn runtime_single_threaded() {
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::current_thread::Runtime::new().unwrap();
runtime.block_on(create_client_server_future()).unwrap();
runtime.run().unwrap();
}
#[test]
fn runtime_single_threaded_block_on() {
let _ = env_logger::try_init();
tokio::runtime::current_thread::block_on_all(create_client_server_future()).unwrap();
}
mod runtime_single_threaded_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item = (), Error = ()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let msg = tokio::runtime::current_thread::block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn spawn() {
test(|f| {
tokio::spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
}
mod runtime_single_threaded_racy {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(tokio::runtime::current_thread::Handle, Box<Future<Item = (), Error = ()> + Send>),
{
let (trigger, exit) = futures::sync::oneshot::channel();
let (handle_tx, handle_rx) = ::std::sync::mpsc::channel();
let jh = ::std::thread::spawn(move || {
let mut rt = tokio::runtime::current_thread::Runtime::new().unwrap();
handle_tx.send(rt.handle()).unwrap();
// don't exit until we are told to
rt.block_on(exit.map_err(|_| ())).unwrap();
// run until all spawned futures (incl. the "exit" signal future) have completed.
rt.run().unwrap();
});
let (tx, rx) = futures::sync::oneshot::channel();
let handle = handle_rx.recv().unwrap();
spawn(
handle,
Box::new(futures::future::lazy(move || {
tx.send(()).unwrap();
Ok(())
})),
);
// signal runtime thread to exit
trigger.send(()).unwrap();
// wait for runtime thread to exit
jh.join().unwrap();
assert_eq!(rx.wait().unwrap(), ());
}
#[test]
fn spawn() {
test(|handle, f| {
handle.spawn(f).unwrap();
})
}
#[test]
fn execute() {
test(|handle, f| {
handle.execute(f).unwrap();
})
}
}
mod runtime_multi_threaded {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime) + Send + 'static,
{
let _ = env_logger::try_init();
let mut runtime = tokio::runtime::Builder::new().build().unwrap();
spawn(&mut runtime);
runtime.shutdown_on_idle().wait().unwrap();
}
#[test]
fn spawn() {
test(|rt| {
rt.spawn(create_client_server_future());
});
}
#[test]
fn execute() {
test(|rt| {
rt.executor()
.execute(create_client_server_future())
.unwrap();
});
}
}
#[test]
fn block_on_timer() {
use std::time::{Duration, Instant};
use tokio::timer::{Delay, Error};
fn after_1s<T>(x: T) -> Box<Future<Item = T, Error = Error> + Send>
where
T: Send + 'static,
{
Box::new(Delay::new(Instant::now() + Duration::from_millis(100)).map(move |_| x))
}
let mut runtime = Runtime::new().unwrap();
assert_eq!(runtime.block_on(after_1s(42)).unwrap(), 42);
runtime.shutdown_on_idle().wait().unwrap();
}
mod from_block_on {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item = (), Error = ()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
let msg = runtime
.block_on(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| {
tokio::spawn(f);
})
}
}
#[test]
fn block_waits() {
let (tx, rx) = oneshot::channel();
thread::spawn(|| {
use std::time::Duration;
thread::sleep(Duration::from_millis(1000));
tx.send(()).unwrap();
});
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let mut runtime = Runtime::new().unwrap();
runtime
.block_on(rx.then(move |_| {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<_, ()>(())
}))
.unwrap();
assert_eq!(1, *cnt.lock().unwrap());
runtime.shutdown_on_idle().wait().unwrap();
}
mod many {
use super::*;
const ITER: usize = 200;
fn test<F>(spawn: F)
where
F: Fn(&mut Runtime, Box<Future<Item = (), Error = ()> + Send>),
{
let cnt = Arc::new(Mutex::new(0));
let mut runtime = Runtime::new().unwrap();
for _ in 0..ITER {
let c = cnt.clone();
spawn(
&mut runtime,
Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})),
);
}
runtime.shutdown_on_idle().wait().unwrap();
assert_eq!(ITER, *cnt.lock().unwrap());
}
#[test]
fn spawn() {
test(|rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|rt, f| {
rt.executor().execute(f).unwrap();
})
}
}
mod from_block_on_all {
use super::*;
fn test<F>(spawn: F)
where
F: Fn(Box<Future<Item = (), Error = ()> + Send>) + Send + 'static,
{
let cnt = Arc::new(Mutex::new(0));
let c = cnt.clone();
let runtime = Runtime::new().unwrap();
let msg = runtime
.block_on_all(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
// Spawn!
spawn(Box::new(lazy(move || {
{
let mut x = c.lock().unwrap();
*x = 1 + *x;
}
Ok::<(), ()>(())
})));
Ok::<_, ()>("hello")
}))
.unwrap();
assert_eq!(2, *cnt.lock().unwrap());
assert_eq!(msg, "hello");
}
#[test]
fn execute() {
test(|f| {
tokio::executor::DefaultExecutor::current()
.execute(f)
.unwrap();
})
}
#[test]
fn spawn() {
test(|f| {
tokio::spawn(f);
})
}
}
mod nested_enter {
use super::*;
use std::panic;
use tokio::runtime::current_thread;
fn test<F1, F2>(first: F1, nested: F2)
where
F1: Fn(Box<Future<Item = (), Error = ()> + Send>) + Send + 'static,
F2: Fn(Box<Future<Item = (), Error = ()> + Send>) + panic::UnwindSafe + Send + 'static,
{
let panicked = Arc::new(Mutex::new(false));
let panicked2 = panicked.clone();
// Since this is testing panics in other threads, printing about panics
// is noisy and can give the impression that the test is ignoring panics.
//
// It *is* ignoring them, but on purpose.
let prev_hook = panic::take_hook();
panic::set_hook(Box::new(|info| {
let s = info.to_string();
if s.starts_with("panicked at 'nested ")
|| s.starts_with("panicked at 'Multiple executors at once")
{
// expected, noop
} else {
println!("{}", s);
}
}));
first(Box::new(lazy(move || {
panic::catch_unwind(move || nested(Box::new(lazy(|| Ok::<(), ()>(())))))
.expect_err("nested should panic");
*panicked2.lock().unwrap() = true;
Ok::<(), ()>(())
})));
panic::set_hook(prev_hook);
assert!(
*panicked.lock().unwrap(),
"nested call should have panicked"
);
}
fn threadpool_new() -> Runtime {
Runtime::new().expect("rt new")
}
#[test]
fn run_in_run() {
test(tokio::run, tokio::run);
}
#[test]
fn threadpool_block_on_in_run() {
test(tokio::run, |fut| {
let mut rt = threadpool_new();
rt.block_on(fut).unwrap();
});
}
#[test]
fn threadpool_block_on_all_in_run() {
test(tokio::run, |fut| {
let rt = threadpool_new();
rt.block_on_all(fut).unwrap();
});
}
#[test]
fn current_thread_block_on_all_in_run() {
test(tokio::run, |fut| {
current_thread::block_on_all(fut).unwrap();
});
}
}
#[test]
fn runtime_reactor_handle() {
#![allow(deprecated)]
use futures::Stream;
use std::net::{TcpListener as StdListener, TcpStream as StdStream};
let rt = Runtime::new().unwrap();
let std_listener = StdListener::bind("127.0.0.1:0").unwrap();
let tk_listener = TcpListener::from_std(std_listener, rt.handle()).unwrap();
let addr = tk_listener.local_addr().unwrap();
// Spawn a thread since we are avoiding the runtime
let th = thread::spawn(|| for _ in tk_listener.incoming().take(1).wait() {});
let _ = StdStream::connect(&addr).unwrap();
th.join().unwrap();
}
#[test]
fn after_start_and_before_stop_is_called() {
let _ = env_logger::try_init();
let after_start = Arc::new(atomic::AtomicUsize::new(0));
let before_stop = Arc::new(atomic::AtomicUsize::new(0));
let after_inner = after_start.clone();
let before_inner = before_stop.clone();
let runtime = tokio::runtime::Builder::new()
.after_start(move || {
after_inner.clone().fetch_add(1, atomic::Ordering::Relaxed);
})
.before_stop(move || {
before_inner.clone().fetch_add(1, atomic::Ordering::Relaxed);
})
.build()
.unwrap();
runtime.block_on_all(create_client_server_future()).unwrap();
assert!(after_start.load(atomic::Ordering::Relaxed) > 0);
assert!(before_stop.load(atomic::Ordering::Relaxed) > 0);
}
+113
View File
@@ -0,0 +1,113 @@
extern crate env_logger;
extern crate futures;
extern crate tokio;
extern crate tokio_io;
use tokio::prelude::*;
use tokio::timer::*;
use std::sync::mpsc;
use std::time::{Duration, Instant};
#[test]
fn timer_with_runtime() {
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(100);
let (tx, rx) = mpsc::channel();
tokio::run({
Delay::new(when)
.map_err(|e| panic!("unexpected error; err={:?}", e))
.and_then(move |_| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn starving() {
use futures::{task, Async, Poll};
let _ = env_logger::try_init();
struct Starve(Delay, u64);
impl Future for Starve {
type Item = u64;
type Error = ();
fn poll(&mut self) -> Poll<Self::Item, ()> {
if self.0.poll().unwrap().is_ready() {
return Ok(self.1.into());
}
self.1 += 1;
task::current().notify();
Ok(Async::NotReady)
}
}
let when = Instant::now() + Duration::from_millis(20);
let starve = Starve(Delay::new(when), 0);
let (tx, rx) = mpsc::channel();
tokio::run({
starve.and_then(move |_ticks| {
assert!(Instant::now() >= when);
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn deadline() {
use futures::future;
let _ = env_logger::try_init();
let when = Instant::now() + Duration::from_millis(20);
let (tx, rx) = mpsc::channel();
#[allow(deprecated)]
tokio::run({
future::empty::<(), ()>().deadline(when).then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
#[test]
fn timeout() {
use futures::future;
let _ = env_logger::try_init();
let (tx, rx) = mpsc::channel();
tokio::run({
future::empty::<(), ()>()
.timeout(Duration::from_millis(20))
.then(move |res| {
assert!(res.is_err());
tx.send(()).unwrap();
Ok(())
})
});
rx.recv().unwrap();
}
+29
View File
@@ -0,0 +1,29 @@
[package]
name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.6"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.6"
description = """
Experimental async/await support for Tokio
"""
categories = ["asynchronous"]
[features]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
hyper = "0.12.8"
+52
View File
@@ -0,0 +1,52 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
Copyright (c) 2016 futures-rs authors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
+55
View File
@@ -0,0 +1,55 @@
# Tokio async/await preview
This crate provides a preview of Tokio with async / await support. It is a shim
layer on top of `tokio`.
**This crate requires Rust nightly and does not provide API stability
guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.34.0-nightly or later.
Add this to your `Cargo.toml`:
```toml
# In the `[packages]` section
edition = "2018"
# In the `[dependencies]` section
tokio = {version = "0.1.15", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
```rust
// The nightly features that are commonly needed with async / await
#![feature(await_macro, async_await, futures_api)]
// This pulls in the `tokio-async-await` crate. While Rust 2018 doesn't require
// `extern crate`, we need to pull in the macros.
#[macro_use]
extern crate tokio;
fn main() {
// And we are async...
tokio::run_async(async {
println!("Hello");
});
}
```
Because nightly is required, run the app with `cargo +nightly run`
Check the [examples](examples) directory for more.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
+2
View File
@@ -0,0 +1,2 @@
[build]
target-dir = "../../target"
+49
View File
@@ -0,0 +1,49 @@
[package]
name = "examples"
edition = "2018"
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
# Break out of the parent workspace
[workspace]
[[bin]]
name = "chat"
path = "src/chat.rs"
[[bin]]
name = "echo_client"
path = "src/echo_client.rs"
[[bin]]
name = "echo_server"
path = "src/echo_server.rs"
[[bin]]
name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.0", path = "../..", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../.." }
tokio-async-await = { path = "../" }
tokio-codec = { path = "../../tokio-codec" }
tokio-current-thread = { path = "../../tokio-current-thread" }
tokio-executor = { path = "../../tokio-executor" }
tokio-fs = { path = "../../tokio-fs" }
tokio-io = { path = "../../tokio-io" }
tokio-reactor = { path = "../../tokio-reactor" }
tokio-signal = { path = "../../tokio-signal" }
tokio-tcp = { path = "../../tokio-tcp" }
tokio-threadpool = { path = "../../tokio-threadpool" }
tokio-timer = { path = "../../tokio-timer" }
tokio-tls = { path = "../../tokio-tls" }
tokio-udp = { path = "../../tokio-udp" }
tokio-uds = { path = "../../tokio-uds" }
+5
View File
@@ -0,0 +1,5 @@
# Tokio async/await examples
These are a separate crate in order to work around some cargo bugs. It also
allows `[patch]` to be used in `Cargo.toml` to ensure the correct lib versions
are being pulled in.
+135
View File
@@ -0,0 +1,135 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate futures; // v0.1
use tokio::codec::{LinesCodec, Decoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use futures::sync::mpsc;
use std::collections::HashMap;
use std::io;
use std::net::SocketAddr;
use std::sync::{Arc, Mutex};
/// Shorthand for the transmit half of the message channel.
type Tx = mpsc::UnboundedSender<String>;
struct Shared {
peers: HashMap<SocketAddr, Tx>,
}
impl Shared {
/// Create a new, empty, instance of `Shared`.
fn new() -> Self {
Shared {
peers: HashMap::new(),
}
}
}
async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()> {
let addr = stream.peer_addr().unwrap();
let mut lines = LinesCodec::new().framed(stream);
// Extract the peer's name
let name = match await!(lines.next()) {
Some(name) => name?,
None => {
// Disconnected early
return Ok(());
}
};
println!("`{}` is joining the chat", name);
let (tx, mut rx) = mpsc::unbounded();
// Register the socket
state.lock().unwrap()
.peers.insert(addr, tx);
// Split the `lines` handle into send and recv handles. This allows spawning
// separate tasks.
let (mut lines_tx, mut lines_rx) = lines.split();
// Spawn a task that receives all lines broadcasted to us from other peers
// and writes it to the client.
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line)).unwrap();
}
});
// Use the current task to read lines from the socket and broadcast them to
// other peers.
while let Some(message) = await!(lines_rx.next()) {
// TODO: Error handling
let message = message.unwrap();
let mut line = name.clone();
line.push_str(": ");
line.push_str(&message);
line.push_str("\r\n");
let state = state.lock().unwrap();
for (peer_addr, tx) in &state.peers {
if *peer_addr != addr {
// TODO: Error handling
tx.unbounded_send(line.clone()).unwrap();
}
}
}
// Remove the client from the shared state. Doing so will also result in the
// tx task to terminate.
state.lock().unwrap()
.peers.remove(&addr)
.expect("bug");
Ok(())
}
fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
// `state` handle is cloned and passed into the task that processes the
// client connection.
let state = Arc::new(Mutex::new(Shared::new()));
let addr = "127.0.0.1:6142".parse().unwrap();
// Bind a TCP listener to the socket address.
//
// Note that this is the Tokio TcpListener, which is fully async.
let listener = TcpListener::bind(&addr).unwrap();
println!("server running on localhost:6142");
// Start the Tokio runtime.
tokio::run_async(async move {
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = match stream {
Ok(stream) => stream,
Err(_) => continue,
};
let state = state.clone();
tokio::spawn_async(async move {
if let Err(_) = await!(process(stream, state)) {
eprintln!("failed to process connection");
}
});
}
});
}
@@ -0,0 +1,53 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
use tokio::net::TcpStream;
use tokio::prelude::*;
use std::io;
use std::net::SocketAddr;
const MESSAGES: &[&str] = &[
"hello",
"world",
"one two three",
];
async fn run_client(addr: &SocketAddr) -> io::Result<()> {
let mut stream = await!(TcpStream::connect(addr))?;
// Buffer to read into
let mut buf = [0; 128];
for msg in MESSAGES {
println!(" > write = {:?}", msg);
// Write the message to the server
await!(stream.write_all_async(msg.as_bytes()))?;
// Read the message back from the server
await!(stream.read_exact_async(&mut buf[..msg.len()]))?;
assert_eq!(&buf[..msg.len()], msg.as_bytes());
}
Ok(())
}
fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Connect to the echo serveer
tokio::run_async(async move {
match await!(run_client(&addr)) {
Ok(_) => println!("done."),
Err(e) => eprintln!("echo client failed; error = {:?}", e),
}
});
}
@@ -0,0 +1,45 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
use std::net::SocketAddr;
fn handle(mut stream: TcpStream) {
tokio::spawn_async(async move {
let mut buf = [0; 1024];
loop {
match await!(stream.read_async(&mut buf)).unwrap() {
0 => break, // Socket closed
n => {
// Send the data back
await!(stream.write_all_async(&buf[0..n])).unwrap();
}
}
}
});
}
fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
let addr = addr.parse::<SocketAddr>().unwrap();
// Bind the TCP listener
let listener = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
tokio::run_async(async {
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = stream.unwrap();
handle(stream);
}
});
}
+33
View File
@@ -0,0 +1,33 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate hyper;
use tokio::prelude::*;
use hyper::Client;
use std::time::Duration;
use std::str;
pub fn main() {
tokio::run_async(async {
let client = Client::new();
let uri = "http://httpbin.org/ip".parse().unwrap();
let response = await!({
client.get(uri)
.timeout(Duration::from_secs(10))
}).unwrap();
println!("Response: {}", response.status());
let mut body = response.into_body();
while let Some(chunk) = await!(body.next()) {
let chunk = chunk.unwrap();
println!("chunk = {}", str::from_utf8(&chunk[..]).unwrap());
}
});
}
+16
View File
@@ -0,0 +1,16 @@
/// Wait for a future to complete.
#[macro_export]
macro_rules! await {
($e:expr) => {{
#[allow(unused_imports)]
use $crate::compat::backward::IntoAwaitable as IntoAwaitableBackward;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
use $crate::std_await;
#[allow(unused_mut)]
let mut e = $e;
let e = e.into_awaitable();
std_await!(e)
}};
}
+83
View File
@@ -0,0 +1,83 @@
use futures::{Future, Poll};
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::ptr;
use std::task::{Poll as StdPoll, RawWaker, RawWakerVTable, Waker};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
pub struct Compat<T>(Pin<Box<T>>);
impl<T> Compat<T> {
/// Create a new `Compat` backed by `future`.
pub fn new(future: T) -> Compat<T> {
Compat(Box::pin(future))
}
}
/// Convert a value into one that can be used with `await!`.
pub trait IntoAwaitable {
type Awaitable;
fn into_awaitable(self) -> Self::Awaitable;
}
impl<T> IntoAwaitable for T
where
T: StdFuture,
{
type Awaitable = Self;
fn into_awaitable(self) -> Self {
self
}
}
impl<T, Item, Error> Future for Compat<T>
where
T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let waker = noop_waker();
let res = self.0.as_mut().poll(&waker);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
StdPoll::Ready(Err(err)) => Err(err),
StdPoll::Pending => Ok(NotReady),
}
}
}
// ===== NoopWaker =====
fn noop_raw_waker() -> RawWaker {
RawWaker::new(ptr::null(), &NOOP_WAKER_VTABLE)
}
fn noop_waker() -> Waker {
unsafe { Waker::new_unchecked(noop_raw_waker()) }
}
unsafe fn clone_raw(_data: *const ()) -> RawWaker {
noop_raw_waker()
}
unsafe fn drop_raw(_data: *const ()) {}
unsafe fn wake(_data: *const ()) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
const NOOP_WAKER_VTABLE: RawWakerVTable = RawWakerVTable {
clone: clone_raw,
drop: drop_raw,
wake,
};
+67
View File
@@ -0,0 +1,67 @@
use futures::{Async, Future};
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{Poll as StdPoll, Waker};
/// Converts an 0.1 `Future` into an 0.3 `Future`.
#[derive(Debug)]
pub struct Compat<T>(T);
pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T, E>> {
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
Ok(NotReady) => StdPoll::Pending,
Err(err) => StdPoll::Ready(Err(err)),
}
}
pub(crate) fn convert_poll_stream<T, E>(
poll: Result<Async<Option<T>>, E>,
) -> StdPoll<Option<Result<T, E>>> {
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(Some(val))) => StdPoll::Ready(Some(Ok(val))),
Ok(Ready(None)) => StdPoll::Ready(None),
Ok(NotReady) => StdPoll::Pending,
Err(err) => StdPoll::Ready(Some(Err(err))),
}
}
/// Convert a value into one that can be used with `await!`.
pub trait IntoAwaitable {
type Awaitable;
/// Convert `self` into a value that can be used with `await!`.
fn into_awaitable(self) -> Self::Awaitable;
}
impl<T: Future + Unpin> IntoAwaitable for T {
type Awaitable = Compat<T>;
fn into_awaitable(self) -> Self::Awaitable {
Compat(self)
}
}
impl<T> StdFuture for Compat<T>
where
T: Future + Unpin,
{
type Output = Result<T::Item, T::Error>;
fn poll(mut self: Pin<&mut Self>, _waker: &Waker) -> StdPoll<Self::Output> {
use futures::Async::{NotReady, Ready};
// TODO: wire in cx
match self.0.poll() {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
Ok(NotReady) => StdPoll::Pending,
Err(e) => StdPoll::Ready(Err(e)),
}
}
}
+4
View File
@@ -0,0 +1,4 @@
#![doc(hidden)]
pub mod backward;
pub mod forward;
+30
View File
@@ -0,0 +1,30 @@
use tokio_io::AsyncWrite;
use std::future::Future;
use std::io;
use std::pin::Pin;
use std::task::{Poll, Waker};
/// A future used to fully flush an I/O object.
#[derive(Debug)]
pub struct Flush<'a, T: ?Sized + 'a> {
writer: &'a mut T,
}
// Pin is never projected to fields
impl<'a, T: ?Sized> Unpin for Flush<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Flush<'a, T> {
pub(super) fn new(writer: &'a mut T) -> Flush<'a, T> {
Flush { writer }
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Flush<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _wx: &Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
convert_poll(self.writer.poll_flush())
}
}
+192
View File
@@ -0,0 +1,192 @@
//! Use I/O with `async` / `await`.
mod flush;
mod read;
mod read_exact;
mod write;
mod write_all;
pub use self::flush::Flush;
pub use self::read::Read;
pub use self::read_exact::ReadExact;
pub use self::write::Write;
pub use self::write_all::WriteAll;
use tokio_io::{AsyncRead, AsyncWrite};
/// An extension trait which adds utility methods to `AsyncRead` types.
pub trait AsyncReadExt: AsyncRead {
/// Tries to read some bytes directly into the given `buf` in an
/// asynchronous manner, returning a future.
///
/// The returned future will resolve to the number of bytes read once the read
/// operation is completed.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::Cursor;
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 5];
///
/// let bytes = await!(reader.read_async(&mut output[..])).unwrap();
///
/// // This is only guaranteed to be 4 because `&[u8]` is a synchronous
/// // reader. In a real system you could get anywhere from 1 to
/// // `output.len()` bytes in a single read.
/// assert_eq!(bytes, 4);
/// assert_eq!(output, [1, 2, 3, 4, 0]);
/// });
/// ```
fn read_async<'a>(&'a mut self, buf: &'a mut [u8]) -> Read<'a, Self> {
Read::new(self, buf)
}
/// Creates a future which will read exactly enough bytes to fill `buf`,
/// returning an error if end of file (EOF) is hit sooner.
///
/// The returned future will resolve once the read operation is completed.
///
/// In the case of an error the buffer and the object will be discarded, with
/// the error yielded.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::Cursor;
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 4];
///
/// await!(reader.read_exact_async(&mut output)).unwrap();
///
/// assert_eq!(output, [1, 2, 3, 4]);
/// });
/// ```
///
/// ## EOF is hit before `buf` is filled
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncReadExt;
/// use std::io::{self, Cursor};
///
/// let mut reader = Cursor::new([1, 2, 3, 4]);
/// let mut output = [0u8; 5];
///
/// let result = await!(reader.read_exact_async(&mut output));
///
/// assert_eq!(result.unwrap_err().kind(), io::ErrorKind::UnexpectedEof);
/// });
/// ```
fn read_exact_async<'a>(&'a mut self, buf: &'a mut [u8]) -> ReadExact<'a, Self> {
ReadExact::new(self, buf)
}
}
/// An extension trait which adds utility methods to `AsyncWrite` types.
pub trait AsyncWriteExt: AsyncWrite {
/// Write data into this object.
///
/// Creates a future that will write the entire contents of the buffer `buf` into
/// this `AsyncWrite`.
///
/// The returned future will not complete until all the data has been written.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::Cursor;
///
/// let mut buf = [0u8; 5];
/// let mut writer = Cursor::new(&mut buf[..]);
///
/// let n = await!(writer.write_async(&[1, 2, 3, 4])).unwrap();
///
/// assert_eq!(writer.into_inner()[..n], [1, 2, 3, 4, 0][..n]);
/// });
/// ```
fn write_async<'a>(&'a mut self, buf: &'a [u8]) -> Write<'a, Self> {
Write::new(self, buf)
}
/// Write an entire buffer into this object.
///
/// Creates a future that will write the entire contents of the buffer `buf` into
/// this `AsyncWrite`.
///
/// The returned future will not complete until all the data has been written.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::Cursor;
///
/// let mut buf = [0u8; 5];
/// let mut writer = Cursor::new(&mut buf[..]);
///
/// await!(writer.write_all_async(&[1, 2, 3, 4])).unwrap();
///
/// assert_eq!(writer.into_inner(), [1, 2, 3, 4, 0]);
/// });
/// ```
fn write_all_async<'a>(&'a mut self, buf: &'a [u8]) -> WriteAll<'a, Self> {
WriteAll::new(self, buf)
}
/// Creates a future which will entirely flush this `AsyncWrite`.
///
/// # Examples
///
/// ```
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::AsyncWriteExt;
/// use std::io::{BufWriter, Cursor};
///
/// let mut output = [0u8; 5];
///
/// {
/// let mut writer = Cursor::new(&mut output[..]);
/// let mut buffered = BufWriter::new(writer);
/// await!(buffered.write_all_async(&[1, 2])).unwrap();
/// await!(buffered.write_all_async(&[3, 4])).unwrap();
/// await!(buffered.flush_async()).unwrap();
/// }
///
/// assert_eq!(output, [1, 2, 3, 4, 0]);
/// });
/// ```
fn flush_async<'a>(&mut self) -> Flush<Self> {
Flush::new(self)
}
}
impl<T: AsyncRead + ?Sized> AsyncReadExt for T {}
impl<T: AsyncWrite + ?Sized> AsyncWriteExt for T {}
+34
View File
@@ -0,0 +1,34 @@
use tokio_io::AsyncRead;
use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::pin::Pin;
/// A future which can be used to read bytes.
#[derive(Debug)]
pub struct Read<'a, T: ?Sized + 'a> {
reader: &'a mut T,
buf: &'a mut [u8],
}
// Pinning is never projected to fields
impl<'a, T: ?Sized> Unpin for Read<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> Read<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> Read<'a, T> {
Read { reader, buf }
}
}
impl<'a, T: AsyncRead + ?Sized> Future for Read<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _waker: &task::Waker) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
convert_poll(this.reader.poll_read(this.buf))
}
}

Some files were not shown because too many files have changed in this diff Show More