Compare commits

...
Author SHA1 Message Date
Carl Lerche 475dabe96d Release tokio v0.1.20, tokio-timer v0.2.21, and remove async-await-preview feature. (#1089)
The `async-await-preview` feature is removed as 0.1 will no longer track
Rust nightly.

This also bumps:
- tokio-timer (0.2.11).
2019-05-14 11:21:24 -07:00
Carl Lerche 951f2fd910 test: re-export macro dependencies (#1077)
Callers may not always have `futures` available at the root of the
crate. Re-exporting dependencies makes them available to the macro at a
deterministic location.
2019-05-03 20:43:40 -07:00
Carl Lerche 4ef736b9d5 async-await: add current_thread::Runtime::block_on_async (#1072)
This function is used by the Tokio macros introduced by #1058  but was
omitted from the PR.
2019-04-30 19:55:22 -07:00
Steven Fackler 219f24cbf1 timer: Replace Handle::deadline with Handle::timeout (#1074)
Deadline was deprecated a while ago and replaced with Timeout, but the
methods on Handle got missed.

Fixes #1071
2019-04-30 10:29:54 -07:00
Carl Lerche ea282efb2e ci: fix isRelease condition (#1066) 2019-04-29 10:37:00 -07:00
Michal 'vorner' Vaner 042224d33c signal: Smaller dependency (#1069)
The signal-hook library got split into lower-level and higher-level
parts. The tokio-signal uses only API from the lower-level one, so it
can depend on it directly.

The only effect of this change is smaller amount of compiled (and
unused) code during compilation. There's no change in the code actually
used.
2019-04-28 19:12:40 -07:00
Ian Hamlin 927eb80ad4 Fix an error in the mit-url in the README.md (#1068) 2019-04-27 12:56:40 -07:00
Carl Lerche 6a8934e897 Fix threadpool dependency (#1061) 2019-04-25 22:23:24 -04:00
Carl Lerche 0e400af78c Async/await polish (#1058)
A general refresh of Tokio's experimental async / await support.
2019-04-25 22:22:32 -04:00
Carl Lerche df702130d6 tcp: fix some tests that spuriously fail (#1060)
This does not remove all cases of using a fixed port in doc tests, but
removing some should reduce the likelihood of spurious failures.
2019-04-25 12:01:39 -07:00
Carl Lerche 949adbb887 chore: remember to remove path deps on release (#1057) 2019-04-24 10:42:39 -07:00
Ryan Dahl b2b796a228 rt: forward panic_handler to tokio::runtime::Builder (#1055) 2019-04-24 10:41:42 -07:00
Igor Gnatenko abb014efc2 tokio: Bump min version of tokio-sync (#1054)
It is needed for lock functionality which tokio now uses.
2019-04-24 08:24:59 -07:00
Lucio Franco e5cf0cc717 Introduce tokio-test crate (#1030) 2019-04-23 20:17:57 -07:00
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
Carl Lerche 11e2af66a8 Bump Tokio to v0.1.15. (#869)
Also bumps:

- tokio-sync (0.1.0)
- tokio-threadpool (0.1.11)
- tokio-timer (0.2.9)
2019-01-25 10:20:09 -08:00
Carl Lerche a4aae1459c chore: move enumerate test to correct location (#867) 2019-01-24 20:47:45 -08:00
Zahari Dichev 12546d1d9c tokio-timer: fix DelayQueue bug when inserting shorter delay (#863)
Reset the delay of the queue in case an item that expires sooner than the last inserted is put
into the queue.
2019-01-24 14:36:41 -08:00
Zahari Dichev fbad6297c5 Add enumerate combinator to Stream (#832) 2019-01-24 11:50:34 -08:00
Jon Gjengset 0ec8986b0b Make reason for try_send errors clearer (#864) 2019-01-23 15:06:24 -08:00
Jon Gjengset c6f8bdb249 Remove T: Debug bound on mpsc Debug impls (#866)
Following from https://github.com/tokio-rs/tokio/pull/865, this PR
removes `#[derive(Debug)]` on `mpsc` sender and receiver types in favor
of explicit `impl fmt::Debug` blocks that don't have a `T: fmt::Debug`
bound.
2019-01-23 18:04:00 -05:00
Jon Gjengset c6f9a069a5 Explicit impl Clone for tx to avoid T: Clone (#865)
`#[derive(Clone)]` on a type `struct Foo<T>` adds an impl that requires that
`T: Clone`:

```rust
impl<T: Clone> Clone for Foo<T>
```

which is unfortunate in the case of senders, because we don't want to require
that the items being sent are `Clone` for the channel sender to be `Clone`.
This PR adds an explicit `impl Clone` for the bounded and unbounded sender
types which does not have the `T: Clone` bound.

Note that this is _also_ an issue with `#[derive(Debug)]`, but that one is
harder to work around as `chan::Tx` _also_ has `#[derive(Debug)]`, as does
`chan::Chan`, so we'd have to add explicit impls for all of them to make
progress.
2019-01-23 15:51:44 -05:00
Sean McArthur 9f356d6244 tokio-sync: add into_inner for TrySendErrors (#862) 2019-01-22 14:48:22 -08:00
Carl Lerche 13083153aa Introduce tokio-sync crate containing synchronization primitives. (#839)
Introduce a tokio-sync crate containing useful synchronization primitives for programs
written using Tokio.

The initial release contains:

* An mpsc channel
* A oneshot channel
* A semaphore implementation
* An `AtomicTask` primitive.

The `oneshot` and `mpsc` channels are new implementations providing improved
performance characteristics. In some benchmarks, the new mpsc channel shows
up to 7x improvement over the version provided by the `futures` crate. Unfortunately,
the `oneshot` implementation only provides a slight performance improvement as it
is mostly limited by the `futures` 0.1 task system. Once updated to the `std` version
of `Future` (currently nightly only), much greater performance improvements should
be achievable by `oneshot`.

Additionally, he implementations provided here are checked using
[Loom](http://github.com/carllerche/loom/), which provides greater confidence of
correctness.
2019-01-22 11:37:26 -08:00
rmcteggart-r7 91f20e33a4 docs: deal with Result instead of using unwrap (#860) 2019-01-20 14:21:17 -05:00
Eliza Weisman 983e9d1b67 timer: Fix DelayQueue delay reset logic (#851) 2019-01-20 08:38:39 -05:00
Stjepan Glavina 4c8f274db9 threadpool: drop incomplete tasks on shutdown (#722)
## Motivation

When the thread pool shuts down, futures that have been polled at least once but not completed yet are simply leaked. We should drop them instead.

## Solution

Multiple changes are introduced:

* Tasks are assigned a home worker the first time they are polled.

* Each worker contains a set of tasks (`Arc<Task>`) it is home to. When a task is assigned a home worker, it is registered in that worker's set of tasks. When the task is completed, it is unregistered from the set.

* When the thread pool shuts down and after all worker threads stop, the remaining tasks in workers' sets are aborted, i.e. they are switched to the `Aborted` state and their `Future`s are dropped.

* The thread pool shutdown process is refactored to make it more robust. We don't  track the number of active threads manually anymore. Instead, there's  `Arc<ShutdownTrigger>` that aborts remaining tasks and completes the `Shutdown` future once it gets destroyed (when all `Worker`s and `ThreadPool` get dropped because they're the only ones to contain strong references to the `ShutdownTrigger`).

Closes #424 
Closes #428
2019-01-17 22:12:25 +01:00
Marek Kotewicz c980837581 docs: missing links in tokio-timer::delay_queue (#845) 2019-01-13 21:20:08 +01:00
Marek Kotewicz eec370cae8 docs: fixed links in tokio-timer (#844)
* docs: fixed links in tokio-timer/src/timer/mod.rs

* docs: fixed links in tokio-timer::clock
2019-01-12 10:06:55 -08:00
Marek Kotewicz 733d432b80 docs: fixed links to tokio_timer::clock::Now (#842)
* docs: fixed links to tokio_timer::clock::Now in tokio-timer/src/timer/mod.rs

* docs: fixed links to std::time::Instant in tokio-timer/src/timer/mod.rs
2019-01-10 23:49:13 +01:00
Carl Lerche 74c473d68f travis: allow nightly Rust CI to fail (#843) 2019-01-10 11:27:58 -08:00
Sean McArthur d95c697781 tokio: update tokio-threadpool minimum version (#838) 2019-01-07 16:49:08 -08:00
Carl Lerche 25e835c5b7 tcp: specify version for tokio dev dependency
This is required for publishing to crates.io
2019-01-06 23:31:24 -08:00
454 changed files with 24337 additions and 7341 deletions
-21
View File
@@ -1,21 +0,0 @@
image: Visual Studio 2017
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+43
View File
@@ -0,0 +1,43 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_DURATION: 10
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
cargo_cache:
folder: $HOME/.cargo/registry
test_script:
- . $HOME/.cargo/env
- cargo test --all
- (cd tokio-trace/test-log-support && cargo test)
- (cd tokio-trace/test_static_max_level_features && cargo test)
- cargo doc --all
i686_test_script:
- . $HOME/.cargo/env
- |
cargo test --all --exclude tokio-tls --exclude tokio-macros --target i686-unknown-freebsd
before_cache_script:
- rm -rf $HOME/.cargo/registry/index
-127
View File
@@ -1,127 +0,0 @@
---
language: rust
sudo: false
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
- rust: stable
- rust: beta
- rust: nightly
env: ALLOW_FAILURES=true
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
#
# Tests are not run as tests may require newer versions of rust.
- rust: 1.26.0
script: |
cargo check --all
# Test combinations of enabled features.
- rust: stable
script: |
shopt -s expand_aliases
alias check="cargo check --no-default-features"
check
check --features codec
check --features fs
check --features io
check --features reactor
check --features rt-full
check --features tcp
check --features timer
check --features udp
check --features uds
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
# This runs TSAN against nightly and allows failures to propagate up.
- rust: nightly-2018-11-18
env: TSAN=yes
script: |
set -e
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# This runs cargo +nightly doc
- name: nightly_docs
rust: nightly
script: cargo doc
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script: |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
cargo test --all --no-fail-fast
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = "linux" && $TARGET = ""
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+7 -122
View File
@@ -1,140 +1,25 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Update doc URL.
# - Create "v0.1.x" git tag.
version = "0.1.14"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.14/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio-async-await",
"tokio",
"tokio-buf",
"tokio-channel",
"tokio-codec",
"tokio-current-thread",
"tokio-executor",
"tokio-fs",
"tokio-futures",
"tokio-io",
"tokio-macros",
"tokio-reactor",
"tokio-signal",
"tokio-sync",
"tokio-test",
"tokio-threadpool",
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-trace",
"tokio-trace/tokio-trace-core",
"tokio-udp",
"tokio-uds",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.3", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-threadpool = { version = "0.1.4", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
#tokio = { path = "." }
#tokio-async-await = { path = "./tokio-async-await" }
#tokio-codec = { path = "./tokio-codec" }
#tokio-current-thread = { path = "./tokio-current-thread" }
#tokio-executor = { path = "./tokio-executor" }
#tokio-fs = { path = "./tokio-fs" }
#tokio-io = { path = "./tokio-io" }
#tokio-reactor = { path = "./tokio-reactor" }
#tokio-signal = { path = "./tokio-signal" }
#tokio-tcp = { path = "./tokio-tcp" }
#tokio-threadpool = { path = "./tokio-threadpool" }
#tokio-timer = { path = "./tokio-timer" }
#tokio-tls = { path = "./tokio-tls" }
#tokio-udp = { path = "./tokio-udp" }
#tokio-uds = { path = "./tokio-uds" }
+18 -18
View File
@@ -14,29 +14,26 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[mit-url]: LICENSE
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio/0.1.14/tokio) |
[API Docs](https://docs.rs/tokio/0.1.20/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[master-dox]: https://tokio-rs.github.io/tokio/doc/tokio/
## Overview
@@ -52,9 +49,9 @@ level, it provides a few major components:
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/0.1.20/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.20/tokio/reactor/index.html
[scheduler]: https://docs.rs/tokio/0.1.20/tokio/runtime/index.html
## Example
@@ -101,7 +98,7 @@ fn main() {
}
```
More examples can be found [here](examples).
More examples can be found [here](tokio/examples).
## Getting Help
@@ -129,10 +126,6 @@ have greater guarantees of stability.
The crates included as part of Tokio are:
* [`tokio-async-await`]: Experimental `async` / `await` support.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-current-thread`]: Schedule the execution of futures on the current
thread.
@@ -140,8 +133,14 @@ The crates included as part of Tokio are:
* [`tokio-fs`]: Filesystem (and standard in / out) APIs.
* [`tokio-futures`]: Experimental `std::future::Future` and `async` / `await` support.
* [`tokio-codec`]: Utilities for encoding and decoding protocol frames.
* [`tokio-io`]: Asynchronous I/O related traits and utilities.
* [`tokio-macros`]: Macros for usage with Tokio.
* [`tokio-reactor`]: Event loop that drives I/O resources (like TCP and UDP
sockets).
@@ -157,12 +156,13 @@ The crates included as part of Tokio are:
* [`tokio-uds`]: Unix Domain Socket bindings for use with `tokio-io` and
`tokio-reactor`.
[`tokio-async-await`]: tokio-async-await
[`tokio-codec`]: tokio-codec
[`tokio-current-thread`]: tokio-current-thread
[`tokio-executor`]: tokio-executor
[`tokio-fs`]: tokio-fs
[`tokio-futures`]: tokio-futures
[`tokio-io`]: tokio-io
[`tokio-macros`]: tokio-macros
[`tokio-reactor`]: tokio-reactor
[`tokio-tcp`]: tokio-tcp
[`tokio-threadpool`]: tokio-threadpool
+2
View File
@@ -0,0 +1,2 @@
[build]
target-dir = "../target"
+49
View File
@@ -0,0 +1,49 @@
[package]
name = "examples"
edition = "2018"
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
# Break out of the parent workspace
[workspace]
[[bin]]
name = "chat"
path = "src/chat.rs"
[[bin]]
name = "echo_client"
path = "src/echo_client.rs"
[[bin]]
name = "echo_server"
path = "src/echo_server.rs"
[[bin]]
name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.18", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../tokio" }
tokio-codec = { path = "../tokio-codec" }
tokio-current-thread = { path = "../tokio-current-thread" }
tokio-executor = { path = "../tokio-executor" }
tokio-fs = { path = "../tokio-fs" }
tokio-futures = { path = "../tokio-futures" }
tokio-io = { path = "../tokio-io" }
tokio-reactor = { path = "../tokio-reactor" }
tokio-signal = { path = "../tokio-signal" }
tokio-tcp = { path = "../tokio-tcp" }
tokio-threadpool = { path = "../tokio-threadpool" }
tokio-timer = { path = "../tokio-timer" }
tokio-tls = { path = "../tokio-tls" }
tokio-udp = { path = "../tokio-udp" }
tokio-uds = { path = "../tokio-uds" }
@@ -1,9 +1,6 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate futures; // v0.1
#![feature(await_macro, async_await)]
use tokio::await;
use tokio::codec::{LinesCodec, Decoder};
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
@@ -61,7 +58,7 @@ async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()>
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
await!(lines_tx.send_async(line)).unwrap();
}
});
@@ -95,7 +92,8 @@ async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()>
Ok(())
}
fn main() {
#[tokio::main]
async fn main() {
// Create the shared state. This is how all the peers communicate.
//
// The server task will hold a handle to this. For every new client, the
@@ -113,23 +111,21 @@ fn main() {
println!("server running on localhost:6142");
// Start the Tokio runtime.
tokio::run_async(async move {
let mut incoming = listener.incoming();
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = match stream {
Ok(stream) => stream,
Err(_) => continue,
};
while let Some(stream) = await!(incoming.next()) {
let stream = match stream {
Ok(stream) => stream,
Err(_) => continue,
};
let state = state.clone();
let state = state.clone();
tokio::spawn_async(async move {
if let Err(_) = await!(process(stream, state)) {
eprintln!("failed to process connection");
}
});
}
});
tokio::spawn_async(async move {
if let Err(_) = await!(process(stream, state)) {
eprintln!("failed to process connection");
}
});
}
}
@@ -1,8 +1,6 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
#![feature(await_macro, async_await)]
use tokio::await;
use tokio::net::TcpStream;
use tokio::prelude::*;
@@ -36,7 +34,8 @@ async fn run_client(addr: &SocketAddr) -> io::Result<()> {
Ok(())
}
fn main() {
#[tokio::main]
async fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
@@ -44,10 +43,8 @@ fn main() {
// Connect to the echo serveer
tokio::run_async(async move {
match await!(run_client(&addr)) {
Ok(_) => println!("done."),
Err(e) => eprintln!("echo client failed; error = {:?}", e),
}
});
match await!(run_client(&addr)) {
Ok(_) => println!("done."),
Err(e) => eprintln!("echo client failed; error = {:?}", e),
}
}
@@ -1,8 +1,6 @@
#![feature(await_macro, async_await)]
#[macro_use]
extern crate tokio;
use tokio::await;
use tokio::net::{TcpListener, TcpStream};
use tokio::prelude::*;
@@ -24,7 +22,8 @@ fn handle(mut stream: TcpStream) {
});
}
fn main() {
#[tokio::main]
async fn main() {
use std::env;
let addr = env::args().nth(1).unwrap_or("127.0.0.1:8080".to_string());
@@ -34,12 +33,10 @@ fn main() {
let listener = TcpListener::bind(&addr).unwrap();
println!("Listening on: {}", addr);
tokio::run_async(async {
let mut incoming = listener.incoming();
let mut incoming = listener.incoming();
while let Some(stream) = await!(incoming.next()) {
let stream = stream.unwrap();
handle(stream);
}
});
while let Some(stream) = await!(incoming.next()) {
let stream = stream.unwrap();
handle(stream);
}
}
+29
View File
@@ -0,0 +1,29 @@
#![feature(await_macro, async_await)]
use tokio::await;
use tokio::prelude::*;
use hyper::Client;
use std::time::Duration;
use std::str;
#[tokio::main]
async fn main() {
let client = Client::new();
let uri = "http://httpbin.org/ip".parse().unwrap();
let response = await!({
client.get(uri)
.timeout(Duration::from_secs(10))
}).unwrap();
println!("Response: {}", response.status());
let mut body = response.into_body();
while let Some(chunk) = await!(body.next()) {
let chunk = chunk.unwrap();
println!("chunk = {}", str::from_utf8(&chunk[..]).unwrap());
}
}
+22
View File
@@ -0,0 +1,22 @@
#![feature(await_macro, async_await)]
use tokio::await;
use tokio::timer::Delay;
use std::time::{Duration, Instant};
#[tokio::test]
async fn success_no_async() {
assert!(true);
}
#[tokio::test]
#[should_panic]
async fn fail_no_async() {
assert!(false);
}
#[tokio::test]
async fn use_timer() {
let when = Instant::now() + Duration::from_millis(10);
await!(Delay::new(when));
}
+114
View File
@@ -0,0 +1,114 @@
trigger: ["master", "v0.1.x"]
pr: ["master", "v0.1.x"]
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
name: rustfmt
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
displayName: Test tokio
cross: true
crates:
- tokio
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates -
cross: true
crates:
- tokio-fs
- tokio-reactor
- tokio-signal
- tokio-tcp
- tokio-tls
- tokio-udp
- tokio-uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
crates:
- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
- tokio-io
- tokio-sync
- tokio-threadpool
- tokio-timer
- tokio-test
- tokio-trace
- tokio-trace/tokio-trace-core
- tokio-trace/test-log-support
- tokio-trace/test_static_max_level_features
- template: ci/azure-cargo-check.yml
parameters:
name: features
displayName: Check feature permtuations
rust: stable
crates:
tokio:
- codec
- fs
- io
- reactor
- rt-full
- tcp
- timer
- udp
- uds
- sync
tokio-buf:
- util
# Run async-await tests
- template: ci/azure-test-nightly.yml
parameters:
name: test_nightly
displayName: Test Async / Await
rust: nightly-2019-04-25
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross_32bit_linux
target: i686-unknown-linux-gnu
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust_version: 1.26.0
- template: ci/azure-tsan.yml
parameters:
name: tsan
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
- rustfmt
- test_tokio
- test_sub_cross
- test_linux
- features
- test_nightly
- cross_32bit_linux
- minrust
- tsan
+1 -3
View File
@@ -10,8 +10,8 @@ use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
@@ -57,7 +57,6 @@ fn udp_echo_latency(b: &mut Bencher) {
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
@@ -67,7 +66,6 @@ fn udp_echo_latency(b: &mut Bencher) {
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
+8 -9
View File
@@ -3,14 +3,13 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
extern crate test;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
use mio::{PollOpt, Ready, Token};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
@@ -22,8 +21,8 @@ fn mio_register_deregister(b: &mut Bencher) {
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
poll.deregister(&sock).unwrap();
});
}
@@ -36,12 +35,12 @@ fn mio_reregister(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.reregister(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
});
poll.deregister(&sock).unwrap();
}
+62 -49
View File
@@ -11,18 +11,18 @@ pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::reactor::Reactor;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::io::{self, Read, Write};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
pub use test::{self, Bencher};
}
mod connect_churn {
use ::prelude::*;
use prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
@@ -36,25 +36,29 @@ mod connect_churn {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
Ok(TcpStream::connect(&addr).and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
let connects_concurrent = connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(connects_concurrent)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
}
@@ -65,8 +69,7 @@ mod connect_churn {
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
@@ -75,47 +78,56 @@ mod connect_churn {
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(shutdown_rx)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
use std::sync::{Arc, Barrier};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
let threads: Vec<_> = (0..n)
.map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()))
.wait()
.unwrap();
})
})
}).collect();
.collect();
barrier.wait();
@@ -141,7 +153,7 @@ mod connect_churn {
}
mod transfer {
use ::prelude::*;
use prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
@@ -200,7 +212,8 @@ mod transfer {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
let server = listener
.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
@@ -210,17 +223,17 @@ mod transfer {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
drain
.map(|_| ())
.map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
.and_then(move |sock| Transfer {
sock: sock,
rem: MB,
chunk: write_size,
})
.map_err(|e| panic!("client err: {:?}", e));
@@ -229,7 +242,7 @@ mod transfer {
}
mod small_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
@@ -238,7 +251,7 @@ mod transfer {
}
mod big_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
+29
View File
@@ -0,0 +1,29 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+24
View File
@@ -0,0 +1,24 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
# Always patch
- template: azure-patch-crates.yml
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+38
View File
@@ -0,0 +1,38 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --all --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+27
View File
@@ -0,0 +1,27 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain %RUSTUP_TOOLCHAIN%
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%B' | grep -qF '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+16
View File
@@ -0,0 +1,16 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
rustup component add rustfmt
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+19
View File
@@ -0,0 +1,19 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo +nightly check --all
# Check benches
- script: cargo check --benches --all
displayName: Check benchmarks
+41
View File
@@ -0,0 +1,41 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
condition: and(succeeded(), ne(variables['isRelease'], 'true'))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }} (PATCHED)
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-threadpool --tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: nightly-2018-11-18
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+22
View File
@@ -0,0 +1,22 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-buf = { path = "tokio-buf" }
tokio-codec = { path = "tokio-codec" }
tokio-current-thread = { path = "tokio-current-thread" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-futures = { path = "tokio-futures" }
tokio-io = { path = "tokio-io" }
tokio-reactor = { path = "tokio-reactor" }
tokio-signal = { path = "tokio-signal" }
tokio-sync = { path = "tokio-sync" }
tokio-threadpool = { path = "tokio-threadpool" }
tokio-timer = { path = "tokio-timer" }
tokio-tcp = { path = "tokio-tcp" }
tokio-tls = { path = "tokio-tls" }
tokio-trace = { path = "tokio-trace" }
tokio-trace-core = { path = "tokio-trace/tokio-trace-core" }
tokio-udp = { path = "tokio-udp" }
tokio-uds = { path = "tokio-uds" }
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
-55
View File
@@ -1,55 +0,0 @@
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
#[cfg(feature = "io")]
pub use tokio_io::{
AsyncRead,
AsyncWrite,
};
pub use util::{
FutureExt,
StreamExt,
};
pub use ::std::io::{
Read,
Write,
};
pub use futures::{
Future,
future,
Stream,
stream,
Sink,
IntoFuture,
Async,
AsyncSink,
Poll,
task,
};
#[cfg(feature = "async-await-preview")]
#[doc(inline)]
pub use tokio_async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt as StreamAsyncExt,
},
};
-2
View File
@@ -1,2 +0,0 @@
[build]
target-dir = "../../target"
-49
View File
@@ -1,49 +0,0 @@
[package]
name = "examples"
edition = "2018"
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
# Break out of the parent workspace
[workspace]
[[bin]]
name = "chat"
path = "src/chat.rs"
[[bin]]
name = "echo_client"
path = "src/echo_client.rs"
[[bin]]
name = "echo_server"
path = "src/echo_server.rs"
[[bin]]
name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.0", path = "../..", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../.." }
tokio-async-await = { path = "../" }
tokio-codec = { path = "../../tokio-codec" }
tokio-current-thread = { path = "../../tokio-current-thread" }
tokio-executor = { path = "../../tokio-executor" }
tokio-fs = { path = "../../tokio-fs" }
tokio-io = { path = "../../tokio-io" }
tokio-reactor = { path = "../../tokio-reactor" }
tokio-signal = { path = "../../tokio-signal" }
tokio-tcp = { path = "../../tokio-tcp" }
tokio-threadpool = { path = "../../tokio-threadpool" }
tokio-timer = { path = "../../tokio-timer" }
tokio-tls = { path = "../../tokio-tls" }
tokio-udp = { path = "../../tokio-udp" }
tokio-uds = { path = "../../tokio-uds" }
-33
View File
@@ -1,33 +0,0 @@
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
extern crate hyper;
use tokio::prelude::*;
use hyper::Client;
use std::time::Duration;
use std::str;
pub fn main() {
tokio::run_async(async {
let client = Client::new();
let uri = "http://httpbin.org/ip".parse().unwrap();
let response = await!({
client.get(uri)
.timeout(Duration::from_secs(10))
}).unwrap();
println!("Response: {}", response.status());
let mut body = response.into_body();
while let Some(chunk) = await!(body.next()) {
let chunk = chunk.unwrap();
println!("chunk = {}", str::from_utf8(&chunk[..]).unwrap());
}
});
}
-89
View File
@@ -1,89 +0,0 @@
use futures::{Future, Poll};
use std::pin::Pin;
use std::future::{
Future as StdFuture,
};
use std::ptr::NonNull;
use std::task::{
LocalWaker,
Poll as StdPoll,
UnsafeWake,
Waker,
};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
pub struct Compat<T>(Pin<Box<T>>);
impl<T> Compat<T> {
/// Create a new `Compat` backed by `future`.
pub fn new(future: T) -> Compat<T> {
Compat(Box::pin(future))
}
}
/// Convert a value into one that can be used with `await!`.
pub trait IntoAwaitable {
type Awaitable;
fn into_awaitable(self) -> Self::Awaitable;
}
impl<T> IntoAwaitable for T
where T: StdFuture,
{
type Awaitable = Self;
fn into_awaitable(self) -> Self {
self
}
}
impl<T, Item, Error> Future for Compat<T>
where T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let local_waker = noop_local_waker();
let res = self.0.as_mut().poll(&local_waker);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
StdPoll::Ready(Err(err)) => Err(err),
StdPoll::Pending => Ok(NotReady),
}
}
}
// ===== NoopWaker =====
struct NoopWaker;
fn noop_local_waker() -> LocalWaker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { LocalWaker::new(w) }
}
fn noop_waker() -> Waker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { Waker::new(w) }
}
unsafe impl UnsafeWake for NoopWaker {
unsafe fn clone_raw(&self) -> Waker {
noop_waker()
}
unsafe fn drop_raw(&self) {
}
unsafe fn wake(&self) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
}
-4
View File
@@ -1,4 +0,0 @@
#![doc(hidden)]
pub mod forward;
pub mod backward;
-114
View File
@@ -1,114 +0,0 @@
#![cfg(feature = "async-await-preview")]
#![feature(
rust_2018_preview,
arbitrary_self_types,
async_await,
await_macro,
futures_api,
)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.5")]
#![deny(missing_docs, missing_debug_implementations)]
#![cfg_attr(test, deny(warnings))]
//! A preview of Tokio w/ `async` / `await` support.
extern crate futures;
extern crate tokio_io;
/// Extracts the successful type of a `Poll<Result<T, E>>`.
///
/// This macro bakes in propagation of `Pending` and `Err` signals by returning early.
macro_rules! try_ready {
($x:expr) => {
match $x {
std::task::Poll::Ready(Ok(x)) => x,
std::task::Poll::Ready(Err(e)) =>
return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending =>
return std::task::Poll::Pending,
}
}
}
#[macro_use]
mod await;
pub mod compat;
pub mod io;
pub mod sink;
pub mod stream;
/*
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_main::prelude::*;
#[doc(inline)]
pub use crate::async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt,
},
};
}
*/
// Rename the `await` macro in `std`. This is used by the redefined
// `await` macro in this crate.
#[doc(hidden)]
pub use std::await as std_await;
/*
use std::future::{Future as StdFuture};
fn run<T: futures::Future<Item = (), Error = ()>>(t: T) {
drop(t);
}
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
run(future);
unimplemented!();
}
*/
/*
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use crate::async_await::compat::backward;
spawn(backward::Compat::new(async || {
let _ = await!(future);
Ok(())
}));
}
*/
+12 -1
View File
@@ -1,3 +1,14 @@
# 0.1.0 (unreleased)
# 0.1.1 (April 22, 2019)
### Added
- Utilities for creating a `BufStream` from iterators and streams (#1011).
- Add `BufStream::into_stream` (#1048).
- Implement `FromBufStream` for `Bytes` (#1009).
- Implement `Error` for `CollectVecError` (#1010).
### Fixed
- Implement `size_hint` for string types (#1012).
# 0.1.0 (February 23, 2019)
* Initial release
+15 -5
View File
@@ -1,22 +1,32 @@
[package]
name = "tokio-buf"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
version = "0.1.1"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-buf/0.1.0"
documentation = "https://docs.rs/tokio-buf/0.1.1/tokio_buf"
description = """
Asynchronous stream of byte buffers
"""
categories = ["asynchronous"]
[dependencies]
bytes = { version = "0.4.10", features = [ "either" ] }
either = "1.5"
bytes = "0.4.10"
either = { version = "1.5", optional = true}
futures = "0.1.23"
[features]
default = ["util"]
util = ["bytes/either", "either"]
[dev-dependencies]
tokio-mock-task = "0.1.1"
+35
View File
@@ -0,0 +1,35 @@
# tokio-buf
Asynchronous stream of byte buffers
[Documenation](https://docs.rs/tokio-buf)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-buf = "0.1.1"
```
Next, add this to your crate:
```rust
extern crate tokio_buf;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-buf) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-32
View File
@@ -1,32 +0,0 @@
//! Error types
pub use super::collect::CollectError;
pub use super::from::CollectVecError;
pub use super::limit::LimitError;
// Being crate-private, we should be able to swap the type out in a
// backwards compatible way.
pub(crate) mod internal {
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
}
-163
View File
@@ -1,163 +0,0 @@
//! Types and utilities for working with `BufStream`.
mod bytes;
mod chain;
mod collect;
pub mod errors;
mod from;
mod limit;
mod size_hint;
mod str;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::limit::Limit;
pub use self::size_hint::SizeHint;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
/// Indicates to the `BufStream` how much data the consumer is currently
/// able to process.
///
/// The consume hint allows the stream to perform certain optimizations that
/// are dependent on the consumer's readiness. For example, the consume hint
/// may be used to request a remote peer to start sending up to `amount`
/// data.
///
/// Calling `consume_hint` is not a requirement. If `consume_hint` is never
/// called, the stream should assume a default behavior. When `consume_hint`
/// is called, the stream should make a best effort to honor by the request.
///
/// `amount` represents the number of bytes that the caller would like to
/// receive at the time the function is called. For example, if
/// `consume_hint` is called with 20, the consumer requests 20 bytes. The
/// stream may yield less than that. If the next call to `poll_buf` returns
/// 5 bytes, the consumer still has 15 bytes requested. At this point,
/// invoking `consume_hint` again with 20 resets the amount requested back
/// to 20 bytes.
///
/// Calling `consume_hint` with 0 as the argument informs the stream that
/// the caller does not intend to call `poll_buf`. If `poll_buf` **is**
/// called, the stream may, but is not obligated to, return `NotReady` even
/// if it could produce data at that point. If it chooses to return
/// `NotReady`, when `consume_hint` is called with a non-zero argument, the
/// task must be notified in order to respect the `poll_buf` contract.
fn consume_hint(&mut self, amount: usize) {
// By default, this function does nothing
drop(amount);
}
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
}
+83 -4
View File
@@ -1,5 +1,5 @@
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.0")]
#![deny(missing_docs, missing_debug_implementations)]
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.1")]
#![deny(missing_docs, missing_debug_implementations, unreachable_pub)]
#![cfg_attr(test, deny(warnings))]
//! Asynchronous stream of bytes.
@@ -10,11 +10,90 @@
//! `Buf` (i.e, byte collections).
extern crate bytes;
#[cfg(feature = "util")]
extern crate either;
#[allow(unused)]
#[macro_use]
extern crate futures;
pub mod buf_stream;
mod never;
mod size_hint;
mod str;
mod u8;
#[cfg(feature = "util")]
pub mod util;
pub use self::size_hint::SizeHint;
#[doc(inline)]
pub use buf_stream::BufStream;
#[cfg(feature = "util")]
pub use util::BufStreamExt;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
}
+22
View File
@@ -0,0 +1,22 @@
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
@@ -29,7 +29,7 @@ impl SizeHint {
///
/// # Panics
///
/// The function panics if `value` is less than `upper`.
/// The function panics if `value` is greater than `upper`.
pub fn set_lower(&mut self, value: u64) {
assert!(value <= self.upper.unwrap_or(u64::MAX));
self.lower = value;
@@ -1,5 +1,6 @@
use never::Never;
use BufStream;
use buf_stream::errors::internal::Never;
use SizeHint;
use futures::Poll;
@@ -20,6 +21,10 @@ impl BufStream for String {
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
impl BufStream for &'static str {
@@ -36,4 +41,15 @@ impl BufStream for &'static str {
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
fn size_hint(s: &str) -> SizeHint {
let mut hint = SizeHint::new();
hint.set_lower(s.len() as u64);
hint.set_upper(s.len() as u64);
hint
}
@@ -1,10 +1,8 @@
use BufStream;
use buf_stream::errors::internal::Never;
use bytes::{Bytes, BytesMut};
use futures::Poll;
use never::Never;
use std::io;
use BufStream;
impl BufStream for Vec<u8> {
type Item = io::Cursor<Vec<u8>>;
@@ -58,9 +56,7 @@ impl BufStream for BytesMut {
}
}
fn poll_bytes<T: Default>(buf: &mut T)
-> Poll<Option<io::Cursor<T>>, Never>
{
fn poll_bytes<T: Default>(buf: &mut T) -> Poll<Option<io::Cursor<T>>, Never> {
use std::mem;
let bytes = mem::replace(buf, Default::default());
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use either::Either;
use futures::Poll;
@@ -43,9 +43,4 @@ where
let res = try_ready!(self.right.poll_buf());
Ok(res.map(Either::Right).into())
}
fn size_hint(&self) -> SizeHint {
// TODO: Implement
SizeHint::default()
}
}
@@ -1,4 +1,5 @@
use super::{BufStream, FromBufStream};
use super::FromBufStream;
use BufStream;
use futures::{Future, Poll};
@@ -52,29 +53,26 @@ where
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
loop {
let res = self.stream.poll_buf()
.map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
let res = self.stream.poll_buf().map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
match try_ready!(res) {
Some(mut buf) => {
let builder = self.builder.as_mut().expect("cannot poll after done");
U::extend(builder, &mut buf, &self.stream.size_hint())
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
U::extend(builder, &mut buf, &self.stream.size_hint()).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
}
None => {
let builder = self.builder.take().expect("cannot poll after done");
let value = U::build(builder)
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
let value = U::build(builder).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
return Ok(value.into());
}
}
@@ -1,7 +1,9 @@
use super::SizeHint;
use SizeHint;
use bytes::{Buf, BufMut};
use bytes::{Buf, BufMut, Bytes};
use std::error::Error;
use std::fmt;
use std::usize;
/// Conversion from a `BufStream`.
@@ -43,14 +45,22 @@ pub trait FromBufStream<T: Buf>: Sized {
/// Error returned from collecting into a `Vec<u8>`
#[derive(Debug)]
pub struct CollectVecError { _p: () }
pub struct CollectVecError {
_p: (),
}
/// Error returned from collecting into a `Bytes`
#[derive(Debug)]
pub struct CollectBytesError {
_p: (),
}
impl<T: Buf> FromBufStream<T> for Vec<u8> {
type Builder = Vec<u8>;
type Error = CollectVecError;
fn builder(_hint: &SizeHint) -> Vec<u8> {
Vec::new()
fn builder(hint: &SizeHint) -> Vec<u8> {
Vec::with_capacity(hint.lower() as usize)
}
fn extend(builder: &mut Self, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
@@ -70,7 +80,7 @@ impl<T: Buf> FromBufStream<T> for Vec<u8> {
Some(upper) if upper <= 64 => {
reserve = upper as usize;
}
_ => {},
_ => {}
}
// hint.lower() represents the minimum amount of data that will be
@@ -108,3 +118,45 @@ impl<T: Buf> FromBufStream<T> for Vec<u8> {
Ok(builder)
}
}
impl<T: Buf> FromBufStream<T> for Bytes {
type Builder = Vec<u8>;
type Error = CollectBytesError;
fn builder(hint: &SizeHint) -> Vec<u8> {
<Vec<u8> as FromBufStream<T>>::builder(hint)
}
fn extend(builder: &mut Vec<u8>, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
<Vec<u8> as FromBufStream<T>>::extend(builder, buf, hint)
.map_err(|_| CollectBytesError { _p: () })
}
fn build(builder: Vec<u8>) -> Result<Self, Self::Error> {
Ok(builder.into())
}
}
impl fmt::Display for CollectVecError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "BufStream is too big")
}
}
impl Error for CollectVecError {
fn description(&self) -> &str {
"BufStream too big"
}
}
impl fmt::Display for CollectBytesError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "BufStream too big")
}
}
impl Error for CollectBytesError {
fn description(&self) -> &str {
"BufStream too big"
}
}
+54
View File
@@ -0,0 +1,54 @@
use bytes::Buf;
use futures::Poll;
use std::error::Error;
use std::fmt;
use BufStream;
/// Converts an `Iterator` into a `BufStream` which is always ready to yield the
/// next value.
///
/// Iterators in Rust don't express the ability to block, so this adapter
/// simply always calls `iter.next()` and returns that.
pub fn iter<I>(i: I) -> Iter<I::IntoIter>
where
I: IntoIterator,
I::Item: Buf,
{
Iter {
iter: i.into_iter(),
}
}
/// `BufStream` returned by the [`iter`] function.
#[derive(Debug)]
pub struct Iter<I> {
iter: I,
}
#[derive(Debug)]
pub enum Never {}
impl<I> BufStream for Iter<I>
where
I: Iterator,
I::Item: Buf,
{
type Item = I::Item;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
Ok(self.iter.next().into())
}
}
impl fmt::Display for Never {
fn fmt(&self, _: &mut fmt::Formatter) -> fmt::Result {
unreachable!();
}
}
impl Error for Never {
fn description(&self) -> &str {
unreachable!();
}
}
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use bytes::Buf;
use futures::Poll;
@@ -40,10 +40,10 @@ where
return Err(LimitError { inner: None });
}
let res = self.stream.poll_buf()
.map_err(|err| {
LimitError { inner: Some(err) }
});
let res = self
.stream
.poll_buf()
.map_err(|err| LimitError { inner: Some(err) });
match res {
Ok(Ready(Some(ref buf))) => {
@@ -59,22 +59,6 @@ where
res
}
fn size_hint(&self) -> SizeHint {
let mut hint = self.stream.size_hint();
let upper = hint.upper()
.map(|upper| upper.min(self.remaining))
.unwrap_or(self.remaining);
hint.set_upper(upper);
hint
}
fn consume_hint(&mut self, amount: usize) {
// TODO: Should this be capped by `self.remaining`?
self.stream.consume_hint(amount)
}
}
// ===== impl LimitError =====
+87
View File
@@ -0,0 +1,87 @@
//! Types and utilities for working with `BufStream`.
mod chain;
mod collect;
mod from;
mod iter;
mod limit;
mod stream;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::iter::iter;
pub use self::limit::Limit;
pub use self::stream::{stream, IntoStream};
pub mod error {
//! Error types
pub use super::collect::CollectError;
pub use super::from::{CollectBytesError, CollectVecError};
pub use super::limit::LimitError;
}
use BufStream;
impl<T> BufStreamExt for T where T: BufStream {}
/// An extension trait for `BufStream`'s that provides a variety of convenient
/// adapters.
pub trait BufStreamExt: BufStream {
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
/// Creates a `Stream` from a `BufStream`.
///
/// This produces a `Stream` of `BufStream::Items`.
fn into_stream(self) -> IntoStream<Self>
where
Self: Sized,
{
IntoStream::new(self)
}
}
+76
View File
@@ -0,0 +1,76 @@
use bytes::Buf;
use futures::{Async, Poll, Stream};
use BufStream;
/// Converts a `Stream` of `Buf` types into a `BufStream`.
///
/// While `Stream` and `BufStream` are very similar, they are not identical. The
/// `stream` function returns a `BufStream` that is backed by the provided
/// `Stream` type.
pub fn stream<T>(stream: T) -> FromStream<T>
where
T: Stream,
T::Item: Buf,
{
FromStream { stream }
}
/// `BufStream` returned by the [`stream`] function.
#[derive(Debug)]
pub struct FromStream<T> {
stream: T,
}
impl<T> BufStream for FromStream<T>
where
T: Stream,
T::Item: Buf,
{
type Item = T::Item;
type Error = T::Error;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.stream.poll()
}
}
/// Converts a `BufStream` into a `Stream`.
#[derive(Debug)]
pub struct IntoStream<T> {
buf: T,
}
impl<T> IntoStream<T> {
/// Create a new `Stream` from the provided `BufStream`.
pub fn new(buf: T) -> Self {
IntoStream { buf }
}
/// Get a reference to the inner `BufStream`.
pub fn get_ref(&self) -> &T {
&self.buf
}
/// Get a mutable reference to the inner `BufStream`
pub fn get_mut(&mut self) -> &mut T {
&mut self.buf
}
/// Get the inner `BufStream`.
pub fn into_inner(self) -> T {
self.buf
}
}
impl<T: BufStream> Stream for IntoStream<T> {
type Item = T::Item;
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.buf.poll_buf()? {
Async::Ready(Some(buf)) => Ok(Async::Ready(Some(buf))),
Async::Ready(None) => Ok(Async::Ready(None)),
Async::NotReady => Ok(Async::NotReady),
}
}
}
+4 -326
View File
@@ -1,329 +1,7 @@
extern crate tokio_buf;
extern crate bytes;
extern crate futures;
use tokio_buf::buf_stream::{BufStream, SizeHint};
use bytes::Buf;
use futures::{Future, Poll};
use futures::Async::*;
use tokio_buf::BufStream;
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
}
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
}
}
// ===== test `SizeHint` =====
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
// ===== test `chain()` =====
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"])
.chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[
Ok(Ready("foo")),
Ok(NotReady),
Ok(Ready("bar"))
]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
// ===== Test `collect()` =====
#[test]
fn collect_vec() {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world, one two three");
}
// ===== Test limit() =====
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world")
.limit(5)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let res = one("hello world")
.limit(10)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"])
.limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
// ===== BufStream impelmentations for misc types =====
#[test]
fn str_buf_stream() {
let mut bs = "hello world".to_string();
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
let mut bs = "hello world";
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
}
// ===== Test utils =====
fn one(buf: &'static str) -> Mock {
list(&[buf])
}
fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
struct Mock {
polls: VecDeque<Poll<&'static [u8], ()>>,
size_hint: SizeHint,
}
#[derive(Debug)]
struct MockBuf {
data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
// Ensures that `BufStream` can be a trait object
#[allow(dead_code)]
fn obj(_: &mut BufStream<Item = u32, Error = ()>) {}
+43
View File
@@ -0,0 +1,43 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"]).chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[Ok(Ready("foo")), Ok(NotReady), Ok(Ready("bar"))]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
+68
View File
@@ -0,0 +1,68 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Bytes;
use futures::Future;
use tokio_buf::BufStreamExt;
#[macro_use]
mod support;
use support::*;
macro_rules! test_collect_impl {
($t:ty $(, $capacity:ident)*) => {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world, one two three"[..]);
}
}
#[test]
fn collect_vec() {
test_collect_impl!(Vec<u8>, capacity);
}
#[test]
fn collect_bytes() {
test_collect_impl!(Bytes);
}
+33
View File
@@ -0,0 +1,33 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
#[macro_use]
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_iter() {
let mut bs = util::iter(Vec::<Buf>::new());
assert_none!(bs.poll_buf());
}
#[test]
fn full_iter() {
let bufs = vec![buf(b"one"), buf(b"two"), buf(b"three")];
let mut bs = util::iter(bufs);
assert_buf_eq!(bs.poll_buf(), "one");
assert_buf_eq!(bs.poll_buf(), "two");
assert_buf_eq!(bs.poll_buf(), "three");
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}
+65
View File
@@ -0,0 +1,65 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use futures::Future;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world").limit(5).collect::<Vec<_>>().wait();
assert!(res.is_err());
let res = one("hello world").limit(10).collect::<Vec<_>>().wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"]).limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
+42
View File
@@ -0,0 +1,42 @@
extern crate tokio_buf;
use tokio_buf::SizeHint;
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
+49
View File
@@ -0,0 +1,49 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
extern crate tokio_mock_task;
use futures::sync::mpsc;
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
use tokio_mock_task::MockTask;
#[macro_use]
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_stream() {
let (_, rx) = mpsc::unbounded::<Buf>();
let mut bs = util::stream(rx);
assert_none!(bs.poll_buf());
}
#[test]
fn full_stream() {
let (tx, rx) = mpsc::unbounded();
let mut bs = util::stream(rx);
let mut task = MockTask::new();
tx.unbounded_send(buf(b"one")).unwrap();
assert_buf_eq!(bs.poll_buf(), "one");
task.enter(|| assert_not_ready!(bs.poll_buf()));
tx.unbounded_send(buf(b"two")).unwrap();
assert!(task.is_notified());
assert_buf_eq!(bs.poll_buf(), "two");
task.enter(|| assert_not_ready!(bs.poll_buf()));
drop(tx);
assert!(task.is_notified());
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}
+38
View File
@@ -0,0 +1,38 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use std::fmt;
use tokio_buf::BufStream;
#[macro_use]
mod support;
fn test_hello_world<B>(mut bs: B)
where
B: BufStream + fmt::Debug,
B::Item: fmt::Debug,
B::Error: fmt::Debug,
{
let hint = bs.size_hint();
assert_eq!(hint.lower(), 11);
assert_eq!(hint.upper(), Some(11));
assert_buf_eq!(bs.poll_buf(), "hello world");
let hint = bs.size_hint();
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(0));
assert_none!(bs.poll_buf());
}
#[test]
fn string() {
test_hello_world("hello world".to_string());
}
#[test]
fn str() {
test_hello_world("hello world");
}
+133
View File
@@ -0,0 +1,133 @@
#![allow(unused)]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Poll;
use tokio_buf::{BufStream, SizeHint};
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
use bytes::Buf;
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
};
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
};
}
// ===== Test utils =====
pub fn one(buf: &'static str) -> Mock {
list(&[buf])
}
pub fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
pub fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
pub struct Mock {
pub polls: VecDeque<Poll<&'static [u8], ()>>,
pub size_hint: SizeHint,
}
#[derive(Debug)]
pub struct MockBuf {
pub data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
View File
-20
View File
@@ -1,20 +0,0 @@
[package]
name = "tokio-channel"
# When releasing to crates.io:
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-channel/0.1.0"
description = """
Channels for asynchronous communication using Tokio.
"""
categories = ["asynchronous"]
[dependencies]
futures = "0.1.23"
-51
View File
@@ -1,51 +0,0 @@
Copyright (c) 2019 Tokio Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
Copyright (c) 2016 futures-rs Contributors
Permission is hereby granted, free of charge, to any
person obtaining a copy of this software and associated
documentation files (the "Software"), to deal in the
Software without restriction, including without
limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software
is furnished to do so, subject to the following
conditions:
The above copyright notice and this permission notice
shall be included in all copies or substantial portions
of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
View File
-14
View File
@@ -1,14 +0,0 @@
#![doc(html_root_url = "https://docs.rs/tokio-channel/0.1.0")]
#![deny(missing_docs, warnings, missing_debug_implementations)]
//! Asynchronous channels.
//!
//! This crate provides channels that can be used to communicate between
//! asynchronous tasks.
extern crate futures;
pub mod mpsc;
pub mod oneshot;
mod lock;
-105
View File
@@ -1,105 +0,0 @@
//! A "mutex" which only supports `try_lock`
//!
//! As a futures library the eventual call to an event loop should be the only
//! thing that ever blocks, so this is assisted with a fast user-space
//! implementation of a lock that can only have a `try_lock` operation.
use std::cell::UnsafeCell;
use std::ops::{Deref, DerefMut};
use std::sync::atomic::Ordering::SeqCst;
use std::sync::atomic::AtomicBool;
/// A "mutex" around a value, similar to `std::sync::Mutex<T>`.
///
/// This lock only supports the `try_lock` operation, however, and does not
/// implement poisoning.
#[derive(Debug)]
pub struct Lock<T> {
locked: AtomicBool,
data: UnsafeCell<T>,
}
/// Sentinel representing an acquired lock through which the data can be
/// accessed.
pub struct TryLock<'a, T: 'a> {
__ptr: &'a Lock<T>,
}
// The `Lock` structure is basically just a `Mutex<T>`, and these two impls are
// intended to mirror the standard library's corresponding impls for `Mutex<T>`.
//
// If a `T` is sendable across threads, so is the lock, and `T` must be sendable
// across threads to be `Sync` because it allows mutable access from multiple
// threads.
unsafe impl<T: Send> Send for Lock<T> {}
unsafe impl<T: Send> Sync for Lock<T> {}
impl<T> Lock<T> {
/// Creates a new lock around the given value.
pub fn new(t: T) -> Lock<T> {
Lock {
locked: AtomicBool::new(false),
data: UnsafeCell::new(t),
}
}
/// Attempts to acquire this lock, returning whether the lock was acquired or
/// not.
///
/// If `Some` is returned then the data this lock protects can be accessed
/// through the sentinel. This sentinel allows both mutable and immutable
/// access.
///
/// If `None` is returned then the lock is already locked, either elsewhere
/// on this thread or on another thread.
pub fn try_lock(&self) -> Option<TryLock<T>> {
if !self.locked.swap(true, SeqCst) {
Some(TryLock { __ptr: self })
} else {
None
}
}
}
impl<'a, T> Deref for TryLock<'a, T> {
type Target = T;
fn deref(&self) -> &T {
// The existence of `TryLock` represents that we own the lock, so we
// can safely access the data here.
unsafe { &*self.__ptr.data.get() }
}
}
impl<'a, T> DerefMut for TryLock<'a, T> {
fn deref_mut(&mut self) -> &mut T {
// The existence of `TryLock` represents that we own the lock, so we
// can safely access the data here.
//
// Additionally, we're the *only* `TryLock` in existence so mutable
// access should be ok.
unsafe { &mut *self.__ptr.data.get() }
}
}
impl<'a, T> Drop for TryLock<'a, T> {
fn drop(&mut self) {
self.__ptr.locked.store(false, SeqCst);
}
}
#[cfg(test)]
mod tests {
use super::Lock;
#[test]
fn smoke() {
let a = Lock::new(1);
let mut a1 = a.try_lock().unwrap();
assert!(a.try_lock().is_none());
assert_eq!(*a1, 1);
*a1 = 2;
drop(a1);
assert_eq!(*a.try_lock().unwrap(), 2);
assert_eq!(*a.try_lock().unwrap(), 2);
}
}
-989
View File
@@ -1,989 +0,0 @@
//! A multi-producer, single-consumer, futures-aware, FIFO queue with back pressure.
//!
//! A channel can be used as a communication primitive between tasks running on
//! `futures-rs` executors. Channel creation provides `Receiver` and `Sender`
//! handles. `Receiver` implements `Stream` and allows a task to read values
//! out of the channel. If there is no message to read from the channel, the
//! current task will be notified when a new value is sent. `Sender` implements
//! the `Sink` trait and allows a task to send messages into the channel. If
//! the channel is at capacity, then send will be rejected and the task will be
//! notified when additional capacity is available.
//!
//! # Disconnection
//!
//! When all `Sender` handles have been dropped, it is no longer possible to
//! send values into the channel. This is considered the termination event of
//! the stream. As such, `Sender::poll` will return `Ok(Ready(None))`.
//!
//! If the receiver handle is dropped, then messages can no longer be read out
//! of the channel. In this case, a `send` will result in an error.
//!
//! # Clean Shutdown
//!
//! If the `Receiver` is simply dropped, then it is possible for there to be
//! messages still in the channel that will not be processed. As such, it is
//! usually desirable to perform a "clean" shutdown. To do this, the receiver
//! will first call `close`, which will prevent any further messages to be sent
//! into the channel. Then, the receiver consumes the channel to completion, at
//! which point the receiver can be dropped.
// At the core, the channel uses an atomic FIFO queue for message passing. This
// queue is used as the primary coordination primitive. In order to enforce
// capacity limits and handle back pressure, a secondary FIFO queue is used to
// send parked task handles.
//
// The general idea is that the channel is created with a `buffer` size of `n`.
// The channel capacity is `n + num-senders`. Each sender gets one "guaranteed"
// slot to hold a message. This allows `Sender` to know for a fact that a send
// will succeed *before* starting to do the actual work of sending the value.
// Since most of this work is lock-free, once the work starts, it is impossible
// to safely revert.
//
// If the sender is unable to process a send operation, then the current
// task is parked and the handle is sent on the parked task queue.
//
// Note that the implementation guarantees that the channel capacity will never
// exceed the configured limit, however there is no *strict* guarantee that the
// receiver will wake up a parked task *immediately* when a slot becomes
// available. However, it will almost always unpark a task when a slot becomes
// available and it is *guaranteed* that a sender will be unparked when the
// message that caused the sender to become parked is read out of the channel.
//
// The steps for sending a message are roughly:
//
// 1) Increment the channel message count
// 2) If the channel is at capacity, push the task handle onto the wait queue
// 3) Push the message onto the message queue.
//
// The steps for receiving a message are roughly:
//
// 1) Pop a message from the message queue
// 2) Pop a task handle from the wait queue
// 3) Decrement the channel message count.
//
// It's important for the order of operations on lock-free structures to happen
// in reverse order between the sender and receiver. This makes the message
// queue the primary coordination structure and establishes the necessary
// happens-before semantics required for the acquire / release semantics used
// by the queue structure.
use mpsc::queue::{Queue, PopResult};
use futures::task::{self, Task};
use futures::{Async, AsyncSink, Poll, StartSend, Sink, Stream};
use std::fmt;
use std::error::Error;
use std::any::Any;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering::SeqCst;
use std::sync::{Arc, Mutex};
use std::thread;
use std::usize;
mod queue;
/// The transmission end of a channel which is used to send values.
///
/// This is created by the `channel` method.
#[derive(Debug)]
pub struct Sender<T> {
// Channel state shared between the sender and receiver.
inner: Arc<Inner<T>>,
// Handle to the task that is blocked on this sender. This handle is sent
// to the receiver half in order to be notified when the sender becomes
// unblocked.
sender_task: Arc<Mutex<SenderTask>>,
// True if the sender might be blocked. This is an optimization to avoid
// having to lock the mutex most of the time.
maybe_parked: bool,
}
/// The transmission end of a channel which is used to send values.
///
/// This is created by the `unbounded` method.
#[derive(Debug)]
pub struct UnboundedSender<T>(Sender<T>);
trait AssertKinds: Send + Sync + Clone {}
impl AssertKinds for UnboundedSender<u32> {}
/// The receiving end of a channel which implements the `Stream` trait.
///
/// This is a concrete implementation of a stream which can be used to represent
/// a stream of values being computed elsewhere. This is created by the
/// `channel` method.
#[derive(Debug)]
pub struct Receiver<T> {
inner: Arc<Inner<T>>,
}
/// Error type for sending, used when the receiving end of a channel is
/// dropped
#[derive(Clone, PartialEq, Eq)]
pub struct SendError<T>(T);
/// Error type returned from `try_send`
#[derive(Clone, PartialEq, Eq)]
pub struct TrySendError<T> {
kind: TrySendErrorKind<T>,
}
#[derive(Clone, PartialEq, Eq)]
enum TrySendErrorKind<T> {
Full(T),
Disconnected(T),
}
impl<T> fmt::Debug for SendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_tuple("SendError")
.field(&"...")
.finish()
}
}
impl<T> fmt::Display for SendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "send failed because receiver is gone")
}
}
impl<T: Any> Error for SendError<T>
{
fn description(&self) -> &str {
"send failed because receiver is gone"
}
}
impl<T> SendError<T> {
/// Returns the message that was attempted to be sent but failed.
pub fn into_inner(self) -> T {
self.0
}
}
impl<T> fmt::Debug for TrySendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_tuple("TrySendError")
.field(&"...")
.finish()
}
}
impl<T> fmt::Display for TrySendError<T> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
if self.is_full() {
write!(fmt, "send failed because channel is full")
} else {
write!(fmt, "send failed because receiver is gone")
}
}
}
impl<T: Any> Error for TrySendError<T> {
fn description(&self) -> &str {
if self.is_full() {
"send failed because channel is full"
} else {
"send failed because receiver is gone"
}
}
}
impl<T> TrySendError<T> {
/// Returns true if this error is a result of the channel being full
pub fn is_full(&self) -> bool {
use self::TrySendErrorKind::*;
match self.kind {
Full(_) => true,
_ => false,
}
}
/// Returns true if this error is a result of the receiver being dropped
pub fn is_disconnected(&self) -> bool {
use self::TrySendErrorKind::*;
match self.kind {
Disconnected(_) => true,
_ => false,
}
}
/// Returns the message that was attempted to be sent but failed.
pub fn into_inner(self) -> T {
use self::TrySendErrorKind::*;
match self.kind {
Full(v) | Disconnected(v) => v,
}
}
}
#[derive(Debug)]
struct Inner<T> {
// Max buffer size of the channel. If `None` then the channel is unbounded.
buffer: Option<usize>,
// Internal channel state. Consists of the number of messages stored in the
// channel as well as a flag signalling that the channel is closed.
state: AtomicUsize,
// Atomic, FIFO queue used to send messages to the receiver
message_queue: Queue<Option<T>>,
// Atomic, FIFO queue used to send parked task handles to the receiver.
parked_queue: Queue<Arc<Mutex<SenderTask>>>,
// Number of senders in existence
num_senders: AtomicUsize,
// Handle to the receiver's task.
recv_task: Mutex<ReceiverTask>,
}
// Struct representation of `Inner::state`.
#[derive(Debug, Clone, Copy)]
struct State {
// `true` when the channel is open
is_open: bool,
// Number of messages in the channel
num_messages: usize,
}
#[derive(Debug)]
struct ReceiverTask {
unparked: bool,
task: Option<Task>,
}
// Returned from Receiver::try_park()
enum TryPark {
Parked,
Closed,
NotEmpty,
}
// The `is_open` flag is stored in the left-most bit of `Inner::state`
const OPEN_MASK: usize = usize::MAX - (usize::MAX >> 1);
// When a new channel is created, it is created in the open state with no
// pending messages.
const INIT_STATE: usize = OPEN_MASK;
// The maximum number of messages that a channel can track is `usize::MAX >> 1`
const MAX_CAPACITY: usize = !(OPEN_MASK);
// The maximum requested buffer size must be less than the maximum capacity of
// a channel. This is because each sender gets a guaranteed slot.
const MAX_BUFFER: usize = MAX_CAPACITY >> 1;
// Sent to the consumer to wake up blocked producers
#[derive(Debug)]
struct SenderTask {
task: Option<Task>,
is_parked: bool,
}
impl SenderTask {
fn new() -> Self {
SenderTask {
task: None,
is_parked: false,
}
}
fn notify(&mut self) {
self.is_parked = false;
if let Some(task) = self.task.take() {
task.notify();
}
}
}
/// Creates an in-memory channel implementation of the `Stream` trait with
/// bounded capacity.
///
/// This method creates a concrete implementation of the `Stream` trait which
/// can be used to send values across threads in a streaming fashion. This
/// channel is unique in that it implements back pressure to ensure that the
/// sender never outpaces the receiver. The channel capacity is equal to
/// `buffer + num-senders`. In other words, each sender gets a guaranteed slot
/// in the channel capacity, and on top of that there are `buffer` "first come,
/// first serve" slots available to all senders.
///
/// The `Receiver` returned implements the `Stream` trait and has access to any
/// number of the associated combinators for transforming the result.
pub fn channel<T>(buffer: usize) -> (Sender<T>, Receiver<T>) {
// Check that the requested buffer size does not exceed the maximum buffer
// size permitted by the system.
assert!(buffer < MAX_BUFFER, "requested buffer size too large");
channel2(Some(buffer))
}
/// Creates an in-memory channel implementation of the `Stream` trait with
/// unbounded capacity.
///
/// This method creates a concrete implementation of the `Stream` trait which
/// can be used to send values across threads in a streaming fashion. A `send`
/// on this channel will always succeed as long as the receive half has not
/// been closed. If the receiver falls behind, messages will be buffered
/// internally.
///
/// **Note** that the amount of available system memory is an implicit bound to
/// the channel. Using an `unbounded` channel has the ability of causing the
/// process to run out of memory. In this case, the process will be aborted.
pub fn unbounded<T>() -> (UnboundedSender<T>, Receiver<T>) {
let (tx, rx) = channel2(None);
(UnboundedSender(tx), rx)
}
fn channel2<T>(buffer: Option<usize>) -> (Sender<T>, Receiver<T>) {
let inner = Arc::new(Inner {
buffer: buffer,
state: AtomicUsize::new(INIT_STATE),
message_queue: Queue::new(),
parked_queue: Queue::new(),
num_senders: AtomicUsize::new(1),
recv_task: Mutex::new(ReceiverTask {
unparked: false,
task: None,
}),
});
let tx = Sender {
inner: inner.clone(),
sender_task: Arc::new(Mutex::new(SenderTask::new())),
maybe_parked: false,
};
let rx = Receiver {
inner: inner,
};
(tx, rx)
}
/*
*
* ===== impl Sender =====
*
*/
impl<T> Sender<T> {
/// Attempts to send a message on this `Sender<T>` without blocking.
///
/// This function, unlike `start_send`, is safe to call whether it's being
/// called on a task or not. Note that this function, however, will *not*
/// attempt to block the current task if the message cannot be sent.
///
/// It is not recommended to call this function from inside of a future,
/// only from an external thread where you've otherwise arranged to be
/// notified when the channel is no longer full.
pub fn try_send(&mut self, msg: T) -> Result<(), TrySendError<T>> {
// If the sender is currently blocked, reject the message
if !self.poll_unparked(false).is_ready() {
return Err(TrySendError {
kind: TrySendErrorKind::Full(msg),
});
}
// The channel has capacity to accept the message, so send it
self.do_send(Some(msg), false)
.map_err(|SendError(v)| {
TrySendError {
kind: TrySendErrorKind::Disconnected(v),
}
})
}
// Do the send without failing
// None means close
fn do_send(&mut self, msg: Option<T>, do_park: bool) -> Result<(), SendError<T>> {
// First, increment the number of messages contained by the channel.
// This operation will also atomically determine if the sender task
// should be parked.
//
// None is returned in the case that the channel has been closed by the
// receiver. This happens when `Receiver::close` is called or the
// receiver is dropped.
let park_self = match self.inc_num_messages(msg.is_none()) {
Some(park_self) => park_self,
None => {
// The receiver has closed the channel. Only abort if actually
// sending a message. It is important that the stream
// termination (None) is always sent. This technically means
// that it is possible for the queue to contain the following
// number of messages:
//
// num-senders + buffer + 1
//
if let Some(msg) = msg {
return Err(SendError(msg));
} else {
return Ok(());
}
}
};
// If the channel has reached capacity, then the sender task needs to
// be parked. This will send the task handle on the parked task queue.
//
// However, when `do_send` is called while dropping the `Sender`,
// `task::current()` can't be called safely. In this case, in order to
// maintain internal consistency, a blank message is pushed onto the
// parked task queue.
if park_self {
self.park(do_park);
}
self.queue_push_and_signal(msg);
Ok(())
}
// Do the send without parking current task.
//
// To be called from unbounded sender.
fn do_send_nb(&self, msg: T) -> Result<(), SendError<T>> {
match self.inc_num_messages(false) {
Some(park_self) => assert!(!park_self),
None => return Err(SendError(msg)),
};
self.queue_push_and_signal(Some(msg));
Ok(())
}
// Push message to the queue and signal to the receiver
fn queue_push_and_signal(&self, msg: Option<T>) {
// Push the message onto the message queue
self.inner.message_queue.push(msg);
// Signal to the receiver that a message has been enqueued. If the
// receiver is parked, this will unpark the task.
self.signal();
}
// Increment the number of queued messages. Returns if the sender should
// block.
fn inc_num_messages(&self, close: bool) -> Option<bool> {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
// The receiver end closed the channel.
if !state.is_open {
return None;
}
// This probably is never hit? Odds are the process will run out of
// memory first. It may be worth to return something else in this
// case?
assert!(state.num_messages < MAX_CAPACITY, "buffer space exhausted; \
sending this messages would overflow the state");
state.num_messages += 1;
// The channel is closed by all sender handles being dropped.
if close {
state.is_open = false;
}
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => {
// Block if the current number of pending messages has exceeded
// the configured buffer size
let park_self = match self.inner.buffer {
Some(buffer) => state.num_messages > buffer,
None => false,
};
return Some(park_self)
}
Err(actual) => curr = actual,
}
}
}
// Signal to the receiver task that a message has been enqueued
fn signal(&self) {
// TODO
// This logic can probably be improved by guarding the lock with an
// atomic.
//
// Do this step first so that the lock is dropped when
// `unpark` is called
let task = {
let mut recv_task = self.inner.recv_task.lock().unwrap();
// If the receiver has already been unparked, then there is nothing
// more to do
if recv_task.unparked {
return;
}
// Setting this flag enables the receiving end to detect that
// an unpark event happened in order to avoid unnecessarily
// parking.
recv_task.unparked = true;
recv_task.task.take()
};
if let Some(task) = task {
task.notify();
}
}
fn park(&mut self, can_park: bool) {
// TODO: clean up internal state if the task::current will fail
let task = if can_park {
Some(task::current())
} else {
None
};
{
let mut sender = self.sender_task.lock().unwrap();
sender.task = task;
sender.is_parked = true;
}
// Send handle over queue
let t = self.sender_task.clone();
self.inner.parked_queue.push(t);
// Check to make sure we weren't closed after we sent our task on the
// queue
let state = decode_state(self.inner.state.load(SeqCst));
self.maybe_parked = state.is_open;
}
/// Polls the channel to determine if there is guaranteed to be capacity to send at least one
/// item without waiting.
///
/// Returns `Ok(Async::Ready(_))` if there is sufficient capacity, or returns
/// `Ok(Async::NotReady)` if the channel is not guaranteed to have capacity. Returns
/// `Err(SendError(_))` if the receiver has been dropped.
///
/// # Panics
///
/// This method will panic if called from outside the context of a task or future.
pub fn poll_ready(&mut self) -> Poll<(), SendError<()>> {
let state = decode_state(self.inner.state.load(SeqCst));
if !state.is_open {
return Err(SendError(()));
}
Ok(self.poll_unparked(true))
}
fn poll_unparked(&mut self, do_park: bool) -> Async<()> {
// First check the `maybe_parked` variable. This avoids acquiring the
// lock in most cases
if self.maybe_parked {
// Get a lock on the task handle
let mut task = self.sender_task.lock().unwrap();
if !task.is_parked {
self.maybe_parked = false;
return Async::Ready(())
}
// At this point, an unpark request is pending, so there will be an
// unpark sometime in the future. We just need to make sure that
// the correct task will be notified.
//
// Update the task in case the `Sender` has been moved to another
// task
task.task = if do_park {
Some(task::current())
} else {
None
};
Async::NotReady
} else {
Async::Ready(())
}
}
}
impl<T> Sink for Sender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
// If the sender is currently blocked, reject the message before doing
// any work.
if !self.poll_unparked(true).is_ready() {
return Ok(AsyncSink::NotReady(msg));
}
// The channel has capacity to accept the message, so send it.
self.do_send(Some(msg), true)?;
Ok(AsyncSink::Ready)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<T> UnboundedSender<T> {
/// Sends the provided message along this channel.
///
/// This is an unbounded sender, so this function differs from `Sink::send`
/// by ensuring the return type reflects that the channel is always ready to
/// receive messages.
#[deprecated(note = "renamed to `unbounded_send`")]
#[doc(hidden)]
pub fn send(&self, msg: T) -> Result<(), SendError<T>> {
self.unbounded_send(msg)
}
/// Sends the provided message along this channel.
///
/// This is an unbounded sender, so this function differs from `Sink::send`
/// by ensuring the return type reflects that the channel is always ready to
/// receive messages.
pub fn unbounded_send(&self, msg: T) -> Result<(), SendError<T>> {
self.0.do_send_nb(msg)
}
}
impl<T> Sink for UnboundedSender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
self.0.start_send(msg)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
self.0.poll_complete()
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<'a, T> Sink for &'a UnboundedSender<T> {
type SinkItem = T;
type SinkError = SendError<T>;
fn start_send(&mut self, msg: T) -> StartSend<T, SendError<T>> {
self.0.do_send_nb(msg)?;
Ok(AsyncSink::Ready)
}
fn poll_complete(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
fn close(&mut self) -> Poll<(), SendError<T>> {
Ok(Async::Ready(()))
}
}
impl<T> Clone for UnboundedSender<T> {
fn clone(&self) -> UnboundedSender<T> {
UnboundedSender(self.0.clone())
}
}
impl<T> Clone for Sender<T> {
fn clone(&self) -> Sender<T> {
// Since this atomic op isn't actually guarding any memory and we don't
// care about any orderings besides the ordering on the single atomic
// variable, a relaxed ordering is acceptable.
let mut curr = self.inner.num_senders.load(SeqCst);
loop {
// If the maximum number of senders has been reached, then fail
if curr == self.inner.max_senders() {
panic!("cannot clone `Sender` -- too many outstanding senders");
}
debug_assert!(curr < self.inner.max_senders());
let next = curr + 1;
let actual = self.inner.num_senders.compare_and_swap(curr, next, SeqCst);
// The ABA problem doesn't matter here. We only care that the
// number of senders never exceeds the maximum.
if actual == curr {
return Sender {
inner: self.inner.clone(),
sender_task: Arc::new(Mutex::new(SenderTask::new())),
maybe_parked: false,
};
}
curr = actual;
}
}
}
impl<T> Drop for Sender<T> {
fn drop(&mut self) {
// Ordering between variables don't matter here
let prev = self.inner.num_senders.fetch_sub(1, SeqCst);
if prev == 1 {
let _ = self.do_send(None, false);
}
}
}
/*
*
* ===== impl Receiver =====
*
*/
impl<T> Receiver<T> {
/// Closes the receiving half
///
/// This prevents any further messages from being sent on the channel while
/// still enabling the receiver to drain messages that are buffered.
pub fn close(&mut self) {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
if !state.is_open {
break
}
state.is_open = false;
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => break,
Err(actual) => curr = actual,
}
}
// Wake up any threads waiting as they'll see that we've closed the
// channel and will continue on their merry way.
loop {
match unsafe { self.inner.parked_queue.pop() } {
PopResult::Data(task) => {
task.lock().unwrap().notify();
}
PopResult::Empty => break,
PopResult::Inconsistent => thread::yield_now(),
}
}
}
fn next_message(&mut self) -> Async<Option<T>> {
// Pop off a message
loop {
match unsafe { self.inner.message_queue.pop() } {
PopResult::Data(msg) => {
return Async::Ready(msg);
}
PopResult::Empty => {
// The queue is empty, return NotReady
return Async::NotReady;
}
PopResult::Inconsistent => {
// Inconsistent means that there will be a message to pop
// in a short time. This branch can only be reached if
// values are being produced from another thread, so there
// are a few ways that we can deal with this:
//
// 1) Spin
// 2) thread::yield_now()
// 3) task::current().unwrap() & return NotReady
//
// For now, thread::yield_now() is used, but it would
// probably be better to spin a few times then yield.
thread::yield_now();
}
}
}
}
// Unpark a single task handle if there is one pending in the parked queue
fn unpark_one(&mut self) {
loop {
match unsafe { self.inner.parked_queue.pop() } {
PopResult::Data(task) => {
task.lock().unwrap().notify();
return;
}
PopResult::Empty => {
// Queue empty, no task to wake up.
return;
}
PopResult::Inconsistent => {
// Same as above
thread::yield_now();
}
}
}
}
// Try to park the receiver task
fn try_park(&self) -> TryPark {
let curr = self.inner.state.load(SeqCst);
let state = decode_state(curr);
// If the channel is closed, then there is no need to park.
if !state.is_open && state.num_messages == 0 {
return TryPark::Closed;
}
// First, track the task in the `recv_task` slot
let mut recv_task = self.inner.recv_task.lock().unwrap();
if recv_task.unparked {
// Consume the `unpark` signal without actually parking
recv_task.unparked = false;
return TryPark::NotEmpty;
}
recv_task.task = Some(task::current());
TryPark::Parked
}
fn dec_num_messages(&self) {
let mut curr = self.inner.state.load(SeqCst);
loop {
let mut state = decode_state(curr);
state.num_messages -= 1;
let next = encode_state(&state);
match self.inner.state.compare_exchange(curr, next, SeqCst, SeqCst) {
Ok(_) => break,
Err(actual) => curr = actual,
}
}
}
}
impl<T> Stream for Receiver<T> {
type Item = T;
type Error = ();
fn poll(&mut self) -> Poll<Option<T>, ()> {
loop {
// Try to read a message off of the message queue.
let msg = match self.next_message() {
Async::Ready(msg) => msg,
Async::NotReady => {
// There are no messages to read, in this case, attempt to
// park. The act of parking will verify that the channel is
// still empty after the park operation has completed.
match self.try_park() {
TryPark::Parked => {
// The task was parked, and the channel is still
// empty, return NotReady.
return Ok(Async::NotReady);
}
TryPark::Closed => {
// The channel is closed, there will be no further
// messages.
return Ok(Async::Ready(None));
}
TryPark::NotEmpty => {
// A message has been sent while attempting to
// park. Loop again, the next iteration is
// guaranteed to get the message.
continue;
}
}
}
};
// If there are any parked task handles in the parked queue, pop
// one and unpark it.
self.unpark_one();
// Decrement number of messages
self.dec_num_messages();
// Return the message
return Ok(Async::Ready(msg));
}
}
}
impl<T> Drop for Receiver<T> {
fn drop(&mut self) {
// Drain the channel of all pending messages
self.close();
while self.next_message().is_ready() {
// ...
}
}
}
/*
*
* ===== impl Inner =====
*
*/
impl<T> Inner<T> {
// The return value is such that the total number of messages that can be
// enqueued into the channel will never exceed MAX_CAPACITY
fn max_senders(&self) -> usize {
match self.buffer {
Some(buffer) => MAX_CAPACITY - buffer,
None => MAX_BUFFER,
}
}
}
unsafe impl<T: Send> Send for Inner<T> {}
unsafe impl<T: Send> Sync for Inner<T> {}
/*
*
* ===== Helpers =====
*
*/
fn decode_state(num: usize) -> State {
State {
is_open: num & OPEN_MASK == OPEN_MASK,
num_messages: num & MAX_CAPACITY,
}
}
fn encode_state(state: &State) -> usize {
let mut num = state.num_messages;
if state.is_open {
num |= OPEN_MASK;
}
num
}
-151
View File
@@ -1,151 +0,0 @@
/* Copyright (c) 2010-2011 Dmitry Vyukov. All rights reserved.
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* 1. Redistributions of source code must retain the above copyright notice,
* this list of conditions and the following disclaimer.
*
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY DMITRY VYUKOV "AS IS" AND ANY EXPRESS OR IMPLIED
* WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
* MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT
* SHALL DMITRY VYUKOV OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
* INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
* OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
* ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are
* those of the authors and should not be interpreted as representing official
* policies, either expressed or implied, of Dmitry Vyukov.
*/
//! A mostly lock-free multi-producer, single consumer queue.
//!
//! This module contains an implementation of a concurrent MPSC queue. This
//! queue can be used to share data between threads, and is also used as the
//! building block of channels in rust.
//!
//! Note that the current implementation of this queue has a caveat of the `pop`
//! method, and see the method for more information about it. Due to this
//! caveat, this queue may not be appropriate for all use-cases.
// http://www.1024cores.net/home/lock-free-algorithms
// /queues/non-intrusive-mpsc-node-based-queue
// NOTE: this implementation is lifted from the standard library and only
// slightly modified
pub use self::PopResult::*;
use std::prelude::v1::*;
use std::cell::UnsafeCell;
use std::ptr;
use std::sync::atomic::{AtomicPtr, Ordering};
/// A result of the `pop` function.
pub enum PopResult<T> {
/// Some data has been popped
Data(T),
/// The queue is empty
Empty,
/// The queue is in an inconsistent state. Popping data should succeed, but
/// some pushers have yet to make enough progress in order allow a pop to
/// succeed. It is recommended that a pop() occur "in the near future" in
/// order to see if the sender has made progress or not
Inconsistent,
}
#[derive(Debug)]
struct Node<T> {
next: AtomicPtr<Node<T>>,
value: Option<T>,
}
/// The multi-producer single-consumer structure. This is not cloneable, but it
/// may be safely shared so long as it is guaranteed that there is only one
/// popper at a time (many pushers are allowed).
#[derive(Debug)]
pub struct Queue<T> {
head: AtomicPtr<Node<T>>,
tail: UnsafeCell<*mut Node<T>>,
}
unsafe impl<T: Send> Send for Queue<T> { }
unsafe impl<T: Send> Sync for Queue<T> { }
impl<T> Node<T> {
unsafe fn new(v: Option<T>) -> *mut Node<T> {
Box::into_raw(Box::new(Node {
next: AtomicPtr::new(ptr::null_mut()),
value: v,
}))
}
}
impl<T> Queue<T> {
/// Creates a new queue that is safe to share among multiple producers and
/// one consumer.
pub fn new() -> Queue<T> {
let stub = unsafe { Node::new(None) };
Queue {
head: AtomicPtr::new(stub),
tail: UnsafeCell::new(stub),
}
}
/// Pushes a new value onto this queue.
pub fn push(&self, t: T) {
unsafe {
let n = Node::new(Some(t));
let prev = self.head.swap(n, Ordering::AcqRel);
(*prev).next.store(n, Ordering::Release);
}
}
/// Pops some data from this queue.
///
/// Note that the current implementation means that this function cannot
/// return `Option<T>`. It is possible for this queue to be in an
/// inconsistent state where many pushes have succeeded and completely
/// finished, but pops cannot return `Some(t)`. This inconsistent state
/// happens when a pusher is preempted at an inopportune moment.
///
/// This inconsistent state means that this queue does indeed have data, but
/// it does not currently have access to it at this time.
///
/// This function is unsafe because only one thread can call it at a time.
pub unsafe fn pop(&self) -> PopResult<T> {
let tail = *self.tail.get();
let next = (*tail).next.load(Ordering::Acquire);
if !next.is_null() {
*self.tail.get() = next;
assert!((*tail).value.is_none());
assert!((*next).value.is_some());
let ret = (*next).value.take().unwrap();
drop(Box::from_raw(tail));
return Data(ret);
}
if self.head.load(Ordering::Acquire) == tail {Empty} else {Inconsistent}
}
}
impl<T> Drop for Queue<T> {
fn drop(&mut self) {
unsafe {
let mut cur = *self.tail.get();
while !cur.is_null() {
let next = (*cur).next.load(Ordering::Relaxed);
drop(Box::from_raw(cur));
cur = next;
}
}
}
}
-426
View File
@@ -1,426 +0,0 @@
//! A one-shot, futures-aware channel
use lock::Lock;
use futures::{Future, Poll, Async};
use futures::task::{self, Task};
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::sync::atomic::Ordering::SeqCst;
use std::error::Error;
use std::fmt;
/// A future representing the completion of a computation happening elsewhere in
/// memory.
///
/// This is created by the `oneshot::channel` function.
#[must_use = "futures do nothing unless polled"]
#[derive(Debug)]
pub struct Receiver<T> {
inner: Arc<Inner<T>>,
}
/// Represents the completion half of a oneshot through which the result of a
/// computation is signaled.
///
/// This is created by the `oneshot::channel` function.
#[derive(Debug)]
pub struct Sender<T> {
inner: Arc<Inner<T>>,
}
/// Internal state of the `Receiver`/`Sender` pair above. This is all used as
/// the internal synchronization between the two for send/recv operations.
#[derive(Debug)]
struct Inner<T> {
/// Indicates whether this oneshot is complete yet. This is filled in both
/// by `Sender::drop` and by `Receiver::drop`, and both sides interpret it
/// appropriately.
///
/// For `Receiver`, if this is `true`, then it's guaranteed that `data` is
/// unlocked and ready to be inspected.
///
/// For `Sender` if this is `true` then the oneshot has gone away and it
/// can return ready from `poll_cancel`.
complete: AtomicBool,
/// The actual data being transferred as part of this `Receiver`. This is
/// filled in by `Sender::complete` and read by `Receiver::poll`.
///
/// Note that this is protected by `Lock`, but it is in theory safe to
/// replace with an `UnsafeCell` as it's actually protected by `complete`
/// above. I wouldn't recommend doing this, however, unless someone is
/// supremely confident in the various atomic orderings here and there.
data: Lock<Option<T>>,
/// Field to store the task which is blocked in `Receiver::poll`.
///
/// This is filled in when a oneshot is polled but not ready yet. Note that
/// the `Lock` here, unlike in `data` above, is important to resolve races.
/// Both the `Receiver` and the `Sender` halves understand that if they
/// can't acquire the lock then some important interference is happening.
rx_task: Lock<Option<Task>>,
/// Like `rx_task` above, except for the task blocked in
/// `Sender::poll_cancel`. Additionally, `Lock` cannot be `UnsafeCell`.
tx_task: Lock<Option<Task>>,
}
/// Creates a new futures-aware, one-shot channel.
///
/// This function is similar to Rust's channels found in the standard library.
/// Two halves are returned, the first of which is a `Sender` handle, used to
/// signal the end of a computation and provide its value. The second half is a
/// `Receiver` which implements the `Future` trait, resolving to the value that
/// was given to the `Sender` handle.
///
/// Each half can be separately owned and sent across threads/tasks.
///
/// # Examples
///
/// ```
/// extern crate tokio_channel;
/// extern crate futures;
///
/// use tokio_channel::oneshot;
/// use futures::*;
/// use std::thread;
///
/// # fn main() {
/// let (p, c) = oneshot::channel::<i32>();
///
/// thread::spawn(|| {
/// c.map(|i| {
/// println!("got: {}", i);
/// }).wait();
/// });
///
/// p.send(3).unwrap();
/// # }
/// ```
pub fn channel<T>() -> (Sender<T>, Receiver<T>) {
let inner = Arc::new(Inner::new());
let receiver = Receiver {
inner: inner.clone(),
};
let sender = Sender {
inner: inner,
};
(sender, receiver)
}
impl<T> Inner<T> {
fn new() -> Inner<T> {
Inner {
complete: AtomicBool::new(false),
data: Lock::new(None),
rx_task: Lock::new(None),
tx_task: Lock::new(None),
}
}
fn send(&self, t: T) -> Result<(), T> {
if self.complete.load(SeqCst) {
return Err(t)
}
// Note that this lock acquisition may fail if the receiver
// is closed and sets the `complete` flag to true, whereupon
// the receiver may call `poll()`.
if let Some(mut slot) = self.data.try_lock() {
assert!(slot.is_none());
*slot = Some(t);
drop(slot);
// If the receiver called `close()` between the check at the
// start of the function, and the lock being released, then
// the receiver may not be around to receive it, so try to
// pull it back out.
if self.complete.load(SeqCst) {
// If lock acquisition fails, then receiver is actually
// receiving it, so we're good.
if let Some(mut slot) = self.data.try_lock() {
if let Some(t) = slot.take() {
return Err(t);
}
}
}
Ok(())
} else {
// Must have been closed
Err(t)
}
}
fn poll_cancel(&self) -> Poll<(), ()> {
// Fast path up first, just read the flag and see if our other half is
// gone. This flag is set both in our destructor and the oneshot
// destructor, but our destructor hasn't run yet so if it's set then the
// oneshot is gone.
if self.complete.load(SeqCst) {
return Ok(Async::Ready(()))
}
// If our other half is not gone then we need to park our current task
// and move it into the `notify_cancel` slot to get notified when it's
// actually gone.
//
// If `try_lock` fails, then the `Receiver` is in the process of using
// it, so we can deduce that it's now in the process of going away and
// hence we're canceled. If it succeeds then we just store our handle.
//
// Crucially we then check `oneshot_gone` *again* before we return.
// While we were storing our handle inside `notify_cancel` the `Receiver`
// may have been dropped. The first thing it does is set the flag, and
// if it fails to acquire the lock it assumes that we'll see the flag
// later on. So... we then try to see the flag later on!
let handle = task::current();
match self.tx_task.try_lock() {
Some(mut p) => *p = Some(handle),
None => return Ok(Async::Ready(())),
}
if self.complete.load(SeqCst) {
Ok(Async::Ready(()))
} else {
Ok(Async::NotReady)
}
}
fn is_canceled(&self) -> bool {
self.complete.load(SeqCst)
}
fn drop_tx(&self) {
// Flag that we're a completed `Sender` and try to wake up a receiver.
// Whether or not we actually stored any data will get picked up and
// translated to either an item or cancellation.
//
// Note that if we fail to acquire the `rx_task` lock then that means
// we're in one of two situations:
//
// 1. The receiver is trying to block in `poll`
// 2. The receiver is being dropped
//
// In the first case it'll check the `complete` flag after it's done
// blocking to see if it succeeded. In the latter case we don't need to
// wake up anyone anyway. So in both cases it's ok to ignore the `None`
// case of `try_lock` and bail out.
//
// The first case crucially depends on `Lock` using `SeqCst` ordering
// under the hood. If it instead used `Release` / `Acquire` ordering,
// then it would not necessarily synchronize with `inner.complete`
// and deadlock might be possible, as was observed in
// https://github.com/rust-lang-nursery/futures-rs/pull/219.
self.complete.store(true, SeqCst);
if let Some(mut slot) = self.rx_task.try_lock() {
if let Some(task) = slot.take() {
drop(slot);
task.notify();
}
}
}
fn close_rx(&self) {
// Flag our completion and then attempt to wake up the sender if it's
// blocked. See comments in `drop` below for more info
self.complete.store(true, SeqCst);
if let Some(mut handle) = self.tx_task.try_lock() {
if let Some(task) = handle.take() {
drop(handle);
task.notify()
}
}
}
fn recv(&self) -> Poll<T, Canceled> {
let mut done = false;
// Check to see if some data has arrived. If it hasn't then we need to
// block our task.
//
// Note that the acquisition of the `rx_task` lock might fail below, but
// the only situation where this can happen is during `Sender::drop`
// when we are indeed completed already. If that's happening then we
// know we're completed so keep going.
if self.complete.load(SeqCst) {
done = true;
} else {
let task = task::current();
match self.rx_task.try_lock() {
Some(mut slot) => *slot = Some(task),
None => done = true,
}
}
// If we're `done` via one of the paths above, then look at the data and
// figure out what the answer is. If, however, we stored `rx_task`
// successfully above we need to check again if we're completed in case
// a message was sent while `rx_task` was locked and couldn't notify us
// otherwise.
//
// If we're not done, and we're not complete, though, then we've
// successfully blocked our task and we return `NotReady`.
if done || self.complete.load(SeqCst) {
// If taking the lock fails, the sender will realise that the we're
// `done` when it checks the `complete` flag on the way out, and will
// treat the send as a failure.
if let Some(mut slot) = self.data.try_lock() {
if let Some(data) = slot.take() {
return Ok(data.into());
}
}
Err(Canceled)
} else {
Ok(Async::NotReady)
}
}
fn drop_rx(&self) {
// Indicate to the `Sender` that we're done, so any future calls to
// `poll_cancel` are weeded out.
self.complete.store(true, SeqCst);
// If we've blocked a task then there's no need for it to stick around,
// so we need to drop it. If this lock acquisition fails, though, then
// it's just because our `Sender` is trying to take the task, so we
// let them take care of that.
if let Some(mut slot) = self.rx_task.try_lock() {
let task = slot.take();
drop(slot);
drop(task);
}
// Finally, if our `Sender` wants to get notified of us going away, it
// would have stored something in `tx_task`. Here we try to peel that
// out and unpark it.
//
// Note that the `try_lock` here may fail, but only if the `Sender` is
// in the process of filling in the task. If that happens then we
// already flagged `complete` and they'll pick that up above.
if let Some(mut handle) = self.tx_task.try_lock() {
if let Some(task) = handle.take() {
drop(handle);
task.notify()
}
}
}
}
impl<T> Sender<T> {
#[deprecated(note = "renamed to `send`", since = "0.1.11")]
#[doc(hidden)]
#[cfg(feature = "with-deprecated")]
pub fn complete(self, t: T) {
drop(self.send(t));
}
/// Completes this oneshot with a successful result.
///
/// This function will consume `self` and indicate to the other end, the
/// `Receiver`, that the value provided is the result of the computation this
/// represents.
///
/// If the value is successfully enqueued for the remote end to receive,
/// then `Ok(())` is returned. If the receiving end was deallocated before
/// this function was called, however, then `Err` is returned with the value
/// provided.
pub fn send(self, t: T) -> Result<(), T> {
self.inner.send(t)
}
/// Polls this `Sender` half to detect whether the `Receiver` this has
/// paired with has gone away.
///
/// This function can be used to learn about when the `Receiver` (consumer)
/// half has gone away and nothing will be able to receive a message sent
/// from `send`.
///
/// If `Ready` is returned then it means that the `Receiver` has disappeared
/// and the result this `Sender` would otherwise produce should no longer
/// be produced.
///
/// If `NotReady` is returned then the `Receiver` is still alive and may be
/// able to receive a message if sent. The current task, however, is
/// scheduled to receive a notification if the corresponding `Receiver` goes
/// away.
///
/// # Panics
///
/// Like `Future::poll`, this function will panic if it's not called from
/// within the context of a task. In other words, this should only ever be
/// called from inside another future.
///
/// If you're calling this function from a context that does not have a
/// task, then you can use the `is_canceled` API instead.
pub fn poll_cancel(&mut self) -> Poll<(), ()> {
self.inner.poll_cancel()
}
/// Tests to see whether this `Sender`'s corresponding `Receiver`
/// has gone away.
///
/// This function can be used to learn about when the `Receiver` (consumer)
/// half has gone away and nothing will be able to receive a message sent
/// from `send`.
///
/// Note that this function is intended to *not* be used in the context of a
/// future. If you're implementing a future you probably want to call the
/// `poll_cancel` function which will block the current task if the
/// cancellation hasn't happened yet. This can be useful when working on a
/// non-futures related thread, though, which would otherwise panic if
/// `poll_cancel` were called.
pub fn is_canceled(&self) -> bool {
self.inner.is_canceled()
}
}
impl<T> Drop for Sender<T> {
fn drop(&mut self) {
self.inner.drop_tx()
}
}
/// Error returned from a `Receiver<T>` whenever the corresponding `Sender<T>`
/// is dropped.
#[derive(Clone, Copy, PartialEq, Eq, Debug)]
pub struct Canceled;
impl fmt::Display for Canceled {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "oneshot canceled")
}
}
impl Error for Canceled {
fn description(&self) -> &str {
"oneshot canceled"
}
}
impl<T> Receiver<T> {
/// Gracefully close this receiver, preventing sending any future messages.
///
/// Any `send` operation which happens after this method returns is
/// guaranteed to fail. Once this method is called the normal `poll` method
/// can be used to determine whether a message was actually sent or not. If
/// `Canceled` is returned from `poll` then no message was sent.
pub fn close(&mut self) {
self.inner.close_rx()
}
}
impl<T> Future for Receiver<T> {
type Item = T;
type Error = Canceled;
fn poll(&mut self) -> Poll<T, Canceled> {
self.inner.recv()
}
}
impl<T> Drop for Receiver<T> {
fn drop(&mut self) {
self.inner.drop_rx()
}
}
-22
View File
@@ -1,22 +0,0 @@
extern crate tokio_channel;
extern crate futures;
use tokio_channel::mpsc::*;
use futures::prelude::*;
use std::thread;
#[test]
fn smoke() {
let (mut sender, receiver) = channel(1);
let t = thread::spawn(move ||{
while let Ok(s) = sender.send(42).wait() {
sender = s;
}
});
receiver.take(3).for_each(|_| Ok(())).wait().unwrap();
t.join().unwrap()
}
-481
View File
@@ -1,481 +0,0 @@
extern crate tokio_channel;
#[macro_use]
extern crate futures;
mod support;
use support::*;
use tokio_channel::mpsc;
use tokio_channel::oneshot;
use futures::prelude::*;
use futures::future::lazy;
use std::thread;
use std::sync::{Arc, Mutex};
use std::sync::atomic::{AtomicUsize, Ordering};
trait AssertSend: Send {}
impl AssertSend for mpsc::Sender<i32> {}
impl AssertSend for mpsc::Receiver<i32> {}
#[test]
fn send_recv() {
let (tx, rx) = mpsc::channel::<i32>(16);
let mut rx = rx.wait();
tx.send(1).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(1));
}
#[test]
fn send_recv_no_buffer() {
let (mut tx, mut rx) = mpsc::channel::<i32>(0);
// Run on a task context
lazy(move || {
assert!(tx.poll_complete().unwrap().is_ready());
assert!(tx.poll_ready().unwrap().is_ready());
// Send first message
let res = tx.start_send(1).unwrap();
assert!(is_ready(&res));
assert!(tx.poll_ready().unwrap().is_not_ready());
// Send second message
let res = tx.start_send(2).unwrap();
assert!(!is_ready(&res));
// Take the value
assert_eq!(rx.poll().unwrap(), Async::Ready(Some(1)));
assert!(tx.poll_ready().unwrap().is_ready());
let res = tx.start_send(2).unwrap();
assert!(is_ready(&res));
assert!(tx.poll_ready().unwrap().is_not_ready());
// Take the value
assert_eq!(rx.poll().unwrap(), Async::Ready(Some(2)));
assert!(tx.poll_ready().unwrap().is_ready());
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn send_shared_recv() {
let (tx1, rx) = mpsc::channel::<i32>(16);
let tx2 = tx1.clone();
let mut rx = rx.wait();
tx1.send(1).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(1));
tx2.send(2).wait().unwrap();
assert_eq!(rx.next().unwrap(), Ok(2));
}
#[test]
fn send_recv_threads() {
let (tx, rx) = mpsc::channel::<i32>(16);
let mut rx = rx.wait();
thread::spawn(move|| {
tx.send(1).wait().unwrap();
});
assert_eq!(rx.next().unwrap(), Ok(1));
}
#[test]
fn send_recv_threads_no_capacity() {
let (tx, rx) = mpsc::channel::<i32>(0);
let mut rx = rx.wait();
let (readytx, readyrx) = mpsc::channel::<()>(2);
let mut readyrx = readyrx.wait();
let t = thread::spawn(move|| {
let readytx = readytx.sink_map_err(|_| panic!());
let (a, b) = tx.send(1).join(readytx.send(())).wait().unwrap();
a.send(2).join(b.send(())).wait().unwrap();
});
drop(readyrx.next().unwrap());
assert_eq!(rx.next().unwrap(), Ok(1));
drop(readyrx.next().unwrap());
assert_eq!(rx.next().unwrap(), Ok(2));
t.join().unwrap();
}
#[test]
fn recv_close_gets_none() {
let (mut tx, mut rx) = mpsc::channel::<i32>(10);
// Run on a task context
lazy(move || {
rx.close();
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
assert!(tx.poll_ready().is_err());
drop(tx);
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn tx_close_gets_none() {
let (_, mut rx) = mpsc::channel::<i32>(10);
// Run on a task context
lazy(move || {
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
assert_eq!(rx.poll(), Ok(Async::Ready(None)));
Ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn stress_shared_unbounded() {
const AMT: u32 = 10000;
const NTHREADS: u32 = 8;
let (tx, rx) = mpsc::unbounded::<i32>();
let mut rx = rx.wait();
let t = thread::spawn(move|| {
for _ in 0..AMT * NTHREADS {
assert_eq!(rx.next().unwrap(), Ok(1));
}
if rx.next().is_some() {
panic!();
}
});
for _ in 0..NTHREADS {
let tx = tx.clone();
thread::spawn(move|| {
for _ in 0..AMT {
tx.unbounded_send(1).unwrap();
}
});
}
drop(tx);
t.join().ok().unwrap();
}
#[test]
fn stress_shared_bounded_hard() {
const AMT: u32 = 10000;
const NTHREADS: u32 = 8;
let (tx, rx) = mpsc::channel::<i32>(0);
let mut rx = rx.wait();
let t = thread::spawn(move|| {
for _ in 0..AMT * NTHREADS {
assert_eq!(rx.next().unwrap(), Ok(1));
}
if rx.next().is_some() {
panic!();
}
});
for _ in 0..NTHREADS {
let mut tx = tx.clone();
thread::spawn(move|| {
for _ in 0..AMT {
tx = tx.send(1).wait().unwrap();
}
});
}
drop(tx);
t.join().ok().unwrap();
}
#[test]
fn stress_receiver_multi_task_bounded_hard() {
const AMT: usize = 10_000;
const NTHREADS: u32 = 2;
let (mut tx, rx) = mpsc::channel::<usize>(0);
let rx = Arc::new(Mutex::new(Some(rx)));
let n = Arc::new(AtomicUsize::new(0));
let mut th = vec![];
for _ in 0..NTHREADS {
let rx = rx.clone();
let n = n.clone();
let t = thread::spawn(move || {
let mut i = 0;
loop {
i += 1;
let mut lock = rx.lock().ok().unwrap();
match lock.take() {
Some(mut rx) => {
if i % 5 == 0 {
let (item, rest) = rx.into_future().wait().ok().unwrap();
if item.is_none() {
break;
}
n.fetch_add(1, Ordering::Relaxed);
*lock = Some(rest);
} else {
// Just poll
let n = n.clone();
let r = lazy(move || {
let r = match rx.poll().unwrap() {
Async::Ready(Some(_)) => {
n.fetch_add(1, Ordering::Relaxed);
*lock = Some(rx);
false
}
Async::Ready(None) => {
true
}
Async::NotReady => {
*lock = Some(rx);
false
}
};
Ok::<bool, ()>(r)
}).wait().unwrap();
if r {
break;
}
}
}
None => break,
}
}
});
th.push(t);
}
for i in 0..AMT {
tx = tx.send(i).wait().unwrap();
}
drop(tx);
for t in th {
t.join().unwrap();
}
assert_eq!(AMT, n.load(Ordering::Relaxed));
}
/// Stress test that receiver properly receives all the messages
/// after sender dropped.
#[test]
fn stress_drop_sender() {
fn list() -> Box<Stream<Item=i32, Error=u32>> {
let (tx, rx) = mpsc::channel(1);
tx.send(Ok(1))
.and_then(|tx| tx.send(Ok(2)))
.and_then(|tx| tx.send(Ok(3)))
.forget();
Box::new(rx.then(|r| r.unwrap()))
}
for _ in 0..10000 {
assert_eq!(list().wait().collect::<Result<Vec<_>, _>>(),
Ok(vec![1, 2, 3]));
}
}
/// Stress test that after receiver dropped,
/// no messages are lost.
fn stress_close_receiver_iter() {
let (tx, rx) = mpsc::unbounded();
let (unwritten_tx, unwritten_rx) = std::sync::mpsc::channel();
let th = thread::spawn(move || {
for i in 1.. {
if let Err(_) = tx.unbounded_send(i) {
unwritten_tx.send(i).expect("unwritten_tx");
return;
}
}
});
let mut rx = rx.wait();
// Read one message to make sure thread effectively started
assert_eq!(Some(Ok(1)), rx.next());
rx.get_mut().close();
for i in 2.. {
match rx.next() {
Some(Ok(r)) => assert!(i == r),
Some(Err(_)) => unreachable!(),
None => {
let unwritten = unwritten_rx.recv().expect("unwritten_rx");
assert_eq!(unwritten, i);
th.join().unwrap();
return;
}
}
}
}
#[test]
fn stress_close_receiver() {
for _ in 0..10000 {
stress_close_receiver_iter();
}
}
/// Tests that after `poll_ready` indicates capacity a channel can always send without waiting.
#[test]
fn stress_poll_ready() {
// A task which checks channel capacity using poll_ready, and pushes items onto the channel when
// ready.
struct SenderTask {
sender: mpsc::Sender<u32>,
count: u32,
}
impl Future for SenderTask {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
// In a loop, check if the channel is ready. If so, push an item onto the channel
// (asserting that it doesn't attempt to block).
while self.count > 0 {
try_ready!(self.sender.poll_ready().map_err(|_| ()));
assert!(self.sender.start_send(self.count).unwrap().is_ready());
self.count -= 1;
}
Ok(Async::Ready(()))
}
}
const AMT: u32 = 1000;
const NTHREADS: u32 = 8;
/// Run a stress test using the specified channel capacity.
fn stress(capacity: usize) {
let (tx, rx) = mpsc::channel(capacity);
let mut threads = Vec::new();
for _ in 0..NTHREADS {
let sender = tx.clone();
threads.push(thread::spawn(move || {
SenderTask {
sender: sender,
count: AMT,
}.wait()
}));
}
drop(tx);
let mut rx = rx.wait();
for _ in 0..AMT * NTHREADS {
assert!(rx.next().is_some());
}
assert!(rx.next().is_none());
for thread in threads {
thread.join().unwrap().unwrap();
}
}
stress(0);
stress(1);
stress(8);
stress(16);
}
fn is_ready<T>(res: &AsyncSink<T>) -> bool {
match *res {
AsyncSink::Ready => true,
_ => false,
}
}
#[test]
fn try_send_1() {
const N: usize = 3000;
let (mut tx, rx) = mpsc::channel(0);
let t = thread::spawn(move || {
for i in 0..N {
loop {
if tx.try_send(i).is_ok() {
break
}
}
}
});
for (i, j) in rx.wait().enumerate() {
assert_eq!(i, j.unwrap());
}
t.join().unwrap();
}
#[test]
fn try_send_2() {
let (mut tx, rx) = mpsc::channel(0);
tx.try_send("hello").unwrap();
let (readytx, readyrx) = oneshot::channel::<()>();
let th = thread::spawn(|| {
lazy(|| {
assert!(tx.start_send("fail").unwrap().is_not_ready());
Ok::<_, ()>(())
}).wait().unwrap();
drop(readytx);
tx.send("goodbye").wait().unwrap();
});
let mut rx = rx.wait();
drop(readyrx.wait());
assert_eq!(rx.next(), Some(Ok("hello")));
assert_eq!(rx.next(), Some(Ok("goodbye")));
assert!(rx.next().is_none());
th.join().unwrap();
}
#[test]
fn try_send_fail() {
let (mut tx, rx) = mpsc::channel(0);
let mut rx = rx.wait();
tx.try_send("hello").unwrap();
// This should fail
assert!(tx.try_send("fail").is_err());
assert_eq!(rx.next(), Some(Ok("hello")));
tx.try_send("goodbye").unwrap();
drop(tx);
assert_eq!(rx.next(), Some(Ok("goodbye")));
assert!(rx.next().is_none());
}
-124
View File
@@ -1,124 +0,0 @@
extern crate tokio_channel;
extern crate futures;
mod support;
use support::*;
use tokio_channel::oneshot::*;
use futures::prelude::*;
use futures::future::{lazy, ok};
use std::sync::mpsc;
use std::thread;
#[test]
fn smoke_poll() {
let (mut tx, rx) = channel::<u32>();
lazy(|| {
assert!(tx.poll_cancel().unwrap().is_not_ready());
assert!(tx.poll_cancel().unwrap().is_not_ready());
drop(rx);
assert!(tx.poll_cancel().unwrap().is_ready());
assert!(tx.poll_cancel().unwrap().is_ready());
ok::<(), ()>(())
}).wait().unwrap();
}
#[test]
fn cancel_notifies() {
let (tx, rx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
WaitForCancel { tx: tx }.then(move |v| tx2.send(v)).forget();
drop(rx);
rx2.recv().unwrap().unwrap();
}
struct WaitForCancel {
tx: Sender<u32>,
}
impl Future for WaitForCancel {
type Item = ();
type Error = ();
fn poll(&mut self) -> Poll<(), ()> {
self.tx.poll_cancel()
}
}
#[test]
fn cancel_lots() {
let (tx, rx) = mpsc::channel::<(Sender<_>, mpsc::Sender<_>)>();
let t = thread::spawn(move || {
for (tx, tx2) in rx {
WaitForCancel { tx: tx }.then(move |v| tx2.send(v)).forget();
}
});
for _ in 0..20000 {
let (otx, orx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
tx.send((otx, tx2)).unwrap();
drop(orx);
rx2.recv().unwrap().unwrap();
}
drop(tx);
t.join().unwrap();
}
#[test]
fn close() {
let (mut tx, mut rx) = channel::<u32>();
rx.close();
assert!(rx.poll().is_err());
assert!(tx.poll_cancel().unwrap().is_ready());
}
#[test]
fn close_wakes() {
let (tx, mut rx) = channel::<u32>();
let (tx2, rx2) = mpsc::channel();
let t = thread::spawn(move || {
rx.close();
rx2.recv().unwrap();
});
WaitForCancel { tx: tx }.wait().unwrap();
tx2.send(()).unwrap();
t.join().unwrap();
}
#[test]
fn is_canceled() {
let (tx, rx) = channel::<u32>();
assert!(!tx.is_canceled());
drop(rx);
assert!(tx.is_canceled());
}
#[test]
fn cancel_sends() {
let (tx, rx) = mpsc::channel::<Sender<_>>();
let t = thread::spawn(move || {
for otx in rx {
let _ = otx.send(42);
}
});
for _ in 0..20000 {
let (otx, mut orx) = channel::<u32>();
tx.send(otx).unwrap();
orx.close();
// Not necessary to wrap in a task because the implementation of oneshot
// never calls `task::current()` if the channel has been closed already.
let _ = orx.poll();
}
drop(tx);
t.join().unwrap();
}
-16
View File
@@ -1,16 +0,0 @@
use futures::Future;
pub trait ForgetExt {
fn forget(self);
}
impl<F> ForgetExt for F
where F: Future + Sized + Send + 'static,
F::Item: Send,
F::Error: Send
{
fn forget(self) {
use std::thread;
thread::spawn(|| self.wait());
}
}
+5 -3
View File
@@ -1,9 +1,11 @@
[package]
name = "tokio-codec"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc URL.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.1"
@@ -18,6 +20,6 @@ Utilities for encoding and decoding frames.
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
bytes = "0.4.7"
futures = "0.1.18"
+5 -3
View File
@@ -1,6 +1,6 @@
use bytes::{Bytes, BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use bytes::{BufMut, Bytes, BytesMut};
use std::io;
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -8,7 +8,9 @@ pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec { BytesCodec(()) }
pub fn new() -> BytesCodec {
BytesCodec(())
}
}
impl Decoder for BytesCodec {
+1 -8
View File
@@ -19,14 +19,7 @@ extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use tokio_io::_tokio_codec::{Decoder, Encoder, Framed, FramedParts, FramedRead, FramedWrite};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
+4 -6
View File
@@ -1,6 +1,6 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{cmp, io, str, usize};
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -103,10 +103,8 @@ impl LinesCodec {
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
str::from_utf8(buf)
.map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
@@ -153,7 +151,7 @@ impl Decoder for LinesCodec {
self.is_discarding = true;
Err(io::Error::new(
io::ErrorKind::Other,
"line length limit exceeded"
"line length limit exceeded",
))
} else {
// We didn't find a line or reach the length limit, so the next
+33 -9
View File
@@ -1,8 +1,8 @@
extern crate tokio_codec;
extern crate bytes;
extern crate tokio_codec;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
use bytes::{BufMut, Bytes, BytesMut};
use tokio_codec::{BytesCodec, Decoder, Encoder, LinesCodec};
#[test]
fn bytes_decoder() {
@@ -27,13 +27,17 @@ fn bytes_encoder() {
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf)
.unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf)
.unwrap();
}
#[test]
@@ -68,17 +72,32 @@ fn lines_decoder_max_length() {
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 2", line);
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 4", line);
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("", line);
assert_eq!(None, codec.decode(buf).unwrap());
@@ -87,7 +106,12 @@ fn lines_decoder_max_length() {
assert_eq!(None, codec.decode(buf).unwrap());
let line = codec.decode_eof(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("\rk", line);
assert_eq!(None, codec.decode(buf).unwrap());
+10 -10
View File
@@ -1,13 +1,13 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use futures::{Stream, Future};
use bytes::{Buf, BufMut, BytesMut, IntoBuf};
use futures::{Future, Stream};
use std::io::{self, Read};
use tokio_codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_codec::{Decoder, Encoder, Framed, FramedParts};
use tokio_io::AsyncRead;
use bytes::{BytesMut, Buf, BufMut, IntoBuf};
const INITIAL_CAPACITY: usize = 8 * 1024;
@@ -45,8 +45,10 @@ struct DontReadIntoThis;
impl Read for DontReadIntoThis {
fn read(&mut self, _: &mut [u8]) -> io::Result<usize> {
Err(io::Error::new(io::ErrorKind::Other,
"Read into something you weren't supposed to."))
Err(io::Error::new(
io::ErrorKind::Other,
"Read into something you weren't supposed to.",
))
}
}
@@ -61,9 +63,7 @@ fn can_read_from_existing_buf() {
let num = framed
.into_future()
.map(|(first_num, _)| {
first_num.unwrap()
})
.map(|(first_num, _)| first_num.unwrap())
.wait()
.map_err(|e| e.0)
.unwrap();
+7 -8
View File
@@ -1,17 +1,17 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, FramedRead};
use tokio_io::AsyncRead;
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf};
use bytes::{Buf, BytesMut, IntoBuf};
use futures::Async::{NotReady, Ready};
use futures::Stream;
use futures::Async::{Ready, NotReady};
use std::io::{self, Read};
use std::collections::VecDeque;
use std::io::{self, Read};
macro_rules! mock {
($($x:expr,)*) => {{
@@ -212,5 +212,4 @@ impl Read for Mock {
}
}
impl AsyncRead for Mock {
}
impl AsyncRead for Mock {}
+6 -6
View File
@@ -1,16 +1,16 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_io::AsyncWrite;
use tokio_codec::{Encoder, FramedWrite};
use tokio_io::AsyncWrite;
use futures::{Sink, Poll};
use bytes::{BytesMut, BufMut};
use bytes::{BufMut, BytesMut};
use futures::{Poll, Sink};
use std::io::{self, Write};
use std::collections::VecDeque;
use std::io::{self, Write};
macro_rules! mock {
($($x:expr,)*) => {{
+10
View File
@@ -1,3 +1,13 @@
# 0.1.6 (March 22, 2019)
### Added
- implement `TypedExecutor` (#993).
# 0.1.5 (March 1, 2019)
### Fixed
- Documentation typos (#882).
# 0.1.4 (November 21, 2018)
* Fix shutdown on idle (#763).
+7 -5
View File
@@ -1,13 +1,15 @@
[package]
name = "tokio-current-thread"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update doc URL
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.4"
documentation = "https://docs.rs/tokio-current-thread/0.1.4/tokio_current_thread"
version = "0.1.6"
documentation = "https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -19,5 +21,5 @@ keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
tokio-executor = { version = "0.1.5", path = "../tokio-executor" }
tokio-executor = "0.1.7"
futures = "0.1.19"
+1 -1
View File
@@ -2,7 +2,7 @@
Single threaded executor for Tokio.
[Documentation](https://tokio-rs.github.io/tokio/tokio_current_thread/)
[Documentation](https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread/)
## Overview
+102 -93
View File
@@ -1,4 +1,4 @@
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.4")]
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.6")]
#![deny(warnings, missing_docs, missing_debug_implementations)]
//! A single-threaded executor which executes tasks on the same thread from which
@@ -32,19 +32,19 @@ mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::park::{Park, ParkThread, Unpark};
use tokio_executor::{Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::future::{ExecuteError, ExecuteErrorKind, Executor};
use futures::{executor, Async, Future};
use futures::future::{Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::rc::Rc;
use std::sync::{atomic, mpsc, Arc};
use std::time::{Duration, Instant};
use std::thread;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
@@ -86,7 +86,7 @@ pub struct TaskExecutor {
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
polled: bool,
}
impl Turn {
@@ -194,17 +194,21 @@ struct CurrentRunner {
id: Cell<Option<u64>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
});
thread_local! {
/// Current thread's task runner. This is set in `TaskRunner::with`
static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
}
}
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one referred
/// to by a `Handle` so that direct task dispatch can be used.
thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
thread_local! {
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one
/// referred to by a `Handle` so that direct task dispatch can be used.
static EXECUTOR_ID: Cell<u64> = Cell::new(0)
}
/// Run the executor bootstrapping the execution with the provided future.
///
@@ -222,7 +226,8 @@ thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
where
F: Future,
{
let mut current_thread = CurrentThread::new();
@@ -246,7 +251,8 @@ where F: Future,
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
TaskExecutor::current()
.spawn_local(Box::new(future))
@@ -312,7 +318,8 @@ impl<P: Park> CurrentThread<P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future), false);
self
@@ -331,41 +338,33 @@ impl<P: Park> CurrentThread<P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).turn(duration)
}
@@ -432,11 +431,24 @@ impl tokio_executor::Executor for CurrentThread {
}
}
impl<T> tokio_executor::TypedExecutor<T> for CurrentThread
where
T: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
self.borrow().spawn_local(Box::new(future), false);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures.load(atomic::Ordering::SeqCst))
.field(
"num_futures",
&self.num_futures.load(atomic::Ordering::SeqCst),
)
.finish()
}
}
@@ -448,7 +460,8 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future), false);
self
@@ -467,17 +480,18 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
let res = self
.executor
.borrow()
.enter(self.enter, || future.poll_future_notify(&notify, 0));
match res {
Ok(Async::Ready(e)) => return Ok(e),
@@ -496,24 +510,19 @@ impl<'a, P: Park> Entered<'a, P> {
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
self.run_timeout2(None).map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
@@ -542,9 +551,7 @@ impl<'a, P: Park> Entered<'a, P> {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
fn run_timeout2(&mut self, dur: Option<Duration>) -> Result<(), RunTimeoutError> {
if self.executor.is_idle() {
// Nothing to do
return Ok(());
@@ -602,10 +609,9 @@ impl<'a, P: Park> Entered<'a, P> {
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
borrow.id,
&mut *self.enter,
borrow.num_futures)
borrow
.scheduler
.tick(borrow.id, &mut *self.enter, borrow.num_futures)
}
}
@@ -676,7 +682,8 @@ impl Handle {
return Err(SpawnError::shutdown());
}
self.sender.send(Box::new(future))
self.sender
.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
@@ -718,51 +725,53 @@ impl TaskExecutor {
/// Get the current executor's thread-local ID.
fn id(&self) -> Option<u64> {
CURRENT.with(|current| {
current.id.get()
})
CURRENT.with(|current| current.id.get())
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
pub fn spawn_local(
&mut self,
future: Box<Future<Item = (), Error = ()>>,
) -> Result<(), SpawnError> {
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => Err(SpawnError::shutdown()),
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
self.spawn_local(future)
}
}
impl<F> tokio_executor::TypedExecutor<F> for TaskExecutor
where
F: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: F) -> Result<(), SpawnError> {
self.spawn_local(Box::new(future))
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future)),
})
}
}
@@ -771,13 +780,12 @@ where F: Future<Item = (), Error = ()> + 'static
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
where
F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.id.set(Some(self.id));
current.set_spawn(self, || {
f()
})
current.set_spawn(self, || f())
})
}
}
@@ -797,7 +805,8 @@ impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
where
F: FnOnce() -> R,
{
struct Reset<'a>(&'a CurrentRunner);
+17 -21
View File
@@ -1,20 +1,20 @@
use super::Borrow;
use tokio_executor::Enter;
use tokio_executor::park::Unpark;
use tokio_executor::Enter;
use futures::{Future, Async};
use futures::executor::{self, Spawn, UnsafeNotify, NotifyHandle};
use futures::executor::{self, NotifyHandle, Spawn, UnsafeNotify};
use futures::{Async, Future};
use std::cell::UnsafeCell;
use std::fmt::{self, Debug};
use std::marker::PhantomData;
use std::mem;
use std::ptr;
use std::sync::atomic::Ordering::{Relaxed, SeqCst, Acquire, Release, AcqRel};
use std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release, SeqCst};
use std::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize};
use std::sync::{Arc, Weak};
use std::usize;
use std::thread;
use std::marker::PhantomData;
use std::usize;
/// A generic task-aware scheduler.
///
@@ -135,7 +135,8 @@ pub struct Scheduled<'a, U: 'a> {
}
impl<U> Scheduler<U>
where U: Unpark,
where
U: Unpark,
{
/// Constructs a new, empty `Scheduler`
///
@@ -200,9 +201,7 @@ where U: Unpark,
pub fn has_pending_futures(&mut self) -> bool {
// See function definition for why the unsafe is needed and
// correctly used here
unsafe {
self.inner.has_pending_futures()
}
unsafe { self.inner.has_pending_futures() }
}
/// Advance the scheduler state, returning `true` if any futures were
@@ -210,11 +209,9 @@ where U: Unpark,
///
/// This function should be called whenever the caller is notified via a
/// wakeup.
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool
{
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool {
let mut ret = false;
let tick = self.inner.tick_num.fetch_add(1, SeqCst)
.wrapping_add(1);
let tick = self.inner.tick_num.fetch_add(1, SeqCst).wrapping_add(1);
loop {
let node = match unsafe { self.inner.dequeue(Some(tick)) } {
@@ -246,7 +243,7 @@ where U: Unpark,
let node = ptr2arc(node);
assert!((*node.next_all.get()).is_null());
assert!((*node.prev_all.get()).is_null());
continue
continue;
};
// We're going to need to be very careful if the `poll`
@@ -369,8 +366,7 @@ impl Task {
impl fmt::Debug for Task {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Task")
.finish()
fmt.debug_struct("Task").finish()
}
}
@@ -559,7 +555,7 @@ impl<U> List<U> {
}
}
/// Prepends an element to the back of the list
/// Appends an element to the back of the list
fn push_back(&mut self, node: Arc<Node<U>>) -> *const Node<U> {
let ptr = arc2ptr(node);
@@ -580,7 +576,7 @@ impl<U> List<U> {
self.len += 1;
return ptr
return ptr;
}
/// Pop an element from the front of the list
@@ -632,7 +628,7 @@ impl<U> List<U> {
self.len -= 1;
return node
return node;
}
}
@@ -749,7 +745,7 @@ impl<U> Drop for Node<U> {
fn arc2ptr<T>(ptr: Arc<T>) -> *const T {
let addr = &*ptr as *const T;
mem::forget(ptr);
return addr
return addr;
}
unsafe fn ptr2arc<T>(ptr: *const T) -> Arc<T> {
+155 -120
View File
@@ -1,6 +1,6 @@
extern crate futures;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate futures;
use tokio_current_thread::{block_on_all, CurrentThread};
@@ -10,8 +10,8 @@ use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::task;
// This is not actually unused --- we need this trait to be in scope for
// the tests that sue TaskExecutor::current().execute(). The compiler
// doesn't realise that.
@@ -22,7 +22,7 @@ use futures::sync::oneshot;
mod from_block_on_all {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()>>) + 'static>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()>>) + 'static>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -36,7 +36,8 @@ mod from_block_on_all {
})));
Ok::<_, ()>("hello")
})).unwrap();
}))
.unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
@@ -72,7 +73,8 @@ fn block_waits() {
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
}))
.unwrap();
assert_eq!(1, cnt2.get());
}
@@ -100,11 +102,14 @@ fn spawn_many() {
mod does_not_set_global_executor_by_default {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()> + Send>) -> Result<(), E> + 'static, E>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()> + Send>) -> Result<(), E> + 'static, E>(
spawn: F,
) {
block_on_all(lazy(|| {
spawn(Box::new(lazy(|| ok()))).unwrap_err();
ok()
})).unwrap()
}))
.unwrap()
}
#[test]
@@ -123,20 +128,22 @@ mod from_block_on_future {
use super::*;
fn test<F: Fn(Box<Future<Item = (), Error = ()>>)>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let cnt = Rc::new(Cell::new(0));
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
tokio_current_thread
.block_on(lazy(|| {
let cnt = cnt.clone();
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
Ok::<_, ()>(())
})).unwrap();
Ok::<_, ()>(())
}))
.unwrap();
tokio_current_thread.run().unwrap();
@@ -150,7 +157,11 @@ mod from_block_on_future {
#[test]
fn execute() {
test(|f| { tokio_current_thread::TaskExecutor::current().execute(f).unwrap(); });
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
});
}
}
@@ -170,8 +181,8 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut rc = Rc::new(());
@@ -189,10 +200,12 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
})).unwrap();
tokio_current_thread
.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
}))
.unwrap();
drop(tokio_current_thread);
@@ -202,12 +215,15 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -216,7 +232,9 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
@@ -225,12 +243,11 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
mod run_in_future {
@@ -241,13 +258,12 @@ mod run_in_future {
fn spawn() {
block_on_all(lazy(|| {
tokio_current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}));
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -256,18 +272,16 @@ mod run_in_future {
block_on_all(lazy(|| {
tokio_current_thread::TaskExecutor::current()
.execute(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}))
.unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
@@ -288,9 +302,7 @@ fn tick_on_infini_future() {
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.spawn(Infini { num: num.clone() })
.turn(None)
.unwrap();
@@ -347,7 +359,8 @@ mod tasks_are_scheduled_fairly {
});
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -359,8 +372,8 @@ mod tasks_are_scheduled_fairly {
fn execute() {
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
.execute(f)
.unwrap();
})
}
}
@@ -370,8 +383,8 @@ mod and_turn {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -379,24 +392,25 @@ mod and_turn {
let mut tokio_current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
Ok(())
})));
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || Ok(()))));
// Turn once...
tokio_current_thread.turn(None).unwrap();
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
c.set(1 + c.get());
// Spawn!
spawn(Box::new(lazy(move || {
dotspawn(
&mut tokio_current_thread,
Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})));
// Spawn!
spawn(Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})),
);
// This does not run the newly spawned thread
tokio_current_thread.turn(None).unwrap();
@@ -409,12 +423,15 @@ mod and_turn {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -423,11 +440,12 @@ mod and_turn {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
mod in_drop {
@@ -455,23 +473,24 @@ mod in_drop {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut tokio_current_thread = CurrentThread::new();
let mut tokio_current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
dotspawn(&mut tokio_current_thread, Box::new(
MyFuture {
dotspawn(
&mut tokio_current_thread,
Box::new(MyFuture {
_data: Box::new(OnDrop(Some(move || {
spawn(Box::new(lazy(move || {
tx.send(()).unwrap();
Ok(())
})));
}))),
}
));
}),
);
tokio_current_thread.block_on(rx).unwrap();
tokio_current_thread.run().unwrap();
@@ -479,12 +498,15 @@ mod in_drop {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -493,7 +515,9 @@ mod in_drop {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
@@ -562,13 +586,17 @@ fn turn_has_polled() {
tokio_current_thread.spawn(receiver.then(|_| Ok(())));
// Turn once...
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver once, but considered it not ready
assert!(res.has_polled());
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled nothing, the receiver is not ready yet
assert!(!res.has_polled());
@@ -577,14 +605,18 @@ fn turn_has_polled() {
sender.send(()).unwrap();
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver, it's ready now
assert!(res.has_polled());
// Now the executor should be empty
assert!(tokio_current_thread.is_idle());
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// So should've polled nothing
assert!(!res.has_polled());
@@ -646,46 +678,41 @@ fn turn_fair() {
// Once an item is received on the oneshot channel, it will immediately
// immediately make the second oneshot channel ready
tokio_current_thread.spawn(receiver
.map_err(|_| unreachable!())
.and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
tokio_current_thread.spawn(receiver.map_err(|_| unreachable!()).and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
Ok(())
})
);
Ok(())
}));
let receiver_2_done = Rc::new(Cell::new(false));
let receiver_2_done_clone = receiver_2_done.clone();
tokio_current_thread.spawn(receiver_2
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_2.map_err(|_| unreachable!()).and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
}));
// The third receiver is only woken up from our Park implementation, it simulates
// e.g. a socket that first has to be polled to know if it is ready now
let receiver_3_done = Rc::new(Cell::new(false));
let receiver_3_done_clone = receiver_3_done.clone();
tokio_current_thread.spawn(receiver_3
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_3.map_err(|_| unreachable!()).and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
}));
// First turn should've polled both and considered them not ready
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(res.has_polled());
// Next turn should've polled nothing
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(!res.has_polled());
assert!(!receiver_1_done.get());
@@ -736,10 +763,12 @@ fn spawn_from_other_thread() {
let (sender, receiver) = oneshot::channel::<()>();
thread::spawn(move || {
handle.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
}))
.unwrap();
});
let _ = current_thread.block_on(receiver).unwrap();
@@ -758,10 +787,12 @@ fn spawn_from_other_thread_unpark() {
thread::spawn(move || {
let _ = receiver_2.recv().unwrap();
handle.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
}))
.unwrap();
});
// Ensure that unparking the executor works correctly. It will first
@@ -769,13 +800,15 @@ fn spawn_from_other_thread_unpark() {
// lazy future below which will cause the future to be spawned from
// the other thread. Then the executor will park but should be woken
// up because *now* we have a new future to schedule
let _ = current_thread.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1)
).unwrap();
let _ = current_thread
.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1),
)
.unwrap();
}
#[test]
@@ -785,10 +818,12 @@ fn spawn_from_executor_with_handle() {
let (tx, rx) = oneshot::channel();
current_thread.spawn(lazy(move || {
handle.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}))
.unwrap();
Ok::<_, ()>(())
}));
+5
View File
@@ -1,3 +1,8 @@
# 0.1.7 (March 22, 2019)
### Added
- `TypedExecutor` for spawning futures of a specific type (#993).
# 0.1.6 (January 6, 2019)
* Implement `Unpark` for `Arc<Unpark>` (#802).
+8 -5
View File
@@ -1,15 +1,15 @@
[package]
name = "tokio-executor"
# When releasing to crates.io:
# - Remove path dependencies
# - Update html_root_url.
# - Update CHANGELOG.md.
# - Update doc URL.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.6"
documentation = "https://docs.rs/tokio-executor/0.1.6/tokio_executor"
version = "0.1.7"
documentation = "https://docs.rs/tokio-executor/0.1.7/tokio_executor"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -23,3 +23,6 @@ categories = ["concurrency", "asynchronous"]
[dependencies]
crossbeam-utils = "0.6.2"
futures = "0.1.19"
[dev-dependencies]
tokio = "0.1.18"
+5 -5
View File
@@ -2,7 +2,7 @@
Task execution related traits and utilities.
[Documentation](https://docs.rs/tokio-executor/0.1.6/tokio_executor)
[Documentation](https://docs.rs/tokio-executor/0.1.7/tokio_executor)
## Overview
@@ -31,10 +31,10 @@ executor, including:
* [`Park`] abstracts over blocking and unblocking the current thread.
[`Executor`]: https://tokio-rs.github.io/tokio/tokio_executor/trait.Executor.html
[`enter`]: https://tokio-rs.github.io/tokio/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://tokio-rs.github.io/tokio/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://tokio-rs.github.io/tokio/tokio_executor/park/index.html
[`Executor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/trait.Executor.html
[`enter`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/park/trait.Park.html
## License
+6 -4
View File
@@ -1,7 +1,7 @@
use std::prelude::v1::*;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::prelude::v1::*;
use futures::{self, Future};
@@ -70,7 +70,10 @@ pub fn enter() -> Result<Enter, EnterError> {
impl Enter {
/// Register a callback to be invoked if and when the thread
/// ceased to act as an executor.
pub fn on_exit<F>(&mut self, f: F) where F: FnOnce() + 'static {
pub fn on_exit<F>(&mut self, f: F)
where
F: FnOnce() + 'static,
{
self.on_exit.push(Box::new(f));
}
@@ -88,7 +91,6 @@ impl Enter {
pub fn block_on<F: Future>(&mut self, f: F) -> Result<F::Item, F::Error> {
futures::executor::spawn(f).wait_future()
}
}
impl fmt::Debug for Enter {
@@ -103,7 +105,7 @@ impl Drop for Enter {
assert!(c.get());
if self.permanent {
return
return;
}
for callback in self.on_exit.drain(..) {
+50
View File
@@ -0,0 +1,50 @@
use std::error::Error;
use std::fmt;
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
+151
View File
@@ -0,0 +1,151 @@
use futures::Future;
use SpawnError;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// This trait is primarily intended to implemented by executors and used to
/// back `tokio::spawn`. Libraries and applications **may** use this trait to
/// bound generics, but doing so will limit usage to futures that implement
/// `Send`. Instead, libraries and applications are recommended to use
/// [`TypedExecutor`] as a bound.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
/// [`TypedExecutor`]: ../trait.TypedExecutor.html
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
+41 -24
View File
@@ -1,4 +1,4 @@
use super::{Executor, Enter, SpawnError};
use super::{Enter, Executor, SpawnError};
use futures::{future, Future};
@@ -33,24 +33,22 @@ impl DefaultExecutor {
/// `DefaultExecutor::current()` on thread A and then sending the result to
/// thread B will _not_ reference the default executor that was set on thread A.
pub fn current() -> DefaultExecutor {
DefaultExecutor {
_dummy: (),
}
DefaultExecutor { _dummy: () }
}
#[inline]
fn with_current<F: FnOnce(&mut Executor) -> R, R>(f: F) -> Option<R> {
EXECUTOR.with(|current_executor| {
match current_executor.replace(State::Active) {
EXECUTOR.with(
|current_executor| match current_executor.replace(State::Active) {
State::Ready(executor_ptr) => {
let executor = unsafe { &mut *executor_ptr };
let result = f(executor);
current_executor.set(State::Ready(executor_ptr));
Some(result)
},
}
State::Empty | State::Active => None,
}
})
},
)
}
}
@@ -61,18 +59,21 @@ enum State {
// default executor is defined and ready to be used
Ready(*mut Executor),
// default executor is currently active (used to detect recursive calls)
Active
Active,
}
/// Thread-local tracking the current executor
thread_local!(static EXECUTOR: Cell<State> = Cell::new(State::Empty));
thread_local! {
/// Thread-local tracking the current executor
static EXECUTOR: Cell<State> = Cell::new(State::Empty)
}
// ===== impl DefaultExecutor =====
impl super::Executor for DefaultExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
DefaultExecutor::with_current(|executor| executor.spawn(future))
.unwrap_or_else(|| Err(SpawnError::shutdown()))
}
@@ -83,8 +84,22 @@ impl super::Executor for DefaultExecutor {
}
}
impl<T> super::TypedExecutor<T> for DefaultExecutor
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
super::Executor::spawn(self, Box::new(future))
}
fn status(&self) -> Result<(), SpawnError> {
super::Executor::status(self)
}
}
impl<T> future::Executor<T> for DefaultExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
if let Err(e) = super::Executor::status(self) {
@@ -144,10 +159,10 @@ where T: Future<Item = (), Error = ()> + Send + 'static,
/// # pub fn main() {}
/// ```
pub fn spawn<T>(future: T)
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
DefaultExecutor::current().spawn(Box::new(future))
.unwrap()
DefaultExecutor::current().spawn(Box::new(future)).unwrap()
}
/// Set the default executor for the duration of the closure
@@ -156,13 +171,15 @@ pub fn spawn<T>(future: T)
///
/// This function panics if there already is a default executor set.
pub fn with_default<T, F, R>(executor: &mut T, enter: &mut Enter, f: F) -> R
where T: Executor,
F: FnOnce(&mut Enter) -> R
where
T: Executor,
F: FnOnce(&mut Enter) -> R,
{
EXECUTOR.with(|cell| {
match cell.get() {
State::Ready(_) | State::Active =>
panic!("default executor already set for execution context"),
State::Ready(_) | State::Active => {
panic!("default executor already set for execution context")
}
_ => {}
}
@@ -200,7 +217,7 @@ unsafe fn hide_lt<'a>(p: *mut (Executor + 'a)) -> *mut (Executor + 'static) {
#[cfg(test)]
mod tests {
use super::{Executor, DefaultExecutor, with_default};
use super::{with_default, DefaultExecutor, Executor};
#[test]
fn default_executor_is_send_and_sync() {
+25 -195
View File
@@ -1,5 +1,5 @@
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.6")]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.7")]
//! Task execution related traits and utilities.
//!
@@ -17,8 +17,12 @@
//! This crate provides traits and utilities that are necessary for building an
//! executor, including:
//!
//! * The [`Executor`] trait describes the API for spawning a future onto an
//! executor.
//! * The [`Executor`] trait spawns future object onto an executor.
//!
//! * The [`TypedExecutor`] trait spawns futures of a specific type onto an
//! executor. This is used to be generic over executors that spawn futures
//! that are either `Send` or `!Send` or implement executors that apply to
//! specific futures.
//!
//! * [`enter`] marks that the current thread is entering an execution
//! context. This prevents a second executor from accidentally starting from
@@ -29,7 +33,19 @@
//!
//! * [`Park`] abstracts over blocking and unblocking the current thread.
//!
//! # Implementing an executor
//!
//! Executors should always implement `TypedExecutor`. This usually is the bound
//! that applications and libraries will use when generic over an executor. See
//! the [trait documentation][`TypedExecutor`] for more details.
//!
//! If the executor is able to spawn all futures that are `Send`, then the
//! executor should also implement the `Executor` trait. This trait is rarely
//! used directly by applications and libraries. Instead, `tokio::spawn` is
//! configured to dispatch to type that implements `Executor`.
//!
//! [`Executor`]: trait.Executor.html
//! [`TypedExecutor`]: trait.TypedExecutor.html
//! [`enter`]: fn.enter.html
//! [`DefaultExecutor`]: struct.DefaultExecutor.html
//! [`Park`]: park/index.html
@@ -39,200 +55,14 @@ extern crate crossbeam_utils;
extern crate futures;
mod enter;
mod error;
mod executor;
mod global;
pub mod park;
mod typed;
pub use enter::{enter, Enter, EnterError};
pub use error::SpawnError;
pub use executor::Executor;
pub use global::{spawn, with_default, DefaultExecutor};
use futures::Future;
use std::error::Error;
use std::fmt;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementers are encouraged to avoid panics. However, a panic is
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
pub use typed::TypedExecutor;
+2 -1
View File
@@ -190,7 +190,8 @@ impl ParkThread {
/// Get a reference to the `ParkThread` handle for this thread.
fn with_current<F, R>(&self, f: F) -> R
where F: FnOnce(&Parker) -> R,
where
F: FnOnce(&Parker) -> R,
{
CURRENT_PARKER.with(|inner| f(inner))
}
+181
View File
@@ -0,0 +1,181 @@
use SpawnError;
/// A value that spawns futures of a specific type.
///
/// The trait is generic over `T`: the type of future that can be spawened. This
/// is useful for implementing an executor that is only able to spawn a specific
/// type of future.
///
/// The [`spawn`] function is used to submit the future to the executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// This trait is useful as a bound for applications and libraries in order to
/// be generic over futures that are `Send` vs. `!Send`.
///
/// # Examples
///
/// Consider a function that provides an API for draining a `Stream` in the
/// background. To do this, a task must be spawned to perform the draining. As
/// such, the function takes a stream and an executor on which the background
/// task is spawned.
///
/// ```rust
/// #[macro_use]
/// extern crate futures;
/// extern crate tokio;
///
/// use futures::{Future, Stream, Poll};
/// use tokio::executor::TypedExecutor;
/// use tokio::sync::oneshot;
///
/// pub fn drain<T, E>(stream: T, executor: &mut E)
/// -> impl Future<Item = (), Error = ()>
/// where
/// T: Stream,
/// E: TypedExecutor<Drain<T>>
/// {
/// let (tx, rx) = oneshot::channel();
///
/// executor.spawn(Drain {
/// stream,
/// tx: Some(tx),
/// }).unwrap();
///
/// rx.map_err(|_| ())
/// }
///
/// // The background task
/// pub struct Drain<T: Stream> {
/// stream: T,
/// tx: Option<oneshot::Sender<()>>,
/// }
///
/// impl<T: Stream> Future for Drain<T> {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
/// loop {
/// let item = try_ready!(
/// self.stream.poll()
/// .map_err(|_| ())
/// );
///
/// if item.is_none() { break; }
/// }
///
/// self.tx.take().unwrap().send(()).map_err(|_| ());
/// Ok(().into())
/// }
/// }
/// # pub fn main() {}
/// ```
///
/// By doing this, the `drain` fn can accept a stream that is `!Send` as long as
/// the supplied executor is able to spawn `!Send` types.
pub trait TypedExecutor<T> {
/// Spawns a future to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// executor takes ownership of the future and becomes responsible for
/// driving the future to completion.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// my_executor.spawn(MyFuture).unwrap();
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: T) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// if my_executor.status().is_ok() {
/// my_executor.spawn(MyFuture).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E, T> TypedExecutor<T> for Box<E>
where
E: TypedExecutor<T>,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}

Some files were not shown because too many files have changed in this diff Show More