Compare commits

...
Author SHA1 Message Date
Carl Lerche 62f34e15ce Bump tokio to 0.1.19. (#1053)
This also bumps:

- tokio-async-await (0.1.7)
- tokio-buf (0.1.1)
- tokio-sync (0.1.5)
- tokio-threadpool (0.1.14)
2019-04-22 15:12:25 -07:00
Eliza Weisman 3ebca76a9a trace: prepare tokio-trace for release (#1051) 2019-04-22 14:15:07 -07:00
Ryan Dahl fea1f780bc threadpool: add panic_handler (#1052) 2019-04-21 16:26:09 -07:00
Eliza Weisman 712ca84033 trace-core: prepare for 0.2 release (#1047) 2019-04-21 10:20:28 -07:00
Jon Gjengset cf06621998 tokio-sync: Add async mutual exclusion primitive (#964)
This PR introduces `Lock`: A concurrency primitive built on top of `Semaphore` that provides a `Mutex`-like primitive that interacts nicely with futures. Specifically, `LockGuard` (in contrast to `MutexGuard`) does _not_ borrow the `Lock`, and can thus be passed into a future where it will later be unlocked.

This replaces #958, which attempted to introduce a less generic version. The primitive proposed there will instead live in [`async-lease`](https://github.com/jonhoo/async-lease).
2019-04-18 13:16:26 -04:00
Lucio Franco 7e51ab05e9 buf: Add IntoStream (#1048)
* buf: Add IntoStream

* Add debug implementation for IntoStream

* Add get_ref, get_mut and into_inner
2019-04-18 11:42:34 -04:00
Eliza Weisman 4bfa4ffcdf trace-core: Dispatchers unset themselves (#1033)
This branch changes `dispatcher::get_default` to unset the thread's
current dispatcher while the reference to it is held by the closure.
This prevents infinite loops if the subscriber calls code paths which
emit events or construct spans. 

Note that this also means that nested calls to `get_default` inside of a
`get_default` closure will receive a `None` dispatcher rather than the
"actual" dispatcher. However, it was necessary to unset the default in
`get_default` rather than in dispatch methods such as `Dispatch::enter`,
as when those functions are called, the current state has already been
borrowed.

Before:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          51 ns/iter (+/- 12)
test span_repeatedly         ... bench:       5,073 ns/iter (+/- 1,528)
test span_with_fields        ... bench:          56 ns/iter (+/- 49)
test span_with_fields_record ... bench:         363 ns/iter (+/- 61)
```

After:
```
test enter_span              ... bench:           3 ns/iter (+/- 0)
test span_no_fields          ... bench:          35 ns/iter (+/- 12)
test span_repeatedly         ... bench:       4,165 ns/iter (+/- 298)
test span_with_fields        ... bench:          48 ns/iter (+/- 12)
test span_with_fields_record ... bench:         363 ns/iter (+/- 91)
```

Closes #1032 

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-16 15:51:45 -07:00
Taiki Endo 88b942652c async-await: fix examples (#1050)
* Fix crate path in `Cargo.toml` of examples
* Add `edition2018` to examples in the documentation to make it compiled
  on Rust 2018
* Fix an example in the documentation
2019-04-16 14:19:38 -04:00
Taiki Endo 5029e80a89 async-await: update to new futures_api (#1049) 2019-04-16 14:07:03 -04:00
Eliza Weismanandcsmoe 847fb59b17 trace-core: Introduce callsite classification in metadata (#1046)
## Motivation

To ease the implementation of `Subscriber::register_callsite`, a field
should be added to `Metadata` to indicate if this callsite is an event or
a span.

## Solution

A new struct, `Kind`, is added to the `metadata` module in
`tokio-trace-core`, and a `Kind` field is added to the `Metadata`
struct. Macros which construct `metadata` now require a `Kind`.

`Kind` is represented as a struct with a private inner enum to allow new
`Kind`s to be added without breaking changes. However, the _addition_ of
the kind field _is_ a breaking change. While this could be done in a
backward-compatible way, it would permit the construction of metadata
with unknown kinds, and since the next `tokio-trace-core` release will
be a breaking change, I opted to make the breaking change instead.

New API tests for the `callsite!` and `metadata!` macros have been added
to guard against future API breakage.

Fixes: #986
Closes: #1008

Co-Authored-By: csmoe <[email protected]>
2019-04-11 11:56:42 -07:00
Jane Lusby b4fe517a16 trace-core: add a function to rebuild cached interest (#1039)
## Motivation

Currently, `tokio-trace-core` permits `Subscriber`s to indicate that
they are "always", "sometimes", or "never" interested in a particular
callsite. When "always" or "never" is returned, then the interest is
cached and the subscriber will not be asked again about that callsite.
This is much more efficient than requiring the filter to be re-evaluated
every time the callsite is hit.

However, if a subscriber wishes to change its filter configuration
dynamically at runtime, it cannot benefit from this caching. Instead, it
must always return `Interest::sometimes`.  Even when filters change very
infrequently, they must still always be re-evaluated every time.

In order to support a use-case where subscribers may change their filter
configuration at runtime (e.g. tokio-rs/tokio-trace-nursery#42),
but do so infrequently, we should introducing a new function to
invalidate the cached interest.

## Solution

This branch adds a new function in the `callsite` module, called
`rebuild_interest_cache`, that will invalidate and rebuild all cached
interest.

## Breaking Change

In order to fix a race condition that could occur when rebuilding
interest caches using `clear_interest` and `add_interest`, these methods
have been replaced by a new `set_interest` method. `set_interest` should
have the semantics of atomically replacing the previous cached interest,
so that the callsite does not enter a temporary state where it has no
interest.

Closes #1038

Co-Authored-By: yaahallo <[email protected]>
2019-04-10 13:51:05 -07:00
Simon Wollwage 7ae010f0f3 async-await: Use Context instead of Waker in poll (#1041)
Rust nightly std::future::Future recently changed Waker
to Context.

Change to use Context

Co-Authored-By: Kintaro <[email protected]>
2019-04-10 09:14:31 -07:00
Carl Lerche 9144b2ff53 sync: remove unnecessary imports (#1043) 2019-04-09 12:26:11 -07:00
Lev Eniseev 2c4549a18a Add example of blocking environment (#1036) 2019-04-09 12:10:15 -07:00
João Oliveira 4f819b7cd1 trace: fix counters example, Span IDs must be greater than zero (#1037)
## Motivation
tokio-trace counter example was panicking due to returning 0 as the first Span Id

## Solution
start ID's with 1 instead
2019-04-05 13:51:20 -07:00
Eliza Weisman 197f88f3bc trace: Change Span::enter and record to take &self (#1029)
## Motivation

The `Span::enter` function previously required an `&mut` reference to
enter a span. This is a relic of an earlier design where span closure
logic was determined by dropping an inner span component, and is no
longer strictly necessary.

Requiring `&mut self` to enter a span leads to awkward patterns in cases
when a user wishes to enter a span and then call methods on the span
(such as recording field values). For example, we cannot say
```rust
let mut span = span!("foo", bar);
span.enter(|| {
    span.record("bar" &false);
});
```
since the span is mutably borrowed by `enter`. Instead, we must clone
the span, like so:
```rust
let mut span = span!("foo", bar);
span.clone().enter(|| {
    span.record("bar" &false);
});
```

Having to clone the span is somewhat less ergonomic, and it has
performance disadvantages as well: cloning a `Span` will clone the
span's `Dispatch` handle, requiring an `Arc` bump, as well as calling
the `Subscriber`'s `clone_span` and `drop_span` functions. If we can
enter spans without a mutable borrow, we don't have to update any of
these ref counts.

The other reason we may wish to require mutable borrows to enter a span
is if we want to disallow entering a span multiple times before exiting
it. However, it is trivially possible to re-enter a span on the same
thread regardless, by cloning the span and entering it twice. Besides,
there may be a valuable semantic meaning in entering a span from inside
itself, such as when a function is called recursively, so disallowing
this is not a goal.

## Solution

This branch rewrites the `Span::enter`, `Span::record`, and
`Span::record_all` functions to no longer require mutable borrows. 

In the case of `record` and `record_all`, this was trivial, as borrowing
mutably was not actually *necessary* for those functions. For `enter`,
the `Entered` guard type was reworked to consist of an `&'a Inner`
rather than an `Inner`, so it is no longer necessary to `take` the
span's `Inner`. 

## Notes

In addition to allowing spans to be entered without mutable borrows,
`Entered` was changed to exit the span automatically when the guard is
dropped, so we may now observe correct span exits even when unwinding.

Furthermore, this allows us to simplify the `enter` function a bit,
leading to a minor performance improvement when entering spans.

Before:
```
test enter_span              ... bench:          13 ns/iter (+/- 1)
```

...and after:
```
test enter_span              ... bench:           3 ns/iter (+/- 1)
```

Note that this branch also contains a change to make the
`subscriber::enter_span` benchmark more accurate. Previously, this
benchmark constructed a new span inside of `b.iter(|| {...})`. This
means that the benchmark was measuring not only the time taken to enter
a span, but the time taken to construct a `Span` handle as well.
However, we already have benchmarks for span construction, and the
intention of this particular benchmark was to measure the overhead of
constructing a span.

I've updated the benchmark by moving the span construction out of the
`iter` closure. Now, the span is constructed a single time and entered
on every iteration. This allows us to measure only the overhead of
actually entering a span. The "before" benchmark numbers above were
recorded after backporting this change to master, so they are "fair" to
the previous implementation. Prior to this change the benchmark took
approximately 53 ns.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 15:06:47 -07:00
Eliza Weisman 44f65afcc6 trace: Allow field names to be separated by .s (#1027)
## Motivation

In order to support conventions that add namespacing to `tokio-trace`
field names, it's necessary to accept at least one type of separator
character. Currently, the `tokio-trace` macros only accept valid Rust
identifiers, so there is no clear separator character for namespaced
conventions. See also #1018.

## Solution

This branch changes the single `ident` fragment matcher for field names
to match *one or more* `ident` fragments separated by `.` characters.

## Notes

The resulting key is still exposed to `tokio-trace-core` as a string
constant created by stringifying the dotted expression. However, if
`tokio-trace-core` were later to adopt a first class notion of
hierarchical field keys, we would be able to track that change in
`tokio-trace` as an implementation detail.

Closes #1018.
Closes #1022.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-03 14:44:11 -07:00
Eliza Weisman 4271a9cd8d trace: Fix subscriber benchmarks panicking (#1028)
This branch fixes the `tokio-trace` Subscriber benchmarks panicking due
to constructing spans with ID 0. They will now use an arbitrary constant
instead.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-02 13:42:31 -07:00
Matthias Prechtl 9d8096b911 Improve documentation of Subscriber::record and Subscriber::event (#1026) 2019-04-02 12:44:01 -07:00
João Oliveira 597f271c08 trace: Remove default trace level and make levels mandatory on span! macro (#1025)
## Motivation 

Was determined that having the span! macro default to the TRACE level is
probably not ideal (see discussion on #952). 

Closes #1013

## Solution 

Remove default trace level and make log lvl mandatory on span! macro,
and add the respective `trace_span!`, `debug_span!`, `info_span!`,
`warn_span!` and `error_span!` macros that behave as span! macro, but
with defined log levels

## Notes 

I think this is it, also removed some captures that were repeated, and
some testcases that also seemed repeated after adding the mandatory log
level, but please review it, if more tests or examples are needed happy
to provide (tried to find a way to get the generated macros log level,
but didn't find one, if there is a way i can add tests to assert that
the generated macro has the matching log level ). thanks
2019-04-02 11:29:23 -07:00
Taiki Endo 599955f716 Replace try! macro with ? operator (#1024) 2019-04-01 13:45:59 -07:00
Ivan Petkov 91bb0f73f5 signal: refactor Windows registrations to be lazy (#1001)
- Use `Handle::default` over `Handle::current` for consistent semantics
- Make all `windows::Event` constructors lazily invoke `global_init`
  so they can be safely constructed off-task
- Don't assume the reactor is alive and event registration will be done
  when calling `global_init`

Add windows regression tests. Unfortunately, Windows doesn't have a
reliable way of programmatically sending CTRL_C or CTRL_BREAK events
to a progress, so the tests can only exercise our internal machinery by
invoking the handler that we register with the OS

Fixes #999
2019-04-01 12:46:22 -07:00
Eliza Weisman 6c9d8abba9 trace: Make Span API functions taking IDs a little more flexible (#1021)
This branch modifies the `tokio_trace::Span` API functions that take
span IDs (the `Span::child_of` constructor, and the `Span::follows_from`
method) so that more types bearing a span ID can be passed as an
argument. Span IDs may now be passed directly without requiring them to
be passed as `Some(id)`. This should make the API slightly more
ergonomic.

Also, it changes the `Span::field` method to take an `AsField` rather
than a `Borrow<str>`.

Signed-off-by: Eliza Weisman <[email protected]>
2019-04-01 11:42:29 -07:00
Carl Lerche 824b7b6759 buf: stream and iter helpers (#1011) 2019-03-29 12:26:13 -07:00
Carl Lerche cb91dd274a buf: impl Error for CollectVecError (#1010) 2019-03-29 08:49:08 -07:00
Eliza Weisman ea7178b8c6 trace-core: Add slightly more useful debug impls (#1014)
This branch improves the `fmt::Debug` implementation for `Metadata`,
and adds `fmt::Display` implementations for `FieldSet` and `ValueSet`.

When formatting a `Metadata`, only present fields are formatted --- if
optional fields, such as the file, line number, and module path are
`None`, they will be excluded. In addition, `Metadata` now formats its
`FieldSet` using `FieldSet`'s `fmt::Display` implementation, which is a
bit less noisy. Finally, the `Debug` output for `Metadata` now includes
the callsite that the metadata originates from.

The intention behind these changes is to make the output from failed
tests somewhat easier to interpret.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-28 14:47:06 -07:00
Carl Lerche a99b8e2e0b buf: impl size_hint for str types + reorg tests (#1012) 2019-03-28 14:02:45 -07:00
Carl Lerche 03859a7dcd buf: implement FromBufStream for Bytes (#1009) 2019-03-27 19:22:06 -07:00
Eliza Weisman d8177f81ac trace: Allow trace instrumentation to emit log records (#992)
## Motivation

`tokio-trace` currently offers a strategy for compatibility with the
`log` crate: its macros can be dropped in as a replacement for `log`'s
macros, and a subscriber can be used that translates trace events to log
records. However, this requires the application to be aware of
`tokio-trace` and manually set up this subscriber.

Many libraries currently emit `log` records, and would like to be able
to emit `tokio-trace` instrumentation instead. The `tokio` runtimes are
one such example. However, with the current log compatibility strategy,
replacing existing logging with trace instrumentation would break
`tokio`'s logs for any downstream user which is using only `log` and not
`tokio-trace`. It is desirable for libraries to have the option to emit
both `log` _and_ `tokio-trace` diagnostics from the same instrumentation
points.

## Solution

This branch adds a `log` feature flag to the `tokio-trace` crate, which
when set, causes `tokio-trace` instrumentation to emit log records as well
as `tokio-trace` instrumentation. 

## Notes

In order to allow spans to log their names when they are entered and 
exited even when the span is disabled, this branch adds an 
`&'static Metadata` to the `Span` type. This was previously stored in
the `Inner` type and was thus only present when the span was enabled.
This makes disabled spans one word longer, but enabled spans remain
the same size.

Fixes: #949

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-26 16:43:05 -07:00
Son ceca2a3cd6 chore: add license to tokio (#1006) 2019-03-26 08:41:41 -07:00
Son 1524ee4b60 trace: Add static level filtering (#987)
## Motivation

`tokio-trace` should have static verbosity level filtering, like the
`log` crate. The static max verbosity level should be controlled at
compile time with a set of features. It should be possible to set a
separate max level for release and debug mode builds.

## Solution

We can do this fairly similarly to how the `log` crate does it:
`tokio-trace` should export a constant whose value is set based on the
static max level feature flags. Then, we add an if statement to the
`span!` and `event!` macros which tests if that event or span's level
is enabled.

Closes #959
2019-03-25 15:19:20 -07:00
MOZGIII 7793d63739 Corrected doc for tokio_buf SizeHint (#1003) 2019-03-25 16:54:49 -04:00
Red Hara e0e26bc223 Fix typo in README.md in examples (#1002) 2019-03-24 13:02:11 -04:00
Eliza Weisman 9c5cad037f trace-core: Add overrideable downcasting to Subscribers (#974)
## Motivation

In order to implement "out of band" `Subscriber` APIs in third-party
subscriber implementations (see [this comment]) users may want to 
downcast the current `Dispatch` to a concrete subscriber type.

For example, in a library for integrating `tokio-trace` with a fancy new
(hypothetical) distributed tracing technology "ElizaTracing", which uses
256-bit span IDs, we might expect to see a function like this:
```rust

pub fn correlate(tt: tokio_trace::span::Id, et: elizatracing::SpanId) {
    tokio_trace::dispatcher::with(|c| {
        if let Some(s) = c.downcast_ref::<elizatracing::Subscriber>() {
            s.do_elizatracing_correlation_magic(tt, et);
        }
    }); 
}
```

This allows users to correlate `tokio-trace` IDs with IDs in the
distributed tracing system without having to pass a special handle to
the subscriber through application code (as one is already present in
thread-local storage, but with its type erased).

## Solution

This branch makes the following changes:
 * Add an object-safe `downcast_raw` method to the `Subscriber` trait,
   taking a `TypeId` and returning an `*const ()` if the type ID 
   matches the subscriber's type ID, or `None` if it does not, and
 * Add `is<T>` and `downcast_ref<T>` functions to `Subscriber` 
   and `Dispatch`, using `downcast_raw`.

Unlike the approach implemented in #950, the `downcast_raw` method is
object-safe, since it takes a `TypeId` rather than a type _parameter_ 
and returns a void pointer rather than an `&T`. This means that
`Subscriber` implementations can override this method if necessary. For
example, a `Subscriber` that fans out to multiple component subscribers
can downcast to their component parts, and "chained" or "middleware"
subscribers, which wrap an inner `Subscriber` and modify its behaviour 
somehow, can downcast to the inner type if they choose to.

[this comment]: https://github.com/tokio-rs/tokio/issues/932#issuecomment-469473501
[`std::error::Error`'s]: https://doc.rust-lang.org/1.33.0/src/std/error.rs.html#204

Refs: #950, #953, https://github.com/tokio-rs/tokio/issues/948#issuecomment-469444293

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 16:21:46 -07:00
Eliza Weisman 30330da11a chore: Fix examples not working with cargo run (#998)
* chore: Fix examples not working with `cargo run`

## Motivation

PR #991 moved the `tokio` crate to its own subdirectory, but did not
move the `examples` directory into `tokio/examples`. While attempting to
use the examples for testing another change, I noticed that #991 had
broken the ability to use `cargo run`, as the examples were no longer
considered part of a crate that cargo was aware of:

```
tokio on master [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
error: no example target named `chat`

Did you mean `echo`?
```

## Solution

This branch moves the examples into the `tokio` directory, so cargo is
now once again aware of them:

```
tokio on eliza/fix-examples [$] via 🦀v1.33.0 at ☸️ aks-eliza-dev
➜  cargo run --example chat
   Compiling tokio-executor v0.1.7 (/Users/eliza/Code/tokio/tokio-executor)
   Compiling tokio-reactor v0.1.9
   Compiling tokio-threadpool v0.1.13
   Compiling tokio-current-thread v0.1.6
   Compiling tokio-timer v0.2.10
   Compiling tokio-uds v0.2.5
   Compiling tokio-udp v0.1.3
   Compiling tokio-tcp v0.1.3
   Compiling tokio-fs v0.1.6
   Compiling tokio v0.1.18 (/Users/eliza/Code/tokio/tokio)
    Finished dev [unoptimized + debuginfo] target(s) in 7.04s
     Running `target/debug/examples/chat`
server running on localhost:6142
```

Signed-off-by: Eliza Weisman <[email protected]>

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-22 15:25:42 -07:00
Carl Lerche 6e4945025c chore: fix Cargo.toml files 2019-03-22 14:10:06 -07:00
Carl Lerche 3c8f110730 Bump Tokio version to v0.1.18 (#997)
Also bumps:

- tokio-signal (0.2.8)
- tokio-current-thread (0.1.6)
- tokio-executor (0.1.7)
- tokio-threadpool (0.1.13)

[ci-release]
2019-03-22 13:55:48 -07:00
Carl Lerche 678f15bd48 ci: skip crates.io dep run when releasing (#995)
#993 introduces changes in a sub crate that other Tokio crates depend
on. To make CI pass, a `[patch]` statement and `path` dependencies are
used.

When releasing, these must be removed. However, the commit that
removes them and prepares the crates for release will not be able to
pass CI.

This commit adds a conditional on a special `[ci-release]` snippet in
the commit message. If this exists, CI is only run with the full "patched"
dependencies.
2019-03-22 11:58:00 -07:00
Carl Lerche b1172f8074 executor: add TypedExecutor (#993)
Adds a `TypedExecutor` trait that describes how to spawn futures of a specific
type. This is useful for implementing functions that are generic over an executor
and wish to support both `Send` and `!Send` cases.
2019-03-21 14:30:18 -07:00
Carl Lerche cdde2e7a27 chore: repo maintenance + no path dependencies (#991)
- Move `tokio` into its own directory.
- Remove `path` dependencies.
- Run tests with once with crates.io dep and once with patched dep.
2019-03-19 14:58:59 -07:00
Eliza Weisman 85487727d4 trace: Span API polish (#988)
This branch makes the following changes to `tokio-trace`'s `Span` type:

* **Remove manual close API from spans**
  In practice, there wasn't really a use-case for this, and it 
  complicates the implementation a bit. We can always add it back later.

* **Remove generic lifetime from `Span`**
  Again, there wasn't actually a use-case for spans with metadata that
  doesn't live for the static lifetime, and it made using `Span`s in 
  other types somewhat inconvenient. It's also possible to implement an
  alternative API for non-static spans on top of the `tokio-trace-core`
  primitives.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-18 12:44:46 -07:00
Lucio Franco 92d51202ef trace: Remove git dep on trace core for crates version (#984) 2019-03-13 15:07:47 -04:00
Lucio Franco cb55bf4012 signal: Fix deprecated use of Handle::current (#981) 2019-03-13 11:56:25 -07:00
Carl Lerche 987ccfc8ac Bump Tokio to v0.1.17 (#983)
Also bumps:
- tokio-sync (v0.1.4)
2019-03-13 11:19:22 -07:00
Sean McArthur 1bc6d75543 sync: add mpsc benchmarks of small, medium, and large message types (#982) 2019-03-13 11:00:42 -07:00
Sean McArthur 27148d6110 sync: free chan Blocks when Chan is dropped (#978) 2019-03-13 10:38:14 -07:00
Carl Lerche a1871b1480 Prepare tokio-trace-core for initial release. (#979) 2019-03-13 10:29:27 -07:00
Eliza Weisman acd08eb23d tokio: Enable trace subscriber propagation in the runtime (#966)
Signed-off-by: Eliza Weisman <[email protected]>
2019-03-13 10:28:45 -07:00
南浦月 90b1a01010 tokio: fix dependency versions (#944)
#943
2019-03-13 07:47:05 -07:00
Thomas Lacroix 676824988e sync: impl Error for oneshot and watch error types (#967)
Refs: #937
2019-03-12 08:51:23 -07:00
Eliza Weisman 46149f031e trace-core: Fix NoSubscriber causing panics (#975)
PR #973 changed the `tokio_trace_core::span::Id::from_u64` function to
require that the provided `u64` be greater than zero. However, I had
forgotten that the implementation of `Subscriber` for the `NoSubscriber`
type (which is used when no default subscriber is set) always returned
`span::Id::from_u64(0)` from its `new_span` method. In combination with
the assert added in #973, this means that every time a span is hit when
no subscriber is set, `tokio-trace-core` will panic.

This branch fixes the panics by having `NoSubscriber` construct span IDs
using a different (arbitrarily chosen) non-zero constant.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 17:08:04 -07:00
Eliza Weisman 5510ba6dba trace-core: Require span IDs to be > 0 (#973)
This branch changes `tokio_trace_core::span::Id::from_u64` to assert
that the integer from which the span ID is constructed is greater than
zero. This is to enable future use of non-zero optimization.

Unfortunately, we can't actually use a `NonZeroU64` _now_, as that type
was only stabilized in Rust 1.28.0, and `tokio`'s current minimum
supported Rust version is 1.26.0.

Adding and documenting the assertion now allows us to change the
internal representation to `NonZeroU64` later (when 1.28.0 is the
minimum supported Rust version), without causing a breaking change.

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 16:18:40 -07:00
Eliza Weisman b8f63308d7 trace-core: Pass dispatcher by ref to dispatcher::with_default (#971)
* trace-core: Pass dispatcher by ref to `dispatcher::with_default`

As requested by @carllerche in https://github.com/tokio-rs/tokio/pull/966#discussion_r264380005, this branch changes the
`dispatcher::with_default` function in `tokio-trace-core` to take the
dispatcher by ref and perform the clone internally. This makes this
function more consistant with other `with_default` functions in other
crates.

Signed-off-by: Eliza Weisman <[email protected]>

* trace: Don't set the default dispatcher on entering a span

Setting the default dispatcher on span entry is a relic of when spans
tracked their parent's ID. At that time, it was necessary to ensure that
any spans created inside a span were observed by the same subscriber
that originally provided the entered span with an ID, as otherwise, new
spans would be created with parent IDs that did not originate from that
subscriber.

Now that spans don't track their parent ID, this is no longer necessary.
However, removing this behavior does mean that if a span is entered
outside of the subscriber context it was created in, any subsequent
spans will be observed by the current default subscriber and thus will
not be part of the original span's trace tree. Since subscribers are not
expected to change frequently, and spans are not expected to move
between them, this is likely acceptable.

I've removed the tests for the old behavior.

Note that this change improves the performance of span entry/exit fairly
significantly. Here are the results of running a benchmark that enters
a span, does nothing, and immediately exits it, before this change:

```
test enter_span              ... bench:          93 ns/iter (+/- 14)
```

...and after:

```
test enter_span              ... bench:          51 ns/iter (+/- 9)
```

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 15:29:00 -07:00
Eliza Weisman 4313d65b38 trace: Switch to using local_inner_macros for instrumentation API (#969)
## Motivation

Currently, it isn't possible to import individual macros from
`tokio-trace` using the macros 1.2 syntax:

```rust
use tokio_trace::{debug, info, span};
```

This is because these macros require that `callsite` and `enabled` are
imported as well.

## Solution

This branch resolves the problem by adding the [`local_inner_macros`]
attribute to the instrumentation API's macros. This allows other macros
from within the crate to be used without requiring them to be explicitly
imported. 

However, this also requires duplicating any macros from other sources
(such as std and `tokio-trace-core`) with wrappers due to the behaviour
of `local_inner_macros`. I've added these wrapper macros as well.

Since the macros got even longer as a result of this, I've moved them
to a separate file to make `lib.rs` easier to read. I've also wrapped
some very long lines in the macros, and removed the explicit drop of
the result of evaluating some event macros (it's no longer necessary
as all event macros now evaluate to `()`).

[`local_inner_macros`]: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/macros/macro-changes.html#local-helper-macros

Fixes #968

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-11 14:48:23 -07:00
Eliza Weisman e780fccce4 trace: Minor documentation improvements (#963) 2019-03-07 21:36:15 -08:00
Eliza Weisman b01e71b3d8 trace-core: API polish (#962)
This branch makes a handful of `tokio-trace-core` API improvements, mostly
around naming. In particular:

 * Rename `dispatcher::with` to `dispatcher::get_default`
 * Rename `Event::observe` to `Event::dispatch`
 * Make `field::ValidLen` trait private

Closes #948
Closes #960

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 15:19:26 -08:00
Eliza Weisman 7f911b6b70 trace-core: Debreak RustDoc links (#961)
This commit fixes a bunch of broken links in the `tokio-trace-core` API
docs.

Refs: #957
2019-03-07 14:42:08 -08:00
Eliza Weisman d88aba8d1c trace: Add arguments struct to subscriber::Record (#955)
This branch changes the `Subscriber::record` method to take a new
arguments struct, `span::Record`. The `field::Record` trait was renamed
to `field::Visit` to prevent name conflicts.

In addition, the `ValueSet::is_empty`, `ValueSet::contains`, and
`ValueSet::record` methods were made crate-private, as they are exposed
on the `Attributes` and `Record` types. 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-07 12:41:10 -08:00
Eliza Weisman 6fbef0a528 trace-core: Add 'static bound to Subscriber (#953) 2019-03-07 11:54:21 -08:00
Blake Smith 9be5f3f9ff Fix TcpStream::try_clone error message (#946) 2019-03-04 14:17:08 -08:00
Carl Lerche e28856cffe Bump Tokio to 0.1.16. (#941)
Also bumps:

* tokio-current-thread (0.1.5)
* tokio-fs (0.1.6)
* tokio-io (0.1.12)
* tokio-reactor (0.1.9)
* tokio-threadpool (0.1.12)
2019-03-01 21:04:43 -08:00
Carl Lerche 85e3bd34af async-await: fix build for latest nightly (#940)
Fixes: #936
2019-03-01 15:40:42 -08:00
Lucio Franco db4019d84a trace: Fix tokio-trace documentation url in the README (#939) 2019-03-01 15:31:59 -08:00
Carl Lerche 195c4b0496 Bump tokio-sync version to v0.1.3 (#938) 2019-03-01 12:57:07 -08:00
Carl Lerche 619d3b163b sync: impl Error for mpsc error types (#937) 2019-03-01 12:24:17 -08:00
Eliza Weisman 5ff6e37c59 trace: Allow specifying a new span's parent (#923)
This branch allows users of `tokio-trace` to explicitly set a span's
parent, or indicate that a span should be a new root of its own trace
tree. A `parent: ` key has been added to the `span!` macros. When a span
is provided, that span will be set as the parent, while `parent: None`
will result in a new root span. No `parent:` key results in the current
behaviour.

A new type, `span::Attributes`, was added to `tokio-trace-core` to act
as an arguments struct for the `Subscriber::new_span` method. This will
allow future fields to be added without causing breaking API changes.
The `Attributes` struct currently contains the new span's metadata,
`ValueSet`, and parent.

Finally, the `span::Span` type in `-core` was renamed to `span::Id`, for
consistency with `tokio-trace` and to differentiate it from
`span::Attributes`. This name was chosen primarily due to precedent in
other tracing systems.

Closes #920 

Signed-off-by: Eliza Weisman <[email protected]>
2019-03-01 11:29:11 -08:00
Carl Lerche 43d69d77e2 Set up CI with Azure Pipelines (#926)
Use Azure Pipelines for CI. This migrates away from Travis and
Appveyor.
2019-03-01 09:12:21 -08:00
Carl Lerche dbb04e310c Fix rustfmt check (#927)
* Add set -e to .travis.yml
* Fix fmt
* Fix codec feature
2019-02-24 15:41:26 -08:00
Carl Lerche 0e2e07812a Bump tokio-buf to v0.1.0 (#925) 2019-02-23 21:58:47 -08:00
Carl Lerche 047d0b821c buf: misc polish (#924)
- Rename feature flag `util`.
- Rename module `util`
- Move `error` module into `util`.
- Move `BufStream` impls into dedicated file.
2019-02-23 10:17:21 -08:00
Carl Lerche 70f4fc481c sync: Add watch, a single value broadcast channel (#922)
A single-producer, multi-consumer channel that only retains the _last_ sent
value. Values are broadcasted out.

This channel is useful for watching for changes to a value from multiple
points in the code base (for example, changes to a configuration value).
2019-02-22 21:54:50 -08:00
Carl Lerche 7039f02bb2 Bump tokio-async-await to 0.1.6 (#921) 2019-02-22 17:15:42 -08:00
Taiki Endo 4985e0c608 async-await: update to new future/task API (#919)
- Rewrite noop_waker with items from the new API and replaces
  LocalWaker with Waker.

- Bump the minimum required version for `tokio-async-await` to
  1.34.0-nightly.

- `Unpin` was added to std prelude.

- Add `cargo check` to .travis.yml

Fixes: #908
2019-02-22 13:30:21 -08:00
Toralf Wittner fd22090df8 tokio-io: Add unsplit. (#807)
Provide a way to restore an I/O object from its `ReadHalf` and
`WriteHalf`.

Closes #803

Co-Authored-By: twittner <[email protected]>
2019-02-22 12:23:36 -08:00
Eliza Weisman 02a5091885 trace: Minor doc improvements (#913)
This branch adds links to the master RustDoc published by CI to the
`tokio-trace` and `tokio-trace-core` README. In addition, it fixes a
broken links in the RustDoc for `tokio-trace` and updates the
`tokio-trace-core` RustDoc to match the README.

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-22 11:07:53 -08:00
Carl Lerche 80162306e7 chore: apply rustfmt to all crates (#917) 2019-02-21 11:56:15 -08:00
Nicholas Young ab595d0825 threadpool: fix typo in documentation (#915) 2019-02-21 09:28:44 -08:00
Carl Lerche 41a2245b85 chore: remove patch statements in Cargo.toml (#914) 2019-02-21 09:28:14 -08:00
Carl Lerche 7ca4f3ec4b Bump tokio-sync to v0.1.2. (#909) 2019-02-21 09:28:05 -08:00
Carl Lerche 0da649727c fs: fix tests (#916) 2019-02-20 21:56:23 -08:00
Linus Färnstrand 1cf5f73651 Read write helpers (#896)
Provides async versions of read / write helpers being stabilized in `std`.
2019-02-20 14:38:49 -08:00
Sean McArthur beb639a030 sync: fix warnings in benches and tests (#912) 2019-02-20 14:07:53 -08:00
Kevin Leimkuhler 75ab7c9e9b trace: Allow Span IDs to be converted back to u64s (#910)
## Motivation

As described in #905, subscribers have no way to get the numeric value of a span ID back _out_ of a `Span`.  

## Solution

Add a `Span::into_u64` method that returns the inner `u64` span ID

Closes #905

Signed-off-by: kleimkuhler <[email protected]>
2019-02-20 13:26:20 -08:00
David Wilemski cec9efeb7a Fix summary of tokio::util::StreamExt (#861)
The `throttle` function was not mentioned in the summary block but is listed as a method for the trait.
2019-02-20 13:24:48 -08:00
Sean McArthur f9345f99bb sync: drop old tasks in oneshot (#911) 2019-02-20 12:50:29 -08:00
Kevin M Granger ab206b976c fs: add CloneFuture for File::try_clone (#850) 2019-02-20 12:25:50 -08:00
Carl Lerche 3d787b16c7 sync: add loom test for mpsc (#903)
This patch updates tokio_sync::mpsc to support using loom for fuzz
testing. It includes a basic fuzz test.
2019-02-20 10:05:56 -08:00
Paul Osborne f513558076 tokio-reactor: impl AsRawFd for reactor for unix (#890)
In order to support nesting a tokio reactor within another event
system exposing the file descriptor for the underlying reactor
is useful and is already implemented for mio::Poll.

Signed-off-by: Paul Osborne <[email protected]>
2019-02-19 20:23:19 -08:00
Sean McArthur d0cdcff8aa sync: improve assert message for bounded channel buffer size 2019-02-19 17:09:36 -08:00
Carl Lerche e3115231dd sync: fix mpsc/sempahore when releasing permits (#904)
This patch fixes Semaphore by adding a missing code path to the release
routine that handles the case where the waiter's node is queued in the
sempahore but has not yet been assigned the permit.

This fix is used by mpsc to handle the case when the Sender has called
`poll_ready` and is dropped before the permit is acquired.

Fixes #900
2019-02-19 16:26:05 -08:00
Andy Russell 2d5aa82341 chore: move doc comments inside macro invocations (#901) 2019-02-19 13:54:52 -08:00
Lucio Franco dd66096ea0 buf: Add BufStreamExt trait and add a core feature (#897)
This change adds an extension trait to `BufStream` and puts the core
trait behind a feature flag for optional use.

This mainly adds the additional functions in an extension trait to
allow the user to select if they want just the core trait or the fully
featured version. Now the user can add the core feature to _not_
include the extension trait. By deafult, this feature is disabled.
2019-02-19 13:30:37 -08:00
Eliza Weisman c08e73c8d4 Introduce tokio-trace (#827)
<!-- Thank you for your Pull Request. Please provide a description above
and review the requirements below.

Bug fixes and new features should include tests.

Contributors guide:
https://github.com/tokio-rs/tokio/blob/master/CONTRIBUTING.md -->

## Motivation

In asynchronous systems like Tokio, interpreting traditional log
messages can often be quite challenging. Since individual tasks are
multiplexed on the same thread, associated events and log lines are
intermixed making it difficult to trace the logic flow. Currently, none
of the available logging frameworks or libraries in Rust offer the
ability to trace logical paths through a futures-based program.

There also are complementary goals that can be accomplished with such a
system. For example, metrics / instrumentation can be tracked by
observing emitted events, or trace data can be exported to a distributed
tracing or event processing system.

In addition, it can often be useful to generate this diagnostic data in
a structured manner that can be consumed programmatically. While prior
art for structured logging in Rust exists, it is not currently
standardized, and is not "Tokio-friendly".

## Solution

This branch adds a new library to the tokio project, `tokio-trace`.
`tokio-trace` expands upon logging-style diagnostics by allowing
libraries and applications to record structured events with additional
information about *temporality* and *causality* --- unlike a log
message, a span in `tokio-trace` has a beginning and end time, may be
entered and exited by the flow of execution, and may exist within a
nested tree of similar spans. In addition, `tokio-trace` spans are
*structured*, with the ability to record typed data as well as textual
messages.

The `tokio-trace-core` crate contains the core primitives for this
system, which are expected to remain stable, while `tokio-trace` crate
provides a more "batteries-included" API. In particular, it provides
macros which are a superset of the `log` crate's `error!`, `warn!`,
`info!`, `debug!`, and `trace!` macros, allowing users to begin the
process of adopting `tokio-trace` by performing a drop-in replacement.

## Notes

Work on this project had previously been carried out in the
[tokio-trace-prototype] repository. In addition to the `tokio-trace` and
`tokio-trace-core` crates, the `tokio-trace-prototype` repo also
contains prototypes or sketches of adapter, compatibility, and utility
crates which provide useful functionality for `tokio-trace`, but these
crates are not yet ready for a release. When this branch is merged, that
repository will be archived, and the remaining unstable crates will be
moved to a new `tokio-trace-nursery` repository. Remaining issues on the
`tokio-trace-prototype` repo will be moved to the appropriate new repo.

The crates added in this branch are not _identical_ to the current head
of the `tokio-trace-prototype` repo, as I did some final clean-up and docs
polish in this branch prior to merging this PR.

[tokio-trace-prototype]: https://github.com/hawkw/tokio-trace-prototype

Closes: #561

Signed-off-by: Eliza Weisman <[email protected]>
2019-02-19 12:15:01 -08:00
Sean McArthur d1d72dc1c8 reactor: use AtomicTask::register to reduce unnecessary task clones (#899) 2019-02-18 13:04:07 -08:00
Sean McArthur 27a42b980c reactor: release write lock before register syscall 2019-02-14 16:11:05 -08:00
Sean McArthur 7a50e09495 reactor: replace AtomicTask with that from tokio-sync 2019-02-14 16:10:48 -08:00
Sean McArthur d7a556fe8b sync: add AtomicTask::take_task() 2019-02-14 16:10:48 -08:00
Sean McArthur 860ca79d62 Check Task::will_notify_current before cloning in AtomicTask 2019-02-14 13:26:53 -08:00
Sean McArthur 7b98bf7da3 Use tokio-sync's AtomicTask in mpsc 2019-02-14 13:26:53 -08:00
Sean McArthur 49774f6af1 Add poll_ready and constructor benchmarks for tokio-sync 2019-02-14 13:26:53 -08:00
Yilin Chen ec22fb9843 reactor: replace ATOMIC_USIZE_INIT with AtomicUsize::new(0) (#889)
ATOMIC_BOOL_INIT is deprecated since 1.34 because the const fn
AtomicUsize::new is now preferred. As deny(warnings) is set,
tokio fails to build on latest nightly. This will fix it.

Signed-off-by: Yilin Chen <[email protected]>
2019-02-09 23:16:31 +01:00
Andreas Rottmann ce2147d2b6 Add a warning regarding the use of Stdin handles (#876)
Also see the discussion on issue #589.
2019-02-06 21:20:15 -08:00
Alan Somers fca41d4e73 Test FreeBSD on cirrus-ci.com (#873) 2019-02-06 21:19:54 -08:00
Carl Lerche a69aca850c Bump tokio-timer v0.2.10 (#886) 2019-02-04 16:09:43 -08:00
Zahari Dichev 13c96187f8 tokio-timer: Fix multi reset DelayQueue bug (#871)
Fixes #868
2019-02-04 14:37:58 -08:00
wangcong 61d4aa98e4 docs: replace Prepends with Appends (#882) 2019-02-04 09:46:02 -05:00
Carl Lerche 9d6d142bed Bump tokio-sync v0.1.1 (#881) 2019-02-01 14:19:34 -08:00
Stephen Carman 95b0eec8af sync: bounded channel can not have 0 size (#879) 2019-02-01 12:54:06 -08:00
Stjepan Glavina e1a07ce50c threadpool: update crossbeam dependencies (#874) 2019-01-30 14:08:43 -08:00
413 changed files with 17772 additions and 5239 deletions
-22
View File
@@ -1,22 +0,0 @@
image: Visual Studio 2017
environment:
matrix:
- TARGET: x86_64-pc-windows-msvc
platform: x64
- TARGET: i686-pc-windows-msvc
platform: x86
install:
- appveyor-retry appveyor DownloadFile https://win.rustup.rs/ -FileName rustup-init.exe
- rustup-init.exe -y --default-host %TARGET%
- set PATH=%PATH%;C:\Users\appveyor\.cargo\bin
- set RUST_BACKTRACE=1
- set LOOM_MAX_DURATION=10
- rustc -V
- cargo -V
build: false
test_script:
- cargo test --all --no-fail-fast --target %TARGET%
+43
View File
@@ -0,0 +1,43 @@
freebsd_instance:
image: freebsd-12-0-release-amd64
# Test FreeBSD in a full VM on cirrus-ci.com. Test the i686 target too, in the
# same VM. The binary will be built in 32-bit mode, but will execute on a
# 64-bit kernel and in a 64-bit environment. Our tests don't execute any of
# the system's binaries, so the environment shouldn't matter.
task:
name: FreeBSD 12.0
env:
LOOM_MAX_DURATION: 10
setup_script:
- pkg install -y curl
- curl https://sh.rustup.rs -sSf --output rustup.sh
- sh rustup.sh -y
- . $HOME/.cargo/env
- rustup target add i686-unknown-freebsd
- |
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
cargo_cache:
folder: $HOME/.cargo/registry
test_script:
- . $HOME/.cargo/env
- cargo test --all --no-fail-fast
- (cd tokio-trace/test-log-support && cargo test)
- (cd tokio-trace/test_static_max_level_features && cargo test)
- cargo doc --all
i686_test_script:
- . $HOME/.cargo/env
- |
cargo test --all --exclude tokio-tls --no-fail-fast --target i686-unknown-freebsd
before_cache_script:
- rm -rf $HOME/.cargo/registry/index
-131
View File
@@ -1,131 +0,0 @@
---
language: rust
sudo: false
addons:
apt:
packages:
# to x-compile miniz-sys from sources
- gcc-multilib
matrix:
include:
- rust: stable
- rust: beta
- rust: nightly
env: ALLOW_FAILURES=true
- os: osx
- env: TARGET=x86_64-unknown-freebsd
- env: TARGET=i686-unknown-freebsd
- env: TARGET=i686-unknown-linux-gnu
# This represents the minimum Rust version supported by Tokio. Updating this
# should be done in a dedicated PR and cannot be greater than two 0.x
# releases prior to the current stable.
#
# Tests are not run as tests may require newer versions of rust.
- rust: 1.26.0
script: |
cargo check --all
# Test combinations of enabled features.
- rust: stable
script: |
shopt -s expand_aliases
alias check="cargo check --no-default-features"
check
check --features codec
check --features fs
check --features io
check --features reactor
check --features rt-full
check --features tcp
check --features timer
check --features udp
check --features uds
# Test the async / await preview. We don't want to block PRs on this failing
# though.
- rust: nightly
env: ALLOW_FAILURES=true
script: |
cd tokio-async-await
cargo check --all
# This runs TSAN against nightly and allows failures to propagate up.
- rust: nightly-2018-11-18
env: TSAN=yes
script: |
set -e
# Make sure the benchmarks compile
cargo build --benches --all
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# === tokio-timer ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-timer --test hammer --target x86_64-unknown-linux-gnu
# === tokio-threadpool ====
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
cargo test -p tokio-threadpool --tests --target x86_64-unknown-linux-gnu
# This runs cargo +nightly doc
- name: nightly_docs
rust: nightly
env: ALLOW_FAILURES=true
script: cargo doc
allow_failures:
- rust: nightly
env: ALLOW_FAILURES=true
script: |
set -e
if [[ "$TARGET" ]]
then
rustup target add $TARGET
cargo check --all --exclude tokio-tls --target $TARGET
cargo check --tests --all --exclude tokio-tls --target $TARGET
else
# Limit the execution time of loom tests.
export LOOM_MAX_DURATION=10
cargo test --all --no-fail-fast
cargo doc --all
fi
before_deploy:
- cargo doc --all --no-deps
deploy:
provider: pages
skip_cleanup: true
github_token: $GH_TOKEN
target_branch: gh-pages
local_dir: target/doc
on:
branch: master
repo: tokio-rs/tokio
rust: stable
condition: $TRAVIS_OS_NAME = "linux" && $TARGET = ""
env:
global:
- secure: iwlN1zfUCp/5BAAheqIRSFIqiM9zSwfIGcVDw/V7jHveqXyNzmCs7H58/cd90WLqonqpPX0t5GF66oTjms4v0DFjgXr/k4358qeSZaV082V3baNrVpCDHeCQV0SvKsfiYxDDJGSUL1WIUP+tqqDm4+ksZQP3LnwZojkABjWz5CBNt4kX+Wz5ZbYqtQoxyuZba5UyPY2CXJtubvCVPGMJULuUpklYxXZ4dWM2olzGgVJ8rE8udhSZ4ER4JgxB0KUx3/5TwHHzgyPEsWR4bKN6JzBjIczQofXUcUXXdoZBs23H/VhCpzKcn3/oJ8btVYPzwtdj5FmVB1aVR/gjPo2bSGi/sofq+LwL/1HJXkM+kjl8m2dLLcDBKqNYNERtVA1++LhkMWAFRgGYe8v8Ryxjiue1NF5LgAIA/fjK0uI1DELTzTf/TKrM+AtPDNTvhOft4/YD+hoImjwk6nv6PBb2TiTYnc79Qf4AZ65tv1qtsAUPuw4plLaccHQAO4ldYVXn4u9c+iisJwvovs6jo06bF3U3qtdI5gXsrI9+T25TrXvYb+IREo0MHzYEM0KlPFnscEArzC3eajuSd36ARFP3lDc+gp2RPs89iJjowms0eRyepp7Cu6XO3Cd2pfAX8AqvnmttZf4Nm51ONeiBPXPXItUkJm49MCpMJywU1IZcWZg=
notifications:
email:
on_success: never
+3 -124
View File
@@ -1,30 +1,7 @@
[package]
name = "tokio"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.15"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
documentation = "https://docs.rs/tokio/0.1.15/tokio/"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
description = """
An event-driven, non-blocking I/O platform for writing asynchronous I/O
backed applications.
"""
categories = ["asynchronous", "network-programming"]
keywords = ["io", "async", "non-blocking", "futures"]
[workspace]
members = [
"./",
"tokio",
"tokio-async-await",
"tokio-buf",
"tokio-codec",
@@ -39,106 +16,8 @@ members = [
"tokio-timer",
"tokio-tcp",
"tokio-tls",
"tokio-trace",
"tokio-trace/tokio-trace-core",
"tokio-udp",
"tokio-uds",
]
[features]
default = [
"codec",
"fs",
"io",
"reactor",
"rt-full",
"sync",
"tcp",
"timer",
"udp",
"uds",
]
codec = ["tokio-codec"]
fs = ["tokio-fs"]
io = ["bytes", "tokio-io"]
reactor = ["io", "mio", "tokio-reactor"]
rt-full = [
"num_cpus",
"reactor",
"timer",
"tokio-current-thread",
"tokio-executor",
"tokio-threadpool",
]
sync = ["tokio-sync"]
tcp = ["tokio-tcp"]
timer = ["tokio-timer"]
udp = ["tokio-udp"]
uds = ["tokio-uds"]
# This feature comes with no promise of stability. Things will
# break with each patch release. Use at your own risk.
async-await-preview = [
"tokio-async-await/async-await-preview",
]
[badges]
travis-ci = { repository = "tokio-rs/tokio" }
appveyor = { repository = "carllerche/tokio", id = "s83yxhy9qeb58va7" }
[dependencies]
# Only non-optional dependency...
futures = "0.1.20"
# Everything else is optional...
bytes = { version = "0.4", optional = true }
num_cpus = { version = "1.8.0", optional = true }
tokio-codec = { version = "0.1.0", path = "tokio-codec", optional = true }
tokio-current-thread = { version = "0.1.3", path = "tokio-current-thread", optional = true }
tokio-fs = { version = "0.1.3", path = "tokio-fs", optional = true }
tokio-io = { version = "0.1.6", path = "tokio-io", optional = true }
tokio-executor = { version = "0.1.5", path = "tokio-executor", optional = true }
tokio-reactor = { version = "0.1.1", path = "tokio-reactor", optional = true }
tokio-sync = { version = "0.1.0", path = "tokio-sync", optional = true }
tokio-threadpool = { version = "0.1.8", path = "tokio-threadpool", optional = true }
tokio-tcp = { version = "0.1.0", path = "tokio-tcp", optional = true }
tokio-udp = { version = "0.1.0", path = "tokio-udp", optional = true }
tokio-timer = { version = "0.2.8", path = "tokio-timer", optional = true }
# Needed until `reactor` is removed from `tokio`.
mio = { version = "0.6.14", optional = true }
# Needed for async/await preview support
tokio-async-await = { version = "0.1.0", path = "tokio-async-await", optional = true }
[target.'cfg(unix)'.dependencies]
tokio-uds = { version = "0.2.1", path = "tokio-uds", optional = true }
[dev-dependencies]
env_logger = { version = "0.5", default-features = false }
flate2 = { version = "1", features = ["tokio"] }
futures-cpupool = "0.1"
http = "0.1"
httparse = "1.0"
libc = "0.2"
num_cpus = "1.0"
serde = "1.0"
serde_derive = "1.0"
serde_json = "1.0"
time = "0.1"
[patch.crates-io]
#tokio = { path = "." }
#tokio-async-await = { path = "./tokio-async-await" }
#tokio-codec = { path = "./tokio-codec" }
#tokio-current-thread = { path = "./tokio-current-thread" }
#tokio-executor = { path = "./tokio-executor" }
#tokio-fs = { path = "./tokio-fs" }
#tokio-io = { path = "./tokio-io" }
#tokio-reactor = { path = "./tokio-reactor" }
#tokio-signal = { path = "./tokio-signal" }
#tokio-tcp = { path = "./tokio-tcp" }
#tokio-threadpool = { path = "./tokio-threadpool" }
#tokio-timer = { path = "./tokio-timer" }
#tokio-tls = { path = "./tokio-tls" }
#tokio-udp = { path = "./tokio-udp" }
#tokio-uds = { path = "./tokio-uds" }
+9 -12
View File
@@ -14,29 +14,26 @@ the Rust programming language. It is:
[![Crates.io][crates-badge]][crates-url]
[![MIT licensed][mit-badge]][mit-url]
[![Travis Build Status][travis-badge]][travis-url]
[![Appveyor Build Status][appveyor-badge]][appveyor-url]
[![Build Status][azure-badge]][azure-url]
[![Gitter chat][gitter-badge]][gitter-url]
[crates-badge]: https://img.shields.io/crates/v/tokio.svg
[crates-url]: https://crates.io/crates/tokio
[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg
[mit-url]: LICENSE-MIT
[travis-badge]: https://travis-ci.org/tokio-rs/tokio.svg?branch=master
[travis-url]: https://travis-ci.org/tokio-rs/tokio
[appveyor-badge]: https://ci.appveyor.com/api/projects/status/s83yxhy9qeb58va7/branch/master?svg=true
[appveyor-url]: https://ci.appveyor.com/project/carllerche/tokio/branch/master
[azure-badge]: https://dev.azure.com/tokio-rs/Tokio/_apis/build/status/tokio-rs.tokio?branchName=master
[azure-url]: https://dev.azure.com/tokio-rs/Tokio/_build/latest?definitionId=1&branchName=master
[gitter-badge]: https://img.shields.io/gitter/room/tokio-rs/tokio.svg
[gitter-url]: https://gitter.im/tokio-rs/tokio
[Website](https://tokio.rs) |
[Guides](https://tokio.rs/docs/getting-started/hello-world/) |
[API Docs](https://docs.rs/tokio/0.1.15/tokio) |
[API Docs](https://docs.rs/tokio/0.1.19/tokio) |
[Chat](https://gitter.im/tokio-rs/tokio)
The API docs for the master branch are published [here][master-dox].
[master-dox]: https://tokio-rs.github.io/tokio/tokio/
[master-dox]: https://tokio-rs.github.io/tokio/doc/tokio/
## Overview
@@ -52,9 +49,9 @@ level, it provides a few major components:
These components provide the runtime components necessary for building
an asynchronous application.
[net]: https://docs.rs/tokio/0.1/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1/tokio/reactor/index.html
[scheduler]: https://tokio-rs.github.io/tokio/tokio/runtime/index.html
[net]: https://docs.rs/tokio/0.1.19/tokio/net/index.html
[reactor]: https://docs.rs/tokio/0.1.19/tokio/reactor/index.html
[scheduler]: https://docs.rs/tokio/0.1.19/tokio/runtime/index.html
## Example
@@ -101,7 +98,7 @@ fn main() {
}
```
More examples can be found [here](examples).
More examples can be found [here](tokio/examples).
## Getting Help
+118
View File
@@ -0,0 +1,118 @@
trigger: ["master"]
pr: ["master"]
jobs:
# Check formatting
- template: ci/azure-rustfmt.yml
parameters:
name: rustfmt
# Test top level crate
- template: ci/azure-test-stable.yml
parameters:
name: test_tokio
displayName: Test tokio
cross: true
crates:
- tokio
# Test crates that are platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_sub_cross
displayName: Test sub crates -
cross: true
crates:
- tokio-fs
- tokio-reactor
- tokio-signal
- tokio-tcp
- tokio-tls
- tokio-udp
- tokio-uds
# Test crates that are NOT platform specific
- template: ci/azure-test-stable.yml
parameters:
name: test_linux
displayName: Test sub crates -
crates:
- tokio-buf
- tokio-codec
- tokio-current-thread
- tokio-executor
- tokio-io
- tokio-sync
- tokio-threadpool
- tokio-timer
- tokio-trace
- tokio-trace/tokio-trace-core
- tokio-trace/test-log-support
- tokio-trace/test_static_max_level_features
- template: ci/azure-cargo-check.yml
parameters:
name: features
displayName: Check feature permtuations
rust: stable
crates:
tokio:
- codec
- fs
- io
- reactor
- rt-full
- tcp
- timer
- udp
- uds
- sync
tokio-buf:
- util
# Check async / await
- template: ci/azure-cargo-check.yml
parameters:
name: async_await
displayName: Async / Await
rust: nightly-2019-04-22
noDefaultFeatures: ''
benches: true
crates:
tokio:
- async-await-preview
# Try cross compiling
- template: ci/azure-cross-compile.yml
parameters:
name: cross_32bit_linux
target: i686-unknown-linux-gnu
# This represents the minimum Rust version supported by
# Tokio. Updating this should be done in a dedicated PR and
# cannot be greater than two 0.x releases prior to the
# current stable.
#
# Tests are not run as tests may require newer versions of
# rust.
- template: ci/azure-check-minrust.yml
parameters:
name: minrust
rust_version: 1.26.0
- template: ci/azure-tsan.yml
parameters:
name: tsan
- template: ci/azure-deploy-docs.yml
parameters:
dependsOn:
- rustfmt
- test_tokio
- test_sub_cross
- test_linux
- features
- async_await
- cross_32bit_linux
- minrust
- tsan
+1 -3
View File
@@ -10,8 +10,8 @@ use std::io;
use std::net::SocketAddr;
use std::thread;
use futures::sync::oneshot;
use futures::sync::mpsc;
use futures::sync::oneshot;
use futures::{Future, Poll, Sink, Stream};
use test::Bencher;
use tokio::net::UdpSocket;
@@ -57,7 +57,6 @@ fn udp_echo_latency(b: &mut Bencher) {
let (tx, rx) = oneshot::channel();
let child = thread::spawn(move || {
let socket = tokio::net::UdpSocket::bind(&any_addr).unwrap();
tx.send(socket.local_addr().unwrap()).unwrap();
@@ -67,7 +66,6 @@ fn udp_echo_latency(b: &mut Bencher) {
server.wait().unwrap();
});
let client = std::net::UdpSocket::bind(&any_addr).unwrap();
let server_addr = rx.wait().unwrap();
+8 -9
View File
@@ -3,14 +3,13 @@
#![feature(test)]
#![deny(warnings)]
extern crate test;
extern crate mio;
extern crate test;
use test::Bencher;
use mio::tcp::TcpListener;
use mio::{Token, Ready, PollOpt};
use mio::{PollOpt, Ready, Token};
#[bench]
fn mio_register_deregister(b: &mut Bencher) {
@@ -22,8 +21,8 @@ fn mio_register_deregister(b: &mut Bencher) {
const CLIENT: Token = Token(1);
b.iter(|| {
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
poll.deregister(&sock).unwrap();
});
}
@@ -36,12 +35,12 @@ fn mio_reregister(b: &mut Bencher) {
let poll = mio::Poll::new().unwrap();
const CLIENT: Token = Token(1);
poll.register(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.register(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
b.iter(|| {
poll.reregister(&sock, CLIENT, Ready::readable(),
PollOpt::edge()).unwrap();
poll.reregister(&sock, CLIENT, Ready::readable(), PollOpt::edge())
.unwrap();
});
poll.deregister(&sock).unwrap();
}
+62 -49
View File
@@ -11,18 +11,18 @@ pub extern crate test;
mod prelude {
pub use futures::*;
pub use tokio::reactor::Reactor;
pub use tokio::net::{TcpListener, TcpStream};
pub use tokio::reactor::Reactor;
pub use tokio_io::io::read_to_end;
pub use test::{self, Bencher};
pub use std::io::{self, Read, Write};
pub use std::thread;
pub use std::time::Duration;
pub use std::io::{self, Read, Write};
pub use test::{self, Bencher};
}
mod connect_churn {
use ::prelude::*;
use prelude::*;
const NUM: usize = 300;
const CONCURRENT: usize = 8;
@@ -36,25 +36,29 @@ mod connect_churn {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
let connects = stream::iter_result((0..NUM).map(|_| {
Ok(TcpStream::connect(&addr)
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
Ok(TcpStream::connect(&addr).and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
let connects_concurrent = connects.buffer_unordered(CONCURRENT)
let connects_concurrent = connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()));
serve_incomings.select(connects_concurrent)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(connects_concurrent)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
}
@@ -65,8 +69,7 @@ mod connect_churn {
// Spawn reactor thread
let server_thread = thread::spawn(move || {
// Bind the TCP listener
let listener = TcpListener::bind(
&"127.0.0.1:0".parse().unwrap()).unwrap();
let listener = TcpListener::bind(&"127.0.0.1:0".parse().unwrap()).unwrap();
// Get the address being listened on.
let addr = listener.local_addr().unwrap();
@@ -75,47 +78,56 @@ mod connect_churn {
addr_tx.send(addr).unwrap();
// Spawn a single future that accepts & drops connections
let serve_incomings = listener.incoming()
let serve_incomings = listener
.incoming()
.map_err(|e| panic!("server err: {:?}", e))
.for_each(|_| Ok(()));
// Run server
serve_incomings.select(shutdown_rx)
.map(|_| ()).map_err(|_| ())
.wait().unwrap();
serve_incomings
.select(shutdown_rx)
.map(|_| ())
.map_err(|_| ())
.wait()
.unwrap();
});
// Get the bind addr of the server
let addr = addr_rx.wait().unwrap();
b.iter(move || {
use std::sync::{Barrier, Arc};
use std::sync::{Arc, Barrier};
// Create a barrier to coordinate threads
let barrier = Arc::new(Barrier::new(n + 1));
// Spawn worker threads
let threads: Vec<_> = (0..n).map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
let threads: Vec<_> = (0..n)
.map(|_| {
let barrier = barrier.clone();
let addr = addr.clone();
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
thread::spawn(move || {
let connects = stream::iter_result((0..(NUM / n)).map(|_| {
Ok(TcpStream::connect(&addr)
.map_err(|e| panic!("connect err: {:?}", e))
.and_then(|sock| {
sock.set_linger(Some(Duration::from_secs(0))).unwrap();
read_to_end(sock, vec![])
}))
}));
barrier.wait();
barrier.wait();
connects.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(())).wait().unwrap();
connects
.buffer_unordered(CONCURRENT)
.map_err(|e| panic!("client err: {:?}", e))
.for_each(|_| Ok(()))
.wait()
.unwrap();
})
})
}).collect();
.collect();
barrier.wait();
@@ -141,7 +153,7 @@ mod connect_churn {
}
mod transfer {
use ::prelude::*;
use prelude::*;
use std::{cmp, mem};
const MB: usize = 3 * 1024 * 1024;
@@ -200,7 +212,8 @@ mod transfer {
let addr = listener.local_addr().unwrap();
// Spawn a single future that accepts 1 connection, Drain it and drops
let server = listener.incoming()
let server = listener
.incoming()
.into_future() // take the first connection
.map_err(|(e, _other_incomings)| e)
.map(|(connection, _other_incomings)| connection.unwrap())
@@ -210,17 +223,17 @@ mod transfer {
sock: sock,
chunk: read_size,
};
drain.map(|_| ()).map_err(|e| panic!("server error: {:?}", e))
drain
.map(|_| ())
.map_err(|e| panic!("server error: {:?}", e))
})
.map_err(|e| panic!("server err: {:?}", e));
let client = TcpStream::connect(&addr)
.and_then(move |sock| {
Transfer {
sock: sock,
rem: MB,
chunk: write_size,
}
.and_then(move |sock| Transfer {
sock: sock,
rem: MB,
chunk: write_size,
})
.map_err(|e| panic!("client err: {:?}", e));
@@ -229,7 +242,7 @@ mod transfer {
}
mod small_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
@@ -238,7 +251,7 @@ mod transfer {
}
mod big_chunks {
use ::prelude::*;
use prelude::*;
#[bench]
fn one_thread(b: &mut Bencher) {
+33
View File
@@ -0,0 +1,33 @@
parameters:
noDefaultFeatures: '--no-default-features'
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust }}
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- ${{ each feature in crate.value }}:
- script: cargo check ${{ parameters.noDefaultFeatures }} --features ${{ feature }}
displayName: Check `${{ crate.key }}`, features = ${{ feature }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate.key }}
- ${{ if parameters.benches }}:
- script: cargo check --benches --all
displayName: Check benchmarks
+14
View File
@@ -0,0 +1,14 @@
jobs:
- job: ${{ parameters.name }}
displayName: Min supported Rust version
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: ${{ parameters.rust_version }}
- template: azure-patch-crates.yml
- script: cargo check --all
displayName: cargo check --all
+24
View File
@@ -0,0 +1,24 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: sudo apt-get install gcc-multilib
displayName: "Install gcc-multilib"
- script: rustup target add ${{ parameters.target }}
displayName: "Add target"
# Always patch
- template: azure-patch-crates.yml
- script: cargo check --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check source
- script: cargo check --tests --all --exclude tokio-tls --target ${{ parameters.target }}
displayName: Check tests
+38
View File
@@ -0,0 +1,38 @@
parameters:
dependsOn: []
jobs:
- job: documentation
displayName: 'Deploy API Documentation'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
pool:
vmImage: 'Ubuntu 16.04'
dependsOn:
- ${{ parameters.dependsOn }}
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
cargo doc --all --no-deps
cp -R target/doc '$(Build.BinariesDirectory)'
displayName: 'Generate Documentation'
- script: |
set -e
git --version
ls -la
git init
git config user.name 'Deployment Bot (from Azure Pipelines)'
git config user.email '[email protected]'
git config --global credential.helper 'store --file ~/.my-credentials'
printf "protocol=https\nhost=github.com\nusername=carllerche\npassword=%s\n\n" "$GITHUB_TOKEN" | git credential-store --file ~/.my-credentials store
git remote add origin https://github.com/tokio-rs/tokio
git checkout -b gh-pages
git add .
git commit -m 'Deploy Tokio API documentation'
git push -f origin gh-pages
env:
GITHUB_TOKEN: $(githubPersonalToken)
workingDirectory: '$(Build.BinariesDirectory)'
displayName: 'Deploy Documentation'
+27
View File
@@ -0,0 +1,27 @@
steps:
# Linux and macOS.
- script: |
set -e
curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain $RUSTUP_TOOLCHAIN
echo "##vso[task.setvariable variable=PATH;]$PATH:$HOME/.cargo/bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (*nix)"
condition: not(eq(variables['Agent.OS'], 'Windows_NT'))
# Windows.
- script: |
curl -sSf -o rustup-init.exe https://win.rustup.rs
rustup-init.exe -y --default-toolchain %RUSTUP_TOOLCHAIN%
set PATH=%PATH%;%USERPROFILE%\.cargo\bin
echo "##vso[task.setvariable variable=PATH;]%PATH%;%USERPROFILE%\.cargo\bin"
env:
RUSTUP_TOOLCHAIN: ${{parameters.rust_version}}
displayName: "Install rust (windows)"
condition: eq(variables['Agent.OS'], 'Windows_NT')
# All platforms.
- script: |
rustc -Vv
cargo -V
displayName: Query rust and cargo versions
+9
View File
@@ -0,0 +1,9 @@
steps:
- bash: |
set -e
if git log --no-merges -1 --format='%s' | grep -q '[ci-release]'; then
echo "##vso[task.setvariable variable=isRelease]true"
fi
failOnStderr: true
displayName: Check if release commit
+16
View File
@@ -0,0 +1,16 @@
steps:
- script: |
set -e
# Remove any existing patch statements
mv Cargo.toml Cargo.toml.bck
sed -n '/\[patch.crates-io\]/q;p' Cargo.toml.bck > Cargo.toml
# Patch all crates
cat ci/patch.toml >> Cargo.toml
# Print `Cargo.toml` for debugging
echo "~~~~ Cargo.toml ~~~~"
cat Cargo.toml
echo "~~~~~~~~~~~~~~~~~~~~"
displayName: Patch Cargo.toml
+16
View File
@@ -0,0 +1,16 @@
jobs:
# Check formatting
- job: ${{ parameters.name }}
displayName: Check rustfmt
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- script: |
rustup component add rustfmt
displayName: Install rustfmt
- script: |
cargo fmt --all -- --check
displayName: Check formatting
+41
View File
@@ -0,0 +1,41 @@
jobs:
- job: ${{ parameters.name }}
displayName: ${{ parameters.displayName }}
strategy:
matrix:
Linux:
vmImage: ubuntu-16.04
${{ if parameters.cross }}:
MacOS:
vmImage: macOS-10.13
Windows:
vmImage: vs2017-win2016
pool:
vmImage: $(vmImage)
steps:
- template: azure-install-rust.yml
parameters:
rust_version: stable
- template: azure-is-release.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }}
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
condition: and(succeeded(), not(variables['isRelease']))
- template: azure-patch-crates.yml
- ${{ each crate in parameters.crates }}:
- script: cargo test
env:
LOOM_MAX_DURATION: 10
CI: 'True'
displayName: cargo test -p ${{ crate }} (PATCHED)
workingDirectory: $(Build.SourcesDirectory)/${{ crate }}
+36
View File
@@ -0,0 +1,36 @@
jobs:
- job: ${{ parameters.name }}
displayName: TSAN
strategy:
matrix:
Timer:
cmd: cargo test -p tokio-timer --test hammer
Threadpool:
cmd: cargo test -p tokio-threadpool --tests
pool:
vmImage: ubuntu-16.04
steps:
- template: azure-install-rust.yml
parameters:
rust_version: nightly-2018-11-18
- template: azure-patch-crates.yml
- script: |
set -e
# Make sure the benchmarks compile
export ASAN_OPTIONS="detect_odr_violation=0 detect_leaks=0"
export TSAN_OPTIONS="suppressions=`pwd`/ci/tsan"
export RUST_BACKTRACE=1
# Run address sanitizer
RUSTFLAGS="-Z sanitizer=address" \
$(cmd) --target x86_64-unknown-linux-gnu
# Run thread sanitizer
RUSTFLAGS="-Z sanitizer=thread" \
$(cmd) --target x86_64-unknown-linux-gnu
displayName: TSAN / MSAN
env:
TSAN: yes
+22
View File
@@ -0,0 +1,22 @@
# Patch dependencies to run all tests against versions of the crate in the
# repository.
[patch.crates-io]
tokio = { path = "tokio" }
tokio-async-await = { path = "tokio-async-await" }
tokio-buf = { path = "tokio-buf" }
tokio-codec = { path = "tokio-codec" }
tokio-current-thread = { path = "tokio-current-thread" }
tokio-executor = { path = "tokio-executor" }
tokio-fs = { path = "tokio-fs" }
tokio-io = { path = "tokio-io" }
tokio-reactor = { path = "tokio-reactor" }
tokio-signal = { path = "tokio-signal" }
tokio-sync = { path = "tokio-sync" }
tokio-threadpool = { path = "tokio-threadpool" }
tokio-timer = { path = "tokio-timer" }
tokio-tcp = { path = "tokio-tcp" }
tokio-tls = { path = "tokio-tls" }
tokio-trace = { path = "tokio-trace" }
tokio-trace-core = { path = "tokio-trace/tokio-trace-core" }
tokio-udp = { path = "tokio-udp" }
tokio-uds = { path = "tokio-uds" }
-26
View File
@@ -1,26 +0,0 @@
use std::future::{Future as StdFuture};
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::run(future);
}
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use tokio_async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
::spawn(future);
}
+4 -4
View File
@@ -3,12 +3,12 @@ name = "tokio-async-await"
# When releasing to crates.io:
# - Update html_root_url.
version = "0.1.5"
version = "0.1.7"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-async-await/0.1.3"
documentation = "https://docs.rs/tokio-async-await/0.1.7"
description = """
Experimental async/await support for Tokio
"""
@@ -21,9 +21,9 @@ async-await-preview = ["futures/nightly"]
[dependencies]
futures = "0.1.23"
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
[dev-dependencies]
bytes = "0.4.9"
tokio = { version = "0.1.8", path = ".." }
tokio = "0.1.8"
hyper = "0.12.8"
+2 -2
View File
@@ -9,7 +9,7 @@ guarantees. You are living on the edge here.**
## Usage
To use this crate, you need to start with a Rust 2018 edition crate, with rustc
1.33.0-nightly or later.
1.35.0-nightly or later.
Add this to your `Cargo.toml`:
@@ -18,7 +18,7 @@ Add this to your `Cargo.toml`:
edition = "2018"
# In the `[dependencies]` section
tokio = {version = "0.1.0", features = ["async-await-preview"]}
tokio = {version = "0.1.15", features = ["async-await-preview"]}
```
Then, get started. In your application, add:
+2 -2
View File
@@ -25,14 +25,14 @@ name = "hyper"
path = "src/hyper.rs"
[dependencies]
tokio = { version = "0.1.0", path = "../..", features = ["async-await-preview"] }
tokio = { version = "0.1.18", features = ["async-await-preview"] }
futures = "0.1.23"
bytes = "0.4.9"
hyper = "0.12.8"
# Avoid using crates.io for Tokio dependencies
[patch.crates-io]
tokio = { path = "../.." }
tokio = { path = "../../tokio" }
tokio-async-await = { path = "../" }
tokio-codec = { path = "../../tokio-codec" }
tokio-current-thread = { path = "../../tokio-current-thread" }
+1 -1
View File
@@ -61,7 +61,7 @@ async fn process(stream: TcpStream, state: Arc<Mutex<Shared>>) -> io::Result<()>
tokio::spawn_async(async move {
while let Some(line) = await!(rx.next()) {
let line = line.unwrap();
await!(lines_tx.send_async(line));
await!(lines_tx.send_async(line)).unwrap();
}
});
@@ -1,4 +1,4 @@
#![feature(await_macro, async_await)]
#![feature(await_macro, async_await, futures_api)]
#[macro_use]
extern crate tokio;
+4 -4
View File
@@ -2,15 +2,15 @@
#[macro_export]
macro_rules! await {
($e:expr) => {{
use $crate::std_await;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
#[allow(unused_imports)]
use $crate::compat::backward::IntoAwaitable as IntoAwaitableBackward;
#[allow(unused_imports)]
use $crate::compat::forward::IntoAwaitable as IntoAwaitableForward;
use $crate::std_await;
#[allow(unused_mut)]
let mut e = $e;
let e = e.into_awaitable();
std_await!(e)
}}
}};
}
+23 -32
View File
@@ -1,16 +1,9 @@
use futures::{Future, Poll};
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::future::{
Future as StdFuture,
};
use std::ptr::NonNull;
use std::task::{
LocalWaker,
Poll as StdPoll,
UnsafeWake,
Waker,
};
use std::ptr;
use std::task::{Context, Poll as StdPoll, RawWaker, RawWakerVTable, Waker};
/// Convert an 0.3 `Future` to an 0.1 `Future`.
#[derive(Debug)]
@@ -31,7 +24,8 @@ pub trait IntoAwaitable {
}
impl<T> IntoAwaitable for T
where T: StdFuture,
where
T: StdFuture,
{
type Awaitable = Self;
@@ -41,7 +35,8 @@ where T: StdFuture,
}
impl<T, Item, Error> Future for Compat<T>
where T: StdFuture<Output = Result<Item, Error>>,
where
T: StdFuture<Output = Result<Item, Error>>,
{
type Item = Item;
type Error = Error;
@@ -49,9 +44,10 @@ where T: StdFuture<Output = Result<Item, Error>>,
fn poll(&mut self) -> Poll<Item, Error> {
use futures::Async::*;
let local_waker = noop_local_waker();
let waker = noop_waker();
let mut context = Context::from_waker(&waker);
let res = self.0.as_mut().poll(&local_waker);
let res = self.0.as_mut().poll(&mut context);
match res {
StdPoll::Ready(Ok(val)) => Ok(Ready(val)),
@@ -63,27 +59,22 @@ where T: StdFuture<Output = Result<Item, Error>>,
// ===== NoopWaker =====
struct NoopWaker;
fn noop_local_waker() -> LocalWaker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { LocalWaker::new(w) }
fn noop_raw_waker() -> RawWaker {
RawWaker::new(ptr::null(), &NOOP_WAKER_VTABLE)
}
fn noop_waker() -> Waker {
let w: NonNull<NoopWaker> = NonNull::dangling();
unsafe { Waker::new(w) }
unsafe { Waker::from_raw(noop_raw_waker()) }
}
unsafe impl UnsafeWake for NoopWaker {
unsafe fn clone_raw(&self) -> Waker {
noop_waker()
}
unsafe fn drop_raw(&self) {
}
unsafe fn wake(&self) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
unsafe fn clone_raw(_data: *const ()) -> RawWaker {
noop_raw_waker()
}
unsafe fn drop_raw(_data: *const ()) {}
unsafe fn wake(_data: *const ()) {
unimplemented!("async-await-preview currently only supports futures 0.1. Use the compatibility layer of futures 0.3 instead, if you want to use futures 0.3.");
}
const NOOP_WAKER_VTABLE: RawWakerVTable = RawWakerVTable::new(clone_raw, wake, wake, drop_raw);
+10 -11
View File
@@ -1,17 +1,15 @@
use futures::{Async, Future};
use futures::{Future, Async};
use std::marker::Unpin;
use std::future::Future as StdFuture;
use std::pin::Pin;
use std::task::{LocalWaker, Poll as StdPoll};
use std::task::{Context, Poll as StdPoll};
/// Converts an 0.1 `Future` into an 0.3 `Future`.
#[derive(Debug)]
pub struct Compat<T>(T);
pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T, E>> {
use futures::Async::{Ready, NotReady};
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(val)) => StdPoll::Ready(Ok(val)),
@@ -21,9 +19,9 @@ pub(crate) fn convert_poll<T, E>(poll: Result<Async<T>, E>) -> StdPoll<Result<T,
}
pub(crate) fn convert_poll_stream<T, E>(
poll: Result<Async<Option<T>>, E>) -> StdPoll<Option<Result<T, E>>>
{
use futures::Async::{Ready, NotReady};
poll: Result<Async<Option<T>>, E>,
) -> StdPoll<Option<Result<T, E>>> {
use futures::Async::{NotReady, Ready};
match poll {
Ok(Ready(Some(val))) => StdPoll::Ready(Some(Ok(val))),
@@ -50,12 +48,13 @@ impl<T: Future + Unpin> IntoAwaitable for T {
}
impl<T> StdFuture for Compat<T>
where T: Future + Unpin
where
T: Future + Unpin,
{
type Output = Result<T::Item, T::Error>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> StdPoll<Self::Output> {
use futures::Async::{Ready, NotReady};
fn poll(mut self: Pin<&mut Self>, _context: &mut Context) -> StdPoll<Self::Output> {
use futures::Async::{NotReady, Ready};
// TODO: wire in cx
+1 -1
View File
@@ -1,4 +1,4 @@
#![doc(hidden)]
pub mod forward;
pub mod backward;
pub mod forward;
+3 -5
View File
@@ -1,11 +1,9 @@
use tokio_io::AsyncWrite;
use std::io;
use std::future::Future;
use std::marker::Unpin;
use std::io;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Context, Poll};
/// A future used to fully flush an I/O object.
#[derive(Debug)]
@@ -25,7 +23,7 @@ impl<'a, T: AsyncWrite + ?Sized> Flush<'a, T> {
impl<'a, T: AsyncWrite + ?Sized> Future for Flush<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _wx: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _context: &mut Context) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
convert_poll(self.writer.poll_flush())
}
+6 -6
View File
@@ -24,7 +24,7 @@ pub trait AsyncReadExt: AsyncRead {
///
/// # Examples
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
@@ -58,7 +58,7 @@ pub trait AsyncReadExt: AsyncRead {
///
/// # Examples
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
@@ -77,7 +77,7 @@ pub trait AsyncReadExt: AsyncRead {
///
/// ## EOF is hit before `buf` is filled
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
@@ -109,7 +109,7 @@ pub trait AsyncWriteExt: AsyncWrite {
///
/// # Examples
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
@@ -138,7 +138,7 @@ pub trait AsyncWriteExt: AsyncWrite {
///
/// # Examples
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
@@ -162,7 +162,7 @@ pub trait AsyncWriteExt: AsyncWrite {
///
/// # Examples
///
/// ```
/// ```edition2018
/// #![feature(async_await, await_macro, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future which can be used to read bytes.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Read<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> Read<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> Read<'a, T> {
Read {
reader,
buf,
}
Read { reader, buf }
}
}
impl<'a, T: AsyncRead + ?Sized> Future for Read<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _context: &mut task::Context) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for ReadExact<'a, T> {}
impl<'a, T: AsyncRead + ?Sized> ReadExact<'a, T> {
pub(super) fn new(reader: &'a mut T, buf: &'a mut [u8]) -> ReadExact<'a, T> {
ReadExact {
reader,
buf,
}
ReadExact { reader, buf }
}
}
@@ -34,7 +30,7 @@ fn eof() -> io::Error {
impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _context: &mut task::Context) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -47,7 +43,7 @@ impl<'a, T: AsyncRead + ?Sized> Future for ReadExact<'a, T> {
this.buf = rest;
}
if n == 0 {
return Poll::Ready(Err(eof()))
return Poll::Ready(Err(eof()));
}
}
+2 -6
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::pin::Pin;
/// A future used to write data.
@@ -19,17 +18,14 @@ impl<'a, T: ?Sized> Unpin for Write<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> Write<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> Write<'a, T> {
Write {
writer,
buf,
}
Write { writer, buf }
}
}
impl<'a, T: AsyncWrite + ?Sized> Future for Write<'a, T> {
type Output = io::Result<usize>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<usize>> {
fn poll(mut self: Pin<&mut Self>, _context: &mut task::Context) -> Poll<io::Result<usize>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
+3 -7
View File
@@ -4,7 +4,6 @@ use std::future::Future;
use std::task::{self, Poll};
use std::io;
use std::marker::Unpin;
use std::mem;
use std::pin::Pin;
@@ -20,10 +19,7 @@ impl<'a, T: ?Sized> Unpin for WriteAll<'a, T> {}
impl<'a, T: AsyncWrite + ?Sized> WriteAll<'a, T> {
pub(super) fn new(writer: &'a mut T, buf: &'a [u8]) -> WriteAll<'a, T> {
WriteAll {
writer,
buf,
}
WriteAll { writer, buf }
}
}
@@ -34,7 +30,7 @@ fn zero_write() -> io::Error {
impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
type Output = io::Result<()>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<io::Result<()>> {
fn poll(mut self: Pin<&mut Self>, _context: &mut task::Context) -> Poll<io::Result<()>> {
use crate::compat::forward::convert_poll;
let this = &mut *self;
@@ -48,7 +44,7 @@ impl<'a, T: AsyncWrite + ?Sized> Future for WriteAll<'a, T> {
}
if n == 0 {
return Poll::Ready(Err(zero_write()))
return Poll::Ready(Err(zero_write()));
}
}
+5 -84
View File
@@ -1,13 +1,6 @@
#![cfg(feature = "async-await-preview")]
#![feature(
rust_2018_preview,
arbitrary_self_types,
async_await,
await_macro,
futures_api,
)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.5")]
#![feature(rust_2018_preview, async_await, await_macro, futures_api)]
#![doc(html_root_url = "https://docs.rs/tokio-async-await/0.1.7")]
#![deny(missing_docs, missing_debug_implementations)]
#![cfg_attr(test, deny(warnings))]
@@ -23,12 +16,10 @@ macro_rules! try_ready {
($x:expr) => {
match $x {
std::task::Poll::Ready(Ok(x)) => x,
std::task::Poll::Ready(Err(e)) =>
return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending =>
return std::task::Poll::Pending,
std::task::Poll::Ready(Err(e)) => return std::task::Poll::Ready(Err(e.into())),
std::task::Poll::Pending => return std::task::Poll::Pending,
}
}
};
}
#[macro_use]
@@ -38,77 +29,7 @@ pub mod io;
pub mod sink;
pub mod stream;
/*
pub mod prelude {
//! A "prelude" for users of the `tokio` crate.
//!
//! This prelude is similar to the standard library's prelude in that you'll
//! almost always want to import its entire contents, but unlike the standard
//! library's prelude you'll have to do so manually:
//!
//! ```
//! use tokio::prelude::*;
//! ```
//!
//! The prelude may grow over time as additional items see ubiquitous use.
pub use tokio_main::prelude::*;
#[doc(inline)]
pub use crate::async_await::{
io::{
AsyncReadExt,
AsyncWriteExt,
},
sink::{
SinkExt,
},
stream::{
StreamExt,
},
};
}
*/
// Rename the `await` macro in `std`. This is used by the redefined
// `await` macro in this crate.
#[doc(hidden)]
pub use std::await as std_await;
/*
use std::future::{Future as StdFuture};
fn run<T: futures::Future<Item = (), Error = ()>>(t: T) {
drop(t);
}
async fn map_ok<T: StdFuture>(future: T) -> Result<(), ()> {
let _ = await!(future);
Ok(())
}
/// Like `tokio::run`, but takes an `async` block
pub fn run_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use async_await::compat::backward;
let future = backward::Compat::new(map_ok(future));
run(future);
unimplemented!();
}
*/
/*
/// Like `tokio::spawn`, but takes an `async` block
pub fn spawn_async<F>(future: F)
where F: StdFuture<Output = ()> + Send + 'static,
{
use crate::async_await::compat::backward;
spawn(backward::Compat::new(async || {
let _ = await!(future);
Ok(())
}));
}
*/
-2
View File
@@ -6,8 +6,6 @@ pub use self::send::Send;
use futures::Sink;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Sink` types.
pub trait SinkExt: Sink {
/// Send an item into the sink.
+2 -3
View File
@@ -3,7 +3,6 @@ use futures::Sink;
use std::future::Future;
use std::task::{self, Poll};
use std::marker::Unpin;
use std::pin::Pin;
/// Future for the `SinkExt::send_async` combinator, which sends a value to a
@@ -28,9 +27,9 @@ impl<'a, T: Sink + Unpin + ?Sized> Send<'a, T> {
impl<T: Sink + Unpin + ?Sized> Future for Send<'_, T> {
type Output = Result<(), T::SinkError>;
fn poll(mut self: Pin<&mut Self>, _lw: &task::LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _context: &mut task::Context) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll;
use futures::AsyncSink::{Ready, NotReady};
use futures::AsyncSink::{NotReady, Ready};
if let Some(item) = self.item.take() {
match self.sink.start_send(item) {
+3 -5
View File
@@ -6,20 +6,18 @@ pub use self::next::Next;
use futures::Stream;
use std::marker::Unpin;
/// An extension trait which adds utility methods to `Stream` types.
pub trait StreamExt: Stream {
/// Creates a future that resolves to the next item in the stream.
///
/// # Examples
///
/// ```
/// #![feature(await_macro, async_await)]
/// ```edition2018
/// #![feature(await_macro, async_await, futures_api)]
/// tokio::run_async(async {
/// // The extension trait can also be imported with
/// // `use tokio::prelude::*`.
/// use tokio::prelude::{stream, StreamExt};
/// use tokio::prelude::{stream, StreamAsyncExt};
///
/// let mut stream = stream::iter_ok::<_, ()>(1..3);
///
+2 -3
View File
@@ -1,9 +1,8 @@
use futures::Stream;
use std::future::Future;
use std::marker::Unpin;
use std::pin::Pin;
use std::task::{LocalWaker, Poll};
use std::task::{Context, Poll};
/// A future of the next element of a stream.
#[derive(Debug)]
@@ -22,7 +21,7 @@ impl<'a, T: Stream + Unpin> Next<'a, T> {
impl<'a, T: Stream + Unpin> Future for Next<'a, T> {
type Output = Option<Result<T::Item, T::Error>>;
fn poll(mut self: Pin<&mut Self>, _lw: &LocalWaker) -> Poll<Self::Output> {
fn poll(mut self: Pin<&mut Self>, _context: &mut Context) -> Poll<Self::Output> {
use crate::compat::forward::convert_poll_stream;
convert_poll_stream(self.stream.poll())
+12 -1
View File
@@ -1,3 +1,14 @@
# 0.1.0 (unreleased)
# 0.1.1 (April 22, 2019)
### Added
- Utilities for creating a `BufStream` from iterators and streams (#1011).
- Add `BufStream::into_stream` (#1048).
- Implement `FromBufStream` for `Bytes` (#1009).
- Implement `Error` for `CollectVecError` (#1010).
### Fixed
- Implement `size_hint` for string types (#1012).
# 0.1.0 (February 23, 2019)
* Initial release
+14 -4
View File
@@ -3,20 +3,30 @@ name = "tokio-buf"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.0"
version = "0.1.1"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-buf/0.1.0"
documentation = "https://docs.rs/tokio-buf/0.1.1/tokio_buf"
description = """
Asynchronous stream of byte buffers
"""
categories = ["asynchronous"]
[dependencies]
bytes = { version = "0.4.10", features = [ "either" ] }
either = "1.5"
bytes = "0.4.10"
either = { version = "1.5", optional = true}
futures = "0.1.23"
[features]
default = ["util"]
util = ["bytes/either", "either"]
[dev-dependencies]
tokio-mock-task = "0.1.1"
+35
View File
@@ -0,0 +1,35 @@
# tokio-buf
Asynchronous stream of byte buffers
[Documenation](https://docs.rs/tokio-buf)
## Usage
First, add this to your `Cargo.toml`:
```toml
[dependencies]
tokio-buf = "0.1.1"
```
Next, add this to your crate:
```rust
extern crate tokio_buf;
```
You can find extensive documentation and examples about how to use this crate
online at [https://tokio.rs](https://tokio.rs). The [API
documentation](https://docs.rs/tokio-buf) is also a great place to get started
for the nitty-gritty.
## License
This project is licensed under the [MIT license](LICENSE).
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in Tokio by you, shall be licensed as MIT, without any additional
terms or conditions.
-32
View File
@@ -1,32 +0,0 @@
//! Error types
pub use super::collect::CollectError;
pub use super::from::CollectVecError;
pub use super::limit::LimitError;
// Being crate-private, we should be able to swap the type out in a
// backwards compatible way.
pub(crate) mod internal {
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
}
-163
View File
@@ -1,163 +0,0 @@
//! Types and utilities for working with `BufStream`.
mod bytes;
mod chain;
mod collect;
pub mod errors;
mod from;
mod limit;
mod size_hint;
mod str;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::limit::Limit;
pub use self::size_hint::SizeHint;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
/// Indicates to the `BufStream` how much data the consumer is currently
/// able to process.
///
/// The consume hint allows the stream to perform certain optimizations that
/// are dependent on the consumer's readiness. For example, the consume hint
/// may be used to request a remote peer to start sending up to `amount`
/// data.
///
/// Calling `consume_hint` is not a requirement. If `consume_hint` is never
/// called, the stream should assume a default behavior. When `consume_hint`
/// is called, the stream should make a best effort to honor by the request.
///
/// `amount` represents the number of bytes that the caller would like to
/// receive at the time the function is called. For example, if
/// `consume_hint` is called with 20, the consumer requests 20 bytes. The
/// stream may yield less than that. If the next call to `poll_buf` returns
/// 5 bytes, the consumer still has 15 bytes requested. At this point,
/// invoking `consume_hint` again with 20 resets the amount requested back
/// to 20 bytes.
///
/// Calling `consume_hint` with 0 as the argument informs the stream that
/// the caller does not intend to call `poll_buf`. If `poll_buf` **is**
/// called, the stream may, but is not obligated to, return `NotReady` even
/// if it could produce data at that point. If it chooses to return
/// `NotReady`, when `consume_hint` is called with a non-zero argument, the
/// task must be notified in order to respect the `poll_buf` contract.
fn consume_hint(&mut self, amount: usize) {
// By default, this function does nothing
drop(amount);
}
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
}
+83 -4
View File
@@ -1,5 +1,5 @@
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.0")]
#![deny(missing_docs, missing_debug_implementations)]
#![doc(html_root_url = "https://docs.rs/tokio-buf/0.1.1")]
#![deny(missing_docs, missing_debug_implementations, unreachable_pub)]
#![cfg_attr(test, deny(warnings))]
//! Asynchronous stream of bytes.
@@ -10,11 +10,90 @@
//! `Buf` (i.e, byte collections).
extern crate bytes;
#[cfg(feature = "util")]
extern crate either;
#[allow(unused)]
#[macro_use]
extern crate futures;
pub mod buf_stream;
mod never;
mod size_hint;
mod str;
mod u8;
#[cfg(feature = "util")]
pub mod util;
pub use self::size_hint::SizeHint;
#[doc(inline)]
pub use buf_stream::BufStream;
#[cfg(feature = "util")]
pub use util::BufStreamExt;
use bytes::Buf;
use futures::Poll;
/// An asynchronous stream of bytes.
///
/// `BufStream` asynchronously yields values implementing `Buf`, i.e. byte
/// buffers.
pub trait BufStream {
/// Values yielded by the `BufStream`.
///
/// Each item is a sequence of bytes representing a chunk of the total
/// `ByteStream`.
type Item: Buf;
/// The error type this `BufStream` might generate.
type Error;
/// Attempt to pull out the next buffer of this stream, registering the
/// current task for wakeup if the value is not yet available, and returning
/// `None` if the stream is exhausted.
///
/// # Return value
///
/// There are several possible return values, each indicating a distinct
/// stream state:
///
/// - `Ok(Async::NotReady)` means that this stream's next value is not ready
/// yet. Implementations will ensure that the current task will be notified
/// when the next value may be ready.
///
/// - `Ok(Async::Ready(Some(buf)))` means that the stream has successfully
/// produced a value, `buf`, and may produce further values on subsequent
/// `poll_buf` calls.
///
/// - `Ok(Async::Ready(None))` means that the stream has terminated, and
/// `poll_buf` should not be invoked again.
///
/// # Panics
///
/// Once a stream is finished, i.e. `Ready(None)` has been returned, further
/// calls to `poll_buf` may result in a panic or other "bad behavior".
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error>;
/// Returns the bounds on the remaining length of the stream.
///
/// The size hint allows the caller to perform certain optimizations that
/// are dependent on the byte stream size. For example, `collect` uses the
/// size hint to pre-allocate enough capacity to store the entirety of the
/// data received from the byte stream.
///
/// When `SizeHint::upper()` returns `Some` with a value equal to
/// `SizeHint::lower()`, this represents the exact number of bytes that will
/// be yielded by the `BufStream`.
///
/// # Implementation notes
///
/// While not enforced, implementations are expected to respect the values
/// returned from `SizeHint`. Any deviation is considered an implementation
/// bug. Consumers may rely on correctness in order to use the value as part
/// of protocol impelmentations. For example, an HTTP library may use the
/// size hint to set the `content-length` header.
///
/// However, `size_hint` must not be trusted to omit bounds checks in unsafe
/// code. An incorrect implementation of `size_hint()` must not lead to
/// memory safety violations.
fn size_hint(&self) -> SizeHint {
SizeHint::default()
}
}
+22
View File
@@ -0,0 +1,22 @@
use std::{error, fmt};
/// An error that can never occur
pub enum Never {}
impl fmt::Debug for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl fmt::Display for Never {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
match *self {}
}
}
impl error::Error for Never {
fn description(&self) -> &str {
match *self {}
}
}
@@ -29,7 +29,7 @@ impl SizeHint {
///
/// # Panics
///
/// The function panics if `value` is less than `upper`.
/// The function panics if `value` is greater than `upper`.
pub fn set_lower(&mut self, value: u64) {
assert!(value <= self.upper.unwrap_or(u64::MAX));
self.lower = value;
@@ -1,5 +1,6 @@
use never::Never;
use BufStream;
use buf_stream::errors::internal::Never;
use SizeHint;
use futures::Poll;
@@ -20,6 +21,10 @@ impl BufStream for String {
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
impl BufStream for &'static str {
@@ -36,4 +41,15 @@ impl BufStream for &'static str {
Ok(Some(buf).into())
}
fn size_hint(&self) -> SizeHint {
size_hint(&self[..])
}
}
fn size_hint(s: &str) -> SizeHint {
let mut hint = SizeHint::new();
hint.set_lower(s.len() as u64);
hint.set_upper(s.len() as u64);
hint
}
@@ -1,10 +1,8 @@
use BufStream;
use buf_stream::errors::internal::Never;
use bytes::{Bytes, BytesMut};
use futures::Poll;
use never::Never;
use std::io;
use BufStream;
impl BufStream for Vec<u8> {
type Item = io::Cursor<Vec<u8>>;
@@ -58,9 +56,7 @@ impl BufStream for BytesMut {
}
}
fn poll_bytes<T: Default>(buf: &mut T)
-> Poll<Option<io::Cursor<T>>, Never>
{
fn poll_bytes<T: Default>(buf: &mut T) -> Poll<Option<io::Cursor<T>>, Never> {
use std::mem;
let bytes = mem::replace(buf, Default::default());
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use either::Either;
use futures::Poll;
@@ -43,9 +43,4 @@ where
let res = try_ready!(self.right.poll_buf());
Ok(res.map(Either::Right).into())
}
fn size_hint(&self) -> SizeHint {
// TODO: Implement
SizeHint::default()
}
}
@@ -1,4 +1,5 @@
use super::{BufStream, FromBufStream};
use super::FromBufStream;
use BufStream;
use futures::{Future, Poll};
@@ -52,29 +53,26 @@ where
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
loop {
let res = self.stream.poll_buf()
.map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
let res = self.stream.poll_buf().map_err(|err| {
let inner = Error::Stream(err);
CollectError { inner }
});
match try_ready!(res) {
Some(mut buf) => {
let builder = self.builder.as_mut().expect("cannot poll after done");
U::extend(builder, &mut buf, &self.stream.size_hint())
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
U::extend(builder, &mut buf, &self.stream.size_hint()).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
}
None => {
let builder = self.builder.take().expect("cannot poll after done");
let value = U::build(builder)
.map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
let value = U::build(builder).map_err(|err| {
let inner = Error::Collect(err);
CollectError { inner }
})?;
return Ok(value.into());
}
}
@@ -1,7 +1,9 @@
use super::SizeHint;
use SizeHint;
use bytes::{Buf, BufMut};
use bytes::{Buf, BufMut, Bytes};
use std::error::Error;
use std::fmt;
use std::usize;
/// Conversion from a `BufStream`.
@@ -43,14 +45,22 @@ pub trait FromBufStream<T: Buf>: Sized {
/// Error returned from collecting into a `Vec<u8>`
#[derive(Debug)]
pub struct CollectVecError { _p: () }
pub struct CollectVecError {
_p: (),
}
/// Error returned from collecting into a `Bytes`
#[derive(Debug)]
pub struct CollectBytesError {
_p: (),
}
impl<T: Buf> FromBufStream<T> for Vec<u8> {
type Builder = Vec<u8>;
type Error = CollectVecError;
fn builder(_hint: &SizeHint) -> Vec<u8> {
Vec::new()
fn builder(hint: &SizeHint) -> Vec<u8> {
Vec::with_capacity(hint.lower() as usize)
}
fn extend(builder: &mut Self, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
@@ -70,7 +80,7 @@ impl<T: Buf> FromBufStream<T> for Vec<u8> {
Some(upper) if upper <= 64 => {
reserve = upper as usize;
}
_ => {},
_ => {}
}
// hint.lower() represents the minimum amount of data that will be
@@ -108,3 +118,45 @@ impl<T: Buf> FromBufStream<T> for Vec<u8> {
Ok(builder)
}
}
impl<T: Buf> FromBufStream<T> for Bytes {
type Builder = Vec<u8>;
type Error = CollectBytesError;
fn builder(hint: &SizeHint) -> Vec<u8> {
<Vec<u8> as FromBufStream<T>>::builder(hint)
}
fn extend(builder: &mut Vec<u8>, buf: &mut T, hint: &SizeHint) -> Result<(), Self::Error> {
<Vec<u8> as FromBufStream<T>>::extend(builder, buf, hint)
.map_err(|_| CollectBytesError { _p: () })
}
fn build(builder: Vec<u8>) -> Result<Self, Self::Error> {
Ok(builder.into())
}
}
impl fmt::Display for CollectVecError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "BufStream is too big")
}
}
impl Error for CollectVecError {
fn description(&self) -> &str {
"BufStream too big"
}
}
impl fmt::Display for CollectBytesError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "BufStream too big")
}
}
impl Error for CollectBytesError {
fn description(&self) -> &str {
"BufStream too big"
}
}
+54
View File
@@ -0,0 +1,54 @@
use bytes::Buf;
use futures::Poll;
use std::error::Error;
use std::fmt;
use BufStream;
/// Converts an `Iterator` into a `BufStream` which is always ready to yield the
/// next value.
///
/// Iterators in Rust don't express the ability to block, so this adapter
/// simply always calls `iter.next()` and returns that.
pub fn iter<I>(i: I) -> Iter<I::IntoIter>
where
I: IntoIterator,
I::Item: Buf,
{
Iter {
iter: i.into_iter(),
}
}
/// `BufStream` returned by the [`iter`] function.
#[derive(Debug)]
pub struct Iter<I> {
iter: I,
}
#[derive(Debug)]
pub enum Never {}
impl<I> BufStream for Iter<I>
where
I: Iterator,
I::Item: Buf,
{
type Item = I::Item;
type Error = Never;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
Ok(self.iter.next().into())
}
}
impl fmt::Display for Never {
fn fmt(&self, _: &mut fmt::Formatter) -> fmt::Result {
unreachable!();
}
}
impl Error for Never {
fn description(&self) -> &str {
unreachable!();
}
}
@@ -1,4 +1,4 @@
use super::{BufStream, SizeHint};
use BufStream;
use bytes::Buf;
use futures::Poll;
@@ -40,10 +40,10 @@ where
return Err(LimitError { inner: None });
}
let res = self.stream.poll_buf()
.map_err(|err| {
LimitError { inner: Some(err) }
});
let res = self
.stream
.poll_buf()
.map_err(|err| LimitError { inner: Some(err) });
match res {
Ok(Ready(Some(ref buf))) => {
@@ -59,22 +59,6 @@ where
res
}
fn size_hint(&self) -> SizeHint {
let mut hint = self.stream.size_hint();
let upper = hint.upper()
.map(|upper| upper.min(self.remaining))
.unwrap_or(self.remaining);
hint.set_upper(upper);
hint
}
fn consume_hint(&mut self, amount: usize) {
// TODO: Should this be capped by `self.remaining`?
self.stream.consume_hint(amount)
}
}
// ===== impl LimitError =====
+87
View File
@@ -0,0 +1,87 @@
//! Types and utilities for working with `BufStream`.
mod chain;
mod collect;
mod from;
mod iter;
mod limit;
mod stream;
pub use self::chain::Chain;
pub use self::collect::Collect;
pub use self::from::FromBufStream;
pub use self::iter::iter;
pub use self::limit::Limit;
pub use self::stream::{stream, IntoStream};
pub mod error {
//! Error types
pub use super::collect::CollectError;
pub use super::from::{CollectBytesError, CollectVecError};
pub use super::limit::LimitError;
}
use BufStream;
impl<T> BufStreamExt for T where T: BufStream {}
/// An extension trait for `BufStream`'s that provides a variety of convenient
/// adapters.
pub trait BufStreamExt: BufStream {
/// Takes two buf streams and creates a new buf stream over both in
/// sequence.
///
/// `chain()` returns a new `BufStream` value which will first yield all
/// data from `self` then all data from `other`.
///
/// In other words, it links two buf streams together, in a chain.
fn chain<T>(self, other: T) -> Chain<Self, T>
where
Self: Sized,
T: BufStream<Error = Self::Error>,
{
Chain::new(self, other)
}
/// Consumes all data from `self`, storing it in byte storage of type `T`.
///
/// `collect()` returns a future that buffers all data yielded from `self`
/// into storage of type of `T`. The future completes once `self` yield
/// `None`, returning the buffered data.
///
/// The collect future will yield an error if `self` yields an error or if
/// the collect operation errors. The collect error cases are dependent on
/// the target storage type.
fn collect<T>(self) -> Collect<Self, T>
where
Self: Sized,
T: FromBufStream<Self::Item>,
{
Collect::new(self)
}
/// Limit the number of bytes that the stream can yield.
///
/// `limit()` returns a new `BufStream` value which yields all the data from
/// `self` while ensuring that at most `amount` bytes are yielded.
///
/// If `self` can yield greater than `amount` bytes, the returned stream
/// will yield an error.
fn limit(self, amount: u64) -> Limit<Self>
where
Self: Sized,
{
Limit::new(self, amount)
}
/// Creates a `Stream` from a `BufStream`.
///
/// This produces a `Stream` of `BufStream::Items`.
fn into_stream(self) -> IntoStream<Self>
where
Self: Sized,
{
IntoStream::new(self)
}
}
+76
View File
@@ -0,0 +1,76 @@
use bytes::Buf;
use futures::{Async, Poll, Stream};
use BufStream;
/// Converts a `Stream` of `Buf` types into a `BufStream`.
///
/// While `Stream` and `BufStream` are very similar, they are not identical. The
/// `stream` function returns a `BufStream` that is backed by the provided
/// `Stream` type.
pub fn stream<T>(stream: T) -> FromStream<T>
where
T: Stream,
T::Item: Buf,
{
FromStream { stream }
}
/// `BufStream` returned by the [`stream`] function.
#[derive(Debug)]
pub struct FromStream<T> {
stream: T,
}
impl<T> BufStream for FromStream<T>
where
T: Stream,
T::Item: Buf,
{
type Item = T::Item;
type Error = T::Error;
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
self.stream.poll()
}
}
/// Converts a `BufStream` into a `Stream`.
#[derive(Debug)]
pub struct IntoStream<T> {
buf: T,
}
impl<T> IntoStream<T> {
/// Create a new `Stream` from the provided `BufStream`.
pub fn new(buf: T) -> Self {
IntoStream { buf }
}
/// Get a reference to the inner `BufStream`.
pub fn get_ref(&self) -> &T {
&self.buf
}
/// Get a mutable reference to the inner `BufStream`
pub fn get_mut(&mut self) -> &mut T {
&mut self.buf
}
/// Get the inner `BufStream`.
pub fn into_inner(self) -> T {
self.buf
}
}
impl<T: BufStream> Stream for IntoStream<T> {
type Item = T::Item;
type Error = T::Error;
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.buf.poll_buf()? {
Async::Ready(Some(buf)) => Ok(Async::Ready(Some(buf))),
Async::Ready(None) => Ok(Async::Ready(None)),
Async::NotReady => Ok(Async::NotReady),
}
}
}
+4 -326
View File
@@ -1,329 +1,7 @@
extern crate tokio_buf;
extern crate bytes;
extern crate futures;
use tokio_buf::buf_stream::{BufStream, SizeHint};
use bytes::Buf;
use futures::{Future, Poll};
use futures::Async::*;
use tokio_buf::BufStream;
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
}
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
}
}
// ===== test `SizeHint` =====
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
// ===== test `chain()` =====
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"])
.chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[
Ok(Ready("foo")),
Ok(NotReady),
Ok(Ready("bar"))
]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
// ===== Test `collect()` =====
#[test]
fn collect_vec() {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world");
assert_eq!(vec.capacity(), 64);
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: Vec<u8> = bs.collect()
.wait().unwrap();
assert_eq!(vec, b"hello world, one two three");
}
// ===== Test limit() =====
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait().unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world")
.limit(5)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let res = one("hello world")
.limit(10)
.collect::<Vec<_>>()
.wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"])
.limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
// ===== BufStream impelmentations for misc types =====
#[test]
fn str_buf_stream() {
let mut bs = "hello world".to_string();
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
let mut bs = "hello world";
assert_buf_eq!(bs.poll_buf(), "hello world");
assert!(bs.is_empty());
assert_none!(bs.poll_buf());
}
// ===== Test utils =====
fn one(buf: &'static str) -> Mock {
list(&[buf])
}
fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
struct Mock {
polls: VecDeque<Poll<&'static [u8], ()>>,
size_hint: SizeHint,
}
#[derive(Debug)]
struct MockBuf {
data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
// Ensures that `BufStream` can be a trait object
#[allow(dead_code)]
fn obj(_: &mut BufStream<Item = u32, Error = ()>) {}
+43
View File
@@ -0,0 +1,43 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
#[test]
fn chain() {
// Chain one with one
//
let mut bs = one("hello").chain(one("world"));
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), "world");
assert_none!(bs.poll_buf());
// Chain multi with multi
let mut bs = list(&["foo", "bar"]).chain(list(&["baz", "bok"]));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_buf_eq!(bs.poll_buf(), "bok");
assert_none!(bs.poll_buf());
// Chain includes a not ready call
//
let mut bs = new_mock(&[Ok(Ready("foo")), Ok(NotReady), Ok(Ready("bar"))]).chain(one("baz"));
assert_buf_eq!(bs.poll_buf(), "foo");
assert_not_ready!(bs.poll_buf());
assert_buf_eq!(bs.poll_buf(), "bar");
assert_buf_eq!(bs.poll_buf(), "baz");
assert_none!(bs.poll_buf());
}
+68
View File
@@ -0,0 +1,68 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Bytes;
use futures::Future;
use tokio_buf::BufStreamExt;
#[macro_use]
mod support;
use support::*;
macro_rules! test_collect_impl {
($t:ty $(, $capacity:ident)*) => {
// While unfortunate, this test makes some assumptions on vec's resizing
// behavior.
//
// Collect one
//
let bs = one("hello world");
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(11);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect one, with size hint
//
let mut bs = one("hello world");
bs.size_hint.set_lower(10);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world"[..]);
$( assert_eq!(vec.$capacity(), 64); )*
// Collect many
//
let bs = list(&["hello", " ", "world", ", one two three"]);
let vec: $t = bs.collect().wait().unwrap();
assert_eq!(vec, &b"hello world, one two three"[..]);
}
}
#[test]
fn collect_vec() {
test_collect_impl!(Vec<u8>, capacity);
}
#[test]
fn collect_bytes() {
test_collect_impl!(Bytes);
}
+33
View File
@@ -0,0 +1,33 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
#[macro_use]
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_iter() {
let mut bs = util::iter(Vec::<Buf>::new());
assert_none!(bs.poll_buf());
}
#[test]
fn full_iter() {
let bufs = vec![buf(b"one"), buf(b"two"), buf(b"three")];
let mut bs = util::iter(bufs);
assert_buf_eq!(bs.poll_buf(), "one");
assert_buf_eq!(bs.poll_buf(), "two");
assert_buf_eq!(bs.poll_buf(), "three");
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}
+65
View File
@@ -0,0 +1,65 @@
#![cfg(feature = "util")]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use futures::Future;
use tokio_buf::{BufStream, BufStreamExt};
#[macro_use]
mod support;
use support::*;
#[test]
fn limit() {
// Not limited
let res = one("hello world")
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(100)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
let res = list(&["hello", " ", "world"])
.limit(11)
.collect::<Vec<_>>()
.wait()
.unwrap();
assert_eq!(res, b"hello world");
// Limited
let res = one("hello world").limit(5).collect::<Vec<_>>().wait();
assert!(res.is_err());
let res = one("hello world").limit(10).collect::<Vec<_>>().wait();
assert!(res.is_err());
let mut bs = list(&["hello", " ", "world"]).limit(9);
assert_buf_eq!(bs.poll_buf(), "hello");
assert_buf_eq!(bs.poll_buf(), " ");
assert!(bs.poll_buf().is_err());
let mut bs = list(&["hello", " ", "world"]);
bs.size_hint.set_lower(11);
let mut bs = bs.limit(9);
assert!(bs.poll_buf().is_err());
}
+42
View File
@@ -0,0 +1,42 @@
extern crate tokio_buf;
use tokio_buf::SizeHint;
#[test]
fn size_hint() {
let hint = SizeHint::new();
assert_eq!(hint.lower(), 0);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_lower(100);
assert_eq!(hint.lower(), 100);
assert!(hint.upper().is_none());
let mut hint = SizeHint::new();
hint.set_upper(200);
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(200));
let mut hint = SizeHint::new();
hint.set_lower(100);
hint.set_upper(100);
assert_eq!(hint.lower(), 100);
assert_eq!(hint.upper(), Some(100));
}
#[test]
#[should_panic]
fn size_hint_lower_bigger_than_upper() {
let mut hint = SizeHint::new();
hint.set_upper(100);
hint.set_lower(200);
}
#[test]
#[should_panic]
fn size_hint_upper_less_than_lower() {
let mut hint = SizeHint::new();
hint.set_lower(200);
hint.set_upper(100);
}
+49
View File
@@ -0,0 +1,49 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
extern crate tokio_mock_task;
use futures::sync::mpsc;
use futures::Async::*;
use std::io::Cursor;
use tokio_buf::{util, BufStream};
use tokio_mock_task::MockTask;
#[macro_use]
mod support;
type Buf = Cursor<&'static [u8]>;
#[test]
fn empty_stream() {
let (_, rx) = mpsc::unbounded::<Buf>();
let mut bs = util::stream(rx);
assert_none!(bs.poll_buf());
}
#[test]
fn full_stream() {
let (tx, rx) = mpsc::unbounded();
let mut bs = util::stream(rx);
let mut task = MockTask::new();
tx.unbounded_send(buf(b"one")).unwrap();
assert_buf_eq!(bs.poll_buf(), "one");
task.enter(|| assert_not_ready!(bs.poll_buf()));
tx.unbounded_send(buf(b"two")).unwrap();
assert!(task.is_notified());
assert_buf_eq!(bs.poll_buf(), "two");
task.enter(|| assert_not_ready!(bs.poll_buf()));
drop(tx);
assert!(task.is_notified());
assert_none!(bs.poll_buf());
}
fn buf(data: &'static [u8]) -> Buf {
Cursor::new(data)
}
+38
View File
@@ -0,0 +1,38 @@
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use futures::Async::*;
use std::fmt;
use tokio_buf::BufStream;
#[macro_use]
mod support;
fn test_hello_world<B>(mut bs: B)
where
B: BufStream + fmt::Debug,
B::Item: fmt::Debug,
B::Error: fmt::Debug,
{
let hint = bs.size_hint();
assert_eq!(hint.lower(), 11);
assert_eq!(hint.upper(), Some(11));
assert_buf_eq!(bs.poll_buf(), "hello world");
let hint = bs.size_hint();
assert_eq!(hint.lower(), 0);
assert_eq!(hint.upper(), Some(0));
assert_none!(bs.poll_buf());
}
#[test]
fn string() {
test_hello_world("hello world".to_string());
}
#[test]
fn str() {
test_hello_world("hello world");
}
+133
View File
@@ -0,0 +1,133 @@
#![allow(unused)]
extern crate bytes;
extern crate futures;
extern crate tokio_buf;
use bytes::Buf;
use futures::Async::*;
use futures::Poll;
use tokio_buf::{BufStream, SizeHint};
use std::collections::VecDeque;
use std::io::Cursor;
macro_rules! assert_buf_eq {
($actual:expr, $expect:expr) => {{
use bytes::Buf;
match $actual {
Ok(Ready(Some(val))) => {
assert_eq!(val.remaining(), val.bytes().len());
assert_eq!(val.bytes(), $expect.as_bytes());
}
Ok(Ready(None)) => panic!("expected value; BufStream yielded None"),
Ok(NotReady) => panic!("expected value; BufStream is not ready"),
Err(e) => panic!("expected value; got error = {:?}", e),
}
}};
}
macro_rules! assert_none {
($actual:expr) => {
match $actual {
Ok(Ready(None)) => {}
actual => panic!("expected None; actual = {:?}", actual),
}
};
}
macro_rules! assert_not_ready {
($actual:expr) => {
match $actual {
Ok(NotReady) => {}
actual => panic!("expected NotReady; actual = {:?}", actual),
}
};
}
// ===== Test utils =====
pub fn one(buf: &'static str) -> Mock {
list(&[buf])
}
pub fn list(bufs: &[&'static str]) -> Mock {
let mut polls = VecDeque::new();
for &buf in bufs {
polls.push_back(Ok(Ready(buf.as_bytes())));
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
pub fn new_mock(values: &[Poll<&'static str, ()>]) -> Mock {
let mut polls = VecDeque::new();
for &v in values {
polls.push_back(match v {
Ok(Ready(v)) => Ok(Ready(v.as_bytes())),
Ok(NotReady) => Ok(NotReady),
Err(e) => Err(e),
});
}
Mock {
polls,
size_hint: SizeHint::default(),
}
}
#[derive(Debug)]
pub struct Mock {
pub polls: VecDeque<Poll<&'static [u8], ()>>,
pub size_hint: SizeHint,
}
#[derive(Debug)]
pub struct MockBuf {
pub data: Cursor<&'static [u8]>,
}
impl BufStream for Mock {
type Item = MockBuf;
type Error = ();
fn poll_buf(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
match self.polls.pop_front() {
Some(Ok(Ready(value))) => Ok(Ready(Some(MockBuf::new(value)))),
Some(Ok(NotReady)) => Ok(NotReady),
Some(Err(e)) => Err(e),
None => Ok(Ready(None)),
}
}
fn size_hint(&self) -> SizeHint {
self.size_hint.clone()
}
}
impl MockBuf {
fn new(data: &'static [u8]) -> MockBuf {
MockBuf {
data: Cursor::new(data),
}
}
}
impl Buf for MockBuf {
fn remaining(&self) -> usize {
self.data.remaining()
}
fn bytes(&self) -> &[u8] {
self.data.bytes()
}
fn advance(&mut self, cnt: usize) {
self.data.advance(cnt)
}
}
+1 -1
View File
@@ -18,6 +18,6 @@ Utilities for encoding and decoding frames.
categories = ["asynchronous"]
[dependencies]
tokio-io = { version = "0.1.7", path = "../tokio-io" }
tokio-io = "0.1.7"
bytes = "0.4.7"
futures = "0.1.18"
+5 -3
View File
@@ -1,6 +1,6 @@
use bytes::{Bytes, BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use bytes::{BufMut, Bytes, BytesMut};
use std::io;
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that just ships bytes around.
#[derive(Copy, Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -8,7 +8,9 @@ pub struct BytesCodec(());
impl BytesCodec {
/// Creates a new `BytesCodec` for shipping around raw bytes.
pub fn new() -> BytesCodec { BytesCodec(()) }
pub fn new() -> BytesCodec {
BytesCodec(())
}
}
impl Decoder for BytesCodec {
+1 -8
View File
@@ -19,14 +19,7 @@ extern crate tokio_io;
mod bytes_codec;
mod lines_codec;
pub use tokio_io::_tokio_codec::{
Decoder,
Encoder,
Framed,
FramedParts,
FramedRead,
FramedWrite,
};
pub use tokio_io::_tokio_codec::{Decoder, Encoder, Framed, FramedParts, FramedRead, FramedWrite};
pub use bytes_codec::BytesCodec;
pub use lines_codec::LinesCodec;
+4 -6
View File
@@ -1,6 +1,6 @@
use bytes::{BufMut, BytesMut};
use tokio_io::_tokio_codec::{Encoder, Decoder};
use std::{cmp, io, str, usize};
use tokio_io::_tokio_codec::{Decoder, Encoder};
/// A simple `Codec` implementation that splits up data into lines.
#[derive(Clone, Debug, Eq, PartialEq, Ord, PartialOrd, Hash)]
@@ -103,10 +103,8 @@ impl LinesCodec {
}
fn utf8(buf: &[u8]) -> Result<&str, io::Error> {
str::from_utf8(buf).map_err(|_|
io::Error::new(
io::ErrorKind::InvalidData,
"Unable to decode input as UTF8"))
str::from_utf8(buf)
.map_err(|_| io::Error::new(io::ErrorKind::InvalidData, "Unable to decode input as UTF8"))
}
fn without_carriage_return(s: &[u8]) -> &[u8] {
@@ -153,7 +151,7 @@ impl Decoder for LinesCodec {
self.is_discarding = true;
Err(io::Error::new(
io::ErrorKind::Other,
"line length limit exceeded"
"line length limit exceeded",
))
} else {
// We didn't find a line or reach the length limit, so the next
+33 -9
View File
@@ -1,8 +1,8 @@
extern crate tokio_codec;
extern crate bytes;
extern crate tokio_codec;
use bytes::{BytesMut, Bytes, BufMut};
use tokio_codec::{BytesCodec, LinesCodec, Decoder, Encoder};
use bytes::{BufMut, Bytes, BytesMut};
use tokio_codec::{BytesCodec, Decoder, Encoder, LinesCodec};
#[test]
fn bytes_decoder() {
@@ -27,13 +27,17 @@ fn bytes_encoder() {
const INLINE_CAP: usize = 4 * 4 - 1;
let mut buf = BytesMut::new();
codec.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INLINE_CAP + 1]), &mut buf)
.unwrap();
// Default capacity of Framed Read
const INITIAL_CAPACITY: usize = 8 * 1024;
let mut buf = BytesMut::with_capacity(INITIAL_CAPACITY);
codec.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf).unwrap();
codec
.encode(Bytes::from_static(&[0; INITIAL_CAPACITY + 1]), &mut buf)
.unwrap();
}
#[test]
@@ -68,17 +72,32 @@ fn lines_decoder_max_length() {
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 2", line);
assert!(codec.decode(buf).is_err());
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("line 4", line);
let line = codec.decode(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("", line);
assert_eq!(None, codec.decode(buf).unwrap());
@@ -87,7 +106,12 @@ fn lines_decoder_max_length() {
assert_eq!(None, codec.decode(buf).unwrap());
let line = codec.decode_eof(buf).unwrap().unwrap();
assert!(line.len() <= MAX_LENGTH, "{:?}.len() <= {:?}", line, MAX_LENGTH);
assert!(
line.len() <= MAX_LENGTH,
"{:?}.len() <= {:?}",
line,
MAX_LENGTH
);
assert_eq!("\rk", line);
assert_eq!(None, codec.decode(buf).unwrap());
+10 -10
View File
@@ -1,13 +1,13 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use futures::{Stream, Future};
use bytes::{Buf, BufMut, BytesMut, IntoBuf};
use futures::{Future, Stream};
use std::io::{self, Read};
use tokio_codec::{Framed, FramedParts, Decoder, Encoder};
use tokio_codec::{Decoder, Encoder, Framed, FramedParts};
use tokio_io::AsyncRead;
use bytes::{BytesMut, Buf, BufMut, IntoBuf};
const INITIAL_CAPACITY: usize = 8 * 1024;
@@ -45,8 +45,10 @@ struct DontReadIntoThis;
impl Read for DontReadIntoThis {
fn read(&mut self, _: &mut [u8]) -> io::Result<usize> {
Err(io::Error::new(io::ErrorKind::Other,
"Read into something you weren't supposed to."))
Err(io::Error::new(
io::ErrorKind::Other,
"Read into something you weren't supposed to.",
))
}
}
@@ -61,9 +63,7 @@ fn can_read_from_existing_buf() {
let num = framed
.into_future()
.map(|(first_num, _)| {
first_num.unwrap()
})
.map(|(first_num, _)| first_num.unwrap())
.wait()
.map_err(|e| e.0)
.unwrap();
+7 -8
View File
@@ -1,17 +1,17 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_codec::{Decoder, FramedRead};
use tokio_io::AsyncRead;
use tokio_codec::{FramedRead, Decoder};
use bytes::{BytesMut, Buf, IntoBuf};
use bytes::{Buf, BytesMut, IntoBuf};
use futures::Async::{NotReady, Ready};
use futures::Stream;
use futures::Async::{Ready, NotReady};
use std::io::{self, Read};
use std::collections::VecDeque;
use std::io::{self, Read};
macro_rules! mock {
($($x:expr,)*) => {{
@@ -212,5 +212,4 @@ impl Read for Mock {
}
}
impl AsyncRead for Mock {
}
impl AsyncRead for Mock {}
+6 -6
View File
@@ -1,16 +1,16 @@
extern crate tokio_codec;
extern crate tokio_io;
extern crate bytes;
extern crate futures;
extern crate tokio_codec;
extern crate tokio_io;
use tokio_io::AsyncWrite;
use tokio_codec::{Encoder, FramedWrite};
use tokio_io::AsyncWrite;
use futures::{Sink, Poll};
use bytes::{BytesMut, BufMut};
use bytes::{BufMut, BytesMut};
use futures::{Poll, Sink};
use std::io::{self, Write};
use std::collections::VecDeque;
use std::io::{self, Write};
macro_rules! mock {
($($x:expr,)*) => {{
+10
View File
@@ -1,3 +1,13 @@
# 0.1.6 (March 22, 2019)
### Added
- implement `TypedExecutor` (#993).
# 0.1.5 (March 1, 2019)
### Fixed
- Documentation typos (#882).
# 0.1.4 (November 21, 2018)
* Fix shutdown on idle (#763).
+6 -4
View File
@@ -3,11 +3,13 @@ name = "tokio-current-thread"
# When releasing to crates.io:
# - Update html_root_url.
# - Update doc URL
# - Update doc url
# - Cargo.toml
# - README.md
# - Update CHANGELOG.md.
# - Create "v0.1.x" git tag.
version = "0.1.4"
documentation = "https://docs.rs/tokio-current-thread/0.1.4/tokio_current_thread"
version = "0.1.6"
documentation = "https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -19,5 +21,5 @@ keywords = ["futures", "tokio"]
categories = ["concurrency", "asynchronous"]
[dependencies]
tokio-executor = { version = "0.1.5", path = "../tokio-executor" }
tokio-executor = "0.1.7"
futures = "0.1.19"
+1 -1
View File
@@ -2,7 +2,7 @@
Single threaded executor for Tokio.
[Documentation](https://tokio-rs.github.io/tokio/tokio_current_thread/)
[Documentation](https://docs.rs/tokio-current-thread/0.1.6/tokio_current_thread/)
## Overview
+102 -93
View File
@@ -1,4 +1,4 @@
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.4")]
#![doc(html_root_url = "https://docs.rs/tokio-current-thread/0.1.6")]
#![deny(warnings, missing_docs, missing_debug_implementations)]
//! A single-threaded executor which executes tasks on the same thread from which
@@ -32,19 +32,19 @@ mod scheduler;
use self::scheduler::Scheduler;
use tokio_executor::park::{Park, ParkThread, Unpark};
use tokio_executor::{Enter, SpawnError};
use tokio_executor::park::{Park, Unpark, ParkThread};
use futures::future::{ExecuteError, ExecuteErrorKind, Executor};
use futures::{executor, Async, Future};
use futures::future::{Executor, ExecuteError, ExecuteErrorKind};
use std::fmt;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::rc::Rc;
use std::sync::{atomic, mpsc, Arc};
use std::time::{Duration, Instant};
use std::thread;
use std::time::{Duration, Instant};
/// Executes tasks on the current thread
pub struct CurrentThread<P: Park = ParkThread> {
@@ -86,7 +86,7 @@ pub struct TaskExecutor {
/// Returned by the `turn` function.
#[derive(Debug)]
pub struct Turn {
polled: bool
polled: bool,
}
impl Turn {
@@ -194,17 +194,21 @@ struct CurrentRunner {
id: Cell<Option<u64>>,
}
/// Current thread's task runner. This is set in `TaskRunner::with`
thread_local!(static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
});
thread_local! {
/// Current thread's task runner. This is set in `TaskRunner::with`
static CURRENT: CurrentRunner = CurrentRunner {
spawn: Cell::new(None),
id: Cell::new(None),
}
}
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one referred
/// to by a `Handle` so that direct task dispatch can be used.
thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
thread_local! {
/// Unique ID to assign to each new executor launched on this thread.
///
/// The unique ID is used to determine if the currently running executor matches the one
/// referred to by a `Handle` so that direct task dispatch can be used.
static EXECUTOR_ID: Cell<u64> = Cell::new(0)
}
/// Run the executor bootstrapping the execution with the provided future.
///
@@ -222,7 +226,8 @@ thread_local!(static EXECUTOR_ID: Cell<u64> = Cell::new(0));
/// [`CurrentThread`]: struct.CurrentThread.html
/// [mod]: index.html
pub fn block_on_all<F>(future: F) -> Result<F::Item, F::Error>
where F: Future,
where
F: Future,
{
let mut current_thread = CurrentThread::new();
@@ -246,7 +251,8 @@ where F: Future,
///
/// [`tokio::spawn`]: ../fn.spawn.html
pub fn spawn<F>(future: F)
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
TaskExecutor::current()
.spawn_local(Box::new(future))
@@ -312,7 +318,8 @@ impl<P: Park> CurrentThread<P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.borrow().spawn_local(Box::new(future), false);
self
@@ -331,41 +338,33 @@ impl<P: Park> CurrentThread<P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).block_on(future)
}
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run()
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).run_timeout(duration)
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
let mut enter = tokio_executor::enter()
.expect("failed to start `current_thread::Runtime`");
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let mut enter = tokio_executor::enter().expect("failed to start `current_thread::Runtime`");
self.enter(&mut enter).turn(duration)
}
@@ -432,11 +431,24 @@ impl tokio_executor::Executor for CurrentThread {
}
}
impl<T> tokio_executor::TypedExecutor<T> for CurrentThread
where
T: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
self.borrow().spawn_local(Box::new(future), false);
Ok(())
}
}
impl<P: Park> fmt::Debug for CurrentThread<P> {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("CurrentThread")
.field("scheduler", &self.scheduler)
.field("num_futures", &self.num_futures.load(atomic::Ordering::SeqCst))
.field(
"num_futures",
&self.num_futures.load(atomic::Ordering::SeqCst),
)
.finish()
}
}
@@ -448,7 +460,8 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// This internally queues the future to be executed once `run` is called.
pub fn spawn<F>(&mut self, future: F) -> &mut Self
where F: Future<Item = (), Error = ()> + 'static,
where
F: Future<Item = (), Error = ()> + 'static,
{
self.executor.borrow().spawn_local(Box::new(future), false);
self
@@ -467,17 +480,18 @@ impl<'a, P: Park> Entered<'a, P> {
///
/// The caller is responsible for ensuring that other spawned futures
/// complete execution.
pub fn block_on<F>(&mut self, future: F)
-> Result<F::Item, BlockError<F::Error>>
where F: Future
pub fn block_on<F>(&mut self, future: F) -> Result<F::Item, BlockError<F::Error>>
where
F: Future,
{
let mut future = executor::spawn(future);
let notify = self.executor.scheduler.notify();
loop {
let res = self.executor.borrow().enter(self.enter, || {
future.poll_future_notify(&notify, 0)
});
let res = self
.executor
.borrow()
.enter(self.enter, || future.poll_future_notify(&notify, 0));
match res {
Ok(Async::Ready(e)) => return Ok(e),
@@ -496,24 +510,19 @@ impl<'a, P: Park> Entered<'a, P> {
/// Run the executor to completion, blocking the thread until **all**
/// spawned futures have completed.
pub fn run(&mut self) -> Result<(), RunError> {
self.run_timeout2(None)
.map_err(|_| RunError { _p: () })
self.run_timeout2(None).map_err(|_| RunError { _p: () })
}
/// Run the executor to completion, blocking the thread until all
/// spawned futures have completed **or** `duration` time has elapsed.
pub fn run_timeout(&mut self, duration: Duration)
-> Result<(), RunTimeoutError>
{
pub fn run_timeout(&mut self, duration: Duration) -> Result<(), RunTimeoutError> {
self.run_timeout2(Some(duration))
}
/// Perform a single iteration of the event loop.
///
/// This function blocks the current thread even if the executor is idle.
pub fn turn(&mut self, duration: Option<Duration>)
-> Result<Turn, TurnError>
{
pub fn turn(&mut self, duration: Option<Duration>) -> Result<Turn, TurnError> {
let res = if self.executor.scheduler.has_pending_futures() {
self.executor.park.park_timeout(Duration::from_millis(0))
} else {
@@ -542,9 +551,7 @@ impl<'a, P: Park> Entered<'a, P> {
&mut self.executor.park
}
fn run_timeout2(&mut self, dur: Option<Duration>)
-> Result<(), RunTimeoutError>
{
fn run_timeout2(&mut self, dur: Option<Duration>) -> Result<(), RunTimeoutError> {
if self.executor.is_idle() {
// Nothing to do
return Ok(());
@@ -602,10 +609,9 @@ impl<'a, P: Park> Entered<'a, P> {
}
// After any pending futures were scheduled, do the actual tick
borrow.scheduler.tick(
borrow.id,
&mut *self.enter,
borrow.num_futures)
borrow
.scheduler
.tick(borrow.id, &mut *self.enter, borrow.num_futures)
}
}
@@ -676,7 +682,8 @@ impl Handle {
return Err(SpawnError::shutdown());
}
self.sender.send(Box::new(future))
self.sender
.send(Box::new(future))
.expect("CurrentThread does not exist anymore");
// use 0 for the id, CurrentThread does not make use of it
self.notify.notify(0);
@@ -718,51 +725,53 @@ impl TaskExecutor {
/// Get the current executor's thread-local ID.
fn id(&self) -> Option<u64> {
CURRENT.with(|current| {
current.id.get()
})
CURRENT.with(|current| current.id.get())
}
/// Spawn a future onto the current `CurrentThread` instance.
pub fn spawn_local(&mut self, future: Box<Future<Item = (), Error = ()>>)
-> Result<(), SpawnError>
{
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => {
Err(SpawnError::shutdown())
}
pub fn spawn_local(
&mut self,
future: Box<Future<Item = (), Error = ()>>,
) -> Result<(), SpawnError> {
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(future, false) };
Ok(())
}
None => Err(SpawnError::shutdown()),
})
}
}
impl tokio_executor::Executor for TaskExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
self.spawn_local(future)
}
}
impl<F> tokio_executor::TypedExecutor<F> for TaskExecutor
where
F: Future<Item = (), Error = ()> + 'static,
{
fn spawn(&mut self, future: F) -> Result<(), SpawnError> {
self.spawn_local(Box::new(future))
}
}
impl<F> Executor<F> for TaskExecutor
where F: Future<Item = (), Error = ()> + 'static
where
F: Future<Item = (), Error = ()> + 'static,
{
fn execute(&self, future: F) -> Result<(), ExecuteError<F>> {
CURRENT.with(|current| {
match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => {
Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future))
}
CURRENT.with(|current| match current.spawn.get() {
Some(spawn) => {
unsafe { (*spawn).spawn_local(Box::new(future), false) };
Ok(())
}
None => Err(ExecuteError::new(ExecuteErrorKind::Shutdown, future)),
})
}
}
@@ -771,13 +780,12 @@ where F: Future<Item = (), Error = ()> + 'static
impl<'a, U: Unpark> Borrow<'a, U> {
fn enter<F, R>(&mut self, _: &mut Enter, f: F) -> R
where F: FnOnce() -> R,
where
F: FnOnce() -> R,
{
CURRENT.with(|current| {
current.id.set(Some(self.id));
current.set_spawn(self, || {
f()
})
current.set_spawn(self, || f())
})
}
}
@@ -797,7 +805,8 @@ impl<'a, U: Unpark> SpawnLocal for Borrow<'a, U> {
impl CurrentRunner {
fn set_spawn<F, R>(&self, spawn: &mut SpawnLocal, f: F) -> R
where F: FnOnce() -> R
where
F: FnOnce() -> R,
{
struct Reset<'a>(&'a CurrentRunner);
+17 -21
View File
@@ -1,20 +1,20 @@
use super::Borrow;
use tokio_executor::Enter;
use tokio_executor::park::Unpark;
use tokio_executor::Enter;
use futures::{Future, Async};
use futures::executor::{self, Spawn, UnsafeNotify, NotifyHandle};
use futures::executor::{self, NotifyHandle, Spawn, UnsafeNotify};
use futures::{Async, Future};
use std::cell::UnsafeCell;
use std::fmt::{self, Debug};
use std::marker::PhantomData;
use std::mem;
use std::ptr;
use std::sync::atomic::Ordering::{Relaxed, SeqCst, Acquire, Release, AcqRel};
use std::sync::atomic::Ordering::{AcqRel, Acquire, Relaxed, Release, SeqCst};
use std::sync::atomic::{AtomicBool, AtomicPtr, AtomicUsize};
use std::sync::{Arc, Weak};
use std::usize;
use std::thread;
use std::marker::PhantomData;
use std::usize;
/// A generic task-aware scheduler.
///
@@ -135,7 +135,8 @@ pub struct Scheduled<'a, U: 'a> {
}
impl<U> Scheduler<U>
where U: Unpark,
where
U: Unpark,
{
/// Constructs a new, empty `Scheduler`
///
@@ -200,9 +201,7 @@ where U: Unpark,
pub fn has_pending_futures(&mut self) -> bool {
// See function definition for why the unsafe is needed and
// correctly used here
unsafe {
self.inner.has_pending_futures()
}
unsafe { self.inner.has_pending_futures() }
}
/// Advance the scheduler state, returning `true` if any futures were
@@ -210,11 +209,9 @@ where U: Unpark,
///
/// This function should be called whenever the caller is notified via a
/// wakeup.
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool
{
pub fn tick(&mut self, eid: u64, enter: &mut Enter, num_futures: &AtomicUsize) -> bool {
let mut ret = false;
let tick = self.inner.tick_num.fetch_add(1, SeqCst)
.wrapping_add(1);
let tick = self.inner.tick_num.fetch_add(1, SeqCst).wrapping_add(1);
loop {
let node = match unsafe { self.inner.dequeue(Some(tick)) } {
@@ -246,7 +243,7 @@ where U: Unpark,
let node = ptr2arc(node);
assert!((*node.next_all.get()).is_null());
assert!((*node.prev_all.get()).is_null());
continue
continue;
};
// We're going to need to be very careful if the `poll`
@@ -369,8 +366,7 @@ impl Task {
impl fmt::Debug for Task {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
fmt.debug_struct("Task")
.finish()
fmt.debug_struct("Task").finish()
}
}
@@ -559,7 +555,7 @@ impl<U> List<U> {
}
}
/// Prepends an element to the back of the list
/// Appends an element to the back of the list
fn push_back(&mut self, node: Arc<Node<U>>) -> *const Node<U> {
let ptr = arc2ptr(node);
@@ -580,7 +576,7 @@ impl<U> List<U> {
self.len += 1;
return ptr
return ptr;
}
/// Pop an element from the front of the list
@@ -632,7 +628,7 @@ impl<U> List<U> {
self.len -= 1;
return node
return node;
}
}
@@ -749,7 +745,7 @@ impl<U> Drop for Node<U> {
fn arc2ptr<T>(ptr: Arc<T>) -> *const T {
let addr = &*ptr as *const T;
mem::forget(ptr);
return addr
return addr;
}
unsafe fn ptr2arc<T>(ptr: *const T) -> Arc<T> {
+155 -120
View File
@@ -1,6 +1,6 @@
extern crate futures;
extern crate tokio_current_thread;
extern crate tokio_executor;
extern crate futures;
use tokio_current_thread::{block_on_all, CurrentThread};
@@ -10,8 +10,8 @@ use std::rc::Rc;
use std::thread;
use std::time::Duration;
use futures::task;
use futures::future::{self, lazy};
use futures::task;
// This is not actually unused --- we need this trait to be in scope for
// the tests that sue TaskExecutor::current().execute(). The compiler
// doesn't realise that.
@@ -22,7 +22,7 @@ use futures::sync::oneshot;
mod from_block_on_all {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()>>) + 'static>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()>>) + 'static>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -36,7 +36,8 @@ mod from_block_on_all {
})));
Ok::<_, ()>("hello")
})).unwrap();
}))
.unwrap();
assert_eq!(2, cnt.get());
assert_eq!(msg, "hello");
@@ -72,7 +73,8 @@ fn block_waits() {
block_on_all(rx.then(move |_| {
cnt.set(1 + cnt.get());
Ok::<_, ()>(())
})).unwrap();
}))
.unwrap();
assert_eq!(1, cnt2.get());
}
@@ -100,11 +102,14 @@ fn spawn_many() {
mod does_not_set_global_executor_by_default {
use super::*;
fn test<F: Fn(Box<Future<Item=(), Error=()> + Send>) -> Result<(), E> + 'static, E>(spawn: F) {
fn test<F: Fn(Box<Future<Item = (), Error = ()> + Send>) -> Result<(), E> + 'static, E>(
spawn: F,
) {
block_on_all(lazy(|| {
spawn(Box::new(lazy(|| ok()))).unwrap_err();
ok()
})).unwrap()
}))
.unwrap()
}
#[test]
@@ -123,20 +128,22 @@ mod from_block_on_future {
use super::*;
fn test<F: Fn(Box<Future<Item = (), Error = ()>>)>(spawn: F) {
let cnt = Rc::new(Cell::new(0));
let cnt = Rc::new(Cell::new(0));
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
let cnt = cnt.clone();
tokio_current_thread
.block_on(lazy(|| {
let cnt = cnt.clone();
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
spawn(Box::new(lazy(move || {
cnt.set(1 + cnt.get());
Ok(())
})));
Ok::<_, ()>(())
})).unwrap();
Ok::<_, ()>(())
}))
.unwrap();
tokio_current_thread.run().unwrap();
@@ -150,7 +157,11 @@ mod from_block_on_future {
#[test]
fn execute() {
test(|f| { tokio_current_thread::TaskExecutor::current().execute(f).unwrap(); });
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
});
}
}
@@ -170,8 +181,8 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut rc = Rc::new(());
@@ -189,10 +200,12 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
let mut tokio_current_thread = CurrentThread::new();
tokio_current_thread.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
})).unwrap();
tokio_current_thread
.block_on(lazy(|| {
spawn(Box::new(Never(rc.clone())));
Ok::<_, ()>(())
}))
.unwrap();
drop(tokio_current_thread);
@@ -202,12 +215,15 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -216,7 +232,9 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
@@ -225,12 +243,11 @@ mod outstanding_tasks_are_dropped_when_executor_is_dropped {
#[should_panic]
fn nesting_run() {
block_on_all(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
mod run_in_future {
@@ -241,13 +258,12 @@ mod run_in_future {
fn spawn() {
block_on_all(lazy(|| {
tokio_current_thread::spawn(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}));
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -256,18 +272,16 @@ mod run_in_future {
block_on_all(lazy(|| {
tokio_current_thread::TaskExecutor::current()
.execute(lazy(|| {
block_on_all(lazy(|| {
ok()
})).unwrap();
block_on_all(lazy(|| ok())).unwrap();
ok()
}))
.unwrap();
ok()
})).unwrap();
}))
.unwrap();
}
}
#[test]
fn tick_on_infini_future() {
let num = Rc::new(Cell::new(0));
@@ -288,9 +302,7 @@ fn tick_on_infini_future() {
}
CurrentThread::new()
.spawn(Infini {
num: num.clone(),
})
.spawn(Infini { num: num.clone() })
.turn(None)
.unwrap();
@@ -347,7 +359,8 @@ mod tasks_are_scheduled_fairly {
});
ok()
})).unwrap();
}))
.unwrap();
}
#[test]
@@ -359,8 +372,8 @@ mod tasks_are_scheduled_fairly {
fn execute() {
test(|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
.execute(f)
.unwrap();
})
}
}
@@ -370,8 +383,8 @@ mod and_turn {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let cnt = Rc::new(Cell::new(0));
let c = cnt.clone();
@@ -379,24 +392,25 @@ mod and_turn {
let mut tokio_current_thread = CurrentThread::new();
// Spawn a basic task to get the executor to turn
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
Ok(())
})));
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || Ok(()))));
// Turn once...
tokio_current_thread.turn(None).unwrap();
dotspawn(&mut tokio_current_thread, Box::new(lazy(move || {
c.set(1 + c.get());
// Spawn!
spawn(Box::new(lazy(move || {
dotspawn(
&mut tokio_current_thread,
Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})));
// Spawn!
spawn(Box::new(lazy(move || {
c.set(1 + c.get());
Ok::<(), ()>(())
})));
Ok(())
})),
);
// This does not run the newly spawned thread
tokio_current_thread.turn(None).unwrap();
@@ -409,12 +423,15 @@ mod and_turn {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -423,11 +440,12 @@ mod and_turn {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
}
mod in_drop {
@@ -455,23 +473,24 @@ mod in_drop {
fn test<F, G>(spawn: F, dotspawn: G)
where
F: Fn(Box<Future<Item=(), Error=()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item=(), Error=()>>)
F: Fn(Box<Future<Item = (), Error = ()>>) + 'static,
G: Fn(&mut CurrentThread, Box<Future<Item = (), Error = ()>>),
{
let mut tokio_current_thread = CurrentThread::new();
let mut tokio_current_thread = CurrentThread::new();
let (tx, rx) = oneshot::channel();
dotspawn(&mut tokio_current_thread, Box::new(
MyFuture {
dotspawn(
&mut tokio_current_thread,
Box::new(MyFuture {
_data: Box::new(OnDrop(Some(move || {
spawn(Box::new(lazy(move || {
tx.send(()).unwrap();
Ok(())
})));
}))),
}
));
}),
);
tokio_current_thread.block_on(rx).unwrap();
tokio_current_thread.run().unwrap();
@@ -479,12 +498,15 @@ mod in_drop {
#[test]
fn spawn() {
test(tokio_current_thread::spawn, |rt, f| { rt.spawn(f); })
test(tokio_current_thread::spawn, |rt, f| {
rt.spawn(f);
})
}
#[test]
fn execute() {
test(|f| {
test(
|f| {
tokio_current_thread::TaskExecutor::current()
.execute(f)
.unwrap();
@@ -493,7 +515,9 @@ mod in_drop {
// `futures::Executor`, so we'll call `.spawn(...)` rather than
// `.execute(...)` for now. If `CurrentThread` is changed to
// implement Executor, change this to `.execute(...).unwrap()`.
|rt, f| { rt.spawn(f); }
|rt, f| {
rt.spawn(f);
},
);
}
@@ -562,13 +586,17 @@ fn turn_has_polled() {
tokio_current_thread.spawn(receiver.then(|_| Ok(())));
// Turn once...
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver once, but considered it not ready
assert!(res.has_polled());
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled nothing, the receiver is not ready yet
assert!(!res.has_polled());
@@ -577,14 +605,18 @@ fn turn_has_polled() {
sender.send(()).unwrap();
// Turn another time
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// Should've polled the receiver, it's ready now
assert!(res.has_polled());
// Now the executor should be empty
assert!(tokio_current_thread.is_idle());
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
// So should've polled nothing
assert!(!res.has_polled());
@@ -646,46 +678,41 @@ fn turn_fair() {
// Once an item is received on the oneshot channel, it will immediately
// immediately make the second oneshot channel ready
tokio_current_thread.spawn(receiver
.map_err(|_| unreachable!())
.and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
tokio_current_thread.spawn(receiver.map_err(|_| unreachable!()).and_then(move |_| {
sender_2.send(()).unwrap();
receiver_1_done_clone.set(true);
Ok(())
})
);
Ok(())
}));
let receiver_2_done = Rc::new(Cell::new(false));
let receiver_2_done_clone = receiver_2_done.clone();
tokio_current_thread.spawn(receiver_2
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_2.map_err(|_| unreachable!()).and_then(move |_| {
receiver_2_done_clone.set(true);
Ok(())
}));
// The third receiver is only woken up from our Park implementation, it simulates
// e.g. a socket that first has to be polled to know if it is ready now
let receiver_3_done = Rc::new(Cell::new(false));
let receiver_3_done_clone = receiver_3_done.clone();
tokio_current_thread.spawn(receiver_3
.map_err(|_| unreachable!())
.and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
})
);
tokio_current_thread.spawn(receiver_3.map_err(|_| unreachable!()).and_then(move |_| {
receiver_3_done_clone.set(true);
Ok(())
}));
// First turn should've polled both and considered them not ready
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(res.has_polled());
// Next turn should've polled nothing
let res = tokio_current_thread.turn(Some(Duration::from_millis(0))).unwrap();
let res = tokio_current_thread
.turn(Some(Duration::from_millis(0)))
.unwrap();
assert!(!res.has_polled());
assert!(!receiver_1_done.get());
@@ -736,10 +763,12 @@ fn spawn_from_other_thread() {
let (sender, receiver) = oneshot::channel::<()>();
thread::spawn(move || {
handle.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender.send(()).unwrap();
Ok(())
}))
.unwrap();
});
let _ = current_thread.block_on(receiver).unwrap();
@@ -758,10 +787,12 @@ fn spawn_from_other_thread_unpark() {
thread::spawn(move || {
let _ = receiver_2.recv().unwrap();
handle.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
sender_1.send(()).unwrap();
Ok(())
}))
.unwrap();
});
// Ensure that unparking the executor works correctly. It will first
@@ -769,13 +800,15 @@ fn spawn_from_other_thread_unpark() {
// lazy future below which will cause the future to be spawned from
// the other thread. Then the executor will park but should be woken
// up because *now* we have a new future to schedule
let _ = current_thread.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1)
).unwrap();
let _ = current_thread
.block_on(
lazy(move || {
sender_2.send(()).unwrap();
Ok(())
})
.and_then(|_| receiver_1),
)
.unwrap();
}
#[test]
@@ -785,10 +818,12 @@ fn spawn_from_executor_with_handle() {
let (tx, rx) = oneshot::channel();
current_thread.spawn(lazy(move || {
handle.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
})).unwrap();
handle
.spawn(lazy(move || {
tx.send(()).unwrap();
Ok(())
}))
.unwrap();
Ok::<_, ()>(())
}));
+5
View File
@@ -1,3 +1,8 @@
# 0.1.7 (March 22, 2019)
### Added
- `TypedExecutor` for spawning futures of a specific type (#993).
# 0.1.6 (January 6, 2019)
* Implement `Unpark` for `Arc<Unpark>` (#802).
+5 -2
View File
@@ -8,8 +8,8 @@ name = "tokio-executor"
# - Cargo.toml
# - README.md
# - Create "v0.1.x" git tag.
version = "0.1.6"
documentation = "https://docs.rs/tokio-executor/0.1.6/tokio_executor"
version = "0.1.7"
documentation = "https://docs.rs/tokio-executor/0.1.7/tokio_executor"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://github.com/tokio-rs/tokio"
license = "MIT"
@@ -23,3 +23,6 @@ categories = ["concurrency", "asynchronous"]
[dependencies]
crossbeam-utils = "0.6.2"
futures = "0.1.19"
[dev-dependencies]
tokio = "0.1.18"
+5 -5
View File
@@ -2,7 +2,7 @@
Task execution related traits and utilities.
[Documentation](https://docs.rs/tokio-executor/0.1.6/tokio_executor)
[Documentation](https://docs.rs/tokio-executor/0.1.7/tokio_executor)
## Overview
@@ -31,10 +31,10 @@ executor, including:
* [`Park`] abstracts over blocking and unblocking the current thread.
[`Executor`]: https://tokio-rs.github.io/tokio/tokio_executor/trait.Executor.html
[`enter`]: https://tokio-rs.github.io/tokio/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://tokio-rs.github.io/tokio/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://tokio-rs.github.io/tokio/tokio_executor/park/index.html
[`Executor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/trait.Executor.html
[`enter`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/fn.enter.html
[`DefaultExecutor`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/struct.DefaultExecutor.html
[`Park`]: https://docs.rs/tokio-executor/0.1.7/tokio_executor/park/trait.Park.html
## License
+6 -4
View File
@@ -1,7 +1,7 @@
use std::prelude::v1::*;
use std::cell::Cell;
use std::error::Error;
use std::fmt;
use std::prelude::v1::*;
use futures::{self, Future};
@@ -70,7 +70,10 @@ pub fn enter() -> Result<Enter, EnterError> {
impl Enter {
/// Register a callback to be invoked if and when the thread
/// ceased to act as an executor.
pub fn on_exit<F>(&mut self, f: F) where F: FnOnce() + 'static {
pub fn on_exit<F>(&mut self, f: F)
where
F: FnOnce() + 'static,
{
self.on_exit.push(Box::new(f));
}
@@ -88,7 +91,6 @@ impl Enter {
pub fn block_on<F: Future>(&mut self, f: F) -> Result<F::Item, F::Error> {
futures::executor::spawn(f).wait_future()
}
}
impl fmt::Debug for Enter {
@@ -103,7 +105,7 @@ impl Drop for Enter {
assert!(c.get());
if self.permanent {
return
return;
}
for callback in self.on_exit.drain(..) {
+50
View File
@@ -0,0 +1,50 @@
use std::error::Error;
use std::fmt;
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
+151
View File
@@ -0,0 +1,151 @@
use futures::Future;
use SpawnError;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// This trait is primarily intended to implemented by executors and used to
/// back `tokio::spawn`. Libraries and applications **may** use this trait to
/// bound generics, but doing so will limit usage to futures that implement
/// `Send`. Instead, libraries and applications are recommended to use
/// [`TypedExecutor`] as a bound.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
/// [`TypedExecutor`]: ../trait.TypedExecutor.html
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
+41 -24
View File
@@ -1,4 +1,4 @@
use super::{Executor, Enter, SpawnError};
use super::{Enter, Executor, SpawnError};
use futures::{future, Future};
@@ -33,24 +33,22 @@ impl DefaultExecutor {
/// `DefaultExecutor::current()` on thread A and then sending the result to
/// thread B will _not_ reference the default executor that was set on thread A.
pub fn current() -> DefaultExecutor {
DefaultExecutor {
_dummy: (),
}
DefaultExecutor { _dummy: () }
}
#[inline]
fn with_current<F: FnOnce(&mut Executor) -> R, R>(f: F) -> Option<R> {
EXECUTOR.with(|current_executor| {
match current_executor.replace(State::Active) {
EXECUTOR.with(
|current_executor| match current_executor.replace(State::Active) {
State::Ready(executor_ptr) => {
let executor = unsafe { &mut *executor_ptr };
let result = f(executor);
current_executor.set(State::Ready(executor_ptr));
Some(result)
},
}
State::Empty | State::Active => None,
}
})
},
)
}
}
@@ -61,18 +59,21 @@ enum State {
// default executor is defined and ready to be used
Ready(*mut Executor),
// default executor is currently active (used to detect recursive calls)
Active
Active,
}
/// Thread-local tracking the current executor
thread_local!(static EXECUTOR: Cell<State> = Cell::new(State::Empty));
thread_local! {
/// Thread-local tracking the current executor
static EXECUTOR: Cell<State> = Cell::new(State::Empty)
}
// ===== impl DefaultExecutor =====
impl super::Executor for DefaultExecutor {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
fn spawn(
&mut self,
future: Box<Future<Item = (), Error = ()> + Send>,
) -> Result<(), SpawnError> {
DefaultExecutor::with_current(|executor| executor.spawn(future))
.unwrap_or_else(|| Err(SpawnError::shutdown()))
}
@@ -83,8 +84,22 @@ impl super::Executor for DefaultExecutor {
}
}
impl<T> super::TypedExecutor<T> for DefaultExecutor
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
super::Executor::spawn(self, Box::new(future))
}
fn status(&self) -> Result<(), SpawnError> {
super::Executor::status(self)
}
}
impl<T> future::Executor<T> for DefaultExecutor
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
fn execute(&self, future: T) -> Result<(), future::ExecuteError<T>> {
if let Err(e) = super::Executor::status(self) {
@@ -144,10 +159,10 @@ where T: Future<Item = (), Error = ()> + Send + 'static,
/// # pub fn main() {}
/// ```
pub fn spawn<T>(future: T)
where T: Future<Item = (), Error = ()> + Send + 'static,
where
T: Future<Item = (), Error = ()> + Send + 'static,
{
DefaultExecutor::current().spawn(Box::new(future))
.unwrap()
DefaultExecutor::current().spawn(Box::new(future)).unwrap()
}
/// Set the default executor for the duration of the closure
@@ -156,13 +171,15 @@ pub fn spawn<T>(future: T)
///
/// This function panics if there already is a default executor set.
pub fn with_default<T, F, R>(executor: &mut T, enter: &mut Enter, f: F) -> R
where T: Executor,
F: FnOnce(&mut Enter) -> R
where
T: Executor,
F: FnOnce(&mut Enter) -> R,
{
EXECUTOR.with(|cell| {
match cell.get() {
State::Ready(_) | State::Active =>
panic!("default executor already set for execution context"),
State::Ready(_) | State::Active => {
panic!("default executor already set for execution context")
}
_ => {}
}
@@ -200,7 +217,7 @@ unsafe fn hide_lt<'a>(p: *mut (Executor + 'a)) -> *mut (Executor + 'static) {
#[cfg(test)]
mod tests {
use super::{Executor, DefaultExecutor, with_default};
use super::{with_default, DefaultExecutor, Executor};
#[test]
fn default_executor_is_send_and_sync() {
+25 -195
View File
@@ -1,5 +1,5 @@
#![deny(missing_docs, missing_debug_implementations, warnings)]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.6")]
#![doc(html_root_url = "https://docs.rs/tokio-executor/0.1.7")]
//! Task execution related traits and utilities.
//!
@@ -17,8 +17,12 @@
//! This crate provides traits and utilities that are necessary for building an
//! executor, including:
//!
//! * The [`Executor`] trait describes the API for spawning a future onto an
//! executor.
//! * The [`Executor`] trait spawns future object onto an executor.
//!
//! * The [`TypedExecutor`] trait spawns futures of a specific type onto an
//! executor. This is used to be generic over executors that spawn futures
//! that are either `Send` or `!Send` or implement executors that apply to
//! specific futures.
//!
//! * [`enter`] marks that the current thread is entering an execution
//! context. This prevents a second executor from accidentally starting from
@@ -29,7 +33,19 @@
//!
//! * [`Park`] abstracts over blocking and unblocking the current thread.
//!
//! # Implementing an executor
//!
//! Executors should always implement `TypedExecutor`. This usually is the bound
//! that applications and libraries will use when generic over an executor. See
//! the [trait documentation][`TypedExecutor`] for more details.
//!
//! If the executor is able to spawn all futures that are `Send`, then the
//! executor should also implement the `Executor` trait. This trait is rarely
//! used directly by applications and libraries. Instead, `tokio::spawn` is
//! configured to dispatch to type that implements `Executor`.
//!
//! [`Executor`]: trait.Executor.html
//! [`TypedExecutor`]: trait.TypedExecutor.html
//! [`enter`]: fn.enter.html
//! [`DefaultExecutor`]: struct.DefaultExecutor.html
//! [`Park`]: park/index.html
@@ -39,200 +55,14 @@ extern crate crossbeam_utils;
extern crate futures;
mod enter;
mod error;
mod executor;
mod global;
pub mod park;
mod typed;
pub use enter::{enter, Enter, EnterError};
pub use error::SpawnError;
pub use executor::Executor;
pub use global::{spawn, with_default, DefaultExecutor};
use futures::Future;
use std::error::Error;
use std::fmt;
/// A value that executes futures.
///
/// The [`spawn`] function is used to submit a future to an executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// The strategy employed by the executor to handle the future is less defined
/// and is left up to the `Executor` implementation. The `Executor` instance is
/// expected to call [`poll`] on the future once it has been notified, however
/// the "when" and "how" can vary greatly.
///
/// For example, the executor might be a thread pool, in which case a set of
/// threads have already been spawned up and the future is inserted into a
/// queue. A thread will acquire the future and poll it.
///
/// The `Executor` trait is only for futures that **are** `Send`. These are most
/// common. There currently is no trait that describes executors that operate
/// entirely on the current thread (i.e., are able to spawn futures that are not
/// `Send`). Note that single threaded executors can still implement `Executor`,
/// but only futures that are `Send` can be spawned via the trait.
///
/// # Errors
///
/// The [`spawn`] function returns `Result` with an error type of `SpawnError`.
/// This error type represents the reason that the executor was unable to spawn
/// the future. The two current represented scenarios are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
///
/// If a caller encounters an at capacity error, the caller should try to shed
/// load. This can be as simple as dropping the future that was spawned.
///
/// If the caller encounters a shutdown error, the caller should attempt to
/// gracefully shutdown.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
///
/// [`spawn`]: #tymethod.spawn
/// [`poll`]: https://docs.rs/futures/0.1/futures/future/trait.Future.html#tymethod.poll
pub trait Executor {
/// Spawns a future object to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// future may run on the current thread or another thread at the discretion
/// of the `Executor` implementation.
///
/// # Panics
///
/// Implementers are encouraged to avoid panics. However, a panic is
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// # }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::Executor;
/// # fn docs(my_executor: &mut Executor) {
/// use futures::future::lazy;
///
/// if my_executor.status().is_ok() {
/// my_executor.spawn(Box::new(lazy(|| {
/// println!("running on the executor");
/// Ok(())
/// }))).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// # }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E: Executor + ?Sized> Executor for Box<E> {
fn spawn(&mut self, future: Box<Future<Item = (), Error = ()> + Send>)
-> Result<(), SpawnError>
{
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
/// Errors returned by `Executor::spawn`.
///
/// Spawn errors should represent relatively rare scenarios. Currently, the two
/// scenarios represented by `SpawnError` are:
///
/// * An executor being at capacity or full. As such, the executor is not able
/// to accept a new future. This error state is expected to be transient.
/// * An executor has been shutdown and can no longer accept new futures. This
/// error state is expected to be permanent.
#[derive(Debug)]
pub struct SpawnError {
is_shutdown: bool,
}
impl SpawnError {
/// Return a new `SpawnError` reflecting a shutdown executor failure.
pub fn shutdown() -> Self {
SpawnError { is_shutdown: true }
}
/// Return a new `SpawnError` reflecting an executor at capacity failure.
pub fn at_capacity() -> Self {
SpawnError { is_shutdown: false }
}
/// Returns `true` if the error reflects a shutdown executor failure.
pub fn is_shutdown(&self) -> bool {
self.is_shutdown
}
/// Returns `true` if the error reflects an executor at capacity failure.
pub fn is_at_capacity(&self) -> bool {
!self.is_shutdown
}
}
impl fmt::Display for SpawnError {
fn fmt(&self, fmt: &mut fmt::Formatter) -> fmt::Result {
write!(fmt, "{}", self.description())
}
}
impl Error for SpawnError {
fn description(&self) -> &str {
"attempted to spawn task while the executor is at capacity or shut down"
}
}
pub use typed::TypedExecutor;
+2 -1
View File
@@ -190,7 +190,8 @@ impl ParkThread {
/// Get a reference to the `ParkThread` handle for this thread.
fn with_current<F, R>(&self, f: F) -> R
where F: FnOnce(&Parker) -> R,
where
F: FnOnce(&Parker) -> R,
{
CURRENT_PARKER.with(|inner| f(inner))
}
+181
View File
@@ -0,0 +1,181 @@
use SpawnError;
/// A value that spawns futures of a specific type.
///
/// The trait is generic over `T`: the type of future that can be spawened. This
/// is useful for implementing an executor that is only able to spawn a specific
/// type of future.
///
/// The [`spawn`] function is used to submit the future to the executor. Once
/// submitted, the executor takes ownership of the future and becomes
/// responsible for driving the future to completion.
///
/// This trait is useful as a bound for applications and libraries in order to
/// be generic over futures that are `Send` vs. `!Send`.
///
/// # Examples
///
/// Consider a function that provides an API for draining a `Stream` in the
/// background. To do this, a task must be spawned to perform the draining. As
/// such, the function takes a stream and an executor on which the background
/// task is spawned.
///
/// ```rust
/// #[macro_use]
/// extern crate futures;
/// extern crate tokio;
///
/// use futures::{Future, Stream, Poll};
/// use tokio::executor::TypedExecutor;
/// use tokio::sync::oneshot;
///
/// pub fn drain<T, E>(stream: T, executor: &mut E)
/// -> impl Future<Item = (), Error = ()>
/// where
/// T: Stream,
/// E: TypedExecutor<Drain<T>>
/// {
/// let (tx, rx) = oneshot::channel();
///
/// executor.spawn(Drain {
/// stream,
/// tx: Some(tx),
/// }).unwrap();
///
/// rx.map_err(|_| ())
/// }
///
/// // The background task
/// pub struct Drain<T: Stream> {
/// stream: T,
/// tx: Option<oneshot::Sender<()>>,
/// }
///
/// impl<T: Stream> Future for Drain<T> {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
/// loop {
/// let item = try_ready!(
/// self.stream.poll()
/// .map_err(|_| ())
/// );
///
/// if item.is_none() { break; }
/// }
///
/// self.tx.take().unwrap().send(()).map_err(|_| ());
/// Ok(().into())
/// }
/// }
/// # pub fn main() {}
/// ```
///
/// By doing this, the `drain` fn can accept a stream that is `!Send` as long as
/// the supplied executor is able to spawn `!Send` types.
pub trait TypedExecutor<T> {
/// Spawns a future to run on this executor.
///
/// `future` is passed to the executor, which will begin running it. The
/// executor takes ownership of the future and becomes responsible for
/// driving the future to completion.
///
/// # Panics
///
/// Implementations are encouraged to avoid panics. However, panics are
/// permitted and the caller should check the implementation specific
/// documentation for more details on possible panics.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// my_executor.spawn(MyFuture).unwrap();
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn spawn(&mut self, future: T) -> Result<(), SpawnError>;
/// Provides a best effort **hint** to whether or not `spawn` will succeed.
///
/// This function may return both false positives **and** false negatives.
/// If `status` returns `Ok`, then a call to `spawn` will *probably*
/// succeed, but may fail. If `status` returns `Err`, a call to `spawn` will
/// *probably* fail, but may succeed.
///
/// This allows a caller to avoid creating the task if the call to `spawn`
/// has a high likelihood of failing.
///
/// # Panics
///
/// This function must not panic. Implementers must ensure that panics do
/// not happen.
///
/// # Examples
///
/// ```rust
/// # extern crate futures;
/// # extern crate tokio_executor;
/// # use tokio_executor::TypedExecutor;
/// # use futures::{Future, Poll};
/// fn example<T>(my_executor: &mut T)
/// where
/// T: TypedExecutor<MyFuture>,
/// {
/// if my_executor.status().is_ok() {
/// my_executor.spawn(MyFuture).unwrap();
/// } else {
/// println!("the executor is not in a good state");
/// }
/// }
///
/// struct MyFuture;
///
/// impl Future for MyFuture {
/// type Item = ();
/// type Error = ();
///
/// fn poll(&mut self) -> Poll<(), ()> {
/// println!("running on the executor");
/// Ok(().into())
/// }
/// }
/// # fn main() {}
/// ```
fn status(&self) -> Result<(), SpawnError> {
Ok(())
}
}
impl<E, T> TypedExecutor<T> for Box<E>
where
E: TypedExecutor<T>,
{
fn spawn(&mut self, future: T) -> Result<(), SpawnError> {
(**self).spawn(future)
}
fn status(&self) -> Result<(), SpawnError> {
(**self).status()
}
}
+5 -4
View File
@@ -1,15 +1,16 @@
extern crate tokio_executor;
extern crate futures;
extern crate tokio_executor;
use tokio_executor::*;
use futures::{Future, future::lazy};
use futures::{future::lazy, Future};
use tokio_executor::DefaultExecutor;
mod out_of_executor_context {
use super::*;
use tokio_executor::Executor;
fn test<F, E>(spawn: F)
where
F: Fn(Box<Future<Item=(), Error=()> + Send>) -> Result<(), E>,
F: Fn(Box<Future<Item = (), Error = ()> + Send>) -> Result<(), E>,
{
let res = spawn(Box::new(lazy(|| Ok(()))));
assert!(res.is_err());
+6
View File
@@ -1,3 +1,9 @@
# 0.1.6 (March 1, 2019)
### Added
- File::try_clone() (#850).
- Async equivalent of read / write file helpers being added to `std` (#896).
# 0.1.5 (January 6, 2019)
* Add examples to `File` API docs (#786).
+7 -7
View File
@@ -8,13 +8,13 @@ name = "tokio-fs"
# - Cargo.toml
# - README.md
# - Create "v0.1.x" git tag.
version = "0.1.5"
version = "0.1.6"
authors = ["Carl Lerche <[email protected]>"]
license = "MIT"
readme = "README.md"
repository = "https://github.com/tokio-rs/tokio"
homepage = "https://tokio.rs"
documentation = "https://docs.rs/tokio-fs/0.1.5/tokio_fs"
documentation = "https://docs.rs/tokio-fs/0.1.6/tokio_fs"
description = """
Filesystem API for Tokio.
"""
@@ -23,13 +23,13 @@ categories = ["asynchronous", "network-programming", "filesystem"]
[dependencies]
futures = "0.1.21"
tokio-threadpool = { version = "0.1.3", path = "../tokio-threadpool" }
tokio-io = { version = "0.1.6", path = "../tokio-io" }
tokio-threadpool = "0.1.3"
tokio-io = "0.1.6"
[dev-dependencies]
rand = "0.6"
tempfile = "3"
tempdir = "0.3"
tokio-io = { version = "0.1.6", path = "../tokio-io" }
tokio-codec = { version = "0.1.0", path = "../tokio-codec" }
tokio = { version = "0.1.7", path = ".." }
tokio-io = "0.1.6"
tokio-codec = "0.1.0"
tokio = "0.1.7"
+1 -1
View File
@@ -2,7 +2,7 @@
Asynchronous filesystem manipulation operations (and stdin, stdout, stderr).
[Documentation](https://docs.rs/tokio-fs/0.1.5/tokio_fs)
[Documentation](https://docs.rs/tokio-fs/0.1.6/tokio_fs)
## Overview
+17 -19
View File
@@ -2,39 +2,35 @@
#![deny(deprecated, warnings)]
extern crate futures;
extern crate tokio_fs;
extern crate tokio_codec;
extern crate tokio_fs;
extern crate tokio_threadpool;
use tokio_fs::{stdin, stdout, stderr};
use tokio_codec::{FramedRead, FramedWrite, LinesCodec};
use tokio_fs::{stderr, stdin, stdout};
use tokio_threadpool::Builder;
use futures::{Future, Stream, Sink};
use futures::{Future, Sink, Stream};
use std::io;
pub fn main() -> Result<(), Box<std::error::Error>> {
let pool = Builder::new()
.pool_size(1)
.build();
let pool = Builder::new().pool_size(1).build();
pool.spawn({
let input = FramedRead::new(stdin(), LinesCodec::new());
let output = FramedWrite::new(stdout(), LinesCodec::new())
.with(|line: String| {
let mut out = "OUT: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let output = FramedWrite::new(stdout(), LinesCodec::new()).with(|line: String| {
let mut out = "OUT: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let error = FramedWrite::new(stderr(), LinesCodec::new())
.with(|line: String| {
let mut out = "ERR: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let error = FramedWrite::new(stderr(), LinesCodec::new()).with(|line: String| {
let mut out = "ERR: ".to_string();
out.push_str(&line);
Ok::<_, io::Error>(out)
});
let dst = output.fanout(error);
@@ -44,6 +40,8 @@ pub fn main() -> Result<(), Box<std::error::Error>> {
.map_err(|e| panic!("io error = {:?}", e))
});
pool.shutdown_on_idle().wait().map_err(|_| "failed to shutdown the thread pool")?;
pool.shutdown_on_idle()
.wait()
.map_err(|_| "failed to shutdown the thread pool")?;
Ok(())
}
+5 -7
View File
@@ -17,30 +17,28 @@ pub fn create_dir<P: AsRef<Path>>(path: P) -> CreateDirFuture<P> {
#[derive(Debug)]
pub struct CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
path: P,
}
impl<P> CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
fn new(path: P) -> CreateDirFuture<P> {
CreateDirFuture {
path: path,
}
CreateDirFuture { path: path }
}
}
impl<P> Future for CreateDirFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
::blocking_io(|| fs::create_dir(&self.path) )
::blocking_io(|| fs::create_dir(&self.path))
}
}
+5 -7
View File
@@ -18,30 +18,28 @@ pub fn create_dir_all<P: AsRef<Path>>(path: P) -> CreateDirAllFuture<P> {
#[derive(Debug)]
pub struct CreateDirAllFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
path: P,
}
impl<P> CreateDirAllFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
fn new(path: P) -> CreateDirAllFuture<P> {
CreateDirAllFuture {
path: path,
}
CreateDirAllFuture { path: path }
}
}
impl<P> Future for CreateDirAllFuture<P>
where
P: AsRef<Path>
P: AsRef<Path>,
{
type Item = ();
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
::blocking_io(|| fs::create_dir_all(&self.path) )
::blocking_io(|| fs::create_dir_all(&self.path))
}
}
+37
View File
@@ -0,0 +1,37 @@
use super::File;
use futures::{Future, Poll};
use std::io;
/// Future returned by `File::try_clone`.
///
/// Clones a file handle into two file handles.
///
/// # Panics
///
/// Will panic if polled after returning an item or error.
#[derive(Debug)]
pub struct CloneFuture {
file: Option<File>,
}
impl CloneFuture {
pub(crate) fn new(file: File) -> Self {
Self { file: Some(file) }
}
}
impl Future for CloneFuture {
type Item = (File, File);
type Error = (File, io::Error);
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
self.file
.as_mut()
.expect("Cannot poll `CloneFuture` after it resolves")
.poll_try_clone()
.map(|inner| inner.map(|cloned| (self.file.take().unwrap(), cloned)))
.map_err(|err| (self.file.take().unwrap(), err))
}
}
+5 -5
View File
@@ -13,7 +13,8 @@ pub struct CreateFuture<P> {
}
impl<P> CreateFuture<P>
where P: AsRef<Path> + Send + 'static,
where
P: AsRef<Path> + Send + 'static,
{
pub(crate) fn new(path: P) -> Self {
CreateFuture { path }
@@ -21,15 +22,14 @@ where P: AsRef<Path> + Send + 'static,
}
impl<P> Future for CreateFuture<P>
where P: AsRef<Path> + Send + 'static,
where
P: AsRef<Path> + Send + 'static,
{
type Item = File;
type Error = io::Error;
fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
let std = try_ready!(::blocking_io(|| {
StdFile::create(&self.path)
}));
let std = try_ready!(::blocking_io(|| StdFile::create(&self.path)));
let file = File::from_std(std);
Ok(file.into())

Some files were not shown because too many files have changed in this diff Show More